The emerging strategy of protein–drug conjugates (PDCs) for targeted cancer therapy holds great potential to improve treatment efficacy by specifically targeting cancer biomarkers and delivering toxic payloads directly to tumor cells, minimizing off-target toxicity. The success of this approach depends on the internalization and retention of the payload in target cells. This study introduces a method using a small protein domain engineered for conditional target affinity, enabling lysosomal trafficking independent of the biological fate of the receptor. Specifically, we describe the development of an EGF receptor binder, CaRAEGFR, with calcium-regulated affinity (CaRA), meaning the target binding strength is tailored by the available calcium concentration. This allows for endosomal dissociation, as calcium levels are lower in endosomes than in the bloodstream. Affinity measurements and structural modeling reveal the molecular basis of the calcium modulated affinity. Live cell imaging demonstrates efficient internalization and lysosomal trafficking of the calcium-dependent domain, while the EGF receptor is recycled to the membrane. When used as a drug carrier, CaRAEGFR effectively delivers the toxin to the lysosomes, resulting in potent cytotoxicity with an IC50 of 0.8 nM in EGFR-expressing cancer cells