The pseudorange error is one of the root causes of localization inaccuracy in GPS. Previous data-driven methods regress and eliminate pseudorange errors using handcrafted intermediate labels. Unlike them, we propose an end-to-end GPS localization framework, E2E-PrNet, to train a neural network for pseudorange correction (PrNet) directly using the final task loss calculated with the ground truth of GPS receiver states. The gradients of the loss with respect to learnable parameters are backpropagated through a Differentiable Nonlinear Least Squares (DNLS) optimizer to PrNet. The feasibility of fusing the data-driven neural network and the model-based DNLS module is verified with GPS data collected by Android phones, showing that E2E-PrNet outperforms the baseline weighted least squares method and the state-of-the-art end-to-end data-driven approach. Finally, we discuss the explainability of E2E-PrNet.