This study aims to quantify the techno-economic potential of using solar photovoltaics (PV) to support heat pumps (HP) towards the replacement of natural gas heating in a representative North American residence from a house owner's point of view. For this purpose, simulations are performed on: (1) a residential natural gas-based heating system and grid electricity, (2) a residential natural gas-based heating system with PV to serve the electric load, (3) a residential HP system with grid electricity, and (4) a residential HP+PV system. Detailed descriptions are provided along with a comprehensive sensitivity analysis for identifying specific boundary conditions that enable lower total life cycle cost. The results show that under typical inflation conditions, the lifecycle cost of natural gas and reversable, air-source heat pumps are nearly identical, however the electricity rate structure makes PV costlier. With higher rates of inflation or lower PV capital costs, PV becomes a hedge against rising prices and encourages the adoption of HPs by also locking in both electricity and heating cost growth. The real internal rate of return for such prosumer technologies is 20x greater than a long-term certificate of deposit, which demonstrates the additional value PV and HP technologies offer prosumers over comparably secure investment vehicles while making substantive reductions in carbon emissions. Using the large volume of results generated, impacts on energy policy are discussed, including rebates, net-metering, and utility business models.