Accurate phase fraction analysis is an essential element of the microstructural characterization of alloys and often serves as a basis to quantify effects such as heat treatment or mechanical deformation. Additive manufacturing (AM) of metals, due to the intrinsic nonequilibrium solidification and spatial variability, creates additional challenges for the proper quantification of phase fraction. Such challenges are exacerbated when the alloy itself is prone to deformation-induced phase transformation. Using commonly available in-house X-ray diffraction (XRD) and electron backscatter diffraction (EBSD) and less commonly used synchrotron-based high-energy X-ray diffraction, we characterized nitrogen-atomized 17-4 precipitation-hardening martensitic stainless steel, a class of AM alloy that has received broad attention within the AM research community. On the same build, our measurements recovered the entire range of reported values on the austenite phase fractions of as-built AM 17-4 in literature, from approximate to 100% martensite to approximate to 100% austenite. Aided by Calphad simulation, our experimental findings established that our as-built AM 17-4 is almost fully austenitic and that in-house XRD and EBSD measurements are subject to significant uncertainties created by the specimen's surface finish. Hence, measurements made using these techniques must be understood in their correct context. Our results carry significant implications, not only to AM 17-4 but also to AM alloys that are susceptible to deformation-induced structure transformation and suggest that characterizations with less accessible but bulk sensitive techniques such as synchrotron-based high energy X-ray diffraction or neutron diffraction may be required for proper understanding of these materials.