Urbanization processes are manifested by the change in the physical morphology of cities. Gaining knowledge about cities through their morphology is naturally linked to the capability of remote sensing (RS) that can monitor city forms with a synoptic view. Yet, our knowledge of the urban form does not linearly increase with the increase of image data. Thus, the role, challenges and potentials of RS in deriving knowledge about urban morphology are worth investigating. We argue that ongoing efforts of mapping urban elements in RS are only marginally contributing to the understanding of cities in terms of urban morphology. We further reason that magnifying the role of RS depends on a suggested workflow involving steps that are external to RS, mainly including characterizing urban morphology through meaningful measurements of mapped elements, and interpreting the measured physical forms as proxies of the socioeconomic status. To exemplify the major steps, we focus on urban poverty (deprivation), and examine its manifestation through the morphology of buildings. Our findings show that challenges appear as soon as the collection of building information from RS images starts. This is mainly caused by inconsistent, incomplete and inaccurate GIS based representation of buildings on images, as well as low quality predictions, hidden from accuracy metrics. Although the potential of deriving meaningful urban morphological patterns from building maps for explaining socioeconomic patterns still holds, several uncertainties remain unsolved, such as the way urban processes are manifested morphologically and how the morphology is captured with the influence of building map quality. Our main conclusion is that as RS imagebased morphological information propagates and fluctuates along the process of knowledge derivation, causing difficulties in quantifying the exact amount of urban knowledge derived. Nonetheless, useful knowledge could already be obtained even with suboptimal data sources and model performances, which opens the opportunity to facilitate transferable and reproducible urban morphology studies by using widely accessible data despite their suboptimal quality.