kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Learning 3D Texture-Aware Representations for Parsing Diverse Human Clothing and Body Parts
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).ORCID iD: 0000-0002-7414-845X
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Computational Science and Technology (CST).ORCID iD: 0000-0002-7257-0761
Adobe Research.ORCID iD: 0000-0002-7627-7765
2026 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Existing methods for human parsing into body parts and clothing often use fixed mask categories with broad labels that obscure fine-grained clothing types. Recent open-vocabulary segmentation approaches leverage pretrained text-to-image (T2I) diffusion model features for strong zero-shot transfer, but typically group entire humans into a single person category, failing to distinguish diverse clothing or detailed body parts. To address this, we propose Spectrum, a unified network for part-level pixel parsing (body parts and clothing) and instance-level grouping. While diffusion-based open-vocabulary models generalize well across tasks, their internal representations are not specialized for detailed human parsing. We observe that, unlike diffusion models with broad representations, image-driven 3D texture generators maintain faithful correspondence to input images, enabling stronger representations for parsing diverse clothing and body parts. Spectrum introduces a novel repurposing of an Image-to-Texture (I2Tx) diffusion model—obtained by fine-tuning a T2I model on 3D human texture maps—for improved alignment with body parts and clothing. From an input image, we extract human-part internal features via the I2Tx diffusion model and generate semantically valid masks aligned to diverse clothing categories through prompt-guided grounding. Once trained, Spectrum produces semantic segmentation maps for every visible body part and clothing category, ignoring standalone garments or irrelevant objects, for any number of humans in the scene. We conduct extensive cross-dataset experiments—separately assessing body parts, clothing parts, unseen clothing categories, and full-body masks—and demonstrate that Spectrum consistently outperforms baseline methods in prompt-based segmentation.

Place, publisher, year, edition, pages
2026.
National Category
Computer Vision and Learning Systems
Research subject
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-374597DOI: 10.48550/arXiv.2508.06032OAI: oai:DiVA.org:kth-374597DiVA, id: diva2:2023287
Conference
The 40th Annual AAAI Conference on Artificial Intelligence, Jan 20-27, 2026, Singapore
Note

QC 20251219

Available from: 2025-12-18 Created: 2025-12-18 Last updated: 2025-12-19Bibliographically approved

Open Access in DiVA

fulltext(42023 kB)529 downloads
File information
File name FULLTEXT01.pdfFile size 42023 kBChecksum SHA-512
c78bea58dff3d5422003646ed18ba21a5b11454ef9ec18ff24215f37a9a3f49460c184b50c9d9267d9e0236401660e95617b819794d00397a012e1d6a638488f
Type fulltextMimetype application/pdf

Other links

Publisher's full textConference website

Authority records

Chhatre, KiranPeters, Christopher

Search in DiVA

By author/editor
Chhatre, KiranPeters, ChristopherKaranam, Srikrishna
By organisation
Computational Science and Technology (CST)
Computer Vision and Learning Systems

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 952 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf