kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Are LLMs Ready for Spatial and Topological "Reasoning" in Fiber Networks?: An Evaluation on Real-World GIS Tasks
KTH, School of Architecture and the Built Environment (ABE), Urban Planning and Environment, Geoinformatics.
KTH, School of Architecture and the Built Environment (ABE), Urban Planning and Environment, Geoinformatics.
2025 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesisAlternative title
Är LLM:er redo för spatialt och topologiskt "resonemang" i fibernätverk? : En utvärdering baserad på verkliga GIS-arbetsflöden (Swedish)
Abstract [en]

Large language models (LLM) have achieved outstanding results in understanding natural languages, performing reasoning and generating code. However the testing of LLMs in spatial domains concerning spatial data and topological reasoning remains largely untested. This research examines the capability of standard LLMs to execute spatial and topological reasoning operations on real-world fiber infrastructure networks using schema-based prompts. The research evaluates five leading models including GPT-4.1, GPT-4o, GPT-o4 Mini, GPT-4 Turbo, and DeepSeek-R1 through 26 operational GIS workflow-based benchmark tasks. The tasks include both basic object-level questions such as cabinet identification, customer location filtering but also complex spatial logic operations that require multiple hops, recursive path tracing and failure impact simulation. The research is based on the model's API to generate zero-shot prompts for all queries without the help of any model training. The study evaluated two prompting methods: regular and Chain-of-Thought (CoT) to determine if an alternative prompting approach enhances spatial task performance.

In the empirical evaluations, LLMs show effectiveness when dealing with low-complexity tasks which have flat schema relationships and defined object types. GPT-o4 Mini and DeepSeekR1 achieved the highest F1-scores and lowest syntax error rates when performing these tasks better than their larger counterparts. Model performance seems to depend more on architecture design, instruction-following ability and prompt grounding rather than model size according to these research findings. Performance declined substantially when models needed to perform complex reasoning steps that involved schema hierarchy navigation. The models demonstrated three primary errors while completing their tasks. Models frequently failed to apply recursive logic, misunderstood nested object relationships, or hallucinated schema elements. This resulted in valid looking but logically incorrect SQL queries. The CoT approach improved model performance specifically for GPT-4.1 yet failed to generate sustained benefits for other models.The research indicates that LLMs have the ability to assist with both semi-automatic quality control and metadata inspection tasks in spatial operations. These models excel at performing schema inspections, helping users generate draft queries as well as providing guidance to nontechnical users through complex databases. However current LLMs demonstrate unreliable performance in critical operational tasks such as failure simulation, data migration and redundancy analysis unless they are combined with fallback systems or human verification processes. The research demonstrates both the present advantages and distinct boundaries of LLMs when working with real-world spatial data which supports the requirement for hybrid workflows and domain adaptation and further research before operational deployment can be considered.

Place, publisher, year, edition, pages
2025. , p. 87
Series
TRITA-ABE-MBT ; 2637
Keywords [en]
Large Language Models (LLMs), Spatial Reasoning, Topological Reasoning, Geographic Information Systems (GIS), Fiber Optic Networks
Keywords [sv]
Large Language Models (LLMs), Spatial Reasoning, Topological Reasoning, Geographic Information Systems (GIS), Fiber Optic Networks
National Category
Earth and Related Environmental Sciences
Identifiers
URN: urn:nbn:se:kth:diva-377770OAI: oai:DiVA.org:kth-377770DiVA, id: diva2:2043348
External cooperation
Digpro AB
Subject / course
Geoinformatics
Educational program
Master of Science - Transport and Geoinformation Technology
Presentation
(English)
Supervisors
Examiners
Available from: 2026-03-05 Created: 2026-03-04 Last updated: 2026-03-05Bibliographically approved

Open Access in DiVA

fulltext(1956 kB)316 downloads
File information
File name FULLTEXT02.pdfFile size 1956 kBChecksum SHA-512
30c8543bc528607c25d39ecbbcdab61355c0578c30de998b2f2927b1ae5ed9fa57897c55c5558c5ccb9a3975f9240e2ff92e59500b5fbcfa30da8b9a20977720
Type fulltextMimetype application/pdf

By organisation
Geoinformatics
Earth and Related Environmental Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 316 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 7548 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf