kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The Case for Hierarchical Deep Learning Inference at the Network Edge
IMDEA Networks Institute, Madrid, Spain.
IMDEA Networks Institute, Madrid, Spain.
IMDEA Networks Institute, Madrid, Spain.
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Information Science and Engineering.ORCID iD: 0000-0002-2739-5060
Show others and affiliations
Number of Authors: 62023 (English)In: NetAISys 2023 - Proceedings of the 1st International Workshop on Networked AI Systems, Part of MobiSys 2023, Association for Computing Machinery (ACM) , 2023, p. 13-18Conference paper, Published paper (Refereed)
Abstract [en]

Resource-constrained Edge Devices (EDs), e.g., IoT sensors and microcontroller units, are expected to make intelligent decisions using Deep Learning (DL) inference at the edge of the network. Toward this end, developing tinyML models is an area of active research - DL models with reduced computation and memory storage requirements - that can be embedded on these devices. However, tinyML models have lower inference accuracy. On a different front, DNN partitioning and inference offloading techniques were studied for distributed DL inference between EDs and Edge Servers (ESs). In this paper, we explore Hierarchical Inference (HI), a novel approach proposed in [19] for performing distributed DL inference at the edge. Under HI, for each data sample, an ED first uses a local algorithm (e.g., a tinyML model) for inference. Depending on the application, if the inference provided by the local algorithm is incorrect or further assistance is required from large DL models on edge or cloud, only then the ED offloads the data sample. At the outset, HI seems infeasible as the ED, in general, cannot know if the local inference is sufficient or not. Nevertheless, we present the feasibility of implementing HI for image classification applications. We demonstrate its benefits using quantitative analysis and show that HI provides a better trade-off between offloading cost, throughput, and inference accuracy compared to alternate approaches.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM) , 2023. p. 13-18
Keywords [en]
deep learning, edge computing, hierarchical inference
National Category
Computer Sciences Computer Systems
Identifiers
URN: urn:nbn:se:kth:diva-334523DOI: 10.1145/3597062.3597278ISI: 001119206300003Scopus ID: 2-s2.0-85164295939OAI: oai:DiVA.org:kth-334523DiVA, id: diva2:1790623
Conference
1st International Workshop on Networked AI Systems, NetAISys 2023, co-located with ACM MobiSys 2023, Helsinki, Finland, Jun 18 2023
Note

Part of ISBN 9798400702129

QC 20230823

Available from: 2023-08-23 Created: 2023-08-23 Last updated: 2024-01-15Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Moothedath, Vishnu NarayananGross, James

Search in DiVA

By author/editor
Moothedath, Vishnu NarayananGross, James
By organisation
Information Science and Engineering
Computer SciencesComputer Systems

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 39 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf