kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Evaluating model serving strategies over streaming data
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Software and Computer systems, SCS.ORCID iD: 0000-0002-8573-0090
KTH, School of Electrical Engineering and Computer Science (EECS), Computer Science, Software and Computer systems, SCS.ORCID iD: 0000-0002-9351-8508
2022 (English)In: Proceedings of the 6th Workshop on Data Management for End-To-End Machine Learning, DEEM 2022 - In conjunction with the 2022 ACM SIGMOD/PODS Conference, Association for Computing Machinery (ACM) , 2022, article id 4Conference paper, Published paper (Refereed)
Abstract [en]

We present the first performance evaluation study of model serving integration tools in stream processing frameworks. Using Apache Flink as a representative stream processing system, we evaluate alternative Deep Learning serving pipelines for image classification. Our performance evaluation considers both the case of embedded use of Machine Learning libraries within stream tasks and that of external serving via Remote Procedure Calls. The results indicate superior throughput and scalability for pipelines that make use of embedded libraries to serve pre-trained models. Whereas, latency can vary across strategies, with external serving even achieving lower latency when network conditions are optimal due to better specialized use of underlying hardware. We discuss our findings and provide further motivating arguments towards research in the area of ML-native data streaming engines in the future.

Place, publisher, year, edition, pages
Association for Computing Machinery (ACM) , 2022. article id 4
Keywords [en]
data streams, machine learning inference
National Category
Other Computer and Information Science
Identifiers
URN: urn:nbn:se:kth:diva-317104DOI: 10.1145/3533028.3533308Scopus ID: 2-s2.0-85133190958OAI: oai:DiVA.org:kth-317104DiVA, id: diva2:1693181
Conference
6th Workshop on Data Management for End-To-End Machine Learning, DEEM 2022 - In conjunction with the 2022 ACM SIGMOD/PODS Conference, 12 June 2022, Virtual, Online
Note

QC 20220906

Part of proceedings: ISBN 978-145039375-1

Available from: 2022-09-06 Created: 2022-09-06 Last updated: 2022-09-06Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Horchidan, Sonia-FlorinaCarbone, Paris

Search in DiVA

By author/editor
Horchidan, Sonia-FlorinaCarbone, Paris
By organisation
Software and Computer systems, SCS
Other Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 136 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf