Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Connectivity Properties of Mainline BitTorrent DHT Nodes
KTH, School of Information and Communication Technology (ICT), Communication: Services and Infrastucture (Closed 20120101), Telecommunication Systems Laboratory, TSLab (closed 2012-01-01).
KTH, School of Information and Communication Technology (ICT), Communication: Services and Infrastucture (Closed 20120101), Telecommunication Systems Laboratory, TSLab (closed 2012-01-01).
KTH, School of Information and Communication Technology (ICT), Communication: Services and Infrastucture (Closed 20120101), Telecommunication Systems Laboratory, TSLab (closed 2012-01-01).
2009 (English)In: 2009 IEEE NINTH INTERNATIONAL CONFERENCE ON PEER-TO-PEER COMPUTING (P2P 2009), NEW YORK: IEEE , 2009, p. 262-270Conference paper, Published paper (Refereed)
Abstract [en]

The birth and evolution of Feer-to-Peer (P2P) protocols have, for the most part, been about peer discovery. Napster, one of the first P2P protocols, was basically FTP/HTTP plus a way of finding hosts willing to send you the file. Since then, both the transfer and peer discovery mechanisms have improved, but only recently have we seen a real push to completely decentralized peer discovery to increase scalability and resilience. Most such efforts are based on Distributed Hash Tables (DHTs), with Kademlia being a popular choice of DHT implementation. While sound in theory, and performing well in simulators and testbeds, the real-world performance often falls short of expectations. Our hypothesis is that the connectivity artifacts caused by guarded hosts (i.e., hosts behind firewalls and NATs) are the major cause for such poor performance. In this paper, the first steps towards testing this hypothesis are developed. First, we present a taxonomy of connectivity properties which will become the language used to accurately describe connectivity artifacts. Second, based on experiments "in the wild", we analyze the connectivity properties of over 3 million hosts. Finally, we match those properties to guarded host behavior and identify the potential effects on the DHT

Place, publisher, year, edition, pages
NEW YORK: IEEE , 2009. p. 262-270
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-30017DOI: 10.1109/P2P.2009.5284530ISI: 000274540500041Scopus ID: 2-s2.0-73549117904ISBN: 978-1-4244-5066-4 (print)OAI: oai:DiVA.org:kth-30017DiVA, id: diva2:398948
Conference
9th International Conference on Peer-to-Peer Computing Seattle, WA, SEP 09-11, 2009
Note

QC 20110221

Available from: 2011-02-21 Created: 2011-02-21 Last updated: 2018-01-12Bibliographically approved
In thesis
1. Kademlia on the Open Internet: How to Achieve Sub-Second Lookups in a Multimillion-Node DHT Overlay
Open this publication in new window or tab >>Kademlia on the Open Internet: How to Achieve Sub-Second Lookups in a Multimillion-Node DHT Overlay
2011 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Distributed hash tables (DHTs) have gained much attention from the research community in the last years. Formal analysis and evaluations on simulators and small-scale deployments have shown good scalability and performance.

In stark contrast, performance measurements in large-scale DHT overlays on the Internet have yielded disappointing results, with lookup latencies measured in seconds. Others have attempted to improve lookup performance with very limited success, their lowest median lookup latency at over one second and a long tail of high-latency lookups.

In this thesis, the goal is to to enable large-scale DHT-based latency-sensitive applications on the Internet. In particular, we improve lookup latency in Mainline DHT, the largest DHT overlay on the open Internet, to identify and address practical issues on an existing system. Our approach is implementing and measuring backward-compatible modifications to facilitate their incremental adoption into Mainline DHT (and possibly other Kademlia-based overlays). Thus, enabling our research to have impact on a real-world system.

Our results close the performance gap between small- and large-scale DHT overlays. With a median lookup latency below 200 ms and a 99\superscript{th} percentile of just above 500 ms, our median lookup latency is one order of magnitude lower than the best performing measurement reported in the literature. Moreover, our results do not show a long tail of high-latency lookups, unlike previous reports.

We have achieved these results by studying how connectivity artifacts on the underlying network ---probably caused by firewalls and NAT devices on the Internet--- affect the DHT overlay. Our measurements of the connectivity of more than 3 million nodes reveal that connectivity artifacts are widespread and can severely degrade lookup performance.

Scalability and locality-awareness have also been explored in this thesis, where different mechanisms have been proposed. Some of the mechanisms are planned to be integrated into Mainline DHT in future work.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2011. p. x, 31
Series
Trita-ICT-ECS AVH, ISSN 1653-6363 ; 11:10
Keywords
dht, p2p, distributed systems, kademlia
National Category
Communication Systems
Identifiers
urn:nbn:se:kth:diva-46469 (URN)978-91-7501-153-0 (ISBN)
Presentation
2011-12-09, C2, Isafjordsgatan 22, Kista, 10:00 (English)
Opponent
Supervisors
Note
QC 20111118Available from: 2011-11-18 Created: 2011-11-03 Last updated: 2011-11-18Bibliographically approved
2. Distributed Peer Discovery in Large-Scale P2P Streaming Systems: Addressing Practical Problems of P2P Deployments on the Open Internet
Open this publication in new window or tab >>Distributed Peer Discovery in Large-Scale P2P Streaming Systems: Addressing Practical Problems of P2P Deployments on the Open Internet
2013 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Peer-to-peer (P2P) techniques allow users with limited resources to distribute content to a potentially large audience by turning passive clients into peers. Peers can self-organize to distribute content to each other, increasing the scalability of the system and decreasing the publisher’s costs, compared to a publisher distributing the data himself using a content delivery network (CDN) or his own servers.

Peer discovery is the mechanism that peers use to find each other. Peer discovery is a critical component of any P2P-based system, because P2P networks are dynamic by nature. That is, peers constantly join and leave the network and each individual peer is assumed to be unreliable. This thesis addresses practical issues in distributed peer discovery mech- anisms in the context of three different large-scale P2P streaming systems: a (1) BitTorrent-based streaming system, (2) Spotify, and (3) our own mobile P2P streaming system based on the upcoming Peer-to-peer Streaming Protocol (PPSP) Internet standard.

We dramatically improve peer discovery performance in BitTorrent’s Mainline DHT, the largest distributed hash table (DHT) overlay on the open Internet. Our implementation’s median lookup latency is an order of magnitude lower than the best performing measurement reported in the literature and does not exhibit a long tail of high-latency lookups, which is critical for P2P streaming applications.

We have achieved these results by studying how connectivity artifacts on the underlying network —probably caused by network address translation (NAT) gateways— affect the DHT overlay. Our measurements of more than three million nodes reveal that connectivity artifacts are widespread and can severely degrade DHT performance.

This thesis also addresses the practical issues of integrating mobile devices into P2P streaming systems. In particular, we enable P2P on Spotify’s Android app, study how distributed peer discovery affects energy consumption, and implement and evaluate backwards-compatible modifications which dramatically reduce energy consumption on 3G.

Then, we build the first complete system that not only is capable of streaming content to mobile devices but also allows them to publish content directly into the P2P system, even when they are behind a NAT gateway, with minimal impact on their battery and data usage.

While our preferred approach is implementing backwards-compatible modifications, we also propose and analyze backwards-incompatible ones. The former allow us to evaluate them in the existing large-scale systems and allow developers to deploy our modifications into the actual system. The latter free us to propose deeper changes. In particular, we propose (1) a DHT-based peer discovery mechanism that improves scalability and introduces localityawareness, and (2) modifications on Spotify’s gossip-like peer discovery to better accommodate mobile devices

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2013. p. x, 43
Series
Trita-ICT-ECS AVH, ISSN 1653-6363 ; 13:19
National Category
Communication Systems
Identifiers
urn:nbn:se:kth:diva-134608 (URN)978-91-7501-917-8 (ISBN)
Public defence
2013-12-11, Aula, Forum, KTH-ICT, Isafjordsgatan 39, Kista, 13:00 (English)
Opponent
Supervisors
Note

QC 20131203

Available from: 2013-12-03 Created: 2013-11-25 Last updated: 2013-12-04Bibliographically approved

Open Access in DiVA

fulltext(305 kB)348 downloads
File information
File name FULLTEXT01.pdfFile size 305 kBChecksum SHA-512
ee07abc59d498c6c0fac0a692932388000e0abdc76fe3db6b47d8d13571f1020a77e68163353cebe56f51a1cca16460e49b50850bfa25331ac3354f44aafd5bf
Type fulltextMimetype application/pdf

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Jimenez, RaúlOsmani, FlutraKnutsson, Björn
By organisation
Telecommunication Systems Laboratory, TSLab (closed 2012-01-01)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 348 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 123 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf