Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Distributed Peer Discovery in Large-Scale P2P Streaming Systems: Addressing Practical Problems of P2P Deployments on the Open Internet
KTH, School of Information and Communication Technology (ICT), Communication Systems, CoS, Network Systems Laboratory (NS Lab).
2013 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Peer-to-peer (P2P) techniques allow users with limited resources to distribute content to a potentially large audience by turning passive clients into peers. Peers can self-organize to distribute content to each other, increasing the scalability of the system and decreasing the publisher’s costs, compared to a publisher distributing the data himself using a content delivery network (CDN) or his own servers.

Peer discovery is the mechanism that peers use to find each other. Peer discovery is a critical component of any P2P-based system, because P2P networks are dynamic by nature. That is, peers constantly join and leave the network and each individual peer is assumed to be unreliable. This thesis addresses practical issues in distributed peer discovery mech- anisms in the context of three different large-scale P2P streaming systems: a (1) BitTorrent-based streaming system, (2) Spotify, and (3) our own mobile P2P streaming system based on the upcoming Peer-to-peer Streaming Protocol (PPSP) Internet standard.

We dramatically improve peer discovery performance in BitTorrent’s Mainline DHT, the largest distributed hash table (DHT) overlay on the open Internet. Our implementation’s median lookup latency is an order of magnitude lower than the best performing measurement reported in the literature and does not exhibit a long tail of high-latency lookups, which is critical for P2P streaming applications.

We have achieved these results by studying how connectivity artifacts on the underlying network —probably caused by network address translation (NAT) gateways— affect the DHT overlay. Our measurements of more than three million nodes reveal that connectivity artifacts are widespread and can severely degrade DHT performance.

This thesis also addresses the practical issues of integrating mobile devices into P2P streaming systems. In particular, we enable P2P on Spotify’s Android app, study how distributed peer discovery affects energy consumption, and implement and evaluate backwards-compatible modifications which dramatically reduce energy consumption on 3G.

Then, we build the first complete system that not only is capable of streaming content to mobile devices but also allows them to publish content directly into the P2P system, even when they are behind a NAT gateway, with minimal impact on their battery and data usage.

While our preferred approach is implementing backwards-compatible modifications, we also propose and analyze backwards-incompatible ones. The former allow us to evaluate them in the existing large-scale systems and allow developers to deploy our modifications into the actual system. The latter free us to propose deeper changes. In particular, we propose (1) a DHT-based peer discovery mechanism that improves scalability and introduces localityawareness, and (2) modifications on Spotify’s gossip-like peer discovery to better accommodate mobile devices

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2013. , x, 43 p.
Series
Trita-ICT-ECS AVH, ISSN 1653-6363 ; 13:19
National Category
Communication Systems
Identifiers
URN: urn:nbn:se:kth:diva-134608ISBN: 978-91-7501-917-8 (print)OAI: oai:DiVA.org:kth-134608DiVA: diva2:668949
Public defence
2013-12-11, Aula, Forum, KTH-ICT, Isafjordsgatan 39, Kista, 13:00 (English)
Opponent
Supervisors
Note

QC 20131203

Available from: 2013-12-03 Created: 2013-11-25 Last updated: 2013-12-04Bibliographically approved
List of papers
1. CTracker: a Distributed BitTorrent Tracker Based on Chimera
Open this publication in new window or tab >>CTracker: a Distributed BitTorrent Tracker Based on Chimera
2008 (English)In: Collaboration and the Knowledge Economy: Issues, Applications, Case Studies / [ed] Paul Cunningham and Miriam Cunningham, Amsterdam: IOS Press , 2008, 941-947 p.Conference paper, Published paper (Refereed)
Abstract [en]

There are three major open issues in the BitTorrent peer discovery system, which are not solved by any of the currently deployed solutions. These issues seriously threaten BitTorrent's scalability, especially when considering that mainstream content distributors could start using BitTorrent for distributing content to millions of users simultaneously in the near future.

In this paper these issues are addressed by proposing a topology-aware distributed tracking system as a replacement for both centralized and Kademlia-based trackers.

An experiment measuring most popular open BitTorrent trackers is also presented. It shows that centralized trackers are not topology aware. We conclude that an ideal topology-aware tracker would return peers whose latency to the requester peer is significantly lower than of a centralized tracker.

Place, publisher, year, edition, pages
Amsterdam: IOS Press, 2008
Keyword
dht, p2p, bittorrent
National Category
Communication Systems Computer Systems
Identifiers
urn:nbn:se:kth:diva-46463 (URN)978-1-58603-924-0 (ISBN)
Conference
eChallenges 2008
Projects
P2P-Next
Funder
EU, FP7, Seventh Framework Programme, 21617
Note
QC 20111117Available from: 2011-11-17 Created: 2011-11-03 Last updated: 2013-12-03Bibliographically approved
2. Connectivity Properties of Mainline BitTorrent DHT Nodes
Open this publication in new window or tab >>Connectivity Properties of Mainline BitTorrent DHT Nodes
2009 (English)In: 2009 IEEE NINTH INTERNATIONAL CONFERENCE ON PEER-TO-PEER COMPUTING (P2P 2009), NEW YORK: IEEE , 2009, 262-270 p.Conference paper, Published paper (Refereed)
Abstract [en]

The birth and evolution of Feer-to-Peer (P2P) protocols have, for the most part, been about peer discovery. Napster, one of the first P2P protocols, was basically FTP/HTTP plus a way of finding hosts willing to send you the file. Since then, both the transfer and peer discovery mechanisms have improved, but only recently have we seen a real push to completely decentralized peer discovery to increase scalability and resilience. Most such efforts are based on Distributed Hash Tables (DHTs), with Kademlia being a popular choice of DHT implementation. While sound in theory, and performing well in simulators and testbeds, the real-world performance often falls short of expectations. Our hypothesis is that the connectivity artifacts caused by guarded hosts (i.e., hosts behind firewalls and NATs) are the major cause for such poor performance. In this paper, the first steps towards testing this hypothesis are developed. First, we present a taxonomy of connectivity properties which will become the language used to accurately describe connectivity artifacts. Second, based on experiments "in the wild", we analyze the connectivity properties of over 3 million hosts. Finally, we match those properties to guarded host behavior and identify the potential effects on the DHT

Place, publisher, year, edition, pages
NEW YORK: IEEE, 2009
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-30017 (URN)10.1109/P2P.2009.5284530 (DOI)000274540500041 ()2-s2.0-73549117904 (Scopus ID)978-1-4244-5066-4 (ISBN)
Conference
9th International Conference on Peer-to-Peer Computing Seattle, WA, SEP 09-11, 2009
Note

QC 20110221

Available from: 2011-02-21 Created: 2011-02-21 Last updated: 2013-12-04Bibliographically approved
3. Sub-Second Lookups on a Large-Scale Kademlia-Based Overlay
Open this publication in new window or tab >>Sub-Second Lookups on a Large-Scale Kademlia-Based Overlay
2011 (English)In: 11th IEEE International Conference on Peer-to-Peer Computing 2011 (P2P’11), IEEE , 2011, 82-91 p.Conference paper, Published paper (Refereed)
Abstract [en]

Previous studies of large-scale (multimillion node) Kademlia-based DHTs have shown poor performance, measured in seconds, in contrast to the far more optimistic results from theoretical analysis, simulations and testbeds. In this paper, we unexpectedly find that in the Mainline BitTorrent DHT (MDHT), probably the largest DHT overlay on the Internet, many lookups already yield results in less than a second, albeit not consistently. With the backwards-compatible modifications we present, we show that not only can we reduce median latencies to between 100 and 200 ms, but also consistently achieve sub-second lookups. These results suggest that it is possible to deploy latency-sensitive applications on top of large-scale DHT overlays on the Internet, contrary to what some might have concluded based on previous results reported in the literature.

Place, publisher, year, edition, pages
IEEE, 2011
Series
IEEE International Conference on Peer-to-Peer Computing, ISSN 2161-3567
Keyword
dht, kademlia, performance, large-scale
National Category
Computer and Information Science
Identifiers
urn:nbn:se:kth:diva-38313 (URN)10.1109/P2P.2011.6038665 (DOI)000298838500010 ()2-s2.0-80055024024 (Scopus ID)978-1-4577-0149-8 (ISBN)
Conference
11th IEEE International Conference on Peer-to-Peer Computing 2011 (P2P’11)
Funder
EU, FP7, Seventh Framework Programme, No. 216217
Note
QC 20110825Available from: 2011-10-13 Created: 2011-08-24 Last updated: 2013-12-03Bibliographically approved
4. Swift: The missing link between peer-to-peer and information-centric networks
Open this publication in new window or tab >>Swift: The missing link between peer-to-peer and information-centric networks
2012 (English)In: Proceeding P2P-Dep '12 Proceedings of the First Workshop on P2P and Dependability, New York: ACM , 2012, 4- p.Conference paper, Published paper (Refereed)
Abstract [en]

A common pitfall of many proposals on new information-centric architectures for the Internet is the imbalance of upfront costs and immediate benefits. If properly designed and deployed, information-centric architectures can accommodate the current Internet usage which is at odds with the historical design of the Internet infrastructure. To address this concern, we focus on prospects of incremental adoption of this paradigm by introducing a peer-to-peer based transport protocol for content dissemination named Swift that exhibits properties required in an Information-Centric Network (ICN), yet can be deployed in the existing Internet infrastructure. Our design integrates components while highly prioritizing modularity and sketches a path for piecemeal adoption which we consider a critical enabler of any progress in the field.

Place, publisher, year, edition, pages
New York: ACM, 2012
Keyword
Distributed systems, Networking, Transport protocol
National Category
Communication Systems
Identifiers
urn:nbn:se:kth:diva-99943 (URN)10.1145/2212346.2212350 (DOI)2-s2.0-84861548802 (Scopus ID)978-1-4503-1148-9 (ISBN)
Conference
1st Workshop on P2P and Dependability, P2P-Dep'12 - In Conjunction with 9th European Dependable Computing Conference, EDCC 2012; Sibiu; 8 May 2012 through 8 May 2012
Note

QC 20120802

Available from: 2012-08-02 Created: 2012-08-02 Last updated: 2013-12-04Bibliographically approved
5. Integrating Smartphones in Spotify's Peer-Assisted Music Streaming Service
Open this publication in new window or tab >>Integrating Smartphones in Spotify's Peer-Assisted Music Streaming Service
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Spotify is a large-scale peer-assisted music streaming service. Spotify’s P2P network serves 80% of music data to desktop clients. On the other hand, the rapidly growing number of mobile clients do not use P2P but instead stream all data from Spotify’s servers.

We enable P2P on a Spotify mobile client and empirically eval- uate the impact of P2P protocols (in particular low-bandwidth traffic between peers) on energy consumption, both on 3G and

Wifi. On 3G, current P2P protocols are highly energy inefficient, but simple modifications bring consumption close to the client-server configuration. On Wifi, the extra energy cost of enabling P2P is much lower.

Finally, we propose a protocol modification to further integrate mobile devices in Spotify’s P2P network according to their capa- bilities (power source, access network). This allows us to break the artificial division between desktop and mobile platforms and dynamically adapt as resources become (un)available to the device.

 

National Category
Electrical Engineering, Electronic Engineering, Information Engineering Communication Systems
Identifiers
urn:nbn:se:kth:diva-134609 (URN)
Note

QS 2013

Available from: 2013-11-25 Created: 2013-11-25 Last updated: 2013-12-03Bibliographically approved
6. Tribler Mobile: P2P Video Streaming from and to Mobile Devices
Open this publication in new window or tab >>Tribler Mobile: P2P Video Streaming from and to Mobile Devices
Show others...
(English)Manuscript (preprint) (Other academic)
Abstract [en]

Peer-to-peer (P2P) mechanisms allow users with limited resources to distribute content to a large audience, without the need of intermediaries.

These P2P mechanisms, however, appear to be ill-suited for mobile devices, given their limited resources: battery, bandwidth, and connectivity. Even Spotify, a commercial straming service where desktop clients stream about 80% of the data via P2P, does not use P2P on mobile devices.

This paper describes Tribler Mobile, a mobile app that allows users to broadcast their own videos to potentially large audiences

directly from their devices. Our system delegates most of the distribution tasks to boosters running on desktop computers. Our mechanisms are designed to be fully-decentralized and consider mobile devices’ limitations.

Tribler Mobile is available as open-source software and have been installed by almost 500 users on their Android devices.

 

National Category
Communication Systems
Identifiers
urn:nbn:se:kth:diva-134610 (URN)
Note

QS 2013

Available from: 2013-11-25 Created: 2013-11-25 Last updated: 2013-12-03Bibliographically approved

Open Access in DiVA

jimenez2013thesis.pdf(2891 kB)1773 downloads
File information
File name FULLTEXT04.pdfFile size 2891 kBChecksum SHA-512
c05b0913a4ff2f0470b6f67faf1dd603998068aa8c0bc79e685a6d5023e6216de70b4cb71e41c53e0b58b820b9389c8d29fb05f7a5e0864550a0027d1fb39f33
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Jimenez, Raul
By organisation
Network Systems Laboratory (NS Lab)
Communication Systems

Search outside of DiVA

GoogleGoogle Scholar
Total: 1782 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 2133 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf