Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Dealing with Network Partitions and Mergers in Structured Overlay Networks
KTH, School of Information and Communication Technology (ICT), Electronic, Computer and Software Systems, ECS.
2009 (English)Licentiate thesis, comprehensive summary (Other academic)
Abstract [en]

Structured overlay networks form a major classof peer-to-peer systems, which are touted for their abilitiesto scale, tolerate failures, and self-manage. Any long livedInternet-scale distributed system is destined to facenetwork partitions. Although the problem of network partitionsand mergers is highly related to fault-tolerance andself-management in large-scale systems, it has hardly beenstudied in the context of structured peer-to-peer systems.These systems have mainly been studied under churn (frequentjoins/failures), which as a side effect solves the problemof network partitions, as it is similar to massive nodefailures. Yet, the crucial aspect of network mergers has beenignored. In fact, it has been claimed that ring-based structuredoverlay networks, which constitute the majority of thestructured overlays, are intrinsically ill-suited for mergingrings. In this thesis, we present a number of research papers representing our work on handling network partitions and mergers in structured overlay networks. The contribution of this thesis is threefold. First, we provide a solution for merging ring-based structured overlays. Our solution is tuneable, by a {\em fanout} parameter, to achieve a trade-off between message and time complexity. Second, we provide a network size estimation algorithm for ring-based structured overlays. We believe that an estimate of the current network size can be used for tuning overlay parameters that change according to the network size, for instance the fanout parameter in our merger solution.Third, we extend our work from fixing routing anomalies to achieving data consistency. We argue that decreasing lookup inconsistencies on the routing level aids in achieving data consistency in applications built on top of overlays. We study the frequency of occurence of lookup inconsistencies and discuss solutions to decrease the affect of lookup inconsistencies.

Place, publisher, year, edition, pages
Stockholm: KTH , 2009. , 87 p.
Series
Trita-ICT-ECS AVH, ISSN 1653-6363 ; 09:01
Keyword [en]
Structured Overlay Networks, Distributed Hash Tables, DHTs, Network Partitions, Network Mergers, Network Size Estimation, Lookup Inconsistencies, Gossiping
National Category
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-10211ISBN: 978-91-7415-290-6 (print)OAI: oai:DiVA.org:kth-10211DiVA: diva2:211208
Presentation
2009-05-08, Sal D, Kungl Tekniska Högskolan, Isajordsgatan 39, Kista, Forum IT-Universitetet, 14:00 (English)
Opponent
Supervisors
Available from: 2009-04-15 Created: 2009-04-09 Last updated: 2010-11-01Bibliographically approved
List of papers
1. Dealing with network partitions in structured overlay networks
Open this publication in new window or tab >>Dealing with network partitions in structured overlay networks
2009 (English)In: Peer-to-Peer Networking and Applications, ISSN 1936-6442, Vol. 2, no 4, 334-347 p.Article in journal (Refereed) Published
Abstract [en]

Structured overlay networks form a major class of peer-to-peer systems, which are touted for their abilities to scale, tolerate failures, and self-manage. Any long-lived Internet-scale distributed system is destined to face network partitions. Although the problem of network partitions and mergers is highly related to fault-tolerance and self-management in large-scale systems, it has hardly been studied in the context of structured peer-to-peer systems. These systems have mainly been studied under churn (frequent joins/failures), which as a side effect solves the problem of network partitions, as it is similar to massive node failures. Yet, the crucial aspect of network mergers has been ignored. In fact, it has been claimed that ring-based structured overlay networks, which constitute the majority of the structured overlays, are intrinsically ill-suited for merging rings. In this paper, we present an algorithm for merging multiple similar ring-based overlays when the underlying network merges. We examine the solution in dynamic conditions, showing how our solution is resilient to churn during the merger, something widely believed to be difficult or impossible. We evaluate the algorithm for various scenarios and show that even when falsely detecting a merger, the algorithm quickly terminates and does not clutter the network with many messages. The algorithm is flexible as the tradeoff between message complexity and time complexity can be adjusted by a parameter.

Keyword
DHTs, Distributed hash tables, Loopy rings, Network mergers, Network partitions, Structured overlay networks
National Category
Computer Science
Identifiers
urn:nbn:se:kth:diva-25807 (URN)10.1007/s12083-009-0037-7 (DOI)000284092500005 ()2-s2.0-70350616363 (Scopus ID)
Note
QC 20101101Available from: 2010-11-01 Created: 2010-11-01 Last updated: 2012-01-18Bibliographically approved
2. Key-based consistency and availability in structured Overlay Networks
Open this publication in new window or tab >>Key-based consistency and availability in structured Overlay Networks
Show others...
2008 (English)In: Proceedings of the 17th International Symposium on High Performance Distributed Computing 2008, HPDC'08, 2008, 235-236 p.Conference paper, Published paper (Refereed)
Abstract [en]

Structured Overlay Networks (SONs) provide a promising platform for high performance applications since they are scalable, fault-tolerant and self-managing. SONs provide lookup services that map keys to nodes that can be used as processing or storage resources. In SONs, lookups for a key may return inconsistent results. Consequently, it is difficult to provide consistent data services on top of SONs that build on key-based search. In this paper, we study the frequency of occurrence of inconsistent lookups. We show that the affect of lookup inconsistencies can be reduced by using node responsibilities. We present our results as a trade-off between consistency and availability of keys.

Keyword
Key-availability, Lookup consistency, Peer-to-peer systems, Structured overlay networks
National Category
Computer Science
Identifiers
urn:nbn:se:kth:diva-25810 (URN)2-s2.0-57349084604 (Scopus ID)978-159593997-5 (ISBN)
Conference
17th International Symposium on High Performance Distributed Computing 2008, HPDC'08; Boston, MA; 23 June 2008 through 27 June 2008
Note
QC 20101101Available from: 2010-11-01 Created: 2010-11-01 Last updated: 2010-11-01Bibliographically approved
3. A Practical Approach to Network Size Estimation for Structured Overlays
Open this publication in new window or tab >>A Practical Approach to Network Size Estimation for Structured Overlays
2008 (English)In: SELF-ORGANIZING SYSTEMS, PROCEEDINGS / [ed] Hummel KA; Sterbenz JPG, Berlin: SPRINGER-VERLAG , 2008, Vol. 5343, 71-83 p.Conference paper, Published paper (Refereed)
Abstract [en]

Structured overlay networks have recently received much attention due to their self-* properties under dynamic and decentralized settings. The number of nodes in all overlay fluctuates all the time due to churn. Since knowledge of the size of the. overlay is a core requirement for many systems, estimating the size in a decentralized manner is a challenge taken up by recent research activities. Gossip-based Aggregation has been shown to give accurate estimates for the network size, but previous work done is highly sensitive to node failures. In this paper, we present a gossip-based aggregation-style network size estimation algorithm. We discuss shortcomings of existing aggregation-based size estimation algorithms, and give a solution that is highly robust to node failures and is adaptive to network delays. We examine our solution in various scenarios to demonstrate. its effectiveness.

Place, publisher, year, edition, pages
Berlin: SPRINGER-VERLAG, 2008
Series
LECTURE NOTES IN COMPUTER SCIENCE, ISSN 0302-9743 ; 5343
Keyword
Cybernetics, Distributed computer systems, Technical presentations
National Category
Computer Science
Identifiers
urn:nbn:se:kth:diva-25811 (URN)000263515100007 ()2-s2.0-58349117706 (Scopus ID)978-3-540-92156-1 (ISBN)
Conference
3rd International Workshop on Self-Organizing Systems, Vienna, AUSTRIA, DEC 10-12, 2008
Note

QC 20101101

Available from: 2010-11-01 Created: 2010-11-01 Last updated: 2017-06-08Bibliographically approved

Open Access in DiVA

fulltext(2678 kB)599 downloads
File information
File name FULLTEXT02.pdfFile size 2678 kBChecksum SHA-512
069412bcf8d661d340b3a8e9a6a04997ddf558714a0a531ae43730461d08502adc5cf40f4ead716a0da2019a3b7213c711bf06651d0ca8ef7f5504996e297c55
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Shafaat, Tallat Mahmood
By organisation
Electronic, Computer and Software Systems, ECS
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 600 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 265 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf