When Gnutella first came onto the scene in March of 2000, it was hailed as the next revolution in computing technology.It received considerable attention in the press, and users rushed to download Gnutella-compatible applications. However, as the year progressed, the public opinion of Gnutella turned more negative. People began to complain that Gnutella was too complicated, that downloads didn’t work often enough, that the network’s size was too small, and that there was no coherent sense of leadership in the development community.

Since late last year, there have been several major changes to the Gnutella Network. These changes will be discussed in reference to the three main problems with the Gnutella Network that have been addressed since late 2000: Download failure, Scalability, and Fragmented Development.

Download Failure

Download failures have plagued the Gnutella Network since its beginning, and turned many users off to the network in its early stages. Downloads fail for a variety of reasons, some of which are more avoidable than others.

In late 2000, many downloads were failing due simply to poorly written servent software which would mishandle download requests.This was a problem not only for users of these poorly written servents, but also for any other users trying to download files from them. The rise in popularity and widespread use of better-written Gnutella servents such as LimeWire v1.2, BearShare v2.0.9, Gnotella v0.9.7, and ToadNode v0.94 has helped the download problem significantly. However, undeveloped servents such as the “original” Gnutella v0.56 are still in use, and remain the source of many download failures.

Another significant cause of download failures in late 2000 was the growing use of browser-based Gnutella search programs such as asiayeah.com and gnute.com. Using these web pages, users were searching for and downloading files from the Gnutella Network without participating in the network either as content distributors or as query routers. Since the browser interfaces allowed them no means to pass network traffic along to other users or to share their own files with the network, they represented a drain on the network’s resources. Each host running a Gnutella node on the network has a certain number of available download “slots,” each of which can support one download from another user. Browser users were taking up download slots on other users’ machines by downloading files from the network, without ever giving download slots of their own back to the network by sharing files. As these download slots were filled, users who were sharing files with the network had to turn away other users requesting downloads, causing such download attempts to fail. Therefore, in conjunction with BearShare, Lime Wire implemented a browser-blocking feature in versions 1.0 and higher of the LimeWire servent, which refuses to upload files to non-sharing browser-based users. LimeWire currently sends these users an HTML document suggesting that they download a software servent such as LimeWire, capable of giving back to and participating fully in the network. BearShare also uses a similar feature.

One final reason that downloads were failing in late 2000 was user behavior on the Gnutella Network. Many people were not sharing any files at all; like browser users, these people use up download slots in their downloads without giving any slots back for uploads. Other users were running their servents for only a few minutes, which was not enough time for anyone else to successfully download one of their files. Though much of this sort of behavior is still going on, Lime Wire has been making a consistent effort to build features into the LimeWire servent to combat these problems, and to educate the Gnutella community on Gnutella “good citizen” behavior. For example, the LimeWire servent shares downloaded files with the network by default (which Gnutella v0.56 did not do) and warns users before closing down the LimeWire servent. For more information on “good citizen” behavior, please see http://www.limewire.com/citizen.htm.

Despite the advances that Gnutella has made since late 2000, download attempts still often fail, and various developers have had different ideas how to best address this problem. Kelly Truelove, founder and CEO of Clip2 DSS, suggested one such way in his recent paper “Gnutella: Alive, Well, and Changing Fast.” Here, Mr. Truelove argued that one possible way to improve download success rates would be to configure servents to respond to queries only if they have the resources to upload the file that matches the query. This way, users receiving query hits would not have to waste time attempting to download a file from a user who does not have the resources to upload it. A user who receives a query hit would be more confident that the download would succeed, and overall download success rates should in fact improve. However, Lime Wire has recently implemented the most revolutionary advance yet in Gnutella download technology, which obviates the need for the sort of technique suggested by Mr. Truelove. When a user attempts to download a file using the new LimeWire Smart Downloader and receives an error message, LimeWire will automatically try to download that file from the other sources that sent a sufficiently similar query hit. If all sources have been unsuccessfully attempted, LimeWire will repeat the cycle, trying sources that gave a busy signal before sources that couldn’t be found. This way, if user A has a rare file that user B is looking for, user A can return a query hit to user B even if user A does not have enough resources right now to transfer the file. User B can try to initiate the transfer, and, if he receives a busy signal, LimeWire will automatically re-attempt to initiate the download on user B’s behalf some period of time later. As an added bonus, the Smart Downloader feature motivates users to stay online longer in order to complete their own downloads. As a result, their shared files are available for upload for a longer period of time.

Though Gnutella still has far to go, it is worth noting that the innovations described above have had a significant impact on download success rates. As of late 2000, an estimated 10% of attempted downloads were successful; now, this figure is greater than 25%. Further, the LimeWire Smart Downloader has only been available since LimeWire v1.2, released February 2, 2001. Lime Wire fully expects the Smart Downloader to prompt the most dramatic improvement yet in Gnutella download success rates.

Scalability

In late 2000, Clip2 DSS reported in “Gnutella: To the Bandwidth Barrier and Beyond” that, on average, the largest responsive section of the public Gnutella Network at any one time consisted of approximately 400-800 hosts. Clip2 DSS attributed this relatively small network size to a so-called “modem bandwidth barrier.” The average network traffic, they reasoned, had grown to exceed dial-up modem bandwidth, preventing modem users from serving as effective nodes on the Gnutella Network. Modem users were actually hurting the network, acting as dead-ends through which normal network traffic could no longer flow, and resulting in a fragmentation of the public Gnutella Network into disconnected components.

Clip2 DSS argued that “further technical innovation” would be required for the public Gnutella Network to surpass the modem bandwidth barrier, and suggested one such technical innovation in the form of the Clip2 DSS Reflector. Low bandwidth connections such as modem users are able to connect to a public Reflector, upload their file index, and leave the Reflector to handle normal network traffic on their behalf. As a result, those nodes that are connected to the Reflector need to use their bandwidth only for file uploads and downloads, which they perform as usual. In allowing the Reflector to act as their proxy on the Gnutella Network, low-bandwidth users are prevented from choking on network traffic that they cannot handle.

The Clip2 Reflector represented one strategy for dealing with the modem bandwidth barrier, but it has not come into popular use on the Gnutella Network. Lime Wire has contributed significantly and in a variety of ways to the effort to come up with different ways to help scale the Gnutella Network. Based on this observation that modem users were becoming dead-end nodes on the network, Lime Wire decided that a different sort of technical innovation might also enhance the network’s scalability. Lime Wire concluded that the network could benefit from a “tiering” effect, where higher-bandwidth nodes are placed closer to the center of the network and lower-bandwidth nodes are placed closer to the periphery. Then, for example, a T1 user would connect to a band of other T1 users, which would all be capable of handling the average network traffic on that band. DSL users form their own band outside the T1 band, and modem users form a final band on the outside of that. The fastest connections, such as T3’s, would form the core of this “tiered” network. Thus, Lime Wire set to the project of re-structuring the network so that slower-bandwidth connections do not create choke points on the network when average traffic exceeds their bandwidth limitations. A few connections are maintained across bands so that each band does have an entry point for access to the other bands’ shared files.

Lime Wire has been able to accomplish this tiering effect using, first, the LimeWire Gateway, and, second, the “connection preferencing” rules built into the LimeWire servent. The LimeWire servent is configured to connect to the LimeWire Gateway on startup, so the two technologies work in concert to bring intelligent structuring and organization to the Gnutella Network. In addition, a number of other clients also connect to the LimeWire Gateway, so this effect will still take place even in the absence of LimeWire servent.

The first tool that Lime Wire has been able to use to help the Gnutella Network scale is the LimeWire Gateway. The LimeWire Gateway is a pong server, providing a newly connecting user with pong information about other hosts on the network (see Gnutella v. 0.4 protocol specification for details on pong messages). Pong messages contain the IP address, number of files, and total file size for a particular user on the Gnutella Network, and enable the user receiving this information to connect directly to the user it describes. The LimeWire Gateway employs various intelligent processes to determine the appropriate pong information to send to a user who connects to it. If I connect to the LimeWire Gateway using a modem connection, I will receive different pong information than if I connect on a T1 connection. The LimeWire Gateway is therefore able to affect the initial connectivity of hosts that connect to it, in order to intelligently structure the network to avoid bandwidth bottlenecks.

Lime Wire’s second tool has been the connection preferencing rules that have been built into the LimeWire servent software. Kelly Truelove quite correctly pointed out in “Gnutella: Alive, Well, and Changing Fast” that LimeWire’s connection preferencing rules tend to push slower connections to the outside of the network, where they will not serve as dead-end connections for faster nodes. LimeWire and BearShare are both configured to drop connections to unresponsive hosts; thus, if a modem user is unable to handle the traffic in a particular section of the network, other users will drop their connection to that modem user, forcing him to an area of the network where he can handle the traffic. Note that this need not happen just because a host has low bandwidth; if a host in unresponsive for any reason (for example, because it is running old or poorly written servent software) LimeWire will drop its connection to that host. In addition, LimeWire has implemented several more sophisticated connection preferencing rules (off by default in version 1.2), whereby connections from users sharing more files are preferenced over connections from users sharing fewer files. Thus, not only are low-bandwidth and other clogged hosts pushed toward the outside of the network, but non-sharing users are as well.

Again, the Gnutella Network still has far to go before it can reach its full scalability potential and accommodate millions of users, but we have already observed a dramatic increase in Gnutella usership since late 2000. Kelly Truelove estimated the average numbers of users on the Gnutella Network at any one time at around 1500, and Lime Wire’s research suggests that this number has recently gone as high as 11,000.(For more information, please see the LimeWire rolling host graph.) This represents an effective multiplication by ten of network size since late 2000, and the trend shows every promise of continuing.

Fragmented Development

The final problem with the Gnutella Network that has been addressed since late 2000 is the lack of a coherently organized body to guide Gnutella’s development.Gnutella lacked any such body in the beginning, and various attempts to create such a body have met with limited success.GPulp has never been able move past the proposal stage, and the so-called “Gnutella2” project has been long on hype but short on specifics.

In collaboration with all of the other leaders of the Gnutella community, including BearShare, Clip2 DSS, FirstPeer, Gnotella, Gnucleus, Mactella, Newtella, and ToadNode, Lime Wire has helped to establish the Gnutella Development Forum, where new ideas about Gnutella are discussed and evaluated in an organized and controlled way. All of the participants in the GDF have agreed to discuss changes and extensions to the Gnutella protocol before implementing them, so that the entire Gnutella community can move forward together. A formal proposal mechanism, including voting procedures, has been put into place, and the GDF has already agreed on several changes to be made in version 0.5 of the Gnutella protocol.

In addition, direct communication among the various Gnutella development entities has picked up since late 2000. Lime Wire has had the privilege of meeting directly with Gnutella leaders such BearShare, ToadNode, and Clip2 DSS, and is confident that these meetings will help enable the Gnutella Network to grow and develop.