DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

[0064] Embodiments of a distributed index mechanism for indexing and searching for content in peer-to-peer networks are described. In one embodiment, peers with sufficient storage and networking capacity may be able to participate in a distributed index. In one embodiment, content may include XML documents, and search queries may be formulated as XPath queries. In one embodiment, content may include text documents, and search queries may include one or more terms from the corresponding documents. In one embodiment, the distributed index may be a distributed hash table, with participating peers each responsible for a range (zone) of values of hashes that may be generated by a hash function. Other embodiments may use other distributed indexing schemes. For example, each participating peer may be responsible for a range (zone) of a given alphabet.

[0065] Embodiments of the distributed index mechanism may be used to implement content management services, presence services and notification services, among other applications and services. In one embodiment, a distributed index mechanism may be used to implement a notification service that indexes queries and routes content to query generators when an item of content that matches the query is published. Some embodiments may be implemented in peer-to-peer networks implemented according to a peer-to-peer platform such as the exemplary peer-to-peer platform described below.

[0066] In one embodiment, a distributed inverted index may be used to map each query item (e.g. keyword or XML path) to a list of postings. The posting definition may depend on the application of the distributed index. Posting objects may include information about corresponding document containing query items associated with the posting objects. In one embodiment, posting objects may include relevance information for query items in corresponding documents. For example, a posting may include one or more of, but is not limited to, a document identifier, document expiration information, location identifiers of the query item in the document, and frequency information of the query item inside the document. In one embodiment, a distributed index may be used to map query items to postings. In one embodiment, a content-addressable network may be used as an underlying distributed index infrastructure. In a content-addressable network, each keyword is transformed using a hash function into one (or possibly a multi-dimensional) number. Each peer in the distributed index is assigned a responsibility for indexing a range (zone) of numbers. In addition, a connection topology and a routing mechanism (e.g. a module such as an application or service) may be used for reaching any zone from any other zone.

[0067] Using the distributed index and connection topology and routing algorithm(s), there is no central identity or server. A first peer may initially handle all the space in the distributed index. When another peer joins the distributed index, the distributed index may be split between the two peers to create two spaces or zones of the distributed index. If another peer joins one of the peers, the zone on the peer may be split with the joining peer. Peers may continue to join and be allotted zones in the distributed index. The distributed index may be one or more dimensions, and may be thought of as an n-dimensional space. The topology generated by the various peers joining the distributed index determines how content are added and how queries are routed.

[0068] Queries and content may be routed through one or more intermediate nodes. Participating peers may include routing information and a routing service to route messages, including add content and query messages, to one or more other peers (i.e. other zones) in the distributed index.

[0069] When content is stored in the distributed index, e.g. a name or document, a hashing algorithm (e.g. MD5 or SHA-1) may be used to generate a hashing key from the content or one or more attributes or portions of the content (e.g. one or more terms from a text document, a document name, a peer identifier, etc.). Whatever content is to be stored is routed to and stored at the location (zone) indicated by the key value. There may be a range of zones (e.g. 0-10000) that may be split among two or more peers. Queries are hashed and routed to zones indicated by the key value generated by the hash to access the corresponding content.

[0070] If a peer receives a query with multiple terms, the terms may be split and routed to multiple peers (zones); results of the multiple term queries are returned to the peer that initially received, split and routed the query, which returns the results (content, or alternatively information on how to access the content) to the querying peer.

[0071] FIG. 34 illustrates several peers participating in a distributed index according to one embodiment. Each peer 200 may be allotted a zone 400 of the distributed index, and is responsible for storing content in that zone and for providing content from that zone in response to queries. In this example, peer 200 A may have been a first peer in the distributed index, and originally included the entire index in zone 400 A. Peer 200 B may have joined peer 200 A, and the index may have been split into zones 400 A and 400 B. Peer 200 C may have later joined peer 200 A, and zone 400 A may have again been split into zones 400 A and 400 C. Peer 200 D may have later joined peer 200 C, and zone 400 C may have been split into zones 400 C and 400 D.

[0072] FIG. 35 illustrates a peer participating in a distributed index according to one embodiment. In this example, peer 200 A includes zone 400 A. In one embodiment, each peer 200 participating in the distributed index may include a hashing mechanism 402 and a routing mechanism. Hashing mechanism may receive incoming content to be added to the distributed index and/or incoming queries for content of the distributed index. Incoming queries or content may be hashed by the hashing mechanism to produce a key (which may also be referred to as a hash) that may be used to determine the zone 400 that the content is to be added to or the zone 400 in which queried content is stored. The key may be passed to the routing mechanism 404 to be routed to the appropriate zone.

[0073] Returning to FIG. 34 , as an example of routing content to be added to the distributed index, peer 200 A may receive a content to be added to the distributed index. Peer 200 A may hash the content (or one or more attributes or portions of the content) to produce a key. The key may indicate the zone to which the content belongs. If the content belongs in zone 400 A, peer 200 A stores the content. If the content does not belong in zone 400 A, peer 200 A may route the content to the peer corresponding to zone 400 indicated by the key. For example, the key may indicate the content belongs in zone 400 C, and peer 200 A may send the content to peer 200 C. In one embodiment, peer 200 A may send the content to peer 200 C and peer 200 C may hash the content to generate the key. In another embodiment, peer 200 A may send the key with the content. Peer 200 C has previously split zone 400 C with peer 200 D. Peer 200 C may determine from the key if the content belongs in zone 400 C. If it does, the content may be added to zone 400 C. If the content does not belong in zone 400 C, peer 200 C routes the content to peer 200 D to be added to zone 400 D. Note that this example may also be applied to routing queries.

[0074] As an example of routing queries, peer 200 B may receive a query for content in the distributed index. Peer 200 B may hash the query to generate a key. Peer 200 B may then determine from the key if the query is for content in zone 400 B. If the query is for content in zone 400 B, then peer 200 B may supply the content to the querying peer. If the query is not for content in zone 400 B, peer 200 B routes the query to peer 200 A. In one embodiment, peer 200 B may send the query and peer 200 A may hash the query to generate the key. In another embodiment, peer 200 B may send the key with the query. Peer 200 A may then use the key to determine if the content is in zone 400 A. If the query is for content in zone 400 A, then peer 200 A may supply the content to the querying peer. In one embodiment, peer 200 A may send the content to peer 200 B, which may then route the content to the querying peer. If the query is not for content in zone 400 A, peer 200 A may route the query to the peer corresponding to the zone 400 indicated by the key. In this example, peer 200 A determines that the query is for content in zone 400 C (and not zone 400 B) and routes the query to peer 200 C as indicated by the query. Peer 200 C determines if the query is for content in zone 400 C or 400 D. If the query is for content in zone 400 C, then peer 200 C may supply the content to the querying peer. In one embodiment, peer 200 C may route the content to peer 200 A, which may then route the content to peer 200 B, which may then route the content to the querying peer. If the query is for content in zone 400 C, peer 200 C may route the query to peer 200 D, which may then route the content to the querying peer. In one embodiment, peer 200 D may route the content to peer 200 C, which may then route the content to peer 200 A, which may then route the query to peer 200 B, which may then route the content to the querying peer.

[0075] Note that in one embodiment, a peer 200 including the zone 400 of the distributed index indicated by the key may respond directly to a requesting peer to provide requested content rather than routing the response through one or more other peers 200 participating in the distributed index. Also, note that when the query reaches the peer 200 including the zone 400 of the distributed index indicated by the key, the requested content may not be stored. In this case, the requesting peer may be notified that the content is not stored in the distributed index.

[0076] In one embodiment, actual content (e.g. documents) may not be stored on the peers participating in the distributed index. Instead, one or more mechanisms for accessing the content may be stored. For example, information for accessing the content such as a URL or other location indicator may be stored in the distributed index. In response to a query, the information for accessing the content may be returned to the querying peer, which may then access the content using the provided information for accessing the content.

[0077] In one embodiment, to index resources addressable by a single attribute such as a file name or an identifier, an implementation of a distributed index based on a content-addressable network may be used. A keyword is transformed into a number using a global hash function. The keyword-value may then be routed to the peer responsible for the zone indicated by the number generated from the keyword. For example, the keyword may be a string such as a person's name or a file name, and the value may be the peer address of the peer responsible for the zone.

[0078] Documents may be formed of many attributes. For example, an XML document may be represented in general by a set of XML paths. A text document may be represented by a set of terms together with some measure of term strengths. Similarly, a query may be a logical expression of terms or XML paths. In one embodiment, to build a distributed index for documents, each piece of content (e.g. a document) may have a unique content identifier. The content identifier may be mapped to a peer assigned to a zone using a global hash function to map the content to the zone. This peer is responsible for parsing the document into its basic components and indexing the components (keywords or XML paths). In addition, the peer may map each basic component into a number using the global hash function, and send the component-posting pair to the peer responsible for the component zone. The posting may include a reference to the content identifier.

[0079] In one embodiment, a document or other content may be retrieved by sending a query to any peer participating in the distributed index. The peer receiving the query parses the query into its basic components and maps the components to the peers responsible for indexing those components. The peers that index the basic components retrieve the content identifier from the postings and send the complete query to the peer responsible for indexing the content identifier. The peer responsible for indexing the content identifier matches the full query with the full content.

[0080] Embodiments of the distributed index mechanism may provide functions including, but not limited to, functions for joining and leaving the network, inserting content in the distributed index, and querying the distributed index. One embodiment may provide one or more mechanisms for indexing and retrieving complex content, such as markup text, free text, XML documents, etc. In one embodiment, the distributed index mechanism may be used for more complex applications such as personalized content delivery (personalized news). In this embodiment, keywords and subjects of interest may be registered in a distributed index, and when a content including the registered metadata appears on the network, all nodes that have registered for that content may be notified.

[0081] In one embodiment, the insert content function may allow edge peers and “freeloaders” (peers not participating in the distributed index) to benefit from the distributed index without having to join the distributed index (i.e. without having to participate in the routing or having to maintain an index).

[0082] In one embodiment, each item of content may have an associated expiration time, at or after which the content will expire if not renewed. This approach may be tolerant to unreliable peers, and may simplify content propagation. In one embodiment, content may be renewed by republishing the content before it expires. Republishing content before expiration may help to ensure that new content is published at the right peer (i.e. in the correct zone and location of the distributed index). For example, the distributed index may become slightly inconsistent due to participating peers joining and/or leaving the distributed index. In another embodiment, to renew an item of content, a peer in the peer-to-peer network may send a renew content request message to a peer participating in the distributed index, and the message may be routed to the peer(s) indexing the content, which may then reset the expiration time.

[0083] Peer-to-Peer Distributed Index of Identity Information

[0084] In one embodiment, a distributed index may be used to store identity information in a decentralized manner on a plurality of peer nodes. Identity information may be used by websites that authenticate users, for example. Distributed indexes may allow identity information to be spread across multiple peer nodes so that the load is spread among the various peer nodes. In one embodiment, the spreading of identity information may be done redundantly so that peer node failures preferably do not affect the system adversely. Peer nodes may be added to the pool as the load grows to provide scalability. In one embodiment, the distributed index of identity information may be a distributed hash table. Other embodiments may use other distributed indexing schemes. One embodiment of a distributed index of identity information may be implemented in peer-to-peer networks implemented according to a peer-to-peer platform such as the exemplary peer-to-peer platform described below.

[0085] As an example, in a peer-to-peer network, a first peer may use an identity A, and a second peer may use an identity B. Each peer identity may have associated information such as a user name, peer identifier and/or name, a password, certificate, and other authentication information and usage information. Identity information such as user names, peer identifiers and/or names, passwords, certificates and associated information may be stored in a distributed index. When a peer is joining the network, the peer may request to use identity A. The mechanism may use the identity A to locate corresponding information in the distributed index. Identity A may be hashed to generate. a key to route the request to the corresponding zone as described above. If a peer is already using identity A, the peer may be asked for a password, or alternatively may be requested to submit a different identity. In one embodiment, if a peer attempts to join the peer-to-peer network using a particular identity, the mechanism may ask for authentication and/or other information to authenticate the peer.

[0086] In one embodiment, a distributed index of identity information may be implemented in a peer-to-peer network implemented according to a peer-to-peer platform such as the exemplary peer-to-peer platform described below and used by peer groups in authenticating peer nodes desiring to join the peer groups. In this embodiment, one or more members of a peer group may provide a peer group membership service using a distributed index of identity information. Peers desiring to join a peer group in the peer-to-peer network may submit identity information (e.g. one or more of a user name, peer name, password, certificate, etc.) to a member peer in the peer group. The peer group may be the peer group implementing the peer group membership service or, in one embodiment, a different peer group using the membership service provided by the peer group. The member peer of the peer group may then provide the identity information to the authentication service, which may hash the identity information to find corresponding identity information (e.g. a peer or user name and associated correct password) and use the corresponding information to determine if the peer may join the peer group, to request the peer to submit more information, and/or to perform other peer group membership tasks.

[0087] FIG. 36 is a flowchart illustrating a method of authenticating users in a network using a distributed index of identity information stored on a plurality of peer nodes according to one embodiment. As indicated at 420 , a distributed index of identity information may be stored across one or more peers. This identity information may be used, for example, in authenticating users in the peer-to-peer network. Each peer may store a zone of the distributed index that includes identity information for a subset of the key(s) used to store and retrieve content from the distributed index. For example, user names may be hashed and used as keys to store and retrieve corresponding identity information. Identity information may include, but is not limited to, information such as a user name, peer identifier and/or name, a password, certificate, and other authentication information and usage information.

[0088] As indicated at 422 , a peer node participating in the distributed index may receive a request for authentication, which may include information identifying a user (referred to as a user identifier), for example a user name. In one embodiment, a hash of user identifiers is used to store and retrieve corresponding identity information from the distributed index. As indicated at 424 , the received user identifier may be hashed to produce a hash (i.e. a key). As indicated at 426 , the hash of the user identifier may be used to determine a zone of the distributed index corresponding to a range of hash values that may include the hash of the user identifier. In one embodiment, the request for authentication may be routed to the peer node storing the determined zone.

[0089] As indicated at 428 , the hash of the user identifier may be used to determine if the user identifier is in use and/or if there is identity information stored at the location indicated by the hash. As indicated at 430 , if the user identity is not in use, then identity information corresponding to the user identifier may be added to the location in the zone indicated by the hash of the user identifier to add the new user identifier (e.g. user name) to the distributed index. In one embodiment, the peer that originated the authentication request may be prompted to provide more identity information (e.g. password, certificate, etc.) for storing at the location in the distributed index corresponding to the hash of the user identifier.

[0090] As indicated at 432 , if the user identity is in use, authentication of the user identifier may be performed using the identity information stored at the location of the zone indicated by the hash of the user identifier. For example, the peer that originated the authentication request may be prompted to provide more identity information (e.g. password, certificate, etc.) to be compared to identity information stored at the distributed index location indicated by the hash of the user identifier. Authentication of the user identifier may be granted or denied based on results of the comparison, and notification of the results may then be provided to the peer that originated the authentication request.

[0091] While the distributed index of identity information is described above for identity information corresponding to peer nodes in a peer-to-peer networking environment, it is noted that identity information for other nodes in other networking environments may be stored in a distributed index of identity information on a plurality of peer nodes.

[0092] Peer-to-Peer Distributed Index of Presence Information

[0093] In one embodiment, a distributed index may be used to store presence information in a decentralized manner on a plurality of peer nodes. Presence information may be used in an Internet messaging system, for example. Distributed indexes allow presence information to be spread across multiple peer nodes so that the load is spread among the various peer nodes. In one embodiment, this spreading may be done redundantly so that a few peer node failures preferably do not affect the system adversely. In one embodiment, a notification service may be implemented using the distributed index to notify peers of other peers joining a peer-to-peer session. In one embodiment, the distributed index of presence information may be a distributed hash table. Other embodiments may use other distributed indexing schemes. One embodiment of a distributed index of presence information may be implemented in peer-to-peer networks implemented according to a peer-to-peer platform such as the exemplary peer-to-peer platform described below.

[0094] One embodiment may provide a message to register for join notifications. Peer nodes may use the message to register to be notified when certain other peers join or insert themselves into the distributed index. When a peer joins a session, peers already present in the session who have registered to be notified if the peer joins the session may be notified of the peer joining the session. This embodiment may be used, for example, as a presence system for buddy list-type notifications in chat applications.

[0095] As an example, a first peer may join a chat session. Information corresponding to the first peer may be stored in the distributed index. The first peer may register to be notified if and when other specified peers join the chat session. When a specified second peer joins the chat session, the second peer's presence information is registered in the distributed index. Because the first peer has registered to be notified if the second peer joins, the first peer is notified of the second peer's presence in the chat session.

[0096] If a peer desires to be informed of the presence of another peer, the peer registers a query for the other peer (e.g. “look for peer X”) in the distributed index. Whenever the other peer joins, the peer that registered the query for the other peer may be notified by the notification service that the other peer has joined. In one embodiment, the notification service may provide information on how to contact the other peer to the peer that registered the query.

[0097] In one embodiment, peer presence information may be an advertisement stored in the distributed has table similar to the advertisements described for the exemplary peer-to-peer platform described below. Peer presence information may be stored as markup language (e.g. XML) documents, text documents, binary documents, or in any other computer data format or combination of formats.

[0098] In one embodiment, participating peers may search or query the distributed index of presence information to discover the presence of other participating peers. In one embodiment, a first peer may send a query message including an identifier for a second peer. The identifier may be hashed and used to determine if the second peer is registered in the distributed index of presence information. If the second peer is registered, information including contact information for the second peer may be returned to the first peer. In one embodiment, a search mechanism may be provided to allow participating peers, including peers joining a session, to search the distributed index for registered peers that may be of interest to the searching peer.

[0099] FIGS. 37 and 38 are flowcharts illustrating presence detection in a network using a distributed index stored on a plurality of peer nodes according to one embodiment. FIG. 37 is a flowchart illustrating registering to receive notification of a peer joining a peer session according to one embodiment. As indicated at 440 , one of the plurality of peer nodes may receive a request for notification of registration of a particular peer from another peer in the network. As indicated at 442 , the request for notification of registration of the particular peer may be stored in the distributed index of presence information. In one embodiment, the request may include identification information for the particular peer (e.g. a user identifier, peer identifier, name, etc.) For simplicity, the term “user identifier” may be used herein to refer to the identification information. In one embodiment, the user identifier may be hashed and the request for notification of registration of the particular peer may be stored in the distributed index of presence information at a location indicated by the hash of the user identifier. The location may be in a particular zone of the distributed index, and if the zone is not on the peer node that received the request for notification of registration of the particular peer, the request may be routed to the peer node that stores the zone.

[0100] As indicated at 444 , the particular peer may desire to join the session and may generate a registration request. The registration request may be received by one of the peer nodes participating in the distributed index. As indicated at 446 , the stored request for notification from the distributed index may be accessed from the distributed index in response to receiving the registration request. In one embodiment, the registration request may include the user identifier for the particular peer. In one embodiment, the user identifier may be hashed and the request for notification of registration of the particular peer may be accessed from the distributed index of presence information at a location indicated by the hash of the user identifier. The location may be in a particular zone of the distributed index, and if the zone is not on the peer node that received the registration request, the request may be routed to the peer node that stores the zone.

[0101] As indicated at 448 , the peer that sent the request for notification of registration of the particular peer may be notified that the particular peer has joined in response to accessing the request for notification of registration of the particular peer. In one embodiment, the notification may include information for contacting the particular peer on the node on which the particular peer is running. In one embodiment, the request for notification of registration of the particular peer may include information identifying the sender of the request for notification that may be used to send the notification of registration of the particular peer to the peer that originated the notification request.

[0102] FIG. 38 is a flowchart illustrating a peer joining a peer session according to one embodiment. As indicated at 460 , the particular peer may generate a registration request to join the session. The registration request may be received by one of the peer nodes participating in the distributed index. As indicated at 462 , presence information for the peer may be stored in the distributed index. In one embodiment, the registration request may include a user identifier for the particular peer. In one embodiment, the user identifier may be hashed and the presence information for the peer may be stored in the distributed index of presence information at a location indicated by the hash of the user identifier. The location may be in a particular zone of the distributed index, and if the zone is not on the peer node that received the request for notification of registration of the particular peer, the presence information may be routed to the peer node that stores the zone. As indicated at 464 , presence information stored in the distributed index to the peer and corresponding to one or more other registered peers in the session may be provided to the registering peer. In one embodiment, the presence information for the one or more other registered peers may include information for contacting nodes on which the other peers are running. Any other peers in the session that have previously registered notification requests for the registering peer may be notified that the peer has joined the session. The notification(s) may include information for contacting the registering peer on a node on which the registering peer is running.

[0103] While the distributed index of presence information is described above for presence information corresponding to peer nodes in a peer-to-peer networking environment, it is noted that presence information for other nodes in other networking environments may be stored in a distributed index of presence information on a plurality of peer nodes.

[0104] Peer-to-Peer Instant Messaging Using Distributed Indexes

[0105] In one embodiment, a distributed index may be used to implement a distributed instant messaging system in a peer-to-peer network. Embodiments may be decentralized, self-organizing, and self-healing. Embodiments of the distributed instant messaging system may allow personal computers or other computing devices on home broadband connections to self-organize, store information about portions of the identity space, and route messages among participating peers in a peer-to-peer fashion. In one embodiment, the distributed index of instant messaging information may be a distributed hash table. Other embodiments may use other distributed indexing schemes. One embodiment of a distributed instant messaging system using a distributed index may be implemented in peer-to-peer networks implemented according to a peer-to-peer platform such as the exemplary peer-to-peer platform described below.

[0106] In one embodiment, the distributed instant messaging system may be implemented as an application or service on top of distributed indexing. The distributed instant messaging system has no central server that may fail and disrupt participation. In the distributed instant messaging system, the participating peers may distribute the load (e.g. peer information storage, authentication, and message routing) among the participating peers. If one or more of the participating peers fail or leave the session, other participating peer nodes may take over the departing peers' “zones” in the distributed index. In one embodiment, one or more zones of the distributed index may be redundantly stored on other peers to provide additional reliability against peer departures. One embodiment may implement identity services and presence detection, for example as illustrated in FIG. 36 and FIGS. 37 - 38 , respectively, through which peers may join an instant messaging session (including authentication) and detect the presence of other joining peers.

[0107] In one embodiment, message(s) may be routed to other participating peers using the distributed index. Instant messaging system messages may include one or more of, but are not limited to, text, binary, audio, video, images, network game information, software, and generally any data format that may be transported over a network.

[0108] FIG. 39 illustrates a plurality of nodes participating in instant messaging using a distributed index of instant messaging information according to one embodiment. Peers 200 A- 200 E are included in an instant messaging group 500 of peers and each store a zone 400 of the instant messaging distributed index. Other nodes 502 on the network may not be included in the instant messaging group. Each zone 400 may store instant messaging information for a subset of instant messaging group participants at locations in the zone indicated by hashes of identifiers of the instant messaging group participants. Each identifier may identify a participating peer in the instant messaging group. Identifiers may be, for example, names (e.g. user names, peer names, etc.), user identifiers, or peer identifiers such as the peer identifiers described for the exemplary peer-to-peer platform described below.

[0109] In one embodiment, the instant messaging distributed index may be used to route messages between peers 200 in the instant messaging group 500 . For example, peer 200 B may want to send a message (e.g. a “chat” message”) to a destination peer in the instant messaging group 500 (e.g. peer 200 D). The message may include an identifier for the destination peer. The identifier may be hashed, and the hash of the identifier may be used to access instant messaging information for the destination peer at a corresponding location in the distributed index. The instant messaging information for the destination peer may include routing information for the destination peer that may then be used to route the message to the destination peer. In one embodiment, the routing information may include contact information (e.g. a network address, an advertisement, etc.) for contacting and communicating with the destination peer. In one embodiment, routing may include the peer including the zone in which the instant messaging information for the destination peer using the routing information to send the message to the destination peer. In one embodiment, routing may include sending the routing information to the peer that originated the message, which may then use the routing information to send the message to the peer. Other embodiments may use other routing methods.

[0110] In one embodiment, the routing information may include an advertisement for the destination peer such as a peer advertisement and/or a pipe advertisement such as the advertisements described for the exemplary peer-to-peer platform described below. A pipe advertisement may be used to set up a pipe (i.e. a virtual communications channel) on the network between the source peer and the destination peer (e.g. peers 200 B and 200 D) and the two peers can exchange messages in a “chat” session via the pipe. In one embodiment, each peer in the instant messaging group 500 may include a routing mechanism such as the routing mechanism 404 illustrated in and described for FIG. 35 that may perform routing of messages between peers in the instant messaging group using the distributed index.

[0111] In one embodiment, the instant messaging distributed index may be used in adding new peers to the instant messaging group 500 . For example, node 502 A may desire to join as a peer 200 in the instant messaging group 500 , and may send a message requesting to join the instant messaging group 500 to one of the peers 200 in the group 500 . In one embodiment, the message may include an identifier used by the node 502 A. The identifier may be hashed, and the hash of the identifier may be used to access a corresponding location in the distributed index. If there is instant messaging information corresponding to the node's identifier in the distributed index, then the node 502 A may be asked to provide additional identification information (e.g. a password, a certificate, etc.) for use in authenticating the node 502 A's identifier. The node 502 A may respond by providing the requested additional identification information. The additional identification information may then be compared to identification information stored in the distributed index at the location indicated by the hash of the node 502 A's identifier to authenticate the node 502 A. If the additional identification information matches the identification information stored in the distributed index, node 502 A is authenticated, and the node 502 A may be added as a peer 200 to the instant messaging group. If the additional identification information does not match the identification information stored in the distributed index, then the node 502 is not authenticated. In one embodiment, node 502 A may be notified that it failed authentication.

[0112] If there is no instant messaging information corresponding to the node's identifier in the distributed index, then the node 502 A may be asked to provide additional identification information (e.g. a password, a certificate, etc.) for use in authenticating the node 502 A on future attempts to join the instant messaging group 500 using the identifier provided by node 502 A. The additional identification information may then be stored in the distributed index at a location indicated by a hash of the identifier provided by node 502 A.

[0113] When a peer 200 is added to instant messaging group 500 , a portion (zone) of the instant messaging distributed index may be stored on the added peer 200 . For example, if node 502 A contacts peer 200 A to join the instant messaging group 500 and successfully joins as a new peer in instant messaging group 500 as described above, then zone 400 A stored on peer 200 A may be divided and one part of zone 400 A may be stored on the new peer (i.e. node 502 A). The other part of zone 400 A may remain stored on peer 200 A. In one embodiment, parts of zones from two or more peers in an instant messaging group may be moved to and stored on a new peer, and may be combined to form a new zone. Thus, adding new peers to instant messaging group 500 distributes the workload of managing the instant messaging system using the distributed index.

[0114] When a peer leaves an instant messaging group, the zone stored on the departing peer may be moved to another peer in the instant messaging group, and may be combined with a zone already on the peer. For example, if peer 200 B desires to leave instant messaging group 500 , zone 400 B may be moved to peer 200 A and combined with zone 400 A. In some embodiments, a zone on a departing peer may be divided into parts and moved to two or more other peers still in the instant messaging group. In one embodiment, when a peer leaves the instant messaging group, instant messaging information corresponding to the peer and stored in the distributed index at a location indicated by a hash of an identifier used by the peer may be modified to indicate that the peer is no longer participating in the instant messaging group. In one embodiment, the corresponding instant messaging information may be deleted from the instant messaging distributed index.

[0115] When a peer joins an instant messaging group, one or more other peers in the instant messaging group may be notified that the peer has joined the instant messaging group. In one embodiment, peers in an instant messaging group may register to be informed if particular peers (identified by particular identifiers) join the instant messaging group. For example, prior to node 502 A requesting to join instant messaging group 500 , peer 200 C may have registered a notification message requesting be notified if a peer using the identifier provided by node 502 A. In one embodiment, the notification message may be stored in the distributed index at a location indicated by a hash of the identifier. If node 502 successfully joins the instant messaging group 500 using the identifier as described above, the stored notification request may be accessed and used to notify peer 200 C that a peer using the identifier has joined the instant messaging group. In one embodiment, peer 200 C may be notified with a message including information configured for use in contacting the new peer corresponding the identifier. In one embodiment, this information may include an advertisement, or alternatively information for accessing an advertisement, configured for use in contacting the new peer such as a peer advertisement or pipe advertisement as described for the exemplary peer-to-peer platform described below.

[0116] When a peer joins an instant messaging group, the peer may query for or otherwise be provided with information identifying one or more other peers currently in the instant messaging group and thus registered in the instant messaging distributed index. For example, if node 502 A successfully joins instant messaging group 500 as a new peer, one or more messages may be sent to the new peer including information identifying peers 200 A- 200 E as participants in the instant messaging group 500 . In one embodiment, this information may include contact information for contacting peers 200 participating in the instant messaging group 500 . In one embodiment, this information may include advertisements (or alternatively information for accessing advertisements) for peers 200 participating in instant messaging group 500 , such as peer or pipe advertisements as described for the exemplary peer-to-peer platform described below.

[0117] In one embodiment, joining peers (or any participating peer) may search or query the instant messaging distributed index of presence information to discover the presence of other participating peers. For example, a first peer may send a query message including an identifier for a second peer. The identifier may be hashed and used to determine if the second peer is registered in the distributed index of presence information. If the second peer is registered, information including contact information for the second peer may be returned to the first peer. This mechanism may be used, for example, b a peer joining the distributed index as a participant to discover other participating peers which the joining peer may desire to communicate with.

[0118] FIG. 40 is a flowchart illustrating using an instant messaging distributed index to route messages between peers participating in an instant messaging group according to one embodiment. As indicated at 470 , a distributed index of instant messaging information may be stored across a plurality of peers participating in an instant messaging group. As indicated at 472 , one of the plurality of peers may generate a message for another one of the peers. In one embodiment, the message may include an identifier of the destination peer. As indicated at 474 , the plurality of peers routes the message to the other peer using routing information for the destination peer in the instant messaging information stored in the distributed index at a location indicated by a hash of an identifier corresponding to the destination peer.

[0119] FIG. 41 is a flowchart illustrating reallocating zones when adding a peer to an instant messaging group according to one embodiment. A distributed index of instant messaging information may be stored across a plurality of peers participating in an instant messaging group. As indicated at 480 , a peer may be added to the plurality of peers storing the instant messaging distributed index. As indicated at 482 , a zone of the distributed index may be stored on the added peer. For example, if a peer successfully joins as a new peer in the instant messaging group as described above for FIG. 39 , then a zone of the instant messaging distributed index stored on a peer in the instant messaging group that the new peer contacted to join the group may be divided and one part of the zone may be stored on the new peer and the other part of the zone may remain stored on the original peer. In one embodiment, parts of zones from two or more peers in an instant messaging group may be moved to and stored on a new peer, and may be combined to form a new zone.

[0120] FIG. 42 is a flowchart illustrating reallocating zones when removing a peer from an instant messaging group according to one embodiment. A distributed index of instant messaging information may be stored across a plurality of peers participating in an instant messaging group. As indicated at 490 , a peer may leave the plurality of peers storing the instant messaging distributed index. As indicated at 492 , a zone of the distributed index stored on the departing peer may be combined with a zone of the distributed index stored on another peer in the instant messaging group. In one embodiment, the zone stored on the departing peer may be divided into two or more parts, and each of the parts may be combined with zones on two or more peers in the instant messaging group.

[0121] Peer-to-Peer Platform

[0122] The following is a description of an open network computing platform designed for peer-to-peer computing in which embodiments of a distributed index mechanism for indexing and searching for content in peer-to-peer networks may be implemented. It is noted that this description is included for exemplary purposes and that embodiments of a distributed index mechanism for indexing and searching for content in peer-to-peer networks as described herein may be used with other peer-to-peer platforms and protocols.

[0123] Embodiments of an open network computing platform designed for peer-to-peer computing are described. The network computing platform may be referred to as a peer-to-peer platform. The peer-to-peer platform may be used to build a wide range of distributed services and applications in which every device is addressable as a peer, and where peers can bridge from one domain into another. The peer-to-peer platform may enable developers to focus on their own application development while easily creating distributed computing software that is flexible, interoperable, and available on any peer on the expanded Web. The peer-to-peer platform may enable software developers to deploy interoperable services and content, further spring-boarding the peer-to-peer revolution on the Internet. The peer-to-peer platform addresses the problems of prior art peer-to-peer systems by providing a generic and service-agnostic peer-to-peer platform that may be preferably defined by a small number of protocols. Each protocol is preferably easy to implement and easy to be adopted into peer-to-peer services and applications. Thus, service offerings from one vendor may be used, perhaps transparently, by the user community of another vendor's system.

[0124] The peer-to-peer platform extends P2P computing to enable a wide range of distributed computing applications and overcome the limitations typically found in prior art P2P applications. The peer-to-peer platform is a network computing technology that provides a set of simple, small, and flexible mechanisms that can support P2P computing on any platform, anywhere, and at any time. The peer-to-peer platform generalizes P2P functionality and provides core technology that addresses the limitations of prior art P2P computing technologies.

[0125] The peer-to-peer platform is a modular platform that provides simple and essential building blocks for developing a wide range of distributed services and applications. The peer-to-peer platform specifies a set of protocols rather than an API. Thus, the peer-to-peer platform can be implemented in any language on any Operating System to provide solutions ranging from providing a simple protocol-based wrapper that enables a small device to join a network of peers to developing a fully integrated application that supports metering, monitoring, high-level security and communication across server-class systems.

[0126] In one embodiment, the peer-to-peer platform architecture may include, but is not limited to, protocols, advertisements, and core services. Network protocol bindings may be used to preferably ensure interoperability with existing content transfer protocols, network transports, routers, and firewalls. The peer-to-peer platform may be used to combine network nodes (peers) into a simple and coherent peer-to-peer network computing platform. Embodiments the peer-to-peer platform may be directed at providing several benefits including one or more of, but not limited to, no single point of failure, asynchronous messaging, the ability for peers to adapt to their network environment, and moving content towards its consumers.

[0127] FIG. 2 illustrates one embodiment of peer-to-peer platform software architecture at the conceptual level. The peer-to-peer platform may include several layers. In one embodiment, the software stack may be described using three layers; a peer-to-peer platform (core) layer 120 , a service layer 140 and an application layer 150 . In one embodiment, the peer-to-peer platform may include a core layer 120 that defines and encapsulates minimal primitives that are common to peer-to-peer networking, including, but not limited to, peers 110 , peer groups 122 , peer discovery 124 , peer communication (e.g. pipes) 126 , peer monitoring 128 , and associated security primitives 130 . This layer may be shared by all peer-to-peer devices so that interoperability becomes possible.

[0128] A peer may be defined as any entity that runs some or all of one or more protocols provided by the peer-to-peer platform core layer. As such, a peer may manifest in the form of a processor, a process or a device. A peer may be anything with a digital heartbeat that supports the peer-to-peer platform core, including sensors, servers, PCs, computers up to and including supercomputers, PDAs, manufacturing and medical equipment, phones and cellular phones. In order to interact with other peers (e.g. to form or join peer groups), the peer needs to be connected to some kind of network (wired or wireless), such as IP, Bluetooth, or Havi, among others.

[0129] The peer-to-peer platform may provide mechanisms through which peers may discover each other, communicate with each other, and cooperate with each other to form peer groups. Peers may discover each other on the network to form transient or persistent relationships called peer groups. A peer group is a collection of peers connected by a network that share a common set of interests and that have agreed upon a common set of rules to publish, share and access any computer content (code, data, applications, or other collections of computer representable resources), and communicate among themselves. Peer groups may also be statically predefined. The peers in a peer group may cooperate to provide a common set of services. A peer group may be viewed as an abstract region of the network, and may act as a virtual subnet. The concept of a region virtualizes the notion of routers and firewalls, subdividing the network in a self-organizing fashion without respect to actual physical network boundaries. In one embodiment, peer groups implicitly define a region scope that may limit peer propagation requests. Conceptually, a peer group may be viewed as a virtual entity that speaks the set of peer group protocols.

[0130] The core layer 120 provides core support for peer-to-peer services and applications. In a multi-platform, secure execution environment, the core mechanisms of peer groups, peer pipes and peer monitoring may be provided. Peer groups 122 may establish a set of peers and naming within a peer group with mechanisms to create policies for creation and deletion, membership, advertising and discovery of other peer groups and peer nodes, communication, security, and content sharing. Pipes provide virtual communication channels among peers. Messages sent in pipes may support transfer of data, content, and code in a protocol-independent manner, allowing a range of security, integrity, and privacy options. In one embodiment, messages may be structured with a markup language such as XML. Peer monitoring 128 enables control of the behavior and activity of peers in a peer group and can be used to implement peer management functions including access control, priority setting, traffic metering, and bandwidth balancing.

[0131] The core layer 120 may include protocols and building blocks to enable key mechanisms for peer to peer networking, including discovery, transport (including firewall handling and limited security), and the creation of peers and peer groups. In one embodiment, the core layer 120 may be thin and small, and may provide interesting and powerful primitives for use by services and applications in the other layers. The core layer 120 may support choices such as anonymous vs. registered users and encrypted vs. clear text content without imposing specific policies on developers. Policy choices may be made, or when necessary, implemented, at the service layer 140 and/or application layer 150 . For example, administration services such as accepting or rejecting a peer's membership in a peer group may be implemented using the functionality provided by the core layer 120 .

[0132] The core components of the peer-to-peer protocol may be used to implement discovery mechanisms for searching, publishing and recovering of core abstractions (e.g. peers, peer group, pipes, endpoints, and advertisements). In one embodiment, these mechanisms may be simple, administration free, and may not require special peers to act as “master” peers. These mechanisms may allow processes in the peer-to-peer network, in absence of help from other applications and/or services, to bootstrap and find out the information necessary to access applications and services that can help. In addition, the core may “return” to this standalone behavior and still function if helper applications or services fail. In one embodiment, safety mechanisms may be put in place in order to avoid a major overflow of “web-crawling.” In one embodiment, applications and/or services that support the peer-to-peer protocol may access, control, and/or override the core components, even to the extreme of implementing a centralized, client-server model based on the core components.

[0133] At the highest abstraction level, the peer-to-peer platform may be viewed as a set of protocols provided at the core layer 120 . In one embodiment, a common thread among peer-to-peer platform peers is protocols, not APIs or software implementations. In one embodiment, the peer-to-peer platform protocols may guarantee interoperability between compliant software components executing on potentially heterogeneous peer runtimes. Thus, embodiments of the peer-to-peer platform may be agnostic to programming language