{{{ #!forumlinks }}} = !BuddyCast3 = This version of !BuddyCast adds ratings, comments, faster .torrent collecting, access to more content due to RSS subscription integration, and superior '''scalability'''. They key problem that we will solve with !BuddyCast3 is the slow speed of peer/content discovery and limited scalability. The slow collection of .torrent files means new content takes hours to discover. The scalability problem is due to the limit of 50 hashes in a [wiki:BuddyCast2] profile. This version will remove this limit. The initial version of !BuddyCast was made in a very conservative manner. [wiki:BuddyCast2] improved the original by only exchanging ''correct'' information of online peers. With version 3 we build further upon this operational system. == new Remote Search message == A new Bittorrent message is created to increase the number of search results. When the user issues a keyword search Tribler looks into the local megacache and issues a ''remote search'' message. Peers respond to this remote search message with the matches they find in their own megacache. The result is that as of Buddycast3 peers search 1 + 10 megacaches for matches for given keywords. This greatly enhances recall and precision. == New Buddycast message format == The new Buddycast message contains information about torrents, taste buddies, and random peers. * Torrent info - preferences: Recently downloaded torrents by the user [[BR]] - collected torrents: Recently collected torrents (include Subscribed torrents) [[BR]] - ratings: Recently rated torrents and their ratings (negative rating means this torrent was deleted) [[BR]] * Taste Buddies - permid - ip - port - similarity * Random Peers - permid - ip - port - similarity == Very far Future == For Buddycast8 or something we hopefully can evolve towards a simplified version of a distributed database with a generic sync protocol. The architecture blends gossip protocols with operational algorithms such as the [http://fsck.com/~jesse/talks/2008/yapc.asia.08-prophet.pdf Prophet DB sync project] and add a security and resource usage model. With a trust-aware gossip kernel we can exchange databases updates with peers we meet. This will lead to eventual consistency and spreading. To avoid complexity we carefully avoid conflict resolution by not allowing the ''update'' primitive. {{{ #!protected #:[[Include(wiki:ProtectedSectionMessage)]] === Rated or recently downloaded torrent info === The torrent information included inside a !BuddyCast3 message consists of different types. First is the rated or recent downloads. This is a list of maximum 50 SHA1 hashes of the most recently rated files by the user or most recently (unrated) downloads by the user. This list of 50 is thus the profile of the user. The encoding is simply two lists of hashes. The first lists contains the hash and a rating value of 0 to 5. This directly maps to the 5 star-based rating representation in the GUI. The second list contains the most recent downloads of the user without a rating. When the user has not rated any files the first list is absent and the second list contains at most 50 of the recent downloads made by the user. When the user has rated between 1 and 49 torrents, those rated torrents take priority and are augmented with unrated recent downloads up to a total of 50. When the user has rated 50 or more torrents, only the most recent rating are included in the message and none of the unrated torrents. === Subscribed torrents using RSS === A new option in [milestone:3.7.0 the upcoming release] is support for [wiki:TriblerRSSFeed RSS subscriptions]. RSS Feeds such as [http://www.legaltorrents.com/rss.xml Legaltorrents.com] contain the descriptions of content and direct links to .torrent files. The old !BuddyCast algorithm only transmitted information to others when a .torrent was manually downloaded by one Tribler user. When people subscribe to RSS feeds they find interesting they also collect torrents automatically. The new ''subscribed torrents'' field in buddycast contain the hashes of the 25 most recently downloaded .torrent files of various feeds. === Recently collected === To improve the torrent collecting process the 25 most recently collected .torrent files are included in the new Buddycast message. The receiver can thus see for which hashes it could request the .torrent file. These recently 25 hashes should have no duplicates with the previous listed 75 hashes. === Tracker statistics === The information on the maximum 50+25+25 hashes is augmented with three fields to lighten the load on the trackers. The amount of seeders, leechers, and the amount of seconds ago this was checked is also included in every !BuddyCast3 message. This means trackers do not need to be contacted for statistics every time a new .torrent is collected. === Taste Buddies === To improve security, reduce complexity, and lower the bandwith usage the profiles of other peers are no longer included. Semantic clustering is still maintained, but detailed profiles can only be obtained from peers themselves. Upon receiving a new !BuddyCast message the similarity of the included taste buddies is still unknown. However, this similarity of new taste buddies is required for quick convergence of Buddycast. The similarity of the Buddycast message sender is used as a temporairy similarity value for the included taste buddies. === tit-for-tat .torrent collecting === The key change to the algorithm of Buddycast is the tit-for-tat for metadata. Due to the [wiki:Give-To-Get] algorithm for Video on Demand we will need to maintain full backwards compatibility with .torrent files for a long time. The usage of Merkle hashes and '''Tribler-only''' swarms is not likely to emerge soon and will be limited to specific content of Public Broadcasters. To deal with the bandwidth requirements of .torrent files !Buddycast3 will give 1 .torrent file to a peer every 4 hours (BuddyCast round). When a peer has given you a single .torrent file, you will allow this peer to download '''two''' .torrent files from you. By comparing the new subscribed and recently collected hashes it is possible that you are interested in more .torrents of a peer. With a single peer you can thus exchange dozens of .torrent files at high speed. The !Buddycast algorithm selects either the most similar peer to connect to or a random peer. After connecting and exchanging a !BuddyCast message with a peer you will start exchanging multiple .torrent files. From the uncollected .torrent files in the received !BuddyCast message you will randomly select one and request it's download. When this peer also requests a .torrent file from you, this will trigger a second .torrent request, etc.. The recently downloaded .torrent field will be updated whenever new .torrent files have arrived. You will only request the next .torrent download when the previous .torrent file finished downloading. This algorithm ensures that when you meet a peer with a non-overlapping recently collected .torrent list; you will automatically exchange all .torrents. This exchange will occur at full upload speed of both Tribler peers. The current BuddyCast2 network contains a mere 2400 .torrent files. This algorithm with RSS feeds should expand this by at least two orders of magnitude. For rate control control you will : - Not download more then 50 .torrent files per peer per 4 hours - Not download more then 8 .torrent files in parallel from various peers === Fast bootstrapping === The fixed 15 seconds BuddyCast interval will be made dynamic and vary between 1 and 60 seconds. Buddycast will run into 4 different modes : - 1 second rounds; Bootstrap mode [[BR]] Only used '''once''', the 2 minutes directly after the software is installed.[[BR]] - 5 second rounds; Accelerated mode [[BR]] Used 30 minutes directly after the software is restarted. (Also following Bootstrap mode) - 15 second rounds; Normal mode [[BR]] Default operation - 60 second rounds; Slow mode [[BR]] Entered when Tribler has been running for more than 24 hours === Comments === (Moved to a later version; Not yet ready for implementation; requires better friends and social network support) A new feature is the ability to add textual comments to a torrent. This addition enables a first primitive version of a message board to Tribler. An outgoing BuddyCast message only contains the comments entered by the user directly. Comments are thus not forwarded by others. A comment has a maximum of 500 characters. === Priorities === - 1 Fast Bootstrapping (1, 5, 15, 60 second rounds) + user override in GUI - 2 Tribler disk usage (collect 1000, 5000''default'', 25000, 100000 torrents) - 3 Sparseness problem in recommender - 4 Reduce memory usage - 5 new message layout (no taste buddy profiles) + tit-for-tat of .torrent downloading - 6 tracker statistics inclusion - 7 RSS - 8 Comments === Splitting BuddyCast === (For future) Isn't BuddyCast becoming too big? Recent torrents, moderations, comments, ip addresses, taste buddies... Especially considering TorrentSmell (or any other scheme) providing infohash:swarm mapping. In case decentralized tracking works, passing of torrent files around is unnecessary; might use the [http://www.bittorrent.org/beps/bep_0009.html metadata extension] instead. So, is it possible to split BuddyCast into blocks having explicitly defined functionality and dependencies? There is a proposal by Johan (as far as I understand it) to maintain 50 idle BitTorrent connections to exchange BuddyCast announces/requests; the proposal also targets decentralized tracking as ip:port records are also included into the exchange. So, too different kinds of activities with too different requirements got packed into a single envelope. }}}