Content Search

Status: operational and deployed for years

The prime functionalities of Tribler are easy finding and fast download. Finding content should be as-easy-as-google.

This research is strongly linked with: decentralised content discovery, DecentralizedRecommendation, and AdversarialContentSearch

Algorithms

Keyword search is the main improvement of V4.1. We had many complaints that the core P2P features of search & download where poor in Tribler V4.0. ContentSearch improvements :

• Return many results Done: Arno implemented remote search
• Works even after Tribler is just installed TODO
• Not consume much memory TODO
• Fast response time ?
• Returns no irrelevant results TODO
• Partial matches to keywords are not shown TODO
• Always use the AND operator when two keywords are provided
• Provides clear user feedback, like TODO
  • Searched for silver in 40,000 files
  • Matches 1 - 20 of 89 for silver
  • No matches for your query Silver AND man.

Version V4.0 of Tribler keeps all .torrent information in main memory. A Google-like keyword search window will replace the browse all feature as the default opening screen

For V4.1 we no longer keep all .torrent information in main memory. Like many other parts of Tribler this info needs to be loaded on-demand. Partial word matches are no longer shown and results with 1 matching keyword of a 2 keyword search are also no longer shown (aka default AND). A search of look 2007 does not return The.Outlook.2007.HDTV.

TIME	ACTION
startup	list of all keyword of local stored .torrent files is loaded(+popularity); the filenames in a .torrent are ignored for this version
keyword search local	matching of keywords in main memory
keyword search remote	send out search to peers with at least 1000 collected torrents
Youtube search remote	send out search to main video webservers
result ordering	results are simply ordered by their popularity (number of seeds); first all matching torrent names, then the youtube/liveleak matches ordered by number of views
results display	matching swarms are shown directly after local keyword search is completed
results highlighting	keywords matches on the torrent name are highlighted in bold, similar to Google
.torrent info loading	.torrent details such as tracker URL are search in MegaCache after display of results
progressive updates	incoming results from the network search (Tribler peers + video sites) are inserted in real-time into the results, respecting the result ordering
limited window flickering	due to the progressive updating, the results screen is constantly re-written. After the remote keyword search is done, the first page of the results screen is sorted and no longer updated. The slower Youtube results are appended.

Attached is a file with 44.000 .torrent names. This serves as a dataset for keyword matching. Note that even the most basic in information retrieval algorithms are not yet included: Keyword frequency, BM25 in Tribler. This is work for the Tribler V4.2 release.

On the V4.1 roadmap : Enable remote queries of your MegaCache

• Keyword search returns numerous results directly after install
• Bootstrap process does not need to be very agressive
• Remote keyword searches are not the main mechnism, only to help non-bootstrapped peers
• Thus restrict to 10 keyword searches per random peer over its entire lifetime
• Your taste buddies+friends have no restrictions on keyword searches
• return maximum 20 alive+unchecked torrent names+details, not bulky hashes
• Allow download of 10 .torrents for given keyword per peer over its lifetime
• Present a counter to the user of how many .torrent files he can query
• Show number of discovered peers and how many .torrent files they have on average

(partly?) Implemented by Arno...

For Tribler verion 4.2 :

Keyword searches are more powerfull

• Buddycast is extended with a field with the last 10 keyword searches
• Upon startup the opening screen of Tribler shows hot keywords of other peers
• Clicking a hot keyword initiates a keyword search query
• Combine the Youtube.com visual gridview with keyword browsing
• Show for each hot keyword a few thumbnails

User Interface

• Main opening screen
• Search for content on Bittorrent/Youtube/Liveleak/etc.
  • Video
  • Audio
  • XXX
  • other
• Search for people on Tribler/MSN/Myspace/Facebook/etc.

The Tribler draft version of this :

long-term Research issues

Mapping of the user desires with the available content. Craft a iterative cycle of re-search using results feedback. Feedback on keyword search convergence. Real-time search keyword suggestion. pairs of keywords & click-log. Implicit tagging. Adversarial content search.

Usage of click-log approach in P2P search with privacy preservation and security

Arjen: click-log approach with maximum privacy. Spam and adversarial behavior is the key challenge of this research. Understanding the trade-off between performance, relevance, scalability, robustness, etc. Exploit the usage of social networks and friends for security & privacy. Use of aggregation to combat privacy leakage. Johan requirements: running experimental code and zero-servers.