Peer-to-Peer Search Engine

So far my concepts on decentralized search have been limited to the federated search paradigm. That is, search engines working fully through meta-search engines, thus allowing users to search with all of them under one interface.

I believe there’s another concept that holds promise for the decentralization of search: Peer-to-Peer (P2P) Search Engines. The idea is to have users combine their resources over a P2P network to form a distributed search engine. This gets around the cost of building and maintaining data-centers comparable to Google’s and Bing’s.

Because of this infrastructure they can easily provide search results free of advertising or censorship. And if they have a large enough userbase they could more effectively search the hidden web than traditional search engines.

Currently, P2P search engines such as YaCy, Faroo and Wowd face significant challenges to mainstream adoption: For starters, web browsers don’t yet have a framework for building P2P applications on. Which is why most of them have to be installed directly on the computer which for alot of people, might be confusing. Also, they need to somehow be better than the two incumbents Google and Bing, which are both easy to use and free.

Any application that can get past those obstacles will prove to be at least as game changing as Firefox was, maybe more so.

Search Panel Concept

Right now, using the search bar in Firefox feels like using two separate interfaces. The search bar has it’s controls and the search page you’re sent to has it’s own controls. The search page invariably has the most complete and better integrated controls. At the same time the browser’s search bar is more convenient for switching between search engines. What would really be great is for the two to work together to make one cohesive interface.

Search Panel Interface:

The idea is that the main controls of the search engine (the search box, page navigation, and link selection etc.) are left up to the search panel. While elements like the organic results and sponsored links are left to the search page to handle.

Note: I have more of these mockups on flickr. If want to look at them closer I recommend you view them using their slideshow feature:

Features:

Tab Through Search Results:
There’s a feature in RockMelt where it shows truncated results in a side panel while they are pre-fetched/rendered, which allows you to tab through them quickly. It’s basically the same idea here except you also have access to the regular results layout through the “expand” button.

Toggle Sidebar Options:
In addition to toggling between “Tab Mode” and “Wide mode”, you can toggle the Sidebar shown on the lefthand of the search results (right now, all the major search engines have one). It defaults to off since it’s not needed most of the time.

Switch between Search Engines using Keywords:
This is kind of like Chrome’s “Press [Tab] to Search…” feature only it’s not sensitive to where you type the keyword. You can type it before, after or in the middle of your search query and it’ll offer (via the autocomplete/suggestions dropdown) to search the query with that engine.

Update: New Set of Mockups on Flickr

I’ve uploaded the images of my search client redesign on Flickr. I will try to better explain them in post sometime soon; for now there’s the image descriptions.

Here’s a link:
http://www.flickr.com/photos/55481031@N04/sets/72157625302472285

Rethinking my Search Client Concept

Since I saw RockMelt’s Search Overlay I’ve been doing some rethinking on my search concept. I noticed their features were made to help users search faster. While my features were mostly about decentralizing search by supporting the OpenSearch specifications. So they were user focused while I was platform focused. An important rule about platforms is they’re only as valuable as the number of users it has.

Plus, in light of the shrinking availability of wholesale search providers, it dawned on me that restricting search providers to displaying results in the same feed-based format would eliminate the only other differentiators they have besides their brand. Thus, it wouldn’t really help decentralize search. Before this news, it had occurred to me that differentiation might be a problem, but it wasn’t this serious of a concern.

I still think there’s hope for decentralized search. If there were a search application that could be smart enough to know which engine the user wants to use for a given search, then people could rely less on general purpose engines. This would be effective for when the user wants to use a website’s search repository (eg. Facebook, Twitter, Flickr, Youtube etc.) or a engine that specialized in certain content (eg. News, TV Shows, Gadgets etc.). Though, it won’t be too useful until it can smartly guess which search engine has the content the user wants, when she has no clue.

Anyways, I’m still convinced that a more client based search application is the way to go and that RockMelt’s website integration and mobile apps for things like Twitter, Facebook and Youtube are good examples how to do it.

Does Rockmelt have dropdown search results?

In my original post on search client I talked about a feature where the search box, instead of taking you to Google, would instead show you a dropdown list of results. Well, it seems that the developers of RockMelt have thought of something very similar:

When you do a search from the search box, instead of taking you to Google, you get a column with the first ten results. You can tab through each result, which is pre-loaded into the browser, so you can actually see the Web pages behind each result in the full browser. This is designed to speed up searching, although at least initially I find myself tabbing through each link, if only for a second or two.

– Techcrunch

I doubt their implementation works exactly like my mockup, but I do hope they support the OpenSearch specifications for it.

 

Update: Yes, they do have dropdown search results! Here’s a screenshot from the RockMelt video:

 

Filter Search and Customize

Some things I haven’t really gone into are advanced search features and customization.

Gear (Tools?) Menu:

Filter Search Dialog:

First lets look at the “Filter Search” dialog. This should look kind of familiar for those who’ve used the “Advanced Search” link on Google, Yahoo etc.

When you’re done just click “Search” and it’ll update the results with your filter.

Customize Dialog:

The “Customize…” dialog, is for customizing the search page. This is where you can choose which search engines are in the search panel. This is also where you choose how you want your results shown, plus how and when to load them. “Instant Search”- searches while you type. “Infinite Scroll”- auto loads the next results as you scroll down.

These next two images show what you might come up with if you go for mostly small search providers.

Now your default is “Custom Search Page”.

If you want to go back to the other page settings just select “Google” from the dropdown menu.

Search Client Concept

Search has become a very important part of how we use the web. It’s often thought of as the gateway to the Internet. Today, it’s so important to the way people use the Internet that some even confuse it with the term “web browser”. With that said, it’s just as important for search to be decentralized as it is for the Internet.

About five years ago someone working at A9 created the OpenSearch specification. The standard is similar in concept to OpenID only instead of decentralizing Authentication it decentralizes Search. Despite it’s early reception the specification has been used very little by search providers and browser makers. Notably, OpenSearch Response Elements, which are the formats search clients would receive search results in, have been mostly ignored.

Now, I’m going to show you my concept of what Firefox might look like as a full pledged search client supporting significant parts of the OpenSearch specifications, especially Response Elements. The following are images showing some use cases around the concept (click them for a larger view):

Start Page:

Here’s the start page. It’s not really a page as much as it’s a application screen.

You might search for “carmen sandiego”.

Results Page:

When you do a search it opens the results in a new tab. This is how results (feeds) are displayed:

The client could customize the layout of the results. Some elements (like ads, images etc.) could be auto-grouped together and  color-coded. The positions and fonts of the Title, URL and Snippet could be changed. Embedded media (Images, Videos, News etc.) could be Enabled/Disabled. Ultimately it would be in the users hands.

Dropdown Results List:

Suppose you clicked one of those links and opened this page:

Notice the little blue icon in the Address Bar. It’s a lot like the RSS feed icon. It lets you know when a page you’re on uses OpenSearch Response Elements. Also notice that “carmen sandiego” has carried over to our search box instead of being held on the search providers API.

When you click on the search box a dropdown list shows you abridged results. You can type in other searches and it reloads as you type. If you want to return to the full page results just press [Enter] or the click Search button.

This is useful for when you want to stay on a page while doing searches. You could open one of the links in a new tab and keep watching a video or reading a blog or whatever.

Appending Results:

Back to the results page. This time showing results for “gumshoe”.

Now, lets scroll to the bottom of the page:

Just click the “More Results” bar to load the next set of results. It will append them under the previous set of results.

This is another thing the client could customize. Some might prefer the traditional ‘next’ page ‘previous’ page layout. Others might want it to auto load as you scroll down.

Temporarily Switching Search Providers:

What if you just want to see what Yahoo’s (or Bing Ask’s etc.) results were without committing to it as the default search? You can do this by clicking on it’s “Also try:” tab:

It won’t change your default search and it will go back to the default after you leave the page.

So there you have it. Most of the designs were inspired by official Firefox 4 concepts (most notably the Add-on’s Manager) and I have some more that I’m thinking about posting when they’re ready.

I’d like to hear what people think of this idea, especially from those involved with Mozilla. Leave a comment if there’s any questions, thoughts or you just want to show your interested.