Searching Web the Human Way

by Sunny Saurabh, co-founder, Antya.com

Search in India has its own share of nuances. 'Maruti' is spelt as 'Maruthi' in most of South India; people wanting to go to Naukri.com will most of the times type 'Naukri' as 'Naukari'; a loan request would be simply followed in the query as 'need money'. An online market research company once told me that a couple registered their daughter's profile on BharatMatrimony.com and kept saying that they had registered it on Shaadi! These and many more such intricacies are typical of Indians, and require a special handling to understand the intent of the user.

Search for 'bus tickets', 'Bollywood movies' or even 'temples Delhi', in any of your favourite search engines. Chances are you may get millions of results but not exactly what you were looking for. This is because the result that you want may remain hidden due to complex search algorithms that are prone to spam, and gaming by web masters. Now if you have a search engine that clears the spam and fishes out the results for you, and ranks them according to the popularity of a strong and trusted community, wouldn't that be great? Welcome to the world of human-powered search.

What is human-powered search?

Simply put, a human-powered search engine is a search engine that takes into account human intelligence and leverages the power of the community to provide the most relevant information to the end user. The term 'human-powered' has been made popular by Jason Calacanis who launched Mahalo.com in May 2007. Other variants of human-powered searches include Chacha.com (where you could chat with a human being in case you don't get the results) and Wikia that has a mini-article followed by an algorithmic search. In India, we have Antya.com that combines the element of search & discovery to provide users with a simple, family-safe, and reliable way to find information on the web.

The Need

India today attracts over a billion searches per month. As more and more people come online every year, the need for relevant information increases. For any new user to search, there is always a need of a simplified search engine that could help locate information with ease. Even an advanced user of the net who has no time to investigate, could do well with a search engine that provides quick, reliable, information on the internet.

If I take myself, my ecosystem of search queries wouldn't usually be above 500 in a year. If I want more from the web, I need to discover them, while I search on my keywords. For example, one of the top queries in India on Google is 'Indian Railways'. A human-powered search engine gives me the flexibility of stumbling upon an Erail.in in addition to the Indian Railways' website, as somebody has already done the research for me.

Challenges

As good as it may sound, human-powered search is not foolproof. There is the 'bias' or 'subjectivity' that a human being brings in. Mahalo.com has 'guides' to validate each result. However, even the guides could be subject to bias. For example, if I search for 'mango', I might be referring to the fruit or the fashion label. Who will make that choice for me? Introduction of a strong community angle on the lines of Wikipedia could help disambiguate such results. In addition, a popularity angle in the form of rating or reviews could insure that 'mango' as a fruit is more likely to have been searched in India than as a fashion label.

One of the biggest questions raised with a human-powered search engine is, whether it could match the scale of the machines. If you look at Mahalo.com, it has about 12,000 human editors handcrafting the top 50,000 search terms, which again may not be enough. Let me answer this in a different way.

Last September, I was in Seoul, South Korea, understanding the dominance of the search portal Naver.com, which leaves even the mighty Google with less than three per cent market share. The search service banks on human knowledge to give you one of the richest sources of information on a search term. A Naver search for a subway station, for instance, will return a map, information on the subway line serving the station, connecting bus lines, restaurants and shops near the station, blog entries, and more. Any new query that came into their charts as gaining a lot of popularity will get constructed in no time. But when it came to long tail queries like personal names, Google.co.kr did a much better job.

One of those evenings, I was having dinner with a friend from Naver.com. It was raining the whole day, but the site had predicted it to be a cloudy day. By 11:00 am, she told me, they had received over a hundred phone calls, forcing them to change the weather forecast to 'Rainy'. If, as a search engine, you could attract such a loyal and active user base, you need not worry about scale.