Redefining Search as Surrch

I’ve been working hard on moving the search engine to desktop this past week, and can finally say that the crawler code is working and indexing as quickly as that with a full MySQL database and dedicated machine.

I’d love to see some benchmarks run on it to see exactly how speedy the index is but no time just yet.

Recently I’ve been through some of the applications we’re planning to make available over the next few months, and the direction we’re going to take on such crucial matters as Open Source and outside developer interaction. I can fill you in now on some of the planned uses for the semantic layer (which is likely to be released as an API, possibly open sourced):

• Self aware pages
  • Tagging pages with keywords has always been used on the internet to let search engines know what kind content the page contains.
  • Using the Surrch API we can generate the necessary keywords on the fly as the page loads. This cuts out a large amount of work for SEO.
  • A Surrch enabled engine wouldn’t even need to look at these keywords, it could generate them itself.
  • Not only a page can be self aware these days, people tag everything – including links. The Surrch API could conceivably be used to tag every single word on a page, creating a page that covers every single keyword possibility. This is overkill – but a demonstration of the power available.
• Targeted advertising by referral URL
  • Only a few engines currently have the power to effectively target advertising, this is one of the methods they use to control the market.
  • When a visitor visits a site, 80% of the time, it’s possible to tell the page they came from.
  • Using the Surrch API we can analyse the referring page, or the whole site under that domain currently in our index to estimate the user’s intention and target advertising effectively. This is completely independent of any other search engine.
• Personal search engine
  • The desktop version of Surrch can blend with our spider technology to build your own searchable website index.
  • The Surrch API will allow you to enter one word topics for your interests, and then extrapolate all the related keywords, enabling you to leave it to run and collect web pages of interest.
• Narrow Search
  • When you begin a search, you enter just one or two keywords in the topic you’re interested in.
  • Related keywords appear, which you can then select from to target your search and remove any doubts about dual meanings of a word for example.
  • This step repeats every time you search, also possible is opinionated search.
• Opinionated Search
  • As I’ve mentioned before, Surrch’s opinion will change according to the websites in it’s index.
  • At the base level, you can select from the average opinion of related keywords and subjects from its entire index.
  • You can find the opinion at other levels as well though, and this is where the power comes in in terms of really targeting what the user is looking for quickly and efficiently. All the following mean that this is the first true example of social search:
    • Find the opinion over a range of dates, good for current events, modern history, changes in trends.
    • Find the opinion over areas of geography, or by domain extension (.co.uk, .com).
    • Find the opinion over a certain group of websites, or just one website in particular – compare that with another site.
    • Find the opinion not only over the above things but also subjects, topics, social and religious groups.
    • At the most ridiculous example level, you could even find what topics 18 year olds on myspace living in leeds most talk about – but that I could probably guess. The point is that this is targeting demographics on a really unprecedented level.
  • Add the sites or web pages to your personal profile that you think most closely reflect your opinions, this data can then be taken into account in all future searches returning greater personal relevancy.

Outside parties have already expressed an interest for other applications of the Surrch API. Some are pretty clever and I wish I could post them without fear of a fish in the mouth. For now you’re going to have to wait – but this is getting to be some very exciting work.