The Search TargetPosted: January 5, 2007
Targeting search is for the most part left at the user’s prerogative, they enter a keyword or key phrase to target their search through the multitude of results collected into the large indexes that are behind Google and Yahoo.
This can be problematic for the search engines who have the responsibility to filter large amounts of spam and misleading content from their indexes to offer an adequate search experience to the user. For the most part this works very well and will return genuinely useful websites for most subjects.
There are many levels of search which we can break down into three tiers of search:
I’m going to go through this at a basic level here, rather than fiddling with minor discrepancies and crossovers between these categories.
The first level, unfiltered crawling, is when a crawler is set to collate websites by following every link it finds without bias. The type of index collated here will obviously be larger, contain more information and be more open to abuse than the other two.
Filtered crawling occurs when a crawler is automatically programmed to break link following to pages that do not conform with its specificied parameters. As a basic example a crawler could be set to only index pages that contain the keyword ‘phill’ in their source. Any pages that don’t contain this word are not added to the index and thus the crawler will not follow any successive links on these pages either. This allows for faster indexing and an improved search for the user – though only if they’re looking at your chosen target. It’s most useful for websites that deal in specialist subjects.
Targeted search often does not involve a crawler and is site specific; indexing only the pages on a particular website or in a content database. Targeted search usually needs no refinement when searching its index for results but can often be poorly implemented and lacking enough data to make true comparisons.
Search Giants such as Google are trying to bridge the gap between these three areas in order to improve the quality of their results – allowing users to create their ‘custom search engine’ and site specific searches allows Google to gather data and statistics directly at the source if you will. Using the data from filtered and targeted search environments means your data will be more refined, relevant and ultimately allow improved algorithms for unfiltered search as well.
Filtered personal searching is a growth area, allowing users to download their own search indexes based around topics of interest to them. Having their personal internet available offline is always going to be a bonus to travelling professionals even as wireless communication becomes yet more prolific.
Desktop search engines such as the above, can allow a user to target their own results and download web pages to cache to view and search at a later date. Shown below is the keyword filtering used to choose which pages are added to the index and which are discarded. The information can then be taken and interpreted into any format the user chooses – for example this index powers an online engine for advice on moving abroad. It could just as easily be used for statistical analysis, offline search or website caching.
By allowing the user multiple accounts it gives the power to create the variety they need and to experiment with what was previously a very complex system available only to the search heavyweights.
This is just one example of how search is changing at its most basic level and has just started over the last year or so to really think about how to meet the demands of the user. In the current climate a new Google could well emerge with a new breed of start-ups on its heels offering a wide variety of services with fresh and innovative ideas it needs to watch out – this year could be the end of the monster search index.
With the advent of affordable terabyte hard disk space and improved high speed broadband expected this year, search technology would be crazy not to become the ultimate desktop tool, we’re looking at a very important time for shaping search.
If you’re interested in some examples of emerging search technology then I suggest you have a meander over some of these websites: