Sunday, October 23, 2005
The Art of Effective Web Searching
The ubiquitous Web search form -- the "search command line" -- has had a profound and transformative effect on information retrieval. That simple little box at the top of every search engine has opened up a boundless new world for millions of people. In doing so, all of the major search engines have made compromises to cast their nets wide and broaden their overall appeal. While this has done wonders to build a general-purpose tool for the majority, there are specific things that we can do to unlock the secrets buried beneath the search command line. The major search engines tend to return particularly accurate and relevant results for two very different types of queries. On one hand, a search for terms that are common but rather specific will usually return an accurate list of popular Web sites that contain canonical information about the term(s). At the other extreme, a search for a relatively rare and statistically improbable set of search terms also tends to return accurate results. For example, a search for "ISBN 0679723420" points you straight to Vladimir Nabokov's classic Pale Fire, without much effort.
Therefore, the art of effective searching usually falls somewhere in the middle of those two kinds of queries. This is especially applicable in the academic and technical communities, where users frequently seek out esoteric and difficult-to-articulate data. While the search engines are getting better at helping the user with these more demanding queries, here are six techniques the power user can adopt to find the best results fast: Search from the outside in. Sometimes search queries simply return too much information, especially when the search engine doesn't know enough to disambiguate between different types of results. The first approach most people try is to add more terms to the query until the right results are found. But this can just as easily eliminate good results as reveal them. An alternative is to start with broad search terms and use the "minus" operator in the search query to selectively eliminate sets of results. This lets the power user quickly scan through the list of results and subtract entire categories of pages with quick modifications to the query, ending up with only the desired matches.
Comparison search. The Web is so large, no single search engine can crawl more than a fraction of it. Also, each search engine's relevancy algorithms (i.e., the logic that determines the order of the search results) differ substantially. The solution is to keep bookmarks handy for the big engines such as Google, Yahoo, and MSN, and be prepared to check all of them on particularly challenging searches. Server-side bookmarks. Everyone uses bookmarks, but one power search technique is to use a server-side bookmark manager rather than the one stored on your computer. When the user finds great pages, the bookmarks can be saved on a dedicated bookmark server such as del.icio.us, or LookSmart's Furl, or one of the advanced search engines such as A9 or Yahoo, via their toolbars. Many of these new services also support the capability to tag individual bookmarks with keywords or add extended diary entries that annotate your personal Web as you search.
Tabbed browsing. Web browsers such as Firefox and Safari support tabbed browsing, which hides new pages in the background of the same window until you need them. This allows the power user to perform a search and preload all of the interesting results in tabs while they work down the list. So, all of the selected pages will be loaded and ready to be read by the time the user is done scanning the list of search results.
Search histories. Another technique is to employ a search engine to track the user's search history. A permanent, searchable history is invaluable when performing serious research on the Web. In fact, search histories are often better than bookmarks in such cases, because a bookmark only helps the user remember one site at a time, while a search history helps the user remember how and where they found the page, and helps them retrace those steps in the future. Vertical search. No one general search engine will ever be able to fully capture the richness and complexity of the Web. Medical databases, travel information, and library catalogs are a few examples of vertical search engines that contain highly specialized data in narrow domains. Search syndication protocols such as Amazon.com's and Open-Search are helping specialized vertical search engines surface their results to other sites to help maximize their accessibility.
The search industry is evolving rapidly, and more tools for information retrieval and information sharing are being developed daily. Search engines are also learning how to personalize results based on the user's individual habits. They are incorporating more and more of the deep vertical data into their primary search interface, and are learning how to simultaneously serve the novice user, while adding tools and capabilities to empower the most sophisticated users. In the meantime, power users can stay on top of their searches with advanced search techniques such as those above.
Brought to you by the Guardian eCommerce Privacy Seal Program.
Therefore, the art of effective searching usually falls somewhere in the middle of those two kinds of queries. This is especially applicable in the academic and technical communities, where users frequently seek out esoteric and difficult-to-articulate data. While the search engines are getting better at helping the user with these more demanding queries, here are six techniques the power user can adopt to find the best results fast: Search from the outside in. Sometimes search queries simply return too much information, especially when the search engine doesn't know enough to disambiguate between different types of results. The first approach most people try is to add more terms to the query until the right results are found. But this can just as easily eliminate good results as reveal them. An alternative is to start with broad search terms and use the "minus" operator in the search query to selectively eliminate sets of results. This lets the power user quickly scan through the list of results and subtract entire categories of pages with quick modifications to the query, ending up with only the desired matches.
Comparison search. The Web is so large, no single search engine can crawl more than a fraction of it. Also, each search engine's relevancy algorithms (i.e., the logic that determines the order of the search results) differ substantially. The solution is to keep bookmarks handy for the big engines such as Google, Yahoo, and MSN, and be prepared to check all of them on particularly challenging searches. Server-side bookmarks. Everyone uses bookmarks, but one power search technique is to use a server-side bookmark manager rather than the one stored on your computer. When the user finds great pages, the bookmarks can be saved on a dedicated bookmark server such as del.icio.us, or LookSmart's Furl, or one of the advanced search engines such as A9 or Yahoo, via their toolbars. Many of these new services also support the capability to tag individual bookmarks with keywords or add extended diary entries that annotate your personal Web as you search.
Tabbed browsing. Web browsers such as Firefox and Safari support tabbed browsing, which hides new pages in the background of the same window until you need them. This allows the power user to perform a search and preload all of the interesting results in tabs while they work down the list. So, all of the selected pages will be loaded and ready to be read by the time the user is done scanning the list of search results.
Search histories. Another technique is to employ a search engine to track the user's search history. A permanent, searchable history is invaluable when performing serious research on the Web. In fact, search histories are often better than bookmarks in such cases, because a bookmark only helps the user remember one site at a time, while a search history helps the user remember how and where they found the page, and helps them retrace those steps in the future. Vertical search. No one general search engine will ever be able to fully capture the richness and complexity of the Web. Medical databases, travel information, and library catalogs are a few examples of vertical search engines that contain highly specialized data in narrow domains. Search syndication protocols such as Amazon.com's and Open-Search are helping specialized vertical search engines surface their results to other sites to help maximize their accessibility.
The search industry is evolving rapidly, and more tools for information retrieval and information sharing are being developed daily. Search engines are also learning how to personalize results based on the user's individual habits. They are incorporating more and more of the deep vertical data into their primary search interface, and are learning how to simultaneously serve the novice user, while adding tools and capabilities to empower the most sophisticated users. In the meantime, power users can stay on top of their searches with advanced search techniques such as those above.
Brought to you by the Guardian eCommerce Privacy Seal Program.
