Previous Page | Deep Web Search Engines | Clustering Search Engines Search Inside Books | Find Experts and Expert Opinions

The deep web (also called the invisible web) is the collection of databases, reference sites, statistics, library catalogs, bibliographies, laws and other information that is not indexed by the major search engines. It is a gold mine of very specialized information. You probably already use the deep web, even though you may not be aware of it.

Search engines find actual web pages. However, deep web information is not typically stored on a specific web page. The page is literally created at the moment you do the search, and disappears when you leave. For example, when you look up flight information, the page is made specifically for you by culling data from an airline's database.

Many deep web resources are password-protected. General search engines ignore such resources. In order to have access to them, you must be aware that a given database exists, and you must also know the password. The list of employees at one's workplace is an example of this. You won't likely find coworker John Doe's phone extension via a Google search, but you will find it in the company's private listing of employees. There are three approaches to searching the deep web:

1. General search engines: General search engines, such as Google and Yahoo!, are getting better at finding popular deep web resources. Google Options and Yahoo! Shortcuts connect you to catalogs, dictionaries, movie times, stock quotes, package trackers, and more.

   For less popular database searches, you can try adding the word, "database" to your search terms. For example using the search words: stalking law database yields more hits pointing to government and academic institutions, than does simply running a search on stalking law.
   

2. Subject Directories: The Yahoo directory abd the Open Directory Project are arranged hierarchically by subject.

3. Deep web directories and search engines that are designed specifically to find deep web sources. They won't directly find the answer you're looking for, but they can point you toward some databases that other search engines miss. In many ways they are similar to the subject directories described above. Here are some deep web search tools:

• Complete Planet
• Direct Search
• Invisible Web Directory
• Librarian's Index to the Internet

Here are some deep web resources for psychologists and other mental health professionals:

• The UK-based Social Sciences Internet Gateway is a human-organized list of topics and links to hundreds of psychological resources. There is also a social sciences search engine, as well as the "Grapevine" where you can find training programs, conferences, jobs and colleagues who have similar research interests.
• Douglas Degelman's Amoeba Web has a clean simple interface, but reflects a goldmine of resources, including full-text psychology related journals.
• The Encyclopedia of Psychology is a searchable database of over 2000 links, with emphasis on science and theories in psychology.
• Russ Dewey's Psych Web: In addition to a very large selection of links to other mental health web sites, you’ll find dozens of brochures and articles to use in your practice, information for students and teachers of psychology, self-help resources, and an index to more than 1500 journals.

Unlike Google, which returns search results in one long sequence, other search engines group the results into categories. Clustering gives your search extra precision, allowing you to hone in on the subtopic that addresses your interests. It is expecially helpful when you're not familiar with the range of concepts related to your topic, or when you would probably get better results by using a synonym.

Here are some useful clustering search and metasearch engines:

• Vivisimo and its recent offspring, Clusty: Although these metasearch engines use the same technology, they yield slightly different results. Clusty is more customizable. Both are very good for narrowing your search, especially when you limit your search to medical documents, government, or one of the other pre-specified areas.
• Teoma: offers "Refinements" to narrow your search. When possible it also displays hubs or "expert" sites that have many links to your topic.
• AskJeeves lists "Related topics" when relevant.
• Metacrawler asks, "Are you looking for?" and lists subtopics related to your search.
• Ixquick, Dogpile and QueryServer also give suggestions for related search concepts.

To experience the power of clustering, use the search term procrastination in Google, and then in Clusty and one of the other clustering search engines. Google will yield the most hits, but the others may be more useful.



Search Inside Books

Amazon has a feature called "search inside" where you can browse through the table of contents and read a few pages of the first chapter. Recently added was the option to search for specific words or phrases. For example, if you go to the Amazon page listing my book, Taming Your Inner Brat: A Guide For Transforming Self-Defeating Behavior if you move your mouse to the book cover, a menu will appear with a search box where you can enter a word or phrase. The menu also shows how many other books were cited by the author, along with a link to a list of these books.

Not all books have the "search inside" feature enabled. It is up to the publisher to instruct Amazon to make this option available.

Amazon has its own search engine, A9. It does more than find books. It also finds articles, reference materials, images, movie reviews and more. You can also save your searches if you register at the website.

Google is gradually adding book titles to its search results. If there is a book title that matches your search words, Google displays this at the top of the results. When you click on the link, you can search within the book, and if you wish, be directed to a website to buy the book.



Find Experts and Expert Opinions

One of the quickest ways to find an expert is to add the word expert to your search terms (e.g., ADHD expert.) Many search results contain links to "expert witnesses" near the top. If you want to eliminate these from your hits, include -witness in your search terms. Example: ADHD expert -witness

AllExperts is a free question-and-answer service staffed by volunteers with experience in various subject areas. You can read the experts' profiles and pick to whom you want to pose your question.

At Google Answers you can pose questions that will be answered by a live researcher who has been tested and selected by Google. There is a fee, ranging from $2.50 to $200. You offer what you think the search is worth, and if you're not satisfied, you'll get a refund. All answers are posted for the public to read. Before plunking down your money, search within Google Answers to see if your question has already been addressed.