The Deep Web, or Silent Web as it's sometimes called, is a hidden resource that every writer should know about. It could well be the answer when you want to track down some facts that Google and other search engines can't.
There is considered to be between 350 and 500 million web pages that can be indexed by the likes of Google, MSN, Yahoo and others. The Deep Web is actually not included in these and is reckoned to be many times greater in size!
The 'surface' or 'visible' web is crawled by 'spiders'. These are small robot programs that travel from site to site picking up information which they relay back to the search engine to compile its index.
Unfortunately many websites can't be accessed by spiders. The spider can call at the door, index the address, but can't get in to record the contents. The reasons for this are varied, and include the fact that website owners can deliberately ban spiders from entering their sites if they wish. Another is that if the site comprises of databases, these are anathema to spiders which can't do anything with them. And it is sites like these, made up of databases full of fascinating information on an bewildering variety of subjects, that mainly comprise the Deep Web.
There are thousands, maybe even millions, of such databases out there on the web, but you need to know where they are and what their address is in order to get into them.
The Deep Web is recognised as producing a much higher quality of information than regular search engines due to the many universities and technical and scientific institutions that contribute.
Some databases, such as SJLibrary.org (a collaboration between San Jose State University and San Jose Public Libraries) restrict access to part or all of their content to their students and faculties, but there are plenty of others that do not.
So, it's time to dip your toes into this vast ocean and see what you can find. To start you off here are some sites to try:
Direct Search is a good one (http://www.freepint.com/gary/direct.htm)
Intute - I found this an excellent site http://www.intute.ac.uk/
Complete Planet http://aip.completeplanet.com/
InfoMine http://infomine.ucr.edu/
FindArticles is a database of - you guessed it - articles http://findarticles.com/
For more information on the subject of Deep Web go to deepweb.com http://www.deepweb.com/
------
Mervyn Love writes on several topics including creative writing. His website http://www.WritersReign.co.uk has a mind-boggling array of resources, articles and links to keep any writer happy for hours. Subscribe to the WritersReign Article Writing course here: http://www.writersreign.co.uk/WRac.html
