DEEP WEB

BY-VISHAL GAYAKWAR2K12/IT/99

What is the Deep Web?The deep Web is the “hidden” part of the Web, Inaccessible to conventional search engines, and consequently, to most users.Sometimes called the “Invisible Web”, includes information contained in searchable databases that can only be reached by a direct query or a specialized search engine. Information is contained in dynamic webpages that are generated upon request to a database. It has no persistent or static URL.

The Surface WebWebpages with static or persistent URLs that can be detected by a search engine crawler.

Once detected, the URL is added to that search engine’s database and can become a result in a query or search

of that search engine.

How big is the Deep Web?

550 billion documents 500 times the content of the surface Web Google has identified 1.2 billion documents An Internet search typically searches .03% (1/3000) of available content.The Deep Web contains 7,500 terabytes of information, compared to 19 terabytes of information in the Surface Web.

How Search Engines Work

Uses a spider program to fetch as many webpages as possible

A program called an indexer then reads these webpages and creates an index, storing the URL and important

content of webpage

Each search engine has its own ranking algorithm that returns

results based on their relevance to the user’s specified keywords or

phrases

What’s in the Deep Web?

Searchable databasesDownloadable files & spreadsheetsImage and multi-media filesData setsVarious file formats such as .pdfLots of government information

How is the Deep Web different from the Surface Web?

A search engine “Spider” or “Crawler” will seek out webpage documents by going from one hyperlink to another and adding each page to it’s catalog as it crawls along. This requires that each page have a static or persistent URL.People, not an automated software program, collect and index URLs in the search engine’s catalog.

Surface webpages are added to search engines in one of two ways:.

Why use the Deep Web?Higher quality sourcesSelected and organized by subject expertsDynamic display Customized data setsSome data is visual, and not word searchableRegular search engines miss vast resources available in the Deep WebA search conducted in a Deep Web site on a specific subject will generally yield a greater number of more relevant results than the same search run in a general search engine.

So what is actually on the DeepWeb,

and how do we get there?Deep Web (layer two)

- Content that is only available through specialized anonymous software (Tor).This includes anonymity networks and darknets (anonymous P2P). This are is a sort of safe-haven for speech and content which has been banned from the surface web.

- examples: Anonymous chat/imageboards. CP, drugs, illeagal transactions, weapons, DRM removal software, pirated software/media/books, black markets

Accessing the Deep Web.onion:

-onion is a domain host suffix designating an anonymous hidden service reachable via the Tor network. -The purpose of using such a system is to make both the information provider and the person accessing the information more difficult to trace, whether by one another, by an intermediate network host, or by an outsider. -.onion adresses are 16-character non-mneumonic hashes, compromised of alphabetic and numeric strings.-The "onion" name refers to onion routing, the technique used by Tor to achieve a degree of anonymity.

Step 1: Alice‘s Tor client obtains a list of Tor nodes from a directory server

Step 2: Alice‘s Tor client picks a random path to destination Server. Green links are encrypted, red links are in the clear.

Step 3: If the user wants access to another site, Alice‘s Tor client selects a second random path. Again, green links are encrypted, red links are in the clear.

Searching Deep Web vs. Surface Web

When using a Deep Web index, such as CompletePlanet, Lycos or DirectSearch, you are first searching through a collection of databases, NOT looking for a specific piece of information

Each database is its own searchable collection of information. Once you find one you want to search, you will then conduct another search within that particular database to find the information you want.

BEST OF DEEP WEB CONTENT

AlphaSearch(http://www.calvin.edu/library/searre/internet/as/)

ProFusion(http://www.profusion.com) WebData(

http:/www.Webdata.com/Webdata.htm) Librarians’ Index to the Internet(

http://www.lii.org/) InvisibleWeb.com (

http://www.invisibleweb.com/)

TOP DEEP WEB CATEGORIES

Public Company Filings Library Catalogs Company Directories Job Postings Historical Documents and Images Out of Print Books Patents Economic Information and many more.....

ADVANTAGES....

To protect your privacy from marketers and identity thieves.

They protect your communications from irresponsible corporations.

If you are looking for any kind of software or e-book, chances are you can get it for free.

If you are interested in classified intelligence ,it may be the best source you can get.

Access websites, banned by cyberoam.

DRAWBACKS

JavaScript, external plug-ins (Flash, Java), or non-secure (https) websites could and will give information about you.

You will be facing a elite troop of attackers. Revealing possible personally identifying

information. Think twice before clicking any link, it might be

from the same people you are trying to hide from. You can look for some serious jail-time, if you get

caught looking over any data, which is considered to be a threat to national security eg. Websites banned by government.

At present,the internet is functionally divided into two areas – 1% of the information content is in the surface web and 99% is in the deep web.Search engines index the surface Web but only access the deep web to a very limited degree.As the web evolves more of the deep web will become more easily available;however,at present one must directly access deep Web sites through their query engines.To do this,you need to know the URL of the deep web site, considering there are over 200,000 deep web site, and more are being continuously added

CONCLUSION