The Concept Of Deep Web
A brief introduction
A lot of people have surely heard about the term “Deep Web”, invented by Mike Bergman, usually in films or television. Despite the way that Hollywood has made it look like (as a dark place full of terrible stuff), the deep web includes over 95% of all the content currently on the internet. Furthermore, almost everyone has accessed this side of the internet, and many do it almost every day.
By definition, the deep web is the set of all the content that web browsers such as Google or Bing can’t index. This includes, for example, the webpage that generates once you enter your personal Gmail account, or the one you see when introducing you credit card details to buy a plane ticket. These pages are only visible by the user accessing them, and aren’t findable on Google, which include databases, scientific publications or academic materials.
What does this mean? Well, it means that to access this type of content you will need a password, a code or a special software (like TOR). Despite what many people think, accessing the deep web is not by any means illegal, and pretty much anyone can do it.
The concept of “Deep web”
The name Deep web references a commonly used analogy which compares the Internet with the ocean. Actually this is a really accurate idea because, while the sea level part (the Internet that most of us know and use) is a very accessible and safe space, the deeper part of the ocean is a lot darker, and cannot be accessed by every search engine. Since this part of the web is so large, it really reinforces the ocean metaphor we mentioned earlier.
The reason why so many illegal transactions are committed there is the fact that all the internet traffic is completely encrypted and cannot be monitored, making it easy to maintain anonymity. This level of security is accomplished by coding the domains with HASH and registering them with .onion extensions among other measures.
Now it can be accessed by using special software, being TOR the most popular one, making it really easy for practically everyone to access the black market or the “Dark web” safely. Since anonymity is highly valued there most of the transactions are made with cryptocurrency, like Bitcoin or Ethereum.
Origins
The original purpose of the deep web was relatively simple: the U.S. military wanted (and needed) a way to communicate through the newly “created” internet, without being detected by other foreign entities.
We need to take into account that in the beginning of the web, there was no actual distinction between the deep and the surface web, because, until 1990 (with the “Archie” search engine) there were no “searchers”. As we have mentioned earlier, the military needs turned into the appearance of Tor, which, a few years after its release, had its development retaken by a group of developers that formed The Tor Project.
But… How does it work?
Like we have said before, Google can’t index these pages. By index, we mean a huge list of websites that Google “knows” about, and introduces as an optional result for a user’s search. Apart from that, each site is ranked by its importance, so every user can easily find what he/she’s looking for.
The pages on the deep part of the web aren’t indexed by google because neither Google nor the page owner wants to do it, either to protect the info or because it’s an illegal website.
Because of this, links to deep web pages look quite weird. For example, http://62gs2n5ydnyffzfy.onion/ is the link to a blog of a IIT student (Illinois Institute of Technology) in which he showcases photos and videos of the underground tunnels and roofs of the IIT. Most of those links aren’t accessible from well known browsers such as Chrome and Firefox.
You may have noticed that the link ends in “.onion”. That’s because the main way to access those is TOR, “The Onion Router”.
Onion web surfing
TOR, or The Onion Router, is a web browser 100% focused on maintaining your privacy. Initially created by the Marines, its name is a metaphor of the way it works. Like actual onions, it has many layers of internet nodes, moving through them randomly, so that the traffic cannot be interfered.
As mentioned before, it was first developed by the Office of Naval Research of the US, even if it later its development broke away from the government. It then got available in the public domain in 2002, and has then received updates and improvements.
The TOR browser, which is based in Firefox.
TOR is forbidden in some countries like China, Venezuela or even Russia; countries with a dictatorial government, that try to restrict the freedom of its citizens. The browser makes it easy to bypass the government controls and access otherwise prohibited websites that the users may want to enter to. The creators of TOR pursue a ideal of privacy and anonymity, often limited by ads and web trackers placed by big companies or governments. These controls are becoming wider and more present these last years; we think that’s a reason why many people are beginning to use Tor. The software is based on Mozilla’s Firefox.
The Silk Road and the illegal deep net
We, of course, need to make a distinction between Deep and Dark web. Even tho most non-indexable websites are totally legal, there are some others, the ones who belong to the Deep Net, which are illegal and prosecuted by the police and government.
The most well known case of this is the infamous Silk Road.
The Silk Road was the first and most famous modern darknet market. Operated through Tor, it offered its users the ability to browse and shop anonymously, a much needed “feature” for the buyers and sellers who were trafficking with illegal drugs.
The site’s development began in late 2010, and was launched on February of the year after. Its creator was called Ross Ulbricht, nicknamed “Dread Pirate Roberts”. At first, the accounts for the sellers were limited, sold in online auctions. Later on a price per account was set up. Purchases were made with cryptocurrency, same as nowadays. Its name is based on a commercial route that existed from the 2nd century BCE to the 18th century, and connected Europe with the main markets of Asia, such as India and China.
It worked in a similar way of today’s eBay: products could be bought and sold by pretty much anyone, and could be later rated by its buyers. This initiative defended a free international market where governments had no authority - under a certain set of moral values: products sold there couldn’t cause victims or harm, so child pornography or weapons were forbidden, for example.
The website was seized by the FBI the 2 of October of 2013, and its creator was arrested. The Bitcoin exchange rates dropped enormously as a consequence as well.
After a series of trials, Ross was sentenced to 30 years of prison, and, after being found guilty of 7 charges (such as money laundering or computer hacking), he was finally condemned to life sentence.
Other illegal uses
Some other webpages vary in their purpose, being way darker than The Silk Road. “Red rooms” are horrible sites where a victim is live streamed being tortured or even killed; illegal pornography portals that include even pedophilia can be found too.
Finally, “darker” markets also exist, where services vary from hacker hiring, to weapon, slavery, stolen goods (like jewelry) and even fake passports/ID offers. One of the most famous “darker” markets is Samsara Market.
Legal uses of the Deep Web
As mentioned before, most pages in the Deep Net are legal. We also make a distinction here, between the ones that are accessible through every web browser (such as your personal gmail inbox, accessed with passwords) and the ones only available using Tor.
First off, we have the “daily” sites. Even people who have never heard of the term “Deep Web” have accessed them. This include a wide range of webpages: personal account settings, the sites generated when introducing personal data in online shops (like Amazon, or a plane flight ticket reserve), or even academic materials or student pages. Many scientific publications are also published here.
The gmail inbox is a clear example of daily life Deep Web access.
Academic papers have a high presence in the deep net. We differentiate between the “paid” and “free” ones.
On one hand we have the “paid” ones. Academic papers are usually sold to big businesses, who publish and distribute them in exchange of a fee, usually expensive (sometimes around 30$ per paper). Those papers are stored in databases in the deep web (like most private databases), which fulfill the definition of deep web.
On the other hand, many scientists and readers disagree with the policies of these companies, which often seek getting a high profit instead of distributing knowledge to the people. Due to this, several movements have emerged in the deep web, whose objective is to divulge this findings and papers to everyone who wants, for free. Sci-hub is one of the most well-known deep web websites which do this, and includes over 70 million papers.
Final conclusions
Deep web has been vastly mythified since its appearance years ago, mainly due to two facts: first, Hollywood has used it as a plot base, thus modifying what actually is, such as “Unfriended: Dark Web”; second, since many people don’t understand how it works, they’re scared of it. As autor Andrew Smith says, “People fear what they don't understand and hate what they can't conquer”.
We think some pieces of media that properly represent it are: “Mr Robot” the well known tv series about hackers, “Deep web” a documentary by Alex Winter which follows the deep web and the events that lead to the Silk Road being shut down, and “Anonymous” also called “Hackers” another documentary, directed by Akan Satayev and based on true events.
These last years, a trend of creating fake videos on youtube has appeared, with clickbait titles such as “UNEXPECTED Real Dark Web Mystery Box Goes Horribly Wrong! (Disturbing Content) Very Scary!” (actual 100% real title), which give misleading information and “darkify” even more the already bad image of the deep web. The Dark web is a absolute minority of the deep web, as we have stated before.
Despite the existence of those illegal markets and sites, we believe that the deep web is necessary if we want to have privacy and security on the internet. It’s not by any means hard to access: anyone can enter it in a matter of seconds using Tor, which also is a very valuable tool for people who really care about their privacy. Unfortunately, it will probably keep being treated like a “hole of horrors” for a long time.
