Shedding Light on the Dark Web – What is it, Really, and How Can it Help Me? 

Dark web, darknet, deep web – all sexy new terms that are often overused and not well understood. Definitions are all over the place ranging from illegal and nefarious, to private, commercial, encrypted and so on. When looking at Internet content in 2017, I use the following definitions to describe the three layers of the web:

Normal Web

Resources on the Internet that you can find using a standard search engine solution such as Google, Bing and Yahoo. The large majority of the Internet is not indexed or discovered by such solutions. Think of “normal web” as the popular places on the Internet, like YouTube, Facebook, Baidu, Wikipedia and similar sites. Of the data that does come up on a “normal web” query, most people only view the first page of results never checking out the thousands of possible ‘hits’ on other pages.

Deep Web

Resources on the Internet that are not indexed by standard search engines (not found in normal web queries). Again, the majority of the Internet is not indexed but it can be found through deep manual searching of the Internet. Just because a search engine hasn’t crawled the site doesn’t mean you can’t find it on your own. There are sites that limit access to data, such as a commercial source like Virus Total or Court records, which require authenticated credentials in order to discover and view data of interest.

Dark Web

Resources on the Internet that are intentionally hidden. This is common for the eCrime and criminal marketplaces, such as botnets, child porn and illicit drug use. However, it is not only the “criminal web,” it is possible that someone intentionally hides a resource that they only want to use periodically and securely, such as a temporary FTP server to share files with a remote contractor.

Darknet is another term I mentioned above. It often relates to the architecture of the hosting of hidden, proxied or anonymous type resources on the Internet. I am hopeful we will not see this term used too much as we need clarity in this market, not confusion.

Some equivocate the use of Internet proxies and anonymization with the dark web. Use of such solutions simply masks your location and/or identity. That is different from attempting to intentionally conceal a resource on the Internet. I regularly use proxies and anonymizations every day and it is for legitimate means. Take, for example, a moniker inside of a global interactive gaming environment. Of course I’m not going to post my real name, social and home address. The same is true for the use of solutions like Tor, where I may not want people collecting information about my IP or geolocation. Thus, privacy is a legitimate need for use of such tools. Freedom of speech is also another motive, where individuals in some countries are oppressed – pursued and killed at times – where proxies and anonymization protects them and their loved ones from totalitarian regime.  

In 2017 a lot of security companies are starting to have their staff join forums, chat rooms and dark web communities. What they don’t realize, being novices at this game, is that most of the individuals inside of such easily garnered dark web resources are actually law enforcement or other ill-trained naïve analysts. There’s a lot of noise to signal in these types of dark web resources as it’s a dance of monitor and counter-monitor and attempting to correlate monikers, relationships, peer-to-peer modeling and so forth. People are smart, especially the ones worried about getting caught, and they have a plan to keep you busy as well as map out your tools, tactics and practices to use against you. Most of the time in such forums I see “blue on blue” where one law enforcement officer or body is investigating another because of a post they made.

More mature experts in the field, of which there are a very small number in this world today, understand the more complex nature of ecosystems such as eCrime, espionage and terrorism that does take place in very dark places of the Internet every day. Much like joining a gang where you have to prove yourself to join, the most nefarious dark web resources have similar methods in place to ensure a trusted environment for criminal activities. This is also coupled with real-world realities, like in some countries where various mafia cells are well established. Several years back an up and coming cyber hacker infringed upon mafia operations and was beaten to within an inch of his life; he gave up hacking after that.  

How Can the Deep Web and Dark Web Help Me?

With an explosive growth of the Internet, and so much of it not even indexed within normal web queries, every company has a need for visibility into the deep web and dark web. This is a great help for brand reputation, research and development, mergers and acquisitions, investigation of a threat and threat agent, and more.

There are new products and services, especially in the past ten years, designed to mine the Internet at a deeper level to discover if company credentials or data is for sale or has been leaked, locate information about a threat or threat agent of interest, and so on. There are cyber threat intelligence services for exactly this type of solution, looking for rogue domains and data which may impact your brand in addition to extensive global monitoring and response to emergent threats specific to your organization.