How much of the World Wide Web is readily available to search engines? Chief information security officers and IT security professionals know the amount of data hidden from plain sight is substantial — something akin to an iceberg carrying nine-tenths of its weight below the surface.

However, according to a new Global Commission on Internet Governance report, the number is much larger. Just 0.03 percent of the so-called Deep Web is available to search engines, while the even-deeper Dark Web is deliberately hidden and unavailable when using standard browsers.

Such a massive piece of virtual real estate that is essentially unmonitored by Internet oversight agencies raises the question: Is there any hope for cybersecurity in the dark?

Deep Web Versus Dark Web

To understand the effect of Dark Web data, it is first important to separate the Deep Web from its shadowy counterpart. The report, “The Impact of the Dark Web on Internet Governance and Security,” defines the Deep Web as “a class of content on the Internet that, for various reasons, is not indexed by search engines.” For most major engines, this lack of indexing is tied to profit. While the information is readily available to those who look, so few are interested that actively crawling for this content provides little to no return on investment.

The Dark Web, meanwhile, is “a part of the Deep Web that has been intentionally hidden and is inaccessible through standard Web browsers.” Powered by networks such as TOR and I2P, this hidden Web makes it possible for users to remain entirely anonymous. While in some cases, this anonymity is used simply as a way to protect free speech or for government agencies to keep top-secret data under wraps, there is another side to this darker corner of the Web filled with cybercrime, the transfer of illegal goods and even terrorism. Are Internet governance and cybersecurity even possible in this environment?

Here Comes the Calvary

According to a recent Naked Security article, it’s only a matter of time before law enforcement and other agencies gain some measure of control over the Dark Web. The article likens the existing hidden Web to the Wild West — even though it was once larger than the settled territories of the United States, even this lawless land eventually found itself bound by law and order.

According to the Global Commission on Information Governance report, the following are six key monitoring areas that are essential to the success of any governance effort:

  1. Mapping the Hidden Services Directory: Both TOR and I2P use a distributed hash table system to hide database information. Strategically deployed nodes could monitor and map this network.
  2. Customer Data Monitoring: There will be no monitoring of consumers themselves, but rather destination requests to track down top-level rogue domains.
  3. Social Site Monitoring: This includes watching over popular sites such as Pastebin to find hidden services.
  4. Hidden Service Monitoring: Agencies must “snapshot” new services and sites as they appear for later analysis, since they disappear quickly.
  5. Semantic Analysis: A shared database of hidden site activities and history should be built.
  6. Marketplace Profiling: Sellers, buyers and intermediary agents committing illegal acts should be tracked.

The bottom line for businesses? While the Dark Web does not pose any immediate or obvious threat, it exists nonetheless and operates as a catchall both for users seeking anonymity and those looking to operate outside the law. Monitoring this hidden corner of the Web is by no means impossible. It comes down to the choices nation-states and private companies are willing to make. How much light must be thrown at the Dark Web to make it safe, while still respecting the right to Internet anonymity? Is a known darkness better than none at all?

Image Source: iStock

More from

Bridging the 3.4 Million Workforce Gap in Cybersecurity

As new cybersecurity threats continue to loom, the industry is running short of workers to face them. The 2022 (ISC)2 Cybersecurity Workforce Study identified a 3.4 million worldwide cybersecurity worker gap; the total existing workforce is estimated at 4.7 million. Yet despite adding workers this past year, that gap continued to widen.Nearly 12,000 participants in that study felt that additional staff would have a hugely positive impact on their ability to perform their duties. More hires would boost proper risk…

The Evolution of Antivirus Software to Face Modern Threats

Over the years, endpoint security has evolved from primitive antivirus software to more sophisticated next-generation platforms employing advanced technology and better endpoint detection and response.  Because of the increased threat that modern cyberattacks pose, experts are exploring more elegant ways of keeping data safe from threats.Signature-Based Antivirus SoftwareSignature-based detection is the use of footprints to identify malware. All programs, applications, software and files have a digital footprint. Buried within their code, these digital footprints or signatures are unique to the respective…

How Do Threat Hunters Keep Organizations Safe?

Neil Wyler started his job amid an ongoing cyberattack. As a threat hunter, he helped his client discover that millions of records had been stolen over four months. Even though his client used sophisticated tools, its threat-hunting technology did not detect the attack because the transactions looked normal. But with Wyler’s expertise, he was able to realize that data was leaving the environment as well as entering the system. His efforts saved the company from suffering even more damage and…

The White House on Quantum Encryption and IoT Labels

A recent White House Fact Sheet outlined the current and future U.S. cybersecurity priorities. While most of the topics covered were in line with expectations, others drew more attention. The emphasis on critical infrastructure protection is clearly a top national priority. However, the plan is to create a labeling system for IoT devices, identifying the ones with the highest cybersecurity standards. Few expected that news. The topic of quantum-resistant encryption reveals that such concerns may become a reality sooner than…