What is OSINT? – Basic Tools and Techniques

The advent of the World Wide Web has given both intelligence agencies and ordinary private users a colossal number of opportunities. The most important of these is the availability of open sources of information and the ability to search them thanks to OSINT technology. Got a problem? Interested in something? Need an answer urgently? Just Google it!

Statistics show that 93% of users search for information using Google and Yandex, while the remaining 7% use YouTube, Bing, Yahoo, Rambler, and others. And if you can’t find something in a search engine, you can always turn to social media, where you can gather the missing information. Simple and straightforward.

An Introduction to OSINT

OSINT (or Open Source INTelligence, as it’s known in its full English version) is a technology for searching, accumulating, and analyzing data collected from accessible internet sources. This sounds too general, so we’ll try to explain the term in more human terms.

Back in 1947, CIA analyst Ken Sherman reported that the country collected approximately 80% of its information from open sources online. A little later, Samuel Wilson, head of the US Department of Defense, said that 90% of intelligence data also comes from open sources, with only the remaining 10% coming from the professional work of agents and spies. So much for the modern-day James Bond, distilled into an internet search bar.

So, how does regular web surfing differ from OSINT? Primarily, it’s the depth of the approach. For many casual internet users, the search ends at the stage where, for professionals, the fun is just beginning.

OSINT allows for approximately 90% of the tasks faced by private detective agencies. The internet offers the most valuable data sources: classifieds sites, marketplaces, blogs, forums, government projects, online banking systems, and social media. The key is to extract truly important and useful information from the overall “sum,” which can sometimes be compared to sifting a mountain of earth through a gold sieve. It’s important to understand that sometimes information is far more valuable and significant than a pile of precious metal. Some data can be worth millions and billions.

So, what’s the difference between a novice and a professional “surfer”? The former simply sees a funny picture, a repost with advice, pages and groups to kill time. A professional notes content publication dates, user engagement and activity, key image details, geotags, and evaluates the target audience—and that’s just the beginning. The next step might involve sending an IP logger to identify IP addresses. Then, a port scan should be performed to determine what technological equipment is located near a specific person: cameras, printers, PCs, routers, and other network-connected equipment. For example, if two or three copiers are found on the list, we can assume the person is at their office.

Here’s another illustrative example. In the address bar of the popular social network VKontakte, a personal ID is appended to the profile’s main address after the slash, which is no secret. However, it can contain not only numbers but also a nickname. The latter is invaluable information, as most online users tend to use the same nicknames across different resources. Thus, just the nickname in the address bar can be used to identify other activities of a specific user: social media pages, messages on relevant forums, resumes, orders on trading platforms, etc.

So, what exactly is OSINT?

This technology allows for the collection of maximum information from open sources for comprehensive professional analysis. This data can be posted in various forms: articles, forum discussion posts, video and audio files, documents, images, animations, GIFs, etc.

Before answering a question or satisfying a need for knowledge, users search for information and subject it to qualitative analysis, which can be extremely time-consuming. Obtaining accurate results for the average user can be a difficult task. Open-source tools, which can be run simultaneously, can help. They will collect data from available sources, leaving you to perform the comparison and analysis.

Below, we’ll explore OSINT tools and technologies in more depth.

Basic Tools and Methods

Shodan. While I’m still actively using Google to answer basic everyday questions, the incredibly powerful search engine Shodan allows hackers to browse exposed assets. The service will immediately show you a selection of results that most fully match your query. The system is most often used to search for assets connected to the network.
This open-source tool allows for high-quality security analysis and testing of specific target vulnerabilities (privacy of personal data, available passwords and ports, IP addresses, etc.). Shodan also provides the most adaptive community search.

Google Dorks. This service has actually been around since 2002, but admit it, have you heard of it? It demonstrates remarkable performance and is a truly intelligent tool based on queries. The open-source service helps users quickly navigate results or the search index.
Maltego. Paterva’s powerful, open-source intelligence tool, built into Kali Linux, is designed for serious target analysis using transformations. It’s written in Java. To use it, you’ll need to register for free on the manufacturer’s website, after which you can begin creating digital fingerprints of your chosen target online. How cool is that?

This tool allows you to convert IP addresses, find and identify AS numbers, and netblocks. With little effort, you can discover specific phrases and target locations.

TheHarvester. A highly focused, yet no less useful tool for finding subdomains, email addresses, IP addresses, and other useful information from a vast array of publicly available information.
Recon-Ng. A useful tool for field reconnaissance using a modular approach. Suitable for users familiar with Metasploit. The tool has built-in modules that allow you to obtain information based on your request and needs. You can use these modules by marking domains in the workspace. These modules are created to perform specialized operations, such as searching for domains related to the original or target domains.
A useful module is bing_linkedin_cache, which allows you to collect information about email correspondence associated with the target domain. It is actively used for implementing social engineering techniques.

On top of that, it’s also a powerful open-source intelligence tool—a must-have for online security researchers.

SpiderFoot. Another useful reconnaissance tool, also open source for Windows and Linux. Written in Python, it has a very user-friendly configuration, works well on almost any platform, and can be integrated into graphical interfaces.

A useful feature is the ability to use queries from over 100 OSINT resources. Collect email data, logins, IP addresses, domain names, and more even faster and more efficiently. You can also learn about network blocks, web servers, and much more, unavailable to the average user.

Use this tool to set up targeted social media advertising campaigns that fully meet your requirements. SpiderFoot collects only useful information, understanding how data is interconnected.

With this tool, you’ll gain a complete understanding of the risk of hacker threats that could make your accounts and profiles vulnerable, leading to the loss of personal data (including payment data). Conduct your own penetration test to improve your threat prevention and overall system security before your data is actually stolen.

Creepy A dedicated geolocation reconnaissance tool that collects data primarily from social media and image and photo hosting services. The service then publishes reports on a map using a dedicated search filter. Reports can be downloaded in CSV or KML format for export to specialized analytics programs.
The tool is open source. Its core functionality is divided into two tabs: “targets” and “map view.”

The resource is written in Python and is available as a packaged binary, allowing for integration into various distributions.

Benefits of Using OSINT

Despite the existence of hundreds of web resources for searching for information on specific individuals or law firms, users still don’t know how to obtain exclusive information.

By using advanced search queries in Google Docs, Bing, Yandex, and DuckDuckGo, users can get surprising, and sometimes even frightening, results.

There are also specialized websites designed to search for people based on specific information provided.

Just imagine resources that search the entire internet based on a single criterion! Simply enter an email address, upload a photo, or even specify an IP address—and voila, your target is found in just one click. In one place, you’ll receive structured and organized information from numerous online resources. Check a specific individual or an entire corporation in just an hour of internet surfing. This will significantly minimize the risks of unscrupulous partnerships or fraud.

Another advantage is the uniqueness of the OSINT system itself. There are no template algorithms for conducting your investigation, as every case is unique and requires an individual approach.

OSINT is represented by a number of platforms that allow you to perform a range of actions in just a few clicks: data search and collection, analytics, studying change trends, comparing results over time, etc.

Thanks to this technology, any user can gather exclusive data that no one would voluntarily provide: this includes studying the pages followed by the target individual, as well as all their “likes,” comments on posts, social circle, and social connections. You can check the individual’s connections and interactions with various people, including bloggers, politicians, officials, media personalities, public organizations, and foundations. These capabilities allow you to assess the risk of interacting with a particular individual in person or during a major transaction. They also help you choose the optimal negotiation strategy based on preferences, which are easily discovered by monitoring the activity of a potential partner on forums and social media.

OSINT is essential for large firms and organizations that want to operate as productively and profitably as possible, while minimizing the risk of leaking valuable commercial information.

What is needed for OSINT work in today’s reality?

The system’s operating algorithm is very simple and requires following a sequential sequence of steps. This sequence has been developed and tested for effectiveness over many years.

• Gather all publicly available source information about the target (personal data, email addresses, photographs, contacts, etc.).
• Define your objectives: what questions need to be addressed and what information is missing to form a complete picture.
• Determine the OSINT tools that effectively address your specific needs.
• Configure your search, then analyze all the collected data.
• Run a repeat search based on the new information obtained.
• Confirm or refute your assumptions.

We can conclude that OSINT is the technology of our present and future. Those who understand its tools and operating principles will always be one step ahead in the competitive struggle and in matters of personal security.