OSINT - Information is power

28/05/2014

Invalid URL provided. Please provide text to translate.

power

The proliferation of Internet use, nearly 2.5 billion users worldwide,

and the ease of content publishing through different media such as social networks
Social networks or blogs have favored the storage of an exorbitant amount of information.
online. Some of the most significant figures are as follows:

• Google stores information from 30 trillion web pages, which amounts to more than
1,000 terabytes of information.
• Facebook has 1.1 billion users, 50 million pages, and 240,000.
millions of photos uploaded to their page.
• Twitter has more than 230 million active users who write daily more
of 500 million tweets.
• Badoo has around 175 million users with a large number of
personal information published.
• Tumblr has about 175 million blogs and around 50 billion
posts.
• Flickr has 84 million users and over 8 billion photos.

These are some of the most well-known representative data. However, it is not necessary to
forget about the amount of information available on the 'Deep Web' since even though not
there are exact figures, its volume is also estimated to be very extensive and can be
very relevant, even on many occasions more than that accessible through methods
conventional.

These figures provide an approximate idea of the enormous amount of data

available publicly on the network, from which it can be obtained
information of great value and utility through techniques such as OSINT.

Description

Open Source Intelligence or "OSINT" refers to

knowledge gathered from publicly accessible sources. The process includes the
search, selection, and acquisition of information, as well as subsequent processing and
analysis of it in order to obtain useful and applicable knowledge in different
fields.

There are numerous open sources from which information can be obtained.
relevant, among which stand out:
 • Media: magazines, newspapers, radio, etc.
• Public information from government sources.
• Forums, social networks, blogs, wikis, etc.
• Conferences, symposiums, papers, online libraries, etc.

Some examples of the use of OSINT are the following:

• Know the online reputation of a user or company.

• Conduct sociological, psychological, linguistic studies, etc.
• Audit of companies and different organizations in order to assess the level of
privacy and security.
• Evaluate market trends.
• Identification and prevention of potential threats in the military or defense context.
national security.
• As a negative aspect, it is used by cybercriminals to launch APT attacks and
Spear Phishing.

Case studies

• A study conducted by researchers from the University of Cambridge (United Kingdom

United) in collaboration with Microsoft Research Cambridge warns that the
preferences shown by clicking on "Likes" are enough to trace
a detailed user profile.
• Researchers from the University of Pennsylvania, in the United States, taking
with information source the status updates of 75,000 people in
Facebook has managed to predict your age, gender, and even your personality type.
based only on the words they used.
• Alessandro Acquisti and Ralph Gross from Carnegie Mellon University conducted
a study in which they used information from various public sources,
 including social media profiles, they reported that they were able to predict with
precision the social security affiliate number of 8.5% of people
born in the United States between 1989 and 2003, nearly five million of
people.
• The universities of Seville and Alicante are developing a platform that
analyze the opinions from the web and social media to help the
institutions or companies to make strategic decisions.
• Researchers from Carnegie Mellon University conclude how the
information shared through social media can lead to the
discrimination in hiring.
• Different European projects aim to obtain and exploit information from networks.
socials, that improve the overall and integrated management of all participants in
crisis and emergency situations, within the 2013 Security program
SEC-2013.6.1-1 The impact of social media in emergencies

Process

The OSINT process consists of the following phases:

• Requirements: it is the phase in which all the requirements that must be established are set.
fulfill, that is, those conditions that must be met to achieve the
objective or to solve the problem that has led to the development of the OSINT system.
• Identifying relevant sources of information: it consists of specifying, based on
the established requirements, the sources of interest that will be collected. It is necessary to
keep in mind that the volume of information available on the Internet is
 practically unapproachable so it is necessary to identify and specify the sources of
relevant information in order to optimize the acquisition process.
• Acquisition: stage in which information is obtained from the sources
indicated.
• Processing: it involves formatting all the information gathered in a structured manner.
that can later be analyzed.
• Analysis: it is the phase in which intelligence is generated from the collected data.
and processed. The goal is to relate information from different sources
searching for patterns that allow for some significant conclusion.
• Presentation of intelligence: it consists of presenting the information obtained from a
effectively, potentially useful and understandable, so that it can be
correctly exploited.

Problems

Two main problems can be identified when using an OSINT system:

• Too much information: as has already been demonstrated, the amount of

public information available on the Internet is more than remarkable. That is why, we
it must carry out a very thorough process when identifying and selecting the
sources of information of interest that will be collected, and that later
will serve for the generation of intelligence. The fact of using an extensive catalog
obviously involves a higher cost when implementing the system,
and in the case of not having the necessary resources available, it causes a
significant slowdown of it.
• Reliability of sources: It is important to assess the sources that will be used beforehand.
nourish the information system since an incorrect selection of them can
provoke erroneous results and misinformation.

Tools

There are a multitude of useful tools and services when it comes to implementing a system.
OSINT. Below are some of them:

• Common search engines: Google, Bing, Yahoo, Ask. They allow consulting all the
information that they index. They also allow specifying concrete parameters.
Hacking with search engines: for example, 'Google Hacking' or 'Bing Hacking'
in a way that searches can be conducted with much greater precision than that which
users commonly use.

Depending on the search engine used, different parameters are employed, although
some of them are common. Some examples of parameterized searches are
the following:

o Files with pdf extension from a specific website: site:[Link] +

ext:pdf
 o Some hacked sites: intitle:"hacked by SultanHaikal"

Through these parameters, sensitive information can be obtained, among other things.
such as usernames and passwords coming from database dumps,
localization of vulnerable servers, access to online hardware devices such as
webcams, surveillance cameras or printers, or personal data such as ID card,
bank accounts, etc.

• Specialized search engines:

o Shodan: Allows among other things to locate computers, webcams,
printers, etc. based on the software, the IP address, the location
geographic, etc. Through this service it is possible to locate information about
interest and, at times, curious and even unsettling, such as for example:
access the control system of an ice skating rink in
Denmark and defrost it, put the entire system in test mode.
traffic control of a city or access the control system of a
hydroelectric plant in France.
o NameCHK: it is a tool that allows you to check if a name of
the user is available on more than 150 online services. In this way, it
you can know the services that a specific user uses, since
usually people keep that name for all the services that
uses. In addition, they have an API that allows automating queries.

o Knowem: it is a tool with similar characteristics to MameCHK

but check the name across more than 550 services, including domains
available.
o Tineye: it is a service that starts from an image and indicates on which sites
web appears. It is similar to the image search incorporated by Google
Images.
o People information search engines: allow searches to
through different parameters such as names, email addresses or
phones. Based on concrete data, they locate users in services such as
social networks, and include possible related data about them such as
phone numbers or photos. Some of the portals that incorporate this
services are: Spokeo, Pipl, 123people or Wink.
• Metadata collection tools:
o Metagoofil: allows the extraction of metadata from public documents
From the extracted information, we
they can obtain email addresses from the staff of a ...
company, the software used for the creation of documents and by
so much power to search for vulnerabilities for this software, names of
employees, etc.
o Libextractor: it is an application similar to Metagoofil that supports many
more formats, although the obtained information is not very useful.
• Services to obtain information from a domain:
o Domaintools: is one of the leading services in this field as
incorporates a large number of features. It is worth noting that it allows
 create alerts for users who register domains, monitor domains and IPs,
create alerts for new domains that contain certain words, and even
a research service for a large number of threats such as 'spear'
phishing, denial of service, spam, fraud or malware.
o Robtex: shows, among other things, the reliability of the domain, its position in
Alexa ranking, the list of subdomains, the mail servers or the
ISP that you use.
o MyIPNeighbors: allows you to obtain the list of domains that share
server with the indicated domain.
• APIs of different services like Facebook, Twitter, Google+, or YouTube:

Through the methods they implement, one can consult in a way

automated the published data.

• Other tools of interest:

o GooScan: allows automating searches on Google by being able to identify from
a simple way subdomains of a specific domain, leaks of
information or possible vulnerabilities.
o SiteDigger: just like GooScan, it allows you to automate searches. It searches in
Google's cache to identify vulnerabilities, errors, issues of
configuration, etc.
o OsintStalker (FBStalker and GeoStalker): they use different social networks
like Facebook, LinkedIn, Flickr, Instagram, and Twitter to collect large
amount of information about a person. They allow locating places and
regularly visited websites, online friends, etc. and display the data
on Google Maps.
o [Link]: allows you to obtain data from Twitter, Flickr, and Instagram. From the
selection of an account extracts dates and GPS information, and creates a database of
Data in CSV or KMZ format to visualize them.
o Theharvester: this tool retrieves emails, subdomains, hosts, names
of employees, open ports, etc. through different services such as
Google, Bing, LinkedIn, and Shodan.
• Palantir and Maltegoal deserve a special mention for implementing a great
number of features and being one of the great references in the field of
intelligence of open sources.

o Palantir: it is a company that has various services as clients.

United States government (CIA, NSA, and FBI) and which focuses on the
software development against terrorism and fraud, through management and
exploitation of large volumes of information.
o Maltego: allows you to visually graph the relationships between people,
companies, websites, documents, etc. from public information.

Conclusions
The intelligence gathered from publicly accessible sources (OSINT) has become a
special relevance in recent years, mainly promoted by the proliferation of
use of the Internet and social media. There is an enormous amount of information.
available on the network, including the 'Deep Web', which can be of great interest in very
various fields that encompass information security, online reputation or the
identification and management of potential risks to national security. Likewise, increasingly...
they carry out more sociological, psychological, or other studies that use
based on the publicly available information on the internet.

Another significant aspect, and which allows one to realize the importance of this type of
information, is the emergence in the labor market of the OSINT analyst figure, which
He is responsible, among other things, for implementing and managing OSINT systems.
McKinsey Global Institute has forecasted that by 2018 there will be a demand in the US.
between 140,000 and 190,000 professionals with statistical and analytical skills
predictive.

All of this has caused different countries to allocate more and more resources to
implement these systems, even creating organizations like Open Source Center
(OSC) in the United States or associations like Eurosinten in Belgium, responsible for
analyze public data in order to identify and prevent threats.

For all the aforementioned, it is undeniable that open source intelligence

can provide a large number of benefits.