Lecture

2 ICT 3214 Security

Analytics

Security Data
Lecture 2 Outline
￮ Security analytics process
￮ Data sources for security analysis
￮ Events of security interests
Security analytics
process
Is
… …there anomaly in the log?
[Link] - - [07/Mar/2004:16:32:50 -0800] "GET /twiki/bin/view/Main/WebChanges HTTP/1.1"
200 40520
[Link] - - [07/Mar/2004:16:33:53 -0800] "GET
/twiki/bin/edit/Main/Smtpd_etrn_restrictions?topicparent=[Link] HTTP/1.1"
401 12851
[Link] - - [07/Mar/2004:16:35:19 -0800] "GET /mailman/listinfo/business HTTP/1.1" 200
6379
[Link] - - [07/Mar/2004:16:36:22 -0800] "GET
/twiki/bin/rdiff/Main/WebIndex?rev1=1.2&rev2=1.1 HTTP/1.1" 200 46373
[Link] - - [07/Mar/2004:16:37:27 -0800] "GET /twiki/bin/view/TWiki/DontNotify HTTP/1.1"
200 4140
[Link] - - [07/Mar/2004:16:39:24 -0800] "GET /twiki/bin/view/Main/TokyoOffice HTTP/1.1"
200 3853
[Link] - - [07/Mar/2004:16:43:54 -0800] "GET /twiki/bin/view/Main/MikeMannix HTTP/1.1"
200 3686
[Link] - - [07/Mar/2004:16:45:56 -0800] "GET /twiki/bin/attach/Main/PostfixCommands
HTTP/1.1" 401 12846
[Link] - - [07/Mar/2004:16:47:12 -0800] "GET /[Link] HTTP/1.1" 200 68
……
 Information vs Intelligence

￮ Raw data offer pieces of information in isolation

￮ Example – Log files

￮ Intelligence is analysed and refined

￮ Example - finding an anomaly in log file after
analysis

￮ Intelligence helps in
￮ decision making to reduce security risks
￮ finding relationship/patterns/trends in data
Security Analytics Process

Data Security
Analysis
Sourcing Intelligence

•Aggregation •Correlation •Reporting

•Normalization •Visualization
Security Analytics Process

• Ask questions that have

objective answers

• Find and collect

relevant data

• Learn through iterations

• Find statistics
 Tasks in Security Analytics Process
￮ Provide domain expertise
￮ Context of analysis

￮ Manage and curate data

￮ Prepare, store, and maintain data

￮ Program and script

￮ Analyse the data, extract knowledge

￮ Use statistics
￮ To learn from the data, intelligence

￮ Develop visualization
￮ Communicate the results effectively
Data sources for
security analysis
 Chicken and Egg problem

￮ Should you ask questions based on data you

need to have
￮ Objective Questions
￮ Questions that help to explore data
￮ Datasets

￮ Determine data sets that you need to answer

your question
￮ Usefulness to mine knowledge
￮ Quality of the data
Log analysis goals
￮ Goals of analysis varies depending on your
business needs
￮ Security and Compliance

￮ Past bad things

￮ Future bad things
￮ Never before seen things
Policy definition
￮ Logging policy
￮ Adequate logging
￮ Log aggregation and retention
￮ Log protection
￮ Log review
￮ Web server log (Apache access log)

￮ DNS lookup on a non-existent domain

(Unix syslog)
Windows Event Logs
￮ Application “[Link]” :
￮ software related incidents on the OS, like crashing of an
application
￮ Security “[Link]” :
￮ information based on default Windows System audit policies,
which includes login attempts and resource access
￮ Setup:
￮ events related to application setup, and statuses of
Windows Updates
￮ System “[Link]” :
￮ logs relevant to device driver status
￮ Forwarded events:
￮ logs from other machines
￮ Windows event log

￮ Administrative tools >

Computer
Management > Event
Viewer

￮ evtx file in directory

C:\WINDOWS\SYSTEM
32\winevt\logs
Applications and Services logs
Store events for a single application or component, in four
subtypes:
￮ Admin:
￮ targeted at end users and system administrators, used to
fix an issue or take other action
￮ Operational:
￮ generally used to diagnose an issue
￮ Debug and Analytic:
￮ intended for developers, used to diagnose problems that
cannot be handled through user intervention
Selecting log sources
￮ Establish a repeatable process for evaluating
and selecting systems and devices for logging
￮ Criticality
￮ Assign a criticality level to a device, e.g.
￮ firewall is more critical than workstation
￮ DNS server has a lower criticality than your credit
processing server
￮ Validation
￮ Get stakeholders to verify together that the source
is indeed a critical asset
Logging policy on log source
selection
￮ Collect log records from external
firewalls and IDS/IPS systems

￮ Collect log records from every

firewall, IPS, server, and desktop in
your network

￮ Review the policy very 3 to 6 months

Events of security
interests
Security Related Host Logs

￮ Host intrusion detection and

￮ Host logs produced by OS prevention
components ￮ Detect and block various
attacks of network, operating
￮ Various network services logs system, and applications
￮ Events recorded can be
related to
￮ Logs of applications running on
￮ Reconnaissance or probe
the system
detected
￮ Changes to executable
files
 Windows Events to Monitor
￮ Events about Windows Event Log service
Events 1100 and￮ Recorded to the Security event log, regardless
1102 may indicate of the audit policy
￮ 1100(S): The event logging service has
malicious behaviors
of shutting down shut down.
￮ 1102(S): The audit log was cleared.
the Log Service or
clearing Security
event log, to cover ￮ 1104(S): The security log is now full.
one’s activity ￮ 1105(S): Event log automatic backup.
￮ 1108(S): The event logging service
encountered an error while processing
an incoming event published from %1
 Windows Events to Monitor
The list is not exhaustive
Event Log Event ID
Application 1000 (Event Source -Application Error)
1002 (Event Source -Application Hang)

7045 (Event Source -Service Control Manager)

Windows Events to Monitor
7040 (Event Source -Service Control Manager)
1056 (Event Source -TerminalServices-RemoteConnectionManager)
12 (Event Source -Kernel-General)
13 (Event Source -Kernel-General)

Security 4720, 4722, 4723, 4724, 4725, 4726, 4738, 4740, 4765, 4766, 4767, 4780, 4794,
4798, 5376, 5377, 4727, 4737, 4728, 4729, 4730, 4731, 4732, 4733, 4734, 4735,
4754, 4755, 4756, 4757, 4758, 4799, 4741, 4742, 4739, 6416, 4688, 4625, 4627,
4634, 4624, 4648, 4649, 4778, 4779, 5632, 4964, 4672, 5142, 5140, 5145, 4664,
4698, 4702, 4719, 4715, 4713, 4717, 4718, 4704, 4705, 5025, 4608, 4609, 4616,
4610, 4614, 4622, 4697, 6281, 6410, 5038, 1100, 1102, 1104, 5136, 5137,5139,
5141
Registry and file system auditing: 4663 (System Access Control List required), 4657
(SACL required)
Linux/Unix Events to Monitor
Web Server Events to Monitor
OS logs
￮ Authentication

￮ Linux syslog, remote user authenticating

with Secure Shell (SSH) daemon

￮ System start up, shut down and reboot

￮ Linux syslog, system shutdown

OS logs
￮ Service start up, shut down and status
change

￮ Solaris syslog, sendmail daemon starts up

￮ Service crash

￮ Linux syslog, FTP server shutting down

involuntarily (due to a crash or a kill
command)
OS logs
￮ Miscellaneous status message

￮ Linux syslog of a time synchronization

daemon (NTPD) ￮ OS logs are
security relevant
￮ Useful for
intrusion
detection, and
￮ incident
response
Network daemon logs
￮ Connection established to the service

￮ Linux syslog, successful connection to a POP3

mail daemon by a remote user “anton”

￮ Connection to server failed

￮ Linux syslog shows a connection failure (due to

access controls) to a telnet service
Network daemon logs
￮ Connection was established, but access was not
allowed

￮ Linux syslog message shows an unsuccessful

connection to the Secure Shell server
￮ Various failure messages

￮ Linux syslog message shows a failure of a

sendmail daemon to continue talking to a
client (likely a spam program)
Network daemon logs
￮ Various status messages

￮ Linux syslog message indicates a successful

Email transfer
Network daemons present the
most common entry ways into
the system remotely and many
of the attacks are targeted
against them
HIDS & HIPS
￮ Dragon HIDS examples, Insecure system
reconfiguration or corruption

￮ A Nessus vulnerability scanner probe is

detected by watching the FTP log

￮ Dragon HIDS host sensor shows a critical

system file deletion alert
HIDS & HIPS

￮ Authentication or authorization failed

Security related network logs
￮ Network logs generated by network
infrastructure
￮ By routers and switches
￮ NIDS, firewalls

￮ Network infrastructure logs

￮ Logins and logouts
￮ Connection established to the service
￮ Bytes transferred in and out
￮ Reboots
￮ Configuration changes
Network Device Logs to
Monitor
Application logs

￮ Application user activity

￮ Privileged user activity
￮ Routine but critical activity
￮ Reconfiguration
Lecture 2 Summary Data
Security
Sourcin Analysis
Intellige
g
nce
￮ Security analytics process
•Aggregation •Correlation •Reporting
•Normalization •Visualization

￮ Data sources for security

analysis
Lecture 2 Summary
￮ Events of security interests