Digital Evidence Location via SVM Outlier Detection
Digital Evidence Location via SVM Outlier Detection
a point which can keep a low false negative rate while cidents. For more detail information about feature cal-
keeping a low false positive rate when using outlier de- culation, please refer to [9, 14]. However using too many
tection techniques. We resolve this problem by using a features for various incidents will cause the problem of
two-tier approach that combines outlier detection to re- over-fitting of both the Outlier Detector and the Post-
duce false negative rate with postprocessor to reduce false processor, which will increase the cost of calculation and
positive rate (see Figure 2). Datasets needing investiga- reduce accuracy. The best way is to select a subset of
tion are first preprocessed, and attributes describing the features used for the investigation. We analyzed various
target object are extracted from investigated datasets by combinations of features for their contribution to the Out-
the Feature Calculator. Then the Outlier Detector is em- lier Detector and the Postprocessor accuracy according
ployed to learn the ”outlier-ness” of the records of interest to the method proposed in [10]. The result of analyzing
to the investigators and to eliminate the records that have about the relationship between incident category and its
a high-probability of being normal. Then the outliers de- feature subset is described in Table 1.
tected by the Outlier Detector and the rest of the records
are passed on to Postprocessor which can be regarded as 3.2 Outlier Detector Using One-class
a secondary classifier to further determine whether they
SVM
are false positives from the outlier detector or potential
evidence with the support of existing knowledge consist- After extracting and selecting the suitable features, the
ing of expert knowledge and existing evidence samples. next step is to eliminate records that have a very high
Records outputted by the postprocessor will be labelled probability of being normal through an outlier detector.
as potential evidence samples that can be used as existing Although almost any anomaly detection methods could be
knowledge in the next iteration, and passed on to the Tar- applied, we employ a commonly used one-class Support
get Object unit to define or update the searching targets Vector Machine (SVM) with a modified Gaussian (RBF)
for further investigation. Repeat the above steps until kernel. The standard RBF kernel function is based on
the evidence of an incident which refutes or supports an the Euclidean Distance function. One weakness of the
investigation hypothesis [3] have been acquired. The for- basic Euclidean distance function is that if one of the in-
mal description of the above components is detailed in the put attributes has a relatively large range, then it can
following sections respectively. overwhelm the other attributes. For example, if there
are two attributes (A1 and A2), and A1 can have val-
ues from 1 to 10000, and A2 has values only from 1 to
3.1 Feature Calculator 10, then A2’s influence on the distance function could be
Feature Calculator (FC) is responsible for extracting fea- overwhelmed by A1’s influence. Besides this, it can not
tures from the original investigated data source. Feature effectively handle applications with both continuous and
extraction and selection from the available data is impor- nominal attributes. So we redefine the RBF kernel func-
tant to the effectiveness of the methods employed because tion as:
the great capability in selecting the suitable features of a −kD(x,y)k
classifier can lead directly to faster training and more ac- KΦ (x, y) = e δ2 .
curate results. Usually the selection of what kinds of fea- Where D(x, y) is the Heterogeneous Value Difference
tures depends on the target objects defined or constructed
Function (HVDM)[15] and defined as:
by the forensic investigator. In theory, the more easily to
select features, the more specific the target object is. In D(x, y) = (Σm 2 1
i=1 di (xi , yi )) .
2
fact, it is very difficult to define a clear target object be-
fore carrying out a forensic investigation of an incident. Where m is the number of attributes, di is the distance
But it is comparatively easy to identify the category the function for ith attribute and defined as:
investigated incident belongs to. So we categorize the tar-
get object into five different classes R2L, DOS, Probe, 1, xi or yi unknown
U2R, Normal for network forensics. Where R2L de- di (xi , yi ) = dvdm (xi , yi ), xi and yi are nominal
ddif f (xi , yi ), xi and yi are numeric
notes unauthorized access from a remote machine, such as
guessing a password; DOS denotes denial-of-service, such Ni,x,j Ni,y,j
as smurf attack; U2R denotes unauthorized access to lo- where dvdm (xi , yi ) = Σkj=1 | Ni,x − Ni,y |, ddif f (xi , yi )
|xi −yi |
cal superuser privileges, such as various “buffer overflow” = 4σi , and
attacks; “probe” denotes surveillance and other probing,
such as host or port scanning [14]. • σi is the standard deviation of the numeric values of
Under the network environment, there are many traffic ith attribute;
features that can be used for intrusion detection or event • Ni,x is the number of instances in the training set T
analysis, such as, source address and port number, desti- that have value x for ith attribute;
nation address and port number, timestamp, etc. Stolfo
and his team have researched this topic in-depth and cal- • Ni,x,j is the number of instances in T that have value
culated 41 different features in all applied to various in- x for ith attribute and output class j;
International Journal of Network Security, Vol.6, No.3, PP.301–308, May 2008 304
Note: the numbers in the above “Feature Subset Vector” column are corresponding to the serial number of 41
various quantitative and qualitative features extracted in [14].
M inimize 21 wT w + vl1 l
Σi=1 ξi − p
Subject to yi (w · Φ(xi )) ≥ p − ξi , ξi ≥ 0, i = 1, · · · , l.
one of the models (such as FSV(dos) model) to filter the • DS(u2r): DataSet(normal) ∪ DataSet(u2r);
false positives under the support of existing evidence or
knowledge (see Figure 4(a)), or the union of multiple mod- • DS(norm): DataSet(normal) ∪ DataSet(r2l) ∪
els, such as (see Figure 4(b)), to systematically evaluate DataSet(u2r) ∪ DataSet(dos) ∪ DataSet(probe).
the output of each classifier under the support of forensic
experts. Besides this, in order to build the classification mod-
The output of the Postprocessor will be the potential els for the Postprocessor, we randomly selected different
evidence or evidence snippet, and a forensic investigator number (K) of attack samples from the filtered attack
can define new target objects or employ other tools (such samples, and trained the classifiers.
as keyword searching tool) on the bases of the output for In order to measure the performance of the proposed
further investigation. method, the ROC curve is used. The ROC curve is a plot
of detection accuracy against the false positive rate [15].
It can be obtained by varying the detection threshold.
4 Experiments and Results Detection rate and false positive rate may be defined as
follows:
In this section, we introduce the experiment methods and
present initial results of the use of our proposed method Detection rate = T P/(T P + F N )
to detect outlier in intrusion detection databases. In all F alse positive rate = F P/(F P + T N ).
experiments we employ a SVM tool called LIBSVM [6]
and the modified RBF kernel. Where TP denotes true positives, FN denotes false nega-
tives, FP is false positives and TN is true negatives. Be-
4.1 Dataset sides this, both the Outlier Detector and the Postproces-
sor employ the SVM algorithm, and we hope that our
The data for our experiments was prepared by the 1999
SVMs can detect outliers with high accuracy, while clas-
DARPA intrusion detection evaluation program from
sifying normal examples with great confidence. This is
MIT Lincoln Labs [9]. The following experiments are
controlled to a great degree by the parameter v. In order
based on the 10% train data subset with 494,021 data
to reduce the complexity of the experiments, all exper-
records. Each record has 41 attributes for each connec-
iments were performed with constant value of v (0.125)
tion plus one class label, and the class label will only be
selected by a great deal of experiments.
used for testing, not participation in the classification.
1 1
0.9 0.9
0.8 0.8
0.7 0.7
Detection Rate
Detection Rate
0.6 0.6
0.5 0.5
0.4 0.4
(a) ROC curves with DS(dos) (b) ROC curves with DS(probe)
1 1
0.9
0.8 0.8
0.7
Detection Rate
Detection Rate
0.6
0.6
0.5
0.4
0.4
Outlier Detector
0.3 Outlier Detector
0.2 Postprocessor(K=20)
Postprocessor(K=20)
Postprocessor(K=50)
0.2 Postprocessor(K=50)
Postprocessor(K=100) Postprocessor(K=100)
0.1 0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 0 0.2 0.4 0.6 0.8 1 1.2 1.4
False Positive Rate(%) False Positive Rate(%)
(c) ROC curves with DS(r2l) (d) ROC curves with DS(u2r)
5 Conclusions
1
future goals.
0.8
0.6
References
0.4
[1] Be. E. Boser, I. Guyon, and V. Vapnik, “A training
0.2 algorithm for optimal margin classifiers,” Computa-
tional Learing Theory, pp. 144-152, 1992.
0
0 100 200 300 400 500 [2] C. Campbell and K. P. Bennett, “A linear program-
Value of Paramter K ming approach to novelty detection,” Advances in
Neural Information Processing Systems, vol. 14, pp.
Figure 7: Relation curves between parameter K and false 395-401, 2001.
positives [3] B. D. Carrier and E. H. Spfford, “Defining event
reconstruction of a digital crime scene,” Journal of
Forensic Sciences, vol. 49, no. 6, pp. 1291-1298, 2004.
[4] B. Carrier, File System Forensic Analysis, Addison
Wesley Professional, 2005.
[5] B. D. Carrier and E. H. Spafford, “Automated digital
DS(norm) datasets individually. The results of experi- evidence target definition using outlier analysis and
ments are presented in Figure 7 which shows the rela- existing evidence,” in 2005 Digital Forensic Research
tionship between the parameter K and the False Positive Workshop (DFRWS), 2005.
rate when the detection rate is a constant (90%). From [6] C. C. Chang and C. J. Lin, LIBSVM: A Li-
the Figure 7, we can draw the conclusion that the increase brary for Support Vector Machines, 2001.
of the parameter K will help decrease the False Positive ([Link] cjlin/libsvm)
at the very start, and this trend will tend to smooth with [7] M. Davy, A. Gretton, A. Doucet, and P. J. W. Rayne,
the increase of K. So we can conclude further that the pa- “Optimised support vector machines for nonstation-
rameter K should have an ideally critical value which will ary signal classification,” IEEE Signal Processing
benefit the decrease of the False Positive to a great de- Letters, vol. 9, pp. 442-445, 2002.
gree. In our experiments, the critical value of K is about [8] F. Desobry and M. Davy, “Support vector-based on-
100. The critical value of K depends on both the inter- line detection of abrupt changes,” in Proceedings of
ested type of event and the quality of training data. In ICASSP, vol. 4, pp. 872-875, 2003.
practice, the selection of K will become even worse, and [9] W. Lee and S. J. Stolfo. “Data mining approaches
how to determine the value of K will be a challenging for intrusion detection,” in Proceedings of the 7th
topic in theory. USENIX Security Symposium, pp. 79-93, 1998.
International Journal of Network Security, Vol.6, No.3, PP.301–308, May 2008 308