Machine learning

 Article

 Talk
 Language
Download PDF
Watch
Edit
For the journal, see Machine Learning (journal).

"Statistical learning" redirects here. For statistical learning in linguistics,

Machine learning (ML) is a field of study in artificial intelligence concerned with the
development and study of statistical algorithms that can learn from data and generalize to unseen
data, and thus perform tasks without explicit instructions.[1] Within a subdiscipline in machine
learning, advances in the field of deep learning have allowed neural networks, a class of
statistical algorithms, to surpass many previous machine learning approaches in performance. [2]
ML finds application in many fields, including natural language processing, computer
vision, speech recognition, email filtering, agriculture, and medicine.[3][4] The application of ML to
business problems is known as predictive analytics.
Statistics and mathematical optimization (mathematical programming) methods comprise the
foundations of machine learning. Data mining is a related field of study, focusing on exploratory
data analysis (EDA) via unsupervised learning.[6][7]
From a theoretical viewpoint, probably approximately correct learning provides a framework for
describing machine learning.
History
edit
See also: Timeline of machine learning

The term machine learning was coined in 1959 by Arthur Samuel, an IBM employee and pioneer
in the field of computer gaming and artificial intelligence.[8][9] The synonym self-teaching
computers was also used in this time period.[10][11]
Although the earliest machine learning model was introduced in the 1950s when Arthur
Samuel invented a program that calculated the winning chance in checkers for each side, the
history of machine learning roots back to decades of human desire and effort to study human
cognitive processes.[12] In 1949, Canadian psychologist Donald Hebb published the book The
Organization of Behavior, in which he introduced a theoretical neural structure formed by certain
interactions among nerve cells.[13] Hebb's model of neurons interacting with one another set a
groundwork for how AIs and machine learning algorithms work under nodes, or artificial
neurons used by computers to communicate data.[12] Other researchers who have studied
human cognitive systems contributed to the modern machine learning technologies as well,
including logician Walter Pitts and Warren McCulloch, who proposed the early mathematical
models of neural networks to come up with algorithms that mirror human thought processes.[12]
By the early 1960s, an experimental "learning machine" with punched tape memory, called
Cybertron, had been developed by Raytheon Company to
analyse sonar signals, electrocardiograms, and speech patterns using rudimentary reinforcement
learning. It was repetitively "trained" by a human operator/teacher to recognize patterns and
equipped with a "goof" button to cause it to reevaluate incorrect decisions.[14] A representative
book on research into machine learning during the 1960s was Nilsson's book on Learning
Machines, dealing mostly with machine learning for pattern classification.[15] Interest related to
pattern recognition continued into the 1970s, as described by Duda and Hart in 1973.[16] In 1981 a
report was given on using teaching strategies so that an artificial neural network learns to
recognize 40 characters (26 letters, 10 digits, and 4 special symbols) from a computer terminal. [17]
Tom M. Mitchell provided a widely quoted, more formal definition of the algorithms studied in
the machine learning field: "A computer program is said to learn from experience E with respect
to some class of tasks T and performance measure P if its performance at tasks in T, as measured
by P, improves with experience E."[18] This definition of the tasks in which machine learning is
concerned offers a fundamentally operational definition rather than defining the field in cognitive
terms. This follows Alan Turing's proposal in his paper "Computing Machinery and
Intelligence", in which the question "Can machines think?" is replaced with the question "Can
machines do what we (as thinking entities) can do?".[19]
Modern-day machine learning has two objectives. One is to classify data based on models which
have been developed; the other purpose is to make predictions for future outcomes based on
these models. A hypothetical algorithm specific to classifying data may use computer vision of
moles coupled with supervised learning in order to train it to classify the cancerous moles. A
machine learning algorithm for stock trading may inform the trader of future potential
predictions.[20]
Relationships to other fields
edit
Artificial intelligence
edit
 Machine learning as subfield of AI[21]

As a scientific endeavor, machine learning grew out of the quest for artificial intelligence (AI).
In the early days of AI as an academic discipline, some researchers were interested in having
machines learn from data. They attempted to approach the problem with various symbolic
methods, as well as what were then termed "neural networks"; these were
mostly perceptrons and other models that were later found to be reinventions of the generalized
linear models of statistics.[22] Probabilistic reasoning was also employed, especially in automated
medical diagnosis.[23]: 488
However, an increasing emphasis on the logical, knowledge-based approach caused a rift
between AI and machine learning. Probabilistic systems were plagued by theoretical and
practical problems of data acquisition and representation.[23]: 488 By 1980, expert systems had come
to dominate AI, and statistics was out of favor.[24] Work on symbolic/knowledge-based learning
did continue within AI, leading to inductive logic programming(ILP), but the more statistical line
of research was now outside the field of AI proper, in pattern recognition and information
retrieval.[23]: 708–710, 755 Neural networks research had been abandoned by AI and computer
science around the same time. This line, too, was continued outside the AI/CS field, as
"connectionism", by researchers from other disciplines including John Hopfield, David
Rumelhart, and Geoffrey Hinton. Their main success came in the mid-1980s with the reinvention
of backpropagation.[23]: 25
Machine learning (ML), reorganized and recognized as its own field, started to flourish in the
1990s. The field changed its goal from achieving artificial intelligence to tackling solvable
problems of a practical nature. It shifted focus away from the symbolic approaches it had
inherited from AI, and toward methods and models borrowed from statistics, fuzzy logic,
and probability theory.[24]
Data compression
edit
This section is an excerpt from Data compression § Machine learning.[edit]

There is a close connection between machine learning and compression. A system that predicts
the posterior probabilities of a sequence given its entire history can be used for optimal data
compression (by using arithmetic coding on the output distribution). Conversely, an optimal
compressor can be used for prediction (by finding the symbol that compresses best, given the
previous history). This equivalence has been used as a justification for using data compression as
a benchmark for "general intelligence".[25][26][27]
An alternative view can show compression algorithms implicitly map strings into implicit feature
space vectors, and compression-based similarity measures compute similarity within these
feature spaces. For each compressor C(.) we define an associated vector space ℵ, such that C(.)
maps an input string x, corresponding to the vector norm ||~x||. An exhaustive examination of the
feature spaces underlying all compression algorithms is precluded by space; instead, feature
vectors chooses to examine three representative lossless compression methods, LZW, LZ77, and
PPM.[28]
According to AIXI theory, a connection more directly explained in Hutter Prize, the best possible
compression of x is the smallest possible software that generates x. For example, in that model, a
zip file's compressed size includes both the zip file and the unzipping software, since you can not
unzip it without both, but there may be an even smaller combined form.
Examples of AI-powered audio/video compression software include NVIDIA Maxine, AIVC.
[29]
Examples of software that can perform AI-powered image compression
include OpenCV, TensorFlow, MATLAB's Image Processing Toolbox (IPT) and High-Fidelity
Generative Image Compression.[30]
In unsupervised machine learning, k-means clustering can be utilized to compress data by
grouping similar data points into clusters. This technique simplifies handling extensive datasets
that lack predefined labels and finds widespread use in fields such as image compression.[31]
Data compression aims to reduce the size of data files, enhancing storage efficiency and
speeding up data transmission. K-means clustering, an unsupervised machine learning algorithm,
is employed to partition a dataset into a specified number of clusters, k, each represented by
the centroid of its points. This process condenses extensive datasets into a more compact set of
representative points. Particularly beneficial in image and signal processing, k-means clustering
aids in data reduction by replacing groups of data points with their centroids, thereby preserving
the core information of the original data while significantly decreasing the required storage
space.[32]
Large language models (LLMs) are also efficient lossless data compressors on
some data sets, as demonstrated by DeepMind's research with the Chinchilla
70B model. Developed by DeepMind, Chinchilla 70B effectively compressed
data, outperforming conventional methods such as Portable Network
Graphics (PNG) for images and Free Lossless Audio Codec (FLAC) for audio. It
achieved compression of image and audio data to 43.4% and 16.4% of their
original sizes, respectively. There is, however, some reason to be concerned
that the data set used for testing overlaps the LLM training data set, making
it possible that the Chinchilla 70B model is only an efficient compression tool
on data it has already been trained on.[33][34]

Data mining
edit
Machine learning and data mining often employ the same methods and overlap significantly, but
while machine learning focuses on prediction, based on known properties learned from the
training data, data mining focuses on the discovery of (previously) unknown properties in the
data (this is the analysis step of knowledge discovery in databases). Data mining uses many
machine learning methods, but with different goals; on the other hand, machine learning also
employs data mining methods as "unsupervised learning" or as a preprocessing step to improve
learner accuracy. Much of the confusion between these two research communities (which do
often have separate conferences and separate journals, ECML PKDD being a major exception)
comes from the basic assumptions they work with: in machine learning, performance is usually
evaluated with respect to the ability to reproduce known knowledge, while in knowledge
discovery and data mining (KDD) the key task is the discovery of
previously unknown knowledge. Evaluated with respect to known knowledge, an uninformed
(unsupervised) method will easily be outperformed by other supervised methods, while in a
typical KDD task, supervised methods cannot be used due to the unavailability of training data.
Machine learning also has intimate ties to optimization: Many learning problems are formulated
as minimization of some loss function on a training set of examples. Loss functions express the
discrepancy between the predictions of the model being trained and the actual problem instances
(for example, in classification, one wants to assign a label to instances, and models are trained to
correctly predict the preassigned labels of a set of examples).[35]
Generalization
edit
Characterizing the generalization of various learning algorithms is an active topic of current
research, especially for deep learning algorithms.
Statistics
edit
Machine learning and statistics are closely related fields in terms of methods, but distinct in their
principal goal: statistics draws population inferences from a sample, while machine learning
finds generalizable predictive patterns.[36] According to Michael I. Jordan, the ideas of machine
learning, from methodological principles to theoretical tools, have had a long pre-history in
statistics.[37] He also suggested the term data science as a placeholder to call the overall field.[37]
Conventional statistical analyses require the a priori selection of a model most suitable for the
study data set. In addition, only significant or theoretically relevant variables based on previous
experience are included for analysis. In contrast, machine learning is not built on a pre-structured
model; rather, the data shape the model by detecting underlying patterns. The more variables
(input) used to train the model, the more accurate the ultimate model will be.[38]
Leo Breiman distinguished two statistical modeling paradigms: data model and algorithmic
model,[39] wherein "algorithmic model" means more or less the machine learning algorithms
like Random Forest.
Some statisticians have adopted methods from machine learning, leading to a combined field that
they call statistical learning.[40]
Statistical physics
edit
Analytical and computational techniques derived from deep-rooted physics of disordered
systems can be extended to large-scale problems, including machine learning, e.g., to analyse the
weight space of deep neural networks.[41] Statistical physics is thus finding applications in the area
of medical diagnostics.[42]
Theory
Approaches


1.
2.
3.
4.

Models
Applications






 

Limitations
Model assessments
Ethics
Hardware
Software






 

Journals


Conferences


See also



 

References
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
50.
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
95.
96.
97.
98.
99.
100.
101.
102.
103.
104.
105.
106.
107.
108.
109.
110.
111.
112.
113.
114.
115.
116.
117.
118.
119.
120.
121.
122.
123.
124.
125.
126.
127.
128.
129.
130.
131.
132.
133.
134.
135.
136.
137.
138.
139.
140.
141.
142.
143.
144.
145.
146.
147.
148.
149.
150.
151.
 152.
153.
154.
155.
156.
157.
158.
159.
160.
161.
162.
163.
164.
165.
166.
167.
168.
169.
170.
171.
172.
173.
174.
175.
176.
177.
178.
179.
180.

Sources

 

Further reading


External links


Last edited 23 hours ago by OAbot

RELATED ARTICLES

 Neural network (machine learning)Computational model used in

 Content is available under CC BY-SA 4.0 unless otherwise noted.

 Privacy policy

 Contact Wikipedia

 Code of Conduct

 Developers

 Statistics

 Cookie statement

 Terms of Use

 Desktop