Machine Learning

By: Koné Mamadou Tadiou, PhD

A.      Foundations of Machine Learning

A. 1.  What is machine learning?

The field of Machine Learning (ML) in Artificial Intelligence focuses on research on the logical foundations and the design of practical systems which learn from experience, adapt to new situations and improve their behavior over time. ML [1] finds inspiration in biological learning entities and also includes many other disciplines like probability theory, computational logic, optimization, Web search, statistics, and control theory.  Its most well known technique is classification which accepts a vector of values (training set of examples) as input and produces a single discrete output value, the classifier.

ML learning algorithms deal in general with representation, evaluation and optimization:

  • A learning process is always preceded by the choice of a formal representation of the classifier. The set of classifiers that a machine can handle is called its hypothesis space.
  • For the best possible outcome, the learning algorithm uses a function called objective function to identify good classifiers by means of scores.
  • A method for selecting the highest-scoring classifiers is known as optimization.Fig. 1: Typical machine learning tag cloud

A. 2.  Motivation for machine learning

Ray Kurzweil, an American author, inventor and futurist, suggested that the methods of intelligence not yet understood is the mystery that attracts us. The motivation behind current research in ML fits perfectly this image. It’s about transferring control from man to machine, empowering the machine in a way that it is able to program itself through selected examples and experience. Machine learning research aims at the possibility of instructing machines in a way that lessen the burden of hand-programming of complex information into future computers. ML methods excel in application domains too fuzzy (ex. perception, computer vision) for humans to manually design an appropriate algorithm.

A. 3.  Example machine learning problems

Here is a set of common classification problems where our goal is to create categories of objects according to particular properties.

  • optical character recognition (OCR): is a type of pattern recognition [3]. that identifies and classifies handwritten characters (ex. what is the phone number in this scanned image?).
  • face detection:  identifies particular faces in an image along some given features.
  • spam filtering: distinguishes legitimate e-mail form spam.
  • news labeling: sorts news according to their subject content (ex. politics, religion, leisure).
  • natural language understanding: determines and classifies words spoken in a discourse by a human.
  • predictions: given a set of variable (ex. clinical, demographic) predict thelikelihood of occurrence of an event (heart attack, stock price fluctuation, prostate cancer resurgence) .

B.      Background of Machine Learning

Machine Learning techniques stem originally from that of Computer Science and Statistics [2]. Its history evolved along the path of the decline of neural networks systems and the emergence to knowledge-intensive systems. In the beginning, a good understanding of the learning process of biological entities was important for the purpose of reproducing aspects of it in computer systems. In its early days, ML also found inspiration from research into computer-assisted tutoring systems with which it shared many of the same objectives and perspectives. Over the years, ML has, in return, inspired developments in this field to generate Intelligent Tutoring Systems based on AI techniques.

C.      Methods of Machine Learning

There are three main algorithms in ML: supervised learning, unsupervised learning and reinforcement learning. Here is below a table which describes the components of these algorithms.

 

Supervised LearningUnsupervised LearningReinforcement Learning
Artificial neural networkArtificial neural network
Bayesian statisticsAssociation rule learningQ-learning
Case-based reasoningHierarchical clusteringLearning automata
Decision treesPartitional clustering
Learning automata
Instance-based learning
Regression analysis
Linear classifiers
Decision trees
Bayesian networks
Hidden Markov models

Fig. 2  Elements of main machine learning algorithms

The most common learning paradigms are induction, clustering, analogy, discovery, genetic algorithms and reinforcement. The evaluation of the success of a learning algorithm are its predictive accuracy, the speed of the learner, the speed of the classifier, the space requirements.

D.      Applications of Machine Learning

Machine learning has become one of the most active and rewarding areas of research [8] due to its widespread use in situations as diverse as natural language processing, speech recognition, spam detection, Web search, computer vision, medical diagnosis [4], finance (credit scoring, fraud detection, stock trading) and robotics [6]. [7]. Many data-intensive scientific and industrial research (ex. British Petroleum, Cessna) use machine learning techniques in scientific discovery.

Remarkable case studies have shown that ML applications have made predictions (ex. breast cancer diagnosis) more accurate (72%) than human beings (65%). Here are details about some of these applications:

Speech recognition.  This technology – pioneered by IBM – made its debut in Text-To-Speech (TTS) or Speech-To-Text software used in transcription. In order to succeed, such tool must be trained right from the start, with the age group and accent (reading aloud some text) of the human subject. It needs data (speech pattern) fed by the subject and some Bayesian inference (probability) to improve its accuracy over time. For example, the NELL  (Never Ending Language Learning) project led by Tom Mitchell (Carnegie Mellon U.) is learning how to read information from the Web.

Computer vision. Many face recognition systems using vision are developed with machine learning technology. With stunning accuracy, US Post Office  uses ML technology to automatically sort around 80% of envelopes with handwritten addresses.

Bio-surveillance. The Center for Disease Control (CDC) in Atlanta, GA, uses ML technology to detect and track disease outbreaks around US. In addition, the Real-time Outbreak and Disease Surveillance (RODS), a public health surveillance software, collects and analyzes disease data. It uses a machine learning software to classify admissions in terms of categories of symptoms  along geographical distribution.

Robot control. Supervised learning techniques is standard practice in the field robotics. For example, it is used for detecting and repelling dust and snow, identifying vegetation and locating obstacles. In self-supervised learning, a robot can generate training samples, effectively teach itself to increase performance.

E.      Perspectives in Machine Learning

As industry leaders like Google (X labs), Microsoft and Yahoo [5] are actively involved in research and making significant investments in Machine Learning, there is a strong feeling that its future is as bright as never before.

Nowadays, the most promising and exciting aspect of research in ML is Deep Learning (DL). It deals with learning higher-level concepts at several levels of representation. DL is helping researchers make new discoveries in speech recognition and computer vision. Apple’s Siri virtual personal assistant as well as Google Street View are interesting illustrations of this new and promising technique. DL is well on its way of outperforming humans in areas like patterns recognition.

F.      References

[1]. Kodratoff, Yves et al. (1990 ), Yves et al. Machine Learning: An Artificial Intelligence Approach, (Vol. 3).

[2] Hastie, Trevor;  Tibshirani , R.;  Friedman, J. (February 2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
URL = .http://www-stat.stanford.edu/~tibs/ElemStatLearn/

[3]. Christopher M. Bishop (2006). Pattern Recognition and Machine Learning, Springer ISBN 0-387-31073-8.

[4]. Wernick, Yang et al. (July 2010). Machine Learning in Medical ImagingIEEE Signal Processing Magazine, vol. 27, no. 4, pp. 25-38.

[5] Yahoo Research (Machine learning group). Current research on Machine Learning Applications at Yahoo. URL = http://research.yahoo.com/Machine_Learning.

[6] Pieter Abbeel (October 2012). Machine learning for Robotics, a video lecture,
URL = http://videolectures.net/ecmlpkdd2012_abbeel_learning_robotics/

[7] Klingspor V.; Demiris, J. ( ). Human-Robot communication and Machine Learning,
Applied Artificial Intelligence Journal, Vol. 11, pp. 719-746, 1997
URL = http://www.iis.ee.ic.ac.uk/yiannis/KlingsporDemirisKaiser97.pdf

[8] Andrew Ng (2013). Machine Learning: Coursera open access online lecture.
URL = https://www.coursera.org/course/ml