A. Foundations of Machine Learning
A. 1. What is machine learning?
The field of Machine Learning (ML) in Artificial Intelligence focuses on research on the logical foundations and the design of practical systems which learn from experience, adapt to new situations and improve their behavior over time. ML  finds inspiration in biological learning entities and also includes many other disciplines like probability theory, computational logic, optimization, Web search, statistics, and control theory. Its most well known technique is classification which accepts a vector of values (training set of examples) as input and produces a single discrete output value, the classifier.
ML learning algorithms deal in general with representation, evaluation and optimization:
- A learning process is always preceded by the choice of a formal representation of the classifier. The set of classifiers that a machine can handle is called its hypothesis space.
- For the best possible outcome, the learning algorithm uses a function called objective function to identify good classifiers by means of scores.
- A method for selecting the highest-scoring classifiers is known as optimization.Fig. 1: Typical machine learning tag cloud
A. 2. Motivation for machine learning
Ray Kurzweil, an American author, inventor and futurist, suggested that the methods of intelligence not yet understood is the mystery that attracts us. The motivation behind current research in ML fits perfectly this image. It’s about transferring control from man to machine, empowering the machine in a way that it is able to program itself through selected examples and experience. Machine learning research aims at the possibility of instructing machines in a way that lessen the burden of hand-programming of complex information into future computers. ML methods excel in application domains too fuzzy (ex. perception, computer vision) for humans to manually design an appropriate algorithm.
A. 3. Example machine learning problems
Here is a set of common classification problems where our goal is to create categories of objects according to particular properties.
- optical character recognition (OCR): is a type of pattern recognition . that identifies and classifies handwritten characters (ex. what is the phone number in this scanned image?).
- face detection: identifies particular faces in an image along some given features.
- spam ﬁltering: distinguishes legitimate e-mail form spam.
- news labeling: sorts news according to their subject content (ex. politics, religion, leisure).
- natural language understanding: determines and classifies words spoken in a discourse by a human.
- predictions: given a set of variable (ex. clinical, demographic) predict thelikelihood of occurrence of an event (heart attack, stock price fluctuation, prostate cancer resurgence) .
B. Background of Machine Learning
Machine Learning techniques stem originally from that of Computer Science and Statistics . Its history evolved along the path of the decline of neural networks systems and the emergence to knowledge-intensive systems. In the beginning, a good understanding of the learning process of biological entities was important for the purpose of reproducing aspects of it in computer systems. In its early days, ML also found inspiration from research into computer-assisted tutoring systems with which it shared many of the same objectives and perspectives. Over the years, ML has, in return, inspired developments in this field to generate Intelligent Tutoring Systems based on AI techniques.
C. Methods of Machine Learning
There are three main algorithms in ML: supervised learning, unsupervised learning and reinforcement learning. Here is below a table which describes the components of these algorithms.
|Supervised Learning||Unsupervised Learning||Reinforcement Learning|
|Artificial neural network||Artificial neural network|
|Bayesian statistics||Association rule learning||Q-learning|
|Case-based reasoning||Hierarchical clustering||Learning automata|
|Decision trees||Partitional clustering|
|Hidden Markov models|
Fig. 2 Elements of main machine learning algorithms
The most common learning paradigms are induction, clustering, analogy, discovery, genetic algorithms and reinforcement. The evaluation of the success of a learning algorithm are its predictive accuracy, the speed of the learner, the speed of the classifier, the space requirements.
D. Applications of Machine Learning
Machine learning has become one of the most active and rewarding areas of research  due to its widespread use in situations as diverse as natural language processing, speech recognition, spam detection, Web search, computer vision, medical diagnosis , finance (credit scoring, fraud detection, stock trading) and robotics . . Many data-intensive scientific and industrial research (ex. British Petroleum, Cessna) use machine learning techniques in scientific discovery.
Remarkable case studies have shown that ML applications have made predictions (ex. breast cancer diagnosis) more accurate (72%) than human beings (65%). Here are details about some of these applications:
Speech recognition. This technology – pioneered by IBM – made its debut in Text-To-Speech (TTS) or Speech-To-Text software used in transcription. In order to succeed, such tool must be trained right from the start, with the age group and accent (reading aloud some text) of the human subject. It needs data (speech pattern) fed by the subject and some Bayesian inference (probability) to improve its accuracy over time. For example, the NELL (Never Ending Language Learning) project led by Tom Mitchell (Carnegie Mellon U.) is learning how to read information from the Web.
Computer vision. Many face recognition systems using vision are developed with machine learning technology. With stunning accuracy, US Post Ofﬁce uses ML technology to automatically sort around 80% of envelopes with handwritten addresses.
Bio-surveillance. The Center for Disease Control (CDC) in Atlanta, GA, uses ML technology to detect and track disease outbreaks around US. In addition, the Real-time Outbreak and Disease Surveillance (RODS), a public health surveillance software, collects and analyzes disease data. It uses a machine learning software to classify admissions in terms of categories of symptoms along geographical distribution.
Robot control. Supervised learning techniques is standard practice in the field robotics. For example, it is used for detecting and repelling dust and snow, identifying vegetation and locating obstacles. In self-supervised learning, a robot can generate training samples, effectively teach itself to increase performance.
E. Perspectives in Machine Learning
As industry leaders like Google (X labs), Microsoft and Yahoo  are actively involved in research and making significant investments in Machine Learning, there is a strong feeling that its future is as bright as never before.
Nowadays, the most promising and exciting aspect of research in ML is Deep Learning (DL). It deals with learning higher-level concepts at several levels of representation. DL is helping researchers make new discoveries in speech recognition and computer vision. Apple’s Siri virtual personal assistant as well as Google Street View are interesting illustrations of this new and promising technique. DL is well on its way of outperforming humans in areas like patterns recognition.
. Kodratoff, Yves et al. (1990 ), Yves et al. Machine Learning: An Artificial Intelligence Approach, (Vol. 3).
 Hastie, Trevor; Tibshirani , R.; Friedman, J. (February 2009). The Elements of Statistical Learning: Data Mining, Inference, and Prediction.
URL = .http://www-stat.stanford.edu/~tibs/ElemStatLearn/
. Christopher M. Bishop (2006). Pattern Recognition and Machine Learning, Springer ISBN 0-387-31073-8.
. Wernick, Yang et al. (July 2010). Machine Learning in Medical Imaging, IEEE Signal Processing Magazine, vol. 27, no. 4, pp. 25-38.
 Yahoo Research (Machine learning group). Current research on Machine Learning Applications at Yahoo. URL = http://research.yahoo.com/Machine_Learning.
 Pieter Abbeel (October 2012). Machine learning for Robotics, a video lecture,
URL = http://videolectures.net/ecmlpkdd2012_abbeel_learning_robotics/
 Klingspor V.; Demiris, J. ( ). Human-Robot communication and Machine Learning,
Applied Artificial Intelligence Journal, Vol. 11, pp. 719-746, 1997
URL = http://www.iis.ee.ic.ac.uk/yiannis/KlingsporDemirisKaiser97.pdf
 Andrew Ng (2013). Machine Learning: Coursera open access online lecture.
URL = https://www.coursera.org/course/ml