Posts

Posted on

Programming Artificial Intelligence 101

Programming Language

Python logoThough several programming languages can be used to develop Artificial Intelligence applications, the language that is most often used and advised to use is Python, because it is a general purpose language designed for readability.

Learning Python

Documentation and resources can be found to use and learn the language on the official Python website, especially with the complete tutorial available for each version of Python.

If you already have programming experience, you can have a brief overview of the syntax and specificity of Python in this video. This great course on problem-solving with algorithms and data structures will give you a complete, step-by-step approach to learning Python with explanations, videos and exercises, while this other tutorial from Stanford quickly goes over Python basics but dives deeper into using the Numpy, SciPy and Matplotlib modules for scientific computing.

Other languages that are often used include: Java, Lisp, Prolog, C++, R…

Python Programming Tools

Here is a series of programming tools and other systems to assist in the programming of AI applications.

  • PyCharm: an Integrated Development Environment – IDE – used in computer programming, specifically for the Python language.
  • Jupyter: the Jupyter Notebook is an open-source web application to create and share documents with live code, equations, visualizations.
  • NumPy is the fundamental package for scientific computing with Python.
  • TensorFlow: an open-source machine learning framework originally developed by Google Brain researchers and engineers.
  • Anaconda: Anaconda is a platform for Python and R data science and machine learning on Linux, Windows, and Mac OS X.
  • TFLearn: a deep learning library featuring a higher-level API for TensorFlow
  • Scikit-Learn: an open-source machine learning library for Python
  • Pandas: open-source data structure and analysis library for Python
  • Caffe: a deep learning framework made with expression, speed, and modularity in mind
  • Keras: a high-level neural networks API, written in Python
  • Pytorch: an open source machine learning library for Python, used for applications such as natural language processing.

Data

List of datasets used for Machine Learning research – Wikipedia

Cloud services providers

Certain AI applications may require large computing power to process important amounts of data. Fortunately, several companies provide such services on a regular or on-demand basis. Here are the most famous cloud computing services.

Robotics

ROS – the Construct: Robots and environment simulator to train robots

Posted on

23. Model Merging, Cross-Modal Coupling, Course Summary

Bayesian Story Merging

By using the probability model discovery previously studied, certain concepts and ideas can be analyzed and merged if similar.

Cross-Modal Coupling

By analyzing the correspondences between clusters of two sets of data, certain data subsets regular correspondences can be sorted out. According to Prof. Patrick Winston, this system of correspondences is very likely to be present in human intelligence.

Applications of AI

  • Scientific approach: understanding how AI works
  • Engineering approach: implementing AI applications

The most interesting applications are not to replace people with artificial intelligence but to work in tandem with people.

Using a lot of computing power and data becomes more common, but an interesting question is how little information is needed to work a certain problem.

Genesis system

The system translates stories into a internal language to understand stories and display them in diagrams. It allows to read stories on different levels, and use different “personas” to understand stories differently.

Humans may not be intelligent enough to build a machine that is as intelligent as them.

Posted on

22. Probabilistic Inference II

Beliefs nets

Continued from previous class

Events diagrams must always be arranged in a way so that there are final nodes and no loops. Recording probabilities in tables for each event, the tables are filled by repeating experience so as to know the probabilities and occurrences of each event.

Bayesian inference

Several models can be drawn for a given set of events. To know which model is right, the Bayesian probabilities formulas can be used to confirm if events are independent or not, make them easier to compute, and choose the more appropriate model.

  • P(a/b) = P(a,b) / P(b)
  • P(a/b) P(b) = P(a,b) = P(b/a) P(a)
  • P(a/b) = P(b/a) P(a) / P(b)

Defining a as a class, and b as the evidence, the probability of the evidence given the class can be obtained through these formulas.

P(class/evidence) = P(evidence/class) P(class) / P(evidence)

Using the evidence from experience, classes can inferred by analyzing the results and corresponding probabilities.

Structure discovery

Given the data from experience / simulation, the right model can be sorted as it better corresponds to the probabilities. This allows to select between 2 existing models.

However if multiple models can be created, volumes of data make it impossible to compare them all. The solution is to use two models and compare them recursively. At each trial, the losing model is modified for improvements until a model fits certain criteria for success.

A trick is to use the sum of the logarithms rather than the probabilities, as large numbers of trials will make numbers too small to compute properly.

To avoid local maxima, a radical rearrangement of structure is launched after a certain number of trials.

Applications

This Bayesian structure discovery works quite well in situations when a diagnosis must be completed: medical diagnosis, lie-detector, symptoms of aircraft or program not working…

Posted on

21. Probabilistic Inference I

Probabilities in Artificial Intelligence

With a joint probability table, recording the tally of crossed events occurrence will allow us to measure the probabilities of each event happening, conditional or unconditional probabilities, independence of events, etc.

The problem with such table is that as the number of variables increase, the number of rows in the table grows exponentially.

Reminders of probabilities formulas

Basic axioms of probability

  • 0 ≤ P(a) ≤  1
  • P(True) = 1 ; P(False) = 0
  • P(a+b) = P(a) + P(b) – P(a,b)

Basic definitions of probability

  • P(a/b) = P(a,b) / P(b)
  • P(a,b) = P(a/b) P(b)
  • P(a/b,c) = P(a/b,c) P(b,c) = P(a/b,c)P(b/c)P(c)

Chain rule of probability

By generalizing the previous formula, we obtain the following chain rule:

Chain rule of probability

Independence

Independent events

P(a/b) = P(a) if a and b are independent

Conditional independence

If a and b are independent

  • P(a/b+z) = P(a/z)
  • P(a+b/z) = P(a/z)P(b/z)

Belief nets

Causal relations between events can be represented in nets. These models highlight that any event is only dependent from its parents and descendants. Recording the probabilities at each node, the number of table and rows is significantly smaller than a general table of all events tallies.