Patterns, not rules

Start with a decision you can follow

You will learn the difference between a rule-based program and a learned model. Both are software. The distinction is where their decision-making behaviour comes from.

Suppose a club gives a discount to members. A person can write: “If the customer is a member, charge the member price; otherwise charge the regular price.” The computer checks membership and follows the instruction. It does not need a pile of past purchases to discover the rule.

A program is a set of instructions. A rule is a condition connected to an action. A rule-based program can be useful, complicated, and carefully designed. Learning from examples is not automatically a better solution.

Now recognise a handwritten digit

Imagine trying to describe every possible handwritten 7 with rules. Some sevens have a crossbar; some lean; some have a curved top. “Has a horizontal line” also matches other shapes. It is difficult to write a short, dependable list of visual rules that handles this variation.

Machine learning offers another approach. Give a training program many examples of images and their correct digit labels. It finds useful patterns and produces a model, a system that can make a prediction for an input.

A label is the answer attached to a training example. For a picture of a seven, the image is the input and 7 is the label. The label is not printed into a new unlabeled image at prediction time; the model must infer it.

Follow the whole process

1. Collect examples. Obtain images from several writers, so the data contains more than one person's handwriting.
2. Label them. Check which digit each image is supposed to represent. An ambiguous mark may need review.
3. Create a model. Choose an approach and its initial settings. At this point a newly created model has not learned your dataset.
4. Train it. Run the learning procedure on the training examples.
5. Use it. Give the trained model a different image and request a prediction.
6. Check it. Compare predictions with known answers on examples that were kept out of training.

Steps 4 and 5 are different. A camera app can make thousands of predictions without retraining after every photograph. We will perform these steps ourselves later in the course.

What does the model contain?

That depends on the kind of model. Here are three kinds:

• Some keep a long list of numbers that were tuned during learning. A neural network is this kind.
• Some keep the training examples themselves. For a new input, they find the most similar stored examples and give the same answer those examples had.
• Some keep a chart of yes-or-no questions, such as “Is there a crossbar?”, and follow it one branch at a time until they reach an answer.

You will build the first two kinds yourself later in the course. The kinds also differ in how easy it is to see why a model decided something. You can read a chart of questions, or look at the stored examples an answer came from. With a long list of tuned numbers, explaining one particular decision can be much harder.

Do not memorise “all models are billions of mysterious numbers.” A small model can still be machine learning. What matters here is that its predictive behaviour is obtained from examples through a learning procedure.

Artificial intelligence, or AI, is the broader field of building systems that perform tasks associated with intelligence. Machine learning is one part of it. Generating text, recognising an image, and recommending an item are different tasks, even when all are advertised as AI.

Which approach fits?

For a membership discount, use the clear membership rule. For recognising varied handwriting, learning from labeled images is a practical option. A real product may combine both: a model predicts a digit, and a written rule asks the user to try again when the input is blank.

A learned pattern can also be the wrong pattern. If every training 7 is on blue paper, colour may become a shortcut. A good training score alone does not reveal that problem. This is why the later lessons insist on checking new examples.

Explain it back

A programmer choosing a discount threshold is writing a rule. A training procedure choosing a useful boundary from labeled examples is learning a model. In both cases people still choose the task, provide the data or rules, and decide how to test the result. Learning does not remove human responsibility.