AI history: from writing the rules to learning the task

Put these two imaginary help requests beside each other: “I cannot sign in” and “my login keeps failing.” A person reading the queue can recognize the resemblance. Getting a computer to make that connection has meant very different kinds of work at different points in AI’s history.

You could write a rule for the two phrases. You could train a classifier on labeled requests. You could give a language model instructions in ordinary language. All three remain available. The history is less a procession of obsolete machines than a changing answer to one question: how much of the task must we describe ourselves, and how much can a system learn?

First, describe what the machine should do

In 1956, researchers gathered for the Dartmouth summer research project. It helped establish artificial intelligence as a named field, with ambitions that included language, abstraction and learning. The questions and mathematical foundations were older than the gathering. What acquired a name was a shared research ambition, not a finished machine.

One route was to give programs rules and facts they could use. A program might search possible moves in a game or apply known facts within a restricted domain. For our imaginary help queue, someone could explicitly connect “cannot sign in” and “login keeps failing” to the same category.

That is a perfectly useful arrangement until the description becomes hard to keep up with. People find new ways to phrase old problems. Facts change. Each exception asks someone to decide how it belongs in the rules. The computer can be consistent about a description that has quietly stopped describing the world very well.

Learning offered another route, and it was a thread of AI research rather than an invention that waited for rules to disappear. Instead of supplying every connection directly, engineers could give a system examples and adjust its numerical parameters during training. They still chose how to represent the input, what error to reduce and how training would proceed. The system found parameter values that reduced that chosen error. The weights and training guide follows one small update closely.

The pictures mattered. So did the machinery.

Recognizing an object in a picture makes the attraction of learning easy to see. Writing a rule for every image would leave an engineer with rather a lot of images still to explain. Deep learning moved more of that recognition into layers trained on examples, where simple visual patterns could feed later computations that distinguish an object.

The 2012 AlexNet paper brought several ingredients together: a large convolutional neural network, which is a layered model suited to processing images; the labeled ImageNet dataset; and GPU computation, with techniques that made training practical. The result concerned image classification on that dataset. It was a substantial achievement with a specific task, not a demonstration that the system understood every picture.

The combination is the revealing part. A learning method needs examples to learn from and enough computation to make training possible. When those surrounding ingredients improve, an idea can move from something conceivable to something useful. The engineer has less image-by-image description to write, but still has to choose the task and training process.

A sentence can supply more than words

Read this invented message: “My account was working yesterday, but now it is locked.” You connect “it” with “account” across the words in between. A model also needs a way to use relationships between separated parts of the text when building its representation of what it receives.

The 2017 Transformer paper proposed a sequence architecture built around attention. Attention combines information from other positions when forming the representation of a token, a unit of text the model handles. That gives the model a way to relate separated words rather than treat each in isolation. Translation was a central task in the paper; the later chat assistant on your screen was not already hiding among its demonstrated results.

Subsequent language-model work paired these architectures with large-scale pretraining. In the 2020 GPT-3 paper, researchers examined tasks specified by instructions and examples in the input, without updating the model’s weights for each task. The instructions could change what the model was asked to do without another round of task-specific training.

That distinction is easy to lose when both processes are called “learning.” An example in a prompt changes the information available for the next prediction. Training changes the learned parameters. For someone building an assistant, those are different ways to shape its behavior, with different work behind them.

An assistant can also surround the language model with retrieved documents and callable tools. The conversation may look seamless, but finding the procedure, writing a plausible answer and completing an operation remain separate achievements. Language made the interface more flexible; it did not make those differences disappear.

More work on this particular answer

There is another place to spend computation: after a question arrives. Before that, training can give a model feedback on its attempts, using rewards to favor behavior that works better. That general approach is called reinforcement learning. OpenAI’s September 2024 o1 announcement described using it to improve multi-step reasoning, along with additional computation spent working through a problem before presenting the result. OpenAI reported improvements from both more training and more work at response time.

Those are two different investments. Training changes the learned system you start with. Response-time computation lets that system do more work on the problem now in front of it. A chain of dependent deductions may benefit from the extra effort. A straightforward classification may not earn back its additional delay and cost.

Return to our pair of login requests. If a simple classifier can send both to the right queue, it may not need the same treatment as a problem whose answer depends on several deductions. Reasoning models added a choice; they did not remove the need to choose.

The queue is still waiting for useful help

For a hypothetical support team, a stable routing rule may be easiest to maintain as ordinary code. Varied phrasing may make a learned classifier more useful. A language model offers more flexible interpretation, with more evaluation work to establish that the flexibility helps. The relevant questions are how much the requests vary and what happens when one goes to the wrong place.

A general benchmark can suggest which systems are worth trying. It cannot tell the team what happens to its own ambiguous requests. For that, compare candidates on the actual queue, including the awkward cases. Behind a wrong route is someone who still cannot sign in, now waiting for the wrong kind of help.

“I cannot sign in” has not become a different problem just because the available software has changed. If maintained rules handle the queue well, a newer system has something concrete to improve on. If the wording defeats them, learning gives you another option; if interpretation needs to be more flexible, a language model gives you another. The useful advance is the one that gets these requests to help, at a cost of checking and correcting mistakes the team can accept.