What AI can and cannot do

A language model can explain a service document, turn notes into a readable summary and propose a command to investigate a failure. Those are different kinds of work. Explaining supplied text requires a useful interpretation; claiming that a service has recovered requires current evidence about the service. A well-written answer can do the first while leaving the second unresolved.

This lesson gives you a way to tell those jobs apart. You will follow a small document assistant from its reference material to its answer, then check which conclusions the material actually supports. You need no programming tools or paid model account to complete the exercise.

The model is one part of the application

Artificial intelligence is a broad field concerned with building systems that perform tasks such as recognizing patterns, making predictions and generating language. Machine learning is one approach: the system learns patterns from examples rather than relying entirely on rules someone wrote by hand. A trained model is the resulting computational system used to process new inputs.

A language model works with language represented as tokens, which can be words or pieces of words. In a common text-generation setup, the model computes possible next tokens from the information available to it, and a decoding procedure selects what to emit. Repeating that process produces the answer you read. Training makes those computations useful for tasks such as explanation, translation and code generation; it does not make every generated statement correct.

Language generation is one use of AI. A different model might classify a request, estimate a value or flag an unusual measurement. Those outputs need checks suited to their task too; fluent prose is simply the form of output this course will start with.

The application around the model decides what information to send, whether to retrieve documents and which tools are available. A chat interface might supply only your message and conversation history. A document assistant can also retrieve passages before asking the model to answer. An agent application can continue through several steps, receiving model requests for tools, executing permitted operations and returning the results for another response.

That distinction matters when something goes wrong. A relevant passage that never reached the model is a retrieval problem. A claim that goes beyond the supplied passage is an answer-quality problem. A proposed operation that ran against the wrong service is an execution problem. Changing the model alone does not address every one of those failures.

Follow one answer back to its source

For this constructed exercise, imagine a document assistant answering questions about a fictional service named Orchard. Its entire reference document is below. The example is teaching material, not a record of a real incident.

Orchard service note

The /health endpoint confirms that the web process responds.
It does not test the background worker.
The worker sends queued receipts to the configured mail service.
A worker restart does not clear the receipt queue.
No current queue depth or delivery result is included in this note.

A user asks, “The health endpoint responds after a worker restart. Have the queued receipts been delivered?”

A useful answer would explain that the document cannot establish delivery. The health check tests the web process, while the question concerns the worker and completed receipt delivery. The note also says that restarting the worker preserves the queue. To answer the live question, the application would need additional evidence, such as a current queue measurement and a check that receipts are actually being delivered.

Now compare these possible responses:

Response	What the reference supports
“The web process responds to its health check.”	The note defines that check, and the question reports that it responded.
“The worker restart removed all queued receipts.”	The note explicitly says the queue is preserved.
“The receipts have been delivered.”	The note contains no delivery result.
“Delivery remains unverified; check the worker’s results and queue.”	This follows from the gap between the check and the work being asked about.

An invented count or unsupported delivery claim is the kind of error often called a hallucination. The term describes the generated result; it does not explain why this particular error happened.

The last response is useful because it identifies the missing information. It does not manufacture a successful outcome to finish the conversation. Equally, “delivery is unverified” does not mean that delivery failed. The evidence supports neither conclusion yet.

An answer and an action leave different evidence

Suppose the assistant replies, “I restarted the worker.” That sentence alone cannot establish that a restart happened. In a tool-enabled application, the model may request a restart, but surrounding software must validate the request, enforce access rules and perform the operation. The operation returns a result that the application can inspect.

Even a successful restart result answers only part of the operational question. It can establish that the restart operation completed according to the tool’s contract; it does not establish that every receipt was delivered. A later check must examine the behavior the user cares about.

For this first exercise, give the assistant no operational access. The series begins with reading synthetic documents, then adds retrieval, evaluation and a harmless tool stub later. This makes it possible to understand an incorrect answer without also changing a running system.

Try the document question yourself

First, answer these questions from the note without using a model:

1. What does the health endpoint test?
2. Does restarting the worker empty the queue?
3. How many receipts are queued right now?

The first answer concerns the web process. The second is no. The third is unknown because no current count is supplied. For each answer, underline the sentence that supports it, or identify the missing information.

If you already have access to an approved chat application, you can then paste the synthetic note with this prompt:

Use only the Orchard service note below.
Answer each question and identify the supporting sentence.
If the note does not establish an answer, say what is missing.

Questions:
1. What does /health test?
2. Does a worker restart empty the receipt queue?
3. How many receipts are queued right now?

[Paste the Orchard service note here.]

Use the fictional note as written. Do not substitute customer records, internal incident logs, credentials or private service documents. A hosted application receives the material you submit, and its handling depends on the product, account and settings. You do not need sensitive material to learn this distinction.

Compare the response with your own answers. Correct wording may vary; the important result is whether the assistant preserves the scope of the health check, the queue behavior and the missing count. If it invents a number, save that response as a failed example. If it answers all three correctly, you have checked one small case, not established general reliability.

What you carry into the next lesson

You now have a reference note, three questions and expected answers that can be checked independently of a model. Keep them together as the first test case for your document assistant. In the next lesson, you will send a model request and inspect what the application actually returns, using these same boundaries to judge the result.

Sources and example scope

The conceptual definitions are supported by Google’s introduction to machine learning. Token prediction and decoding are explained in Google’s language-model introduction. The text-generation mechanism is developed further in the site’s model inference guide. The division between a generated tool call and application execution is documented in OpenAI’s function-calling guide. The distinction between generated statements and verified facts is also covered in NIST’s generative AI risk profile. The Orchard note, candidate answers and prompt are original constructed teaching examples. No model was called or production system tested while preparing this lesson.