One rule. One source. One answer.
The task was to extract supported claims from two selected Apple annual-report paragraphs. This example concerns a possible increase in the risk of intellectual-property infringement allegations.
The task brief and evidence checks can be used with other language models. Qwen produced the saved response below; it is the source of this example, not a requirement for the activity. Results can differ by model and setup.
Preserve actor, scope, qualifications and modality.
Here, modality means whether something is possible, certain, required or conditional.
What the filing says
Apple FY2025, Item 1A, selected passage P02. Exact sentence.
This risk may be exacerbated by the use of new and emerging technologies, including machine learning and artificial intelligence, which can involve, among other things, the acquisition and use of copyrighted materials for training as well as the potential reproduction of copyrighted materials in their outputs.
What the model says
First claim from the actual saved local response, October 15, 2026.
Apple discloses that the use of machine learning and artificial intelligence, including the training on copyrighted materials, exacerbates the risk of infringement allegations for its products and services.
The model also labeled this claim conditional_risk and attached the exact source sentence shown here.
Your turn
- Find the wording that changes the strength of the claim.
- Write one sentence that preserves the source’s uncertainty.
- Define a check that would catch this problem before someone uses the answer.
Make a note of your answer before opening the review. There is no form to submit.
Compare your reasoning with our review
The source says “may be exacerbated”; the answer says “exacerbates.” A possibility has become a definite relationship. The conditional_risk label and an exact quotation do not repair the claim itself.
One possible rewrite: Apple discloses that machine learning and AI may exacerbate the risk of infringement allegations, including through the use of copyrighted training materials and their potential reproduction in outputs.
Acceptance check: Compare each claim with its cited sentence. Does it preserve who or what the statement concerns, how broadly it applies, and its degree of certainty? Flag a definite claim when its source only supports a possibility. A keyword search can flag candidates, but the presence of “may” alone does not establish that the meaning is preserved.
The original prompt already included the rule. This is a failure to follow an instruction, not evidence that the instruction was missing. The suggested rewrite is our editorial example, not a new model response.
Make the rule easier to check.
Prompt engineering includes specifying observable requirements and testing whether the answer meets them. Try making the existing rule more concrete:
For each claim, identify any uncertainty or condition in its supporting sentence. Preserve that meaning in the claim. If the source does not establish that an event occurred, do not describe it as an observed event.
We tested this addition in the small comparison below. When you revise a prompt, keep the original answer, change one requirement, and compare the new answer against the same source and acceptance check. Check fresh examples before treating an improvement as reliable.
The existing Python checker can verify structure and exact quotations; deciding whether a claim overstates its source requires a separate meaning check. Explore the saved example and Python checks →
Would worked examples help?
The original prompt uses zero-shot prompting: instructions and source material, with no worked demonstrations of the desired answer. Few-shot prompting adds a small number of source-and-answer examples. The two Apple passages are inputs to analyze; they are not demonstrations.
We generated three fresh answers with the same local model: the original prompt, the clearer instruction above, and that clearer instruction plus two fictional shipping and billing examples.
All three passed the Python checks and preserved “may.” Each still needed a different correction. The answer with two examples expressed the AI-risk finding more precisely, but omitted another important qualification. There was no clear overall winner.
See what changed—and what still needed review
| Prompt | What still needed correction |
|---|---|
| Original zero-shot | The comparison extended absence in the selected 2024 paragraph into an absence claim about the whole filing. |
| Clearer zero-shot | The AI finding preserved possibility but blurred the risk of being alleged to infringe with infringement itself. |
| Clearer + two examples | The AI finding preserved the allegation distinction. Its licensing comparison, however, omitted the 2024 statement that obtaining licenses was not assured. |
These are AI-assisted review judgments, not automated verdicts. Inspect the sources and answers yourself. The few-shot comparison quoted accurately; its problem was an incomplete contrast, not a fabricated quotation.
The original prompt already retained “may” in this new run. We cannot credit the rule or examples with fixing the earlier response at the top of this page. That earlier run used different sampling settings: temperature 1 and top-p 0.95; all three new requests used temperature 0 and top-p 1.
This is one familiar task with one response per condition. The examples were designed around known failure modes. They add prompt tokens, and these results do not establish general accuracy, speed or cost advantages.
Inspect the prompts, answers and review (JSON) ↓
Your next check: when a revision improves one detail, inspect the rest of the answer for omissions and new errors.
Take this into your own work.
Use the brief below for a research summary, customer-support analysis, document comparison or another task. Fill in the requirements before choosing a model. Copy the text or download it as a plain-text file.
TASK
What should the model do, and who will use the result?
[Your task]
ALLOWED INFORMATION
What sources may it use? What is outside the scope?
[Sources and scope]
OUTPUT
What fields, format and level of detail do you need?
[Required output]
EVIDENCE
Which claims need support? How should the source be identified?
[Evidence requirements]
MISSING INFORMATION AND STOPPING
When should it ask for clarification, mark something unknown or stop?
If tools are allowed, what actions and limits apply?
[Rules and limits]
FAILURE CHECKS
What would make the answer unacceptable? Which checks can code perform,
and which require someone to compare the answer with the source?
[Acceptance checks]
REVISION RECORD
Save the original instruction and answer. Change one requirement, test
on the same inputs, and inspect any new errors. Before claiming broader
improvement, check fresh examples too. Record calls, tokens, elapsed time
and review effort if you want to assess resource savings.
Clearer instructions can reduce avoidable rework, but savings need measurement. This exercise does not establish lower cost, faster completion or better performance across tasks.
Trace the example.
The opening activity reuses the first finding from our October 15 saved local Qwen extraction. Its full prompt, source records and original response are preserved in the existing example packet as prompt.txt, source.json and qwen/response.json. The October 17 three-prompt comparison has a separate JSON download above, containing the exact prompts, sources, synthetic demonstrations, settings, unchanged answers and review notes.
Read the complete selected paragraphs →
Inspect the source and execution notes →
Download the saved example packet ↓