LLM costs, context and evaluation
A cheaper model request is useful only if its result meets the task's requirements. Compare quality, latency and cost together using the same representative examples. Reasoning settings, repeated context and data formats are different changes: each needs its own measurement and a check that the output still works.
A reading guide by Idir OuhabMeasure the whole task
Set a baseline before changing the model or prompt. Define an acceptable answer, collect realistic inputs and record failures as well as successful runs. Include tool calls, retries and review work when comparing alternatives. A reduction in input tokens alone does not establish a lower total cost per completed task, and a longer reasoning trace does not establish a better answer.
Change one variable in a repeatable comparison
For a document-extraction workflow, keep a small set of representative documents and the expected fields. Run the current configuration, then compare one change: a reasoning level, a stable reusable prefix, or a different input representation. Check field correctness and missing information before comparing usage and elapsed time. Include unusual documents and failed requests. Keep the examples and model settings with the results so you can repeat the comparison after an update.
A suggested reading order
Follow the sequence or start with the question you need to resolve.
More Reasoning Isn’t Always Better
Begin with the argument for choosing reasoning effort for the task, rather than treating a higher setting as an automatic improvement.
Model-selection suggestions are starting points for evaluation, not a benchmark of your workload.
4 min read
Prompt Caching: A Technical Guide to LLM Efficiency
Next, look at repeated context and how a stable prefix can make reuse possible under a provider's caching rules.
Availability, retention and cache boundaries differ by model and API. Confirm a cache hit in usage data before assuming a saving.
3 min read
I Made a Mistake About TOON (And Here's the Data That Proved Me Wrong)
Read the TOON correction before the original proposal. It shows why a token-count result needs to be checked against representation, task and output quality.
This correction qualifies the earlier claim; it is the starting point for evaluating TOON here.
3 min read
TOON vs JSON: token savings and limits of the original recommendation
Read the original TOON article as the historical argument that prompted the correction, with that correction in view.
Historical context: its headline saving is not a general recommendation or a guarantee for current models and inputs.
3 min read
Companies have discovered that AI costs money
Finish with the ROI commentary to connect technical measurements to the evidence needed for an investment decision.
The article discusses reported survey results and uses satire. Those figures do not establish the return of an individual project.
5 min read
Before you decide
Use these questions to make the next decision explicit.
Define the result that counts
Choose representative tasks and acceptance criteria. Identify failures that make an answer unusable even when it is fluent.
Keep the comparison fair
Use the same inputs and checks. Change one important setting at a time, and record the model, prompt and tool versions.
Read actual usage
Inspect returned usage fields and the provider's current pricing and cache conditions. Include retries, tools and any review needed to finish the task.
Recheck when the system changes
Keep difficult examples in the evaluation set. Repeat the comparison after changing the model, context, workflow or acceptance criteria.
Scope and limits
This is a comparison framework, not a promise of savings or a current model-price table. The articles include technical explanation, corrections and opinion. Use the provider documentation and measurements from your own task to make an implementation decision.
Check the implementation details
Caching is provider-specific. Check the documentation for the API you actually call, including supported models, prefix matching, retention and usage reporting.
Turn a cost question into a testable decision
A team workshop can work through a concrete evaluation example. Advisory can examine a proposed comparison, the evidence it needs or a workflow's cost and quality trade-offs. The task, available data and scope determine the work.