#Cutting agent cost with a skill

Your agent spends most of its tokens working things out. It reads help text, guesses a flag, reads the error, and tries again. Each attempt is another turn, and each turn sends the whole conversation back to the model. When the knowledge the agent needs isn't written down where it can read it, the agent pays to discover it again on every task.

A skill that states the exact steps removes those decisions. The agent doesn't save tokens by reading less. It saves them because it has nothing left to work out. With the guesswork gone, a cheaper model can often complete tasks that otherwise needed a frontier model.

This tutorial shows how to measure that with an evaluation: run your tasks without a skill, find where the cost goes, write the skill, and compare the cost per task on several models.

#The core workflow

1. Pick the tasks you want to make cheaper and write a rubric for each
2. Run them without the skill on the models you are considering
3. Find the turns the agent spends working things out
4. Write the skill so it answers those questions up front
5. Run each model with and without the skill in one evaluation
6. Choose the cheapest configuration that meets your quality bar

#Prerequisites

• Tessl installed and authenticated (tessl login). See Installation.
• Publisher access or higher in the workspace. Member access can't generate scenarios or run evaluations.
• A Tessl project linked to the directory you run the commands from. If you create or link the project in a non-interactive shell and belong to more than one organization, pass --org <name> or set TESSL_ORG. See Prove a skill works using evaluation.
• A plugin that contains only the skill you want to optimize. The comparison removes the whole plugin, so other skills in it would count toward the result. See Create a plugin if you have a standalone SKILL.md.
• Credits for the runs. Before each run starts, the CLI shows its estimated credit cost. See pricing for current rates.

#Step 1: Pick the tasks and write a rubric

Choose 3 to 5 tasks that your agents run often and that cost the most: long sessions, many retries, or work you send to your most expensive model. Cost savings repeat on every run, so frequent tasks matter most.

Each task is a scenario with a task.md (the request, as a user would type it) and a criteria.json (the rubric a second model uses to score the result). You can generate starting scenarios from the skill, or from real commits and pull requests in your repository:

tessl scenario generate ./my-plugin --count 5 --wait
tessl scenario download <generation-id> --output ./my-plugin/evals

--wait keeps the generate command running until generation ends or a wait limit is reached. Without it, a non-interactive shell such as CI returns at once. When generation finishes, the command prints its generation id; download that id. If the command reports that it timed out, run tessl scenario view <generation-id> and download only after it shows the generation is complete.

Then edit the rubrics so they check what a good result needs. A cheaper configuration is only cheaper if it still passes, so the rubric must catch the shortcuts a model takes when it doesn't know the right way: data changed by hand, a required step skipped, an error worked around instead of fixed. See Prove a skill works using evaluation for the file formats.

#Step 2: Measure the cost without the skill

Define one test variant per model, each without the skill. Setting the context fixture to null removes the plugin you are evaluating. It removes the whole plugin, not one skill, which is why the plugin should contain only the skill you are optimizing:

[
  { "label": "frontier-no-skill", "model": "<frontier-model>", "fixtures": { "context": null } },
  { "label": "mid-tier-no-skill", "model": "<mid-tier-model>", "fixtures": { "context": null } },
  { "label": "small-no-skill", "model": "<small-model>", "fixtures": { "context": null } }
]

tessl eval run --list-agents lists the agents and models available in your region. Save the file as no-skill.json and run each task once per model:

tessl eval run ./my-plugin --variant-json ./no-skill.json -n 1 --label cost-no-skill

This run is for finding where the cost goes, so one repeat is enough. With 5 tasks it makes 15 attempts: 3 variants, 5 tasks, 1 repeat.

#Step 3: Find where the cost goes

When the run finishes, view it:

tessl eval view <run-id>

The variant table shows score, cost per attempt, tokens, and turns for each model. Look for two signs:

• Many turns for a short task. The agent is exploring, guessing, or retrying.
• A low score on a cheaper model. The agent couldn't work out what to do, or it got past an error the wrong way.

Open the run in the web app from the link that tessl eval view prints. Select a task to see each attempt's score and the scorer's reason for every rubric item. Members of the workspace can download an attempt's solution to see what the agent did.

Write down each thing the agent had to work out. Typical examples:

• which command runs a step, and which flags it needs,
• what an error code means and how to fix it,
• which order the steps must run in,
• what format a file must have,
• what to do with input that doesn't fit, such as a row that fails validation.

Each item on that list is a question the agent answers at runtime, on every task. Those are the questions the skill must answer.

#Step 4: Write the skill so it answers those questions

Turn each item into an exact instruction. A skill written to cut cost reads like a runbook. It doesn't explain the system; it tells the agent what to do:

## Every change has the same steps

1. `./ops lock release <namespace> --reason "<why>"` for every namespace the change touches.
2. `./ops plan <operation> ...`. It prints `plan <id> sha <sha>`.
3. `./ops apply <id> --confirm <sha>`, with the id and sha from step 2.
4. `./ops lock set <namespace>` for every namespace from step 1.

## Errors

- `E_LOCKED <namespace>`: release the lock (step 1).
- `E_STALE <id>`: another change was applied after this plan. Plan again.

When you write it:

• Write the exact command, with every flag the task needs. A flag the agent has to discover costs a turn.
• Name what goes wrong without it. For example: "Without --keep-redirects every existing link stops working." The agent won't find out that it missed a flag that doesn't cause an error.
• Map each error to its fix, so the agent doesn't stop to investigate.
• Say what to do with data that doesn't fit, so the agent doesn't invent its own rule.
• Write the description so the skill loads for the requests that need it. A skill the agent doesn't load saves nothing. See Test whether a skill activates.

Remove ambiguity before you remove words. A skill that leaves the agent to work out one step still costs the turns that step takes.

#Step 5: Compare with and without the skill

Define two variants per model: one with the skill and one without. A variant that leaves out fixtures keeps the plugin you are evaluating:

[
  { "label": "frontier-skill", "model": "<frontier-model>" },
  { "label": "frontier-no-skill", "model": "<frontier-model>", "fixtures": { "context": null } },
  { "label": "mid-tier-skill", "model": "<mid-tier-model>" },
  { "label": "mid-tier-no-skill", "model": "<mid-tier-model>", "fixtures": { "context": null } },
  { "label": "small-skill", "model": "<small-model>" },
  { "label": "small-no-skill", "model": "<small-model>", "fixtures": { "context": null } }
]

Run all the variants in one evaluation, so they are compared under the same conditions. Results move from day to day, so don't compare a variant from one run with a variant from another. Use three repeats:

tessl eval run ./my-plugin --variant-json ./compare.json -n 3 --label cost-compare

With 5 tasks this makes 90 attempts: 6 variants, 5 tasks, 3 repeats. Check the estimated credit cost the CLI prints before you confirm. For a cheaper first look, use -n 1, then repeat with -n 3 before you share a result.

#Step 6: Choose a configuration

Before you look at the results, decide which score counts as a pass for one attempt, for example 100% or 80%. Use the same threshold for every variant.

View the run with tessl eval view <run-id>. For each model, compare the variant with the skill to the variant without it:

• Cost per attempt shows what each task costs. It covers solving the task, not scoring it. Some agents don't record cost. An empty cell means no cost was recorded, not that the task was free.
• Score shows whether the cheaper configuration still does the job.
• Turns and tokens show where the saving comes from.
• Cost per passing attempt is the number to decide on when some attempts fail. Divide the total cost of a variant's attempts by the number of attempts whose score meets your pass threshold. Calculate it only for a variant where every attempt has both a recorded cost and a score. If any attempt is missing either, the result is not reliable, so don't use it to compare that variant. A model that is cheap per attempt but fails often can cost more per task that you can use.

Choose the cheapest configuration whose attempts pass often enough for your use. Often that is a smaller model with the skill.

Results vary from run to run. Before you change a production setup, repeat the comparison in a second run and check that the difference holds. The web app marks differences that are smaller than the spread between repeats in this run. That spread comes from one run, so it doesn't show that the result repeats. Only a second run shows that. See Read an evaluation's results.

#Keep it current

Keep the scenarios in the plugin's evals/ directory and run the comparison again:

• when you change the skill,
• when the tool or process it describes changes,
• when a new model comes out that you might switch to.

A change to the tool that the skill doesn't cover brings the guesswork back, and the cost with it.

#Next steps

• Compare with test variants covers comparing models, agents, and skill versions.
• Read an evaluation's results explains cost, tokens, turns, and variation within and between runs.
• Improving a skill covers reviews and evaluations for skill quality.