Grunt Mode and Local Grunt Adapter: Token Efficiency and ROI

The useful measure of AI efficiency is how much cloud usage, local compute and review a completed task requires. Grunt Mode and Local Grunt Adapter are our open-source implementations of a practical approach: delegate suitable source processing to local models, return compact results, and keep the calling assistant responsible for judgment.

We use both in our own workflow. The opportunity is to reduce repeated source ingestion by the cloud model while maintaining an acceptable result. The tradeoff is the additional coordination, returned output and review that delegation introduces.

Token efficiency starts with the full cloud workflow

For a file summary or extraction, the direct path sends the source into the cloud model. The delegated path can keep that source local: the caller sends instructions and file references, and receives a smaller result. That can reduce cloud context, especially when the source is substantial and the answer is compact.

Count every attempt needed to reach an accepted result. A local answer that requires repeated cloud correction can consume more usage than a direct run. Likewise, an input the cloud caller already has in context may provide little avoided-source benefit. Provider prompt caching and differently priced input/output tokens also affect the cost comparison.

Model the break-even point before calling it ROI

The calculator below separates cloud-token accounting from operating cost. Change the source size and the handoff assumptions, then include local hardware, electricity, operations and extra human review. Every default is an illustrative assumption, not a Grunt benchmark or a provider price quote.

In the starting example, delegation uses fewer cloud tokens but the modeled monthly benefit is negative after local and review costs. Increasing the amount of suitable work, reducing unnecessary handoffs, or improving first-pass acceptance changes that result. The number to optimize is cost per accepted task at the required quality.

The adapter records local usage and reviewed outcomes separately. Its local_cloud_usage tool accepts actual paired cloud totals only when the caller has both runs. Modeled estimates remain separate from measured comparisons. Grunt Mode also reports exploratory per-caller usage estimates; those estimates do not deduct every orchestration cost. The public usage documentation explains the accounting limits.

Data flow: what moves between the cloud and your computer

A capable assistant can read a file, extract fields, summarize a log and make a decision about the result. But those are different jobs. A focused extraction may be a useful assignment for a smaller local model, while deciding whether the output is trustworthy still needs the caller's judgment.

We wanted to make that handoff explicit. Give the worker a bounded input, ask for a compact result, and record whether the result helped. That is the implementation side of the question we explored in our cloud versus local AI cost analysis: what does it take to get useful work done, including review?

We ended up with two independent tools. They can be used separately, and neither requires our Crest desktop app.

Local Grunt Adapter connects the assistant to the worker

Local Grunt Adapter is a Python MCP adapter for models running through Ollama. A locally installed MCP client can ask it to summarize, extract, classify, rewrite or draft a focused code change. The worker cannot execute generated commands or edit source files.

For file-based work, the caller supplies a project folder and named files. The adapter reads those files locally and returns the result for review. An optional native Mac panel shows actual local activity, model allocation and usage metadata; source text and generated answers stay out of the panel.

The public repository includes a deliberately small, fictional example:

Customer: River Labs
Seats: 12
Status: trial

Ask for the customer and seat count, with a schema requiring those two fields, and the expected result is:

{"customer": "River Labs", "seats": 12}

Our live smoke test exercised that extraction through the real MCP connection and local model. It also checked caching, artifact output and caller feedback. A test passing on this example does not establish accuracy on your own records, but it gives you a reproducible starting point.

After a task, the caller records whether the result was accepted, rejected or redone. Completion alone does not count as success.

The adapter normally keeps short tasks with the caller. Choosing a tool, assembling arguments, waiting and checking an answer all have overhead. A tiny task can take more effort to delegate than to finish directly. The routing thresholds are conservative heuristics, and they can return keep_with_caller without loading a model.

Grunt Mode adds reviewed quality gates

Grunt Mode is a TypeScript task gateway with CLI, HTTP and a minimal legacy MCP interface. It uses a separate local llama.cpp-compatible backend.

Its central idea is shadow evaluation. A task category starts in shadow mode: the local model produces a candidate, the caller continues its normal work, and feedback records how the two compared. Promotion to local mode requires at least 50 shadow examples and 95% agreement. Promotion is explicit; local mode adds spot checks and can demote a category when reviewed agreement drops.

Those are the gates implemented in the code, not a claim that every task category has passed them. Agreement is evidence to inspect. It does not prove that a model is correct.

When Grunt returns use: big, it hands control back to the caller. It does not call a cloud model automatically. That division of responsibility matters in an agent workflow, especially when a local candidate fails a check.

The adapter and Grunt Mode solve related problems through different interfaces. The adapter makes bounded local work easy to call from existing assistants. Grunt Mode adds a per-category process for deciding when local results are reliable enough to use.

The privacy boundary includes the returned answer

Running inference locally does not automatically keep information out of a cloud assistant. If private text appears in tool arguments, it has already reached the caller. If a local model returns a summary containing the same information, that summary can reach the caller too.

For known private files, the adapter accepts sensitive=true. It processes the files locally and returns a reference to a private output artifact, without answer text or a preview. A person can inspect that artifact locally. Reading it back through a cloud-visible tool would defeat the boundary.

Heuristic screening can also trigger this handling, but detection can miss private information. Callers should mark known private inputs explicitly, use neutral filenames and instructions, and avoid sending source text in tool arguments.

Grunt Mode's confidential flag has a narrower purpose: it suppresses persisted content and bypasses result caches. Its responses remain visible to the caller. It is not equivalent to the adapter's private-output workflow.

These are workflow boundaries, not a system-wide prompt filter. Neither project intercepts unrelated app prompts or undoes data already sent elsewhere. Our local AI deployment guide covers the broader operational choices around running models yourself.

Preparing the release exposed problems we needed to fix

The most useful part of preparing an open-source release was finding assumptions that needed to become enforced behavior.

In the adapter, generated output could enter the cache before sensitivity screening. We moved screening ahead of cache writes and validation errors, tightened artifact creation permissions, and added regression coverage. The adapter security review records those changes and their limits.

In Grunt Mode, backend URL overrides needed loopback enforcement, the HTTP service needed bearer authentication, and confidential requests needed to bypass the exact cache as well as the semantic cache. We also stopped retaining free-text feedback reasons on confidential records. The Grunt security review documents that work.

For the October 27 release, 41 adapter tests and 83 Grunt tests passed. Both repositories passed GitHub-hosted CI. Four opt-in Grunt tests for live models and the Linux supervisor were skipped; that environment was not validated on this Mac.

We published clean source snapshots with MIT licenses, setup instructions and synthetic examples. Private task databases, configuration backups, previous task reports, model weights and private development history stayed out. Secret scans and dependency audits found no detected secrets or known dependency vulnerabilities in the checked release inputs. Those checks reduce risk; they do not certify safety.

How to evaluate this in your own workflow

Run a direct baseline and a delegated version on the same representative task set. Include all input, output, cached input, retries and review in each cloud total. Record acceptance and latency, then account for local operating cost and additional human effort. Keep model quality and the definition of an accepted answer comparable.

Subscription allowances, API invoices and modeled token equivalents are different measures. Use the provider's actual accounting when evaluating a plan limit or a bill. The calculator above is a planning aid; it does not establish measured savings for either project.

For teams, the same discipline applies to learning an AI workflow: define a repeatable task and the acceptance criteria, then decide which part belongs on a local worker. Both repositories include setup instructions and synthetic examples.