Measure AI cost per accepted draft
Calculate the cost of accepted AI work by including failed attempts, review time and repairs, using a transparent illustrative job ledger.
What belongs in the numerator?
Choose the decision before choosing the cost boundary. To compare operating workflows, include generation, failed attempts, review and necessary repairs for the same set of jobs. To study only provider usage, report that narrower number separately. Calling both “AI cost” hides the work that turns a response into an acceptable draft.
OpenAI’s evaluation guidance supports defining criteria and assessing responses against them. An acceptance count therefore needs an explicit rubric. The costs and labor rate in the example below are fictional USD inputs; they do not describe any provider’s current pricing or the author’s client work. [Working with evals]
A complete illustrative job ledger
Suppose a test batch contains 12 requested drafts. Generation across first attempts and retries costs $1.80. Review takes 60 minutes and accepted-output editing takes 30 minutes. The chosen accounting rate is $40 per hour. Eight drafts are accepted: five unchanged and three after repair. Four are rejected.
| Included item | Calculation | Cost |
|---|---|---|
| Generation including failed attempts | Confirmed batch usage | $1.80 |
| Review of all outputs | 1.0 hour × $40 | $40.00 |
| Required editing | 0.5 hour × $40 | $20.00 |
| Total operating cost | 1.80 + 40 + 20 | $61.80 |
| Cost per requested draft | 61.80 ÷ 12 | $5.15 |
| Cost per accepted draft | 61.80 ÷ 8 | $7.725; display $7.73 |
Generation cost per requested draft is only $1.80 ÷ 12 = $0.15. That is a valid narrow metric, but it does not describe the $7.73 cost of an accepted result under this boundary. Keep unrounded inputs in the ledger and round only the displayed ratio.
Do not assign zero cost to rejected work. Reviewers still spent time detecting the failure, and generation may still be billed. If a rejected draft triggers another attempt, attach that attempt to the same requested job so the denominator does not grow merely because the system retried.
What counts as an accepted draft?
- Accepted unchanged: meets the declared brief and evidence rules without substantive repair.
- Accepted after repair: usable only after recorded editing work.
- Rejected: cannot be used under the rubric or requires a new attempt.
- Pending: not reviewed yet; exclude from a completed-batch acceptance denominator and show it separately.
Schema validity is not editorial acceptance. OpenAI notes that structured outputs can still contain substantive mistakes. A well-shaped response with an invented price should fail the evidence rule even when its JSON parses correctly. Record the type of rejection so the workflow can be improved deliberately. [Structured model outputs]
Treat setup costs consistently. If you include prompt development or integration labor, allocate it over a stated period or job volume and show the allocation separately. A one-time implementation cost should not disappear from one candidate’s calculation while remaining in another’s.
Which improvement would change the cost?
Holding the illustrative $61.80 cost constant, accepting ten drafts instead of eight would reduce cost per accepted draft to $6.18. That is a sensitivity calculation, not a forecast. Achieving it may require more review, different prompts or a narrower task, each of which can change the numerator.
Report quality, time and cost together. A cheaper accepted draft may still be unsuitable if it misses a delivery deadline or passes only a weak rubric. The ledger supports a bounded operating decision; it does not establish that AI is universally cheaper than another production method.
Sources
AI-assisted writing and an AI-generated editorial cover. Source links and checked dates are listed below; illustrative examples are labeled in the text.
- Working with evals — checked 2026-10-18
- Structured model outputs — checked 2026-10-18