Run an AI content production pilot with a representative scope, agreed acceptance criteria, revision and cost records, and a clear decision rule. Evaluate every requested deliverable. Use the editable pilot brief and scorecard to keep first-pass approval, approval at the reporting cutoff and unresolved work separate.
Choose a scope that exposes your real constraints.
For an illustrative software team, a pilot could include one short product demonstration, one presenter-led explanation, and one five-slide carousel. Use a current product reference pack and the language the team actually needs. This is an example scope, not a universal minimum or a promise about any subscription.
Include a difficult but routine requirement, such as a product name that needs pronunciation review or a real interface sequence that must remain accurate. A pilot made entirely of easy generic scenes may reveal little about the work that normally blocks publication.
Agree on acceptance before generation.
| Dimension | Observable acceptance criterion |
|---|---|
| Product accuracy | Claims and screens match the approved current references. |
| Voice and captions | Words are correct, understandable, and synchronized without covering essential visuals. |
| Edit completeness | No unintended gaps, clipped lines, missing scenes, or unresolved placeholders. |
| Brand fit | Uses the agreed visual and verbal references with identifiable deviations reviewed. |
| Delivery | Correct exports, slide order, captions, destinations, and version records are included. |
Use criteria tied to the brief. “Looks premium” may be a useful direction, but it is not specific enough to resolve a disagreement about an export. Identify who makes the final business decision.
Keep the complete production record.
Record when the brief was ready, when the first full export arrived, how much human review was needed, what changed, and when the final decision was made. Distinguish active work from waiting for a reviewer. Count rejected and unfinished pieces alongside accepted ones.
For approval, record both the first submission decision and the final decision. A piece accepted after two revisions contributes to final approvals but not first-pass approvals. A five-slide carousel remains one requested piece if that is how the scope was defined.
Compare like with like.
If you compare with a freelancer, agency, or internal process, use the same product references, output scope, review requirements, and cost boundaries. Include your team’s coordination and correction time. Keep supplier labor already included in an invoice from being counted twice.
Use the scope and cost calculators to make assumptions visible. Do not convert a single fast example into a blanket time-savings claim. Report the spread of completed jobs and explain where the workflow needed assistance.
Make a decision with specific next steps.
A useful outcome can be “continue for product images and carousels, revise the voice workflow before expanding videos.” It can also be “repeat the pilot with better references” when missing inputs made the result uninformative. The decision should identify the constraint the next batch will test.
If the approved creatives run as ads or organic posts, measure that stage separately with appropriate tracking and context. Approval establishes that the content met the review criteria. It does not establish that the creative increased sales, lowered acquisition cost, or will perform similarly for another audience.
Set up the pilot before reviewing the work.
Edit this fictional software-team brief for your own trial. It defines the counting unit, review boundary and reporting cutoff before results are known. The fields contain planning instructions, not an Aitoma service commitment or measured customer outcome.
Download a blank pilot brief or download the blank per-piece scorecard. The brief sets the rules; the scorecard records what happened to each requested piece.
Read the whole pilot at the same cutoff.
Here is an invented three-piece result. All three pieces reached a first complete submission. Two are approved at the reporting cutoff; the presenter explanation still needs a pronunciation correction. These numbers illustrate the record structure and are not Aitoma performance data.
| Piece | First review | Rounds submitted | Decision at cutoff | Active review |
|---|---|---|---|---|
| DEMO-01 · demo | Revise | 1 | Approved | 18 minutes |
| PRES-02 · presenter | Revise | 2 | Revise: pronunciation unresolved | 27 minutes |
| CAR-03 · five-slide carousel | Approved | 0 | Approved | 9 minutes |
Download the worked pilot scorecard. It adds version references, acceptance criteria, issue owners and evidence placeholders to these rows. Keep the original first decision when a later export is approved.
In this guide, a revision round is a consolidated request followed by a replacement submission. Several small edits delivered together count as one round. Supplier-internal corrections before the first business-review submission are outside this first-pass measure; include their cost and effort wherever your measurement boundary requires it.
Report the denominators beside the percentages.
| Measure | Calculation in this example | What it tells you |
|---|---|---|
| First-pass approval among first submissions | 1 ÷ 3 = 33.3% | One of the three complete first submissions passed business review without another submission. |
| Approved share of requested scope at cutoff | 2 ÷ 3 = 66.7% | Two of the three requested pieces are approved so far. This is provisional while work remains unresolved. |
| Unresolved share of requested scope | 1 ÷ 3 = 33.3% | One requested piece still needs a decision or correction. |
| Active human review recorded so far | 18 + 27 + 9 = 54 minutes | The team has recorded 54 person-minutes of review across all three pieces, including the unresolved one. |
Reporting only the two approved pieces as “2 out of 2 approved” would hide the unresolved third piece. Keep the originally agreed scope visible. If you add, withdraw or replace a deliverable, show the scope change separately rather than silently shrinking the denominator.
If a requested piece has never reached a first complete submission, exclude it from the first-submission denominator and report it as not yet submitted in the full-scope record. If no piece has been submitted, first-pass approval is not yet calculable. If all pieces eventually receive a final decision, report that final outcome with its date and count.
These measures describe production and review. They do not measure advertising performance, and this three-piece example is too small to establish a general supplier success rate. Pair the counts with the actual defect notes, format mix and your agreed limits. Use the cost-per-approved-creative calculator when you also have comparable cost records.
Worksheet added 22 October 2026.
Keep a record of the same-brief trial.
Use this free worksheet to compare two providers on one production job. It gives you 21 prompts and separate fields for each provider’s result and supporting file or source. Results start blank; no vendor is pre-scored.
Download the Excel trial worksheet · Download the CSV version
- Record the exact plan and shared brief before testing. Use the same current references, deliverable and acceptance criteria.
- Check the first export, request one focused revision, then inspect the complete revised file. Record unresolved problems and the reviewer’s decision.
- Record hands-on minutes separately from elapsed approval time. Include extra generation charges and explain how you allocate subscription costs.
An accepted export needs a recorded decision and an identifiable final file. Blank cells mean untested, and a production trial does not establish advertising performance. The Excel version includes a decision dropdown; CSV preserves the fields without formatting or dropdowns.