A Joust Tool. Free. No email required.

AI Pilot Pre-Flight Checklist

Twelve questions to help your team prepare for an AI pilot.

Why a pre-flight check?

An AI pilot needs more than a working model. Someone has to own the workflow, handle exceptions, measure the result, and help the team adopt the change. This checklist helps you review those preparations.

This checklist surfaces those prerequisites before you commit budget. Twelve yes/no questions across four categories. Each "no" is a specific, named gap you can either close or scope around. The score gives you a simple readiness signal. The category breakdown tells you where the gaps cluster.

Use the answers to plan your next steps with the team. The score summarizes your responses; it does not predict whether a pilot will succeed or replace a technical review.

  1. Pick one specific pilot you are considering or already running. Not your AI strategy in general, one specific workflow.
  2. Answer each question yes or no. Click the question or the checkbox to mark "yes." Be honest. A "we are working on it" is a no.
  3. Read the score and category breakdown. Each unchecked question shows specific guidance on what to fix. Save your assessment as a PDF for your team.
Or load a sample to see what the output looks like:

The twelve questions

0
of 12
Not started
Answer the questions above to see your readiness score.
Each "yes" earns one point. Review the unchecked items with your team before deciding how to proceed.

Category breakdown

Each category contains three questions. The bars below show where your readiness is strong and where the gaps cluster.

What each category means

Workflow ownership
A named person is responsible for the workflow and its results. They need time and authority to resolve problems during the pilot.
Workflow definition
Map the work from the initial request through completion, including exceptions and handoffs. Show where AI would be used and where people would review its output.
Measurement
Measure the business outcome alongside technical performance. A more accurate model is useful only if it also meets the requirements of the job.
Change management
Involve the people doing the work and budget for training, integration, review, and ongoing support.
The methodology behind the checklist

The questions reflect recurring issues in Joust engagements: workflow ownership, process definition, measurement, and the practical work of adoption.

The four categories help you see which preparations need attention. Review the individual gaps as well as the total: a high score can still leave one critical dependency unresolved.

On scoring. Each yes adds one point. The 0-4, 5-8, and 9-12 bands group the answers for planning. In our roadmap work, higher scores have been associated with more pilots reaching production. That observation is not a probability of success for your project.

On answering. Mark an item yes when it is in place. If someone else is responsible, confirm the arrangement with them. Work in progress stays unchecked so it remains visible in the preparation list.

On scope. This checklist applies to one pilot at a time. A platform rollout (multiple workflows, multiple teams) needs a different conversation. If you are planning that, the checklist is still useful per workflow, but the score does not aggregate cleanly.

What we are not checking for

This checklist covers operational preparation. You will also need to assess:

  • Model selection. Which model, vendor, and deployment meet the requirements of the work.
  • Technical integration depth. System access, APIs, data pipelines, monitoring, and the effort needed to connect them.
  • Regulatory and compliance review. The requirements that apply to the data and proposed use. Include this review in the pilot plan.
What Joust does with this checklist

An AI Operations Roadmap examines three to seven workflows over five weeks. We work through the preparations with your team, estimate the costs and benefits, and set out a plan for implementation.

References and where this approach comes from

Joust internal sources

  • When an AI pilot stalls (Joust, 2026), the long-form companion to this checklist. joustagency.com/blog/the-audit-is-not-a-product/
  • Mapping a workflow before an AI pilot (Joust, 2026), the methodology paper. joustagency.com/blog/workflow-first/
  • Joust AI Operations Roadmap findings, aggregated patterns across mid-market engagements (2024-2026).

External research informing the questions

  • McKinsey & Company. The state of AI in early 2024. A survey of adoption, reported benefits, and risk-management practices.
  • Brynjolfsson, Li, Raymond. Generative AI at Work. NBER Working Paper 31161 (2023). Highlights the importance of well-defined task structure for AI productivity gains, supporting the "workflow definition" category. nber.org/papers/w31161
  • RAND Corporation. The Root Causes of Failure for Artificial Intelligence Projects (2024). Interviews with 65 experienced practitioners about project purpose, data, infrastructure, and technical feasibility.

What we deliberately exclude

  • Model-quality benchmarks. Assess these separately against the requirements of the task.
  • Vendor case studies as a scoring input. Results from another company do not establish readiness for this pilot.
  • Company-wide maturity scores. These questions concern one workflow at a time.
Talk it through with Ron Read the long-form companion
Get in touch