For most of the history of training, a completed course was a weak little promise. Someone finished the module, passed the quiz, turned in the assignment — and you let yourself believe some learning had probably happened. It was never strong evidence, but it was something. The artifact stood in for the cognition.
That stand-in is dead. Everyone taking your training now has an AI in their pocket — ChatGPT, Copilot, Gemini, Claude, take your pick — that will generate the essay, pass the quiz, and produce the assignment in less time than it takes to read the prompt. The completed artifact no longer implies a single thing about what's in the learner's head, because the artifact is now free. I've spent twenty years building training and quality programs where the question that mattered wasn't "did they finish" but "did it change what they do on the job." That question just got existential, and most of the industry hasn't noticed yet.
Two questions, and the field is only honest about one
When you strip it down, effective training has to answer two questions:
- Did the design actually teach the objective? Does this course require the thinking it claims to build?
- Did learning actually occur — real cognition, in this specific person?
We've always been mediocre at the first and we faked the second. AI didn't create those weaknesses. It just made faking the second one trivial, and in doing so it exposed how thin the proof always was.
The proxies that just stopped working
Look honestly at how learning gets "measured" in most organizations, and you'll see a stack of proxies that AI has quietly invalidated:
- Completion. Completion was always attendance. It tells you someone clicked through. It never told you they learned. Now it doesn't even tell you they read it.
- The multiple-choice quiz. A recall test on facts the learner will look up the moment they need them — and that the AI will answer instantly during the quiz itself. You're measuring whether they can retrieve, not whether they can do, and you're not even measuring retrieval cleanly anymore.
- The submitted artifact. The essay, the deck, the plan, the analysis. This was the strongest of the weak proxies — surely producing the thing meant they learned the thing. Not anymore. The thing is the easiest part to generate. Grading the artifact now grades the learner's prompt, not the learner.
If your evaluation rests on any of these, you are certifying learning you can no longer see. The certificate on the wall says "completed." It does not say "can."
The signals that still hold
Two kinds of evidence survived the AI flood, because neither can be produced by a model on the learner's behalf:
Alignment — does the assessment require the thinking the outcome claims? This is the design discipline. If your objective is "evaluate which vendor fits a given scenario," a recall quiz literally cannot measure it — evaluation lives in the justification of a choice, not in the recall of a fact. Most assessments are mis-aligned in exactly this way: high-order outcome, low-order test. (The fastest tell is the word "understand." Ban it. You can't see "understand," so you can't assess it — see how to build a course outline for the fix.) AI can't fake your way out of a misalignment; it can only expose it.
Unfakeable performance — can they do it and reason about it, unscripted? This is the part AI broke and then, in breaking, clarified. The reliable signal isn't the finished work. It's what happens when you make the learner defend the finished work in real time, or teach it back to someone, or do the real task and explain the choices as they go. A learner who had ChatGPT write the plan cannot survive five honest questions about why the plan is built the way it is. A learner who actually understands it can — and gets better under pressure, not worse.
The mental shift is small and total: the training is the input, not the evidence. Let them use AI to produce the deliverable — that's the modern world, and fighting it is a waste of energy. Then assess the thing AI can't hand them: the live, unscripted reasoning about their own work.
What this looks like for a team of one
You don't need a research lab. You need to stop treating "done" as "learned" and move the proof one inch closer to the behavior:
- Match the assessment to the cognitive level. If the outcome is recall, a teach-back covers it. If it's "evaluate" or "build," the assessment is them doing it and defending the judgment calls — not a quiz about it.
- Make them defend or teach, not just submit. A five-minute "walk me through why you did it this way, and what you'd change" reveals more than a fifty-question test. Teaching it back to a colleague (or, increasingly, to an AI playing a confused novice) exposes the gaps instantly — you can't teach what you don't understand.
- Demote the certificate, promote the evidence. "Completed the course" becomes what it always was: showed up. The thing worth recording is "demonstrated they can reason it under questioning." That's the line that means something to a manager.
You don't need unfakeable. You need better than a checkmark.
Here's the honest part, because someone will raise it: no assessment is perfectly cheat-proof. A determined person with enough time can game almost anything. Fine. You don't need perfect — you need better than the completion checkmark you have today, and that is a shockingly low bar that the entire industry is currently failing to clear. The goal isn't an unbreakable test. It's evidence of actual cognition that a reasonable person would trust more than a finished worksheet. That's achievable right now, with the tools you already have.
The reframe
AI didn't kill assessment. It killed the lazy proxies we'd been hiding behind, and it did us a favor by doing it in the open. For years we certified learning we couldn't actually see and mostly got away with it. We can't anymore — and that's the moment the field gets honest.
So stop asking "did they finish the course?" That question is now meaningless. Ask the two that aren't: does this assessment require the thinking I claimed to build, and can this person reason it out loud when I push? Answer those and you're measuring learning. Skip them and you're printing certificates that prove someone had an internet connection.
AI can draft the course and the learner can use AI to do the work. What it can't do is hold the understanding for them when you ask them to explain it. That — the unscripted reasoning, judged against what the outcome actually claimed — is the last honest signal of learning. Build your evaluation on that one, because it's the only one left standing.
— Tom
Start with an outcome the assessment can actually prove
The free Course Outliner takes a topic and an audience and gives you measurable objectives and an assessment tied to the outcome — the alignment half, in seconds. No signup.
Try the free Course Outliner →