Most school AI policies explain what staff and students should do. Fewer explain what happens when an AI system produces harmful content, exposes personal data, undermines an assessment or behaves differently after an update.
An incident playbook closes that gap. It gives a teacher a safe first action and gives leadership a consistent route for safeguarding, privacy, technology and governance decisions.
Define an AI incident broadly
Do not limit incidents to cybersecurity breaches. Include harmful or inappropriate output, personal-data exposure, biased recommendations, unsafe automation, material misinformation, assessment-integrity failures and unauthorised AI use that creates real risk.
Near misses matter too. A failure caught before it affects a learner may reveal the same control weakness as a harmful event.
Use a seven-step response
1. Protect people
Stop the harmful interaction, provide support and follow existing safeguarding or emergency procedures where necessary. AI policy never replaces those established duties.
2. Contain the system
Pause the affected feature or account if continued use could increase harm. Do not delete evidence while trying to “fix” the problem.
3. Preserve evidence proportionately
Record time, system, version if known, user action, output and relevant settings. Avoid spreading sensitive screenshots more widely than necessary.
4. Classify the incident
Separate low-impact content errors from privacy, safeguarding, discrimination, security or high-impact decision issues. Classification determines who must be involved.
5. Escalate to the right owner
Create named routes for safeguarding, data protection, IT/security, curriculum/assessment and senior leadership. The Artificial Intelligence for Educational Leadership is designed for leaders who need this cross-functional governance perspective.
6. Decide and communicate
Determine whether the tool can resume, needs new controls or should remain paused. Communicate with affected people and follow any notification obligations under applicable policy or law.
7. Learn
Ask why the existing control failed. Update configuration, training, procurement requirements or policy and share the lesson at the right level.
Set three severity levels
Level 1 – Local correction: low-impact, reversible issue with no sensitive data or learner harm. Teacher or local owner records and corrects it.
Level 2 – Management review: repeated errors, assessment impact, inappropriate content, minor privacy exposure or uncertain cause. Tool may be limited while investigated.
Level 3 – Critical response: safeguarding risk, serious data exposure, discrimination, material security incident or high-impact automated decision. Escalate immediately through established institutional procedures.
Prepare staff with scenarios
Training should include what to do when an output is wrong or unsafe, not only how to use AI. The PGCert in AI in Teaching and Learning can support educators developing deeper responsible practice, while the Certified AI Literacy Professional provides a broader AI literacy foundation.
Run tabletop cases: a chatbot gives a harmful response to a learner; a teacher uploads identifiable student work to an unapproved tool; an AI-marking assistant produces a pattern of questionable scores; a vendor changes a model during term.
Define who can pause a tool
Incident response fails when everyone can report but no one has authority to act. Name the roles that can suspend an integration, remove a tool from the approved list, stop automated marking or require a vendor response.
For critical systems, also define the fallback. Teaching and safeguarding processes should not depend on a single AI service remaining available.
Connect incidents to procurement
Contracts and vendor evaluation should answer operational questions before an incident: Who investigates? Which logs are available? How quickly will the supplier notify material changes? Can the institution suspend processing? How are data deleted?
Where personal data are involved, specialist governance such as the Certified AI Data Protection Officer perspective may be necessary.
Keep an incident learning register
Record category, cause, affected system, immediate action, root/control issue, owner and closure evidence. Review trends each term. Repeated low-severity incidents can reveal a training or design weakness before a serious event occurs.
Avoid recording unnecessary student details in the central register; link sensitive case information through the institution’s established protected systems where needed.
Final takeaway
A good school AI incident response process is simple at the front line and rigorous behind it. Protect people, contain the problem, preserve evidence, classify, escalate, decide and learn.
The purpose is not to create fear around AI. It is to make experimentation safer by ensuring that staff know what to do when technology does not behave as expected.
Responses