Practice position
AI assists. People decide.
Use approved systems, minimise information, check important output and keep accountable professional judgement human-led.
Bias can enter at several points
Historic data reflects who reached services, how professionals recorded them and which outcomes organisations valued. A model trained on those records may learn existing inequalities rather than reveal an objective truth.
Generative AI can reproduce stereotypes in apparently harmless work, including fictional cases, summaries and tone changes. A polished answer may over-associate substance use with offending, blame or particular communities.
Accuracy for whom?
An average performance figure can conceal weaker results for smaller or under-represented groups. Commissioners should ask how a system was tested across relevant populations, whether the local setting resembles the evaluation setting and what happens when confidence is low.
Build challenge into the workflow
Ask whose perspective is missing, what labels were used, which people are under-represented and whether an apparently efficient process shifts burden onto people with less digital access. Review language with practitioners and people with lived experience, and monitor effects rather than treating a pre-launch equality check as final.
Practical checklist
- Examine data and label origins
- Request performance by relevant group
- Test fictional content for stereotypes
- Include lived experience in design and review
- Monitor unequal errors and access after launch
Authoritative starting sources
ICO: fairness in the AI lifecycle ↗WHO ethics and governance of AI for health ↗Written by Tony D’Agostino / TD Consultancy. Reviewed . This page provides general professional education, not legal or clinical advice.