Practice position
AI assists. People decide.
Use approved systems, minimise information, check important output and keep accountable professional judgement human-led.
Look beyond model bias
Health inequality can arise even when a model performs similarly across groups. Digital access, language, literacy, disability, device cost, confidence, housing instability and mistrust can affect who can use an AI-enabled route and who drops out.
If an automated route becomes the easiest or only route, people who need human contact may receive a poorer service. Productivity for the organisation can become extra work or exclusion for the person using it.
Ask distributional questions
Who receives the benefit, who carries the risk and who supplies the data? Which errors are more serious for people already facing stigma or surveillance? What non-digital alternative remains available? These questions belong in commissioning and evaluation, not as a late accessibility check.
Use evidence and involvement together
Quantitative monitoring can reveal differences in reach and outcome, but it may not explain why they occur. Combine disaggregated data with accessible involvement of people using services, people outside treatment, frontline staff and community organisations.
Practical checklist
- Retain a realistic non-digital route
- Assess language, literacy and accessibility
- Measure reach and errors by relevant group
- Ask who carries extra work or risk
- Act on lived-experience feedback
Written by Tony D’Agostino / TD Consultancy. Reviewed . This page provides general professional education, not legal or clinical advice.