What is Machine Learning Bias?
Systematic errors in AI systems that produce unfair or discriminatory outcomes. Bias can come from skewed training data, flawed algorithms, or feedback loops. In privacy contexts, biased systems may disproportionately surveil or deny services to certain groups.
AI doesn't eliminate human bias—it amplifies it. Machine learning systems learn from data, and if that data reflects historical discrimination, the AI will too.
Sources of Bias
Data Bias
- Historical bias: Training data reflects past discrimination (e.g., hiring data where women were underhired)
- Representation bias: Underrepresented groups have less data, so models perform worse for them
- Measurement bias: The thing being measured doesn't capture what we care about
- Aggregation bias: One model for all groups when different groups need different treatment
Algorithmic Bias
- Optimization: Model optimizes for wrong metric (clicks over fairness)
- Feedback loops: Model's predictions influence future data (recommendation systems)
- Proxy discrimination: Using correlated features (ZIP code as proxy for race)
Privacy and Surveillance Implications
- Facial recognition: Higher error rates for women and people of color—wrongful arrests
- Predictive policing: Reinforces over-policing of minority neighborhoods
- Credit scoring: AI may encode historical lending discrimination
- Hiring: Resume screening AI may reject qualified candidates from non-traditional backgrounds
- Advertising: Job and housing ads shown differently by demographic
Mitigation
- Diverse training data and teams
- Auditing for disparate impact
- Human oversight of high-stakes decisions
- Transparency and explainability
- Regulatory frameworks (EU AI Act, sector-specific rules)
Related Terms
AI Hiring Discrimination
The use of AI in hiring processes that can systematically discriminate against candidates based on protected characteristics inferred from resumes, video interviews, social media, and other data.
Automated Decision-Making
The use of algorithms and AI systems to make decisions about individuals — including credit approval, hiring, insurance pricing, benefits eligibility, criminal sentencing, and content moderation — often without human oversight, transparency, or the ability to appeal.
Facial Recognition Ban
Legislative and regulatory actions to prohibit or restrict the use of facial recognition technology — particularly by law enforcement and in public spaces — driven by accuracy concerns, racial bias, mass surveillance risks, and the fundamental threat to anonymity in public life.
Have more questions?
Use our guided flow to get the right next privacy step for Machine Learning Bias.
Open Guided Flow