Algorithmic Bias Still Failing: Who Gets Left Behind in 2026
In Spring 2021, Mary Louis, a prospective tenant with a housing voucher, applied for an apartment in Massachusetts. SafeRent’s automated screening algorithm rejected her application without accounting for the fact that over 73% of her rent would be paid directly by a public housing authority.
The resulting low score led the landlord to decline her, forcing Louis to spend months finding an alternative residence, eventually settling for a property that cost $200 more per month in a neighborhood she had not chosen. The algorithm did not need to know her race to produce this outcome; the inputs performed that work for it. SafeRent eventually settled for $2.275 million in November 2024 and was forced to revise its screening tools for voucher holders. However, the core question remains: why was the system designed to produce such exclusion in the first place? This is what algorithmic bias looks like in 2026.
It is not a theoretical concern waiting for a test case, but a documented pattern supported by settlements, and the same structural problems continue to operate across the industry.
The hiring pipeline has a bias problem; it can’t explain it away
The employment sector remains a primary site for automated discrimination. Derek Mobley applied for hundreds of jobs through Workday’s AI screening system and was consistently rejected, often within an hour of submission and sometimes in the middle of the night.
His case argued that Workday’s tools discriminated based on race, age, and disability through what the algorithm had “learned” to prefer from historical hiring patterns. By May 2025, the case achieved preliminary collective certification as a nationwide action, with courts ruling that the software vendor could be held liable alongside the employers who used the tool.
The underlying mechanism matters more than the legal timeline. A University of Washington study recently tested three AI hiring models on identical applications that differed only in the applicants’ names. White-associated names were preferred 85% of the time, while Black-associated names accounted for only 9%. These tools were not explicitly designed to discriminate; they learned from historical data that had been doing so for decades.
This confirms that algorithmic bias in hiring does not require a biased engineer. It simply requires biased training data and a lack of oversight regarding what the model actually prioritizes.
Loan approvals and housing: The algorithm as gatekeeper
SafeRent’s scoring system penalised Black and Hispanic applicants partly because of lower median credit scores. A gap built over decades of lending practices.
The algorithm inherited that disparity, encoded it as a reliable financial signal, and ran it across every landlord on the platform. Mortgage algorithms frequently demonstrate the same pattern. Black and Hispanic borrowers with creditworthiness comparable to that of white borrowers are still quoted significantly higher rates.
Variables such as zip code, employment history, and educational background carry the race signal even when race itself is absent from the dataset. The model genuinely believes it is making a neutral financial calculation, unaware that it is discriminating based on proxy variables. What the SafeRent case settled, beyond the financial penalty, was a question that courts had long been reluctant to answer. Can the algorithm vendor be held liable when the landlord makes the final call?
The court’s affirmative answer suggests that an algorithm performing that much of the decision-making cannot claim to be just software.
Criminal justice: Where algorithmic bias carries the most weight
COMPAS has been scoring defendants for sentencing and parole decisions in U.S. courts since 2000. A 2025 study published in the Indiana Law Journal used causal inference methods to find that the system doesn’t just carry racial bias forward from its training data; it amplifies it. The researchers found the algorithm actively worsens racial disparities rather than smoothing them.
The scoring factors include employment history, neighborhood, and family criminal records. None of these are race, yet all of them correlate with race through structural inequality built over generations. A score generated in seconds can follow an individual through sentencing, parole hearings, and post-release monitoring, presented each time as an objective, data-driven analysis.
While courts have largely continued to use these tools, the distance between what the research shows and what the courtroom accepts represents a significant accountability failure.
Why bias mitigation hasn’t fixed it
The standard industry response to documented algorithmic bias is an audit followed by a mitigation commitment. However, neither has reliably solved the issue. Each approach runs into specific technical and social barriers:
| Mitigation Approach | What It Claims to Fix | Where It Actually Breaks Down |
|---|---|---|
| Bias Audits | Proxy variables like zip codes carry the same signal. | Proxy variables like zip codes carry the same signal. |
| Removing Variables | Eliminates explicit demographic inputs. | Proxy variables like zip code carry an identical signal. |
| Diverse Training Data | Reduces representation gaps. | Doesn’t change what the model is being asked to predict. |
| Fairness Constraints | Reduces one form of bias. | Often shifts it to a different group or outcome. |
| Human-in-the-Loop | Adds an oversight layer. | Reviewers tend to defer to algorithmic scores anyway (Automation Bias). |
None of these approaches reaches the actual source of the problem. Biased training data, proxy variables, and optimization targets shaped by historical discrimination survive every audit because most audits measure outputs rather than origins.
The regulatory gap and what’s actually required
The EU AI Act classifies hiring algorithms, credit scoring, and criminal justice AI as high-risk, with full compliance due by August 2026. Fines are substantial, reaching €15 million, or 3% of global annual turnover, for high-risk violations, and up to €35 million, or 7%, for the most serious prohibited practices. The regulation demands what many organisations are currently unprepared for. Bias-controlled training data, ongoing monitoring after deployment, and oversight capable of stopping a harmful decision before it happens.
In the U.S., courts are performing the work that federal legislation has yet to achieve. Vendors building AI hiring tools are being held liable under anti-discrimination law, regardless of whether the discrimination was intentional.
As the litigation docket fills up, it is becoming clear that algorithmic neutrality is not a valid legal defense when the system itself is making the decision.
What IT and procurement teams should actually check
Deploying a third-party AI tool for hiring, credit evaluation, or risk scoring means assuming a significant share of the associated liability.
Before deployment, procurement teams must ask critical questions regarding independence. Has the vendor run a truly independent, third-party bias audit? What did the training data reflect, and has it been tested for disparate impact across demographic groups? What specific outcome variable is the model optimising for? And does that target encode historical patterns that would be difficult to defend in court?
After deployment, it is vital to monitor outputs by demographic group. A tool that passes bias testing in a controlled environment can “drift” in production when the real-world population differs from the one used in training.
The SafeRent algorithm was doing exactly what it was built to do, and that was the problem.
Distilled
In the landscape of 2026, algorithmic bias has shifted from a technical glitch to a core legal liability. High-profile settlements in hiring and housing have established that automated systems often amplify structural disparities rather than neutralize them.
These cases prove that an algorithm performing the bulk of decision-making cannot be dismissed as just software. And the responsibility for its outcomes rests entirely with the organization that deploys it. This bias persists because standard mitigation efforts focus on measurable outputs while leaving the underlying sources, biased training data, and optimization targets unchanged. As global regulations like the EU AI Act approach full enforcement, the burden of proof has shifted to the implementer.
For IT leaders, the takeaway is clear: vendors do not assume your liability, and failing to test for disparate impact before deployment is equivalent to deploying a biased system by design.
