CAN AI DISCRIMINATE?
By Michael Farhi
Yes, artificial intelligence (AI) can discriminate and the California Civil Rights Council (Council) has established new regulations that govern the use of Automated Decision Systems (ADS) such as AI. Such regulations apply in the workplace for employment decisions ranging from recruitment and applicant screening, to performance evaluations and promotions, to disciplinary actions and decisions.
The focus of these new regulations, which were finalized and went into effect in California on October 1, 2025, is (1) to clarify and define how the state’s anti-discrimination laws apply to ADS including AI, (2) to make it clear that employers will be held liable for discrimination that results from the use of such systems, regardless of whether it is accidental or intentional, and (3) to provide guidance to employers on the use of AI in order to comply with the new regulations.
The Council was tasked with creating these new regulations after concerns that the expanding use of ADS (including AI) in the workplace for employment decisions would lead to an increase in discrimination against protected classes. According to the Council, tools like AI can provide a wealth of benefits to employers, such as streamlining the application process, efficiency, consistency, cost savings, and monitoring employee work and performance. However, AI can also “exacerbate existing biases and contribute to discriminatory outcomes.” These biases and discriminatory outcomes can manifest themselves in several ways, whether it be mirroring the features of a male-dominated field such as electricians or plumbers — and thereby rejecting female applicants for those positions — or reinforcing gender and racial stereotypes through job advertisement delivery systems by directing job advertisements based on gender or race.
Highlighting these concerns is the case of Mobley v. Workday Inc. (2024). Derek Mobley, a 40-year-old black man with anxiety and depression, alleged he was discriminated against by software vendor Workday Inc.’s AI decision making tool, and that Workday Inc., is liable as an agent of an employer for that discrimination. Mobley claimed that he applied for 100 jobs and that Workday Inc.’s AI tool rejected his applications for all 100 jobs even though he met the qualifications for each of the positions. He believes that he isn’t the only person that Workday Inc.’s AI tool discriminated against. There are potentially over one billion people who were discriminated against due to Workday Inc.’s AI tool. While the case is far from over, a federal court recently put the case on track to possibly become the largest class action suit in history.
KEY PROVISIONS IN THE NEW REGULATIONS
There are several key provisions in the new AI regulations which can be found in the California Fair Employment and Housing Act (FEHA).
- Anti-discrimination laws apply to AI: Employers and third parties such as software vendors that provide ADS to Employers will be held liable under FEHA for discrimination that is caused by AI tools.
- ADS-related discrimination in employee selection is prohibited: In general, the use of “an ADS or selection criteria that discriminates against an applicant or employee is prohibited, unless the criteria is job-related and consistent with business necessity.“
- Proactive testing to avoid claims of discrimination: To protect against or defendant against a claim of discrimination, employers should conduct “anti-bias testing or similar proactive efforts to avoid unlawful discrimination, including the quality, efficacy, recency, and scope of such effort, the results of such testing or other effort, and the response to the results.”
- Preservation of Records: Employers are required to preserve all employment and personnel records for at least four years from creation including all records created using ADS. These records include all “applications, personnel records, membership records, employment referral records, selection criteria, automated-decision system data, and other records created or received by the employer or other covered entity dealing with any employment practice and affecting any employment benefit of any applicant or employee.”
- Accommodations may be required: The use of ADS to “analyzes an applicant’s tone of voice, facial expressions or other physical characteristics or behavior may discriminate against individuals based on race, national origin, gender, disability, or other characteristics protected under the Act. To avoid unlawful discrimination, an employer or other covered entity may need to provide reasonable accommodation to an applicant as required by Article 8 (religious creed) or Article 9 (disability) of these regulations.”
- AI and Individualized Job Application Assessments: An employer with 5 or more employees cannot rely solely on AI for assessments. Employers are required to conduct their own individualized assessment before denying a job applicant based on criminal background.
KEY TAKEAWAYS FOR EMPLOYERS BEFORE NEW JERSEY ADOPTS THESE LAWS
The increase in efforts to regulate AI demonstrate that the use of AI in employment decisions does not mean the end of discrimination in the workplace. To the contrary, these new regulations create another legal vulnerability for employers. Employers must learn and understand the new regulations and stay up to date with the latest changes to protect themselves from liability.
Here are a few ways that employers can protect themselves in advance of new regulations:
- Keep records of the use of AI in all employment decisions as workers have the right to challenge employment decisions that were made by biased AI tools.
- Frequently and proactively conduct anti-bias testing of ADS including AI tools being used in employment decisions and keep a detailed record of the completion of the testing and results.
- Limit the use of AI tools or pair it with a human for making employment decisions such as the screening of potential job applicants.
- When selecting a vendor to provide ADS including AI tools, confirm whether the vendor’s AI is an open or closed network, any and all bias testing that has been conducted on AI tool including results, and the sources used to train the AI to be able to make automated decisions.
- Create a standard policy regarding the use of AI tools in employment decisions.
- Lastly, contact an experienced and knowledgeable attorney that can provide expert guidance on California’s new AI regulations. This is the best way to ensure that you have a comprehensive AI policy that protects you and your company from discrimination claims.