Campus News Graphic

Researchers at Penn State and Columbia University have developed artificial intelligence technology that can detect unfair discrimination within specific demographics.

The tool was created to identify discrimination relating to "protected attributes," like race and gender, made by humans or artificial intelligence systems. This detection system stems from the idea of "causality," in which one thing is a cause, and another is an effect, according to a release from Penn State News.

However, finding discrimination based on artificial intelligence can prove "extremely challenging," as cited in the release.

"For example, the question 'Is there gender-based discrimination in salaries?' can be reframed as 'Does gender have a causal effect on salary?' or in other words, 'Would a woman be paid more if she was a man?'" Aria Khademi, graduate student in information sciences and technology at Penn State, said.

Given the difficulty the researchers found when trying to answer hypothetical questions, the team relies on

"sophisticated counterfactual inference algorithms" to find the closest answers they can. Further, the group has also studied public data, such as information from the U.S. Census Bureau to isolate factors such as gender and race.

Through the artificial intelligence system, the researchers were able to start evaluating discrimination in scenarios from gender-pay gaps to data from New York's "Stop and Frisk," which holds information about drivers stopped by law enforcement in New York City.

"You cannot correct for a problem if you don't know that the problem exists," Vasant Honavar, professor and Edward Frymoyer Chair of Information Sciences and Technology at Penn State, said in a statement. "To avoid discrimination on the basis of race, gender or other attributes you need effective tools for detecting discrimination. Our tool can help with that."

If you're interested in submitting a Letter to the Editor, click here.