Understanding Social Biases in Artificial Intelligence Systems

Ming Hsu

Haas School of Business
Helen Wills Neuroscience Institute
University of California, Berkeley

(RowDefault[, (Spacer1), , (Spacer1), ])  #WolframTechConf

Easy For (Most) Humans, Not So Easy (Yet) For Machines

Easy For (Most) Humans, Not So Easy (Yet) For Machines

({{,  }})

What We Want

System that perceive the social world and make social actions.

Can be validated: often difficult

Can generalize: because social world can be highly fluid

Can make decisions: weight consequences

Can be corrected: because we know humans have biases

What We (Promise To) Deliver

A system that can:

Learn about how people think about other people

Generalize to novel stimuli

Behave similarly as humans in lab and field

Including social biases

Correct (to some extent) social biases

What We Mean By “Social Awareness”

Illustrative Example: Using Google News To Predict Resume Callbacks

Labor Market: Using Google News To Predict Resume Callbacks

Labor Market: Using Google News To Predict Resume Callbacks

Empirical Strategy: From Text Corpora To Social Behavior

From Words To Thoughts: Word2Vec

From Words To Thoughts: Word2Vec To Social Perception Dimensions

From Words to Thoughts

Perceived Warmth and Competence of Social Groups

Labor Market: Using Google News To Predict Resume Callbacks

Education: Using Google News to Predict Professor Response Rates

Social Awareness: Detecting Biases

Social Awareness: Detecting Biases

Conclusions

We address key validity challenges in statements about social bias that pervade the media and popular culture.

We find that popular AI/ML models built on naturally occurring text:

Contain knowledge about social groups that correspond to human stereotypes.  I.e., they reflect human thoughts.

This knowledge also corresponds with actual discrimination faced by these social groups. I.e., they reflect human actions.

These findings are not a result of data mining and are scientifically valid

Examples of Future Directions

How these biases evolve over time (e.g., attitude toward Irish in 1917 v. 2017 are very different)?

How does the human brain achieve these capacities?

How to "debias" these AI systems?

Acknowledgements