System Card for Dall-E shows examples of known biases

jas0nh0ng · September 22, 2022, 6:15pm

I just chatted with someone from OpenAI, and she pointed me to this:

openai/dalle-2-preview/blob/main/system-card.md#probes-and-evaluations

# DALL·E 2 Preview - Risks and Limitations

### Note: This document summarizes the initial risk analysis and mitigations for the DALL·E 2 system and is only up to date as of April, 2022. Please see the [OpenAI Blog](https://openai.com/blog/) for more up-to-date information.

**Summary**

-   Below, we summarize initial findings on potential risks associated with DALL·E 2, and mitigations aimed at addressing those risks as part of the ongoing Preview of this technology. We are sharing these findings in order to enable broader understanding of image generation and modification technology and some of the associated risks, and to provide additional context for users of the DALL·E 2 Preview.

-   Without sufficient guardrails, models like DALL·E 2 could be used to generate a wide range of deceptive and otherwise harmful content, and could affect how people perceive the authenticity of content more generally. DALL·E 2 additionally inherits various biases from its training data, and its outputs sometimes reinforce societal stereotypes.

-   The DALL·E 2 Preview involves a variety of mitigations aimed at preventing and mitigating related risks, with limited access being particularly critical as we learn more about the risk surface.

**Content warning**

- This document may contain visual and written content that some may find disturbing or offensive, including content that is sexual, hateful, or violent in nature, as well as that which depicts or refers to stereotypes.

[Introduction](#introduction)

[System Components](#system-components)

This file has been truncated. show original

Here are images for “builder”

And here are images for “flight attendant”

The system card shows more examples. This is definitely worth us looking over more.