Humans are reviewing Copilot users’ bizarre and abusive image-editing requests

A reminder for anyone who thinks their AI chats are private: A human may see what they upload. A report from 404 Media this week describes contractors reviewing Copilot users’ photos, editing requests, and the images Copilot produces. Some of what they see is disturbing.

The third-party companies hired to conduct these reviews present their workers with an original photo and a user’s request for an edit. They are also shown two potential AI edits and asked to judge each on technical merit. Because this is the internet, this goes well beyond requests to put your cat in a Halloween costume. Many of the requests are sexual in nature and some involve children, the report said. The faces in the original photos are not censored, meaning reviewers can see who the subjects are.

Editing requests and results that reviewers are asked to evaluate include requests to shorten skirts, enlarge body parts, and place people in sexual positions. In many cases, the edits appear nonconsensual. For example, one contractor mentioned being shown multiple sets of upskirt photos. Another said they were asked to rate foot-fetish images of children’s cartoon characters. Users had asked Copilot to create pictures of Ariana Grande suffering from anorexia. Images depicting animal sacrifice crossed one reviewer’s desk.

Interviewees told 404 Media that contractors are paid to rate the output quality of an image, but not to flag inappropriate content. They check things like whether instructions were followed properly, whether parts of the image were preserved or distorted, and the overall look of the image. They don’t make legal or safety judgments about the content.

Microsoft uses customer data from some accounts to help improve its products. But when 404 Media asked, it didn’t say whether users can exclude uploads from these rating tasks.