A Social Ranking System Straight Out of Black Mirror

Gif created by Jim Miller / Splatterplop

In “Nosedive,” the first episode of Black Mirror’s third season, people rate each interaction they have. A lovely conversation would net both participants a 5, while a fight would result in 1s. A person’s societal status, as well as related benefits like premium airline bookings and access to certain events and venues, revolves around one’s score. As the title suggests, the protagonist struggles in this system, particularly during a run of particularly unfortunate luck.

The idea of Big Brother isn’t new; versions of an Orwellian state exist in many forms, from the Patriot Act to third-party data mining. But real-time rankings based on one’s every action and conversation? That’s a leap into dystopia.

It also will become reality.

China has begun rolling out its “Social Credit System” (SCS). Right now participation is voluntary, but in 2020 all 1.3 billion Chinese citizens will be in the system designed to give each person a score that indicates one’s trustworthiness. The government hopes the SCS will encourage a philosophy of honesty: “It will forge a public opinion environment where keeping trust is glorious. It will strengthen sincerity in government affairs, commercial sincerity, social sincerity and the construction of judicial credibility.”

Perhaps the most frightening thing about the SCS is its reach. One’s score will reflect five different categories:

1. credit history
2. fulfillment capacity (fulfilling contractual obligations)
3. personal information
4. behavior and preference
5. interpersonal relationships

The Publicity Department of the Communist Party of China (CPC) Central Committee, the Supreme People’s Court, and the China Banking Regulatory Commission (CBRC) recently issued a notice that authorities in each province will create online platforms to publicly expose debtors and those who have defaulted on loans.

While it’s not necessarily unusual to keep a database of people with bad credit or a history of defaulting, the SCS reaches beyond finances and contractual obligations by attempting to rate a person’s behavior. Predilections, extra curriculars, and daily habits are subject to scrutiny and judgment. What do you typically buy when you go to the store? Baby food? Beer? Sports equipment? How many hours a day do you watch television or play video games? The answers to these questions give the system insight—accurate or not—about one’s personality, habits, and overall level of responsibility. Penalizing someone for, say, buying alcohol could succeed in guiding users away from such purchases and ultimately change their behavior.

The system would also judge people based on who their friends are, what they do, and one’s interactions with them. A “positive energy” assessment rewards people for posting compliments and kind missives outline, especially about the government and the country itself. And in an example of guilt by association, people can get punished for what their friends do online, which incentives people to encourage one another to praise the government and the system rather than to deride it.

Given how ambitious this system is, one has to question how the algorithms powering it would work. The government has commissioned eight private companies to develop the systems and software for making these calculations. One of them is China Rapid Finance, a consumer lender with 3.7 million users and the creator of the WeChat messaging app (850 million users) and works in junction with Tencent, a media and telecommunications juggernaut. The other is Sesame Credit, a product of Ant Financial Services Group, which is affiliated with the global trading company Alibaba and the Chinese government. These groups already have a leg up in gathering data about people’s habits: Sesame Credit works with ride-sharing services like Didi Chuxing (450 million users) and Baihe, an online dating service. Ant Financial uses something called AliPay, which is kind of like PayPal or Apple Pay but with larger reach—it allows people to purchase everything from movie tickets to meals to products online.

It’s unclear which of the eight companies will actually run the SCS once its officially implemented, or how that company or its algorithms will work.

Even though the system won’t be mandatory for over two years, millions of people have already signed up. Whether citizens are afraid of the consequences of not participating or whether they’re hoping to net benefits or privileges by being early guinea pigs is unclear. Citizens who get classified as trustworthy stand to be rewarded—it may be easier for them to get loans or other perks, such as early check-ins on airplanes or special seating in restaurants.

Because Baihe is a prominent online dating site, a user’s score could also boost or lower their chances of being matched with suitable partners, so the system could also impact who dates or marries whom. It could also influence friendships, as people may want to connect with those who have high scores and avoid those with lower scores.

China’s State Council General Office has published guidelines for rewards and punishments and is “Building coordination mechanisms for joint incentive mechanisms for promise-keeping and joint punishment mechanisms for trust-breaking.” Punishments for breaking trust include being placed on a “black list” that would make it difficult for a citizen to function in society.

The ending of “Nosedive” gives us a frighteningly plausible look at what might happen to someone who finds herself deemed untrustworthy: ostracization from society. But at that point, at least she can say whatever she wants.