Gamification and AI: Go Directly to Jail, Do Not Pass Go

Gamification and AI: Go Directly to Jail, Do Not Pass Go

This article relates to We Have Been Harmonized

As American political scientist Joseph Nye postulated in the 1980s, there are two ways to control people in geopolitics: hard power (i.e., coercion via violence) or soft power (i.e., enticement via incentive). Successful geopolitical strategy is often about knowing when to use soft power instead of force.

In We Have Been Harmonized, author Kai Strittmatter explains how the Chinese Communist Party has realized the potential to harness both soft and hard power to create absolute control over all aspects of its nationalized society. The Party is using information technology to enhance its security-state apparatus. Beyond camera surveillance — tracking people by gait, voice and face — and internet censorship — no images of Winnie the Pooh allowed! — the Chinese government has harmonized its population via two less-talked-about means: Artificial Intelligence (AI) and gamification. It is the combination of these two tools that has given the Chinese Communist Party a virtually unassailable lock on power.

Artificial intelligence is the use of computer-based machines to learn and adapt based on environmental factors and input. Computers mimic and frequently outstrip human learning capabilities in some cognitive domains. Mixed with cloud-computing and massive datasets, AI can often model and predict outcomes much faster and more accurately than humans (who are tainted by experiential and cognitive biases). Over the past decade, AI research in the United States and China has been at the forefront of state-backed funding. Competition over its development has been described as the modern-day space race, albeit this time Sino-American versus Soviet-American.

When it comes to control, AI represents the threat of force, or hard power. Combined with camera, location and voice surveillance, it epitomizes the panopticon. Unlike the panopticon, though, where prisoners behave as if the guard is watching them even though the guard cannot be watching at all times, through AI, people are being watched all the time. One's every move, interaction and discussion can be observed and analyzed. If not immediately, the data is saved for future analysis.

AI is already being used predictively in China to make calculated guesses about someone's intentions and actions before they occur — much as science fiction writer Philip K. Dick foreshadowed in his short story, "The Minority Report," where people are arrested for crimes they have not yet committed. China's successful response to stem COVID-19 accidentally demonstrated how far the surveillance state has come. As China closed down, there was little need for human contact tracers — the tracing could be done via AI. The Chinese government used its elaborate camera and mobile tracking system to follow a sick person's every move and identify every individual they came into contact with. Though great for COVID response, imagine what such a system might mean for someone harboring non-harmonious views of the Chinese Communist Party (i.e., they have the gall to mention there is an influenza disease spreading in Wuhan).

Not many Chinese citizens are allowed to entertain such views. Enter soft power — enticing people to think, believe or do what you want them to. The Chinese Communist Party proudly uses AI to ensure Party control coercively. As Strittmatter discovers, Chinese tech executives will proudly note that racial profiling is built into their surveillance systems (e.g., camera recognition software) for enhanced security. However, it is the use of soft power through gamification that has, arguably, most stabilized the Party's total power over society.

Gamification is the application of game-design techniques to non-game situations. It is often used in marketing and business to get people to buy things they otherwise might not. It uses aspects of competition and reward to entice people to participate willingly in an endeavor, ideally initiating a long-term habit (e.g., there is a reason you buy your coffee at the same café — reward points). Casinos have mastered the science of gamification, but the process is all around us: online schools (badges and certificates), Waze Maps (identify where the police are), movie reward cards (free popcorn), SkyMiles (free flights on dates no one wants to fly), Twitter (amassing "retweets") and Duolingo (compete with others while learning a language). Humans love winning and they are suckers for free stuff. It doesn't matter if the competition and the winnings are immaterial. Gamification uses our psychological shortcomings to entice us to do things.

In the past, totalitarian states merely used coercion. Often, it was enough. The threat of bodily harm to oneself or one's family and friends tends to keep people in line. But as most regimes discover, eventually such a large groundswell of disenchantment can build that the state cannot threaten force on everyone at all times. Most authoritarian regimes fall apart when a certain threshold is reached, whereby the system is no longer legitimate, and force cannot be used to maintain power. The People's Republic of China narrowly avoided this fate on June 4, 1989, during the Tiananmen Square uprising.

Gamification helps to eliminate this threat. Whereas AI allows for constant surveillance and the removal of those who are not yet harmonized (i.e., toeing the party line), enticed participation in propaganda and community surveillance games actually changes how people behave and think, thereby, in theory at least, eventually reducing the need for surveillance at all. China has begun gamifying mobile apps – one of which is required for all Party members and the other for all citizens (starting in 2020).

The first app, Study (Xi) Strong Country, uses gamification for propaganda consumption. It had over 100 million users as of 2019. The app is filled with Communist writings and the essays of current leader-for-life, Xi Jinping. It logs how long people read these essays and canons, rewarding them with points. Users can also take comprehension quizzes to earn even more points. Of course, everything done on the app is logged by the state. People know this, so they participate and compete with one another to intake more propaganda and party-based rhetoric. Many of us have faked reading content for a boring class or workshop. However, this is a convenience no Party member can afford. They need to pass quizzes to show their loyalty. They need to have the app open for so long each day to demonstrate they are reading the materials. Eventually, using the app becomes habit and gamification keeps people competing more than they would otherwise.

The second app is an even more insidious use of gamification for totalitarian purposes. There are currently several different variations of it (one is called "Honest Shanghai"), depending on where you live in China; these are all beta tests for the planned national rollout. It's similar to a human version of Pokemon Go, where instead of trapping imaginary monsters on street corners, you find and report real people for being "untrustworthy." The app tracks your trustworthiness as well, using AI as well others' reporting. You start with a base score of 1,000. If your score goes too low — for behavior such as cutting in lines or hanging out with other people that have low scores — you get penalized. Except, these aren't just in-game penalties; there are real-life consequences. Some are mild — you are banned from public transport for a limited time; some are severe — you are not allowed to leave the country. Through the gamification of this app, citizens enforce Party-sanctioned behaviors on others and themselves. It becomes habit-forming. Eventually, people shouldn't even fathom doing something that would diminish their trustworthiness score.

Though there are real penalties for doing poorly in the game, the app still represents soft power. Most reactions to the app are not bewilderment at being required to participate, but rather excitement that other non-harmonious and untrustworthy people will be easily identified. They are willingly participating and surveilling themselves and others on behalf of the police. Instead of top-down, authoritarian enforcement, the Chinese Communist Party has used gamification to entice the population to use the wisdom of the crowd — often against its own interests.

This is a watershed moment in the world of surveillance. In the West, people must deal with free speech bumping into Twitter-shaming and "canceling" by those on the same side of the political aisle — e.g., J.K. Rowling and Noam Chomsky being ostracized by the left. Though such methods are controversial and perhaps annoying, they are not analogous to state-sponsored bullying. Being disliked by someone you've never met an ocean away on Twitter pales in comparison to being reported by your neighbors for not taking your recycling bin off the curb in a timely fashion and then penalized by the government. However, as the concept of privacy continually erodes among younger generations in the West, particularly through social media apps — several of which, including TikTok and WeChat, are connected to the Chinese Communist Party — it would not be surprising to see gamified citizenship apps rise in popularity here too.

- Stephen Mrozek - pseudonym for an American university professor who may wish to take their daughter to see the Great Wall one day

Filed under Society and Politics

This "beyond the book article" relates to We Have Been Harmonized. It originally ran in September 2020 and has been updated for the October 2021 paperback edition. Go to magazine.

This review is available to non-members for a limited time. For full access become a member today.