Engineering Narrow-Mindedness: The Dangers of Simplified Morality

Echo chambers foster narrow-mindedness by simplifying moral values for pleasure and promoting moral outrage without genuine engagement.

### The Dangers of Simplifying Morality: Echo Chambers, Outrage, and Gamification
* 🤔 The talk explores the idea that we may be **simplifying our morality and values for comfort and pleasure**, leading to a **dangerous trend in society**.
* 📉 The speaker expresses concern that engineered systems are promoting "**sticky narrow-mindedness**," which can be detrimental to **nuanced understanding and genuine moral engagement**.
* 📺 The distinction between echo chambers and filter bubbles is highlighted, emphasizing that **echo chambers foster distrust of outsiders**, akin to **cult-like behavior**, while filter bubbles merely **limit exposure to differing views**.
* 🔍 The concept of "**evidential preemption**" is introduced, illustrating how individuals within echo chambers can reinforce their beliefs by **predicting and confirming negative portrayals** of opposing views.
* 📊 The talk discusses how new media communication technologies, like social media, contribute to the **flourishing of echo chambers and extreme political polarization**, with both left and right echo chambers exhibiting **similar toxic traits**.
* ⚖ The speaker critiques the reliance on quantification and metrics in various systems, such as education, which can lead to a **false sense of clarity and understanding**, stripping away the **richness of qualitative insights**.
* 📱 The phenomenon of "**moral outrage porn**" is examined, where individuals engage with representations of moral outrage for **immediate gratification** without the burdens of **genuine moral action**, leading to a **simplification of moral beliefs**.
* 🎮 Finally, the dangers of gamification in public discourse are discussed, where **simplified goals and rewards can distort values**, encouraging a **narrow focus that undermines the complexity of real-world issues**.

So, I've been thinking a lot about echo chambers and online outrage. So, here is the big idea for this talk: my interest is in the possibility that we're simplifying our morality and our values for pleasure. Having more complex and nuanced values is a little more painful to go about in the world, and having a simpler one is a little more safe, secure, and communal. And I'm even more worried that because of this possibility, one of the things that might happen is that people out there are consciously designing systems to encourage us to simplify our value system. Okay, the idea is this is a kind of conceptual engineering idea. I'm worried the big reveal that I'm supposed to say for the end, but I'm gonna say now. The big worry I have is that people are engineering systems designed to create sticky narrow-mindedness, right? That's the idea. That's where all the stuff is headed. So, first, echo chambers. Probably the reason I'm here and talking to you is because I wrote this thing that went by

apparently, philosophy pieces can go viral. This thing went viral and in more readers than I could even imagine. This article was mine, was based on this book, which I think everyone should read, by Kathleen Hall Jamieson and Jessica Pella on echo chambers, which is a study of Rush Limbaugh on Fox News in the 90s and early aughts, and how they operated. So, one of the things I noticed was everyone in sociology and political philosophy and journalism were using these two terms as synonymous. One is filter bubbles, and the other is echo chambers. And if you go back and read the original text that introduced these ideas, I'm a philosopher, so I'm always doing this like this. This is the book where the idea of echo chambers came from. You'll discover that they're actually totally different concepts. Okay, so a filter bubble, which an author named Eli Pariser made this term famous, is something where you don't hear the other side. Okay, the idea of a filter bubble is all your friends on Facebook and Twitter

have your political alignment, so you're never exposed to the other side, right? It's a not hearing in Jameson Capella's book. They talked about an echo chamber or something where you were taught to distrust the other side, right? And those are two totally different concepts, right? And I think it's very clear from an analysis of what happens with Fox News and Rush Limbaugh that people who are inside the Rush Limbaugh world actually are perfectly have access to and hear counter-arguments on the other side. They have a story that tells them that the other side is corrupt, untrustworthy. There's a liberal elite in charge of the newspapers, and it's all made-up fiction, right? So, an echo chamber in the original text and in my excavation of it is a structure where insiders are taught to distrust outsiders, and where everyone who doesn't share your core beliefs is an outsider, right? This has the structure, this should be a familiar structure, this kind of has the structure of a cult, right? This is what

cults do they teach people on the inside that everyone on the outside is malignant and out to get them. So, it's an isolation not of like avenues of communication, but it's an isolation of who you trust. So, one of the most interesting things about echo chambers that comes up in the Jameson capella is that echo chambers are almost always associated with conspiracy theories. So, there's a good explanation of this from a philosopher named Andre Begbie. Here's this great paper that I found really useful in understanding this called evidential preemption. And this is how it works. So, an evidential preemption, let's say I'm wrestling both right. Let's say for the moment pretend that you're my followers and imagine I'm Marcia Limbaugh and I say, See, you look, everyone out there in the media is out to get me. So, I make a prediction when you go out there and when you read other people talking about me. They're gonna say I'm a liar. They're gonna say I'm getting all

this wrong but that just a conspiracy out to get me your hero right so notice what I've done. I, Rush Limbaugh, have made a prediction about what other people are going to say, and then you go out there, and then you see that CNN says those same things about me, and my prediction is confirmed. So, you have a further reason to trust me. That makes sense. So, it's kind of, I mean, I think of just like intellectual judo. Not only am I like blocking incoming evidence, but I'm making you trust me more in that block, right? So, this is gonna be really important to just lodge this in your head. So, here's the question I had after I finish writing this piece we want to talk about today. Why are our echo chambers and extremely vicious political polarization flourishing in today's communication environment? This is a polite way to say Facebook and Twitter, right? Okay, what's the relationship between new media communication technologies and echo chambers? So, I left. I have no idea. It's a serious question for me because, as you can tell, echo chambers

they are cult mechanisms. We've seen them. They've been with humanity as long as we know, right? Why is this explosion now? It seems to be happening online, so what's the connection? And in particular, what is this? If you read, if you... Well, I was researching this. I spent a lot of time reading American alt-right websites like Breitbart, and I read the comments section every day for three months, and I actually wanted to die. I hate to stop for like put like emotional and health reasons, right? But you find this too, and the students, conspiracy theories, and moral outrage, right? In this particular note, I wanted to understand. By the way, I should also say I also think there are echo chambers on the left, and when you go into those, you find exactly that same mix. I'm not saying, by the way, the two are the same as far as I can tell. The is this enormous gargantuan mammoth, but they're there. They exist on both sides, and they hate seem to have that same to do in that same flavor. That's why I want

to understand. So here's our method. It's a method that people say comes from a philosopher, Dan Gilbert Ryle, but you actually find it in Schopenhauer. It's called creature construction. We're gonna imagine ourselves into the role of a large-scale belief manipulator. Okay, imagine you want to take control of people's beliefs, right, by injecting ideas into the discussion. How would you do it? So put on your most evil hats, and we're gonna look for techniques of belief manipulation. So there are gonna be three parts of this. Here are the three different streams. Trust me, they're all gonna sound different, and we're gonna get to the same place, hopefully. The first is false clarity. The second is porn. I told you we're gonna talk about porn, and if so, it's why you're here, right? We're gonna talk about porn in here. Philosopher talk about porn. I wrote a paper about porn. I wrote two papers about porn. Sorry, in games. This is, by the way, this is apparently my brand now. I've written papers about games and papers

about porn. So, this is what I do now: gamification of Twitter, and hopefully, these will all connect for you. So, here's the first idea about clarity and false clarity. This idea I've been having: the clarity sometimes is a thought ender. So, if you look at the history of philosophy and the history of a lot of what it is to be rational, you get these descriptions of these endless processes where you have to figure out everything and get to the bottom of everything. But we are not beings that can think infinitely; we are limited beings, right? We have a small amount of time, and so we need to manage our cognitive resources. We know we need to know where to spend our intellectual efforts or intellectual focus, right? So, we need a quick and dirty way of knowing when to end an investigation without having explored it all the way to the bottom, right? If every line of thinking you had explored to the bottom, then you would never get anywhere. You'd have to be like a philosopher, which we don't actually do.

things successfully right. Most people in ordinary life need to know quickly when an investigation is probably thought through enough, so we need to be heuristic. If you don't have this term, this is a rule of thumb, right? We need a quick and easy, quick and dirty principle that lets us figure out most things, not perfectly but quickly. So here's my proposal: we use a sense of confusion as a heuristic for continuing investigations, and we use our sense of clarity as our heuristic for ending investigations. That make sense? So when things feel clear to us, we stop thinking about them. So if that's right, there's a vulnerability here, and you can start, if you put on your evil epistemic manipulator hats, you can start to see what that vulnerability is. So if you're evil, this is what you should think: if you can give a system of thought a feeling of clarity, people will stop thinking about it. They'll stop investigating it, right? So you want it to feel clear. This is kind of a manipulation of attention, so this top

thought terminate heuristic. I think of something like an invisibility cloak for your brain, right? So here's a way to put it. John le Carré, when I give this lecture in America to American students, no one knows who that is. I'm assuming in here most of you know John le Carré is a spy writer, also actually worked in British intelligence, so he knew what was up. He said in his famous series and George Smiley, every book you would remind you the best spy didn't look like James Bond. The best spy was boring, normal, bland, something that your eyes glanced over, right? Something that said it was uninteresting, nothing is worth thinking about here. Similarly, stage magicians, if you know how they work, the way stage magician works is if the stage musician wants to do something down here, they make this hand look really interesting and flashy, and they make this hand seem really boring and dull, right? They know how to send signals of boringness and dullness. So the idea here is this, make sense? You, if you control these signals of

interesting, boring, right, confusing, and think about clear and settled. You can start to manipulate people's attention, so here's a test run. A lot of this is philosophy, but there's evidence from cognitive science that this actually already happens. So, there's this charm from cognitive science called processing fluency. Processing fluency is just how hard or easy it is for something you think about, something right. It's easy to think about, it's processing fluent. It was hard to think about, it's not. So, we use processing fluency as a shortcut to acceptance. So, psychologists have found that the easier task is to, if you look at a purpose like a proposed belief, the easier it is to understand the sentence, the more likely you are to accept it. And the harder it is to understand, the less likely you are to accept it. There's actually a reason this is a good rule of thumb, this is a good heuristic, because most of the time, things are easier to think about if you have experience with them, and things are harder to think about if you don't have experience with

them right, so it's a good starting rule of thumb, but it can totally be manipulated. For example, clear replicated results show that when something is an idea, first of all, if you see an idea, write a sentence you've never seen before, and you're asked to rate its reliability, how likely it is to be true, you'll all assign it something. And if we show you the same thing a month later, your belief in it will go up because it's familiar, right? It's easier to process. Here's another thing. This is the thing I find most terrifying. Almost everything is more likely to accept a sentence written in a legible font than a hard-to-read font. It is. It's easier to process, right? Seems so. It grabs this heuristic at bars. This can be gamed, right? Evil manipulators can take control of lists. This is what slogans are. This is what mottos are. This is what, in turn, even net memes are, right? They work because it's presented in a familiar format. That familiar format gives you a

reason to accept it. So, next step, that's the first little warm-up. So, let's be hostile belief manipulators. We want to create a system that maximizes the feeling. I'm sorry, this is slightly out of order because I'm dumb. I just had an incredible idea this morning and tried to rewrite part of my talk, so this is not something you're supposed to do. So, to understand this - so what I want to hack if I'm an evil manipulator and I want to give you a sense of clarity. I want to hack your sense of understanding things, right? I want to make it feel to you like you understand something even if you don't understand it. So, what is understanding? There's a really useful literature from the philosophy of science about what the nature of understanding is. And a little bit of background, most philosophers have been really interested in the nature of knowledge, which tends to be understood as being sure about individual facts. And a lot of philosophy of science people say like that's not what we actually care

about we care about something more important. We are about understanding, which is something broader, which is AK about a relationship between facts. So you're sharing some theories out there. Katherine Elegans says understanding is grasping a coherent relationship and not just seeing the individual facts but how they fit together, right? John Come on Vague says that understanding is facility with the terrain. So it's not just having individual facts but able to move between the parts and the ability to accommodate new information. And Michael's Trevon says also when you have understanding, you can communicate it, right? You can get that understanding to other people. So here's the phenomena of understanding. So if we want to be hostile, minted belief manipulators is what we want to do. We want a Mac create a system that maximizes the feeling of things sliding into place and the coherent network maximizes the feeling of seeing connections, maximizes the capacity to generate some kind of explanation, and maximizes the sense of communicability. Make sense? So and we evil manipulators, even those are kind

of hard to do have an advantage. People who are trying to get genuine understanding have to actually pay attention to what's true. We don't, right? We can engineer our system for like maximum, the maximum feeling of clarity. They say they actually comes from my mother adapted a little bit. My mother used to tell me that I liked fast food more than home-cooked food, but that's because the fast food people didn't care about my health, so they just maximized deliciousness. But if you care about health, you can't maximize deliciousness. That's the same idea, right? The make sense. If I care about truth, I have to balance things. If I just want to get you on my side, I can just make it maximally brain delicious, right? Make sense. I can pursue the scent of understanding without actually pursuing real understanding. So here's one obvious case study. I'm not going to talk about this lot because I think it's super obvious. A Rush Limbaugh style echo chamber that has a conspiracy theory maximizes this sense, right? If you have a conspiracy

theory. It explains everything in terms of one great global conflict in which all these people are out to get Rush Limbaugh, right? Then you can explain all these complexities in the world, and it's not just that you have a place for them; you have a robust explanation. Usually, like the reason there's this disagreement, it's not mysterious; it's because the New York Times is in the grip of this cabal of people that are really, they're run by Hillary Clinton, right? So you can give explanations, and typically in conspiracy theories, everything is connected, right? And if you make instead of a thousand different forces going on in the world, but one central force going on in the world, you can have the feeling of it creating explanations for everything. And notice that the conspiracy theories that you find in Rush Limbaugh type circles, it's really easy to generate explanations, right? So make sense. So I think it's really obvious here that echo chambers create this feeling of clarity and get you to stop thinking about the theory. Okay, I wouldn't do to all the hate this for a

second because you all probably think you're not believing in one of these, and then we'll give you something else that I think you may be part of, that I'm certainly part of. Here's another example of false clarity. I think Bureau traffic metrics, indicators, and quantitative assessments also give you the same false sense of clarity. So, there's this fabulous book from Theodore Porter, who's a history of the culture of quantification, and you all should read this. This book felt to me like an unlocking of everything that was interesting and important about the world right now. He's a book called Trusted Numbers, and he says that there are two ways of knowing: qualitative and quantitative. He thinks that both are important, but he thinks we've radically overvalued quantitative ways of knowing, right? And here's his explanation. He says when you get a transition from qualitative to quantitative ways of knowing, what happens is qualitative ways of knowing tend to be rich, nuanced, and sensitive, and quantified ways of knowing strip out

richness of information, nuance, and context, civic sensitivity, and make information that's portable and comprehensible in radically different places. Right, so here's an example. This is, I think, my single favorite example ever. You know, in England, we used to measure land in the hide. Get rid of the hide. A hide is the amount of land required to feed an average family and keep them alive. Okay, notice that hides. Can you feel this now? Notice that hides are radically sensitive and changing to local environments, right? A hide near a river is really small. A hide in the field is medium-sized. A hide in a desert is large. It's also sensitive to fluctuations. So, a field, this particular part of the woods, might be flushed most years, but we know one out of every ten years it drops to nothing, and a family would die, right? So, a hide measures this, but in order to govern a hot, the only people that who know who the hide is are the people who live there, right? Central bureaucracies can't figure out what hides are. So, when we start getting

centralized governments, and we need to manage land from the center. We move towards the acre. Notice the acre loses all functional sensitivity, right? Loses all the rich contextual knowledge. Loses the most maybe important thing to know. But what it is, is it's easily aggregateable by anyone, right? Anyone can build a map with it. Anyone can use it. Sänger salary Engelmayer has another fabulous book, which is basically an extension of that to the future. It's called the Seductions of Quantification. She was a part of several UN teams that made various, you know, things like the Human Development Index that assigned a number at every country based on its degree of human quality. She was on the commissions for this, and then she turned an anthropologist's eye to trying to figure out how and when these metrics got made and why they got made. And one of the things she found was once you get these rankings, they get stuck in everybody's brain, and everyone just starts using them, right? The thing they care the most about is the place of their country on the rankings. She's

amazing definition. She says indicators and metrics. This is her definition of the idea: present a false front of authority by hiding the complexity and subjectivity of their manufacture. Right, what it is, a creative metric is you have a thousand pieces of information that you interpreted, and what you output is, here's a single number, right? And it's vastly powerful because once you have that number, you can create a ranking, right? You can see exactly where everything stands. So, I think it's helpful here too for me. I always think about this in terms of the educational system because I work in education. And I'm not sure about you, but in my educational system, we're getting a march short quantification more and more every year. The newest march is that every budgetary request from my requirement has to be justified by quantitative data about how we'll help student success, or student success is defined only as graduation rate. Okay, so I want you to imagine for the moment the difference in a GPA system in education and one based on qualitative. Imagine I someone told me

that GPA is an American term. So, what work happens in America is everything. It's a letter grade: A minus, B plus, B, and each of those is translated to a number. So, you know, an A is a 4.0, a B is a 3.0, and then we have a number, right? And then you can advertise. So, the GPA stands for a grade point average, which is the average of all your scores in all your classes. So, everyone comes out of college with the GPA. Do you have something equivalent here? I was told the GPA is like weirdly American anyway. So, when you come out of college, you have a GPA. Let's think that represents your entire college experience, right? Same with high school. When you apply, the most important thing is your GPA. So, imagine the difference in a GPA system and education and one based entirely on qualitative evaluation, right? The qualitative evaluations you can make rich and context-sensitive. I could tell you, student, what they needed to work on, what they should be proud of. I could keep in mind what they wanted to

do in their life right, but that couldn't be aggregated, couldn't be overviewed by administrators, right? I couldn't summarize it quickly if I was an admissions officer, so we need GPAs instead. So, what GPAs give you is easy cognitive facility, a sense of power and understanding, right? Because what you can do with them is you can aggregate a student's lifetime performance into a single number, you would aggregate a school district's performance into a single number, you would set cutoff points for eligibility. So, if you look at how GPAs get sorted, right, if a bunch of people are applying to one school, that school will just organize them in a spreadsheet by GPA and assign an arbitrary cutoff and then just not look at anyone below that arbitrary cutoff, right? That gives you an enormous sense of understanding. You can manipulate the element, order them, and see them connections instantly and instantly rank things. There's this incredibly interesting book called The Engines of Anxiety by sociologist named Wendy Iceland and another sociology saint-michel Sauter, and it's you know

the US News and World Report's rankings of law schools. So, there's a ranking of law schools by number. It charts what happened to legal education when that ranking showed up, and the answer is it destroyed it and the profession may be right. So, what are the things she thinks, and I think this is super interesting. Sorry, mate, this is an important example. So, um, she says before that every school had a mission statement, and the mission statement tented goal, and a lot of different schools could have different goals, right? One school could be, We're gonna create top corporate lawyers. Another school could be, We're gonna create lawyers engaged with the community, right? And with good moral standing engines of it. When the US News & World Report comes along, the things that cares the most about are entering GPA and LSAT scores of the students and legal employment rates when the students leave, right? So, what happens is any student that admits people on any other basis drops in the rankings. So, they say when this came around, it created an enormous value realignment.

where law schools were no longer allowed to have different missions with different values, right? Instead, they had to conform to whatever values were behind the rankings, or they would lose students and lose funds. The reason the US News and World Report is so satisfying, so catchy, is before it, students had a really hard time figuring out what school they would go to, right? But now, instead, there's this easy, legible ranking that presents an authoritative, as finalized, as understood, simple ranking like this. Okay, so here's the worried US News & World Report rankings. Get students to stop considering their own values in thinking about the wide variety of values of law schools through a show of false clarity, a ranking. So here's the idea I want you to concentrate on going forward as I've been packing your brain with weird stuff. Quantitative systems produce a sensation of clarity by discarding information in nuance and focusing on only some features that make sense. So if you cut away a lot of the nuance, then it's much easier to

stack up data together and aggregate it. You get a feeling of power, but only when you discard a lot of information. So, to sum up part 1, the sense of clarity can be used in thinking. It's a useful point of vulnerability for Thought hacking, and there are two methods: oversimplifying explanations and oversimplifying numerical representations of complex data, right? I just want to be clear here. I'm not saying numbers are bad or not saying science is bad. I'm saying some things are easily measured, and some things, like educational equality, are really hard to measure. But we're like too attracted to hyper-simplified measures of it. That's part 1. Part 2, what you've been waiting for, porn. So, this is from a paper I co-authored with Becca Williams. There's also, by the way, I'll tell you later, we also wrote a New York Times about it. You can read. So, we were really interested in what started as, I will admit, a dumb drunk Facebook conversation between me and Becca on someone else's

thread about the definite. You know philosophers love to define things, and we were drunk, and we were just trying to find porn in the generic sense. This means - there's this new use of porn: food porn, real-estate porn, closet porn. My wife loves this. Let me give you some examples: food porn. My wife loves this site called things neatly organized calm, and whenever she feels stressed, is it just like thousands of pictures like that, and everyone on the site calls an organization porn. Poverty porn. While we were reading the paper, a comedian Saturday Night Live was like, I'm sick and tired of all this impeachment porn, and we were like, We know what you mean, right? So there's this new sense of porn that we're all using. We all know what it means. What's the definition? So we borrow from Michael Ray's definition of sexual pornography. Here's the actual definition. I'm not going to read this. I just want to show it to you so you can be impressed by how technical it is, but the short version is Michael Raines.

definition of sexual pornography is that to use something as sexual pornography is to look at that thing's represented sexual content for immediate sexual gratification without interest in personal intimacy. That makes sense, so we thought that sounds right. Let's generalize it. Here's our account of a generic porn: you use something as ex porn for any X when you use a representation of X for immediate gratification, freed from the typical attending costs and consequences of actually engaging with the real thing, like tense. So, food porn: you get immediate gratification from a representation of food, the cost and effort and calories of actually eating it, right? Real estate porn, which is really common in the West Coast. Okay, kids know we can afford anything, probably London - I'm sure London tears real estate, getting the immediate gratification of representations of real estate without the cost and effort of buying and maintaining it. Makes sense. Okay, if that's right, we should be able to see new version - porn. Here's one: moral outrage porn. Moral outrage porn is using a representation

of moral outrage for immediate gratification, freed from the costs and consequences of actual moral engagement. So, actual and moral engagement includes trying to get things right, trying to be nuanced, being attentive to the truth, acting on it. If you can't tell, I think most of online life is moral outrage porn. Does it make sense? Okay, so I want to be really clear. Every given this talk, someone misread this. I'm not saying moral outrage is bad. I'm saying moral outrage porn is bad, right? If someone says sexual pornography is bad, they're not saying sex is bad, right? So, I am NOT saying there's this like Jonathan Haidt, he's a, you know, American figure, is like there's too much outrage, ah, like around the world, we just need to be civil and respect each other. I'm not saying that at all. I think genuine moral outrage is important. It's crucial. And it's because real moral outrage is so important that the porn version is particularly terrifying. I think the porn version undermines the function of the genuine thing. So, moral

outrage porn involves using your moral outrage for the sake of immediate gratification while avoiding the demands of actually good action. This makes intuitive sense. Can you kind of feel like what this would be like? And I just want to be clear, not all porn is bad, right? I don't think there's anything wrong with using a representation of food for immediate gratification, but there is something wrong intuitively with using a representation of moral outrage for immediate gratification. So, what's wrong? One way to put it is that there's something wrong about using your moral beliefs for pleasure, right? That's not what moral beliefs are for. Let's get this okay. So, I think the deeper danger is the use of moral outrage porn invites us to instrumentally alter our moral belief systems and moral life for pleasure. What this means is instead of trying to get your moral beliefs to match what's morally true or good, you move them around to maximize the pleasures of moral outrage for right. So, how would you change your moral belief?

system to maximize the pleasure of it. If you didn't care about getting things morally right, think you would simplify it, right? If things are complicated, you keep having to pause, you don't get as much moral outrage, and you don't get it with much of that pleasing sense of security. And you'd want to surround yourself with input that maximizes the pleasure, like obvious portrayals of on the other side and lots of people on your side wrong you on right. That would increase the pleasure. So, using moral outrage porn invites us to, and this is a technical term that I'm making up, I guess, instrumentalize our values and moral beliefs, right? It asks us to change our moral invites us to change our moral beliefs and simplify them to increase the pleasure we get from engaging with this pornographically. So, think about the difference. If you actually wanted to get your mo beliefs right, you'd be sensitive to nuance, you'd be open to the possibility that you're wrong. If you were just out to maximize the pleasures of moral outrage porn.

you'd want a simple belief system that was easily triggered and constant exposure to simplistic portrayals that reinforce the belief system while triggering the pleasurable outrage. I'm just hoping that this is seeming familiar to you from your experience of looking at people talking online. So, let me put together parts 1 & 2. Let's say you want someone to accept a system and then stop thinking about it. You would want to create a belief system that condemned in clear and simplistic terms the other side in a way that made you feel smug and superior, and at the same time offer a clear and usable explanation of what everyone else was organized to get your group right. If those things go together, you got something that's pleasurable and payment for simplifying your moral system, and then you get a sense of clarity so you stop worrying with the system. That make sense so far? Hopefully, the two parts are fitting. Okay, part 3. We're going to take another tangent, hopefully, I'll bring it back for you.

games and gamification. So this is okay. I'm gonna advertise for a site. This is my book Games Agency as Art. It's coming out from AUP in March 2020. This is seven years of my life in a book, and it is about why games are a beautiful new art form. If you understand why, you'll think gamification is terrible. If you don't, wait for the book. There's a paper version that's out already. Yes, take a picture by the book. The first chapter is available on my website. I'll tell you about that later for free. So, one of the basic ideas is that games, for me, are temporary environments with temporary goals. You enter an alternate environment, and you pick up new goals while you're there, right? I mean, there's this basic idea: my wife and I love each other, and then we finish dinner. Will you pull out a board game? Then we try to kill each other for an hour, and it's really nice. And then we put it away again. So, I got into this project because I was teaching.

some philosophy of art about games, and I read all the stuff about games as an art form, and I didn't believe any of it because it all talked about games as a kind of fiction or a movie, and that didn't seem right to me. It seemed like that was just the application of an old theory, so I went looking for a theory about the artistic medium of games. So, what's the meat like? You know, oil is the medium, oil paint, watercolors, the medium of water coloring. What's the medium of games? First thought, drools, right? Can't quite be right because this is a game, this rock, this bouldering, this is what I do, and a lot of what makes the game good is not just the rules or the physical actuality, right? This is another, this whole talk is just me excuse to talk about this example. This is an indie game called pew pew pew pew pew pew pew pew pew. You control this, you're moving this way when you fire your jet, you go up, and then when you're not firing, you

drop and you shoot forward, and you have to blow things out of your way or dodge them. You don't play with a controller; you play two people together with microphones. One controls the jet pack by going, and the other controls the laser by going. Give you, and the hardest part is you start to laugh wildly because you're supposed to play onstage in front of an audience, and you can't make the noise you like. Mouth dries from spit anyway, and that's obviously what's really important. There is the ambience of the room, the microphone sensitivity, how dry your mouth is, right? It's not just the rules, is it? The obstacles, that's not quite it either. I think Reiner Knizia, who is a great German board game designer, had it closest when he said this: A scoring system creates the players and motivations in the game, right? So one way to put it is the game designer creates the goals of your in-game self. In the book, the way I put it is the medium of games is agency. That make sense? The game

designer creates abilities, and they create goals. They give you a different agent, and you enter into it, and you find out, you find out, oh, in this game, I'm cooperating with all my friends to comprehend emic. In this one, you figure out, oh, I'm on this team, and we're trying to be that team, right? You find out new goals, and you find out what abilities you have. So, I'm gonna try to connect it to you now, okay? One of the reasons I think this is so appealing is because life is terrible in a lot of ways, and games aren't terrible in a particular way. I think one of the ways that life makes us suffer the most is that what we're trying to do is often very unclear, right? Values are unclear, there're STUV all, it's unclear what the right values are. Second, their application is often unclear. Third, there's like a thousand values, right? And it's hard to trade them off, and it's hard to see how they balance, and then there's a billion other values.

from other people. So how the hell do we build a society in games? There's one goal; it's quantified. You know exactly where you stand. You know that everyone else is pursuing the same goal by the same means. That makes sense. So I think life is kind of this existential value torture, and games give us a little bit of oasis from that. Make sense? We're a little bit, you know, exactly what the point is, and the reason is you're offered a simplified, quantified goal that tells you exactly how well you're doing and that it's easily applicable to the world, right? Other pieces are ting to come together a little bit. Okay, so games tell you what to care about, and we've developed this enormous capacity, anyone who's played gay, to do this thing where you read a sheet of paper, and it tells you that something is worth points, and you just care about that, right? So my worry is that can be good in games, but when we do that on Twitter, we destroy society. Okay, in games, I think it's okay because the

games, the goals and games are temporary, disposable, right? The point doesn't connect to anything else. In a lot of games, the world is temporary, but in gamification, you simplify goals in actual life. So, gamification, here's my worry: gamification threatens to change the target. When you gamify something in the world, like education or Twitter, when you give people points for engaging in public discourse, right? You might suddenly, instead of caring about getting things right in public discourse, you might suddenly just care about doing whatever gets you the most points. And we know what gets you the most points on Twitter: it's moral outrage. Content with moral outrage gets four times as much attention and spread as any other content. Well-measured result. So, consider a phenomenon I've been thinking about a lot: call it value capture. Value capture is what happens when you have rich and subtle values. You're placed in a social institutional system that offers you simplified, typically quantified versions of those values, and then everything turns to crap. Here are some examples: academic assessment, grades.

retweet numbers, Facebook Likes, numerable wines, scoring Rotten Fitbit numbers, rotten tomato scores, maybe money. So, like going into school interested in learning but coming out caring about your GPA or the ranking of your law school, or starting to exercise for your health but then just getting obsessed with maximizing your Fitbit step numbers. I just saw someone nudge their partner or your BMI, trying to control your diet for health, beginning obsessed and said with lowering your BMI and weight. Starting to tweet for the sake of truth, justice, and conversation, and ending up just doing whatever makes you go viral the most. Those are all value capture instances, by the way. I just want to say a lot of the philosophy I do is very personal, and I'm a former game addict, and I can just feel the same systems and Twitter just being like, I'm gonna level up, I'm gonna pass into a thousands of followers now. That's sorry. So, notice in these cases there's a game scoring mechanism. We're offering you a clear quantified measure of some dimension and

you tend to become motivated to change your values towards it because it makes the activity of pursuit clearer and crisper and more pleasurable like a game. But to do that, you have to simplify your values. So Twitter, I've already talked about this enough, you understand why I think Twitter is like a gamification of public discourse. So gamification also threatens to get us to instrumentalize our values, right? If you simplify your values and you start caring about something attached to points, you'll feel more like a game and it'll be more pleasurable and things will be more essentially clear. Then you actually have to cope with the hideous existential nuance of the value horror of the pluralistic world. So another way to put it, in games we put value blinders on, let's make sense. Like in games, you just care about, you don't care about anything else, stop getting points, right? We're trained in games to just care about whatever it is that gives you points. You care about victory conditions and nothing else. Students have a game that just gives you

relief from horror. But if I was a belief manipulator, I'd want you to have value blinders on, right? I'd want you just to care about the thing that helped my side, and I wouldn't want you to pay attention to the other stuff. An easy way to do that would be to attach the values I wanted you to have to a quantified point-like system, right? Okay, so let's put them all together. If you want a system to build a belief system, you're a truth manipulator, a belief manipulator, and you want to build a system to get it to spread and stick to people. Here's what you want to give it: easy explanations for as many phenomena as possible, that's false clarity; a simplified moral system that condemns everybody on the outside, that's an echo chamber; you want to gamify it so people will focus on a narrow goal and get pleasure for doing so, that gives you the clarity of numbers, connects to the pleasure of games, and then you put it.

all in an echo chamber. Here's another way to put it, and I'll be honest with you, I've been thinking about this a lot, and I came up with this way of putting it today at 10 a.m. while drinking a cider. It's Jon's, so we'll see if this works. There's one very food-spattered page. All these systems reinforce and encourage narrowness and moral vision, right? You just, they get you to care about a small number of things, and they try to make it in a very near-honest is sticky, right? If you're open in nuance and you see the world in all its complicated ways, you have lots of counter-evidence, and you have to revise your values. If you have this trainee explainable, well-engineered system that is sufficiently close to the outside, they'll be trapped in it, right? It's kind of a belief system trap. So, so here's how it works. Moral outrage porn gives you pleasure for adopting a simple system. Numbers hide nuance and make the measured thing most salient, right?

gamification gives people pleasure for adopting the quantified and simplified goals, and echo chambers dismiss complexifying new ideas from the outside. If you put these together, this feels to me like my online experience; this feels to me like what I'm seeing in the world. So, when you put these all together—seductive clarity, moral outrage porn, gamification, and echo chambers—what you get is systems that give you pleasure as a reward for adopting the moral and value systems, and systems that are designed to be sticky by narrowing your vision so you just focus on what the system cares once you to care about and nothing else. Their techniques were engineering narrow-mindedness. Thanks. [Applause]

C. Thi Nguyen: The Gamification of Public Discourse (Royal Institute of Philosophy)

The Royal Institute of Philosophy

The Impact of Echo Chambers on Morality and Values

Echo chambers and filter bubbles are distinct concepts, with the former teaching individuals to distrust outsiders and the latter isolating them from opposing viewpoints.

The rise of online communication technologies is contributing to the flourishing of echo chambers and political polarization.

These echo chambers often promote conspiracy theories and moral outrage, creating a sticky narrow-mindedness and simplifying complex value systems.

The method discussed is called creature construction, aimed at manipulating people's beliefs through idea injection.

False clarity is identified as a key technique, where a sense of clarity leads individuals to stop thinking and investigating further.

Processing fluency, or the ease of understanding an idea, is also a powerful manipulator, as it can influence acceptance and belief in the idea.

The speaker discusses the concept of understanding and how it can be manipulated for one's benefit.

They highlight the difference between knowledge and understanding, emphasizing the importance of grasping the relationships between facts.

The speaker provides examples of how false clarity can be created, such as through conspiracy theories and the overvaluation of quantitative metrics.

The Impact of Quantification on Governance and Education

Quantifying metrics, like the hide and acre for land measurement, shows a shift from local sensitivity to centralization.

Creative metrics, such as country rankings, present a false front of authority by simplifying complex information.

In education, the increasing reliance on GPAs for student evaluation provides cognitive ease and power but overlooks the richness of qualitative assessments.

The Impact of US News and World Report's Law School Rankings

The US News and World Report's law school rankings disrupted legal education by prioritizing GPA, LSAT scores, and employment rates, leading to a uniformity in mission and values among schools.

This ranking system created a false sense of clarity for students, making it easier to choose a school while discarding the nuance of different educational values.

The concept of "porn" has evolved to include various non-sexual representations, offering immediate gratification without the real-life costs and consequences, leading to a new understanding of moral outrage porn as well.

Moral outrage porn uses moral outrage for immediate gratification, avoiding genuine moral engagement.

This phenomenon undermines the importance of real moral outrage and can lead to a simplification of moral beliefs for the sake of pleasure.

The desire for pleasure from moral outrage can create a distorted belief system, reinforcing simplistic views and triggering pleasurable outrage, ultimately instrumentalizing moral values.

Games are unique in their artistic medium, with agency being the key factor that differentiates them from other forms of art.

The game designer plays a crucial role in creating goals and abilities for players, offering a clear and quantified purpose that contrasts with the complexities of real life.

While games provide a temporary escape from life's uncertainties, gamification in the real world can lead to a misguided focus on achieving points rather than meaningful engagement.

The Impact of Gamification on Values and Beliefs

Gamification in social media and other systems often leads individuals to prioritize quantified measures, such as likes and scores, over their original values and intentions.

This narrow focus can create a belief system trap, where individuals become obsessed with achieving specific goals and neglecting the complexity of the world around them.

The combination of seductive clarity, moral outrage, gamification, and echo chambers reinforces narrow-mindedness and simplifies moral and value systems, making them more appealing and sticky.