> This "pulling balls from an urn" analogy at the beginning is very misguided. Where is the agency in this analogy? If you build tools with the intention of them directly being useful to people, they're likely to be useful. If you build tools to be parasitic (i.e. useful to people who are trying to benefit at the expense of others), they're likely to be parasitic. It's quite simple.

I find this very naiive, as this at best only applies on a short timescale and for the immediate effect.

History is full of inventions that were created to help people, that is then quickly turned into something harmful, and vice versa.

Also, a lot of "progress" has come as side effects of pure curiosity, espcially within science. I don't think anyone would be able to build modern electronic computers, if it wasn't for some other people a few decades earlier doing basic research in fields such as electromagnetism, quantum physics or chemistry.

I agree with your point about scientific breakthroughs coming via pure curiosity. I’m just saying that we can’t forget the human agency at play here. We didn’t have to build the atom bomb, and it wasn’t exactly simple to turn insights from physics into the atom bomb (it took a team of the best scientists and technologists many years). Also, we can influence the direction of how technology is used. Even setting up the atom bomb to be used involves a ton of decisions and implementation that can be questioned and molded by individual people.

Essentially I believe that fear of powerful technology comes from an irrational distrust in people as incompetent or malicious. Same with desire for powerful technology to control or replace people (e.g. algocracy). It seems to reflect a lot of the rhetoric and motivations of the eugenics movement. But the problem is, it can perhaps be a self-fulfilling prophecy. People are capable of being incredibly competent if encouraged to do so. So I think we should go in that direction, of empowerment and freedom rather than control or replacement. And if our governing structures or institutions or culture are making it difficult to go in that direction, we should fight those.

The Manhatten project was started after Einstein (a pacifist) urged Roosevelt that Hitler could build it first.

https://www.osti.gov/opennet/manhattan-project-history/Event....

At the time Hiroshima and Nagasaki were bombed, about 50000 Chinese people died every week due to Japanese occupation. If the alternative were an invasion, those two bombs probably saved millions of lives.

https://worldpopulationreview.com/country-rankings/world-war...

Von Neumann lobbied to make sure that USA/USSR would end up with a situation of Mutually Assured Destruction. He believed that would be the best way to avoid large wars between the Soviets and NATO.

In all these cases the development/use/threat of nukes were done with, arguably, the best of intentions. In their minds, they were employing agency to help protect the world.

I would argue that the eugenic movement was the same. There may have been some malice, but there was definitely many eugenicists that truely believed that eugenics was the only way to prevent dysgengics, followed by a malthusian collapse.

One could even argue that Hitler saw himself as the hero of Germany, the one that would save Germany from enemy "tribes", foreign and domestic (jews). If so, he may have seen the Holocaust as a heoric and brave act, and it was certainly executed with extreme competency.

Mao's cultural revolution, Pol Pot Cambodian genocide, and the Hutu's genocide vs the Tutsies were all very similar to the Holocaust, even if less known. In all cases, a group of Others were seen as a threat, and eliminated.

In most of these cases, I think the perpetrators saw themselves as the heroes, almost like the heroes of Minas Tirith fighting the hordes of Mordor.

It is extremely hard to come up with a way to live in the world that minimizes harm. And, I think, it is those who has the strongest conviction that they are the Good (TM) struggeling against Evil, that has the greatest tendency to cause harm, as they are convinced that the end justifies the means.

Yeah I agree that no one acts in a knowingly malicious way. But considering for example racists or eugenicists, it's important to note that no one is born a racist or eugenicist. Those mentalities are always either (1) something other people convince them is the right mentality or (2) a kind of coping mechanism, a way to maintain our belief that we are good or we are competent in the face of some negative signals that may say otherwise.

Thus, given the fact that people do change (otherwise they wouldn't have changed to such an unnatural state to begin with), we should rely on the agency of people calling other people out and people being able to increase their self-awareness and learn from their mistakes. I mean, if you disagree with someone, you don't have to convince them, you just have to convince others to join your side. People with different beliefs standing up for them is what creates balance in the world.

Of your above examples, I also agree with your point about it always being some group vs the 'others'. But again, none of those people were born with those desires to harm others. It was something developed and something unnatural, likely the response to some negative events. But I don't think we shouldn't distrust human agency altogether just because some people use theirs to cause harm. And we don't have to calculate precisely the total benefit or harm of our future actions in order to have permission to express our agency. I guess it's really about reaching a state of self-awareness to identify if we're acting in an authentic way.

Coming back to technology, I'd say with people that want to automate away or control people, they should reflect on why they've come to distrust other people in general. I mean they certainly trust themselves to try to automate away or control other people. Why not trust others similarly? What events or people in their life led them to that mindset?

> it's important to note that no one is born a racist or eugenicist.

I would agree about eugenicist, that one requires some understanding about heritability.

But for racism (or similar prejudices based on gender, sexuality, etc), my personal experience, with my own children, indicate that the tendency to group and judge people by their appearance is something that kids to automatically around age 2-3, and that they have to learn to stop. (Kind of like they automatically learn to lie and fight, and have to learn to stop doing that)

And since my children are girls with brown skin in an environment where more than half the kids are white, they focus(ed) a lot on their own skin color, and how that makes people different, just as they do about gender and gender expression. Simply by living in the world, they pick up gender and gender expression. If they see violations of gender-typical behaviour or expression, their first reaction is negative, and they have to be told that it is ok for a boy to hit a boy.

As for the racism part, luckily I have only seen mild expressions of that beyond the basic tendency to use skin color to categorize people. There was maybe one occurance a few years back with my oldest daughter, which made me sit down with her and explain what racism is and why it is not ok. (When she understood that racism could be used against her mother and herself, this was pretty easy)

I think if you travel a bit around the world, especially outside of Europe and the Americas. If you go small towns and villages in Africa, India, South-East Asia, China, Korea and Japan. If you get to know the locals there, not as a tourist, but on a more personal level, you will find that in most places, some kind of racist thinking exists almost everywhere.

If you travel with a person of a different race from yours, it will become extra obvious. Lets say that one of you is white, one is black, asians are likely to treat you quite differently. In some places, there is strong prejudice against Indian people, in some place prejudice agasint Chinese people, etc.

Now if one of you is openly gay or transsexual, the experience would be even mor extreme, an might turn dangerous in some places. In other places (like Thailand) a transsexual person might receive even less attention than in the most progressive places in the US, though. (In Thailand, it is so normal to be a trans-woman, that coming out of the closet is not seen as an achievement. So even the positive feedback that a trans person might get around progressives in the West would be gone. The thais would just instantly accept that the trans woman is socially a woman.)

Oh, and if you think that all those slightly racist or sexist african or asian people are evil people just because of the racism or sexism, maybe you need to look in the mirror.

They simply live in a place that has never had the kind of inclusivity movement that western countries have had after WW2. They are not evil, they simply live in a culture* where some kinds of prejudice is still the norm. Get to know them, and you will find that many of them are still good people with good intentions and kind hearts.

* Not saying that all asians are the same, I mean if you do end up in some village somewhere where the local culture tolerates or even encourages racism or sexism, that village's culture is what I refer to.

Ok point taken. But above you were mentioning examples of genocide…. I guess we could decompose something like racism (which I guess extends to other ways of categorizing in-group vs out-group) as (1) natural tendencies to group and make judgments based on groups, which as you mentioned certainly exist, (2) media/culture that shapes us on top of our natural tendencies, and (3) the desire to harm/fight/destroy other groups (which is probably a spectrum of intensity).

I guess that (3) is the part that I was saying is often unnatural. (And (2) also of course. I mean it’s clear how Hollywood is a racist propaganda machine for example…) But I guess with genocidal maniacs (or people trying to institute mass control or automation away from human agency) that’s really a question of (3). And of course, there are I’m sure events in those maniacs lives that led them to their views. But certainly they’re not seeing the whole picture, certainly they’re acting out of fear, or to maintain a sense of self-worth or as some other kind of coping mechanism.

Unfortunately, not only do I belive that your (3) is HIGHLY natural, I believe it predates humans by 10s of millions of years, if not more.

Now ask yourself. Do you consider any group of people to be evil, just because they they belong to that group? Maybe the MAGA crowd or Antifa crowd? Maybe just FOX employees or CNN employees? Maybe some group of rich people or maybe some group of poor people, such as immigrants from some specific country? Maybe Christian or Islamic fanatics?

If there is any such group you tend to see as Evil just because of their group membership, you are already at risk of ending up with the us-vs-them mentality that is the prerequisite of (3).

I freely admit that I'm (at an early stage) of such a state now, as we speak. In the Ukraine vs Russia conflict, I cheer for Ukraine. That means that I don't feel too much compassion for Russian soldiers that die. It is not important enough to me, that I really think that those Russian soldiers are Evil, but it is enough that I've lost most of my empathy for them, as long as they keep fighting.

Whenever some group that you see as Other start to become a threat for some group or some goal that you attach high value to, there is a tendency to start to dehumanize that group of Others. If you think they are Evil, that means the process already came quite far. Right after evil, is to see them as not deserving the protection normally given to humans. Effectively, they become orcs, or virmin, to be exterminated.

This may be hard to comprehend for someone raised in an affluent, peaceful environment. But once there is a really serious conflict of interests with another group, things change. There are a few factors that can massively expedite this process, such as: 1) People that experience hunger/famine seem to often become extremely hostile to competitor groups after. 2) People who experience (relative) powerty tend to have similar, but weaker reactions. (Think, living through the financial crisis in the west). 3) Real hostile actions perpetrated by members of the opposing group. (Ie George Floyd, the 2020 protests, January 6, 9/11, just to take some American ones). More serious ones are war crimes perpetrated by Germans in WW2 or Russians in Ukraine, if you identify with the victims. 4) Real or perceived threats from the opposing group. (Like Russia threatening to use nukes, or (from the Republican perspective) Democrats opening to borders to get more Democrat voters) 5) A subtle feeling of inferiority. The best example is how wealthy Jews were hated in Europe prior ot WW2, and especially in Germany. Especially when the Germans were living through hard times in the 20s and early 30s, the wealth of the Jews at a time when ethnic Germans were poor and starving, was a to the story that Germans told themselves about being a superior race. Only by defining the Jews as Evil, could the Germans retain their sense of dignity. Because the alternative was too painful. Maybe the Jews were in some way superior to the Germans?

But more than anything, I believe the main cause of these kinds of unreconcilable conflicts, is when we think that we are the Good Guys. The only way to maintain that belief, is to always identify our competitors, adversaries or enemies as Evil. To think that we are willing to fight in a conflict vs some other group just to further our own interest can be too much of a challenge to our self-perception of the Good Guys. Much easier if the enemy is Evil, and not really human.

There is an alternative, but it is a difficult one. Imagine that a lion is trying to kill your child or other loved one. You have a gun. What do you do? In principle, you may be an animal preservationist. But for YOU your loved one is (most likely) more valued than some random lion. You don't need to think of the Lion as Evil, and you may even feel sad as you pull the trigger. Such a loss! Maybe even if you miss, and the Lion kills your loved one, you still don't hate it. Maybe you even let the Lion escape with the dead body, as it is already too late to save your loved one.

Imagine you are a soldier in the trenches of Ukraine (on either side) and you are loyal to your side. Instead of seeing the enemy as Evil, you still think of them as just as human as yourself, just with other values and interests. As you kill him, you are sadedned with empathy, but you still kill him, because the things you are fighting for are more important to you than his life. Maybe the guy next to you cheer, as if the person dying is a good thing, and maybe that makes you even more sad inside. You are not a pacifist, you may be the best soldier in the squad, but you are also not driven by hate. When the enemy surrenders, the guy next to you may want to shoot them, but you make sure they are taken prisoner instead.

This approach is much harder than seeing the opponents as Evil. It requires a higher level of maturity. But after the conflict is over, not only do you not face war crime charges, you may be more likely to be able to live with what you have done. The guy next to you may have nightmares about his crimes until the end of his days.

I wholeheartedly agree that there are times that call for defending your in-group against an out-group. It could be fighting for scarce resources or when your lives are on the line. But do you also agree that there are times where it is out of place? The unabomber perhaps? I mean he was a bit of a one-off (a group by himself). But what about the early eugenicists, for example, was it called for or out of place?

A lot of the time behavior is out of place, at least when measured by my personal set of values. The worst examples of this, in my opinion, tend to happen when some group start to think of themselves as fundamentally Good, and their competitor group as fundamentally Evil.

Another factor that is almost as dangerous, is when some group (or a powerful individual) lose touch with reality. Lets say Putin really thought that most people in Ukraine want to "re-join" Russia, and that only a small group of "Nazis" prevent that.

Then there are individual episodes, like Unabomber, where someone on the fringe (maybe with some mental disorder, a psychopath or crazy fundamentalist) does something that seems evil. First of all, these are usually relatively minor in their effect, but can be blown up a lot. I'm more worried about our reactions to such events than the events themselves, in that if we are too quick to place the responsibility of such events on some group those people are associated with, the act of an individual can make us hate the whole group.

Finally, you keep coming back to eugenicists. There are certainly plenty of examples of "eugenicists" that did some things that we consider highly destructive today and others who did things we consider immoral today. Then there was some a large group of people that didn't actually do much, but that were worried about a dystopian future on the other end of a dysgenic development. Finally, there were plenty of scientists doing science in the field, ranging from pure pseudoscience to some of the most important discoveries of the 20th century. Some of these may have been hardline racists, others very liberal.

Finally, we must consider that eugenics, in some moderate form, had very wide support in many western countries for almost a century, even into the 1970s in many places. Even if we don't share those values anymore, labelling whole generations of people as Evil because of this is like labelling large parts of the population in countries where racism is still tolerated, as Evil.

And from a purely descriptive point of view, I would even say that the concern that caused the eugenics movement were not totally unfounded. At a time when most people lived on a farm, many people had a direct relation to the importance of which animals were allowed to reproduce and which ones were not, on the farm.

In particular, this was relevant for those who wanted to build a welfare state where "natural selection" on humans would be eliminated (this could be a socialist or social democratic welfare state, but also a national socialist welfare state, as seen in Germany). The kinds of pure capitalist societies that were common during the 19th century did come with some kind of social Darwinist natural selection built in, so was not seen as likely to have to add explicit eugenic policies to compensate for the welfare state.

As it turned out, it seems that a dysgenic drift in our DNA appears to be too small to be measurable. We've even had the Flynn effect, where it has appeared that mental ability has been going up generation after generation.

Let's imagine that we'd seen the opposite, where countries with welfare states had a radical drop in mental ability over a few generations (as the eugenicists predicted). Lets say that led to those countries losing their ability to organize as advanced, liberal democracies and advanced economies. At that point, eugenics might return.

So to sum it up, my view is as follows:

1) Try to avoid seeing people as Good and Evil. Know that if you do that, you are being illiberal, and you may make provoke needless conflict.

2) Understand your values and the values of other groups and individuals that you may have to deal with.

3) Figure out what differences you are willing to ignore (possibly only if they stay at some distance from you). Liberalism is precisely the willingness to ignore minor value differences. Illiberalism is to demand that the people around you have the same values as you.

4) Figure out what values you are willing to have a confrontation over, and whether you will only go into a confrontation defensively, as a response to an attack, and what values are so important that you need to actively try to impose them on others. Also, figure out the level of intensity you are willing to use in your confrontation, ranging form purely verbal, through imposition through the legal system all they way to the use of violent force. All these considerations need to include some cost/benefit considerations, the details of which are far beyond the scope here.

5) Know that even if you do end up in some confrontation with the opposing group, that doesn’t make them Evil, and it doesn’t make you Good. By giving up this excuse, you may discover something painful, namely that you are acting in ways that do not correspond according to a persona you have taken on as a pro-social, liberal and non-violent individual or group. You may end up doing things that you never thought you would do and may face cognitive dissonance and perhaps an existential crisis (often a cause of PTSD).

Now, without resorting to the Good vs Evil narrative, you should figure out what it is about your values that cause you to be willing to fight the conflict. If you cannot justify your actions from your values, you are being irrational, and should consider resolving the conflict.

Definitely agree with the summary of your views. I think your point on the motivation of eugenics coming from the desire to build a welfare state is very interesting. (I need to think more about the motivation for people that want to build a more AI-driven society or who fear AI technology in today's day and age).

Though, on a separate note, I would question your view of Putin as suddenly "losing touch with reality". Is it not possible that it's just a strategic response to NATO encroachment? https://www.youtube.com/watch?v=JrMiSQAGOS4

I was re-watching the video you linked. Here is an interesting quote:

"If you really want to wreck Russia, what you should do is to encourage it to try and conquer Ukraine. Putin again is much too smart to do that."

This is what I though up until February this year. But when Russia went straight for Kiev, it very much looked like an attempt to conquer them, with the intent to set up a puppet regime or possibly annex them outright.

Putin is isolated from reality in a number of ways (I'm not claiming any mental illness, more likely the problem is that he's been rewarding yay-sayers in a way that prevented him to fully grasp Russia's situation).

1) His premise that Russia has some recognized right to exercise power over their "Sphere of influence" that includes other souvereign nations. While some poeple, even outside Russia, can agree to that, most of those who matter don't, including China. Even though some may not appreciate that more countries have been joining nato, the right of each nation to make their own decisions about their alliences is seen by most as a higher priority principle. 2) His assertion that the regime in Ukraine is not legitimate after 2013. In 2014, some weight could be given to that argument, but today, it is clear that the population in Ukraine generally recognizes their government. Claiming that it is illegitimate, is similar to naming the ROC the legitimate government in China. 3) Related to (2), the belief that the population of Ukraine would welcome Russian forces as they entered their territories were clearly ill-informed. In reality, Ukraine has been fighting with all the conviction and unity you can expect from a country that values its independence. 4) Putin's apparent belief that his military was strong and capable was similarly mistaken. On paper, Russian forces should be able to overwhelm Ukraine in days or weeeks, based on numbers of troops and materiel. In reality, the organization, morale and logistics/planning of the invasion force was horribly bad. 5) Related to (1): Putin's apparent expectation that Nato would remain completely passive was partly wrong. European nations reacted immediately with shock at the invasion, with military budgets almost instantly give increases that in total was multiple times larger than Russia's total military budget. Also, instead of acting as a deterrent, Putin's aggression has shown neutral neighbours that they need to be members of Nato if they want safety. (Like Finland in Sweden, who used to be very happty to stay neutral). For countries already members of Nato, Russia's aggression also meant that many trading partners realized the danger of being dependent on Russia for strategic resources, particularily oil and gas. If most of the EU is able to become independent of such trade, that will be a huge economic loss for Russia. 6) Related to (3) and (5), the fact that Ukraine was able and willing to defend themselves so valiantly, led to weapon deliveries to Ukraine that I don't think Putin expected.

So in sum, a series of misjudgments by Putin (based on, I think, not receiving realistic information about military and political matters), is now leading Russia into potentially a catastrophic worsening of their situation, both economically, politically and in terms of national security. If they maintain their current trajectory, I anticipate either loss or a very drawn-out conflict in Ukraine with no clear "winner". I predict that Russia will end up isolated from the west, and become in effect a puppet of China.

If so, I'm not sure to what extent India will want to maintain their relatively good relations to Russia either, as India's main competitor (I think) may no longer be the West, but rather China.

This is a lot of speculation on what exactly Putin's calculations were. Of course war is bad for Russia in the short-term but given NATO encroachment, you can't just let yourself be slowly encroached on forever.

I think your point about countries that were typically neutral becoming not neutral is an incredibly important data point here. Though I read it in a different way. Specifically, the fact that Switzerland became partisan (Switzerland!) seems to really support what Mearsheimer was saying (i.e. support for Ukraine by the west is much more about financial incentives than standing up for human rights). I mean, did Switzerland, who was neutral even in WWII suddenly gain a conscience now? (Though I guess another way to read it is just that Switzerland feels very safe at this moment to be partisan, or that power dynamics have shifted significantly in Europe in recent times.)

But anyways, let's just take a step back for a second... it's clear that the US empire is much more powerful than Russia at this point. Of course if the US decides to back Ukraine with a lot of resources and support, it's going to be a struggle for Russia. The fact that it's a struggle does not seem to say much.

> Bostrom's brand of fear mongering does not appear to be based in science, logic, or basic common sense. I mean, a lot of world issues are obvious to the naked eye: overpopulation, consumption, housing prices, destruction of natural habitats.

I really wonder whether people commenting here have actually read the article. It's quite involved, I've already spent a few hours and am far from through. It's fair to assume that most people have spent less time.

Your comment is not very much related to the type of events that he is describing in the article. How are housing prices even remotely a technological development leading to devastation of civilization? If anything they are just an artifact of a particular economic incentive structure.