How to Create a Malevolent Artificial Intelligence

For those of you who have been following my work, it should come as no surprise that I have an ambivalent view of technology.

Technology is arguably the predominant reason that we live safer, longer, and healthier than ever before, particularly when we include medical technology – sanitation, antibiotics, vaccines – and communication technologies – satellites, the internet, and smartphones. It has immense potential, and it has been the driving force for innovation and development for centuries.

But it has a dark side. Technology, once a strong democratizing force, now drives more inequality. It allows governments and corporations to spy on citizens on a level that would make Orwell's worst nightmares look like child's play. It could lead to a collapse of the economic system as we know it, unless we find, discuss, and test new solutions.

To a certain extent, this is already happening, albeit not in a uniformly distributed fashion. If we consider a longer timeframe – perhaps a few decades – things could get far more worrisome. I think it's worth thinking and preparing sooner, rather than despair once it's too late.

Many distinguished scientists, researchers, and entrepreneurs have expressed such concerns for almost a century. On January 2015 dozens, including Stephen Hawking and Elon Musk, signed an Open Letter, calling for concrete research on how to prevent certain potential pitfalls, noting that, "artificial intelligence has the potential to eradicate disease and poverty, but researchers must not create something which cannot be controlled".

And this is exactly what Roman Yampolskiy and I explored in a paper we recently published, titled Unethical Research: How to Create a Malevolent Artificial Intelligence.

Cybersecurity research involves investigating malicious exploits as well as how to design tools to protect cyber-infrastructure. It is this information exchange between ethical hackers and security experts, which results in a well-balanced cyber-ecosystem. In the blooming domain of AI Safety Engineering, hundreds of papers have been published on different proposals geared at the creation of a safe machine, yet nothing, to our knowledge, has been published on how to design a malevolent machine.

It seemed rather odd to us that virtually all research so far had been focused preventing the accidental and unintended consequences of an AI going rogue – i.e. the paperclip scenario. While this is certainly a possibility, it's also worth considering that someone might deliberately want to create a Malevolent Artificial Intelligence (MAI). If that were the case, who would be most interested in developing it, how would it operate, and what would maximize its chances of survival and ability to strike?

Availability of such information would be of great value particularly to computer scientists, mathematicians, and others who have an interest in AI safety, and who are attempting to avoid the spontaneous emergence or the deliberate creation of a dangerous AI, which can negatively affect human activities and in the worst case cause the complete obliteration of the human species.

This includes the creation of an artificial entity that can outcompete or control humans in any domain, making humankind unnecessary, controllable, or even subject to extinction. Our paper provides some general guidelines for the creation of a malevolent artificial entity, and hints at ways to potentially prevent it, or at the very least to minimize the risk.

We focused on some theoretical yet realistic scenarios, touching on the need for an international oversight board, the risk posed by the existence of non-free software on AI research, and how the legal and economic structure of the United States provides the perfect breeding ground for the creation of a Malevolent Artificial Intelligence.

I am honored to share this paper with Roman, a friend and a distinguished scientist who published over 130 academic papers and has contributed significantly to the field.

I hope our paper will inspire more researchers and policymakers to look into these issues.

You can read the full text at: Unethical Research: How to Create a Malevolent Artificial Intelligence.

News coverage:

Understanding the Refugee Crisis in Syria and Europe

I am receiving tons of messages about my last social media posts on the crisis in Syria, the response of the various states (European or not), the responsibilities and the consequences.

I am creating a course trying to make sense of all this, collecting and selecting the best resources to add.

Watch the course:

If you have any video to suggest, feel free to add a comment.

Announcing a New Project: Eternally Curious

Good news, everyone! I've been meaning to do this for at least five years, and today I'm so happy to finally announce it to the world. It's a new video series of highly curated and well-produced content called Eternally Curious, where I explore all things I'm interested in.

I'm kicking off the season with this first video: "Why Are People Stupid?".

Video link:

I hope you'll enjoy it.

Remember to subscribe to my YouTube channel here:
And to support me on konoz!

On Trust

Trust. It's a strange feeling.

Being trusting of others is my default state. I assume people are generally OK, and that they act in selfish or in deliberately evil manners only out of necessity or in extreme cases of boredom. I know that this includes millions of variations and possibilities, but still, I like to generally assume I can trust strangers.

All of this can change of course in a matter of seconds. I like this quote from Mike Tyson (paraphrased):

Everybody has a plan until they get punched in the face.

That punch in the face can come at any time, and it typically does when you least expect it.

I was on a plane from Los Angeles to London today. I sat down, got my things set up, and briefly went to the bathroom. I come back two minutes later, only to find that my iPad was gone. Disappeared. The iPad was no longer. It was an ex-iPad.

The moment you realize you have been fucked and that you have no control over things, a torrent of emotions comes rushing to your head. First you try to remember the details before the fact. Did you really have the iPad there? Yes, you put in in the pocket in front of your seat. Was it not inside the bag? Pretty sure it wasn't, but check the bag, just to be sure. Gosh, I shouldn't have gone to the bathroom while people were still arriving and sitting down. Did you backup the photos and videos you took? $700 down the toilet for taking a leak kind of burns, but the photos! Those are memories, money is replaceable. What about that blog post you wrote? Did you back that up? You should back up more often...

Then comes the suspicion. You are sure: it was there, and somebody took it. Who could have done that? Maybe it was just a bored teenager. Maybe it was an asshole who wanted a new shiny screen to watch bad blockbuster movies on and read the daily mail. Did you have a code on the lock screen? How difficult was it? Only 4 numbers, stupid Apple security, a monkey could crack that in a few hours. Not that it matters, they'll probably format it before even trying to open it. Is the find my iPad activated? Was it in airplane mode? If so, it's of no use. Who could have done that? Look around you. Maybe this guy. Or that woman, she looks awfully suspicious. That's what you get for trusting people so much. Pretty stupid move. Don't they have security cameras or something? Always there to harass us for this false sense of security, maybe they can turn out to be useful for once.

While this is going on, you start to think that you might be getting paranoid for no reason. Maybe it was just displaced. Go to the flight attendant, and ask if someone has found it.

As I walk back to my seat, I hear the announcement going off on the intercom, and I get the gaze of the person sitting next to me. Excuse me, have you seen an iPad with a red cover? I left it here, maybe it fell under the seat or something? "No, haven't seen anything, sorry", he says.

As I go through the emotional roller coaster I try to step back and think this through rationally. There is no point in worrying or getting emotional. You're more likely to think straight if you don't get carried away. And if you don't find it, that's it. Move on. They're not going to search 400 passengers to retrieve your lost iPad, get over it.

"Is this yours?", says the guy next to me, as he hands me my lost treasure. Speechless, I hesitate. "Yes", I utter tentatively, "I never check the pocket in front of the seat", he adds.

"Thanks", I sigh in relief.

As I collect my thoughts on what just happened, I can quite literally feel my brain shifting state, and giggle at the double 180 degree change in world view my mind has gone through in less that 10 minutes.

Then it hits me. I remember his face when I came back from the bathroom. He was staring at me. It was a mix of surprise and terror. You didn't register it immediately, but you noticed, then got distracted when you found out that your iPad was missing, and couldn't think straight anymore. You saw him taking a good look around the seats and bags, or at least pretending to, while the hostess was making the announcement. Then you remembered his words when you asked the second time, "When you sat down, did you notice if there was an iPad, or was it already gone?", "I didn't see anything, I wish I could help you, I didn't take it", "Of course, I was just trying to pinpoint at which point it disappeared", I conclude.

But there is something bugging me. How come he couldn't find it, after I asked him twice, and it was right in front of him? It was right there. How could he miss it? Maybe he thought someone had left it there and then he took it, hoping nobody would come to claim it. Then when he saw me he got scared, he tried to keep it hidden, then realized I was eventually going to find out, and looked for a way to return it while making it look like he didn't know it was there.

Sneaky bastard.

Wait a minute. Where is this coming from? This isn't you. You trust people. Maybe he was being honest. If it's true that he never checks the pocket where the airplane magazines are stored, then his story checks out. No mystery, no evil intent.

But then how did the iPad get into his seat's pocket? Are you sure you didn't displace it yourself, and you just don't remember?

At this point, I realize there is no point in going any further with this, it's an infinite spiral from which you can never get out.

Little by little, all these experiences shape us and make us who we are. The more we allow fear and suspicion to take a hold of us, the more we become alienated and we distance ourselves from others.

The challenge is to remain open, and to not let negativity take over. That's a lesson that we need to deal with every day.

VidCon: Motivation. Inspiration. Fascination.

How would you describe your experience of in three words? It's a question that I find myself asking more frequently, both to myself and to other people, and at every iteration the interest and the expectation grows accordingly.

It forces you to think, reflect, and internalize emotions and situations that would otherwise pass by you, forever out of reach, evanescent, fleeting entities that disappear the moment you experience them.

And so I ask that question.

I savor the moment when I look into the eyes of my interlocutor, shining as they move to the upper left, a sign that they are accessing that part of the creative brain that creates new, spectacular pathways into the synaptic connectome of their mind. And I await with a smile of satisfaction, knowing that they are creating new memories, that this process of voluntary reflection will help them solidify what they have experienced, thus appreciate it more deeply.

You can tell when they are making the effort, walking that extra step that is undoubtably more difficult, but that pays off exponentially more than simply glazing over and answering in autopilot. Then comes the sudden epiphany, thoughts have been processed, memories formed, and the smile becomes contagious, as they become finally aware of what they have been missing out until the moment you changed their mindset and forced them to look at themselves under a different perspective. Words have been attached to these new structures, and the act of voicing them will reinforce them, like building a solid foundation from which cathedral and castles are erected, in all their splendor and immensity.

Now comes my favorite part. Will they open the doors of their mental cathedral with you, thus sharing a commons space, and quite literally opening up to you, making themselves vulnerable? A quite challenging and scary thought, albeit no less rewarding than the previous one.

I thoroughly enjoy walking into new buildings – be them humble houses or majestic skyscrapers – all made of mind-stuff.

Excitement. Expectation. Vulnerability. But also exploration, openness, and connection.

And so I ask that question. And I eagerly wait to see the building that will be created before my eyes. Minds. Engines of creation.

In a way, isn't that what all great art does?

I was sharing this moment with Matthew Clarke, the creator of "Convos With My 2-Year-Old". I find it remarkable how he was able to capture the essence of being in that situation, a father exploring this new jungle of emotions, the universe of the mind of a little girl growing up, who can create elaborate worlds and establish new, unpredictable, and surprising connections. And he made the effort, he did walk that extra step, where you become aware of your experience, process it, internalize it, create archetypes of it and re-process it into a new format, accessible to the minds of others. Video. Such a powerful, mind-expanding medium.

I respect those able to walk that extra step, who can give us a glimpse into their mind and into the nature of their experience, thanks to which we can feel more connected, and less alone.

Art. Great art is about communication, and intention.

I like to ask questions. To scratch that part of the mind that isn't often visited, too far from the usual routine of everyday existence. I did that with Kevin from Vsauce 2, and we shared that special moment. I did that with Derek from Veritasium, and Henry from MinutePhysics, and many other great YouTube creators, whose passion, curiosity, and art reaches millions of inquisitive minds across the globe.

This is what I loved about VidCon. That I could ask these questions, and instead of receiving glazed eyes, I saw cathedrals being erected in front of me, cities of thoughts and ideas that I could walk and explore.

Motivation. Inspiration. Fascination.

Thank you VidCon.

More Tweets and Memories

Syndicate content