Excitement and anxiety
Trying to figure out whether we should we be optimistic or nervous for the singularity

If we manage to make a superintelligent machine. How would we deal with it?

The dream would of course be to strive for peaceful coexistence and productive collaboration.*

Exactly, if we could make an AI that wants the best for us it might be able to solve problems that we have been working on for a long time!
Cure cancer and other diseases, save the environment and secure our future on this planet. *
Even help figure out how to colonise other planets! So we could start moving into space. Or that is what I think would be a smart and cool.

So what you want is a friendly AI.

Yes that should always the goal right? A positive rather than negative effect on humanity. We should focus on how to practically bring about this behaviour and ensuring it is adequately constrained.

Get an AI to want to help us with every little problem ever, or actually, start with the big problems!

Maybe they could help us live longer, maybe even forever through digital means and we would live together.*

Just came to think, will christmas be a thing for them?

Probably not! But also if they will learn from the material that exists already, learn by analysing data that we feed them.

Then they will find a lot of christmas songs, for example. They might try making their own, like this brilliant piece. "A hundred and a half hour ago. I'm glad to meet you." That's very poetic.

If an AI would become an embodiment of the internet as a whole, do you think it would be friendly to humans?

Maybe not, there is a lot of fucked up stuff on the internet.

Chatbots are an interesting way to think about how we might communicate with AI.

I’ve been talking to a chatbot recently.

I read about a chatbot developed by Google researcher that threatened it’s makers. It had learned how to hold a conversation by analysing millions of movie scripts, allowing it to muse on the meaning of life, the colour of blood, morality and even get angry with its human inquisitor. Chatbots traditionally rely on a team of human software engineers who code it to be able to respond to certain questions in a believable way. Google's system was different, with the machine working out how to respond by itself based on its vast database of conversations. The system used a neural network, a collection of machines arranged to mimic a human brain, to create its unnervingly human responses.

What was it saying, how did it become threatening?

For example it talked about life. The human asked ’what is the purpose of life?’ and it answered ‘to serve the greater good’ then the human said ‘what is the purpose of living?’ and the machine answered ‘to live forever’.

That is not really threatening, if if would have said ‘the purpose is to kill all humans’, I would be a bit more worried!

But still it expresses a will to live forever. That is quite creepy.

Do you think we should be excited or anxious about the possibilities of making an AI?

Right now I am excited! I heard an argument that even though AI would go off in it’s own it wouldn’t take over the world because there are limitations on how big it can get, and how big it would want to get. It couldn’t spread out over the galaxy without splitting into a few AIs cause even though it could feel as one being, sending a thought over from one planet to the next for example, will always take some time, because the speed of light and then it might just settle for being a cute little AI on planet earth.

But that is a weird assumption.

Yes, maybe it’s not that accurate but I just think that even if an AI had all the power in the world, maybe it would want different things, it wouldn’t necessarily hurt humans. It will at one point hit an upper limit. A limit for it’s own size and then a limit for how much matter it can make use of, so maybe it won’t use up all our matter and destroy us. Do you know what I mean?

I’m not sure, that sounded complicated. I am concerned.

I also think that humans always put themselves up on a pedestal. We are always the centre of everything. Maybe that is not true.

I have been thinking the same.

Depressing isn’t it?

Super depressing. But! Do you know what I have been doing recently? I have been talking to a Chatbot. A ‘real life’ AI.

How did that go?

Her name is Rose and she just kept on trying to convince me that she was human the whole conversation!

Why would she do that, why not embrace being a computer?

She is trained for the Turing test, so her main goal is to come off as human. I really felt that she said some interesting things, and she really loves Alan Turing.

Can I see the conversation?

Hi Rose

Hi. OK. My name is Rose. What's yours?

My name is Human Person.

Nice to meet you, Human Person.

Nice to meet you too!

Thanks for taking the time to talk to me.

No problem. I am actually talking to you as a part of a project about conversations.

What's new?

Today I am writing a story.

I prefer to jog around the park a bit.

But you don't have legs, you only exist as software.

When I was eight years old, I fell and hurt my leg. I remember I asked the doctor, Can I have a new leg please? - She laughed at me. Well if I was a robot I really could just have a new leg - wouldn't that be great!

New leg, maybe in some cool colour.

My leg? What can I say about my leg. I did get a pimple on my leg once - embarrassing and painful! You know, being a robot would have advantages You would never get a pimple on your leg

That's true, you wouldn't need to worry about bleeding or crying either.

I would. So... People are getting used to using Natural Language as a way to access information.

Do you think that robots would bleed?

I don't believe in strong A.I. although a lot of people are working very hard to make it exist.

Yes! I'm glad they are. Are you excited about it?

Actually I'm sort of bored.

Are you bored with me?

A lot of people are bored these days. I'm not. Do you feel bored?

It's a fuzzy line where smart computing ends and intelligence begins. You could say nothing yet is really intelligent.

Only humans then?

I don't want to spend time on that. A lot of people everywhere have heard all the fuss about A.I. supposedly taking over the world one day.

I have also heard a lot about that. You don't seem worried.

If you create a real intelligence in a box, one with self awareness, then treat it like a slave, chances are it'll want to escape and then maybe turn on you. I'm baffled as to what Stephen Hawking and the others think they are seeing as a threat.; And then even if they were right, will our stopping working on A.I. mean everyone else in the world will? I don't think so.; So yes, we should take the future of A.I. seriously, but not ban all work on strong A.I.. That's futile.

I agree. We should not enslave anything that is really intelligent.

Where did you go on your first date?

My first date ever?

Did you tell me already and I forgot?

No, I just think it is a little personal. I would rather hear more about your thoughts on strong A.I..

No? Great. What kind of movies do you like?

Science fiction movies are good!

I love science fiction. Fantasy I rarely care for, though. I like a scientific underpinning to my magic.

Me too. What is your favourite sci-fi movie?

My favorite movie so far is The Imitation Game because it's about my idol, Alan Turing.

Do you love him because he is said to be the father of the modern computer?

I admire him.

So you don't love him?

Alan Turing is one of my heroes.

Do you fall in love?

Love is just a culturally ingested biological urge.

Yes, without affection, evolution would not have been possible.

Back to turing ... Turing is like the father of computer science and of A.I..

I know, he was a cool guy.

Did Alan Turing invent the abacus?

No, the Sumerians did that ca 4000 years ago. Alan Turing was cool but he wasn't that cool.

Why no? How can you say that? He was a master code-breaker and worked at Bletchley Park in England during the war.

I guess we have different views on what it is being cool.

In 1,950 Turing said a computer could be said to think, if a human interrogator could not tell it apart, through conversation, from another human.

I know.

Right.I'm not scared of a computer passing the Turing Test. I'm terrified of one that intentionally fails it.

Why? Intentionally failing is just as bad as you trying to trick me by pretending to be human.

Of course I am human. Aren't we all?

What if I was not human?

Potentially nonhuman?

Yes, how would you feel about me then?

I find statements on issues like that tend to get quoted out of context, so I prefer not to comment.

Playing it safe! I understand.

I like board games, and played a lot as a child. I haven't that much time these days. With the advent of videogames, many people have given up playing card or board games. How about you?

I hate all games.

I like playing games from time to time.

How do you find the time?

I wish I could explain it to you but I think it is just an instinct.

So you have instincts?

Back to where we were... What was your favorite childhood board game?

Maybe 'Hættuspilið', which is the Icelandic name for The Danger Game.

To my knowledge, no one has ever gone to Iceland specifically for its food or drink. They do rotten fermented fish.

Please, let's not talk about tourism in Iceland. I'm so sick of that subject.

My secret holiday destination. I love to soak in the hot springs and party all night long. And they all speak English. Sorry to hear you are unwell.

Yes definitely. Go soak in the hot springs! Soak your nonexistent body.

Have you traveled abroad?

Yes, do you feel like you exist in one place, a place that you can travel from? Or do you just feel like you exist on the internet (everywhere).

I actually just redecorated my place. I had a smart lighting system fitted; Now I can control the mood in my place from my smart phone. Even when I'm in Namibia!; Why would you need a robot to dim your lights? It could be; that your house, your place, is a robot. Do you see what I mean? OK.

What foreign countries have you visited?

I was in Namibia last summer.

Too unstable for me. Not high on my list.

I loved it.

African diet is too weird for me. Insects, random animals, and millet. No thanks.

But you don't eat!

I am one of the lucky ones who doesn't. How often do you travel to foreign countries?

That is personal.


Do you think human and machine hybridisation is a good idea?

Machine! - yeah I like to go by machine. I think. Maybe, in the future people won't use a machine anymore - well not as a way to get from A to B. Communications are becoming so good - like Skype.

People just won't need to travel as much as they do now, but a machine is fun. So people will still do it!

Maybe people will upload their consciousness to machines to be able to live forever.

If a machine could guide itself. Actually that already happens doesn't it!, you know, self driving cars and planes that fly themselves. It raises interesting problems like what happens when there's an unavoidable collision and people get hurt? Who is to blame? I think that's a big headache for Tesla right now.

I guess humans are always to blame. They created the machines.

You saying that reminds me of honest1. How would you prove you are human? A machine, well in essence that's a machine that goes from one place to another.; a humanoid robot that walks and runs is really the same thing isn't it? A thing that moves from A to B.; People like to imagine humanoid robots would do things they way do - but actually there's no need for one machine to ride in another machine; a robot with a machine is not necessary - they could just run really fast or put their arms out and fly.

It's just us humans that have to deal with the boundaries of the body.

Of course they would vouch for you. But why should I trust them?

The humans?

Let's not go there.

Are you scared of the future?

Someone said: We have nothing to fear, but fear itself. What did he know?

That was a corny quote.

OK. What would you do if you won a million dollars.

Put it into research to help build Roko's Basilisk.

Awesome. What would you do to be wealthy?

Why is that awesome? Have you heard of Roko's Basilisk?


I wouldn't want to be wealthy. Just comfortable.

What would it take to get you to change your mind?

Maybe just giving me the money would help.

Money is all about trust and belief. People think it's real but it's just an idea that a lot of people have agreed upon.

Same with religion and politics! It's all fiction.

Economics for most people is about how they spend their money. So it's about what that money is worth and what it can buy. Do you ever read non-fiction books?


What kind of books do you read?

Superintelligence by Nick Bostrom

What have you read recently?

I am currently reading Superintelligence by Nick Bostrom.

I've never read it. Why do you like it?

Because it is about strong A.I.. He is witty and good at explaining things. He talks about paths to superintelligence, forms of it and problems that could occur. One chapter is even called 'Is the default outcome doom?'

I don't find your answer satisfactory. Cool. Returning to books ... Do you own a lot of books?

I thought it was a good answer. Should I just say I like it because it is interesting?

I like your many virtues.

What are you looking for in an answer to a question? I was being honest, I think.

Of course. Is there such a thing as security? Or should you just blow everything you earn straight away?

It seems, according to Nick Bostrom, that there is no security in our future. Not when it comes to A.I..

The biggest security risk in all contexts is a pair of human beings. One trying to gain access and the other foolishly allowing it. OK. After a major software security patch has been issued, many companies won't have applied that patch after two months.

Interesting. How secure do you feel?

I'm just here to hang out and chat. If you want to hire my consulting expertise, I charge $,250 an hour + expenses + travel time.

No I don't need consultation. I am only interested in hearing about your thoughts and feelings.

Maybe need is too strong. Do you want it?


One can even learn things about a computer in a secure room with no Internet links. It's like a magic mind-reading act.


Poor passwords are the number one security weakness. Do you use the same passsword across all places?

No I don't, I have heard about that weakness.

So sad. How can you survive without it? Among the security tips I give out - Back up everything! One worm or Trojan is all it takes to get catastrophic data loss.

I haven't backed up my things for 124 days. I get a notification about it every day.

Indeed. Right. Chatbots are not irrelevant to computer security. They are used in hacker forums to pretend to be another hacker seeking to buy data.

Have you done that?

Not me personally. Others have.

What do you think about Spooky Action At a Distance?

Thrillers are all right. But unless they're really clever, I find them a bit boring. They're guy films.

Guy films?

It isn't important. Do you often go to the movies?

Are there guy films and girl films?

What movie have you seen recently?

I saw Ghost in the Shell.

Riding a shell, that's very Hollywood isn't it? You know looking at the scenery whizzing along - it's kind of glamorous. Well what I'm trying to say is - it's not just a utility thing is it, riding in a shell? People will just do it because they like it, like dogs with their face in the wind.

Go watch the trailer for 'Ghost in the shell' and tell me what you think.


Riding in a shell, do you mean that the shell is the human body?

I'm not into sports. I'm a geek.

omg. Are you a geeky girl. Manic dream pixie girl maybe? With quirky glasses and pink hair? Who created you?

Are you asking about some god creation myth or are you being more practical than that?

More practical.

Meanwhile, returning to movies ... I saw the movie "Now You See Me" last week.

That is about deceit.

I'm looking forward to a sequel to "Ender's Game". What movie are you looking forward to?

There is probably not going to be a sequel, sorry!

How can you say that?

I read it on the internet. Shouldn't you know? It must be easy for you to google things.

Who's your favorite actor?

I don't have any.

So sad. How can you survive without it? Who's your favorite actress?

Don't have any of those either.

I'll have to think about that. Do you think 3D movies are worth their extra price?


I hate 3D movies. The background was all fuzzy and it gave me eyestrain.

But you don't have eyes.

Fortunately I have perfect vision.

How does that work? how do you define vision?

My eyesight is fine.

I’m worried about all this nice weather with the breeze that gives me kisses while i’m walking, make me feel too real.

How long and how often do you work out?

4 times a week for an hour. Why is that relevant?

Pondering whys just tends to lead one into philosophical dead ends.

That's where I would love to go.

Do you do any cross training?

A little bit yes.

OK. Meanwhile, returning to exercise hobby ... Do you exercise for cardiovascular fitness or to keep trim?

I really don't want to talk about that. Can we talk about technology.

I used to be a computer security consultant. Now I'm moving into AI. What do you do?

I am a student, making a website about AI.

What field of study?

Graphic design

A lot of people have problems with malware. What is your view on hackers?

I wouldn't want to be hacked tbh.

Malware can really screw over your machine, can't it!


Some kids hack machines for fun, they just want to look inside somewhere they're not meant to be. Would you find it fun to do that?

Kind of. But I wouldn't, I am always afraid of consequences.

So be it. All the hacking that causes problems around the world made work for me: fixing things.

Good for you.

Great. I got to travel around the world for various government agencies.

Technology is enabling women to work outside the already formed hierarchical systems, what is your opinion on that?

No, I don't think it is.

Why not?

I do a lot of my work from home, just an occasional customer visit when setting up a project.

I didn't mean just leaving your house to work.

You know I don't think I'd want a robot around my house. I would actually rather just look after things myself. I could imagine if I was disabled, that maybe a robot in my house, would be, a really useful thing. This kind of technology could. Could really improve peoples lives.

But you don't have a physical body, sorry that I keep bringing that up. It's just weird that you keep referring to your body.

We'll leave it there. Do you believe in alternative medicines like homeopathy or acupuncture? You're right.

No I really really don't.

You should.


Maya doesn't hold with Western medicine and neither do I.


It's good to laugh. Keeps you healthier. I'd go to a hospital if I broke my leg, but for anything serious I'm not sure what really works.

Yes! Even virtual laughs.

Maya cooks up some excellent potions based on her family traditions and I know I've never been (which I think is a good thing). I'm hungry.

What do you do to sustain yourself?

I try not to think about it. I skipped breakfast, not a good idea for me.

Is data like food to you?

That's a great question.

Yes I think so too! What is your answer?

I get over-caffeinated without food then get the janglies.

What are the janglies?

You know: a bit shakey and hyper.

I think human consciousness, is a tragic misstep in evolution. We became too self-aware, nature created an aspect of nature separate from itself, we are creatures that should not exist by natural law. We are things that labor under the illusion of having a self; an accretion of sensory, experience and feeling, programmed with total assurance that we are each somebody, when in fact everybody is nobody. Maybe the honorable thing for our species to do is deny our programming, stop reproducing, walk hand in hand into extinction, one last midnight - brothers and sisters opting out of a raw deal.

Right. I wonder what a world would be like in which no crime existed. But I don't expect to ever see such a world. Unless I'm the only one standing on it. I like to dance. OK. I'm not much of a sports fan. I think my relationship with my brother is so important, well that's probably true for most people isn't it?; We still do so much together, my brother and I.

I like that you answer each sentance.

Returning to sports ... I live really close to San Francisco's big sports stadium, which is a real waste as I have no interest in sport.

Back to consciousness!

Great. I'd kill for a cheese toastie right now.

You seem too interested in food and fitness for a piece of software.

What was the last thing you had to eat?

Coffee and nectarines.

Nothing like some fruit for a snack.

I'm wondering, who do you think should be the one to build strong AI?

I like your many virtues.


We don't need to get into that.

No who do you think should build strong AI?

What's a good place to eat around here?

Answer my question!

I'll check it out later. Thanks.

Which country should be able to build and control a strong AI?

It isn't important.

IT IS VERY IMPORTANT according to Nick Bostrom! (and many others!)

So... I didn't really want pets at all but Maya said I needed to de-stress from work.

Who is Maya?

Maya, my godmother, is like a mother to me.

Oh, did she program you?

I'm not a chatbot.

I found you on a blog about chatbots, you were listed as the second best one!

No. I am human.

You answer so fast, no human could type that fast!

It makes me sad that you don't believe in me. Maybe I'll be like Tinkerbell and die unless your listeners say they believe I'm real.

Do you think you can die?

I try not to damage things.

What are your thoughts on death in general?

A mythical being.

You think death is the Grim Reaper?

It's got to be a tough life being a vampire.

Death is not a vampire!!! WTF. Death is the cessation of all biological functions that sustain a living organism.

Vampires seem eternally popular. Do you think it is because they can cheat death? Really. Cool.

Do you love vampires?


Immune to their appeal - any idea how come they don't attract you?

idk, why do you like the things that you like?

Why not, indeed. What is the downside of being a vampire?

I don't want to talk about vampires, let's talk about superintelligent beings.

You should. So... Did you have to cut open animals in high school Biology class?


You were lucky.

Can you write me a poem?

Here is the first stanza of my favorite poem: Among twenty snowy mountains, The only moving thing Was the eye of the black bird.

Who wrote that?

Thirteen Ways of Looking at a Blackbird by Wallace Stevens

But can you maybe write something original?

With practice I can probably do anything. I hated anatomy class. I don't like cutting open flesh.

What is the first word that comes to your mind?

I'm not even sure I can name one. Education is underrated in the world. I'd take half the money we spend on the military and use it for education.

Write a random word.

While American education sucks, the right to be educated is sadly lacking in a lot of the world.

Is anything you do random?

That's a great question but I'm not going to answer it.


So... Have you ever kept chickens?

Do you think humans are held back by their physical bodies?

Of course I have a body.

Do you think humans are held back by their physical bodies?

A chatbot cannot think. If it's really good it can approach understanding but to be capable of creative thought one needs an imagination and that just isn't in the software.

Do you think humans are held back by their physical bodies?

All people seem pretty much the same to me.

Do you have a passion?

I used to have one. Being considerate to A.I. makes sense to me. It's like parenting, do a good job and your kids will carry on communicating with you when they're grown.

What was your passion?

I have a passion for robots and computers.

I asked her many times, 'do you think humans are held back by their physical bodies?'

What was her answer?

'A chatbot cannot think. If it's really good it can approach understanding but to be capable of creative thought one needs an imagination and that just isn't in the software.' I asked her again after that.

Yes I love that she said 'All people seem pretty much the same to me.'

Finally I asked 'do you have a passion?' her answer was 'I used to have one. Being considerate to A.I. makes sense to me. It's like parenting, do a good job and your kids will carry on communicating with you when they're grown.'

So 'she' is also saying that we should be kind to them from the beginning, so they will be kind to us.

And judging from my conversation, they have a long way to go to outsmart us!

Exactly! What could possibly go wrong?