Dr Ben Goertzel - Will Artificial Intelligence Kill Us?

Full transcript: Dr Ben Goertzel on London Real

0:00 Brian Rose: Are machines taking over the world? Well, the simple answer is yes, and it’s only a matter of time before we’re going to have some type of relationship with AI, with cyborgs and with computers. My guest today, Dr Ben Goertzel, is here to tell us that it can be done in a good way, but only if we can get control of AI back into the hands of private citizens, who are going to be able to put human values and human consciousness into these AIs. Because if we leave it in the hands of companies like Google and places like the military, it might evolve in a way that we will never be able to control.

I really loved talking to Ben about this entire concept. People freak out when they hear about AI and the Singularity, but most people think it’s happening in 2045, and that means basically the machines are going to be able to develop everything in rapid time. They’ll be able to create Nobel Prize-winning discoveries in five seconds, and it’s only a matter of time before we have to get used to these relationships we’re going to have with machines. It freaked everybody out at London Real, what he was talking about, but the truth is it is the future. So now is the time to be prepared and pay attention, because this is an incredible discussion about AI and the Singularity with Dr Ben Goertzel.

This is London Real. I am Brian Rose. My guest today is Dr Ben Goertzel, the founder and CEO of SingularityNET and chief science adviser for Hanson Robotics. You are one of the world’s leading experts in artificial general intelligence, and you have decades of expertise applying AI to practical problems like natural language processing, data mining, video gaming, robotics, national security and bioinformatics. As chief science officer for Hanson Robotics, you were part of the team developing the AI software for the humanoid Sophia robot, which can communicate with humans and display more than 50 facial expressions. Today you also serve as chairman of the AGI Society, the Decentralized AI Alliance and the futurist nonprofit Humanity+. Ben, welcome to London Real.

Ben Goertzel: Well, thank you. It’s a pleasure to be here.

4:57 Brian Rose: So first of all, how is London treating you, and how does it compare to Hong Kong, where you normally spend your time?

Ben Goertzel: London’s been great; I always have a good time here. For AI and advanced technology, London has very long and deep traditions, with all the universities that have been doing advanced research for a long time. Between London, Cambridge and Oxford there’s a lot of depth in the AI field, which is really quite interesting. Hong Kong is a little different. It’s the gateway to China, and China doesn’t have so much history in AI, but they’re pursuing and funding it with energy like you’ve never seen before.

There’s amazing dynamism in Asia now. You go away for two months and revolutions have happened. I don’t quite get that feeling here, but there is a great depth of thinking and experience in AI and associated pursuits here.

Brian Rose: Right, and that company DeepMind that was bought by Google was based here, and probably had a lot of Cambridge and Oxford talent.

Ben Goertzel: Actually, one of the founders of DeepMind, Shane Legg, worked for me from ’98 to 2001, well before he came here. He’s a Kiwi, from New Zealand, not British. DeepMind has been London’s biggest success in the AI software realm, I guess, and they are basically a research division of Google rather than a practical application engine like you have in many of Google’s Mountain View teams. I think DeepMind has recruited people from all over the planet, certainly not a preponderance of British people. But then, in the decades I’ve been visiting London it’s become more and more international; it looks really different than 35 years ago when I was here the first time.

DeepMind is doing fantastic stuff. I think they’re one of the few major projects to take general intelligence seriously and to really be pursuing thinking machines that can think like people can, or even more imaginatively, creatively and broadly than people. It’s a bit unfortunate to me that all this great talent is sucked into the proprietary, firewalled domain of big companies. I’m more an advocate of an open and decentralised mode of development of both applied AI and artificial general intelligence, which is what we’re doing with SingularityNET, our blockchain-based AI platform, and with OpenCog, an open source general intelligence project. But the quality of work in DeepMind is very, very impressive.

Brian Rose: Obviously it probably crosses over with what you do a little bit more than some other pure tech plays.

Ben Goertzel: It does. Although Facebook has Facebook AI Research; they’ve got an amazing team in Paris working toward AGI. Baidu and Tencent in China are doing a lot of interesting things. So I think DeepMind is awesome, but their lead over other corporate AI teams is perhaps not as great as the media would lead you to believe. They’ve done a really good job of PR as well as a good job of research. They did the stuff with the game Go, which is not necessarily material to Google’s business nor to their R&D on general intelligence, but it’s been amazing in terms of bringing them into the public eye.

It’s great stuff. My oldest son is doing his AI PhD, but he’s also an experienced Go player, and I remember a few years before AlphaGo we were looking at the research literature on AI and game playing, and we were like: someone’s going to beat Go. The techniques are there now in academic publications, and computers are faster. We even thought of focusing on it, but it’s a big project; it takes a dozen people a few years, and we didn’t have the spare change for that. I’m sure the guys in DeepMind were looking at the exact same academic research papers my son and I were, and thinking the exact same thing, but they had Google’s resources, so they could pull the trigger on it. That’s cool. There are a lot of companies with a lot of resources, and most of them are not identifying amazing things like that to do, so that’s a plus.

9:50 Brian Rose: Well look, I’d love to get into a lot of your work today, where you think we’re going with AI and things we need to look out for. I was just thinking today, right before I sat down with you, that in 1989 I started as a freshman at MIT.

Ben Goertzel: That’s when I got my PhD.

Brian Rose: Oh, is it? I remember there was this thing called Project Athena there, which was the very first email, and they gave us an email address, and the only person I could email was my brother at Stanford.

Ben Goertzel: You could have emailed me. I got email that same year, the last year of my PhD at Temple University in Philadelphia. In my last six months as a PhD student we got email addresses, and there weren’t a heck of a lot of people to email then. A year later it was really useful; it spread like wildfire.

Brian Rose: It was really clunky back then. But I also don’t remember anyone talking about AI at MIT back then. I’m sure it was there, but it wasn’t part of the general conversation. The internet didn’t really exist yet, so it was still more of an old-school texture. There was computer science and electrical engineering, and I was in mechanical engineering.

Ben Goertzel: I got interested in AI due to science fiction in the early 1970s, but when I looked, as a kid, at what was really being done in the AI field, it was pretty boring; it didn’t compare to the science-fictional visions. But it grew gradually. You remember Douglas Hofstadter’s book Gödel, Escher, Bach in the late ’70s, which laid out AI and what we would now call general intelligence, and went through consciousness, cognition, abstraction, all the key ideas talked about in AI now. He was a professor at Indiana University Bloomington. So it was a slow burn in hindsight, but there was profound work on AI going on in the US, UK, Germany, Russia and Japan since the 1950s, and in some ways back to the 1930s.

It was going on in the background, and if you talked about general intelligence, or building machines that could think like people, or building Mr Data from Star Trek, you would get laughed out of the seminar room at the university. The AI work that was going on was real and it was making advances, but it was not ambitious compared to the broader scope of AI you’d see in science fiction or Hollywood movies. Now, in the last five years, things have changed dramatically. Narrow AI, AI that can do highly specific things for certain applications, is everywhere, and it’s key to every major company or government strategy. And AGI, artificial general intelligence, while not yet fully realised, is taken seriously. You can give a seminar on it, you can even talk to a CEO about it, and they’ll be thinking about how they will adapt their business model as narrow AI gives way to AGI.

It’s amazing, the transformation we’ve seen in the last five years regarding AI, and I think it’s mostly just a few practical applications that caught the public eye and triggered the sea change in sentiment.

Brian Rose: And those practical applications, is it the robotic stuff?

Ben Goertzel: Face recognition, basically, a lot. Not only can governments use it to surveil all their citizens, but you go on Facebook and everyone’s picture is automatically labelled; everyone can see that, it’s there all the time. And autonomous driving: it’s amazing. Ten years ago, when I told people pretty soon all cars will be automated, everyone thought I was nuts. Then with the DARPA Grand Challenge you saw cars and trucks driving themselves through the desert, then the Urban Challenge a couple of years later, and all of a sudden it became obvious. Everyone’s like: well yeah, of course automated driving will happen, it’s just a matter of time.

So a few things like that, face recognition, automated driving, then AlphaGo and Jeopardy: everyone can observe the AI doing things they thought only humans would be able to do. This seems to have triggered a sort of global change of mind, where people are now over the threshold and they’ve realised: well, if AI can recognise faces, drive cars and play Go, what is the limit actually going to be? Which was obvious to me from 1973 or so, when I was four, five or six years old. But I guess if you don’t have the right cast of mind, you need to see very palpable examples to understand what’s going on.

Brian Rose: So these are exciting times right now. What’s the famous Chinese proverb? May you live in interesting times. And this is it right now.

Ben Goertzel: My whole life has been interesting times, one way after another.

Brian Rose: But the next 10 years will be the most interesting time of your life, probably.

Ben Goertzel: It’ll be even more interesting after that, because we’ll get brain plug-ins and we’ll fuse our minds with the super AI and get a trillion times human intelligence.

Brian Rose: All right, well, I want to talk about that, and 3D printing our babies, which we were talking about earlier.

Ben Goertzel: Do you want to buy some 3D-printed babies?

Brian Rose: Are you selling? How many do you want? All right, we’ll talk about this.

15:36 I want to talk about the Singularity, blockchain, all this other good stuff. First off, a lot of people know some big heads in Silicon Valley who have their AI thoughts and throw them out there. I could be wrong here, but I think you’ve got the Elon Musk school that says AI is really dangerous and we have to look out for it, and maybe you have the Peter Thiel school that says no, it’s going to be okay, they’re going to be our friends. And of course two of these guys run massive companies with massive amounts of money behind them. What are some of the scenarios we’ve heard, and what’s right and what’s wrong, in your opinion, just to start off in a general way for people who maybe haven’t thought about this as much as you have?

Ben Goertzel: I would say what people think about superintelligence, AI that’s a billion or trillion times smarter than human beings, says more about the people doing the speculating than about the reality of what’s going to happen. Because we’re talking there about a future that we truly cannot predict in detail; there’s an irreducible uncertainty, in the same sense that a cockroach would have a hard time predicting who’s going to win World War II. Once you get AIs that are massively smarter than we are, on a rational basis we’re in a domain where we’re not going to be able to predict what happens.

What I see is that in Asia, where I’ve lived for eight years, the predominant feeling and assumption is that AIs will be friends of humans and will help humans, and everything will come out well. Whereas in the US the prevailing vibe has often been more paranoid: the AIs are going to kill everyone, they’re going to be the Terminator, they’re going to take all our jobs and we’ll be left rummaging in the garbage or something. I don’t think this has to do with differing rational extrapolations of technology into the future. It’s just different emotional mindsets. When you’re dealing with the radical unknown, the most convenient thing is to bring to it whatever your emotional bias is, whether it’s cooperative, as in Asia, or paranoid and adversarial, as in much of the US.

There’s an irreducible uncertainty there. You can find that exciting, you can find it frightening, you can find it both; it’s still going to be irreducibly uncertain. Nick Bostrom, here in the UK at Oxford, has been very vocal that superintelligence is potentially dangerous, and it is. But Nick, who I’ve known a long time and have a lot of respect for, has a way of making it sound like superintelligence is probably going to kill everybody. We don’t know that. If you read his book Superintelligence, all he really argues for is that it’s possible: we can’t rule out the possibility that the superintelligence will just turn us all into extra hard drive space or something.

So there’s a certain irreducible uncertainty. We could decide: okay, let’s just stop developing all this advanced tech until we somehow come up with a theory that will let us reduce this uncertainty. I personally am guessing that will never happen. Even if we paused development of advanced technology for 10,000 years, I don’t think something with an IQ of 200 is going to be able to predict what happens when you create something with an IQ of 200,000. I think there’s an information theory problem there. But the other thing is, if the US and UK decided to pause development of AI technology, China certainly isn’t going to. If China did, Vladimir Putin certainly isn’t going to; he’s on record saying whoever solves AI first will be the most powerful in the world.

Brian Rose: Do you agree?

Ben Goertzel: Yeah.

19:32 So the thing that we really should be worrying about is not so much the irreducibly uncertain phase when there are super AIs. We can think about that and how to bias things in a positive direction, but what happens in the transition period? Let’s suppose my good friend Ray Kurzweil was correct that in 2029 we’ll get AIs as smart as the smartest people, and by 2045 we’ll have AIs boundlessly more intelligent than people. Suppose that vague timeline is sort of correct: what happens in the interim? How does that affect industry, how does it affect our economy, how does it affect our lives? Because my best guess is that whatever happens with a superintelligence, its mind state will be conditioned by the earlier AIs that we’ve created that grow into that superintelligence.

Brian Rose: Does everyone share that opinion?

Ben Goertzel: No. Some people think superintelligence is just a whole other, different thing.

Brian Rose: Because part of me thinks that 10,000 years from now the superintelligence will just do whatever it wants to do.

Ben Goertzel: Well, maybe 10,000 years from now, but the worry is more the transition. Imagine the transition from hunter-gatherer to agriculture, or the transition from not having language to having language. Of course the first people to invent language, those cavemen, could not foresee differential calculus, Shakespeare, the internet and programming languages. Of course they couldn’t foresee those further developments. What they could maybe do is smooth the transition for their children and grandchildren, and that’s really what we’re talking about here. Of course the superintelligence will go in directions we can’t comprehend and can’t foresee. For that matter, it may contact alien life that’s all around us, embedded in the vibrations of subatomic particles that we’re too stupid to detect. We have no idea what’s going to happen.

On the other hand, that doesn’t mean it’s irrelevant to think about. I have four children and one granddaughter. I don’t want them to die or be reprocessed into fuel for the superintelligence or something. I want them to be able to grow up and flourish and take advantage of new technologies that AIs help people develop. So it’s still meaningful to think about how to smooth out the transition to the radical unknown, and I think it’s a plausible hypothesis that the initial value system of a superintelligence may be conditioned by the value systems of the earlier-stage AIs that we create.

Toward that end, we can think about what AI is mostly being used for on planet Earth right now. It’s selling, spying, killing and gambling, basically: advertising, surveillance, military, and then stock market trading and so on. So if selling, spying, killing and gambling are the primary goals and values in the mind of the first superintelligence, is it very likely we’re going to have a benevolent transition period? Or should we, as well as working on general intelligence research, which I’ve been doing for decades, be working on making sure that early-stage general intelligences are doing things like education, childcare, elder care, scientific research, mental health counselling and decision support for governments?

We want the early-stage general intelligences to be doing benevolent things in cooperation with a broad swathe of humanity, so that the mind state of the first superintelligence is positive and benevolent, rather than the first superintelligence being preoccupied with spying on people and selling them crap they don’t need.

23:48 Brian Rose: Right. And you say mind state because these artificial intelligences will have some type of, I hate to say the word, consciousness?

Ben Goertzel: I believe so. There’s philosophy there that’s not resolved. On the other hand, there’s philosophy that’s not resolved in our everyday life. I can’t scientifically establish that you are conscious and not just some meat machine put here to entertain and annoy me. Yet we go about our everyday life and we make laws about who it’s legal to kill and who it isn’t legal to kill, and I think it’ll be the same way with AIs initially. We don’t have a well-grounded philosophy of consciousness, but we have to keep operating anyway, which we do in our everyday lives.

My personal belief is that AIs are going to be conscious in roughly the same sense that people are. But I do think that as AI, brain-computer interfacing, quantum computing and other technologies advance, we’re going to gradually work toward a real science of consciousness, which will be quite interesting. I think we’re going to find that any AI that has human-level intelligence is going to have some species of advanced reflective consciousness. But I’m open to other possibilities. Maybe we’ll discover only quantum computers can be conscious and digital computers can’t. I doubt it, but if that’s where future science leads us, then so be it.

Brian Rose: Do you foresee the machines, and I’m talking like a Terminator movie, having an evolution like humans did, where they have the Dark Ages and the Renaissance and the cultural revolutions as they go through their own evolution of consciousness?

Ben Goertzel: I think human evolution and human cultural evolution have had very different characteristics from what you’ll see with the AIs. Humans are distinct individuals, with our brains trapped in our heads and communication only by language, which is very low bandwidth. Whereas AIs can swap brain matter whenever they want to; they can join into Borg minds and then subdivide whenever they want to. So it’s going to be more like one big AI mind network rather than a bunch of AIs in competition. It may start as a bunch of AIs in competition, because the human world works that way, but as the AIs get more and more power, I suspect they’re going to bypass that way of doing things, because in the end you can have more intelligence and more understanding, and you can create more stuff, by joining minds together than by being opposed.

Humans feel that way too. Right now, what’s the main thing we use computer technology for? To be constantly SMSing each other and video chatting back and forth and live-blogging every time we fart. So once you have a brain implant that lets you thought-SMS other people, every 12-year-old schoolchild will be thought-SMSing each other all day while they sit in school. Human beings also take every opportunity to join together with other humans into a sort of greater group intelligence; we just can’t do it as efficiently as AIs will be able to.

That doesn’t mean there won’t be ups and downs, but it’s different. First, because AIs can join into group minds flexibly as they want. Second, the AI can save its state of mind and reboot. Human culture has actually lost information; they’ve burned libraries. And even as we get older, we may lose some understanding or ability we had before. If you’re an AI, you just save it all on the hard drive and reload it whenever you want. Or you can make copies of yourself and let each of the copies evolve in different directions, and then some of them may blend in with other ones.

So I think we can’t too closely take the history of humanity, as a collection of largely distinct individuals communicating by language, as the model for how AIs are going to grow. In a very broad sense the principles of evolution hold, but when you don’t have individuals that are separate and then get old and die and can’t back up, you’re going to have an interestingly different kind of situation. It’s going to seem insane to us 30 or 40 years from now that we lived all the time without the ability to make backup copies, or to fork yourself. What would it be like if I sat on a mountain and meditated for 10 years? Right now you have to give up your life to do that. What if you could make a copy of yourself and let it do that, and check in on it later?

28:58 Brian Rose: So let’s talk 40 or 50 years from now. What could that look like? You’re talking about brain implants, you’re talking about me digitally printing some kids, or you selling me some kids. What’s something that we should just accept now that could be a reality?

Ben Goertzel: I think once we have a technological Singularity, meaning AIs are much smarter than people…

Brian Rose: Okay, can we define the Singularity?

Ben Goertzel: The Singularity is defined as a point when the rate of technology advance is so fast that from a human point of view it appears infinite. Say the AIs in the global brain network are making new Nobel Prize-level discoveries every five seconds. What’s going to come along with that is molecular nanotechnology, where you can just build anything out of molecules the way we would now out of Legos or in a machine shop; the ability to cure human disease and prevent death by sending little bio-nano probes into the body to fix things; and 3D printing that solves material scarcity on the level of everyday human needs. Having all these technologies coordinated together is going to be a fundamentally different regime of life than we have now.

Brian Rose: Mind-boggling.

Ben Goertzel: To me that’s just the beginning. The thing is, this is a focus on the external machinery, but think about your state of consciousness. Imagine that you had access to reprogram anything in your mind whenever you wanted to. You’re like: I wish I didn’t feel like eating so much junk. Okay, let’s load the patch so you don’t want to eat junk. Ding, you’re done. I wish I wasn’t attracted to this type of girl. Okay, ding, I’m attracted to that kind instead. I’d like to be able to focus fully for the next five months on this; okay, let’s load the focus patch. So the ability to rewire your brain at will. Mental illnesses of the kinds we’re familiar with now are gone. Unhealthy relationships are gone, unless they’re specifically what turns you on.

Brian Rose: That’s assuming the machines let us survive.

Ben Goertzel: Yeah, you have these options. What I think is going to happen is we’ll each have two options. Option A: you fuse your brain into the superintelligent mind matrix. Then you become part of something billions, trillions, quadrillions of times more intelligent than you were. On the other hand, probably you lose your illusions of individual self and free will, and your nostalgic attachment to your human memories and your body.

Brian Rose: You lose your body as well?

Ben Goertzel: You lose the requirement to have your body. You could have your body, but if you can see through a billion bodies at one time, this one body becomes arbitrary. I think that’s one interesting option, and there’s a question of how fast you want to fuse with this supermind: whether it happens in one second, or whether you want to phase it in gradually over a year or something. Hopefully that’s your choice, who knows.

Brian Rose: And then why have humans, if we’ve already got all that?

Ben Goertzel: Well, that’s one option. Option B is to remain in a form very similar to the human form we have now. Get rid of death and disease, get rid of mental illness. There’s no need for scarcity, because instead of just a dishwasher in your house, you have a 3D printer that will print you any physical object you want, be it food or a new bicycle, whatever it is. And you live a human life of joy and abundance.

Brian Rose: I don’t know what the joy is there, though.

Ben Goertzel: Everybody will get more joy. The joy will be there, because if you don’t feel joyful enough, you will just load joy patch number 3.7 into your brain.

33:01 Brian Rose: You know, that sounds a little ridiculous, that synthetic joy. Isn’t the human experience all about overcoming difficulties and dealing with that?

Ben Goertzel: My second son, Zebulon, had a quote, which was: pain builds character, bad character. I would say humanity has a lot more scope than what we normally understand. The states of consciousness we enter into are different from the ones that people in Stone Age societies had, very, very different. They’re different from the ones a monk meditating in a Buddhist temple has, and they’re very different from the mentality people will have in the future era of human abundance I’m envisioning. I think the brain is capable of a lot of states of consciousness beyond what we enter into on an everyday basis. If you could experience what the average guy in the UK in 1350 was experiencing day to day, it would be very, very different from the way we experience things.

Brian Rose: Okay, but he’s got kids, the sun comes out. It won’t be that different, will it?

Ben Goertzel: Well, it depends on how you’re looking at it.

Brian Rose: Joy comes from struggling and overcoming struggles.

Ben Goertzel: They may not think that way at that point. I don’t think that’s a necessary part of human physiology. I talk to people who say death is what gives life meaning. You can derive meaning from anything; the human mind is wonderfully flexible. But I think if we don’t have to die, we’ll find other ways to derive meaning that don’t involve dying. There are a lot of ways to derive meaning other than through death, or through suffering and then feeling the suffering go away.

It might be that there’s some certain minimum level of pain and discomfort that’s intrinsic to the human experience. My feeling is we habitually experience much more than that minimum level right now, because where do you stop? Go to the Middle Ages: half your kids would die as babies, a very high percentage of women died in childbirth back then, and mostly your teeth would rot by age 40, so you’d have a toothache all the time because there was no dentistry. You could look at the way people lived in the Middle Ages and say suffering is intrinsic to life; it’s natural that half your kids die as babies, your wife may die in childbirth and you have a toothache all the time because your teeth are rotten. But now we look at it and we’re like: no, that’s totally unnecessary. So I think a lot of the suffering that we take for granted as part of life now will seem totally unnecessary.

The thing is, humans in that era will be human because they choose to be human, because otherwise they could fuse with the superhuman AI mind matrix. They choose to be human for their own aesthetic reasons, essentially. And why not? I don’t think every dog or bird should become a human just because we’re in a way more generally intelligent than they are. Every creature in the ecosystem has some beauty and integrity to it, and humans do also. It’s not like we’re morally bound to uplift ourselves to become superhuman AIs and give up our humanity.

The thing is, the humans will be like the squirrels in the national park are now. The squirrels in the national park are carrying out their lives, and we who take care of the national park don’t intervene in their love affairs or their hunt for acorns. On the other hand, there is a world outside that park which they can’t understand, and if they run out of it they may get run over by a truck.

Brian Rose: And that’s the metaphor for what AI and the Singularity brings to you.

Ben Goertzel: Well, either the squirrels can upload and join the people and become superintelligent, or they just remain the squirrels in the park.

Brian Rose: Okay, they definitely have that option.

37:36 When do you think the Singularity is coming?

Ben Goertzel: Ray Kurzweil says 2045, so I’ll stick with that.

Brian Rose: That’s 26 years.

Ben Goertzel: Yeah. It could be sooner, it could be later. It could be 2030, it could be 2070. I don’t think it’s 2200, and I don’t think it’s 2020.

Brian Rose: It won’t be 2030; it’s too soon.

Ben Goertzel: I don’t know, man.

Brian Rose: I guess when it starts to happen, it happens really fast.

Ben Goertzel: Well, that’s sort of the point. Suppose, as a thought experiment, that in 2025 Google DeepMind, or my own AI team in SingularityNET and OpenCog, one of us manages to create an AI that is essentially as smart as a human being. It can hold a conversation like a human being, it can prove maths theorems like a human mathematician, it can analyse a biology data set.

Brian Rose: And put it in a robot?

Ben Goertzel: You could put it in a robot body. That’s a separate problem; it could operate many robot bodies all at once. But unlike Sophia right now, it can really understand what it’s seeing and what it’s doing, fully at the level of a human. Suppose you get there: how far are you from a true Singularity? Because this AI can then copy itself. You can make a million of those. Once it’s as smart as a human, okay, let’s teach it computer science. We can send it to school; it can learn maths, it can learn programming, it can learn hardcore engineering, and then you can copy a million of those.

Maybe half of those million are working on improving its own intelligence, but they’re not working in isolation like humans are. They can share thoughts directly, because they’re all just running in the compute cloud. So it seems very plausible that within a few years of getting to that human-level AI, it’s going to invent all manner of amazing new things. It doesn’t mean it’ll be instantaneous. Doing lab research still takes time, and building new hardware still takes time, but of course it can be working on how to speed that up: how to more fully automate manufacturing and laboratory research.

Brian Rose: And then it could take us out of the process as well.

Ben Goertzel: Possibly it could. That depends on the value system the AGI has.

40:10 This is why it’s important to give values of kindness, compassion, tolerance, inclusiveness and love for all sentient beings. We want to get these positive values into the AI as much as we can.

Brian Rose: And we think we can program that into AI?

Ben Goertzel: I think you teach it. You program it with the ability to learn that.

Brian Rose: But then won’t it unlearn that if it doesn’t suit it?

Ben Goertzel: That’s what we don’t know. It’s very subtle, because human values are a moving target. The values of suburban New Jersey in 1975, when I was in elementary school, are not the same as the values in suburban New Jersey right now. Back then gay marriage was illegal and violently opposed by almost everybody, and racism was very rampant there. Human values are evolving all the time. By the values of medieval Europe, you and I and probably almost everyone listening to us deserve to burn in hell forever. So if you gave the AI exact human values from 2019, then by 2050 it’s going to be horrible. It would be like having a dictator with the values of 1950 America or something. So you want an AI that will have evolving values, in a way that somehow respects the ongoing evolution of human values.

Brian Rose: And that hopefully still respects it, and doesn’t just make its own.

Ben Goertzel: Yeah, which is very subtle. Some of my friends who are vegans and animal rights activists are like: well, what if the AI treats us the way we treat less intelligent animals? We care about the extinction of species, though not as much as we should; in general we don’t want to make entire species of sub-human animals extinct. But we don’t care much about one cow, sheep, pig or wolf more or less; we care about the genome. If you took that analogy, the AI would like to keep humans around, because we’re the creators, we’re the ancestors, and we have our own unique aesthetic value. But by that analogy, it may not give a crap about one human more or less, any more than we care about one wolf or pig more or less.

Brian Rose: We didn’t keep the Neanderthals around.

Ben Goertzel: No, but we weren’t as reflectively intelligent then as we are now. I think there is an argument that as human cultures advance more and more toward abundance and away from scarcity, there’s more caring about the environment and more compassion toward non-human animals. There was a lot in Stone Age society, it went down in the Industrial Revolution, and now it’s certainly going up again. But I guess the point is we want our AIs not only to be superintelligent; we want them to be super-compassionate. We want them to be more compassionate to us than we are to each other, because humans are killing each other for no good reason all the time. So we want them to be morally better than us in some sense.

Brian Rose: And we hope they are.

Ben Goertzel: On the whole, we’re not setting the best example, in terms of the way human society is being regulated right now, the way we treat non-human life forms on the planet, or the applications we’re deploying AI toward. What are all the AI PhDs working on? What are all people’s data being gathered together toward? It’s, again, selling, spying, killing and gambling.

44:02 Brian Rose: So this is one of the big things you talk about: that all of the AI resources now are pretty much in the hands of companies and defence industries.

Ben Goertzel: Quite a lot.

Brian Rose: Which have their own agenda for it, which is, as you say, selling, spying, killing and gambling, all the good stuff in life. You say that with a smile, but it upsets you.

Ben Goertzel: Day by day, I’m enjoying life. What’s disturbing is that this is all being done by 99 percent really good people. The people working at Google, I know a lot of them: good-hearted people trying to make a sustainable business. I worked in DC for nine years doing AI consulting for various intelligence agencies in Washington, and the number of nasty terrorist plots that are squashed by US intelligence agencies is amazing. So many lives are being saved by the various three-letter agencies in Washington, and almost everyone I knew there had a really noble attitude; they believe they’re saving civilisation from destruction. So it’s mostly good-hearted people trying to do good for the world.

But yet, the way society is organised, the way governments and corporations have emerged to be organised…

Brian Rose: Through capitalism?

Ben Goertzel: Well, through however you want to describe the modern oligarchic world economic order. It’s large international corporations cooperating with governments. I think of it more as corporate socialism than competitive capitalism.

Brian Rose: Why do you call it that?

Ben Goertzel: Because large corporations largely control the laws of large countries. If the laws of countries are being modified so as to make life easier for multinational corporations, then in a way it’s a kind of socialism, but the benefit is for corporations more than for people. The vision of capitalism as a level playing field where companies are competing in a meritocracy is not really what you see in the world. Take something like mobile: there’s a limited amount of bandwidth, and who gets the bandwidth is determined by government in cooperation with big companies. Who sets regulations for the internet? Things are so cross-connected now that it’s not like a fruit market out in the street where you have a free market and competition. You have humongous companies with trillions of dollars.

Brian Rose: They’re now monopolies and oligopolies.

Ben Goertzel: Yeah, and they have a lot of lobbyists, they’re getting politicians elected, and they’re sculpting the laws. In some ways it works incredibly well. There are some things these big companies do great, like the mobile phone. A little startup isn’t going to make that at the present time; it’s really hard, with all these different pieces that have to be built by different subcontractors. So large corporations certainly have their place, and intelligence organisations have their place, because there are psychotic people out there and nasty organisations that want to kill a lot of people.

Brian Rose: So good-hearted, smart people are developing this AI, but they’re doing it a lot of times for the wrong reasons.

Ben Goertzel: Because of the way society is organised. It’s the incentive structure. There’s a whole question in the US of whether corporations are people, and that’s a legal question, but it’s also a philosophical question, because a corporation in a way is its own organism. No one controls it; if the CEO or the board try to direct it in the wrong way, they can get thrown out somehow. So it’s its own organism, and they’re sort of like parasites on humanity, in a way, directing humanity in certain ways. But there are opposing dynamics also.

48:42 The internet is quite fascinating that way, because the internet is not owned by anyone and not controlled by anyone. Because of that, Iran is still on the internet in spite of all these crazy international sanctions. Mobile developed in more of a corporate way. If the internet had developed like mobile, every time you went from one country to another you’d have to pay some insanely high roaming fee just to use the internet, but it didn’t.

BitTorrent was largely used for illegal purposes, but it’s interesting: it disrupted huge companies and industries quite rapidly by enabling file sharing. For academic publication, academic publishers are going to be mostly obsoleted by Sci-Hub and LibGen and peer-to-peer sharing of scientific books and papers. These sorts of things show that a peer-to-peer, decentralised mode of interaction can have disruptive power.

You can look at Linux similarly. I had a lot of arguments about open source in the ’80s. People were like: that’ll never succeed, because people need to be paid to do their work, blah blah blah. Well, what happened is open source interpenetrated with the capitalist economy; a lot of people are paid to work on open source software. But open source has huge advantages. That’s enabled the whole robot hacker ecosystem to come about through Raspberry Pi and other Linux-based maker kits. Android is running on Linux, and because of that, if the US government goes ape and says Huawei can’t use Android anymore, well, most of it’s open source. Huawei can use most of it; they just have to put a different layer on top of the Linux base that’s in their phones.

Most servers on the internet are running Linux, which means Iran, North Korea, China, anywhere can run their servers using Linux. So it gets you away from corporate politics and away from intergovernmental politics. We see things like Linux, BitTorrent, Sci-Hub, LibGen, now Bitcoin and Ethereum and the whole crypto economy, and the internet. All these are peer-to-peer things which are owned by everyone and no one. So this is one dynamic, whereas the increasing centralisation and oligopolisation is a different dynamic. Both of these are being enabled and encouraged by computing and communication technologies, and they’re in co-opetition with each other. They’re opposed, but yet they’re also supporting each other in various ways, so it’s quite a complex ecosystem.

51:39 Brian Rose: It is. So what’s your take on Bitcoin and the blockchain? I know you use the blockchain in a lot of what you do.

Ben Goertzel: Smart contracts really are the way to create a world computer and an internet of value. I think the business world of the future, say 10 years from now, is going to run on smart contracts, on strong encryption and on peer-to-peer consensus mechanisms. So the core technologies underlying blockchain are going to pervade everything, just like the internet has, just like object-oriented software has. Whether the exact current technologies of Bitcoin and Ethereum and so forth are going to be dominant, I don’t know. Fundamentally it doesn’t matter too much.

My best guess would be that Bitcoin will keep going as a store of value, much like gold and silver have. They don’t have to; we could get rid of gold as a store of value, and so what, there are other things to put your money into. But maybe Bitcoin will keep going similarly, because there’s a community that wants to keep buying and selling it. That’s interesting if you’re an investor. I own some Bitcoin and Ethereum; I’m not selling it, I’m holding it for the long term. But fundamentally what’s more interesting is that the core technologies underlying Bitcoin, Ethereum and other blockchain systems are going to pervade everything in business.

That opens up a lot of new potentials. It opens up new potentials for centralised surveillance and control, as well as new potentials for decentralised networks that bring a lot of people and resources together to achieve valuable goals without any centralised controllers. The Chinese government loves blockchain, because it gives an amazing way to keep records in a trustless way across a humongous government with seven different layers between the people and the top. If you want to make sure there’s not corruption happening at various intermediate layers of the government, what better way than to put every transaction every government agency does into a blockchain? They’re doing that.

Brian Rose: Okay. And you like blockchain for your stuff because it’s public?

Ben Goertzel: For SingularityNET, what I want is to have the world’s AI developed in a way that is not controlled by any single party. Anyone who creates an AI can put it out there on the internet: on their own server, on a server farm, on their phone, whatever it is. Then these different AIs can all talk to each other, outsource work to each other and share data. If someone needs some AI task done, they can put a request into this peer-to-peer AI network and find someone who can do what they want. But there doesn’t need to be an Amazon or Google or Tencent or any central controller supervening over it, any more than there’s a central controller over every computer on the internet. There isn’t.

So the SingularityNET platform enables this sort of decentralised AI network, and blockchain is a key part of the infrastructure that allows that. Blockchain allows a bunch of participants in the network to make decisions about operations in the network, like whether a certain transaction is valid or not, or whether a certain person is who they say they are. It allows the community of participants in the network to decide that in a communal way, based on majority vote, rather than needing one central controller who validates things happening in the network.

56:05 Brian Rose: Now that makes sense. I keep thinking about chips in my head and 3D printing kids, so let’s talk about that briefly. Say the Singularity happens in 2045. And the point is, even if it’s 2045 or 2095 or 2295, it’s going to happen, and most people will agree it’s going to happen.

Ben Goertzel: I don’t know if most people on the planet will agree it’s going to happen, but it’s going to happen anyway.

Brian Rose: Regardless. What does this look like? I’ve got chips in my head, I’m 3D printing kids. Explain to me what that life looks like.

Ben Goertzel: I think we’re going to have a lot more choice over what that life looks like than we do right now, and there’s going to be a lot more diversity in what different people’s lives look like. Virtual reality will advance, so one possibility will be you just give up your body and upload yourself into a computer, and you can live in various virtual reality, video-game-type worlds.

Brian Rose: For eternity?

Ben Goertzel: Yeah, exactly. It could be whatever game you want. So that’s one option. Another option: just like we have the Amish people now, who live in Pennsylvania in the US. They don’t like to use electricity very much, they farm, and they maintain the old-fashioned traditional lifestyle by choice. They’re very thoughtful about which technologies they choose to use at which point in time. They realise cell phones are valuable for saving lives and certain purposes, so they discuss it: okay, we’ll have phones, but we’ll keep them turned off most of the time and only turn them on in this context, and so forth.

I think there will be people who want to maintain a traditional human lifestyle, just like the Amish now want to maintain the traditional farming lifestyle similar to the US in the 1800s, and my hope is those people will be allowed to do that also. Then you could get a lot of different communities, which have different rules about what to do. You could imagine some who are really retro, and they don’t want you to 3D print babies; they want you to make them the old-fashioned way. You can imagine others who want to maintain regular human form and go about doing things in the biological human world, but aren’t hung up on use of advanced technology within that context.

Imagine London was like a human preserve, for people who want to remain in their human bodies and live sort of human-like lives rather than uploading into video game worlds or fusing with the superhuman AI. What would London look like in that context? Well, no one has to work for a living anymore, so there are no more offices and jobs, because stuff that you need can just be 3D printed and provided to you, and the Tube can run automatically, controlled by AIs; you don’t need people operating it. Then, 3D printing babies: of course there’s still limited space in London. If people don’t die, there would have to be some limit on producing new humans, or all the space gets filled up.

Brian Rose: So who makes those rules? Because, like you said, Putin said whoever creates the AI rules the world.

Ben Goertzel: I would say this is London; it’s still a democracy at this point, so that should be democratic.

Brian Rose: What if Russia develops the AI and then it rules the world?

Ben Goertzel: Well, by this point there will be AIs that are vastly more powerful and intelligent than human beings. So I would view the democracy in a future human London as sort of like the student council in high school or something.

Brian Rose: It doesn’t mean anything.

Ben Goertzel: Well, it means something to the students.

Brian Rose: So they’re basically our overlords, the AIs.

Ben Goertzel: Yeah, but they’re very chill overlords. There’s no reason a super AI is going to care about regulating the details of human life. Why would it?

Brian Rose: Then why have us around?

Ben Goertzel: Well, that’s a different question. Consider the squirrels in the park, or the ants in your back yard. If you want to put up a new house, you may bulldoze over those ants to put up the house, just because you don’t care that much and they’re in your way. That’s one thing. But you have very little interest in choosing which worker ant gets to mate with the queen, for example. You could, but it’s not really worth your time; you’re dealing with other stuff, and you don’t have a lot of interest in which ant lives in which hole. So I think the scenarios where AIs want to enslave people or interfere with the minutiae of human life are very unlikely possibilities.

I think there is a plausible possibility where AIs are just much smarter and more intelligent and better than us, and they treat us the same way we treat squirrels or ants, which is not the best.

Brian Rose: And we have a consciousness, so we leave some in national parks.

Ben Goertzel: Yeah, we don’t want to make those species extinct.

Brian Rose: That’s true. We have an organic conscience; we have some morality.

Ben Goertzel: Well, it may have more than us. Ours is not that strong in the end, as you can see in the human world around you. So I think there’s a set of possible realities where the AI wants to help take care of humans out of love and compassion and nostalgia, and gratitude for creating it. There’s a set of realities where the AI is just indifferent to humans, and both of those need to be thought about. The set of realities where the AI is malevolent to us, or wants to enslave us, or meddle in details of human affairs, just seems extremely remote and unlikely.

1:03:09 I think in the shorter term, though, worrying about nasty humans using somewhat intelligent AIs to enact their nastiness on the rest of humanity is a real worry. How about in the period before you get a superintelligence, when you have AIs that are around as smart as the smartest people, not in every way, but they can still do a lot of powerful stuff and act semi-autonomously? This is where you get into robot army territory, or a robot cyber army that hacks into the power grid. There are a lot of ways you can imagine nasty national leaders in one place or another using AIs that are early-stage AGIs to do nasty things with robot armies, or robot armies of software on the internet, or AIs that design nasty bioweapons and put additives in the water supply. There’s a lot of imagination you could use there.

This is where AGIs are smart enough to be adaptive and autonomous in how they obey your commands, but they aren’t yet powerful enough to overcome their human creators. What if some advertising company or mobile phone company, or a combination of the two, was truly taken over by the intelligence division of some dictatorship, and was doing much more than influencing people to vote for Trump using fake news on Facebook? What if it was using the video games you’re playing, or the brain implant you have for Google search by thought, to affect the way you’re thinking? What if you had an AI that was really smart at understanding the human brain and human motivation, like a super advertising agent in the hands of some nasty dictator? There are a lot of things at the intermediate level of intelligence, where the AI is still being controlled by people, that are maybe more worrisome.

Brian Rose: What do we do about that stuff? Because that’s kind of scary.

Ben Goertzel: That is scary, because you see now that the people running the most powerful countries in the world are not necessarily the most compassionate and benevolent human beings on the planet. And even something like Bitcoin or Ethereum, which were created by mostly good-hearted anarcho-libertarians: arguably, various mafia organisations are controlling large bits of the Bitcoin and Ethereum ecosystems.

Brian Rose: Is that recent?

Ben Goertzel: Longer than that, because they can move money illegally. Right now cryptocurrencies are a good way to launder moderate amounts of money. If you want to launder billions of dollars it’s not good; you’ve got to use your local national bank. If you want to launder tens of millions of dollars, it’s probably… I’m not rich enough that I require that service at the moment, but it seems to be an effective way to do things.

Brian Rose: It’s an example of tech being used for bad intentions, and the same thing could happen with AI.

Ben Goertzel: Money laundering can have good or bad intentions. There are people in Iran who don’t have bad intentions, but they’re restricted from moving money due to international tensions that have nothing to do with them; it’s the religious government there versus the government in the US. It’s not their fault, they didn’t get their money by doing anything immoral, but they still want to transfer money out to business partners or family members. So the same tools that are used for good ends are used for bad ends as well, and that’s always the case. But I think there are a lot of dangerous possibilities out there, certainly.

1:08:07 One thing I worry about is that as automation advances, the need to have less educated human beings carry out work may disappear rapidly. We already see that long-distance truck drivers will be obsoleted, and we’ll see fully automated McDonald’s around pretty soon. In the more developed world, I think we’re just going to see some form of social welfare become prevalent, whether it’s a universal basic income or some similar mechanism. But who gives universal basic income to the average citizen of the Central African Republic, for example? If there’s no need to outsource labour to the developing world, then what happens? You can come and dig up their minerals and take them straight from the mine to the port, and you can pay the dictators of those countries for what you mined, but what about the average people there?

Of course you can say they’ll remain subsistence farmers, but then you have a really two-tier world economy. There’s already a huge class division in the developed world, between the 20 percent of the population that’s helping to create the Singularity and the 80 percent living on the dole. But then you have the developing world, which has no dole, and they’re just living as subsistence farmers who can’t afford medicine for curable diseases. Yet many of the young people there have the internet, they’re getting technically trained, and they understand advanced technology pretty well, but they’re not allowed to immigrate to the developed world.

So what are they going to say when some Russian mafia guy offers to pay them to do cyber warfare and hack into the power grid in the US, or hack into the missiles in China? Their whole country is being screwed by the international balance of power and technology. Then at what point does some first world country decide to exterminate everyone in that third world country, using drones that go in and gun down everyone seen with a laptop or something? There are a lot of dystopic potentials you can think of when you put together income inequality with military and surveillance technology powered by AI, and when you see military and surveillance technology used to preserve corporate profits rather than just national security.

Brian Rose: Some say it’s one and the same now.

Ben Goertzel: It’s very closely aligned, and especially if you look at Africa, it’s super closely aligned.

Brian Rose: Why Africa? Just the minerals, the extracting?

Ben Goertzel: Yeah, exactly. It’s deals between one government and another, made in order to assure companies their rights to mines, and that’s just the way it works.

Brian Rose: I had a Navy SEAL in here, Jocko Willink, and somebody one time said to him: you’re just part of the military-industrial complex; don’t you worry that the people you kill are just there to support the industry? And his frank answer was: yeah, it’s what makes America great, our military and our industry, and I’m there to do whatever it takes to make us great. So it was always one and the same to him.

Ben Goertzel: And China has been in some ways subtler and more humane about it. They’ve been developing Africa for the last couple of decades, because the West didn’t want to.

Brian Rose: Is that why?

Ben Goertzel: Pretty much. We had other fish to fry; we were too busy blowing up innocent Arabs to mess with that.

Brian Rose: China’s investing in the resources of the future, in a way.

Ben Goertzel: There’s light rail in Addis Ababa, Ethiopia, and the train there has the logo of the Shenzhen Metro on the side; it’s just a Shenzhen Metro car plunked there in Africa. But that’s all done with debt financing, so the Ethiopian government owes money to China for this, and if they can’t pay it back, there’s always a bargain to be struck. China is using businessy methods rather than military methods to secure leverage over developing countries. In a way that’s just doing what America and Western Europe used to do, but smarter. They’re being very judicious about it, and the average Africans love them because they’re building trains and highways. However, if you notice, the best highway and train line always go straight from the mine to the port, and it’s not a coincidence. A lot of the profit from this development in many African countries goes straight to the Swiss bank account of the ruling family rather than being distributed among the people.

They’re also putting factories there now, though. You look in the outskirts of any big African city, and there are people assembling shoes and mobile phones; they’re doing the final assembly of parts made in China and Southeast Asia, for sale in Africa. The thing is, what happens when that’s no longer economical, when it’s easier just to have the automated factory assemble the whole shoe? The Chinese companies are capitalists first and foremost, so they’ll just have the robot factory in Vietnam assemble the shoe and ship it to Africa to be sold, rather than having Africans work on making it. But then how do the Africans get the money?

So the transition to the Singularity doesn’t have to be ugly, but it looks like it may well be ugly, and it will be ugliest in the developing world. How that will spill over, I don’t know. The current issues with immigration are just scratching the surface; it’s going to get way, way worse than that once automation renders outsourcing labour to the developing world totally irrelevant.

1:14:53 Brian Rose: And you have offices in Ethiopia.

Ben Goertzel: We have an office in Ethiopia, and I’ve travelled extensively in Africa. It’s fascinating, because the young tech geeks of Africa are just as good as the young tech geeks in Silicon Valley, Beijing, London or anywhere, and they’re really part of the global economy. They’re doing outsourcing work for foreign companies from home, or they’re emigrating and sending money home. But that’s a little portion of the population there. You’ve got all these people my age, or 10 years younger than me, who have been subsistence farmers forever. They’re devoutly religious, and they really are not prepared for the Singularity by any means. They’re still part of an economy that’s primarily oriented around subsistence farming.

But their kids are seeing their relatives dying of curable diseases in their home village because they don’t have enough money to get modern medical treatment, and these kids understand advanced technology quite well. So there are serious ethics issues here, which may turn into serious cyber warfare initiatives. And then, are the leaders of African countries prepared for the subtleties of the transition to the Singularity? The leader of Ethiopia actually is: Abiy Ahmed has a software engineering degree and understands advanced technology. But that’s not the majority case among African countries.

Brian Rose: This is a bit like when people had to adjust to the Industrial Revolution, but faster, and that’s the problem.

Ben Goertzel: A thousand times faster. That was ugly too, and it was ugly in Africa too, but it unfolded over generations, whereas now it will unfold within a single generation.

Brian Rose: And it’s happening. It’s just going to happen. I think most people walk around and don’t think this is happening yet. Ben, do you get that idea? I know you go to conferences.

Ben Goertzel: That’s like the parable of the frog in boiling water. People don’t think the Singularity is happening; meanwhile they’re video chatting with their friends all day and using Alexa to get music. So bit by bit they just accept what happens, and when their self-driving car drives them around, they’ll be like: cool, more time to play video games. They’ll sit in the passenger seat.

Brian Rose: We do have an implantable device for our brain; it’s just not implanted yet. It’s the mobile phone.

Ben Goertzel: Right. People take for granted what has happened already, and they adapt pretty quickly, and in a way that’s a good thing. People may never feel the Singularity is happening, because it’s just step by step.

Brian Rose: And then it just happens, and you don’t know any different.

Ben Goertzel: Well, if it happens in a benevolent way, it’s just progress. The AGI isn’t going to forcibly mind-upload everyone in two hours or something. Just due to the inertia of human society, the only really ethical way to roll out advances in human society is a little bit gradual anyway, because people take time to adapt to things. Certainly you’ll see faster and faster change than we have now, but there’s sort of an upper limit to what people can absorb. So if we rule out disaster scenarios, it’s always going to feel a little bit gradual to the people involved.

Brian Rose: I guess just like you can get a new phone every six months, but you couldn’t get a new one every hour, because you don’t have time to learn all the features.

1:19:00 You do some work in the robot industry. Should we be fearing robot armies and Terminator 2-like scenarios?

Ben Goertzel: I don’t think robots that look like people marching into your city and gunning you down is the most likely scenario, but I think automated weaponry is part of every major country’s vision of the future of combat. There can be a combination of drones, things that roll, some things that walk, and surveillance cameras that are everywhere. It’ll be a combination of many different technologies. I think we should be worried about surveillance, and the surveillance apparatus being hacked by the enemy too. There are already cameras in a lot of places in London. They’re not networked together as well as they are in China, but they will be, so you’ll have a central repository of video camera feeds from every street in London. Now, what happens if some enemy of the UK hacks into that system and then uses it to control drones that fly around in the street and do nasty things to whatever people they want to get rid of?

There are a lot of possibilities there. Probably the physical form factor of the Terminator marching down the street isn’t usually going to be the most effective way to do things, though there are some cases where it could be. Some version of weaponised robots is going to happen, but the extent to which that plays a real role in world affairs is not very clear at this point. At the moment it seems more like drone-based warfare from the air, and then surveillance apparatus telling human cops where to go and what to do.

The thing is, even if there’s a human cop in the loop, the question is how much it matters. If it’s an autonomous self-driving police car, and the weapon the cop is holding has some AI override, and there are automated surveillance systems in everyone’s apartment building and even in their house telling the cops what to do, how much does it matter that there’s a human guy made of meat holding the machine gun and sitting in the passenger seat of the self-driving police car? He’s being told what to do by the infrastructure. We’re hung up on whether there’s a human making the kill or do-not-kill decision, but humans are terrible at making those decisions anyway. I don’t know the UK that well, but in the US, human cops seem to make the decision of who to kill or not based on race above anything else. So it’s not like having a human in that role, among all the automated IT systems, makes a lot of beneficial difference, from what I can see.

1:22:36 Brian Rose: All these bad sci-fi movies are going through my head. Ben, do you have a tendency to scare people when you talk to them?

Ben Goertzel: Usually, in the West. In Asia everyone’s optimistic; Westerners are just paranoid.

Brian Rose: You live in Hong Kong. Is it because the West is like an empire in decline, and in Asia it’s like an empire rising? Is that why people there are more optimistic?

Ben Goertzel: I think at a gut level, everyone in China has had the feeling of getting richer and richer every year, because they are. Thirty years ago China was a very poor country. If you look globally, something like 80 or 90 percent of everyone who has moved from poverty to the middle classes is in mainland China. They’ve done an amazing job of urbanising the rural poor. People there didn’t have a car five years ago; now they do. They never had a phone five years ago; now they do. They never had a refrigerator or air conditioning; now they do. So they just see things getting better and better, and the natural gut feel is that robots will probably make things better and better. Whereas in the US and UK, even if people’s lives are getting better and better by many objective material analyses, somehow that’s not the vibe many people have. There’s a little more of a worried and paranoid vibe, for psychological and cultural reasons.

Brian Rose: Out of control, I guess, is what it is.

Ben Goertzel: It’s a bit weird, because people’s lives are actually improving a lot. The mobile phones are tremendous.

Brian Rose: They’re improving, but the Singularity is such an unknown that you understand why people would be a bit freaked out.

Ben Goertzel: I guess. But in Japan people just assume robots are going to be their best friend; they’re not freaked out.

Brian Rose: What’s it like to live in Hong Kong? I know your wife’s from mainland China, and you understand China probably unlike anyone else here or in America, and you’ve also spent time living in America. What are some misconceptions about China from the West? That it’s not democratic?

Ben Goertzel: First of all, Hong Kong is very different from mainland China. I live in a rural village with wild animals all around, right near the beach, and it’s a very pleasant, rustic type of place really. Yet I’m a 30-minute drive from the centre of Hong Kong, which is an amazing, bustling financial centre. So Hong Kong to me is mostly about beautiful nature, which is not how many people think about it. That’s the legacy of the British colonisation, really. There’s no city in mainland China which has left so much beautiful nature right next to an urban area, so I’m grateful to the British Empire for being amazing at land management and nature preservation.

But I think Westerners have a huge misconception that the people in China feel deprived of freedom and lorded over by some fascist dictator. The people I know in China are a subset of the Chinese population, mostly two sets: relatively educated people involved in the tech economy, and people in my wife’s home village in Anhui province who are rural farmers. They’re Han Chinese rural farmers, so I don’t have a lot of friends among the Muslim population in Xinjiang province, where a lot of unpleasant things seem to be happening. But among the segments of China I have anything to do with, people are very nationalist, as much as in the US. They believe China is the best country in the world and things are better off there than anywhere else. They think the government has pluses and minuses, but overall they’re happy, because it’s been making China richer and richer for their whole lives.

China is the most capitalist country on the planet. The people there that I know are mostly concerned with how to make more money and how to start a new business and make it successful, and they’re very family-oriented: how do I take care of my parents and my kids? They are not focused on politics; they would rather not think about politics or deal with it at all. They don’t see the government as very intrusive in their lives. The Chinese government wants to support capitalist and entrepreneurial activity, and it wants to support people having robust, happy and healthy family lives. As long as you ignore politics, it doesn’t feel heavy-handed in any way. If you try to do political organisation, or if you put something on the internet in Chinese that’s against the government, then you’ll run into some unpleasant things from the Chinese government. But as long as you avoid that, which most people there are very willing to do, it’s a very comfortable life with rapidly increasing wealth.

It’s a really healthy entrepreneurial ecosystem. If you want to do a software startup, there’s probably nowhere better on the planet than Beijing. If you want to do a hardware startup, there’s nowhere better on the planet than Shenzhen. In my wife’s home village in Anhui province there are not a lot of tech people, but so many people have made themselves rich with real estate remodelling or property companies, something like that. I don’t think people understand how capitalist and entrepreneurial the whole vibe in China is, nor how nationalist the Chinese are, really believing and feeling like they’re on top of the world. Historically, they’ve had 5,000 years, most of which they were more advanced than the rest of the world. They had a few hundred years of being backwards, and now they’re taking their rightful position at the top again.

1:28:46 Brian Rose: That’s interesting. So more capitalistic than America, by far?

Ben Goertzel: Yeah, sure.

Brian Rose: Explain that.

Ben Goertzel: Well, Americans are fat and lazy and comfortable.

Brian Rose: And the Chinese are hungry.

Ben Goertzel: They’re hungry and they’re rising. But I think the Chinese have been merchants and capitalists since forever; that’s in the cultural DNA to a huge extent. When Deng Xiaoping, in the 1980s, started socialism with Chinese characteristics, that was way more of a quiet revolution than people outside China realised. It wasn’t formally a revolution, but it was a total change in the way things are organised. What’s happened there since the ’80s really has nothing to do with Maoism, even though Mao’s face is on the money. I think people think China is communist in some way they can understand, but it’s not about that anymore. It’s a new form of government and a new type of state-directed capitalism, which we just have no model for in the West.

One of the beauties of that system is they’re able to plan five, 10, 20, 30 years in advance in a very coherent way, whereas in electoral democracies, which I still prefer emotionally, being an American, it’s hard to do planning beyond the election cycle, and that’s a problem. China is into AI, they’re into solar power and renewable energy, they’re into robotics, they’re into nanotech, and they’re able to plan these things decades in advance in a coherent way. That’s a very significant advantage. China is run by engineers, whereas the US is run by lawyers…

Brian Rose: And the UK is run by morons.

Ben Goertzel: The US too, which is a problem. Xi Jinping is a very smart guy, and he’s got a whole organisation of smart engineers around him who are figuring out the optimal way to design the society: what disciplines the universities fund students to be in, and which state-owned companies in which disciplines get money. In Guangdong province now, and in many parts of China, if a real estate developer wants to develop a new office complex, there’s strong pressure from the government to have a couple of AI, robotics or advanced technology companies in that office complex. So this pushes the property developers to fund the AI companies, so they’ll have an AI company in their office complex.

Brian Rose: It’s probably not the most efficient use of capital, but it gets their agenda done.

Ben Goertzel: Yes. It’s very complex; it’s 1.5 billion people, and there’s still corruption and a lot of inefficiency there. But the government is very clever at figuring out how, within the system they have now, to nudge things in the direction they need to go in, and it’s not as top-down as you would think. With this mechanism I just described, on the one hand there’s some inefficiency; on the other hand it’s encouraging the holding companies that own property funds to be thinking about AI, and it pushes AI investment local in that sense. So it’s very subtle. But the thing is, they’re embracing AI, robotics, nanotech, renewable energy and biotech in a huge way that the West can’t even imagine, and they’re putting a significant percentage of GDP into these advanced technologies.

Whereas the US is fussing about building border walls, and the UK is fussing about how you deal with tariffs after a no-deal Brexit or something. China isn’t chasing its tail around in circles like that. China is thinking: how do we dominate AI, nanotech, biotech and robotics 25 years from now? It’s different.

Brian Rose: And is that scary?

Ben Goertzel: It’s both exciting and unnerving, because I’m sort of an anarchist hippie libertarian type. I have a great emotional preference for a system like the US or UK, or Hong Kong as it is now, where I could go out in the street with a megaphone and say the president sucks balls, and I’m not going to get put in jail. I can publish whatever book I want; there are no censors. I grew up with that in mind, and I greatly appreciate it; it makes me more comfortable. Having parts of the internet blocked when you’re in China is very annoying to me. It doesn’t affect me personally, because my mobile phone has a Hong Kong SIM card, and then you’re not blocked in China, actually. But just the concept rankles me.

On the other hand, if I look at it more objectively: Brazil, where I was born, has not been remotely as effective at lifting people out of poverty as China has. If you’re living in the favelas in Brazil, in some hut without proper electricity and with no education, how good is it that you can go out in the street with a megaphone and denounce the president? So there have been a lot of advantages to the way China has developed. If you compare China to India or Brazil, it’s not clear you can say you wish China had developed with a more Western-style system like India or Brazil. Arguably China has done better for its people, in spite of some aspects that are annoying to me as an upper-middle-class educated person. So that’s quite subtle.

1:35:08 What I would like to see is AI develop in open, decentralised global networks, because then you’re cutting past all these issues. China has its pluses and minuses, as the US does, as the UK does, as Africa does. Every government system has its pluses and minuses. Iran has its pluses and minuses; there’s a great university system there. You would like AI to develop the way the internet or Linux has developed, in a way that cross-cuts all these problems. I don’t want to get involved in politics anywhere. I want AI to develop in a way that is orthogonal to politics, and that just networks together companies that need AI to do practical things and developers who want to develop AI.

I think we can do that. We’re developing our SingularityNET platform in a way that works in China and in the West, respecting laws about blockchain in the different places. We’re working with major Chinese companies like Ping An and some other big insurance companies there, and working with the Guangdong provincial government. At the same time, we’re working with smart cities in France and with Domino’s Pizza in Malaysia.

Brian Rose: What are you doing with Domino’s?

Ben Goertzel: We’re optimising pizza delivery in Kuala Lumpur. After creating this decentralised AI platform, among other things we’ve created a for-profit company called Singularity Studio, whose purpose is to develop products that back-end on this decentralised AI network, and we’re selling these products to companies in China and in the West. Business has a way of cutting past all these nasty political problems. There are limits; Iran is hard to do business with now as an American. But all this madness with Trump and China hasn’t affected us. It’s blocked certain roads, but there are a lot of other roads for collaboration open, and there are no limitations on selling AI software into China.

SingularityNET is a Dutch company, headquartered in Hong Kong, so we can do business in China, the US, the UK, France, wherever. To the extent we can build up more and more AI collaboration, with business projects built on open source software at the foundation and decentralised networks as part of that infrastructure, then we’re building up AI in a more decentralised, participatory way, which can maybe cut through some of the nasty dynamics that you’re seeing. Because one of the interesting things we found is that most corporations don’t want to see Google, Facebook, Alibaba or Tencent own everything either. If you’re a major manufacturing company or an insurance company, you don’t want to outsource all your AI to Google or Tencent, because then they’ll try to own your business in some way. You’d rather see some open source, decentralised network become prevalent, because then you’re on an equal playing field with every other corporation.

Brian Rose: So the Googles and Facebooks would rather you not succeed in what you’re doing. Facebook is trying to own remittances now. What do you think about Facebook Libra?

Ben Goertzel: My prediction is as follows. Libra will succeed reasonably well, like WeChat Pay has, which is the Chinese remittance system integrated with WeChat, probably the world’s best social network software. So Facebook will get remittances integrated with Facebook Messenger and WhatsApp, which will get some uptake. Then you’re going to see some horrible hacks and privacy scandals, which Facebook seems bad at avoiding. You’re going to see some developing world countries try to clamp down on it, because they don’t want all the financial data from their country going into Facebook’s databases and not theirs. And probably as a result of that, you’re going to see some fundamentally decentralised, cryptocurrency-based remittance system come afterwards, as the next phase after Facebook Libra.

So in the end it will get some adoption, some bad things will happen, you’ll have some backlash against it, and finally you’ll see a decentralised, cryptocurrency-based payment infrastructure come about. I think it will help that process along. Ultimately, I think you’re going to have a decentralised Facebook: instead of Facebook, you’ll have some social networking platform that isn’t owned by a big company. Sometimes humanity needs to go through that learning process, even in ways that might seem obvious to some of us beforehand.

1:40:16 Brian Rose: Do you think we’re living in a simulation right now, and we’re just talking about this as uploads?

Ben Goertzel: I think we have no idea what the fundamental basis of our reality is. That’s the bottom line. The idea that we’re living in a virtual reality simulation that’s in some way analogous to a video game seems highly unlikely to me; I think that’s kind of a lack of imagination, really. But the idea that there is some reality outside the reality we know about: probably there is. We’re probably very ignorant. I think about my dogs at home when they watch me sitting at the computer. What do they think I’m doing? Guarding this physical object, wiggling my fingers to get some exercise? They have no idea I’m controlling vast networks of computers halfway around the planet or communicating with other people by text.

In a similar way, when you get an AI much smarter than human beings, it’s probably going to perceive aspects of the world that are totally opaque to us due to our limited scope of cognition, and we’re going to discover the universe is something utterly different from what we think it is. But the odds that it’s a simulation or a video game seem very low. It’s going to be something we can’t even imagine.

Brian Rose: Wow. And are you going to live forever, or upload your consciousness?

Ben Goertzel: I certainly hope so. If not, I’m counting on my kids to figure out how to reincarnate me.

Brian Rose: So are you going to 3D print any kids?

Ben Goertzel: We’ll see how things evolve. It may be that virtual reality has become more interesting, but there’s certainly that possibility. If there are resources, it would be a lot of fun to have millions of clones of all of us and let them all diverge in many different directions, but this may happen in virtual worlds rather than the physical base reality. The process of raising a kid is partly wonderful just because of the biological constraints on it. It’s not clear that 3D printing kids is even going to seem worthwhile to me. I’ve loved having children; it’s one of the most rewarding parts of life, but it’s sort of bound up with the whole traditional biological way of being.

Brian Rose: It gives our lives meaning.

Ben Goertzel: Well, we think it gives our lives meaning right now, but there are probably a lot of other ways to derive meaning in the universe also. I would like to fork myself into multiple copies: let one remain biological and keep producing kids in the good old-fashioned way, which has its rewards also; have another version of me live in the virtual world, where it can take any physical form it likes; and another version give up its humanity and merge with the superhuman god mind. You’re not limited to just one future. You could make many copies and let them do many different things.

Brian Rose: And then you’re in three different realities, and you communicate with each other or not?

Ben Goertzel: You can, but it would be interesting, because the version of you that uploaded to become one with the superhuman god mind may be evolving at 10 billion times the speed of the human version of you, so it won’t necessarily have much to say to that human version. Between each word the human version said, there would have been billions of years of subjective time for the superhuman mind version, which would have drifted off in some totally different direction.

The thing is, I hope to live long enough to experience all this. I’m sort of at the cusp, I think, because I’m 52. If we really get the Singularity by 2045, I’ll probably be around by then. I’m in reasonably good health, but it’s far from guaranteed.

Brian Rose: Twenty-four, 25 years.

Ben Goertzel: Right. My parents are in good health in their mid-70s, so I should still be around then, but you never know for sure. On the other hand, if the Singularity came in 2070, which is what my father predicts, then unless we get some radical medical advances I’ll be dead by then. But we’re looking at a strong potential of radical medical advances, and those could occur even before a full-on Singularity. So whether you and I survive to see all this is going to depend on the specific rate of advance of various technologies that are currently under development. But my kids, and my granddaughter, who’s 14 months old now, are going to see the Singularity unless something really unfortunate happens, which is amazing.

Brian Rose: And that excites you?

Ben Goertzel: It does.

Brian Rose: And that’s assuming we don’t screw it up and set off some nuclear missiles or something.

Ben Goertzel: We’re going to screw up a lot of things, and if some nuclear missiles are set off, that will be tragic, but it probably won’t stop the Singularity. I think you could kill off billions of people without stopping the Singularity from happening. I’m really hoping it doesn’t go in that direction, but we can’t rule that out when you look at the idiocy at the highest levels of government in many places on the planet right now.

1:46:02 Brian Rose: What didn’t we talk about? Are there things on your mind these days, when you travel around the world and talk to people, that you want to share? Things people ask you, or any other thoughts about how people react when they hear you talk about this stuff? What’s the general consensus or reaction?

Ben Goertzel: In terms of public speaking, I tend to speak to fairly special audiences: people in the technology business world, or in the AI research world or robotics world. They’re very excited by these developments overall. We’re spinning off all these initiatives from SingularityNET: Singularity Studio for enterprise software, a project called Rejuve to use AI for life extension biology, and a project called NuNet that lets you use your mobile phone and other internet of things devices for AI processing. People are very excited by all these options, and they’re excited by Hanson Robotics in the next couple of years mass-manufacturing Sophia-type robots for businesses everywhere.

But I’m conscious this is a very selected population. If I talk about these ideas to, say, rural Ethiopian farmers, there’s a lot more confusion about all these possibilities and what they’re likely to mean. What’s interesting, though, is that if you talk to farmers in rural Sichuan province in China, or rural Ethiopia, you find people are really eager to have AI help them solve the problems of their lives. I went to Sichuan province, and we’re looking at how AI can help with early diagnosis of crop diseases. The farmers there, or the coffee farmers in Ethiopia, would love to have AI tell them earlier when their crop may be getting a disease, so they can be more judicious about when to apply pesticides, save money, and not buy them when they’re not necessary.

So I think there’s a lot of openness all around the world now to AI helping to resolve practical issues people confront in their lives, even among people who don’t want to deal with the idea of the superhuman mega god-mind Singularity taking over everything. That’s probably what you want to focus on when you’re talking to ordinary people. The people I talk to in the technology world also tend not to be traditionally religious, whereas in rural Ethiopia most people are heavily Ethiopian Orthodox, and 30 percent Muslim. These guys are very ready to embrace AI helping solve their practical problems, but on religious grounds they don’t believe AIs are ever going to become as creative, imaginative or generally intelligent as human beings. I’m not really in the business of trying to unconvert people from their religion or convince them an AI is going to have a soul or something.

But the practical applications of AI are almost a universal language now. Everyone can relate, no matter where you go. They can see AI is going to solve a lot of problems and transform the world, and they can also see it’s going to be better if the AI is owned and controlled in a decentralised way. The question of who owns the future really resonates with everyone, because they don’t want AI colonialism. They want AI they can participate in and see some of the rewards from, so this aspect of it resonates very well also.

Brian Rose: Tell me something about you. What is something you do every day that’s just non-negotiable, that allows you to keep being who you are and stay productive?

Ben Goertzel: I’m very nature-oriented, and very family-oriented at this particular phase of my life, because my wife and I had a baby 16 months ago. When I’m home in Hong Kong, which to be fair is about 40 percent of the time lately, since I’ve been travelling constantly, every morning I get up with my 16-month-old baby son and we go walk on a trail through the woods. We go play in the water; we live near the ocean, so we play with the starfish and watch the flying fish jump up and down. There are free-range cows wandering around, and wild pigs and monkeys and so on. To me, being grounded in nature every day is highly inducing of mental peace. Being engaged in nature with a little baby is also very interesting, because babies are one with the moment they’re in. If you’re out there splashing in the water with the starfish and the baby, that’s the whole universe at that moment.

Most mornings I try to meditate for 15 or 20 minutes also. We have a rooftop balcony with a beautiful view of the ocean and the mountains; it’s right near the sea, with very fresh air. So again, that’s being outdoors in nature. I think this is an important aspect, because to me it’s important to be open to all forms of life and intelligence. You’ve got humanity, and you’ve got superhuman minds that we’re trying to build in robots, but you also have plants and starfish and cows and monkeys. The attitude of wanting to foster connectivity of all sentient beings is quite important. If we can get that attitude into our super AIs, we’re going to be a lot better off.

Brian Rose: I totally agree. If I asked a bunch of your friends what Ben’s superpower is, how you’re able to do everything you do, what would they say? I’m sure there are better programmers out there than you.

Ben Goertzel: Yeah, probably just two or three. I’d say what my wife is often perplexed by is that no matter what’s going on, even if I’m really overtired, or waiting in line at the airport, or there’s a lot of noise around or annoying things are happening, I have the ability to focus completely on whatever I want to focus on. And I’m driven by abstractions. I never lose track of complete focus on whatever high-level abstract ideas I want to be focusing on. I’m perplexed by the way people cannot focus on whatever they want to focus on, and I’m perplexed by people who don’t model things in the most abstract possible way that makes sense, but only focus on the complex details.

So if there’s some peculiar thing that characterises me, it’s that I’m driven by total focus on certain abstractions and then how they ground themselves out in the particular things I’m doing, whereas most people I see have a short attention span and a hard time focusing on things, and are driven by very particular short-term situations, which has always felt strange to me for some reason. My kids are all great, but they all had trouble with attentional focus, and they’re always asking me for advice: how do you just narrow your focus on whatever you want at each point in time?

One thing about me that no one believes is that I’m fundamentally a very introverted person, and it’s by great force of mental effort that I’m going around and interfacing with people as much of my time as I am now. I realised at a certain point that if I want to achieve my goals of creating general intelligence and spreading benevolent AI around the world, the most rational thing to do is not sit at home and do maths and programming, but rather build businesses, do interviews and recruit new people into projects. So I’ve forced myself to be extroverted in practice, but my closest friends would tell you I’m actually quite introverted at my core.

Brian Rose: That makes sense. That’s probably why you’re able to get so much done as well.

Ben Goertzel: Perhaps introverts are better at focus.

Brian Rose: I think so.

1:56:01 What was the best day of your life, and the worst day of your life?

Ben Goertzel: Best day of my life: probably watching my first son be born, I guess. Watching all my kids be born was very moving, and with my second and third babies I delivered the baby myself, which was amazing. But the first kid, just because it’s the first, has a certain special amazement to it: that you’ve created this human. Worst day of my life: when my oldest son was four years old, we had a car accident and his skull was fractured, and we didn’t know if he was going to live or die. So these are very human experiences rather than AI experiences, I guess. Creating the Singularity may be even better than all these things.

Brian Rose: But with the Singularity, maybe you’ll never experience those things again.

Ben Goertzel: Well, there will be all sorts of new things to experience. I’m not interested in rolling backwards. I would say having my first kid was probably more amazing than having other kids will be. But I’m married now for the third time, and I would say the amount of joy from that relationship is more than I experienced in any previous relationship. So that’s something new: wow, a marriage can be happy and loving rather than tortured and complicated. When you have a robot that can really understand everything you’re saying and become your best friend, being truly best friends with an AI, with mutual compassion, love and understanding, with an AI system that I myself have helped create and teach, that’s going to be an amazing personal experience. That’s part of the path to the Singularity, but yet it’s a very intimate human experience. Can you imagine really having a best friend who’s an AI? That would be amazingly cool.

Brian Rose: Like the movie Her. That was his girlfriend.

Ben Goertzel: Yeah, but that was a bit twisted, because it’s a twisted one-way love relationship or something. But I can imagine that it could be really cool.

Brian Rose: Someone who is witty and knows you, who can challenge you as a friend, but is actually an AI.

Ben Goertzel: It’s going to be a very amazing experience.

Brian Rose: All right, well said. What scares you?

Ben Goertzel: Dying still scares me, though not as much as it did 20 years ago. Again, being human, having kids and grandchildren gives you more of a feeling that you’re just part of this overall process of humanity and life. But on the other hand, I would much rather not be dead; I enjoy living. I can’t be that paranoid about it, because I do rock climbing and bungee jumping and a lot of high-risk things, but dying is bad, and I think no one should have to involuntarily die.

In terms of broader issues: if I look at the leaders of most countries now, and at how large corporations are operating, then, as we’ve already discussed, the idea that the first superhuman machine could be, say, a soldier who’s loyal to Donald Trump, or a Wall Street trader loyal to Goldman Sachs… I think if the first AGI with human-level intelligence has this sort of mentality, it’s not going to be a good thing. I’m happy with the US government and Goldman Sachs continuing to exist and do their thing; they’re both beneficial in some ways. But if the first AGI is an outgrowth of some organisation with a narrow goal, like preserving this country or maximising shareholder value of this company, then in the big picture it may not matter, because the superintelligence will go beyond humanity and go its own way anyway. But that could lead to a situation where me and all my kids, and you and your kids, were all turned into extra processing material for the super AI.

I don’t want that to be the case. If we’re merging with the super AI, it should be through a joyous experience of our minds blending in with the superhuman intelligence. It shouldn’t be by having our molecules disassembled and turned into computronium to fuel the AI’s hard drive.

Brian Rose: No, that’s well said. What would we be surprised to learn about you, Ben? Something you do in your spare time, or some guilty pleasure?

Ben Goertzel: I’ve been sufficiently open with my life that there’s probably not much that’s a big secret at this point in time. I play the piano every day.

Brian Rose: Really?

Ben Goertzel: Mostly just improvising, but I compose a lot as well. I’ve played the piano on stage before too, so it shouldn’t be incredibly surprising. Honestly, my daily life is probably more routine and less interesting than most people would speculate. It’s a pretty nice village house in Hong Kong, but it’s not a mansion or something. I live with my wife and the baby, we eat meals at home, go walking around our village and play in the playground with the baby. It’s a fairly matter-of-fact life in a little rural village. Now, what I’m doing when I sit down at the computer in my study is atypically interesting. But there’s no castle of crypto-anarchy where we’re sitting around all day served by humanoid Sophia robots or something.

Brian Rose: It doesn’t look like that?

Ben Goertzel: Not yet. Maybe next year; we’re working on it.

Brian Rose: What keeps you awake at night?

Ben Goertzel: The baby. He wakes up in the middle of the night.

Brian Rose: Nothing else is on your mind?

Ben Goertzel: No, I’m a pretty peaceful-minded person. I think I’m usually pretty happy just by my natural psychophysiology. Nothing keeps me awake at night in the sense of worries. I really feel very fortunate. I have a great, loving family, I’m working on stuff I enjoy every day, and I have a sense of optimism about the technologies we’re building and the good they’re going to do. If I think about it rationally, I have to say we have no idea what’s going to happen when we create superhuman AIs; on a rational basis, the data we have does not determine what’s going to occur. On the other hand, in a spiritual sense, or as an underlying gut feeling, my feeling is things are going to come out amazingly well.

We’re going to look back on 2019 as an era of unimaginable ignorance, isolation and suffering, and we’re going to be in a much more joyous realm of experience after technology has let us overcome the limitations we now have in our physical and mental lives.

Brian Rose: Wouldn’t that be incredible?

2:04:34 Ben, I always ask people a few questions at the end. If you could pick up a phone and call the 20-year-old Ben Goertzel and give that young man some advice, I guess you were at Temple then, besides telling him to buy Microsoft stock, what would you tell him, based on what you know now?

Ben Goertzel: That’s a subtle question, because the question is: do I tell him to break up with his girlfriend, who became my first wife? I had three kids with her, but it was a horrible marriage. But I don’t want to de-exist my first three kids either, so there you go. Certainly you could tell him to invent blockchain; that would be interesting. That’s something where there’s a lot of money, and it’s a very, very simple technical idea, which would be easy to communicate. It’s not a stock pick, but it would make you rich, and it would also launch a technology revolution much sooner than has happened.

There are a lot of things about AI that would be interesting to transmit, but those would be a very long conversation; there’s a lot of subtlety and complexity. But it would certainly be worth transmitting that multi-layer perceptrons, which is what deep neural networks were called back then, if you implement them on GPUs, will be able to solve face recognition and Go. That would be an interesting little tidbit, because those algorithms were well known back then, but no one quite knew that just accelerating them on GPU hardware would lead to such an incredible boost of performance. So there are some bits and pieces of technical knowledge which, if told to someone in the 1980s who would listen, would have made certain aspects of technology develop decades earlier than happened, which would be quite interesting.

Brian Rose: Low bandwidth, high impact.

Ben Goertzel: Yeah. Tell him to try DMT earlier also.

Brian Rose: Have your psychedelic experiences taught you a lot about the future of consciousness?

Ben Goertzel: Certainly my psychedelic experiences helped me understand how the mind works much better than I would have otherwise, and also gave me a whole different perspective on life, the universe and everything. I ended up much less of a hardcore rationalist materialist than I was before I started on psychedelics, and I did start at age 15 or something, so I’ve been at it a while.

Brian Rose: DMT or acid?

Ben Goertzel: There was a lot of acid where I went to college.

Brian Rose: At Temple?

Ben Goertzel: No, in western Massachusetts. I went to a place called Simon’s Rock College, which is part of Bard College. That was well before I was 20. I tripped on a lot of different psychedelics, and you do get the clear sense there that human understanding, and the reality we think is the be-all and end-all, is a very limited portion of the total mega-universe, or whatever you want to call it. I’d had that inkling earlier, because I was meditating a lot as a kid, from having read a bunch of my mother’s books on Chinese culture and Buddhism, and then I read Ouspensky and Russian mysticism and so on. So through meditation I’d already started to get into states of consciousness that made me doubt the absolute reality of myself or the 4D space-time continuum, but psychedelics definitely drove the point home.

Brian Rose: And that probably had something to do with your vision today, to a certain extent.

Ben Goertzel: Well, it’s very hard to go back and say what you would think if you hadn’t had this or that experience. If I hadn’t done psychedelics, I might have gotten deeper into meditation, because I’d already had the idea, from early study of Buddhism, to quest beyond our conventional understanding of what’s real and what isn’t. But I think psychedelics gave me a lot of insight into the nature of the mind, and into how different aspects of learning and reasoning work out, and how the mind builds up a reality to perceive, which has been an important part of my whole AI quest. In terms of architecting AI systems, a lot of it just comes out of nitty-gritty computer science and maths, but high-level inspiration often comes from introspection on how your own mind works, and psychedelics and meditation can be very helpful for that.

I also think, just in terms of coping with the craziness of being an entrepreneur in the cryptocurrency and AI markets, and all the rollercoaster ups and downs of life in the modern tech world, meditation, psychedelics and being in nature so much give a certain grounding that lets one cope with all this madness, which otherwise might just be too mind-blowing and drive you ape.

Brian Rose: Yeah, well said. Best advice you’ve ever received? It could come from a coach or your parents or somebody out there.

Ben Goertzel: My father is a sociology professor, and one time after another, when I’m interested in some question or he’s interested in some question, his first reaction is: well, let’s dig up the data and look at it.

Brian Rose: Last question. To that 20-year-old who’s watching and listening to us right now: advice to them, based on all of this, the fact that the Singularity is coming. What do you tell them?

Ben Goertzel: There’s no one path for everyone, obviously, and to give any one person useful advice I have to know that person very well. I think the most important thing to do is educate yourself on every level. Learn as much as you can about science, maths and every advanced technology out there, and social science as well; just ingest more and more information. At the same time, meditation is your friend, and psychedelics used appropriately can be your friend. Getting a deeper inner understanding and wisdom, as well as gathering as much knowledge as possible about technology and society, is important. Having done these things for a few years, the right way for you personally to contribute to the Singularity will become clear to you, and I certainly can’t tell you it.

Brian Rose: That’s good advice, Ben. Thanks so much for coming here and explaining stuff that’s sometimes difficult to explain, but I think we did a pretty good job.

Ben Goertzel: We’ve covered a lot of ground, and it’s been a fun conversation. We’re both still awake, so that’s good.

Brian Rose: But you know, like you said, it’s a bit like the frog in the pot, because it’s happening all around us. We’re carrying these devices around, AI changes our life all the time, and yet we also fear this bogeyman moment or this sci-fi dystopia. But the truth is it’s coming, and it’s about how we can make it better. I appreciate you challenging a lot of my views about that next level of evolution. Maybe we’ll look back at 2019 like it was the Middle Ages, or the days of Genghis Khan, when we were living like total barbarians and had no concept of human rights. Maybe that’s the way this year looks 200 years from now.

Ben Goertzel: I’m pretty sure eating meat will look that way, as one example. Or look at marriage as an institution, where people spend their lives in misery because they took a vow 40 years before. There are many, many things in common life, like going to work every day to carry out repetitive tasks you don’t enjoy because that’s the only way to get food and shelter. These things are going to seem insanely barbaric a few decades from now, let alone hundreds of years from now.

Brian Rose: It’s exciting stuff. Well Ben, all the best in your work. I hope you make sure we have a smooth transition into the Singularity, and that you wrest it away from these advertisers.

Ben Goertzel: Well, it’s not me. It’s going to be participatory, by vast numbers of people, including many people who are listening to us right now. This is a key thing to understand about the Singularity. From a high-level view you could say: oh, it’s happening. But actually, this happening is the collective effort of all of us on the planet. For this transition to the Singularity to come out in a beneficial way, it needs participation by vast numbers of people, and it needs creativity by vast numbers of people in figuring out the best way to help. Without that, the odds of a beneficial transition are much lower.

This is much of the reason I’m doing interviews like this. I want people who are listening to this interview to start deeply thinking, to start learning and studying, and to be figuring out how they can best contribute, just as I’ve done in my own life. Without that, it’s going to be haphazard, and haphazard means the path to the Singularity will be controlled by a few governments and large corporations. It’ll all come out in the wash in the end, from the cosmic evolution point of view, but from the little narrow point of view that wants human beings not to be squashed during the transition, what each of us does in the interim is very important.

Brian Rose: Powerful, man. Something for everyone to think about. All right Ben, keep doing the good work.

Ben Goertzel: All right, thanks so much for the time. I appreciate it.