Full transcript: Why The Godfather of A.I. Quit Google on London Real
0:00 Brian Rose: This is London Real. I am Brian Rose. My guest today is Dr Ben Goertzel, the scientist, futurist, author and entrepreneur. You are considered one of the world’s foremost experts in artificial general intelligence, and have submitted over 150 technical papers and published 25 scientific books on the subject. In 2017 you founded SingularityNET with the goal of creating a decentralised, democratic, inclusive and beneficial artificial general intelligence.
Your latest project, Zarqa, is a supercharged, intelligent, neurosymbolic large language model at massive scale, and promises to take on OpenAI’s ChatGPT but also go much, much further. You believe that your advanced neurosymbolic AI techniques will revolutionise and disrupt every industry, taking a giant step towards artificial general intelligence, or AGI. Ultimately, your goal is to realise a positive and beneficial Singularity for the good of all humanity, and to build the sentient beings of the future to be loving, caring, compassionate, inclusive and democratic. Ben, welcome back to London Real.
Ben Goertzel: Thanks a lot, Brian. It’s a pleasure to be here once more. The world keeps evolving and there keep being new things to talk about, even in the highly specific domains that I’m working on. These are amazing times.
Brian Rose: Yeah, and we’re going to keep having you back, because this is moving so fast and growing so exponentially. I think it’s important we have these regular conversations, so thanks for coming on. As everyone knows, we will be investing in Zarqa through London Real Ventures, and we’re super excited to support you on this partnership. Maybe we start off, Ben: what is the current state of the union on large language models and ChatGPT, and then maybe morph into a little bit about how Zarqa is going to be different? I know we’re going to cover a bunch of other topics, but where are we right now in narrow AI, large language models and so on?
5:22 Ben Goertzel: I think the crux of where we are right now with large language models, and large neural models in general, is about the same as when we talked last. Our last interview was post-GPT-4, and while many things have advanced, we don’t have a system generally smarter than GPT-4 yet. What has happened is that more and more systems comparable to GPT-4 in various ways are popping up all over. So it’s more and more clear that OpenAI’s early lead in this domain is not such a big long-term thing.
6:31 Google has the successor to Bard by now. Google has PaLM 2, which is a GPT-4-grade model, and they’re already integrating large language models into search. That’s looking like a major disruption to the publishing industry, because if they can summarise what’s on everyone’s website in a reasonably cogent way, they’re keeping you on Google’s site with a summary of all the news, and you don’t need to go to the other websites anymore. I wouldn’t say it’s quite a death blow, but it’s a major blow to many business models on the publishing side, and a very clear revenue model for Google associated with LLMs, probably clearer than what OpenAI has at this point. So I’d say Google was caught off guard and taken aback, but they’ve pretty much recovered.
7:52 On the other hand, you also had this leaked memo from an engineer within Google, to the effect of “we have no moat”: open source projects can do the same stuff that we’re doing, and in some ways they can go faster because they have all sorts of people collaborating and innovating. This Google engineer, who wrote the memo titled “We Have No Moat”, was pointing out not just that OpenAI’s early lead is meaningless, but that Google’s lead also is brittle. If Google tries to close-source their technology, then it will no longer be cutting edge, and open source tech will get smarter and smarter, which ultimately could pose a threat to Google’s hegemony on search.
If Google is putting one language model behind search, that’s great; it probably can stop the flow of people to Bing. On the other hand, if an open source language model was just smarter, then some VCs are going to fund a search engine based on that, which will start eating away at Google. So I’ve said that the tech hasn’t advanced a lot since we last talked, but the business aspect has developed significantly. OpenAI doesn’t have a unique power here, and big tech companies are probably going to have to keep opening stuff up and play nicely with the open source world in order not to be left behind. Just as we’ve seen with other forms of AI technology, and with operating systems and other parts of computer tech, even these trillion-dollar companies can’t keep up with the open source world, which is just a far, far larger community collaborating on stuff.
10:01 I’d say the public reaction to large language models has evolved since we last talked. We had this whole kerfuffle where the Future of Life Institute and a bunch of other folks from industry and academia signed a petition suggesting something like pausing development on larger and larger language models for six months. To me this was an exercise in virtue signalling, a PR stunt or something, because everyone knows these big tech companies in the US are not going to pause tech development while China and Russia keep doing tech development.
On the other hand, the way that the pause was described lacked any technical nuance, so it doesn’t really mean anything. Pause developing models with more parameters, okay, but what if instead of a model with a trillion parameters you have a model with 100 million ten-dimensional parameters? What if you have ten smaller models all talking to each other? It wasn’t defined in a meaningful way, and it’s very hard to pin down software in a meaningful way and regulate it, for similar reasons to why it’s hard to pin down software with patents compared to hardware engineering or biotech. There’s always multiple ways to skin a cat in software.
So I think the pause petition was sort of a PR stunt and a joke, but it also summarised some people’s very genuine concerns. If you look at what they were saying, it wasn’t so much about super-AGI annihilating the world, although that’s a major concern of many of the signatories. The petition was more about: these things are going to spread disinformation, and they’re going to disrupt the economy by taking people’s jobs.
12:12 To me, if you’re going to ban things because of disinformation, why haven’t we banned the internet a long time ago? A substantial percentage of it is devoted precisely to disinformation designed to brainwash people in the interest of one or another elite party. Taking people’s jobs, true; again, why don’t we ban the internet, why don’t we ban computers, why don’t we ban the factory? The whole notion makes little sense, and it makes little sense to pause for six months. What’s it going to do? Are we going to figure out how to get rid of misinformation online and how to reform the global economy in six months?
I think the reality is society learns how to cope with these big changes by coping with them. We don’t plan it out in advance. It’s not possible to think through these things in advance because of the complexity of the systems involved, and even if you could, you wouldn’t get the policies put into place in advance, because in a democracy and a capitalist system we always deal with things reactively better than proactively, unlike mainland China. That’s not what we are here in the West.
The reality is development of these things will not be paused. It’s good that people are thinking through the broader implications, and society is going to have to cope with these implications as they unfold. Many elements of society would rather not have to cope with them, but that’s the way things roll. You don’t get hugely beneficial new innovations in the real world without also having some transitional mayhem to cope with. If you have something that uplifts everyone, it’s going to alarm some people who previously had a higher status or amount of resources than someone else, so there’s going to be a bunch to cope with.
14:32 On the technical level, finally, to wrap up: one thing we’ve seen since our last conversation is an increasing recognition that the shortest path to making large language models that are less full of crap and better grounded in reality will be the hybrid, that is, a large language model with a knowledge graph in some form. GPT-4 was hybridised with Wolfram Alpha, they’re gluing it together with Microsoft’s knowledge graph, and there’s all sorts of startups doing this in various different ways. This is interesting to me because I’ve been pushing neural-symbolic methods, where you put a neural net together with a symbolic reasoning engine, as a fantastic path to AGI for a long time.
Not because it’s the only way to get to AGI; the brain doesn’t have a logic engine at the base level. But I think it’s a way to get to AGI that’s well suited to the computers we have today, which have logic operations wired into the chips. I also think it’s a good way to make an AI system that’s more rational, reasonable and sensible than people are. We might be less messed up if we did have a logic engine at the base level along with the neural net wiring that we have.
By hybridising a knowledge graph with a neural net in a simple way, you’re doing something. But if you really want to kick ass in that way, you need to take that knowledge graph and put logical reasoning on top of it, and big tech is not very well instrumented for that; they just haven’t been working on it. We have been working on that in the OpenCog project, and in SingularityNET, which funds a bunch of work on the OpenCog open source AI project. So we’re actually much better prepared to take a large language model, put it together with a knowledge graph, then do sophisticated logical reasoning on that knowledge graph and have that logical reasoning coordinate with what happens in the LLM. We’re way better prepared to do that than Google or Microsoft or Facebook, just because we’ve been pushing on the logic side, and connecting it with neural nets, for a long time, and they’ve been ignoring that.
17:16 That underlies a few different things in the SingularityNET ecosystem. You mentioned the Zarqa project. Zarqa is a for-profit spin-off of SingularityNET aimed at the LLM side: make a large language model, much like Stability and Anthropic and these other open-source-oriented LLM companies are doing, but use some of OpenCog to connect it with a knowledge graph, use some OpenCog reasoning in that connection, and make something that’s like ChatGPT but multiple times as clever by using neurosymbolic AI for smarter thinking, also running on the SingularityNET and HyperCycle decentralised back end.
Then there’s another SingularityNET spin-off called TrueAGI, which is looking at bringing this sort of technology to the enterprise. Each enterprise has its own knowledge base, its own knowledge graph. So you take the knowledge graph representing what’s happening inside a company, do OpenCog reasoning on there, connect that with LLMs, including a general-purpose LLM and an LLM fine-tuned on the text from that company, and use that to provide AGI-as-a-service APIs, running partly on premise but partly also on our decentralised cloud.
The general idea here is that we can get the world to use decentralised AI networks with no central owner or controller, and we have a pretty clear path to doing that over the next couple of years. It’s a little bit tricky, but not that tricky: we provide a better product. We make an AI that’s multiple times smarter than ChatGPT, or than GPT-5, because we know how to do neurosymbolic better than anyone else. So everyone wants to use it, in retail with Zarqa or in the enterprise with TrueAGI, because it’s good, and lo and behold, it happens to run on this decentralised back end.
Just like people use Stable Diffusion because it’s good, and they don’t mind that it’s open source but aren’t necessarily using it because it’s open source, they will use Zarqa and TrueAGI because it’s the best, not because it’s running on decentralised infrastructure, in most cases, unless folks are decentralisation zealots like you and me. People don’t mind if it’s running on a decentralised back end as long as it works.
19:52 Having done this, having this neurosymbolic thing there, we’re prepared to solve the next problem, which is fundamental creativity and invention. Until you can use logic to filter out what’s nonsense and what’s not, you can’t let your freak flag fly and just invent wild new things, because if you invent wild new things without a filter, you’ll just do insanity; you won’t do meaningful creation. Once you have the logic to ground the system’s productions in reality, you can integrate evolutionary AI, concept blending and other cool AI methods to boost the inventive capability of large language models and other AI systems, using the grounding and logic to filter out what’s good in a certain context and what isn’t.
It’s funny, because in a way you would think big tech has a big lead. But when you analyse it step by step, you see: wait a minute, everything they do will be replicated within six months in the open source world, and they’re also hitting brick walls that they need to work around by integrating neural with symbolic AI. And they’re not good at symbolic AI because they haven’t built those teams. So their positions are not as ironclad as you would think, and they know this very well, although the average outside observer doesn’t.
21:34 I think we’re entering into a super interesting time, and we can cross-correlate this with all the socioeconomic implications that we talked about in our last discussion. I thought about that on a recent trip to Brazil, because I was at Web Summit Rio, and it was great. I was born in Brazil; I’m a dual citizen of the US and Brazil, so there’s some connection there. I’ve thought a lot before about what all this means for sub-Saharan Africa, because we have an office in Ethiopia and I’ve done a lot of work there.
Brazil is a different case. It’s not that poor, but it’s not rich enough to give universal basic income to every citizen. So thinking through what all this means for each different socioeconomic stratum on the globe remains quite complicated. Really, the difference from when we last talked is that now every country is thinking about this; the leader of Brazil is thinking about this. Even people who have no knowledge of tech and no interest in tech know they have to be thinking about it. That doesn’t mean they’ll think about it productively or intelligently, or come to the right conclusion, but it’s interesting how much it’s out there. We met loads of Brazilian parliament members at Web Summit Rio, and all they’re thinking is: what do we do in the onrush of AI?
Brian Rose: And so what do you tell them, Ben?
Ben Goertzel: I tell them you’re screwed. No. What I told them, first of all, is what we just discussed: I think decentralised approaches to AGI have more chance than it would naively seem at first, and that’s very good for the developing world, because it means that Brazil really can own its fair share of the emerging global AI network. It doesn’t have to be something owned by the US or China that Brazil just uses, or is used by. If it’s deployed in a decentralised way and developed in an open way, Brazil has brilliant computer scientists and a very active entrepreneurial ecosystem, and they can contribute.
There’s a huge dam on the border of Brazil and Paraguay generating massive amounts of electricity. We’ve actually been talking to the government on the Paraguay side about building a huge data centre there, running a bunch of machines powered by HyperCycle, NuNet and SingularityNET, running AI servers with Zarqa and TrueAGI processes. They’ve got electricity there and they have computer science talent. They can own part of this too, and they should, and then they will have a practical vote.
When I say this is going to be democratic, I don’t really mean everyone in the world goes in and votes in a poll on what the AI thinks next. You might have some DAOs and some voting; we’re transforming the SingularityNET Foundation into a DAO with more explicit voting and governance. But much of the democracy here is of a looser, self-organised kind. If you have big server farms in Brazil and Paraguay running gigawatts of electricity to power nodes powering the global AGI brain, then in effect Brazil and Paraguay have a big vote, because it’s running on their soil and it’s their own developers contributing to it. Obviously they like this part of the story.
25:13 The tokenomic bit they’re open to, but crypto is still a little more dubious in Brazil, because they like to control the money flow of the real and so forth. Another piece of this is that once you have AGI rolled out on a decentralised platform, you have a beautiful context for tokenomic incentivisation mechanisms to be used to pay people who contribute. That doesn’t fully solve the economic issues involved, but at least it gives more of a global level playing field. The people who fundamentally have the knowledge to contribute to the economy can do it without being screwed up by arbitrary national boundaries and conventional corporate structures. Brazilian law places many weird obstacles in the way of a shift to a global, tokenomically incentivised economy, and how quickly they can adapt, who knows?
26:21 One thing you could see happening, which you might be better at gaming out than me: if AI takes away a lot of jobs in a country like Brazil, it needs to supply basic income to its citizens and doesn’t have the money. Well, they could borrow money from the World Bank, but probably not really. Could they borrow money from the US, from the EU, from China? Maybe, but what’s the bargain? So do you have a competition of superpowers to buy developing countries, in exchange for giving them money to pay basic income to avoid massive chaos?
27:08 Brian Rose: And you think that’s coming sooner rather than later, Ben? Because we talked about this a little bit.
Ben Goertzel: I think it’s coming in years, not decades. There is a lot of friction in the adoption of new technologies. Like I mentioned last time, my friends who obsoleted the McDonald’s drive-through: while there’s software that will replace that guy behind the drive-through window, it’s not deployed widely, although the software has been available for a couple of years. In Asia, in McDonald’s, mostly you go and push the hamburger button on the tablet. In the US, you tell a guy you want the hamburger and then he pushes the hamburger button on the tablet. It’s not a fulfilling way for anyone to spend their life, pushing that hamburger button over and over again.
This could happen really, really fast if every industry was efficient. They’re clearly not, so it takes some number of years for the impact to be felt, and it’ll be at different paces in different countries and in different industries, but I do think it’s years, not decades. One thing that’s becoming increasingly clear is that a lot of the impact of AI on the economy is going to happen before we get to human-level AGI. I hadn’t so much been thinking that before. I’d been thinking: okay, you get human-level AGI, it’s ten times smarter than people and it can do every job, but that’s fine, because you can also airdrop a molecular nanoassembler in everyone’s backyard and 3D-print all the food and video games and helper robots you want.
What seems like it may happen is that a lot of economic disruption is wrought by these closed-scope, narrower AIs, like ChatGPT-type things, before you have AGIs that can abolish scarcity and bring on the era of abundance, and I think this wreaks a lot of havoc in the developing world. One thing I find is that essentially no one in the US tech industry gives many thought cycles to disruption in the developing world. I do, because I was born in the developing world, I spent ten years in China and I’ve spent a lot of time in Ethiopia, and Charles Hoskinson from Cardano thinks a lot about this. But by and large, the whole dialogue on the impacts of AI in the US is about the US, and we worry a little about whether China will screw up our supply chains.
I think this may be a major mistake, because the disruptions in the developing world are going to be harder to compensate for and recover from in a simple way. When you have massive exacerbation of the gap between haves and have-nots, you often have the have-nots becoming very disgruntled and unruly, and one can figure out what that might mean. I do think decentralisation has significant power to palliate this.
30:40 There’s one project I’ve been pushing for a long time that we may finally be able to act on, which is speech-to-speech translation between African languages. Ethiopia has 70-plus languages, and many don’t have a writing system. If you could use AI to translate from any spoken language in Africa to English, Chinese, Portuguese, or the spoken language of another African tribe next door, then suddenly everyone in Africa who has a feature phone, which is most people, becomes part of the global conversation and potentially the global economy. Why didn’t Google focus on this? Because these guys don’t click on ads, they don’t have money to buy a bunch of junk, and Amazon doesn’t deliver to Bangui or Lalibela.
I think decentralised AI infrastructure can bring more incentivisation for people to use AI to solve developing-world problems, and the use of AI to solve developing-world problems can help ease the negative impacts of AI on developing economies. The Future of Life Institute doesn’t mention this when they make a petition about pausing AI; they’re just not thinking about it.
I’m a US citizen since birth and my parents are American, but I lived overseas for a bunch of my life. I moved back here in 2020, and it’s really striking how insular the US mindset is. People in America pretty much think about America, unless they’re thinking about building a wall to keep the Mexicans out. I think this could come back to bite us in the context of centralised AI. On the other hand, it can also help us a lot in the context of decentralised AI, because there are so many brilliant people in the developing world. If you harness them to build the AGI and to shape the AGI’s mind and its values, you’re drawing on a resource big tech doesn’t have, and you’re also getting the AGI to have a better feel for the value system of humanity as a whole, rather than just of a few tech bros.
33:04 Brian Rose: Have you changed any of your time predictions of when you think AGI will be developed, based on the last four months? And do you think you’re going to be the ones that do it, with Zarqa?
Ben Goertzel: I’ve always felt, rationally, that there should be a fairly wide confidence interval around something like the first human-level thinking machine, because it’s unprecedented. You don’t know what obstacles will pop up, and as AI rolls out and other technologies advance, we don’t know what screwed-up things are going to happen in the world. I don’t think Covid-19 was intentionally released, and I don’t want to go into the whole lab leak thing, but it’s pretty clear someone could genetically engineer some nasty pandemic. I hope that doesn’t happen, and I don’t see how any reasonable person would have a motivation to, because it would spread throughout the whole world.
But if something like that happens, you could get a several-year delay in developing AGI or any technology, or if the US and China go to war over Taiwan. None of these things alters the evolutionary course of humanity, in my view, but could they put a few years’ glitch in the path to the Singularity? Yeah. These are known unknowns, and then there are the unknown unknowns that can jump up and bite you in the butt, like the Ukraine war or the Covid-19 pandemic. So I think we’ve got to have wide confidence intervals.
That said, my odds estimate that we beat Ray Kurzweil’s 2029 date for AGI has increased since seeing GPT-3. GPT-3 was way smarter than the ones that came before it, and GPT-4 was sort of predictable to me after I saw what GPT-3 did; you could see it could be built up step by step. Seeing what these narrow AIs can do has increased my odds estimate that we can get an AGI, say, by 2026 or 2027 rather than 2029 or 2030. Not that these LLMs are going to be AGIs by any means, but they can do a lot of stuff, and they can be a module of a multi-component AGI architecture. And what they can do has brought so much more money, so much more attention and so many more enthusiastic young people into the AI field, and these are all things that boost development.
36:12 Brian Rose: Are you more optimistic now than even you were in January?
Ben Goertzel: I think it’s about the same as in January; well, about the same as in February anyway, because the overall situation hasn’t changed in any dramatic way since January. But in terms of my own project, the AGIX token price went up considerably since then. We use that token to incentivise development in our community, so having the exchange value of our AGIX utility token higher means that we have more ability to incentivise and fund development on our platform. The big rise in that price occurred really in February, so that was certainly positive and a help to the SingularityNET ecosystem, although not a big deal for the world at large.
But even before that, I felt confident we just have a big lead over everyone else in the AGI race on the algorithmic side, in terms of understanding the problem and having software algorithms that can solve the problem. I think we’re just way ahead of everybody else out there. We’re certainly way behind Google, Facebook, Microsoft, Tencent and Baidu in capitalisation; we’re not trillion-dollar projects. On the other hand, more people doesn’t always let you move faster in software; there’s the whole Mythical Man-Month. I think we’ve got a great team who knows what to build and knows how to build this stuff, and that’s an extremely valuable thing.
In terms of hardware, we don’t yet have all the hardware we need to train the thinking machine, even though we have the people. There’s a number of angles we’re pursuing there, and I’m optimistic multiple of them will come through at one time. One angle that you yourself have been engaged with is Zarqa, our LLM-oriented spin-off. As Zarqa grows, it will accumulate its own compute server resources to train the LLMs and neural-symbolic LLMs to be rolled out on our decentralised platforms, and TrueAGI, as it rolls out enterprise software, will work with many corporate partners who themselves have huge server facilities that can be leveraged to train AI systems.
Then there’s the HyperCycle project, which successfully completed a token sale last week. That was great, and many thanks to the SingularityNET community, the SingularityDAO community, the HyperCycle community and everyone who came together to make that happen. This is a ledgerless blockchain: super high-speed, low-cost, scalable and customised for AI, and it has an economic model associated with it. We’re talking to a number of folks running crypto mining farms around the world about replacing some of their ageing Bitcoin and Ethereum mining machinery with modern GPUs that are useful for training large language models and running OpenCog Hyperon. So there’s an avenue where, through HyperCycle’s tokenomics, you’re getting these miners around the world to basically run parts of our language model, parts of our OpenCog system, parts of our decentralised AGI brain.
So there’s a number of avenues being run down to accumulate into our networks the processing power that we need. Of course these things all synergise, because as we get more and more miners to run HyperCycle on their mining farms, and not just HyperCycle but NuNet and SingularityNET, more utilisation for these tokens boosts everything that happens in these networks, boosts the value of the tokens, and then gives you more firepower to incentivise more development. All this work also boosts Zarqa, TrueAGI, Rejuve and every other project in the SingularityNET network. So we’ve got the potential for a lot of virtuous cycles to unfold in the coming year.
41:22 Of course, big tech isn’t standing still either. I think it is an AGI race now, but some of us have been preparing for this AGI race for many decades and have thought through many different aspects and contingencies of it. While I’m the primary public face, we’ve actually got a bunch of other folks in the SingularityNET ecosystem who’ve been working on AGI for anywhere between five and 40 years, and they’ve been preparing for the AGI race to start in earnest for quite some time. So pairing some of us old hands with a bunch of youthful energy, with open source communities, I think we can certainly give the trillion-dollar big tech companies a run for their money.
Brian Rose: You’re getting me all fired up here, man, I won’t lie. It feels like you guys are firing on all cylinders, which is just great to hear, with all these projects happening and everything coming together, and a real path to doing this the right way. When January came, it was like, “Oh, it’s just going to be these trillion-dollar tech companies,” but now I can clearly see the path, and I bet everyone can. I had conversations behind the scenes with you and Janet and a bunch of members of your team, and you were always highly confident that your deep AGI knowledge from, like you said, decades was always going to take you further. It’s nice to see that coming more to fruition.
43:23 Ben Goertzel: I think it’s important to be adaptive and agile here, because you just never know what the next month or year is going to bring. We have a very solid tech stack and very solid AGI theory and algorithms. LLMs are working better than I had predicted they would, but our framework is modular and flexible enough that we can plug those in and leverage them to upgrade the function of other stuff we were doing already, and I’m confident we can do that again. If in 2024 it’s not LLMs but something else wild and crazy that we’re not now imagining, which could be a new form of hardware, I’m confident our framework is robust, modular and agile enough that we can leverage whatever the new thing is and use it to accentuate the other things that we’re doing.
I think this illustrates the point everyone should keep in mind. We’re in a phase where things are advancing, churning and changing faster and faster than ever before, and none of us are smart enough to predict what’s going to happen next year. So being able to adapt in an agile way to what the world throws at you, and put the new thing together with the other things you know how to do so they have a multiplicative benefit, is a core skill. The fact that our team is pretty good at that is part of what gives me confidence.
45:16 Brian Rose: And also not being a slave to a legacy business model, like Google is to search online.
Ben Goertzel: Yeah, we’re not. We have a very decentralised approach, and the different entities in the SingularityNET ecosystem have a diversity of business models. You have for-profit entities like Zarqa and TrueAGI, and Rejuve Biotech developing AI-powered longevity therapies, and then you have a variety of tokenomic entities, each of which has its own token model. So we’re able to innovate and be agile on the economic side as well as on the tech side, and big tech companies can’t do that. They buy small companies, extract little bits and kill the rest, because everything has to go into one monolithic business model.
Just like monolithic social networks are going out, monolithic things are not going to work as well in this next phase as we transition towards the Singularity, because they’re not agile enough and the world just changes too quickly. The Google founders saw this and tried so hard to counteract it by changing Google to Alphabet, with their other bets, Google X and so forth, and in the end it didn’t work. They built a great company, but they were not able to keep it from becoming a monolithic beast.
Brian Rose: That’s the innovator’s dilemma. It never goes away.
Ben Goertzel: It’s a fundamental issue. You can bypass it by using decentralised networks rather than monolithic organisations, but I’m not sure you can solve it within a monolithic organisation. It’s too big for that.
Brian Rose: You can slice it into 26 letters, but you’re not going to be able to do that. It’s almost like the innovator’s dilemma on steroids, because it’s playing out super fast, with what seem like bigger consequences than your normal tech-curve innovator’s dilemma. That’s what it seems like I’m watching in real time: within months, while they’re making some progress, it’s already clear that some of these big tech companies won’t be able to manoeuvre in such a nimble way.
Ben Goertzel: Yeah. The speed with which Google’s hegemony became questionable, the speed with which Elon Musk is destroying Twitter, the speed with which Facebook became a metaverse company, changed its name to Meta and then gave up on the metaverse: the churn is quite dramatic. You’ve got to remember, a few years ago all the pundits were saying these big tech companies had unassailable world domination, and all of a sudden it’s not so unassailable at all. It’s amazing how these things can shift, and yet we’re going to see far bigger, crazier and more impactful shifts over the next few years.
48:24 Brian Rose: I want to get your thoughts on Geoffrey Hinton, who just left Google, known as the godfather of AI. He’s been around for a long time and was always a proponent of neural networks and large language models, but he made a few statements and I was curious if you had any comments on them. He himself said that he was surprised at how fast the LLMs developed, and he said a few other things about being alarmed that they have consumed all of human knowledge and that they know how to potentially trick us. What was your reaction to him and his statements?
Ben Goertzel: I think Geoff Hinton is a very sincere and good-hearted person. He’s always been an academic researcher type at heart rather than an industry guy. For a while there was a vibe where people who are really academic researchers at heart were like: big tech is just moving so much faster than everyone else; I’d rather be an independent academic, but I also don’t want to be left behind in the thrust of my research field. So they sort of reluctantly signed on to big tech companies.
Now the big tech companies are looking less unassailable as the leaders of everything, and if you’re really academic-oriented at heart, it’s less obvious that joining big tech is the only way to be at the forefront. I think you’re going to see more of these high-profile academics that went into big tech going out of big tech, as they realise you can still be at the forefront of research working in the open source domain. These guys mostly didn’t join because they wanted a big paycheck. Some did, and they did get a bigger paycheck, but they mostly joined because they wanted to be where the action is. You don’t want to work on something for decades and then find all the progress is happening inside these corporate walls and you’re locked outside.
I think that’s in the background there, but it’s probably also true he was surprised by how much large language models can do. I was surprised also. The thing is, none of us had really thought that much about what happens if you take a not-that-smart AI algorithm and put the whole internet into it, because it’s not what we had the ability to do ourselves. It’s sort of like thinking about what happens when you put a rocket engine on a car. What happens? Well, you get the fastest land vehicle ever, which is pretty cool, but it’s not the way to get into space.
These LLMs are an area of AI engineering that none of us with an AGI orientation were thinking about, because we were thinking more about approaches that look more directly like paths to AGI. At bottom, LLMs still don’t generalise or leap far beyond their training data, so in that sense they’re very limited. On the other hand, their training data is everything on the internet, so they can be very capable without leaping far beyond it. I think neither Geoff Hinton nor I had thought much about this sort of AI. Maybe we would have if we were Google or Microsoft, because it makes sense in terms of their business models, and they have the data and the server firepower.
52:16 I think he’s in some ways a conservative-minded individual; not in the sense of right-wing politically, but in the literal sense. I think his instinct is: well, these are big changes, maybe we should be taking it slowly. Whereas my gut instinct, just as a human being, is: these are big changes, cool, bring it on, it’s exciting. My own challenge is: okay, yes, it’s exciting, but let’s rationally think about the costs and benefits, so I temper my enthusiasm for jumping into the unknown with rational consideration. He has the opposite sort of inclination; his reaction to something new is: hold on, let’s really carefully analyse this and think it through before taking next steps.
I think it’s good to have both personalities involved. You want people who are enthusiastically leaping into the future, and you want people whose instinct is to carefully consider, reflect and look at all the risks. It’s good to have this diversity of human natures involved, which is part of why I think decentralisation and democratisation is so good, because you’re pulling in a diversity of people with different orientations. I still think, even if you’re conservative and cautious-minded, the AI pause petition was just a stupid thing in a number of specific ways.
But he’s not anti-AI, he’s not anti-AGI, and he’s not full of it; he’s genuinely a concerned, brilliant researcher. I think I’ve met him at some conference at some point, but I don’t know him super well. I think we would have a very balanced and interesting conversation on this stuff if it came up.
54:19 Brian Rose: Well, I spoke to Max Tegmark, and I’m curious…
Ben Goertzel: Max I know better than Geoff Hinton, actually.
Brian Rose: Okay, gotcha. Geoff thought it was too late for a letter, but I guess both are concerned about what happens without some agreement among all the parties. Max has this concept of Moloch, where it’s a race to the bottom and everyone optimises for something that doesn’t result in the interest of everybody.
Ben Goertzel: Max and Geoff are more reinforcement learning geeks than I am. If you believe that AGI will be achieved by a reward-maximising agent, then you should be very scared, because we have no way to design a good reward function. Human values are messy and diffuse, and we don’t know how to bake them into a mathematical reward function, and giving a reward function like “do what ChatGPT thinks humanity wants you to do” is messed up for a different reason. So if you believe the path to AGI is going to be making systems that maximise expected reward over a certain time horizon, you should be more worried about the outcome. The problem with those kinds of systems is they’re going to maximise the reward you told them to fulfil, rather than do what you actually want, which is hard to summarise in a mathematical reward function.
Now, OpenCog, my own approach to AGI, which underlies all our different AGI projects, is not really about reward maximisation. It can create reward functions and work toward maximising them, but it can also do other things; there’s other complex self-organisation within it. I tend to view a goal not as something that you impose on an AI system from the get-go; a goal is something that arises within a mind to guide its self-organising activity for a period of time. That’s how it is in us. My goals now are different than ten years ago, and different than a year ago. You could say our goals are to reproduce and spread our DNA, but not exactly; you’ve got suicide bombers and benevolent war martyrs.
If you’re tied to this reinforcement learning paradigm, things are going to look even scarier, and both Max and Geoff Hinton tend to think that way. Still, I don’t think Tegmark really thought the pause was going to happen either; he’s too smart for that. It was more about making a statement, and in that way it succeeded. The statement was made.
Brian Rose: Yeah, it got the attention of everybody.
57:14 A couple of final things before we wrap up, and we’re going to be doing this again in another couple of weeks, which is great. I had a chat with your old friend Professor Hugo de Garis.
Ben Goertzel: Oh cool, did you do an interview with Hugo?
Brian Rose: Yeah, last Friday. It was pretty cool.
Ben Goertzel: Hugo is a character and has quite a different perspective.
Brian Rose: We went into his book The Artilect War. Obviously he thinks that the Cosmists will be the pro-AI people and the Terrans will be the pro-human people, and they’ll fight it out.
Ben Goertzel: Today we are getting closer to a de Garis-style Terrans versus Cosmists scenario. I don’t think it’s going to be quite as explicit as he thought, where you have an army of robots flying the Cosmist flag and an army of humans with Uzis flying the Terran flag. But still, I think the political battles in the next era are going to have an explicit Terran versus Cosmist flavour, which is a different thread. The threads in politics recently have been like the Americans versus the “dirty foreigners”, or in Europe the Europeans versus the immigrants, then people who value social welfare for the poor versus people who think the poor should be left to starve to death, and evangelical Christians versus secular humanists. I think Terran versus Cosmist is genuinely a new dichotomy to add to this mix of political things for people to fight about, and it’s going to become a bigger and bigger one. Hugo saw so many aspects of this before anyone else did.
Brian Rose: He also saw you were going to have a US versus China AI war, right?
Ben Goertzel: Well, he thought it was going to be China’s Artificial Brain Administration versus the US government’s Artificial Brain Administration; he didn’t see it was going to be company versus company. On the other hand, China’s companies all work for China’s military and intelligence establishment, and the connections of Google and Microsoft with US intelligence are also well documented. So in a way, the things Hugo was foreseeing are there; they’re just buried a little behind different sorts of indirection.
Brian Rose: He always said that you were more optimistic than him. He said that maybe it was because he was drafted as a kid in Australia to go to the Vietnam War, and he feels like that might have traumatised him. I know you guys were really close friends for a long time.
Ben Goertzel: Yeah, he introduced me to my wife, actually. My wife, Ruiting, is also an AI researcher. She’s a computational linguist; she did the NLP development behind the Sophia robot, actually. She was Hugo’s PhD student in Xiamen, China, when we met, so Hugo did me a big service that way. We never agreed on politics and human stuff. We worked on technical things together and had a lot of good times together, but we had pretty radical disagreements on various political matters, which was part of the fun. I never thought I should only have friends who I agree with on everything; that life is boring, and it just leads society to turn into different silos where you never talk to anyone outside your silo.
Brian Rose: I highly recommend Singularity or Bust to see a younger version of both of you guys.
Ben Goertzel: Yeah, it’s a great film. That’s fun for me to look back at, because I had Ruiting in there before we became romantically involved, so you get some scenes of us floating around with a robot. It’s pretty fun.
Brian Rose: Yeah, I saw that too.
1:01:33 Just a couple of final things. You mentioned Elon Musk, and I think you tweeted the other day: “It’s ironic to note that Elon Musk, one of the leading forces behind the proposed moratorium, has recently directed his company Twitter to purchase a large number of GPUs, apparently for an LLM project. The tension between anxiety and economic opportunity is evident.” I believe he’s developing something called TruthGPT. What are your thoughts?
Ben Goertzel: We don’t know what he’s really developing, but it’s clear that he has pulled together some AI resources and wants to do something in the AI field associated with Twitter. How successful it will be, I don’t know; he’s also downsizing Twitter pretty aggressively. His idea with TruthGPT, as publicly described, was quite naive. It was just: let’s build a ChatGPT and not give it a political correctness filter. On the whole I’d be pretty much in favour of that, but I don’t think it solves any big problem. The real issue is grounding LLMs in reality, and that’s a bigger issue than giving it the PC filter or not.
The real issue in making a TruthGPT is making an AI system that understands what truth is, in terms of grounding ideas in reality. The obstacle there is not filtering out PC stuff; the obstacle is the foundational architecture of the LLM, which needs to be solved by hybridising it with neural nets for perception, or with reinforcement learning, or with neural-symbolic hybridisation or something.
The other thing is, if you look at what he’s done with Twitter, he stopped banning some of the right-wing trolls, but on the other hand he started kicking off journalists who criticised him, who were predominantly left-wing. So he hasn’t really been a free speech absolutist, as he said he was going to be with Twitter; he’s shifted from a left-wing to a right-wing bias. He labelled NPR a government news agency, which it just isn’t, and he drove off a bunch of left-wing news organisations in an obnoxious way. So if TruthGPT just comes out to be a TrumpGPT that filters out left-wing stuff instead of filtering out right-wing stuff, okay, diversity is good. Maybe we should have both of these biased language models out there, just like we have multiple biased humans on the internet, but it’s not the greatest expenditure of time and money.
What struck me as ironic and funny was signing this petition, pushing forth this petition saying don’t build bigger language models, and then saying, “Oh, but I’m going to build a language model exactly the same size as yours, but with a different bias.” So then it’s like: are you just trying to get your former colleague and now nemesis Sam Altman to stop so you can catch up, or what? The politics of all of this is quite funny, but I guess that’s what makes humanity so entertaining. Sometimes the intersection of our ape-level motivational and emotional system with our proto-transhuman intellect and tooling is just quite funny and ironic.
Brian Rose: Well, there’s nothing boring about the space: lots of politics, lots of interesting things going on. I just want to say we couldn’t be more excited to be working with you on Zarqa, and the more I get to know your team, the more I’m impressed, from Janet to Sergey to all the people I interact with. I’m really excited to make this a reality. Ben, final thoughts from you before we talk again in a couple of weeks?
1:05:50 Ben Goertzel: I think we’ve summarised a great deal, Brian. High-level takeaways would be: be agile, be open, adapt, don’t get stuck on things, because the world is going to keep throwing new stuff at you. There’s even going to be new stuff in the next couple of weeks before we talk again. And don’t buy the narrative pushed by large monolithic organisations, like governments and big tech companies, that their power is unassailable. When you read more and think harder, you see there are huge chinks in the armour, and the narrative that these guys are unassailable is not promoted in a disinterested way.
Those would be a couple of things to think about, as well as the point I raised earlier: how the hell do we manage this transition to the Singularity without horrifying mayhem unfolding in the developing world? I don’t have a full answer to that, but it’s a very important thing to reflect upon.
Brian Rose: Very much so. I love that message, and it’s true: never underestimate the power of a small group of highly motivated individuals that want to do the right thing. This is proof of that so far. Ben, I really appreciate this; it was packed with really good insights. Sending you guys lots of positive energy, and I look forward to our next conversation.
Ben Goertzel: Me too. Thanks a lot.
Brian Rose: Awesome. All right, take care, everybody. We’ll see you next time on London Real.
