Digital Humanity: From Real Minds to Synthetic Hallucination? | AI, LLMs, digital identity, consciousness & authenticity

Gareth King (00:38)

David, thank you so much for joining us and welcome to the show.

David Schmudde (00:42)

Yeah, thanks for having me.

Gareth King (00:46)

Before we get into it today, can you tell us a little bit about yourself and the journey that's led you to this point?

David Schmudde (00:54)

So currently I'm the CTO of Yorba. We work in the digital identity space. A lot of it's about the digital footprint that you have online, but there's a bigger issue around digital identity that we deal with sort of under the hood. And also, it's important to think about the bigger picture here because we're just kind of fighting things now as they are.

But better things are possible and our hands are probably going to be forced with what's happening in the AI wave, with the agentic AI posing as humans.

Gareth King (01:27)

We know a lot of people talk about the dead internet theory. What the statistics are telling us is that over fifty percent of online content is now considered synthetically generated. Would you say that theory is reality now, or do we have a bit further to go until things would be what you would consider a dead internet?

David Schmudde (01:46)

There's a bit further to go. What I think about the internet that is so obvious it somehow remains hidden is that it's really, really, really big. It's unimaginably large. So, there is still plenty out there that's human generated. There's plenty out there that there's craft to it, their websites or what they're communicating.

What I mean by plenty, I mean there's so much out there that it would take a lifetime to consume. I'm talking about outside of what is being machine generated and also outside of the centrally controlled platforms. So today, there's still more for you to read and look at on the internet than you have time to consume.

The question is really about findability. That's the core question. And there could be a case where all of this slop, all of this machine generated stuff, plus the centralising forces of these social media platforms, there's almost nothing else that can be found very easily. And today I do concede, of course, that it's probably most people's experience that they see a lot more AI generated stuff than they used to. Maybe fifty percent of their timelines is AI generated.

Gareth King (03:04)

For a lot of people, a huge amount of their time in the digital world is spent, say, on a phone. And a lot of that time is spent on that social media that you'd mentioned. And I think I'd love for us to talk about the role that social media itself has played in getting us to this point through the monetisation of attention.

I read something from the head of Instagram who was basically explaining to people that the way to be really relevant in 2026 was to be more human. The problem is obviously cutting through the rest of the shit out there. And I think what was so interesting to me about this post from this guy was that Instagram is such a huge culprit of being full of the slop, as people say. And it rewards that through its algorithm.

Why do you think he's now coming out with a statement like that when the platform that he's been overseeing has been driving so much of this awful stuff?

David Schmudde (04:05)

Well, there's a few ways to take that question, of course with Instagram, it's not a recent phenomenon, but it has been the platform's mission since the beginning to add filters, for example, to your photos. Are the filters making things more real?

And then after like this naive sort of photo era, when it became a real social network, are the images projected from the users of Instagram real? Like you know, the angles they're taking, the sort of like posturing they're doing. Are those things real? So, these are like well-known aspects of the platform in its lifetime. And then of course, yes, today there's a lot more AI generated content. I mean, this is part of the whole culture of the company.

The best analogy that I can think of is the role of the A&R person in the music industry. And the music industry has always commercialised authenticity, but I don't mean like in the sense that Jimi Hendrix was real or like Kurt Cobain was real. I mean in the sense that something like country music, for example, what they were sort of manufacturing at the end was a facsimile of what was actually happening in the Appalachians or what was actually happening in the South.

And it was through the lens of these mostly men that lived in New York, right? And the point is it's like they invented country music as a real thing, but it was an invention of New York tastemakers, right?

So, in this case, whether it be A&R people in New York City or Instagram execs in Silicon Valley, the high level marketing speak here is the same where they want to proffer something that is real and authentic to the people who are listening or consuming or watching, even if it's not, because what is real in life, isn't all that pretty, right? And people don't want to engage it all the time.

Gareth King (06:15)

You know, and I just thought it was quite a strong and striking pivot for somebody in that position who's overseen what's been going on there. But let's go back to I guess the amount of content online that's not generated by a human.

Now you said something that of course it makes it harder to find that human generated stuff. Now, once upon a time obviously you could have been sure that what you were engaging with was most likely from a human. It was human thoughts, it was human creativity, it was human mindset being put into a digital form. But now, we're not so sure.

Is it maybe gonna be the case forever that what we're reading actually coming from a human or not doesn't really even matter beyond our own individual, I guess, feelings or needs in the moment. You know, a lot of people are turning to these LLMs, whether they're as therapists, as friends, as relationships or whatever, rather than just shutting it off because it's not human.

How many people would do you reckon are like that? Is this gonna be a minor percentage or we're all likely to be susceptible to that?

David Schmudde (07:23)

I guess from a Turing Test perspective, doesn't really matter, right? I mean, if it passes the Turing Test, it means you can't differentiate between it and a human. And so that threshold is not entirely gone, but it's getting quite close.

And these people with, for example, AI companions, AI girlfriends or AI boyfriends or friends, for them, this person has an, this thing has an identity, right? So as long as it's consistent, I mean, what does it matter that it's synthetic versus authentic? So, people will continue to get closer to these entities, and this is just sort of the reality going forward.

The bigger sort of question to me is the political context. And so, this idea that like a technology is ethically neutral, right? And it just depends on how you use it, which is kind of the pervasive idea that we have in our society, is incorrect. It is demonstrably incorrect in a technology, say like nuclear power, where if you don't want to have a disastrous outcome, once you build a nuclear power plant, you have to have a certain political structure. You can't have anarchy anymore. That's off the table. You must have a certain political structure. Otherwise, that thing is going to have a catastrophic outcome, right? So, the politics and the technology are tied.

And Joseph Weisenbaum talked a lot about the politics of AI. Now he was writing about this after he created the Eliza Chatbot in the 1970s, in the 1980s he was writing about this. And the political position of an artificial intelligence bot, right, in his estimation, is necessarily conservative. It must be conservative because it can only look backwards. We have no evidence, no demonstrable evidence that within the identity of this LLM or in his days symbolic AI, regardless, we have no demonstrable evidence that it has a sense of, for example, intuition. It has a sense of forward-lookingness, where of course it has imagination in a way that we can see, in the self-evident ways that we can see it on screen when it creates something, it has like some sense of imagination.

But is that imagination based only on looking backwards, which some of our imagination is, or is it some combination of looking backwards and looking forwards? And can AI actually do that?

So, the question about this political, you know, technology is around one, that it demonstrably looks backwards. That's how they're trained. We don't have any evidence that it does look forward in an intuitive way that we can identify.

And so, the third component of this is then the orthogonal fact that these are all well-funded, a certain number, small number of well-funded individuals with a certain cultural disposition, right, that is baking it into the AI.

And that's why I think that to kind of wrap this up, I think that's why the versions of ChatGPT so far, for example, are so such so sycophantic, right? Because like one, that that's what they think that people want. And two, it talks like an optimistic Silicon Valley dude that thinks, oh that's a great idea. And it just like constantly feeds into their experience of their life. It just ‘cause their whole life is people validating their terrible takes. That's their entire life. And so, the thing that they create acts exactly like that. Surprise, you know.

So, like these things are interrelated. And there is some kind of question as to like how sort of diverse and innovative, like actually innovative that these beings can get. And are people do people have an appetite for that, right? Do they want this because there will be some people that only want the sycophants, but there are a lot of people that want to be surprised and want unexpected things in their lives.

Gareth King (11:40)

And it kinda got me thinking as you were speaking there about the way that you said it has to be conservative in nature, because it can only draw from the past. And you know, I think that's one of the arguments that a lot of people have around it, which is it's just an aggregator, it can just synthesise information, et cetera, et cetera, et cetera.

Now, do you personally think that LLMs will be able to gain the ability to imagine like humans can and to look into the future and project into the future like that? Or do you think they're gonna hit a point where, you know, the smoke and mirrors start to fall away and like a lot of people are currently saying, they'll be exposed for their limitations?

David Schmudde (12:24)

The limits of LLMs, I think, are sort of indeterminate at the moment. I mean, a lot of people have a lot of opinions. I will be coy about mine a little bit because I don't have a lot of confidence in where this is going, but I do know that there is a lot of research, for example, being put into combinations of systems where expert systems and LLMs are layered together to get better outcomes.

And so, we're taking this like older symbolic AI approach and grafting it on because some people do think that there are real limits to what an LLM can ingest. And if it starts ingesting its own content, like what happens then if like all it is doing is producing its own output and reingesting that? Like there's a lot of unknown about that.

And then there's this this reality of course is like the digital space is limited. It's both infinite and limited in a way that meat space is not, right? And so in our space there could be another sort of leap, for example, with LLMs when they can actually move in our space, right? So, we don't really know how that can play out either.

Gareth King (13:41)

But alright, let's bring that concept right back down to earth for a little bit around LLMs and what they're generating and then human generation and as you said, LLMs regenerating based on their output.

Now, something that I've been quite interested to read, and it is a little bit concerning to myself personally, is the apparent loss of cognitive abilities as people have started becoming more and more reliant on LLMs.

You know, a lot of people would use something like Chat GPT, right, to write something for themselves. And one of the big debates around LLM created writing, for this example, is that it's got these very obvious patterns that it follows.

But I guess my question on this is what's influencing what more? So have these LLMs obviously been trained on a lot of human created output and reach this kind of aggregate style mode, you know, all these apparent signs that people say it has, which then people adopt and pass off as their own. The LLM sees that and it's like, okay, this is more human. That means I double down on that.

But then also secondarily, humans not actually doing that writing themselves now start to train themselves on the LLM's output. And so, I guess my where I'm going with this is do you think that this line between human identity and machine identity in this context, but then also in a wider digital space, is blurring simply because of our I guess our seeking for efficiency and ease? Like what are we doing to ourselves?

David Schmudde (15:24)

I don't know. I don't know, of course. Of course, I can't project the outcomes of what you're describing. But I guess I'll try to make this less of a cop out and more of an answer by citing postmodern art. I mean, you have an artist like Sol Lewitt, and I don't know if Sol Lewitt could draw. I have no idea. But there's a lot of artists post 1940s, you know, that weren't they don't know how to draw, right? And what does that mean that you're a visual artist and you don't know how to draw and you don't know how to paint? You can't render something?

Well, I don't know what that the impact of that is, but honestly, leave the drawing to the drawers, leave the illustrating to the illustrators, right? Because I think what Donald Judd or Sol Lewitt or some of these, Dan Flavin, some of these other like postmodern artists gave us really, really interesting art.

And the thing, the reason I cited Sol Lewitt in particular is like he was working at such a high level that he didn't even make his own art. This is conceptual art, right? I think of the idea. I don't even make the thing. The art is mine, but I didn't even put pen to paper or whatever medium it is.

And so, it was something lost when artists don't know how to draw. I guess, but there's a lot of artists out there. So yeah, but is something gained by having artists like Donald Judd and Sol Lewitt in the world? I think absolutely. I think absolutely it's gained. I think I think postmodern art, although often misunderstood, you know, not well understood, is really a joy to have in my life along the side of other types of art, other periods, Renaissance, you know, other periods of art.

Gareth King (17:19)

Yeah, I mean it is an interesting one there, you know, artists that can't draw. And, you know, my first reaction to hearing something like that is it's like, well, yes, because there's so many different mediums of art that you can create. But then what you were saying there around the postmodern artists who might have the concept and then the creation of the thing is outsourced to other people.

David Schmudde (17:43)

This is very much a parallel to LLMs.

Gareth King (17:47)

That's exactly where I was going. And I think that a lot of it that's exactly what I was gonna say. It parallels a lot of the debate that I've seen personally online around let's call traditional artists versus AI artists. And there's this really huge, massive battle right now.

And you know, the battle lines are drawn along, you didn't create anything, you just typed some words as a prompt. And the machine created something. And look, I can understand the logic behind that argument because that machine is drawing on everything else that is potentially inspiration for a physical artist that would still need to be inspired by that and then physically create something, which of course the machine is not doing.

You know, I've seen I've seen some people that make incredible AI generated visuals. And look, I would have not the first clue of what their prompts would be to deliver stuff like that. And to me, I look at that and I'm like, wow that's a piece of art.

But then a lot of other people are like, no, you're not an artist because you just had the idea and the machine made it. And yeah, one thing that I wanted to say is I'd never heard the postmodern art example used in any of those discussions.

David Schmudde (19:01)

I mean, and because people don't study history, right? And so, like they don't know, but like Sol Lewitt, I mean, he published his algorithm. Like you can see a piece and read the algorithm that made it, and coming back to identity, the space that he was working in was a space that was more like a film production. But the amazing thing about Hollywood is that the unions got strong enough where they say, no, you are gonna credit everybody that worked on this film.

And so, you have this long list of credits, right? Which I think for me, like even if only five percent of the people in the world care, or one percent about these credits, like for me that's still a huge win that the artisans or the people who worked on it get the credit, get on screen, and they're part of the film production. And the film can still start with a film by Steven Spielberg. So, the identity is still, this is my movie. I made this. It's me, one person, filmed by Steven Spielberg. Except for it with Spielberg, the unions force him to acknowledge that he had help. It's not just the divine coming from heaven, then this thing manifested onto the screen.

And that that's maybe the less interesting thing about LLMs to me is that there isn't more people involved to like, I don't know, create friction in the process, create real friction. Other egos that like aren't agreeing with you, not because you're wrong, but because they had like really shitty morning. And that's the only reason that they're just, and then but actually because they disagree with you, you're like you had a second thought. And then there's something came out of that, right?

Now that's super interesting to me. Going back and forth in LLMs, I mean, you know, it's fine. I don't have something against it as an artist, but it doesn't have the texture.

Gareth King (20:56)

Yeah, that's an interesting point too. You know, I'm probably not as close to this stuff as someone like yourself is, but do you think that they're currently developing models with more of that humanity built into them? And where does that go then once these things get better and better at human mimicry?

David Schmudde (21:14)

These things are products. This seems like it can only go one way. One of the best-case outcomes of this, whole thing is if one, it doesn't destroy us all. And two, if actually these bets are bad bets on these companies, and they end up being commoditised. And if these LLMs become commoditised, if it turns out like they're actually commodities and not incredibly valuable IP, then you get some really interesting stuff, right? But as long as it's the way it is, I don't know if that's good for the world or not, but like I the current thing is not super great.

Gareth King (21:49)

Just on that point then, what are your biggest concerns about where it's heading?

David Schmudde (21:58)

So as a CTO, things are looking good because the cost of labour is gonna go down, right? These LLMs are still in my mind a little oversold. But if the bets in Silicon Valley are right in terms of like their employment picture, basically we have a situation where these programmers that work for Facebook and for Google and I mean they've created the thing that will replace them, right?

So that's what I mean by the cost of labour. So, from the CTO perspective, this is looking pretty good. From me as a human. Large swathes of unemployment, especially with something like programming where there's a real craft to it. It's quite a pleasure reading good code, well thought out code.

Gareth King (22:46)

I've heard that. When I look at it personally, I'm just bamboozled. So, I'll take you, I'll take your word for it on that one.

But just on that point, then around the cost of labour rapidly heading down towards zero. The sale price of a lot of services, products, etc. has got a lot of labour and effort value and cost baked into it, in the sale price. Do you think then if all of that labour and effort value is removed, how do you think people will be able to maintain the effort-inclusive prices on a lot of their products and services?

David Schmudde (23:26)

The way that this works, time and time again, is you get productivity gains in the marketplace and then real prices fall, but somehow the end consumer doesn't benefit like that, right.

So, I’ll give an example. Take desktop publishing and newspapers, right? Desktop publishing and was a revolution for newspapers. Doing layout, I mean, these were more than 10x gains. These were gigantic gains being able to do layout on a computer. Let alone spell checking and all the other stuff that you get baked into this. But I mean putting a newspaper out every day. These people were putting newspapers out every day, sometimes twice a day, like morning and evening editions.

So, you think about what desktop publishing, what the Atari STs, what the Macintoshes in the 1980s were giving to those enabling for those industries. And what happened in newspapers? Newspapers became more valuable, the newspapers themselves, and the real wages flattened. It's not like, I can do more so I'm more valuable, so I get paid more, right? Real wages flattened and people lost their jobs.

And most of the value went, guess where? To the very top. And newspaper prices didn't fall. Now maybe they maybe they levelled. I don't know the actual statistic, but they didn't in terms of versus inflation, you know, the price of your newspaper. But this the value clearly went, the majority of the value clearly went to one place, right?

And there is with the way that the way that the world is going, well, at least in the United States, and we also see this in Europe, where I live, but the way that it's going, there's no reason to think that this this income, this gap in wealth, this wealth gap is going to get smaller with a new technology. Because every single time in my lifetime, when there's a new technology that creates real world benefits, the monetary value goes one direction. There's no exceptions.

Gareth King (25:25)

Totally agree. And I was asking because I'd also read someone saying the other day, very confidently saying I should say, that once productivity gains are so increased that that will make bosses pay the workers more for their increased productivity. And I was I was quite taken aback by that. I was like, I've never, but you know, I guess we'll see. I personally can't imagine it happening, and…

But all of that said, what are you hopeful about with this technology as it grows, as our use of it grows and as it kind of interweaves itself further and further into our daily lives from top to bottom?

David Schmudde (26:02)

My short-term hope for it is pretty clear and it's orthogonal to AI. We've gotta get serious about having a solution for people to have identities online. Like not identities where they can be tracked, right, by the government or tracked by advertisers, but identities where I can come in anonymously and say that I'm over sixteen years old or I'm over eighteen years old, where certain aspects of our identity that are critical for infrastructural purposes, for helping to build the society that we want are there.

You live in Australia, right? So, you've had a number of different sort of legislations in the last 10 years around online content, whether it be pornography, whether it be journalism, and legislators are passing these laws and technologists can't keep up. Like I thought I thought it was supposed to be the reverse. I thought technology was supposed to be so good and the law can't keep up.

But legislators are saying, for example, we finally want to solve the pornography issue, right, with access to pornography. And the technologists say, well, we don't know how to do that. Well, I thought you were the innovators. I thought you were the innovators. And you can't give me a technology that can serve both needs, both building the society we want and also helping people keep the anonymity that's required, that is requisite in a democracy?

This is this is the antithesis of innovation, right? Because we know that these societies need it. So, these technologies have existed, by the way. We have solutions. We have we have things on the board. Adoption’s really hard. And also, it is a really hard problem, the balance between anonymity and giving up personal information and tracking and da-da.

So, it's a super hard problem. So, I don't want to underplay it, but it's a diminishingly small number of people working on this problem in technology. And they certainly aren't funded like you're funded at Facebook, right? I said the magic word though, funding.

So, the hope is that AIs force this issue a little bit because AIs are they're going to have identities online. These LLMs are going to have identities online where it's not just slop, but they're doing things on behalf their agents, right? They're agents that are doing things on behalf of me.

It's just too convenient. It's just such a huge marketplace of possibility. And if AIs get identity, it's finally going to push the identity issue forward, right? Because we need some way to know that an AI is actually an AI, right? And that means that the converse must be true, that the person who is not the AI is definitely not an AI.

And that's at least one step forward to moving the technology towards what democracies around the world say they want. And the best parts of these democracies don't want to track their citizens. I mean, you know, it's a tense area. But the best part of them do not. They the best part of them have constitutions that actually say you legally can't do this.

If we say that that's kind of the baseline, then we are in a better position to finally push this forward. And LLMs kind of forced that because it's just too lucrative of a market to ignore.

Gareth King (29:31)

Do you think that's gonna be driven by legislation or the people working on it? Like who's gonna jump first to kind of get to that balance point that you just mentioned?

David Schmudde (29:40)

Well, the legislation is there, right? I mean, these age gating laws, whatever you think about them, maybe they're using it as a front to do other things, like legislators are using this, you know, in in ways to get other gains. But whatever you think about them, we I think generally agree that we want to treat children differently than we want to treat adults online.

So, the legislation, especially recently, has come has been happening, whether we want it or not. And it's just the same thing as the GDPR privacy legislation. I mean, in the sense that, like, you know, these legislators are doing the best they can, and then they put out a flawed law, right? Like the GDPR. And there's some good stuff in there, and then there's like cookie banners, right? There's stuff like that. That's what people see.

So, legislators are doing the best they can, but the technologists aren't keeping up. And the state, for example, in Europe, the states are putting money into these IDs and they are putting they're trying to do these things, but there's a lot always a lot of tension in releasing them and you know, their citizens are correctly sceptical of the state releasing IDs and this is especially in Europe, with the history of World War Two, you know, this is very reasonable position to take.

But the people that have not put the money in are the innovators, quote unquote, the venture capitalists, these people in Silicon Valley. So, they could maybe I think now because of the market opportunity, they could finally jump second, right? And then say that they're they jumped first, right? Who cares? But they'll say, oh we got this new innovation. And so yeah, maybe that's how it plays out. I'm hoping.

Gareth King (31:26)

What do you think the human identity online of the near and, you know, further future will look like?

David Schmudde (31:35)

What do I hope or what do I think? I mean, I'll do hope. The thing that's missing that is different between the digital space and the meat space is the fact that the protocol for engendering trust is explicit in digital space, is implicit in our space.

Meaning like I meet you on the street and you know, I look at you funny and you don't trust me. I come up to you with a smile, you trust me. And none of this is like explicit. Like there's no written law that says if I smile at you and come up to you, you trust me, but you know.

And either way, I might have different intentions and what I'm showing. But the point is, is that there is like some I don't want to say protocol, but there is some shorthand building relationships in this in our space. And so, the most optimistic part of me is thinking that there is a better way to negotiate relationships online. And a lot of it has to do with implicit knowledge and context that isn't managed. Like actively managed.

And what I mean by that is this was a bad idea back then and we should I wish, they would have known. But like this idea, for example, that I can manage my privacy on every single website I go to and I give them consent to do X, Y, and Z and I read all that, that is unmanageable. No individual could manage that. It was a really stupid idea, right?

There is fundamental research being done in computer science that has sort of a softer negotiating power, where if two unknown agents meet each other in the digital space, they can negotiate through a diplomat that is like not a rigid protocol. And it has a softer logic to it. And maybe that comes through LLMs, I don't know. But we'll see.

Gareth King (33:36)

Look, thanks so much for all of that, David. You've really given us a lot to think about when it comes to the human identity in the digital world. What have you got coming up and where can people follow what you're up to?

David Schmudde (33:47)

Well, yeah, so mostly what I got coming up is my two-year-old boy. He's every day he's a little bit bigger. And then of course, you know, work at what I already said at the top, Yorba, which is at Yorba dot co. And so that's the professional work.

And then if you liked what you heard here, this sort of like society and technology studies sort of aspect, I write s few times a year at Beyond the Frame, which is at my last name. So, it's at S C H M U D Dot D E. That's S C H M U D Dot D E. You just type that in your address bar and there's more of me.

Gareth King (34:32)

David, thanks so much.

David Schmudde (34:33)

Thanks a lot.