I Spent Two Years Building the Median — Hariprasad Sudharshan

If I told you that If I told you that New York New York New York suddenly stopped suddenly stopped suddenly stopped working today, what would you working today, what would you working today, what would you say are the reasons why there are say are the reasons why there are no planes taking off from Kennedy Airport no planes taking off from Kennedy Airport ? The subway suddenly ? The subway suddenly ? The subway suddenly stops at the stops at the stops at the next station and next station and next station and comes to a halt. None of your comes to a halt. None of your comes to a halt. None of your contactless payments are contactless payments are contactless payments are working. The elevators working. The elevators working. The elevators you get into don't you get into don't you get into don't know what know what know what floor they are on. The phones aren't floor they are on. The phones aren't floor they are on. The phones aren't ringing. Wall Street ringing. Wall Street ringing. Wall Street and the whole world are and the whole world are and the whole world are freezing. What freezing. What freezing. What could be the reason? could be the reason? Software. Unless there Software. Unless there was a software was a software was a software error that affected the error that affected the error that affected the entire planet. Nothing entire planet. Nothing entire planet. Nothing else can stop else can stop else can stop this modern world. this modern world. This is how much New York and our entire world are York and our entire world are York and our entire world are actually run by actually run by actually run by invisible invisible invisible software. Not the software. Not the software. Not the AI ​​software AI ​​software we mostly we mostly talk about. This is talk about. This is talk about. This is traditional, traditional, traditional, deterministic, deterministic, rule-based, reliable software that software that created the created the created the modern world for us. It has modern world for us. It has been created by hand by been created by hand by people for over 70 years. I am Hari. I'm a people for over 70 years. I am Hari. I'm a product specialist who has been product specialist who has been building software for building software for almost two almost two decades. And I have decades. And I have decades. And I have one sincere theory.

one sincere theory. If If there is a there is a there is a truly new truly new truly new industry in human history, it is the industry in human history, it is the software industry software industry . Because . Because . Because take navigation, take navigation, agriculture, agriculture, construction, or construction, or construction, or even astronomy— even astronomy— even astronomy— we've had it in various we've had it in various we've had it in various forms and guises forms and guises forms and guises before. Maybe before. Maybe before. Maybe they've become they've become they've become more advanced now, more advanced now, more advanced now, but this is not news to us but this is not news to us but this is not news to us . Today—from . Today—from . Today—from the commute to the commute to the commute to watching this watching this watching this video, from the nuclear video, from the nuclear video, from the nuclear reactor to the reactor to the reactor to the space station, space station, space station, powering a powering a powering a $125 trillion economy—the only $125 trillion economy—the only common common common factor is factor is factor is software. So it would be a software. So it would be a software. So it would be a disaster disaster disaster if for some reason the if for some reason the if for some reason the software stopped, like in 2000 software stopped, like in 2000 software stopped, like in 2000 . And our modern . And our modern . And our modern world owes a lot to world owes a lot to this this type of type of type of software. software. software. The deterministic one The deterministic one The deterministic one that you trust with that you trust with that you trust with your your your pacemaker and pacemaker and pacemaker and when you fly your when you fly your when you fly your Boeing 747. So if you Boeing 747. So if you Boeing 747. So if you look at the product look at the product look at the product here, on the left you have here, on the left you have here, on the left you have engineers. Engineers engineers. Engineers engineers. Engineers are hired and they are the are hired and they are the are hired and they are the source of solutions for source of solutions for source of solutions for each product with each product with each product with their unique their unique their unique flavor. I flavor. I flavor. I would even say that their would even say that their would even say that their belief and value systems belief and value systems are reflected in the are reflected in the software software they release. The tool chain was

tool chain was completely completely completely subject to subject to subject to deterministic deterministic deterministic rules. The software that runs rules. The software that runs rules. The software that runs the world and the the world and the the world and the toolchains around toolchains around toolchains around it are in a it are in a it are in a cross- cross- cross- feedback loop, feedback loop, feedback loop, where assemblers where assemblers where assemblers created C, C created created C, C created created C, C created Unix, Unix created everything, and Unix, Unix created everything, and Unix, Unix created everything, and then we created Kafka, then we created Kafka, then we created Kafka, Zookeeper, and all the way to Kubernetes, Zookeeper, and all the way to Kubernetes, Zookeeper, and all the way to Kubernetes, which run which run which run the world today. The only the world today. The only the world today. The only judgment in this judgment in this judgment in this picture is human picture is human picture is human judgment. And we all judgment. And we all judgment. And we all know how know how know how different we are. And it is from this different we are. And it is from this different we are. And it is from this difference that difference that difference that every every every product advantage arises. This block product advantage arises. This block product advantage arises. This block here is a high-level here is a high-level here is a high-level representation of the entire representation of the entire software industry software industry and the and the and the global economy. global economy. global economy. Who owns the Who owns the Who owns the intellectual intellectual intellectual property? Where is the property? Where is the property? Where is the continuous cycle? And continuous cycle? And continuous cycle? And how this cycle, where each how this cycle, where each how this cycle, where each brick in the chain of brick in the chain of brick in the chain of tools, tools, tools, helps us helps us helps us conquer new conquer new conquer new heights. Is heights. Is heights. Is software software deterministic or deterministic or probabilistic—that is the probabilistic—that is the probabilistic—that is the question.

question. question. Keep this picture in your Keep this picture in your Keep this picture in your head. Let's head. Let's head. Let's see how see how see how one card will change. one card will change. Namely: yes, in the age of AI, Namely: yes, in the age of AI, engineers today are being engineers today are being engineers today are being helped, or helped, or helped, or guided, or, as guided, or, as guided, or, as they sometimes say, they sometimes say, they sometimes say, completely replaced by completely replaced by completely replaced by models that I would call models that I would call models that I would call rented. The rented. The good thing is that this good thing is that this creates a level playing field creates a level playing field creates a level playing field where everyone has where everyone has where everyone has equal access to the equal access to the equal access to the most modern most modern most modern models and models and models and tools. tools. Teams that previously Teams that previously couldn't afford the couldn't afford the couldn't afford the talent from Google or talent from Google or talent from Google or Meta to build this Meta to build this Meta to build this modern software modern software modern software can now gain can now gain can now gain intelligence through APIs that intelligence through APIs that intelligence through APIs that supposedly solved the supposedly solved the supposedly solved the development problem. development problem. Let's see if Let's see if they really decided and they really decided and they really decided and what's what's what's really happening. If you really happening. If you really happening. If you look closely and look closely and look closely and look under the hood, you look under the hood, you look under the hood, you will see that everything will see that everything will see that everything around the model: around the model: around the model: the tools, the agents, the the tools, the agents, the the tools, the agents, the tools, the MCP—it is tools, the MCP—it is tools, the MCP—it is all the same old all the same old all the same old deterministic deterministic software. Those who software. Those who build these know for sure build these know for sure build these know for sure that these are not AI that these are not AI products. It's the same products. It's the same products. It's the same old deterministic old deterministic old deterministic software, perhaps built software, perhaps built software, perhaps built using models.

using models. And what remained And what remained unchanged and became the unchanged and became the unchanged and became the result? Ultimately, result? Ultimately, result? Ultimately, the world needs software the world needs software the world needs software that reliably that reliably that reliably does the same does the same does the same job every time. So, the job every time. So, the job every time. So, the end product is end product is end product is deterministic software that deterministic software that deterministic software that drives the global drives the global drives the global economy as we economy as we economy as we know it. If so, know it. If so, know it. If so, what is the role of AI here? what is the role of AI here? Its role is, first, Its role is, first, to accelerate the development of to accelerate the development of to accelerate the development of deterministic software at deterministic software at deterministic software at the lowest possible the lowest possible the lowest possible cost. And write cost. And write cost. And write deterministic software deterministic software deterministic software that did not that did not that did not exist before due to exist before due to exist before due to budget constraints budget constraints budget constraints or overly complex or overly complex or overly complex tasks. So, these are two tasks. So, these are two tasks. So, these are two use cases use cases that I think will that I think will that I think will emerge over the emerge over the emerge over the next 5–10 years. next 5–10 years. So, the bottom line is this: So, the bottom line is this: we will we will we will use a use a use a probabilistic probabilistic probabilistic mechanism to mechanism to mechanism to create reliable create reliable create reliable deterministic software, deterministic software, deterministic software, supplementing, and some would supplementing, and some would supplementing, and some would even say even say replacing, human replacing, human replacing, human judgment. Is judgment. Is judgment. Is this possible? Where are we today? We this possible? Where are we today? We this possible? Where are we today? We will see this in this will see this in this will see this in this report. Who knows this report. Who knows this report. Who knows this person? I am a big

person? I am a big person? I am a big fan of hers. He is a superman fan of hers. He is a superman , just like his friends. , just like his friends. Okay, now I see Okay, now I see you understand everything, because you you understand everything, because you you understand everything, because you recognized these recognized these recognized these men, and he is a men, and he is a member of the elite " member of the elite " faster 10 seconds" club. faster 10 seconds" club. Two of them are relatively Two of them are relatively famous, but all three are famous, but all three are superhumans. Isn't it? So superhumans. Isn't it? So , in 2015, , in 2015, , in 2015, nine people competed, and the difference nine people competed, and the difference nine people competed, and the difference between them was only between them was only between them was only 27 hundredths of a second. And 27 hundredths of a second. And 27 hundredths of a second. And Gatlin lost Gatlin lost Gatlin lost gold by one hundredth of a gold by one hundredth of a gold by one hundredth of a second. Can you second. Can you second. Can you believe this? This is believe this? This is believe this? This is beyond the comprehension of the beyond the comprehension of the beyond the comprehension of the human mind. human mind. Scientific measurements Scientific measurements showed that this is how showed that this is how showed that this is how Usain Bolt beat Usain Bolt beat Usain Bolt beat Gatlin that night. Gatlin that night. On another evening On another evening the result could have been the result could have been the result could have been completely different. I still completely different. I still completely different. I still run the hundred-meter dash in run the hundred-meter dash in run the hundred-meter dash in 13 seconds, not 10.

13 seconds, not 10. 13 seconds, not 10. What about you? That's the What about you? That's the What about you? That's the difference between us difference between us difference between us and these and these and these extraordinary extraordinary extraordinary athletes athletes athletes competing at the limits of competing at the limits of competing at the limits of human ability human ability . But as a fan, I will . But as a fan, I will . But as a fan, I will stand up for my stand up for my stand up for my favorites, as I just favorites, as I just favorites, as I just did. However, I did. However, I did. However, I also communicate with also communicate with also communicate with professionals in this professionals in this professionals in this field. And in their field. And in their field. And in their opinion, the gap is opinion, the gap is opinion, the gap is so small that so small that so small that previous achievements previous achievements previous achievements or world records don't or world records don't or world records don't matter when matter when matter when they take the field they take the field . On any given day, anyone . On any given day, anyone . On any given day, anyone can beat can beat can beat anyone. I will apply the anyone. I will apply the anyone. I will apply the same criterion to same criterion to same criterion to models to models to models to highlight a few highlight a few highlight a few points. Six points. Six points. Six laboratories. A recent laboratories. A recent laboratories. A recent study from study from study from Stanford: a few Stanford: a few Stanford: a few dozen Elo points dozen Elo points dozen Elo points this way or that. And the this way or that. And the this way or that. And the leaderboard leaderboard leaderboard changes with each changes with each changes with each release cycle. And release cycle. And this week's popular question: Which this week's popular question: Which model do you model do you model do you use and use and use and how much how much how much smarter do you smarter do you smarter do you feel feel feel using it? Is using it? Is using it? Is n't that the main n't that the main n't that the main topic of discussion at topic of discussion at topic of discussion at dinner tonight? But dinner tonight? But dinner tonight? But I can say one thing: this is I can say one thing: this is I can say one thing: this is not necessarily a good not necessarily a good not necessarily a good topic for the topic for the topic for the boardroom. Why? Because boardroom. Why? Because boardroom. Why? Because often our often our often our emotional side, like emotional side, like emotional side, like that fan's, that fan's, that fan's, seeps into the seeps into the seeps into the negotiation rooms.

negotiation rooms. negotiation rooms. After all, we are all human. After all, we are all human. After all, we are all human. I'm not saying that you shouldn't I'm not saying that you shouldn't I'm not saying that you shouldn't compare models. I'm compare models. I'm compare models. I'm just saying: don't make a just saying: don't make a just saying: don't make a small gap in the small gap in the small gap in the frontier an frontier an frontier an advantage in your advantage in your product roadmap. Let's product roadmap. Let's dig a little dig a little dig a little deeper, because there are some deeper, because there are some deeper, because there are some numbers worth numbers worth numbers worth breaking down. Anthropic 1503, breaking down. Anthropic 1503, breaking down. Anthropic 1503, Mistral 1460—87 points Mistral 1460—87 points Mistral 1460—87 points difference either way. difference either way. This does not mean that This does not mean that Anthropic is 87 points Anthropic is 87 points Anthropic is 87 points smarter than Mistral. smarter than Mistral. In fact, these scores In fact, these scores should be seen as a should be seen as a should be seen as a relative ranking relative ranking relative ranking obtained through obtained through obtained through comparison. This comparison. This comparison. This actually shows how actually shows how actually shows how these models these models these models relate to each relate to each relate to each other in that other in that assessment environment. This is not a assessment environment. This is not a scale of absolute scale of absolute scale of absolute intelligence. intelligence. But here's the main But here's the main problem that everyone problem that everyone problem that everyone ignores. In all these ignores. In all these ignores. In all these conversations between different conversations between different conversations between different teams, I saw a teams, I saw a teams, I saw a lot of organizations. lot of organizations.

While we think we are While we think we are comparing models, I comparing models, I comparing models, I have noticed that often have noticed that often have noticed that often they are not even they are not even they are not even comparing models. We comparing models. We comparing models. We mistakenly compare the mistakenly compare the mistakenly compare the harness or product harness or product harness or product to the models. Claude Code to the models. Claude Code to the models. Claude Code vs. Cursor or Codex—this is vs. Cursor or Codex—this is vs. Cursor or Codex—this is not a not a not a model comparison. This is more model comparison. This is more model comparison. This is more like a like a like a Jamaica vs. USA comparison. You are Jamaica vs. USA comparison. You are Jamaica vs. USA comparison. You are comparing an athlete and the comparing an athlete and the comparing an athlete and the training system training system training system around that athlete. around that athlete. around that athlete. So, a model is So, a model is So, a model is an athlete or an engine. an athlete or an engine. an athlete or an engine. It provides It provides It provides basic basic basic functionality. functionality. The harness determines how The harness determines how this engine this engine this engine is used. is used. is used. Tools, Tools, Tools, context, instructions, context, instructions, context, instructions, orchestration, memory, orchestration, memory, orchestration, memory, workflows, workflows, MCP tool calls, that's the wrapping MCP tool calls, that's the wrapping around it. around it. around it. The product determines The product determines The product determines why why why anyone should care at all, anyone should care at all, anyone should care at all, because it is the end because it is the end because it is the end result. Today, result. Today, result. Today, you and I can you and I can you and I can rent a mover and rent a mover and rent a mover and almost all the almost all the almost all the tools that come with tools that come with tools that come with it. it. Some of them are closely Some of them are closely integrated, others are not integrated, others are not . So, if everyone . So, if everyone . So, if everyone is using the is using the is using the same source, as same source, as same source, as we saw in the we saw in the we saw in the previous previous previous image, image, image, then what is the difference? What used to be the case then what is the difference? What used to be the case then what is the difference? What used to be the case and and and allowed different allowed different allowed different companies to thrive companies to thrive companies to thrive in different fields in different fields in different fields was based on the was based on the was based on the diversity and diversity and diversity and uniqueness of uniqueness of uniqueness of engineering thinking engineering thinking . Since the model . Since the model . Since the model is rented, we is rented, we is rented, we increasingly see that increasingly see that increasingly see that when opportunities

when opportunities when opportunities are available to everyone, what do are available to everyone, what do are available to everyone, what do we have as a result we have as a result ? The model and ? The model and ? The model and tools don't care tools don't care who prints and from which who prints and from which who prints and from which part of the world. Or whether part of the world. Or whether part of the world. Or whether you are an E6, E7, you are an E6, E7, you are an E6, E7, or E1 level engineer. Same problems or E1 level engineer. Same problems , same tools. , same tools. , same tools. She gives the next She gives the next She gives the next person the same thing she person the same thing she person the same thing she gave you. And I want to gave you. And I want to gave you. And I want to call this answer the call this answer the call this answer the "median." And this "median." And this "median." And this median is constantly median is constantly median is constantly changing with each changing with each changing with each release cycle, as we release cycle, as we release cycle, as we saw in the chart. saw in the chart. Previously Previously solved problems solved problems solved problems get fed into these get fed into these get fed into these large language models, large language models, large language models, and that's how these releases and that's how these releases and that's how these releases come out one after come out one after come out one after another. Just like in another. Just like in another. Just like in software development software development software development . I'm oversimplifying . I'm oversimplifying . I'm oversimplifying , but basically this is what , but basically this is what , but basically this is what makes makes makes the median go up, which the median go up, which the median go up, which is good. But at the same time, is good. But at the same time, is good. But at the same time, it is also a kind of trap it is also a kind of trap . Because it's . Because it's . Because it's good enough to release, good enough to release, good enough to release, but it's hard to tell but it's hard to tell but it's hard to tell what the what the what the next person will release. Because the next person will release. Because the next person will release. Because the next person or next person or next person or competitor could competitor could competitor could release something similar release something similar release something similar or even identical.

or even identical. This This could not happen until 2023. The could not happen until 2023. The median product median product was expensive. It took a was expensive. It took a was expensive. It took a team, time, and a team, time, and a team, time, and a stack of technologies. stack of technologies. Today, this Today, this solved problem is solved problem is solved problem is available for free available for free . And this free . And this free . And this free task is available to task is available to task is available to everyone. And you go to everyone. And you go to everyone. And you go to the client just like the client just like the client just like everyone else. And then, everyone else. And then, everyone else. And then, with so many with so many with so many similar similar similar products products products flooding the market, flooding the market, flooding the market, it's distribution that it's distribution that it's distribution that becomes the advantage, not becomes the advantage, not becomes the advantage, not your product. So what is this your product. So what is this your product. So what is this advantage that you advantage that you advantage that you can create in can create in can create in this new this new this new environment? I’ll quickly environment? I’ll quickly environment? I’ll quickly run through run through run through GitHub’s numbers to back up GitHub’s numbers to back up GitHub’s numbers to back up my point: this model my point: this model my point: this model and tooling and tooling and tooling are creating a huge are creating a huge are creating a huge amount of new amount of new software. And in software. And in my opinion, it's really my opinion, it's really my opinion, it's really good good good software. But software. But software. But the results are similar. This is the results are similar. This is the results are similar. This is not a crowd of imitators.

not a crowd of imitators. not a crowd of imitators. Absolutely not. This is a crowd of Absolutely not. This is a crowd of Absolutely not. This is a crowd of people who have arrived at people who have arrived at people who have arrived at the same answer the same answer the same answer because the source or because the source or because the source or template is the same template is the same template is the same . And the word . And the word . And the word our industry has chosen our industry has chosen our industry has chosen for this is “AI slop.” for this is “AI slop.” Software—AI— garbage. I want to garbage. I want to garbage. I want to take that word from you take that word from you take that word from you and give it a and give it a and give it a new meaning. new meaning. new meaning. Let me Let me Let me be fair first. be fair first. be fair first. If you've been developing for a If you've been developing for a If you've been developing for a long time and came across long time and came across long time and came across this this this AI-powered development, it AI-powered development, it AI-powered development, it seemed like magic, seemed like magic, seemed like magic, right? Because we right? Because we right? Because we took the "black took the "black took the "black box" and box" and box" and directed directed directed a query to it. You would definitely a query to it. You would definitely a query to it. You would definitely feel at some feel at some feel at some point, "Oh, that turned out point, "Oh, that turned out point, "Oh, that turned out really well." Two really well." Two really well." Two weeks ago I released an weeks ago I released an weeks ago I released an iOS app. I was a iOS app. I was a iOS app. I was a developer, but I developer, but I developer, but I never dreamed of never dreamed of never dreamed of creating applications, creating applications, creating applications, because, remember, we because, remember, we because, remember, we had our own niches: had our own niches: had our own niches: embedded systems, embedded systems, embedded systems, application development. application development. Today, AI makes Today, AI makes everyone a developer. everyone a developer. everyone a developer. It's good for me, and I It's good for me, and I It's good for me, and I say that sincerely. The word " say that sincerely. The word " slop" has a slop" has a slop" has a negative connotation. negative connotation. negative connotation. And it is And it is And it is used without used without used without any any any empirical analysis.

empirical analysis. empirical analysis. That's why I think That's why I think That's why I think these 100,000 repositories these 100,000 repositories these 100,000 repositories and the proliferation of AI development are and the proliferation of AI development are bringing bringing new people into the field who have new people into the field who have new people into the field who have never never never programmed. The programmed. The success metrics for success metrics for these new products these new products these new products cannot be assessed cannot be assessed cannot be assessed through the lens of through the lens of through the lens of expert opinion. I would like to expert opinion. I would like to expert opinion. I would like to measure the success of those measure the success of those measure the success of those millions of AI millions of AI applications by whether applications by whether applications by whether they meet they meet they meet the expectations of their the expectations of their the expectations of their creators. Because who creators. Because who creators. Because who determines what a determines what a determines what a successful product looks like? For successful product looks like? For successful product looks like? For us, this could be something us, this could be something us, this could be something that generates that generates that generates profit. For someone profit. For someone profit. For someone else, it might be else, it might be else, it might be something they can something they can something they can show their colleagues: “ show their colleagues: “ Hey, look, I Hey, look, I Hey, look, I built this great built this great built this great website.” This is also a success. And website.” This is also a success. And website.” This is also a success. And I've seen millions of I've seen millions of I've seen millions of people who have actually people who have actually people who have actually done this done this done this with Lovable and Replit, with Lovable and Replit, with Lovable and Replit, and it's cool. They are not and it's cool. They are not and it's cool. They are not AI snobs. What do you AI snobs. What do you AI snobs. What do you get when you get when you get when you give a good give a good give a good task to a coding task to a coding task to a coding agent? Most often agent? Most often agent? Most often today it is a useful today it is a useful today it is a useful and ready-made artifact.

and ready-made artifact. Maybe it's similar to the Maybe it's similar to the others, because, you know, it's the others, because, you know, it's the others, because, you know, it's the same same same dough mold. But most dough mold. But most dough mold. But most software software developed this developed this way falls into what way falls into what way falls into what I call the " I call the " median." I'm not telling median." I'm not telling median." I'm not telling you to stop you to stop you to stop developing or releasing developing or releasing developing or releasing products. This is perfectly products. This is perfectly products. This is perfectly acceptable for acceptable for acceptable for internal internal internal use. One use. One use. One caveat: caveat: caveat: stop thinking of stop thinking of stop thinking of this as a this as a this as a differentiating factor. But differentiating factor. But differentiating factor. But this "median" is missing this "median" is missing this "median" is missing one thing. We also one thing. We also one thing. We also need to know both need to know both need to know both the advantages and the advantages and the advantages and disadvantages. This is not a new disadvantages. This is not a new disadvantages. This is not a new problem. In the 90s, everyone problem. In the 90s, everyone problem. In the 90s, everyone was trying to solve the was trying to solve the was trying to solve the search problem, and search problem, and search problem, and most had a most had a most had a basic crawler and basic crawler and basic crawler and index. And whoever index. And whoever index. And whoever solves this problem solves this problem solves this problem really well or really well or really well or has an element of has an element of has an element of novelty will rise novelty will rise novelty will rise to the top, because this is the to the top, because this is the to the top, because this is the differentiation factor differentiation factor . So today I'm . So today I'm . So today I'm saying that the median of saying that the median of saying that the median of most most software you software you can create is can create is can create is actually actually actually free. But if free. But if free. But if you want you want you want to compete in the to compete in the to compete in the open market, you open market, you open market, you need that element of need that element of need that element of novelty. So, a little bit novelty. So, a little bit novelty. So, a little bit about my about my about my competence before competence before competence before I go any further, because I've been I go any further, because I've been talking for quite some time now. I didn't start talking for quite some time now. I didn't start creating products when the creating products when the creating products when the models models models got better. I have got better. I have got better. I have created over a created over a created over a dozen public dozen public dozen public applications and applications and applications and contributed to LlamaIndex and contributed to LlamaIndex and contributed to LlamaIndex and other repositories.

other repositories. The last 3 years have been The last 3 years have been quite exciting, quite exciting, quite exciting, which is what led me to which is what led me to which is what led me to this topic, but also this topic, but also this topic, but also exhausting. I exhausting. I exhausting. I want to show you this want to show you this want to show you this because the projects I will because the projects I will because the projects I will demonstrate are demonstrate are demonstrate are not the first amateur not the first amateur not the first amateur attempts. I was trying attempts. I was trying attempts. I was trying to solve a real to solve a real to solve a real problem and ended up problem and ended up problem and ended up creating a generic creating a generic creating a generic solution. I never solution. I never solution. I never expected this. So I expected this. So I expected this. So I want to introduce a few want to introduce a few want to introduce a few terms. The " terms. The " create first" reflex. create first" reflex. When we all When we all got into AI and got into AI and got into AI and stuff, we stopped stuff, we stopped stuff, we stopped doing what we used to do— doing what we used to do— when development was when development was when development was expensive and difficult, I was expensive and difficult, I was expensive and difficult, I was more searching than more searching than more searching than building. Today everything building. Today everything building. Today everything has changed. It is easier to give has changed. It is easier to give has changed. It is easier to give a task to a a task to a coding agent than to coding agent than to coding agent than to search, read these search, read these search, read these extensive extensive extensive search results, and make search results, and make search results, and make a decision. We don't a decision. We don't a decision. We don't want to make want to make want to make decisions anymore. And this is becoming the decisions anymore. And this is becoming the decisions anymore. And this is becoming the new norm in our new norm in our new norm in our craft. So what do we craft. So what do we craft. So what do we do? Let's do? Let's do? Let's ask the agent ask the agent coder to create a RAG coder to create a RAG system. And I call system. And I call system. And I call it the " it the " create first" reflex. If create first" reflex. If you have a problem, open an agent. Yes open an agent. Yes , you get a framework in the , you get a framework in the , you get a framework in the first 2 minutes, and I'm first 2 minutes, and I'm first 2 minutes, and I'm sure if you're sure if you're sure if you're used to this kind of used to this kind of used to this kind of development, you'll make a development, you'll make a development, you'll make a demo in 2 hours—a demo in 2 hours—a working version that you working version that you working version that you can show to can show to can show to someone, albeit with someone, albeit with someone, albeit with flaws, but it flaws, but it flaws, but it works. And you works. And you works. And you could really get started could really get started could really get started in 4 weeks. Really

in 4 weeks. Really in 4 weeks. Really good product with good product with good product with support and everything support and everything support and everything else. But only else. But only else. But only then do you find out: then do you find out: then do you find out: okay, I created the 400th okay, I created the 400th okay, I created the 400th or 4000th RAG system. I or 4000th RAG system. I or 4000th RAG system. I took RAG as an example here took RAG as an example here . So, I pointed out the . So, I pointed out the . So, I pointed out the same problem and this is what same problem and this is what same problem and this is what we got. And I we got. And I we got. And I asked: “Hey, why did you asked: “Hey, why did you asked: “Hey, why did you create another RAG create another RAG system when there are system when there are system when there are more mature, more mature, more mature, feature-rich feature-rich feature-rich open-source solutions available with open-source solutions available with open-source solutions available with thousands of stars and thousands of stars and thousands of stars and proven time? We proven time? We proven time? We could just could just could just use them. We use them. We use them. We shouldn't have shouldn't have shouldn't have built this." We wouldn't have built this 3 years ago built this." We wouldn't have built this 3 years ago . And he . And he admitted: “Hey, okay, I'm to admitted: “Hey, okay, I'm to admitted: “Hey, okay, I'm to blame here. You said, " blame here. You said, " Can you create?" I Can you create?" I Can you create?" I took it and created it. If took it and created it. If took it and created it. If you had told me about this, you had told me about this, you had told me about this, I would have done something different I would have done something different ." A typical RAG is what ." A typical RAG is what ." A typical RAG is what was actually was actually was actually created, even though it is a created, even though it is a created, even though it is a solved problem. solved problem. But if you're creating But if you're creating this for the first time, it's going to be this for the first time, it's going to be this for the first time, it's going to be exciting for you because you'll exciting for you because you'll exciting for you because you'll feel like: I provided a PDF, feel like: I provided a PDF, feel like: I provided a PDF, directed it to the directed it to the directed it to the LLM backend, and started LLM backend, and started LLM backend, and started asking questions.

asking questions. asking questions. The breakdown into parts The breakdown into parts The breakdown into parts and all those factors that and all those factors that and all those factors that I understand or don't I understand or don't I understand or don't understand, the vector understand, the vector understand, the vector database will decide database will decide database will decide for me and I got the for me and I got the for me and I got the solution. Small solution. Small solution. Small wins produce wins produce wins produce a lot of dopamine, and I a lot of dopamine, and I a lot of dopamine, and I fall into the same fall into the same fall into the same traps as you. And this is a traps as you. And this is a traps as you. And this is a detailed answer detailed answer . I defined this on the . I defined this on the . I defined this on the previous slide. previous slide. Now I'll stop Now I'll stop talking about these talking about these talking about these trivial problems trivial problems trivial problems and show one of and show one of and show one of my applications. my applications. This project is one of the This project is one of the representatives of the first representatives of the first representatives of the first two years, when I two years, when I two years, when I developed many developed many developed many applications. As you applications. As you applications. As you all know, all know, all know, my my my " " " create first" reflex kicks in. I did this create first" reflex kicks in. I did this create first" reflex kicks in. I did this many times to many times to many times to come up with this term. come up with this term. I created this by I created this by sacrificing sacrificing sacrificing walks and walks and walks and entertainment. I entertainment. I entertainment. I missed a lot of missed a lot of missed a lot of parties and parties and parties and movie nights. And I movie nights. And I movie nights. And I built like built like built like crazy. It was like a crazy. It was like a crazy. It was like a game of chess that game of chess that game of chess that you become you become you become addicted to. And I had addicted to. And I had addicted to. And I had every AI every AI platform imaginable platform imaginable platform imaginable .

. . Maximum plan. Maximum plan. Maximum plan. Sometimes for free. Sometimes for free. After all this effort, After all this effort, I finally did a I finally did a I finally did a soft launch. I'm a soft launch. I'm a soft launch. I'm a bit of a perfectionist. bit of a perfectionist. So by then I was So by then I was ecstatic because I saw ecstatic because I saw ecstatic because I saw my idea instantly my idea instantly my idea instantly become something real. become something real. Although as a product manager I would manager I would manager I would need a large need a large need a large team and team and team and architects. I was able to architects. I was able to architects. I was able to do it all do it all do it all by myself. I was thrilled by myself. I was thrilled by myself. I was thrilled until I until I until I realized I hadn't yet realized I hadn't yet realized I hadn't yet compared it to compared it to compared it to reality. The truth is that people around me reality. The truth is that people around me told me that they had told me that they had seen similar seen similar seen similar calculators elsewhere. calculators elsewhere. And before anyone And before anyone thinks I stole from thinks I stole from thinks I stole from them or they stole from me, the them or they stole from me, the them or they stole from me, the amazing thing is that no one amazing thing is that no one amazing thing is that no one copied from anyone else. It's the same thing copied from anyone else. It's the same thing copied from anyone else. It's the same thing . This is a . This is a . This is a copy job. When I copy job. When I copy job. When I analyzed some of analyzed some of analyzed some of these open and these open and these open and closed closed closed calculators, I calculators, I calculators, I sometimes found them to be the sometimes found them to be the sometimes found them to be the same. Not even same. Not even same. Not even similar. And how could people in similar. And how could people in similar. And how could people in different places different places different places come to a similar come to a similar come to a similar decision? Even if decision? Even if decision? Even if you look at the you look at the you look at the LLM routing, LLM routing, LLM routing, everything everything everything matches everywhere. Like those matches everywhere. Like those matches everywhere. Like those models, the solutions also models, the solutions also models, the solutions also coincide. And they coincide. And they coincide. And they compete for a compete for a compete for a tiny space. It's tiny space. It's tiny space. It's like having the same

like having the same like having the same developers on developers on developers on your team and your your team and your your team and your competitors having the same competitors having the same developers. So you developers. So you can can can only develop intuition through only develop intuition through only develop intuition through creation, and I'm creation, and I'm creation, and I'm sure those sure those sure those developers would agree with me developers would agree with me developers would agree with me . Now . Now . Now we have something we have something we have something to recalibrate to to recalibrate to to recalibrate to find a successful path. find a successful path. So after two years of So after two years of spending almost spending almost spending almost all my time, I got tired all my time, I got tired all my time, I got tired because I was spending a lot of because I was spending a lot of because I was spending a lot of effort on every effort on every effort on every interesting idea, I took it interesting idea, I took it interesting idea, I took it and built it thanks to the and built it thanks to the and built it thanks to the " " " create first" reflex. And almost create first" reflex. And almost create first" reflex. And almost every time I found myself in a every time I found myself in a every time I found myself in a crowded market crowded market crowded market where there were already two or where there were already two or where there were already two or three players, or I three players, or I three players, or I was the first, but as soon as I took was the first, but as soon as I took was the first, but as soon as I took a month a month a month off, off, off, 60 similar 60 similar 60 similar products appeared. And whoever has the products appeared. And whoever has the products appeared. And whoever has the best best best distribution distribution distribution will obviously win this will obviously win this will obviously win this battle. So I would like to battle. So I would like to battle. So I would like to introduce another introduce another introduce another new term. Okay, new term. Okay, new term. Okay, development first, development first, development first, what next? In the what next? In the what next? In the AI ​​development life cycle, there is a AI ​​development life cycle, there is a novelty budget. So, if you novelty budget. So, if you can offer can offer can offer even one unit of even one unit of even one unit of true novelty, true novelty, true novelty, like PageRank did like PageRank did like PageRank did for Google, you can for Google, you can for Google, you can stay ahead of the stay ahead of the stay ahead of the average for at least a average for at least a average for at least a long time. If long time. If long time. If you choose a problem you choose a problem you choose a problem and want to become the and want to become the and want to become the best solution best solution best solution to it in the world, you to it in the world, you to it in the world, you have a good chance of have a good chance of have a good chance of beating your competitors.

beating your competitors. The first two years I The first two years I spent, you know, days spent, you know, days spent, you know, days and nights on architecture and nights on architecture , agent design, , agent design, , agent design, prompt chains, prompt chains, prompt chains, skills, and someone skills, and someone skills, and someone shared, "Hey, this is my shared, "Hey, this is my shared, "Hey, this is my new workflow new workflow new workflow in Claude. This is my new in Claude. This is my new in Claude. This is my new workflow in Claude." workflow in Claude." workflow in Claude." I did all of this. And I did all of this. And I did all of this. And trust me, in trust me, in trust me, in most cases most cases most cases you will get similar you will get similar you will get similar , if not , if not , if not identical, results. So I had to identical, results. So I had to identical, results. So I had to redesign redesign redesign my calculator my calculator my calculator that I was showing you, that I was showing you, that I was showing you, and this time I and this time I and this time I spent almost all my spent almost all my spent almost all my time figuring out how to make it stand time figuring out how to make it stand time figuring out how to make it stand out. I didn't out. I didn't out. I didn't spend nearly as much time showing spend nearly as much time showing spend nearly as much time showing off my off my off my code to Claude or my code to Claude or my programming skills as I programming skills as I did before. Because it did before. Because it did before. Because it seemed like magic. The seemed like magic. The seemed like magic. The average result average result average result of the model is not of the model is not of the model is not subjective. This is a subjective. This is a subjective. This is a collection of best collection of best collection of best practices. So I chose a practices. So I chose a practices. So I chose a direction and treated it direction and treated it direction and treated it like a like a like a budget. After two budget. After two budget. After two years of burning years of burning years of burning several million several million several million tokens on many tokens on many tokens on many max rates max rates , I feel like everything , I feel like everything , I feel like everything else is now else is now else is now free. And the free. And the free. And the highest highest highest return on return on return on investment is the investment is the investment is the novelty budget. So novelty budget. So novelty budget. So before you start before you start before you start developing, name developing, name developing, name one thing that is new.

one thing that is new. If there is none, you If there is none, you can use the can use the can use the middle solution, because middle solution, because middle solution, because there are many people who have there are many people who have there are many people who have already created a solution already created a solution already created a solution for your problem, I'm for your problem, I'm for your problem, I'm sure. Name sure. Name sure. Name me a problem for me a problem for me a problem for which there are no solutions. I'll which there are no solutions. I'll which there are no solutions. I'll go and find her for go and find her for go and find her for you. And then incorporate you. And then incorporate you. And then incorporate this innovation plan into this innovation plan into this innovation plan into your project roadmap and your project roadmap and your project roadmap and planning. I'm planning. I'm planning. I'm doing it now. And doing it now. And doing it now. And review it the same way review it the same way review it the same way you you you review review review project costs. project costs. project costs. Honestly, if you can't Honestly, if you can't Honestly, if you can't name name name the novelty in your the novelty in your the novelty in your project, then it's worth project, then it's worth project, then it's worth admitting that it's just a admitting that it's just a admitting that it's just a learning exercise. This is learning exercise. This is learning exercise. This is completely normal. completely normal. After two years of failures, lessons, After two years of failures, lessons, and the mechanics of dealing with and the mechanics of dealing with and the mechanics of dealing with averages, averages, I came up with these I came up with these four rules. And these are four rules. And these are four rules. And these are not rules for not rules for not rules for secondary projects secondary projects . I want you to . I want you to . I want you to use them use them use them as a roadmap for your as a roadmap for your as a roadmap for your performances performances performances next quarter.

next quarter. next quarter. After all, your After all, your After all, your products products products will definitely have duplicates, will definitely have duplicates, will definitely have duplicates, and sometimes it can be and sometimes it can be and sometimes it can be open source open source open source software. Why software. Why software. Why spend so much spend so much spend so much effort on its effort on its effort on its creation? creation? creation? Use it, Use it, Use it, fork it. So, fork it. So, fork it. So, one piece of advice: be as one piece of advice: be as principled as possible. AI assistants assistants will mostly give you will mostly give you probable answers. No matter what it probable answers. No matter what it probable answers. No matter what it was about. was about. And these things will be the And these things will be the same for almost same for almost same for almost everyone. It may be everyone. It may be everyone. It may be similar, but it is already within the scope of what has similar, but it is already within the scope of what has been decided long ago. In a world been decided long ago. In a world of average of average of average decisions, your own opinion decisions, your own opinion is the cheapest is the cheapest is the cheapest novelty you novelty you novelty you can have. So can have. So can have. So take action, because the model take action, because the model cannot offer this. Second. The path cannot offer this. Second. The path from request to from request to from request to finished product finished product is an is an is an average average average path. And there is path. And there is path. And there is absolutely nothing absolutely nothing absolutely nothing wrong with that. If you wrong with that. If you wrong with that. If you feel it's for feel it's for feel it's for internal internal internal use and your use and your use and your team should team should team should use it—go for it use it—go for it . I fully support . I fully support . I fully support this. And now the third. This is this. And now the third. This is this. And now the third. This is one of the most interesting one of the most interesting one of the most interesting things. When AI says " things. When AI says " done," it means done," it means done," it means to me: okay, AI to me: okay, AI to me: okay, AI has produced its has produced its has produced its average average average product. Now I have to product. Now I have to product. Now I have to start working from start working from start working from this point. This is this point. This is this point. This is a signal from any of a signal from any of a signal from any of your your your coding agents, because this is where coding agents, because this is where your work begins. And your work begins. And remember this. These remember this. These remember this. These coding agents have coding agents have coding agents have so far given out so far given out so far given out the latest great the latest great the latest great design for free.

design for free. They won't invent the They won't invent the next Kubernetes, next Kubernetes, next Kubernetes, Zookeeper, Kafka, or any other Zookeeper, Kafka, or any other Zookeeper, Kafka, or any other similar similar similar components. They components. They components. They will create a framework, and they will create a framework, and they will create a framework, and they do it quite do it quite do it quite well in many well in many well in many places, but this will be the places, but this will be the places, but this will be the last great last great last great design for free. And design for free. And design for free. And what is your what is your what is your job? If you job? If you job? If you want to want to want to succeed, you need to succeed, you need to succeed, you need to find the following. Here are a find the following. Here are a find the following. Here are a few methods that I few methods that I few methods that I find useful. I have find useful. I have find useful. I have shared this method with shared this method with shared this method with many leaders many leaders many leaders in different in different in different organizations and organizations and organizations and received positive received positive received positive feedback. If you are a feedback. If you are a feedback. If you are a manager and are not currently manager and are not currently manager and are not currently building anything with this building anything with this building anything with this AI, this is for you. You AI, this is for you. You AI, this is for you. You will see many demo will see many demo versions. I'm sure versions. I'm sure versions. I'm sure you get you get you get bombarded with them every week. And that bombarded with them every week. And that bombarded with them every week. And that seems like a big seems like a big seems like a big deal, because I've seen deal, because I've seen deal, because I've seen leadership instincts leadership instincts leadership instincts not be tuned in, not be tuned in, not be tuned in, because when they were because when they were because when they were forming their forming their forming their intuition, the result intuition, the result intuition, the result they're seeing they're seeing they're seeing now took them a now took them a now took them a quarter or more of their quarter or more of their quarter or more of their time and required an entire time and required an entire time and required an entire team. Today, a team. Today, a team. Today, a new new new engineer appears and claims to engineer appears and claims to engineer appears and claims to have built the same thing that would have have built the same thing that would have have built the same thing that would have previously previously previously taken a taken a taken a team and a quarter of a team and a quarter of a team and a quarter of a job. So now is the job. So now is the job. So now is the most difficult time most difficult time most difficult time to make decisions about to make decisions about to make decisions about what to invest your what to invest your what to invest your resources in. Especially resources in. Especially resources in. Especially when you are not immersed in when you are not immersed in when you are not immersed in these technologies. Because these technologies. Because these technologies. Because no one can no one can no one can rewire themselves

rewire themselves rewire themselves just by reading or just by reading or just by reading or watching YouTube or watching YouTube or watching YouTube or LinkedIn posts. You LinkedIn posts. You LinkedIn posts. You are reconfiguring are reconfiguring are reconfiguring through creation. You through creation. You through creation. You develop intuition develop intuition develop intuition through creation. So through creation. So , the strategy is simple. , the strategy is simple. , the strategy is simple. Three questions. This won't Three questions. This won't Three questions. This won't solve your solve your solve your problems, but these are problems, but these are problems, but these are useful questions to useful questions to useful questions to ask. ask. ask. One. Who has already One. Who has already One. Who has already done this? This question is done this? This question is done this? This question is compelling, because in compelling, because in compelling, because in a world of exciting AI, a world of exciting AI, a world of exciting AI, we need to we need to we need to curb the “ curb the “ build first” reflex build first” reflex build first” reflex fueled by AI. Two fueled by AI. Two . What do we have that the . What do we have that the . What do we have that the other other other team, our team, our team, our competitors, or another competitors, or another competitors, or another person on the team doesn't have? This person on the team doesn't have? This person on the team doesn't have? This instills novelty instills novelty instills novelty as a priority in as a priority in as a priority in the minds of the minds of the minds of team members. Because team members. Because team members. Because the best prompt, as the best prompt, as the best prompt, as some would say, will some would say, will some would say, will just give you a just give you a just give you a mediocre problem mediocre problem mediocre problem and a mediocre solution. and a mediocre solution. And third, where will And third, where will this application live? In the this application live? In the this application live? In the era of exciting AI, era of exciting AI, era of exciting AI, imposing a imposing a imposing a performance performance performance mindset when everyone mindset when everyone mindset when everyone is excited about development is excited about development is excited about development is a way to show is a way to show is a way to show that demoing is an that demoing is an that demoing is an expensive way expensive way expensive way to expose the lack of a to expose the lack of a to expose the lack of a distribution model. And distribution model. And distribution model. And I have been strictly I have been strictly I have been strictly applying all of this applying all of this applying all of this for the past for the past for the past year. And you may year. And you may year. And you may ask, what happened ask, what happened ask, what happened to that project to that project to that project I told you about I told you about ? I asked these

? I asked these ? I asked these questions. Okay, questions. Okay, questions. Okay, if I have to provide if I have to provide if I have to provide value that is value that is value that is above average, then what above average, then what above average, then what stands between an open stands between an open stands between an open model and a model and a model and a token generation system? And token generation system? And token generation system? And what price should the company pay for this what price should the company pay for this ? Because open scales are ? Because open scales are in fashion now, but in fashion now, but in fashion now, but open scales are not open scales are not open scales are not open inference. And open inference. And open inference. And I learned this the I learned this the I learned this the hard way, because there are hard way, because there are hard way, because there are 10 layers between the 10 layers between the 10 layers between the open scales and the open scales and the open scales and the system that generates system that generates system that generates these tokens. We these tokens. We these tokens. We take for take for take for granted that granted that granted that the labs provide the labs provide the labs provide us with ML Ops, is SRE on a completely us with ML Ops, is SRE on a completely us with ML Ops, is SRE on a completely different scale different scale different scale that the world has not yet that the world has not yet that the world has not yet seen. So, precision, seen. So, precision, seen. So, precision, KV cache, batch KV cache, batch KV cache, batch processing, processing, processing, execution parallelism—and there are different execution parallelism—and there are different execution parallelism—and there are different ways to ways to ways to combine them. So for combine them. So for combine them. So for some of them you have to some of them you have to some of them you have to hire hire hire experts. And I don't know experts. And I don't know where these where these where these experts come from, because this is not a experts come from, because this is not a experts come from, because this is not a technology or technology or technology or skill that the skill that the skill that the industry already has. And industry already has. And industry already has. And some of them, layer some of them, layer some of them, layer nine, are nine, are nine, are recycling. This is recycling. This is recycling. This is structural. This is structural. This is structural. This is impossible to buy. So impossible to buy. So , if, as I said, , if, as I said, , if, as I said, you take one you take one you take one problem and you want problem and you want problem and you want to be the best at to be the best at to be the best at solving it, you're not solving it, you're not solving it, you're not going to get above going to get above going to get above average average just just just by using these AI by using these AI tools. I searched

tools. I searched tools. I searched and found that none of the and found that none of the and found that none of the 244 calculators 244 calculators 244 calculators I showed earlier I showed earlier I showed earlier did this. And I did this. And I did this. And I changed the whole changed the whole changed the whole application to something like application to something like application to something like how much does it how much does it how much does it cost cost cost to run identical scales to run identical scales to run identical scales inside your inside your inside your enterprise enterprise enterprise versus running them versus running them versus running them in the Neo cloud and in the Neo cloud and in the Neo cloud and getting tokens. getting tokens. That's what this calculator gives you That's what this calculator gives you . Because if . Because if . Because if you have open you have open you have open models and you want models and you want models and you want to use them, to use them, to use them, what's the difference? We've what's the difference? We've what's the difference? We've had great had great had great feedback from many feedback from many feedback from many businesses that businesses that businesses that really want to go down really want to go down really want to go down this path, because when this path, because when this path, because when people hear about people hear about people hear about open models, they get open models, they get open models, they get excited. For excited. For excited. For what? Organizations considering what? Organizations considering deploying models deploying models with open scales with open scales with open scales receive these scales receive these scales receive these scales for free. But you for free. But you for free. But you should check. Here is the should check. Here is the should check. Here is the QR code. So QR code. So QR code. So scan it, scan it, scan it, implement it into your implement it into your implement it into your work, and see work, and see work, and see where it takes you.

where it takes you. where it takes you. It is open source It is open source software and it is software and it is free. So, free. So, free. So, this is where it all this is where it all this is where it all begins and begins and begins and ends. ends. Thanks to the power of LLM , the development process , the development process , the development process has become much simpler over the past 3 years has become much simpler over the past 3 years has become much simpler over the past 3 years . But . But . But the changes and dynamics of the changes and dynamics of the changes and dynamics of software development software development software development ultimately have ultimately have n't changed. n't changed. Prompt-based products are useful. Prompt-based products are useful. I'm releasing them too. This is I'm releasing them too. This is I'm releasing them too. This is not junk. But they not junk. But they not junk. But they have low have low have low durability in durability in durability in open open open competition. And they competition. And they competition. And they can only survive can only survive can only survive if you have a if you have a if you have a huge huge huge distribution channel. So, distribution channel. So, distribution channel. So, if you want to if you want to if you want to succeed in this succeed in this succeed in this field, I repeat: field, I repeat: field, I repeat: take one take one take one problem and problem and problem and try to become the try to become the try to become the best in the world at best in the world at best in the world at solving it. Yes, solving it. Yes, solving it. Yes, we may not we may not we may not be able to be able to be able to solve it, but solve it, but solve it, but such an intention will definitely such an intention will definitely such an intention will definitely raise you above the raise you above the raise you above the average level. The average level. The only question only question left is: will left is: will left is: will your AI- your AI- your AI- powered products be able powered products be able to thrive? I to thrive? I spent 2 years on spent 2 years on spent 2 years on average average average decisions, but it was a decisions, but it was a decisions, but it was a learning experience. It was learning experience. It was learning experience. It was worth every worth every worth every token spent.

token spent. token spent. At the beginning, I asked At the beginning, I asked At the beginning, I asked two questions. What two questions. What two questions. What will happen to will happen to will happen to the the the software? In software? In software? In my opinion, nothing will happen to the software that my opinion, nothing will happen to the software that powers this $ powers this $ 100 trillion 100 trillion 100 trillion economy economy economy . The world will . The world will . The world will continue to run continue to run continue to run on deterministic on deterministic on deterministic software, at least for the software, at least for the software, at least for the foreseeable foreseeable foreseeable future, and future, and future, and perhaps forever. The era of perhaps forever. The era of perhaps forever. The era of AI means we AI means we AI means we can write it can write it can write it faster, but it won't faster, but it won't faster, but it won't replace replace replace deterministic deterministic software. So in software. So in conclusion, I want to conclusion, I want to conclusion, I want to say: spend say: spend say: spend everything you have on one everything you have on one everything you have on one thing, namely novelty thing, namely novelty . Squeeze every . Squeeze every . Squeeze every drop, every ounce of drop, every ounce of drop, every ounce of your intelligence, for your intelligence, for your intelligence, for novelty, because novelty novelty, because novelty is scarcity, and mediocrity is free is scarcity, and mediocrity is free . The choice is . The choice is . The choice is yours. Thank you.