Changing how Engineering works with Ironclad CTO Sunita Verma

Hey friends, shout out to Text Control Hey friends, shout out to Text Control for sponsoring the show. Developers are for sponsoring the show. Developers are for sponsoring the show. Developers are asking a new question. Do we still need asking a new question. Do we still need asking a new question. Do we still need thirdparty libraries if AI can just thirdparty libraries if AI can just thirdparty libraries if AI can just generate all the code? For common UI generate all the code? For common UI generate all the code? For common UI tasks, maybe not always, but deep tasks, maybe not always, but deep tasks, maybe not always, but deep functionality is different. Document functionality is different. Document functionality is different. Document editing, PDF generation, rendering, editing, PDF generation, rendering, editing, PDF generation, rendering, conversion, enterprise document conversion, enterprise document conversion, enterprise document workflows are the results of years of workflows are the results of years of workflows are the results of years of engineering, lots of extensive testing, engineering, lots of extensive testing, engineering, lots of extensive testing, and millions of realworld deployments. and millions of realworld deployments. and millions of realworld deployments. AI can generate a lot of code, but it AI can generate a lot of code, but it AI can generate a lot of code, but it can't instantly generate decades of can't instantly generate decades of can't instantly generate decades of proven document technology. And TX text proven document technology. And TX text proven document technology. And TX text control gives developers that foundation control gives developers that foundation control gives developers that foundation so you can focus on building your so you can focus on building your so you can focus on building your application and not rebuilding a application and not rebuilding a application and not rebuilding a document engine. Learn more at document engine. Learn more at document engine. Learn more at textcontrol.com. textcontrol.com. textcontrol.com. [music] Hi, I'm Scott Hansselman. It's another Hi, I'm Scott Hansselman. It's another episode of Hansel Minutes. Today I have episode of Hansel Minutes. Today I have episode of Hansel Minutes. Today I have the pleasure of chatting with Shenita the pleasure of chatting with Shenita the pleasure of chatting with Shenita Verma. She is the CTO at Ironclad. How Verma. She is the CTO at Ironclad. How Verma. She is the CTO at Ironclad. How are you? [music] are you? [music] are you? [music] >> Good. Very good. Thank you. Excited to >> Good. Very good. Thank you. Excited to >> Good. Very good. Thank you. Excited to be here.

be here. be here. >> Yeah, thanks for hanging out. Um, you >> Yeah, thanks for hanging out. Um, you >> Yeah, thanks for hanging out. Um, you have been around a while and you worked have been around a while and you worked have been around a while and you worked at Google for 18 years. You've uh worked at Google for 18 years. You've uh worked at Google for 18 years. You've uh worked at Semantic many many many years ago. at Semantic many many many years ago. at Semantic many many many years ago. Uh, you got your masters at um, UC Santa Uh, you got your masters at um, UC Santa Uh, you got your masters at um, UC Santa Barbara. So, I feel a little bit of a Barbara. So, I feel a little bit of a Barbara. So, I feel a little bit of a kinship because I feel like we are kinship because I feel like we are kinship because I feel like we are people who have seen it all, but you're people who have seen it all, but you're people who have seen it all, but you're choosing to stick around during this AI choosing to stick around during this AI choosing to stick around during this AI moment and help engineering teams moment and help engineering teams moment and help engineering teams navigate. navigate. navigate. >> Yes. And um I would say we are people >> Yes. And um I would say we are people >> Yes. And um I would say we are people who have seen a lot, maybe not all. who have seen a lot, maybe not all. who have seen a lot, maybe not all. >> Yeah. >> Yeah. >> Yeah. >> And um the reason I'm still um I'm still >> And um the reason I'm still um I'm still >> And um the reason I'm still um I'm still in the game is I've been working in AI in the game is I've been working in AI in the game is I've been working in AI for many years. for many years. for many years. >> I went through the winter. I went >> I went through the winter. I went >> I went through the winter. I went through the phase when it was machine through the phase when it was machine through the phase when it was machine learning before it became deep learning learning before it became deep learning learning before it became deep learning before it became you know generative AI before it became you know generative AI before it became you know generative AI and for me um given that I'm a I'm a and for me um given that I'm a I'm a and for me um given that I'm a I'm a technology optimist and I believe in the technology optimist and I believe in the technology optimist and I believe in the potential of technology for bringing potential of technology for bringing potential of technology for bringing about positive transformation and this about positive transformation and this about positive transformation and this is as big a transformation as we could is as big a transformation as we could is as big a transformation as we could hope to experience in our lifetime so I hope to experience in our lifetime so I hope to experience in our lifetime so I wanted to be there.

wanted to be there. wanted to be there. >> Yeah. Is it is it hard to be a >> Yeah. Is it is it hard to be a >> Yeah. Is it is it hard to be a technology optimist? I'm finding a technology optimist? I'm finding a technology optimist? I'm finding a little bit someone called me toxically little bit someone called me toxically little bit someone called me toxically positive recently. positive recently. positive recently. >> [laughter] >> [laughter] >> [laughter] >> I think is a compliment, but it may be a >> I think is a compliment, but it may be a >> I think is a compliment, but it may be a backhanded compliment. But like I'm backhanded compliment. But like I'm backhanded compliment. But like I'm choosing to to in this moment where choosing to to in this moment where choosing to to in this moment where everyone's saying AI is going to ruin everyone's saying AI is going to ruin everyone's saying AI is going to ruin everything to try to be positive and say everything to try to be positive and say everything to try to be positive and say it's actually going to make things it's actually going to make things it's actually going to make things better if we decide to make things better if we decide to make things better if we decide to make things better. I I agree with that framing that better. I I agree with that framing that better. I I agree with that framing that if we decide to make things better and I if we decide to make things better and I if we decide to make things better and I think a lot of it is in our hands as think a lot of it is in our hands as think a lot of it is in our hands as well as to how we leverage the well as to how we leverage the well as to how we leverage the technology for what purposes and for technology for what purposes and for technology for what purposes and for what let's say end goals and this is not what let's say end goals and this is not what let's say end goals and this is not different Scott from any other different Scott from any other different Scott from any other technology curve we may have seen prior technology curve we may have seen prior technology curve we may have seen prior um take social media take mobile take um take social media take mobile take um take social media take mobile take infrastructure everything went through a infrastructure everything went through a infrastructure everything went through a similar curve so I don't expect it'll be similar curve so I don't expect it'll be similar curve so I don't expect it'll be that different with this technology that different with this technology that different with this technology curve as well although the technology curve as well although the technology curve as well although the technology itself is substantially different. itself is substantially different. itself is substantially different. >> Yeah. It doesn't feel like just another >> Yeah. It doesn't feel like just another >> Yeah. It doesn't feel like just another tool like we got we got iPhones, we got tool like we got we got iPhones, we got tool like we got we got iPhones, we got cellular, we got the cloud. Those felt cellular, we got the cloud. Those felt cellular, we got the cloud. Those felt like big deals. This feels like an even like big deals. This feels like an even like big deals. This feels like an even bigger deal than the cloud and that was bigger deal than the cloud and that was bigger deal than the cloud and that was a big deal.

a big deal. a big deal. >> Yeah. And it it's a bigger deal in my >> Yeah. And it it's a bigger deal in my >> Yeah. And it it's a bigger deal in my opinion because the technology on the opinion because the technology on the opinion because the technology on the surface feels very easy, very relatable surface feels very easy, very relatable surface feels very easy, very relatable because natural language is the because natural language is the because natural language is the interface, but under the hood it works interface, but under the hood it works interface, but under the hood it works really differently and that can actually really differently and that can actually really differently and that can actually result in surprises for a lot of people. result in surprises for a lot of people. result in surprises for a lot of people. So it it has a very I don't know what to So it it has a very I don't know what to So it it has a very I don't know what to say. It has a very weird dynamic to the say. It has a very weird dynamic to the say. It has a very weird dynamic to the technology. technology. technology. >> Yeah. I feel like one of the things that >> Yeah. I feel like one of the things that >> Yeah. I feel like one of the things that people aren't talking about is how it people aren't talking about is how it people aren't talking about is how it makes us feel. Mhm. makes us feel. Mhm. makes us feel. Mhm. >> that you know we are talking to a big >> that you know we are talking to a big >> that you know we are talking to a big number matrix and we're talking to math number matrix and we're talking to math number matrix and we're talking to math but you put a voice on it or someone but you put a voice on it or someone but you put a voice on it or someone adds a feature in open AI where it it adds a feature in open AI where it it adds a feature in open AI where it it breathes or it chuckles or it goes um breathes or it chuckles or it goes um breathes or it chuckles or it goes um and they're going out of their way to and they're going out of their way to and they're going out of their way to anthropomorphize anthropomorphize anthropomorphize these numbers. these numbers. these numbers. >> Yeah. >> Yeah. >> Yeah. >> And I feel sometimes that that's going >> And I feel sometimes that that's going >> And I feel sometimes that that's going to make people feel a certain kind of to make people feel a certain kind of to make people feel a certain kind of way. And particularly in software way. And particularly in software way. And particularly in software engineering teams where they might feel engineering teams where they might feel engineering teams where they might feel like well this thing this person this like well this thing this person this like well this thing this person this entity is going to replace me. It's entity is going to replace me. It's entity is going to replace me. It's smarter than me and I don't believe it smarter than me and I don't believe it smarter than me and I don't believe it is smarter than us. It may know things is smarter than us. It may know things is smarter than us. It may know things we don't know but I don't I don't we don't know but I don't I don't we don't know but I don't I don't believe it's more intelligent than we believe it's more intelligent than we believe it's more intelligent than we are.

are. are. >> Yeah. Yeah. So the way I at least I have >> Yeah. Yeah. So the way I at least I have >> Yeah. Yeah. So the way I at least I have thought about it even though I know the thought about it even though I know the thought about it even though I know the internal workings of the technology the internal workings of the technology the internal workings of the technology the way to think about it is it has been way to think about it is it has been way to think about it is it has been able to compress human knowledge like able to compress human knowledge like able to compress human knowledge like whatever it has learned it has learned whatever it has learned it has learned whatever it has learned it has learned from us both across all the human beings from us both across all the human beings from us both across all the human beings that created the knowledge and across that created the knowledge and across that created the knowledge and across the time dimension. So we as our brains the time dimension. So we as our brains the time dimension. So we as our brains like as no single human being can like as no single human being can like as no single human being can compress that much knowledge in their compress that much knowledge in their compress that much knowledge in their brain. So yes in that sense it brain. So yes in that sense it brain. So yes in that sense it definitely has more knowledge compressed definitely has more knowledge compressed definitely has more knowledge compressed into it but intelligence part yeah I I'm into it but intelligence part yeah I I'm into it but intelligence part yeah I I'm not so sure and the reason is I actually not so sure and the reason is I actually not so sure and the reason is I actually came across a paper very recently on came across a paper very recently on came across a paper very recently on recursive super intelligence from Shinga recursive super intelligence from Shinga recursive super intelligence from Shinga University and one of the things they University and one of the things they University and one of the things they said is they were using the setup is said is they were using the setup is said is they were using the setup is that they were using AI to train AI and that they were using AI to train AI and that they were using AI to train AI and one of the things they found is that one of the things they found is that one of the things they found is that >> AI kept optimizing within the given >> AI kept optimizing within the given >> AI kept optimizing within the given strategy and didn't actually change the strategy and didn't actually change the strategy and didn't actually change the strategy which sort of makes sense given strategy which sort of makes sense given strategy which sort of makes sense given how it works because it has already how it works because it has already how it works because it has already found um what do I say a local optima found um what do I say a local optima found um what do I say a local optima and it wants to stay in that local and it wants to stay in that local and it wants to stay in that local optima it while as human beings we are optima it while as human beings we are optima it while as human beings we are much more creative we are much more sort much more creative we are much more sort much more creative we are much more sort of out of the box thinkers if we are not of out of the box thinkers if we are not of out of the box thinkers if we are not getting the good result given in a given getting the good result given in a given getting the good result given in a given situation we'll say you know what let's situation we'll say you know what let's situation we'll say you know what let's just think of let's just rethink the just think of let's just rethink the just think of let's just rethink the whole paradigm itself and that's not whole paradigm itself and that's not whole paradigm itself and that's not something the AI can do so that's what I something the AI can do so that's what I something the AI can do so that's what I still that's why I still associate still that's why I still associate still that's why I still associate critical thinking imagination creativity critical thinking imagination creativity critical thinking imagination creativity with human beings with human beings with human beings >> that's a great point. I think, you know, >> that's a great point. I think, you know, >> that's a great point. I think, you know, distillation is different than remixing.

distillation is different than remixing. distillation is different than remixing. And if I get a mixtape, I might make a And if I get a mixtape, I might make a And if I get a mixtape, I might make a copy of it and then it might become a copy of it and then it might become a copy of it and then it might become a less quality mixtape, but as a human, less quality mixtape, but as a human, less quality mixtape, but as a human, I'm going to want to remix it, change I'm going to want to remix it, change I'm going to want to remix it, change it, do something different with it. So, it, do something different with it. So, it, do something different with it. So, I think you're right. I I was giving a I think you're right. I I was giving a I think you're right. I I was giving a session recently at a for a bunch of session recently at a for a bunch of session recently at a for a bunch of nontechnical AI users, and I talked nontechnical AI users, and I talked nontechnical AI users, and I talked about the JPEGing of knowledge, right? about the JPEGing of knowledge, right? about the JPEGing of knowledge, right? And it's like someone went they said, And it's like someone went they said, And it's like someone went they said, "Oh, is all of Wikipedia in that one "Oh, is all of Wikipedia in that one "Oh, is all of Wikipedia in that one file?" And it's like, well, no, they file?" And it's like, well, no, they file?" And it's like, well, no, they kind of just said kind of just said kind of just said >> the sky is blue >> the sky is blue >> the sky is blue >> instead of giving me each individual >> instead of giving me each individual >> instead of giving me each individual blue pixel and all the little details. blue pixel and all the little details. blue pixel and all the little details. It was like we went file, save as, and It was like we went file, save as, and It was like we went file, save as, and we picked JPEG and we made a very, very we picked JPEG and we made a very, very we picked JPEG and we made a very, very small thing. small thing. small thing. >> We threw away a lot. It is lossy. So >> We threw away a lot. It is lossy. So >> We threw away a lot. It is lossy. So then we had a whole conversation around then we had a whole conversation around then we had a whole conversation around lossy compression. And it doesn't feel lossy compression. And it doesn't feel lossy compression. And it doesn't feel lossy though. lossy though. lossy though. >> And I think that's the trick. >> And I think that's the trick. >> And I think that's the trick. >> Yeah, that's the trick. Exactly. I was >> Yeah, that's the trick. Exactly. I was >> Yeah, that's the trick. Exactly. I was going to say that's a trick. Yes. going to say that's a trick. Yes. going to say that's a trick. Yes. >> Yeah. So let me ask you this though. you >> Yeah. So let me ask you this though. you >> Yeah. So let me ask you this though. you you you know that we're in a branding you you know that we're in a branding you you know that we're in a branding moment as well in the sense of it's fun moment as well in the sense of it's fun moment as well in the sense of it's fun to say AI and LLMs have made people to say AI and LLMs have made people to say AI and LLMs have made people think that deep learning and machine think that deep learning and machine think that deep learning and machine learning is all just next token learning is all just next token learning is all just next token prediction even though the ecosystem of prediction even though the ecosystem of prediction even though the ecosystem of of machine learning and deep learning of machine learning and deep learning of machine learning and deep learning and AI in general has been around for and AI in general has been around for and AI in general has been around for for decades but now that we have these for decades but now that we have these for decades but now that we have these bots and everyone has stopped typing bots and everyone has stopped typing bots and everyone has stopped typing code and is now chatting with the bot is code and is now chatting with the bot is code and is now chatting with the bot is there something or anything that you there something or anything that you there something or anything that you once believed about engineering organiz once believed about engineering organiz once believed about engineering organiz organizations 5 years ago say that now organizations 5 years ago say that now organizations 5 years ago say that now your belief has changed and you believe your belief has changed and you believe your belief has changed and you believe something different.

something different. something different. >> Yeah. One thing that has definitely >> Yeah. One thing that has definitely >> Yeah. One thing that has definitely changed I I'll say two things actually. changed I I'll say two things actually. changed I I'll say two things actually. I'll say things that have changed and I'll say things that have changed and I'll say things that have changed and I'll say things that have not changed I'll say things that have not changed I'll say things that have not changed for organizations. I think what has for organizations. I think what has for organizations. I think what has changed is um the leverage basically uh changed is um the leverage basically uh changed is um the leverage basically uh where engineering organizations get where engineering organizations get where engineering organizations get their leverage from and what I mean by their leverage from and what I mean by their leverage from and what I mean by that is if you think about let's say that is if you think about let's say that is if you think about let's say pre-agentic coding AI era pre-agentic coding AI era pre-agentic coding AI era >> in order to deliver an outcome you >> in order to deliver an outcome you >> in order to deliver an outcome you needed a group of engineers and the needed a group of engineers and the needed a group of engineers and the speed of let's say code generation and speed of let's say code generation and speed of let's say code generation and outcome generation was limited to quite outcome generation was limited to quite outcome generation was limited to quite a bit of extent by how how much code a bit of extent by how how much code a bit of extent by how how much code could people write how much good code could people write how much good code could people write how much good code could people write could people write could people write >> but code writing is not the bottleneck >> but code writing is not the bottleneck >> but code writing is not the bottleneck anymore. So that part of the leverage anymore. So that part of the leverage anymore. So that part of the leverage has shifted and that's what actually in has shifted and that's what actually in has shifted and that's what actually in my opinion makes smaller teams equally my opinion makes smaller teams equally my opinion makes smaller teams equally compelling in terms of producing compelling in terms of producing compelling in terms of producing outcomes as larger teams. What has not outcomes as larger teams. What has not outcomes as larger teams. What has not changed and this is the part I actually changed and this is the part I actually changed and this is the part I actually believe very strongly about is I never believe very strongly about is I never believe very strongly about is I never thought at least I personally never thought at least I personally never thought at least I personally never thought engineering an engineer's job thought engineering an engineer's job thought engineering an engineer's job was only to write code.

was only to write code. was only to write code. >> Writing code was one [clears throat] >> Writing code was one [clears throat] >> Writing code was one [clears throat] part of the job. The part the bigger part of the job. The part the bigger part of the job. The part the bigger part of the job was thinking about the part of the job was thinking about the part of the job was thinking about the problem, thinking about creative problem, thinking about creative problem, thinking about creative solutions to the problem, evaluating solutions to the problem, evaluating solutions to the problem, evaluating alternatives to the problem, judging alternatives to the problem, judging alternatives to the problem, judging basically whether the solution that basically whether the solution that basically whether the solution that you've come up with as a good one or you've come up with as a good one or you've come up with as a good one or not, brainstorming the outcomes not, brainstorming the outcomes not, brainstorming the outcomes basically with different people. And I basically with different people. And I basically with different people. And I actually think with and then obviously actually think with and then obviously actually think with and then obviously coding, testing and all those things coding, testing and all those things coding, testing and all those things were there as well. And with the tooling were there as well. And with the tooling were there as well. And with the tooling now in place, all those other aspects now in place, all those other aspects now in place, all those other aspects that I'm talking about have become even that I'm talking about have become even that I'm talking about have become even more important. So that part of the job more important. So that part of the job more important. So that part of the job maybe has changed for better in in in maybe has changed for better in in in maybe has changed for better in in in that sense for me. So I think engineers that sense for me. So I think engineers that sense for me. So I think engineers who were always looking for who were always looking for who were always looking for creative ways to solve problems, they creative ways to solve problems, they creative ways to solve problems, they are still having a lot of fun. are still having a lot of fun. are still having a lot of fun. >> Yeah, I am having fun. But I think I'm >> Yeah, I am having fun. But I think I'm >> Yeah, I am having fun. But I think I'm mourning what has been lost, but I'm not mourning what has been lost, but I'm not mourning what has been lost, but I'm not quite sure if I've lost problem solving, quite sure if I've lost problem solving, quite sure if I've lost problem solving, if I've lost toil. Um, there was an if I've lost toil. Um, there was an if I've lost toil. Um, there was an engineer at Microsoft who recently had a engineer at Microsoft who recently had a engineer at Microsoft who recently had a spicy tweet where he said that the spicy tweet where he said that the spicy tweet where he said that the typing part of coding is over.

typing part of coding is over. typing part of coding is over. >> I see. Um, I think typing I I I don't >> I see. Um, I think typing I I I don't >> I see. Um, I think typing I I I don't agree with that for the following agree with that for the following agree with that for the following reason. You could have put a voice reason. You could have put a voice reason. You could have put a voice interface to an existing even the you interface to an existing even the you interface to an existing even the you know the editors early earlier editors know the editors early earlier editors know the editors early earlier editors and if voice interface was good enough and if voice interface was good enough and if voice interface was good enough it would have it could have taken a it would have it could have taken a it would have it could have taken a dictation in some sense for you and it dictation in some sense for you and it dictation in some sense for you and it could have written the code for you. But could have written the code for you. But could have written the code for you. But I think what has changed nowadays is the I think what has changed nowadays is the I think what has changed nowadays is the ability to come up with the code itself ability to come up with the code itself ability to come up with the code itself through these tools. through these tools. through these tools. >> Okay. Interesting. But then I still >> Okay. Interesting. But then I still >> Okay. Interesting. But then I still people still feel like they've been people still feel like they've been people still feel like they've been they've had something taken from them they've had something taken from them they've had something taken from them >> that that that I do I do see this you >> that that that I do I do see this you >> that that that I do I do see this you know I would say a biodal distribution know I would say a biodal distribution know I would say a biodal distribution in some sense where there is a group of in some sense where there is a group of in some sense where there is a group of engineers who felt coding was such a engineers who felt coding was such a engineers who felt coding was such a enjoyable experience and for some it enjoyable experience and for some it enjoyable experience and for some it genuinely was genuinely was genuinely was >> that they miss it and for others it was >> that they miss it and for others it was >> that they miss it and for others it was just a tool and they actually enjoyed just a tool and they actually enjoyed just a tool and they actually enjoyed the problem solving part more. the problem solving part more. the problem solving part more. That is a really thoughtful analysis. I That is a really thoughtful analysis. I That is a really thoughtful analysis. I think you're right. And one of the think you're right. And one of the think you're right. And one of the analogies that I've been testing out is analogies that I've been testing out is analogies that I've been testing out is I was in Japan and I got to see Japanese I was in Japan and I got to see Japanese I was in Japan and I got to see Japanese carpentry, thousand-year-old practice of carpentry, thousand-year-old practice of carpentry, thousand-year-old practice of knowing how the wood fits and they would knowing how the wood fits and they would knowing how the wood fits and they would spend months to build a house. But then spend months to build a house. But then spend months to build a house. But then I came back to America and I went to I came back to America and I went to I came back to America and I went to IKEA and I got some shelves for my kid IKEA and I got some shelves for my kid IKEA and I got some shelves for my kid to go to college. So how do I live in a to go to college. So how do I live in a to go to college. So how do I live in a world where there is both IKEA and there world where there is both IKEA and there world where there is both IKEA and there is Japanese carpentry and appreciate is Japanese carpentry and appreciate is Japanese carpentry and appreciate them both and their place that they both them both and their place that they both them both and their place that they both exist.

exist. exist. >> Yeah. So I think this this is how I >> Yeah. So I think this this is how I >> Yeah. So I think this this is how I think about it is the craft part of think about it is the craft part of think about it is the craft part of basically basically basically how to bring together a solution how to how to bring together a solution how to how to bring together a solution how to construct a solution make it best make construct a solution make it best make construct a solution make it best make it well optimized make it well scalable it well optimized make it well scalable it well optimized make it well scalable make it reliable. I don't think any of make it reliable. I don't think any of make it reliable. I don't think any of those things have changed. maybe just those things have changed. maybe just those things have changed. maybe just the more me mechanical parts of the job the more me mechanical parts of the job the more me mechanical parts of the job have definitely been changed or taken have definitely been changed or taken have definitely been changed or taken away by these tools. away by these tools. away by these tools. >> Um so yeah I I actually feel engineers >> Um so yeah I I actually feel engineers >> Um so yeah I I actually feel engineers and I'm one of those who feel that you and I'm one of those who feel that you and I'm one of those who feel that you know who take joy and and um know who take joy and and um know who take joy and and um satisfaction in solving a problem well satisfaction in solving a problem well satisfaction in solving a problem well [clears throat] [clears throat] [clears throat] >> feel even more empowered actually in >> feel even more empowered actually in >> feel even more empowered actually in today's environment. Yeah, I used to today's environment. Yeah, I used to today's environment. Yeah, I used to feel like I was IO bound because my mind feel like I was IO bound because my mind feel like I was IO bound because my mind would have the answer and then I'm just would have the answer and then I'm just would have the answer and then I'm just waiting for the input output part which waiting for the input output part which waiting for the input output part which was my fingers. was my fingers. was my fingers. >> Yeah. >> Yeah. >> Yeah. >> Uh and I'm curious as a senior as a very >> Uh and I'm curious as a senior as a very >> Uh and I'm curious as a senior as a very as a very senior person are you getting as a very senior person are you getting as a very senior person are you getting back into coding even though you're a back into coding even though you're a back into coding even though you're a CTO and doing PRs because I'm seeing CTO and doing PRs because I'm seeing CTO and doing PRs because I'm seeing more and more more and more more and more >> really good CTOs are actually doing PRs >> really good CTOs are actually doing PRs >> really good CTOs are actually doing PRs on their own teams. I am I have checked on their own teams. I am I have checked on their own teams. I am I have checked very few PRs as of now but I have built very few PRs as of now but I have built very few PRs as of now but I have built a lot of prototypes. Ah a lot of prototypes. Ah a lot of prototypes. Ah >> I built a lot of prototypes um just to >> I built a lot of prototypes um just to >> I built a lot of prototypes um just to showcase the concepts that are going showcase the concepts that are going showcase the concepts that are going through my mind and I actually similar through my mind and I actually similar through my mind and I actually similar to you one of the things I to you one of the things I to you one of the things I felt that I had lot of ideas in my head felt that I had lot of ideas in my head felt that I had lot of ideas in my head and I was limited by and I was limited by and I was limited by >> how many of them I could code >> how many of them I could code >> how many of them I could code >> and now I can actually code many of >> and now I can actually code many of >> and now I can actually code many of those ideas build those prototypes show

those ideas build those prototypes show those ideas build those prototypes show show it to my team that this is how at show it to my team that this is how at show it to my team that this is how at least I was thinking of solving this least I was thinking of solving this least I was thinking of solving this problem here does it align with your problem here does it align with your problem here does it align with your worldview and we are able to have a much worldview and we are able to have a much worldview and we are able to have a much more sort of I would say a much more um more sort of I would say a much more um more sort of I would say a much more um deeper conversation deeper conversation deeper conversation >> yeah well I'm also finding us asking >> yeah well I'm also finding us asking >> yeah well I'm also finding us asking ourselves if we can build anything, ourselves if we can build anything, ourselves if we can build anything, >> what is the right thing to build? >> what is the right thing to build? >> what is the right thing to build? >> Cuz I see some organizations just >> Cuz I see some organizations just >> Cuz I see some organizations just throwing spaghetti at the wall, just throwing spaghetti at the wall, just throwing spaghetti at the wall, just coding anything. Uh I would assume in coding anything. Uh I would assume in coding anything. Uh I would assume in your job deciding what the right your job deciding what the right your job deciding what the right direction to go is far more interesting direction to go is far more interesting direction to go is far more interesting than just writing everything. than just writing everything. than just writing everything. >> Exactly. Exactly. That that's exactly >> Exactly. Exactly. That that's exactly >> Exactly. Exactly. That that's exactly how I'm using my time. how I'm using my time. how I'm using my time. >> Yeah. Interesting. So then as a crafts >> Yeah. Interesting. So then as a crafts >> Yeah. Interesting. So then as a crafts person, are you a taste maker? Are you person, are you a taste maker? Are you person, are you a taste maker? Are you helping your teams develop their sense helping your teams develop their sense helping your teams develop their sense of code smell? Uh, and and what do you of code smell? Uh, and and what do you of code smell? Uh, and and what do you feel about junior engineers? Because a feel about junior engineers? Because a feel about junior engineers? Because a lot of juniors just came out of school lot of juniors just came out of school lot of juniors just came out of school right as AI happened and they feel a right as AI happened and they feel a right as AI happened and they feel a little bit like what what was the point little bit like what what was the point little bit like what what was the point of my degree and I'm trying to remind of my degree and I'm trying to remind of my degree and I'm trying to remind them that you learned how to smell and them that you learned how to smell and them that you learned how to smell and how to taste during your degrees. how to taste during your degrees. how to taste during your degrees. >> Yeah. And this is where actually one of >> Yeah. And this is where actually one of >> Yeah. And this is where actually one of the things we did very uh deliberately the things we did very uh deliberately the things we did very uh deliberately and very intently in in our in our and very intently in in our in our and very intently in in our in our company is company is company is >> we basically said we going to hire a >> we basically said we going to hire a >> we basically said we going to hire a combination or we're going to have a combination or we're going to have a combination or we're going to have a combination of senior and junior people combination of senior and junior people combination of senior and junior people in the team. So we were one of those in the team. So we were one of those in the team. So we were one of those companies that actually didn't say that companies that actually didn't say that companies that actually didn't say that you know all the entry- level jobs are you know all the entry- level jobs are you know all the entry- level jobs are gone because gone because gone because who will basically become you know who will basically become you know who will basically become you know senior engineers principal engineers of senior engineers principal engineers of senior engineers principal engineers of tomorrow those that are actually coming tomorrow those that are actually coming tomorrow those that are actually coming out of school and the two like different out of school and the two like different out of school and the two like different dimensions that I see is senior

dimensions that I see is senior dimensions that I see is senior engineers or very senior engineers engineers or very senior engineers engineers or very senior engineers sometimes have a hard time unlearning sometimes have a hard time unlearning sometimes have a hard time unlearning >> but they are the ones who actually have >> but they are the ones who actually have >> but they are the ones who actually have the strong judgment who actually have a the strong judgment who actually have a the strong judgment who actually have a strong taste to evaluate the output of strong taste to evaluate the output of strong taste to evaluate the output of the AI. Junior engineers are all in on the AI. Junior engineers are all in on the AI. Junior engineers are all in on all the tooling, but they don't have the all the tooling, but they don't have the all the tooling, but they don't have the judgment or the smell that you were judgment or the smell that you were judgment or the smell that you were talking about. They don't have those talking about. They don't have those talking about. They don't have those pattern evaluations built in yet because pattern evaluations built in yet because pattern evaluations built in yet because they haven't acquired that experience. they haven't acquired that experience. they haven't acquired that experience. So, we are actually pairing people So, we are actually pairing people So, we are actually pairing people together. Um, I'm not saying that there together. Um, I'm not saying that there together. Um, I'm not saying that there are people who are basically not using are people who are basically not using are people who are basically not using agentic coding tools in our company agentic coding tools in our company agentic coding tools in our company anymore. So there no s that we don't anymore. So there no s that we don't anymore. So there no s that we don't have that anymore in our company but I have that anymore in our company but I have that anymore in our company but I think the that pairing is helping quite think the that pairing is helping quite think the that pairing is helping quite a bit because junior engineers are a bit because junior engineers are a bit because junior engineers are getting to learn how to spot things getting to learn how to spot things getting to learn how to spot things because AI is not the one of the things because AI is not the one of the things because AI is not the one of the things we are bas one of the things I'm we are bas one of the things I'm we are bas one of the things I'm encouraging my team to do is always encouraging my team to do is always encouraging my team to do is always challenge AI don't take what it is challenge AI don't take what it is challenge AI don't take what it is saying as the ground truth so challenge saying as the ground truth so challenge saying as the ground truth so challenge it with your um or counter basically it with your um or counter basically it with your um or counter basically counteract with your uh formulation and counteract with your uh formulation and counteract with your uh formulation and this is where junior engineers can learn this is where junior engineers can learn this is where junior engineers can learn from senior engineers.

from senior engineers. from senior engineers. >> And I'll give a very specific example. >> And I'll give a very specific example. >> And I'll give a very specific example. Um, one of the agentic coding tools um Um, one of the agentic coding tools um Um, one of the agentic coding tools um wrote an SQL query that was highly wrote an SQL query that was highly wrote an SQL query that was highly unoptimized. If I got if it had gone in unoptimized. If I got if it had gone in unoptimized. If I got if it had gone in production, it would probably have production, it would probably have production, it would probably have brought our production system down. And brought our production system down. And brought our production system down. And the engineer reviewing it obviously had the engineer reviewing it obviously had the engineer reviewing it obviously had experience, understood how our experience, understood how our experience, understood how our production works and is like this is not production works and is like this is not production works and is like this is not the right query. This is not how we the right query. This is not how we the right query. This is not how we would have written it. and actually would have written it. and actually would have written it. and actually provided a pretty detailed review to the provided a pretty detailed review to the provided a pretty detailed review to the other engineer who had written or other engineer who had written or other engineer who had written or generated that part of the code. So I generated that part of the code. So I generated that part of the code. So I think that judgment that that smell that think that judgment that that smell that think that judgment that that smell that ability to evaluate is even more ability to evaluate is even more ability to evaluate is even more important today than it was before. important today than it was before. important today than it was before. >> Absolutely. And I think it's important >> Absolutely. And I think it's important >> Absolutely. And I think it's important to note that the early and career people to note that the early and career people to note that the early and career people in my opinion are going to get their in my opinion are going to get their in my opinion are going to get their sense of smell from having those lived sense of smell from having those lived sense of smell from having those lived experiences from messing up production experiences from messing up production experiences from messing up production or in this case almost messing up or in this case almost messing up or in this case almost messing up production. Uh, and otherwise they're production. Uh, and otherwise they're production. Uh, and otherwise they're not going to develop those skills ever. not going to develop those skills ever. not going to develop those skills ever. >> And I I worry a little bit for >> And I I worry a little bit for >> And I I worry a little bit for developers that are only have only ever developers that are only have only ever developers that are only have only ever used AI, have only ever talked to used AI, have only ever talked to used AI, have only ever talked to themselves in the AI mirror. Who is themselves in the AI mirror. Who is themselves in the AI mirror. Who is going to hold their hand and sit with going to hold their hand and sit with going to hold their hand and sit with them? Because we came up in a time where them? Because we came up in a time where them? Because we came up in a time where we were pairing with seniors. Mhm.

we were pairing with seniors. Mhm. we were pairing with seniors. Mhm. >> So, I love to hear that that you're >> So, I love to hear that that you're >> So, I love to hear that that you're actually pairing people up with a formal actually pairing people up with a formal actually pairing people up with a formal investment in juniors cuz it's a rough investment in juniors cuz it's a rough investment in juniors cuz it's a rough job market out there and a lot of job market out there and a lot of job market out there and a lot of juniors are having trouble finding juniors are having trouble finding juniors are having trouble finding finding work. finding work. finding work. >> I I I know that. And by the way, like I >> I I I know that. And by the way, like I >> I I I know that. And by the way, like I am very grateful that our CEO was very am very grateful that our CEO was very am very grateful that our CEO was very aligned with the strategy aligned with the strategy aligned with the strategy >> and this is what I told him that this is >> and this is what I told him that this is >> and this is what I told him that this is why we are doing it and he was totally why we are doing it and he was totally why we are doing it and he was totally aligned with that part. In fact, he's aligned with that part. In fact, he's aligned with that part. In fact, he's written some LinkedIn post about this as written some LinkedIn post about this as written some LinkedIn post about this as well. well. well. >> Oh, I love that. I love that. Hey >> Oh, I love that. I love that. Hey >> Oh, I love that. I love that. Hey friends, it's Scott and I want to give a friends, it's Scott and I want to give a friends, it's Scott and I want to give a shout out to one of our new sponsors, shout out to one of our new sponsors, shout out to one of our new sponsors, SER API. That's SEP API. It's a web SER API. That's SEP API. It's a web SER API. That's SEP API. It's a web search API that gives developers real search API that gives developers real search API that gives developers real time structured access to search engine time structured access to search engine time structured access to search engine results. Works great with AI agents and results. Works great with AI agents and results. Works great with AI agents and AI assistants. I actually use it on my AI assistants. I actually use it on my AI assistants. I actually use it on my own personal OpenClaw instance. You've own personal OpenClaw instance. You've own personal OpenClaw instance. You've got usage for research tools, market got usage for research tools, market got usage for research tools, market intelligent platforms, rag workflows, intelligent platforms, rag workflows, intelligent platforms, rag workflows, anything that needs fresh information anything that needs fresh information anything that needs fresh information from the web. You can get your results from the web. You can get your results from the web. You can get your results back from the web search as clean back from the web search as clean back from the web search as clean structured markdown which actually uses structured markdown which actually uses structured markdown which actually uses about 50% fewer tokens and JSON on about 50% fewer tokens and JSON on about 50% fewer tokens and JSON on average. Very very clean very simple average. Very very clean very simple average. Very very clean very simple integration into your existing AI integration into your existing AI integration into your existing AI workflows. There's actually a 100 plus workflows. There's actually a 100 plus workflows. There's actually a 100 plus SER APIs that are available to you. So SER APIs that are available to you. So SER APIs that are available to you. So instead of building or maintaining instead of building or maintaining instead of building or maintaining scraping infrastructure and trying to scraping infrastructure and trying to scraping infrastructure and trying to scrape the web, SER API handles the scrape the web, SER API handles the scrape the web, SER API handles the complexity, delivers search results as complexity, delivers search results as complexity, delivers search results as structured data that your application structured data that your application structured data that your application can just use. Like I said, there's a 100 can just use. Like I said, there's a 100 can just use. Like I said, there's a 100 plus APIs. You've got access to Google, plus APIs. You've got access to Google, plus APIs. You've got access to Google, to Bing, to Amazon, to Walmart, to to Bing, to Amazon, to Walmart, to to Bing, to Amazon, to Walmart, to YouTube, lots and lots of different YouTube, lots and lots of different YouTube, lots and lots of different sources. They've got enterprisegrade

sources. They've got enterprisegrade sources. They've got enterprisegrade security and compliance. All the security and compliance. All the security and compliance. All the certifications, making it a really certifications, making it a really certifications, making it a really strong option for organizations where strong option for organizations where strong option for organizations where security and compliance matter. Best security and compliance matter. Best security and compliance matter. Best part, you get 250 searches every month part, you get 250 searches every month part, you get 250 searches every month for free. You can check them out at for free. You can check them out at for free. You can check them out at serapi.com/hancel serapi.com/hancel serapi.com/hancel minutes. That's s e rp minutes. That's s e rp minutes. That's s e rp pappi.com/hancelman minutes for 250 free searches a month. minutes for 250 free searches a month. Um, another thing that uh you have said Um, another thing that uh you have said Um, another thing that uh you have said that is a little bit uh provocative is that is a little bit uh provocative is that is a little bit uh provocative is that you think the team should be a that you think the team should be a that you think the team should be a little bit smaller. So you would little bit smaller. So you would little bit smaller. So you would basically shard your teams. So rather basically shard your teams. So rather basically shard your teams. So rather than having a giant team, you'd have a than having a giant team, you'd have a than having a giant team, you'd have a bunch of little teams and then they bunch of little teams and then they bunch of little teams and then they would individually own stuff. What do would individually own stuff. What do would individually own stuff. What do you think the optimal size for an AI you think the optimal size for an AI you think the optimal size for an AI augmented team is? right now augmented team is? right now augmented team is? right now >> I I probably answer the question >> I I probably answer the question >> I I probably answer the question slightly differently slightly differently slightly differently >> and I've been thinking about it quite a >> and I've been thinking about it quite a >> and I've been thinking about it quite a bit because I've worked in larger team bit because I've worked in larger team bit because I've worked in larger team environments as well [clears throat] environments as well [clears throat] environments as well [clears throat] >> and what used to happen earlier was um >> and what used to happen earlier was um >> and what used to happen earlier was um because of again people being the or because of again people being the or because of again people being the or human output being the sort of the long human output being the sort of the long human output being the sort of the long >> pole [clears throat] >> pole [clears throat] >> pole [clears throat] >> the teams were basically especially in >> the teams were basically especially in >> the teams were basically especially in larger companies were organized based on larger companies were organized based on larger companies were organized based on specializations and part of the specializations and part of the specializations and part of the specialization will still remain in my specialization will still remain in my specialization will still remain in my opinion So things like you know uh fault opinion So things like you know uh fault opinion So things like you know uh fault tolerance, reliability, scalability tolerance, reliability, scalability tolerance, reliability, scalability especially because there are deployment especially because there are deployment especially because there are deployment um uh data points that need to come into um uh data points that need to come into um uh data points that need to come into picture. AI will take some quite a bit picture. AI will take some quite a bit picture. AI will take some quite a bit of time to get there in my opinion. But of time to get there in my opinion. But of time to get there in my opinion. But I think what is happening at other I think what is happening at other I think what is happening at other levels is people are able to generalize

levels is people are able to generalize levels is people are able to generalize better like they're able to do more better like they're able to do more better like they're able to do more general work general work general work >> better and as a result you don't need >> better and as a result you don't need >> better and as a result you don't need some of those specialized teams that you some of those specialized teams that you some of those specialized teams that you needed earlier and hence the sizes of needed earlier and hence the sizes of needed earlier and hence the sizes of the teams are the teams are the teams are >> becoming more smaller as a result of >> becoming more smaller as a result of >> becoming more smaller as a result of [clears throat] that and um to some [clears throat] that and um to some [clears throat] that and um to some extent it's also eliminating or reducing extent it's also eliminating or reducing extent it's also eliminating or reducing the comm communication and coordination the comm communication and coordination the comm communication and coordination overhead that naturally exists in a overhead that naturally exists in a overhead that naturally exists in a larger group environment. larger group environment. larger group environment. So I think uh I do think smaller teams So I think uh I do think smaller teams So I think uh I do think smaller teams um done well will have an advantage um done well will have an advantage um done well will have an advantage going forward and it's sort of I'm not going forward and it's sort of I'm not going forward and it's sort of I'm not necessarily saying I align with that necessarily saying I align with that necessarily saying I align with that framing completely but I think Sam framing completely but I think Sam framing completely but I think Sam Ortman had said that you know one person Ortman had said that you know one person Ortman had said that you know one person companies I don't completely subscribe companies I don't completely subscribe companies I don't completely subscribe to that view but um definitely the team to that view but um definitely the team to that view but um definitely the team sizes are shrinking. sizes are shrinking. sizes are shrinking. >> Yeah. Another way to look at it that >> Yeah. Another way to look at it that >> Yeah. Another way to look at it that might be a little bit more positive is might be a little bit more positive is might be a little bit more positive is this idea of collective ownership. Yeah. this idea of collective ownership. Yeah. this idea of collective ownership. Yeah. you know when systems were smaller and you know when systems were smaller and you know when systems were smaller and they weren't big giant distributed cloud they weren't big giant distributed cloud they weren't big giant distributed cloud systems like they are today you know we systems like they are today you know we systems like they are today you know we would talk about the full stack engineer would talk about the full stack engineer would talk about the full stack engineer that was a phrase that was bandied about that was a phrase that was bandied about that was a phrase that was bandied about and then we decided well the stack is and then we decided well the stack is and then we decided well the stack is too deep I can't possibly know the full too deep I can't possibly know the full too deep I can't possibly know the full stack stack stack >> but now I feel like I can and I can have >> but now I feel like I can and I can have >> but now I feel like I can and I can have team ownership and I can also you know I team ownership and I can also you know I team ownership and I can also you know I might have in the past had a a build might have in the past had a a build might have in the past had a a build server engineer or a QA engineer or a server engineer or a QA engineer or a server engineer or a QA engineer or a database engineer now I'm finding that database engineer now I'm finding that database engineer now I'm finding that people are more willing to jump in and people are more willing to jump in and people are more willing to jump in and say I've never done that for I'll give say I've never done that for I'll give say I've never done that for I'll give it I'll give it a try and I'll see if I it I'll give it a try and I'll see if I it I'll give it a try and I'll see if I can take on that PR. It's a little bit can take on that PR. It's a little bit can take on that PR. It's a little bit outside my comfort zone. And I think if outside my comfort zone. And I think if outside my comfort zone. And I think if we have that that um growth mindset of we have that that um growth mindset of we have that that um growth mindset of learning learning learning >> that I could try things that aren't in

>> that I could try things that aren't in >> that I could try things that aren't in my space because the bot will be there my space because the bot will be there my space because the bot will be there to to support me and my team and the to to support me and my team and the to to support me and my team and the mentors and mentees on my team are there mentors and mentees on my team are there mentors and mentees on my team are there to support me. Uh you know a smaller to support me. Uh you know a smaller to support me. Uh you know a smaller team can have deeper collective team can have deeper collective team can have deeper collective ownership. ownership. ownership. >> Yeah. Yeah. And and we are seeing that >> Yeah. Yeah. And and we are seeing that >> Yeah. Yeah. And and we are seeing that in our environment as well. Do you in our environment as well. Do you in our environment as well. Do you change the how do you take an change the how do you take an change the how do you take an organization because we're not talking organization because we're not talking organization because we're not talking about necessarily downsizing an about necessarily downsizing an about necessarily downsizing an organization as we're talking about organization as we're talking about organization as we're talking about changing the shape of it because you changing the shape of it because you changing the shape of it because you might have hundreds of engineers and now might have hundreds of engineers and now might have hundreds of engineers and now you're going to announce that you're a you're going to announce that you're a you're going to announce that you're a collection of five person startups. collection of five person startups. collection of five person startups. Does that structurally change things? Does that structurally change things? Does that structurally change things? And I'm curious also as a second And I'm curious also as a second And I'm curious also as a second question is there are you changing question is there are you changing question is there are you changing titles? We're seeing more and more titles? We're seeing more and more titles? We're seeing more and more members of technical staff which is a members of technical staff which is a members of technical staff which is a way of just saying Jack and Jill of all way of just saying Jack and Jill of all way of just saying Jack and Jill of all trades. trades. trades. >> Yeah. So, so two-part answer to that >> Yeah. So, so two-part answer to that >> Yeah. So, so two-part answer to that question. One, we've been around for question. One, we've been around for question. One, we've been around for approximately I think 10 10 plus years approximately I think 10 10 plus years approximately I think 10 10 plus years now. now. now. >> So, there is a existing culture um parts >> So, there is a existing culture um parts >> So, there is a existing culture um parts of which are pretty good and parts of of which are pretty good and parts of of which are pretty good and parts of which are basically you know can do with which are basically you know can do with which are basically you know can do with some change and we've done that part of some change and we've done that part of some change and we've done that part of the change but I think we do want to the change but I think we do want to the change but I think we do want to keep the good parts of the culture and keep the good parts of the culture and keep the good parts of the culture and good parts of the culture around good parts of the culture around good parts of the culture around collaboration, knowledge sharing and so collaboration, knowledge sharing and so collaboration, knowledge sharing and so on. Um part of it is we have already on. Um part of it is we have already on. Um part of it is we have already done changes around how teams are done changes around how teams are done changes around how teams are organized, how big or small teams are.

organized, how big or small teams are. organized, how big or small teams are. Um I would say it wasn't exactly easy. Um I would say it wasn't exactly easy. Um I would say it wasn't exactly easy. It takes time for and it was more about It takes time for and it was more about It takes time for and it was more about shifting the mental models than any kind shifting the mental models than any kind shifting the mental models than any kind of let's say any any other any other of let's say any any other any other of let's say any any other any other change. But I think once those mental change. But I think once those mental change. But I think once those mental models have shifted, once people have models have shifted, once people have models have shifted, once people have realized how working with this realized how working with this realized how working with this technology can supercharge the outcomes technology can supercharge the outcomes technology can supercharge the outcomes that they are able to do, that they are able to do, that they are able to do, >> I think the shift has been shift has >> I think the shift has been shift has >> I think the shift has been shift has been good been good been good >> and it took some time but uh we are on >> and it took some time but uh we are on >> and it took some time but uh we are on the other side of it now. the other side of it now. the other side of it now. >> Now I'm I'm 99% sure that you're not >> Now I'm I'm 99% sure that you're not >> Now I'm I'm 99% sure that you're not going to tell me that productivity and going to tell me that productivity and going to tell me that productivity and AI is measured in lines of code. AI is measured in lines of code. AI is measured in lines of code. >> No, I was never a subscriber to that. AI >> No, I was never a subscriber to that. AI >> No, I was never a subscriber to that. AI I never subscribed to that. I never subscribed to that. I never subscribed to that. >> Excellent. So we agree that PRs and >> Excellent. So we agree that PRs and >> Excellent. So we agree that PRs and lines of code and all those things can lines of code and all those things can lines of code and all those things can be gamed. What is the the efficiency be gamed. What is the the efficiency be gamed. What is the the efficiency measurement? What is the ROI? Is it measurement? What is the ROI? Is it measurement? What is the ROI? Is it customer outcomes? Is it incidents customer outcomes? Is it incidents customer outcomes? Is it incidents closed? Is it time to learning? closed? Is it time to learning? closed? Is it time to learning? >> Yeah. First of all, I I actually don't >> Yeah. First of all, I I actually don't >> Yeah. First of all, I I actually don't again I I also don't subscribe to the again I I also don't subscribe to the again I I also don't subscribe to the efficiency narrative because this is my efficiency narrative because this is my efficiency narrative because this is my framing which is efficiency narratives framing which is efficiency narratives framing which is efficiency narratives you can only take them down up to a you can only take them down up to a you can only take them down up to a certain point because end state is zero.

certain point because end state is zero. certain point because end state is zero. There's no waste. Basically zero waste. There's no waste. Basically zero waste. There's no waste. Basically zero waste. >> Exactly. So I don't think of efficiency. >> Exactly. So I don't think of efficiency. >> Exactly. So I don't think of efficiency. I think again going back to your framing I think again going back to your framing I think again going back to your framing about growth mindset. I think in terms about growth mindset. I think in terms about growth mindset. I think in terms of growth basically and growth can be an of growth basically and growth can be an of growth basically and growth can be an infinite basically if you want it can be infinite basically if you want it can be infinite basically if you want it can be an infinite play. So that's how I we an infinite play. So that's how I we an infinite play. So that's how I we think about it think about it think about it >> and um again I also never measured a >> and um again I also never measured a >> and um again I also never measured a productivity in lines of code. I think productivity in lines of code. I think productivity in lines of code. I think what we are looking at is what outcomes what we are looking at is what outcomes what we are looking at is what outcomes are we able to deliver in a given time are we able to deliver in a given time are we able to deliver in a given time frame with the size of the team that we frame with the size of the team that we frame with the size of the team that we have have have >> and is that becoming better or not while >> and is that becoming better or not while >> and is that becoming better or not while keeping in mind the quality of the keeping in mind the quality of the keeping in mind the quality of the outcome as well because that's a tension outcome as well because that's a tension outcome as well because that's a tension side of it. You can produce all the code side of it. You can produce all the code side of it. You can produce all the code you want. Uh there's no restriction on you want. Uh there's no restriction on you want. Uh there's no restriction on that but quality of the outcome and that but quality of the outcome and that but quality of the outcome and whether it actually solves a problem for whether it actually solves a problem for whether it actually solves a problem for our customers is an important benchmark. our customers is an important benchmark. our customers is an important benchmark. So we are looking at both uh outcomes as So we are looking at both uh outcomes as So we are looking at both uh outcomes as well as how much customer value are we well as how much customer value are we well as how much customer value are we able to deliver with a given team in a able to deliver with a given team in a able to deliver with a given team in a given time. given time. given time. >> Yeah. See that that is a humanistic >> Yeah. See that that is a humanistic >> Yeah. See that that is a humanistic perspective that is a you know did we perspective that is a you know did we perspective that is a you know did we make someone's life better? Did the make someone's life better? Did the make someone's life better? Did the product did the user story ship and did product did the user story ship and did product did the user story ship and did it ship with quality and did it ship it ship with quality and did it ship it ship with quality and did it ship well? You know what that's that's the well? You know what that's that's the well? You know what that's that's the that's the correct artifact not metrics that's the correct artifact not metrics that's the correct artifact not metrics and token token maxing. It's all that's and token token maxing. It's all that's and token token maxing. It's all that's nonsense.

nonsense. nonsense. >> I was never into token maxing [laughter] >> I was never into token maxing [laughter] >> I was never into token maxing [laughter] as well. as well. as well. Yeah. So, yeah, I appreciate that Yeah. So, yeah, I appreciate that Yeah. So, yeah, I appreciate that there's a productivity measurement trap there's a productivity measurement trap there's a productivity measurement trap also because trying to drive it down to also because trying to drive it down to also because trying to drive it down to zero just doesn't it's not it's not the zero just doesn't it's not it's not the zero just doesn't it's not it's not the thing. And you mentioned before it's thing. And you mentioned before it's thing. And you mentioned before it's human attention ultimately is now the human attention ultimately is now the human attention ultimately is now the thing that is the most valuable. I'm thing that is the most valuable. I'm thing that is the most valuable. I'm making code faster than I can review it. making code faster than I can review it. making code faster than I can review it. And it's easy to get into the trap of And it's easy to get into the trap of And it's easy to get into the trap of just saying, "Oh, it's fine. Merge." just saying, "Oh, it's fine. Merge." just saying, "Oh, it's fine. Merge." >> Yeah. I'm curious if you've had any >> Yeah. I'm curious if you've had any >> Yeah. I'm curious if you've had any challenges with that where people uh challenges with that where people uh challenges with that where people uh they I think someone called them meat they I think someone called them meat they I think someone called them meat proxies where I am proxying between the proxies where I am proxying between the proxies where I am proxying between the AI. So you you're my boss, you ask me a AI. So you you're my boss, you ask me a AI. So you you're my boss, you ask me a question, I ask the AI and I tell you question, I ask the AI and I tell you question, I ask the AI and I tell you the answer. What am I for? the answer. What am I for? the answer. What am I for? >> I I actually have a joke maybe if you >> I I actually have a joke maybe if you >> I I actually have a joke maybe if you have time I can tell you about that. But have time I can tell you about that. But have time I can tell you about that. But um maybe just going back to your um maybe just going back to your um maybe just going back to your question um one of the things we had one question um one of the things we had one question um one of the things we had one of the approach we had taken right from of the approach we had taken right from of the approach we had taken right from the beginning is when we introduce AI in the beginning is when we introduce AI in the beginning is when we introduce AI in our software development process we'll our software development process we'll our software development process we'll introduce it across the board as in introduce it across the board as in introduce it across the board as in across all steps of the software across all steps of the software across all steps of the software development life cycle not just development life cycle not just development life cycle not just generation not just review but also in generation not just review but also in generation not just review but also in other parts downstream as well because other parts downstream as well because other parts downstream as well because one of the things I was very clear is one of the things I was very clear is one of the things I was very clear is you make one part faster you're going to you make one part faster you're going to you make one part faster you're going to basically create bottlenecks in other basically create bottlenecks in other basically create bottlenecks in other areas and that's going to happen because areas and that's going to happen because areas and that's going to happen because you have this bunch of machines you have this bunch of machines you have this bunch of machines producing code on one side and if you producing code on one side and if you producing code on one side and if you have humans no matter how many humans have humans no matter how many humans have humans no matter how many humans you put you're not you can't outrun the you put you're not you can't outrun the you put you're not you can't outrun the machine basically in some sense machine basically in some sense machine basically in some sense [clears throat] [clears throat] [clears throat] >> but I have to say despite all our best >> but I have to say despite all our best >> but I have to say despite all our best intentions and again this is where the intentions and again this is where the intentions and again this is where the mental model ship takes a little bit of mental model ship takes a little bit of mental model ship takes a little bit of time there are certain parts of the code time there are certain parts of the code time there are certain parts of the code that still get reviewed manually and and

that still get reviewed manually and and that still get reviewed manually and and even that is proving to be overwhelming even that is proving to be overwhelming even that is proving to be overwhelming sometimes for the team sometimes for the team sometimes for the team >> and um we are it's a sort of work in >> and um we are it's a sort of work in >> and um we are it's a sort of work in progress on our side we are exploring progress on our side we are exploring progress on our side we are exploring other ways in which um code review needs other ways in which um code review needs other ways in which um code review needs to get done to get done to get done >> and one of the approach we've been >> and one of the approach we've been >> and one of the approach we've been thinking about is hey can we take a risk thinking about is hey can we take a risk thinking about is hey can we take a risk based approach to code review that based approach to code review that based approach to code review that riskier parts get more eyes on them and riskier parts get more eyes on them and riskier parts get more eyes on them and more human eyes on them while the more more human eyes on them while the more more human eyes on them while the more standard parts of the code can actually standard parts of the code can actually standard parts of the code can actually be reviewed by another AI and just go be reviewed by another AI and just go be reviewed by another AI and just go through and one of the things we are through and one of the things we are through and one of the things we are very careful is we want the agent doing very careful is we want the agent doing very careful is we want the agent doing the review which has to be different the review which has to be different the review which has to be different model from the agent producing the code model from the agent producing the code model from the agent producing the code because otherwise they'll introduce the because otherwise they'll introduce the because otherwise they'll introduce the same biases on both sides. Yeah, we've same biases on both sides. Yeah, we've same biases on both sides. Yeah, we've been doing a lot of multimodel been doing a lot of multimodel been doing a lot of multimodel adversarial reviews. I'll even take a adversarial reviews. I'll even take a adversarial reviews. I'll even take a model from three of the major models and model from three of the major models and model from three of the major models and go across Claude, GPT, and Gemini and go across Claude, GPT, and Gemini and go across Claude, GPT, and Gemini and have them all vote have them all vote have them all vote >> on on on things to try to avoid any >> on on on things to try to avoid any >> on on on things to try to avoid any biases because we're finding that in my biases because we're finding that in my biases because we're finding that in my opinion uh in all clawed shops there's a opinion uh in all clawed shops there's a opinion uh in all clawed shops there's a there's a bias and it will have um have there's a bias and it will have um have there's a bias and it will have um have uh what's the word? Blind spots.

uh what's the word? Blind spots. uh what's the word? Blind spots. >> Yeah, we do that on the eval side. So >> Yeah, we do that on the eval side. So >> Yeah, we do that on the eval side. So quality emails, this is how they are quality emails, this is how they are quality emails, this is how they are done. done. done. >> Oh, excellent. Excellent. Um, you you've >> Oh, excellent. Excellent. Um, you you've >> Oh, excellent. Excellent. Um, you you've also acknowledged like burnout. I feel a also acknowledged like burnout. I feel a also acknowledged like burnout. I feel a little bit like last year everyone told little bit like last year everyone told little bit like last year everyone told me that AI was going to like make me 20% me that AI was going to like make me 20% me that AI was going to like make me 20% more productive, and I assume that means more productive, and I assume that means more productive, and I assume that means I would get Friday off work, but I'm I would get Friday off work, but I'm I would get Friday off work, but I'm assuming that you're working more than assuming that you're working more than assuming that you're working more than ever. So, how do you, you know, AI is ever. So, how do you, you know, AI is ever. So, how do you, you know, AI is supposed to remove toil, but now I can supposed to remove toil, but now I can supposed to remove toil, but now I can generate 10 things and they all need generate 10 things and they all need generate 10 things and they all need reviewing. reviewing. reviewing. >> Yeah. >> Yeah. >> Yeah. >> How do we balance that out? Like I know >> How do we balance that out? Like I know >> How do we balance that out? Like I know there's there's underlying capitalism there's there's underlying capitalism there's there's underlying capitalism and growth growth growth, but how do you and growth growth growth, but how do you and growth growth growth, but how do you grow your brain and become smarter and grow your brain and become smarter and grow your brain and become smarter and not burn out? Because AI was supposed to not burn out? Because AI was supposed to not burn out? Because AI was supposed to not burn us out and everyone's feeling not burn us out and everyone's feeling not burn us out and everyone's feeling pretty burned out right now. So the the pretty burned out right now. So the the pretty burned out right now. So the the way at least um I want us to think about way at least um I want us to think about way at least um I want us to think about is there is rate of production of is there is rate of production of is there is rate of production of technology and there is a rate of technology and there is a rate of technology and there is a rate of diffusion of that technology in our diffusion of that technology in our diffusion of that technology in our actual workflows work streams across the actual workflows work streams across the actual workflows work streams across the companies where we sell and so on companies where we sell and so on companies where we sell and so on >> and there's just lot of existing um say >> and there's just lot of existing um say >> and there's just lot of existing um say systems and work that needs to be taken systems and work that needs to be taken systems and work that needs to be taken into account and I think like I'm again into account and I think like I'm again into account and I think like I'm again I'm also not in one of the persons who I'm also not in one of the persons who I'm also not in one of the persons who say you know AI will solve all problems say you know AI will solve all problems say you know AI will solve all problems We still have to get involved. We still We still have to get involved. We still We still have to get involved. We still have to use our creativity and judgment.

have to use our creativity and judgment. have to use our creativity and judgment. And I think the rate of diffusion is And I think the rate of diffusion is And I think the rate of diffusion is slower. And we have I think eventually slower. And we have I think eventually slower. And we have I think eventually the pace of technology will have to the pace of technology will have to the pace of technology will have to match the pace of diffusion in my match the pace of diffusion in my match the pace of diffusion in my opinion. opinion. opinion. >> And it's a little bit uh maybe it's a >> And it's a little bit uh maybe it's a >> And it's a little bit uh maybe it's a little controversial for me to say that little controversial for me to say that little controversial for me to say that but I think it's a bit premature to say but I think it's a bit premature to say but I think it's a bit premature to say that you know the rate at which the that you know the rate at which the that you know the rate at which the technology is getting produced will technology is getting produced will technology is getting produced will basically you you'll see the outcome at basically you you'll see the outcome at basically you you'll see the outcome at on the other side from at the same rate. on the other side from at the same rate. on the other side from at the same rate. We are not seeing outcomes at the same We are not seeing outcomes at the same We are not seeing outcomes at the same rate on the other side. And there are rate on the other side. And there are rate on the other side. And there are plenty of studies that basically show plenty of studies that basically show plenty of studies that basically show whether for whatever reasons, whether for whatever reasons, whether for whatever reasons, organizational inertia, diffusion, organizational inertia, diffusion, organizational inertia, diffusion, mental model, like the outcomes are not mental model, like the outcomes are not mental model, like the outcomes are not commensurate with the rate at which the commensurate with the rate at which the commensurate with the rate at which the technology is getting produced. technology is getting produced. technology is getting produced. >> Yeah. And and I think and the simple >> Yeah. And and I think and the simple >> Yeah. And and I think and the simple acknowledgement of that has a lot of acknowledgement of that has a lot of acknowledgement of that has a lot of value. saying when leadership says the value. saying when leadership says the value. saying when leadership says the quiet part out loud and tells people quiet part out loud and tells people quiet part out loud and tells people that it's okay to feel a certain way and that it's okay to feel a certain way and that it's okay to feel a certain way and it's okay to like let's just focus on it's okay to like let's just focus on it's okay to like let's just focus on did we did the customer get what they did we did the customer get what they did we did the customer get what they needed that that makes for a healthier needed that that makes for a healthier needed that that makes for a healthier organization I think organization I think organization I think >> I agree I agree and it's a customer >> I agree I agree and it's a customer >> I agree I agree and it's a customer outcomes that we want to continue to outcomes that we want to continue to outcomes that we want to continue to focus on focus on focus on >> yeah exactly so there are there are >> yeah exactly so there are there are >> yeah exactly so there are there are companies out there who are not doing companies out there who are not doing companies out there who are not doing anything with AI just as there are anything with AI just as there are anything with AI just as there are companies that have an immature SDLC I companies that have an immature SDLC I companies that have an immature SDLC I know companies that don't even have a know companies that don't even have a know companies that don't even have a formal formal formal CI/CD. Yeah. And like it's 2026 and CI/CD. Yeah. And like it's 2026 and CI/CD. Yeah. And like it's 2026 and those people should be very concerned.

those people should be very concerned. those people should be very concerned. >> If you were to drop into a standard >> If you were to drop into a standard >> If you were to drop into a standard software engineering organization like software engineering organization like software engineering organization like today, like you're the new CTO, brand today, like you're the new CTO, brand today, like you're the new CTO, brand new, hello, here's what do we what do we new, hello, here's what do we what do we new, hello, here's what do we what do we change first? How do we introduce this change first? How do we introduce this change first? How do we introduce this new moment into people who are years new moment into people who are years new moment into people who are years behind? So one of the beliefs I also behind? So one of the beliefs I also behind? So one of the beliefs I also have is um I think um it's actually two have is um I think um it's actually two have is um I think um it's actually two things I would say and this is what we things I would say and this is what we things I would say and this is what we had done um at ironcloud. ironclad was had done um at ironcloud. ironclad was had done um at ironcloud. ironclad was using the the sort of like the earlier using the the sort of like the earlier using the the sort of like the earlier versions of AI not the generative AI versions of AI not the generative AI versions of AI not the generative AI when I landed when I landed when I landed >> right >> right >> right >> and one of the things we basically and >> and one of the things we basically and >> and one of the things we basically and this was a deliberate choice again our this was a deliberate choice again our this was a deliberate choice again our CT our CEO was very aligned on this CT our CEO was very aligned on this CT our CEO was very aligned on this >> is we will actually make it easy for >> is we will actually make it easy for >> is we will actually make it easy for people or we'll make it a choice for people or we'll make it a choice for people or we'll make it a choice for people to adopt the technology more as a people to adopt the technology more as a people to adopt the technology more as a learning and growth play we basically learning and growth play we basically learning and growth play we basically say this is a learning opportunity say this is a learning opportunity say this is a learning opportunity >> and by the way one of the things I think >> and by the way one of the things I think >> and by the way one of the things I think you know it and I know in the in the in you know it and I know in the in the in you know it and I know in the in the in the technology space the thing that the technology space the thing that the technology space the thing that works best in technology space is works best in technology space is works best in technology space is adaptability you adapt to basically the adaptability you adapt to basically the adaptability you adapt to basically the new technological moments that come in new technological moments that come in new technological moments that come in otherwise you are obsolete basically at otherwise you are obsolete basically at otherwise you are obsolete basically at that point that point that point >> so I think one of the things we >> so I think one of the things we >> so I think one of the things we basically said is it's a growth basically said is it's a growth basically said is it's a growth opportunity the company is standing opportunity the company is standing opportunity the company is standing behind the learning that we are behind the learning that we are behind the learning that we are providing and that learning will help providing and that learning will help providing and that learning will help you do your jobs better and you do your jobs better and you do your jobs better and >> so we basically ran actually a 20-day >> so we basically ran actually a 20-day >> so we basically ran actually a 20-day program last year we I taught classes is program last year we I taught classes is program last year we I taught classes is in that program because I had worked in in that program because I had worked in in that program because I had worked in deeply in the technology helping people

deeply in the technology helping people deeply in the technology helping people understand how to contextualize it, how understand how to contextualize it, how understand how to contextualize it, how to contextualize it using the technology to contextualize it using the technology to contextualize it using the technology in the day-to-day work as well. And um in the day-to-day work as well. And um in the day-to-day work as well. And um we basically ran the 20-day program we basically ran the 20-day program we basically ran the 20-day program ahead of a hackathon that we wanted to ahead of a hackathon that we wanted to ahead of a hackathon that we wanted to run and fast forward to almost whatever run and fast forward to almost whatever run and fast forward to almost whatever 10 months from that um we are in a very 10 months from that um we are in a very 10 months from that um we are in a very good place as a company in terms of good place as a company in terms of good place as a company in terms of adoption. adoption has been pretty adoption. adoption has been pretty adoption. adoption has been pretty voluntary and this is the the outcome voluntary and this is the the outcome voluntary and this is the the outcome measurement part that I talked about. I measurement part that I talked about. I measurement part that I talked about. I got the same questions from the team. I got the same questions from the team. I got the same questions from the team. I said are you going to measure my lines said are you going to measure my lines said are you going to measure my lines of code even though I'm not writing the of code even though I'm not writing the of code even though I'm not writing the code and I'm like I have never been a code and I'm like I have never been a code and I'm like I have never been a proponent of lines of code. proponent of lines of code. proponent of lines of code. >> Yeah. Yeah. >> Yeah. Yeah. >> Yeah. Yeah. >> So I think maybe TLDDR of your answer to >> So I think maybe TLDDR of your answer to >> So I think maybe TLDDR of your answer to your question is I think shifting the your question is I think shifting the your question is I think shifting the mental model is the thing I would start mental model is the thing I would start mental model is the thing I would start with with with >> even though it may feel like it's a >> even though it may feel like it's a >> even though it may feel like it's a slower approach and it the returns come slower approach and it the returns come slower approach and it the returns come slightly later in the cycle versus um a slightly later in the cycle versus um a slightly later in the cycle versus um a more approach of hey starting today we more approach of hey starting today we more approach of hey starting today we are all basically shifting to AI um are all basically shifting to AI um are all basically shifting to AI um because I don't think the mental model because I don't think the mental model because I don't think the mental model in a company like that the way you are in a company like that the way you are in a company like that the way you are talking about would be set to accept it talking about would be set to accept it talking about would be set to accept it in in a way in a way once you've in in a way in a way once you've in in a way in a way once you've basically run the education programs or basically run the education programs or basically run the education programs or some kind of learning program.

some kind of learning program. some kind of learning program. >> Yeah, I'm really glad to hear that >> Yeah, I'm really glad to hear that >> Yeah, I'm really glad to hear that you're so focused on education because I you're so focused on education because I you're so focused on education because I know in 5 years or whenever I retire, know in 5 years or whenever I retire, know in 5 years or whenever I retire, [snorts] [snorts] [snorts] the early in career people are going to the early in career people are going to the early in career people are going to be 5 years into their career or they're be 5 years into their career or they're be 5 years into their career or they're going to be 10 years in and they're now going to be 10 years in and they're now going to be 10 years in and they're now seniors and they're going to be in a seniors and they're going to be in a seniors and they're going to be in a very very different world than we are. very very different world than we are. very very different world than we are. And I hope that these systems that we And I hope that these systems that we And I hope that these systems that we leave them are not fragile and the leave them are not fragile and the leave them are not fragile and the organizations that we leave them are organizations that we leave them are organizations that we leave them are resilient and we do focus on artificial resilient and we do focus on artificial resilient and we do focus on artificial intelligence is not intelligent and it intelligence is not intelligent and it intelligence is not intelligent and it is artificial and it is here to serve us is artificial and it is here to serve us is artificial and it is here to serve us and make us smarter. So I hope that the and make us smarter. So I hope that the and make us smarter. So I hope that the orgs that we leave are going to be orgs that we leave are going to be orgs that we leave are going to be smarter people. smarter people. smarter people. >> I'm I'm with you Scott on that and that >> I'm I'm with you Scott on that and that >> I'm I'm with you Scott on that and that is one of the reasons we said we want to is one of the reasons we said we want to is one of the reasons we said we want to have a mix of people in the company. have a mix of people in the company. have a mix of people in the company. people who will become you know people who will become you know people who will become you know >> seniors and principles and you know >> seniors and principles and you know >> seniors and principles and you know leaders of tomorrow. leaders of tomorrow. leaders of tomorrow. >> Yeah. Well, I appreciate you so much and >> Yeah. Well, I appreciate you so much and >> Yeah. Well, I appreciate you so much and thank you for spending time with me thank you for spending time with me thank you for spending time with me today. today. today. >> No, thank you. It was thank you for all >> No, thank you. It was thank you for all >> No, thank you. It was thank you for all the insightful questions and it was a the insightful questions and it was a the insightful questions and it was a lot of fun. lot of fun. lot of fun. >> We've been chatting with Sunnita Verma. >> We've been chatting with Sunnita Verma. >> We've been chatting with Sunnita Verma. She's a CTO at Ironclad and this has She's a CTO at Ironclad and this has She's a CTO at Ironclad and this has been another episode of Hansel Minutes been another episode of Hansel Minutes been another episode of Hansel Minutes and we'll see you again next week.

and we'll see you again next week. and we'll see you again next week. [music]