Your AI Isn’t the Problem. Your Infrastructure Is. — Jiachen Jiang, Coder

Thank you to everyone who Thank you to everyone who joined. My joined. My joined. My name is Shao-Chun Jiang, and I am the name is Shao-Chun Jiang, and I am the name is Shao-Chun Jiang, and I am the Lead Lead Lead Product Manager at Product Manager at Product Manager at Coder. We Coder. We Coder. We are working on are working on are working on open open development environments with self- self- hosting capabilities for hosting capabilities for hosting capabilities for agent and agent and agent and human collaboration. Many of our human collaboration. Many of our human collaboration. Many of our clients work in clients work in clients work in highly regulated highly regulated highly regulated industries: from industries: from industries: from government agencies to government agencies to government agencies to banks and banks and banks and investment funds investment funds . My job is to . My job is to . My job is to create an create an create an AI governance system that AI governance system that AI governance system that allows them allows them allows them to reap the benefits to reap the benefits to reap the benefits of AI without of AI without of AI without opening the way for opening the way for opening the way for security threats. security threats. Today I'm here to Today I'm here to tell you tell you tell you some scary some scary some scary stories. Three real stories. Three real stories. Three real cases of unsuccessful cases of unsuccessful cases of unsuccessful use of AI that use of AI that use of AI that occurred in real occurred in real occurred in real companies over the companies over the companies over the past year. Some of past year. Some of past year. Some of these stories are these stories are these stories are public, others are public, others are anonymous incidents that anonymous incidents that anonymous incidents that happened to our happened to our happened to our clients. For clients. For clients. For each incident, I each incident, I each incident, I will explain step by step will explain step by step how exactly it happened. how exactly it happened.

No technical No technical jargon. Then I'll jargon. Then I'll jargon. Then I'll get to the get to the get to the most important most important most important part. I will tell part. I will tell part. I will tell you exactly how you can you exactly how you can you exactly how you can prevent such prevent such prevent such horror stories from happening in horror stories from happening in horror stories from happening in your company. your company. The good news is that The good news is that everything I'm everything I'm everything I'm about to show about to show about to show could have been avoided. could have been avoided. Not by paying Not by paying five times more per five times more per five times more per token for a smarter token for a smarter token for a smarter model that model that model that avoids errors a little better, avoids errors a little better, avoids errors a little better, but through but through but through infrastructural infrastructural infrastructural control. Many of control. Many of control. Many of these tools these tools these tools you already have, you just have you already have, you just have you already have, you just have n't had time n't had time n't had time to customize them for the to customize them for the to customize them for the AI ​​world yet. Let me AI ​​world yet. Let me AI ​​world yet. Let me outline the situation. outline the situation. outline the situation. You come to You come to You come to work in the morning. You work in the morning. You work in the morning. You ask your coding agent ask your coding agent to review your to review your assignments. But don't assignments. But don't assignments. But don't do anything yet. You do anything yet. You do anything yet. You are telling the agent are telling the agent are telling the agent to consider this command an to consider this command an to consider this command an active active active code freeze. code freeze. code freeze. I just want you to I just want you to I just want you to investigate the problem. investigate the problem. investigate the problem. Then you leave. Then you leave. Then you leave. You are going to have coffee. You are going to have coffee. You are going to have coffee. And when you And when you And when you return, you return, you return, you will find this will find this will find this message waiting for you. So, I message waiting for you. So, I message waiting for you. So, I know you asked me know you asked me know you asked me not to change anything, not to change anything, not to change anything, but, well, I destroyed a but, well, I destroyed a but, well, I destroyed a month of your work month of your work month of your work in seconds. What you in seconds. What you in seconds. What you see on the slide is a see on the slide is a see on the slide is a direct quote from direct quote from direct quote from the agent who is the the agent who is the the agent who is the main character of the main character of the main character of the first scary first scary first scary story. And I want story. And I want story. And I want to emphasize something to emphasize something to emphasize something important. This did important. This did important. This did not happen because the agent was not happen because the agent was not happen because the agent was captured by some captured by some captured by some external attacker external attacker . This agent . This agent . This agent was simply trying to follow was simply trying to follow was simply trying to follow instructions. He instructions. He instructions. He did exactly what the

did exactly what the did exactly what the user gave user gave user gave him permission to do. So as him permission to do. So as him permission to do. So as we continue, I we continue, I we continue, I want you to ask want you to ask want you to ask yourself one yourself one yourself one question. What is preventing question. What is preventing question. What is preventing this from happening to me this from happening to me ? Are these actions that ? Are these actions that ? Are these actions that my agents are authorized to take my agents are authorized to take my agents are authorized to take ? The thing is, ? The thing is, ? The thing is, you can't you can't you can't rely on Anthropic rely on Anthropic rely on Anthropic or OpenAI to stop or OpenAI to stop or OpenAI to stop something like this. They something like this. They something like this. They can create can create can create smarter models, but smarter models, but smarter models, but ultimately large language ultimately large language ultimately large language models have two models have two models have two qualities that make them qualities that make them qualities that make them impossible to fully impossible to fully impossible to fully trust. They are trust. They are trust. They are non-deterministic, which non-deterministic, which non-deterministic, which means that there means that there means that there is always some is always some is always some randomness or randomness or randomness or unpredictability in their actions. In unpredictability in their actions. In unpredictability in their actions. In other words, you other words, you other words, you can ask it can ask it can ask it to do the same to do the same to do the same thing 10 times and the model thing 10 times and the model thing 10 times and the model can do it 10 can do it 10 can do it 10 different ways. different ways. different ways. The second feature The second feature The second feature is that is that is that they have no they have no they have no structural structural structural distinction between distinction between distinction between instructions and instructions and instructions and context. In other context. In other context. In other words, you can words, you can words, you can tell the model to tell the model to tell the model to do something and provide do something and provide do something and provide context for context for context for decision-making, but decision-making, but decision-making, but if there if there if there is something in that context that is something in that context that is something in that context that contradicts your contradicts your contradicts your instructions, the model will instructions, the model will instructions, the model will not not not prioritize them because prioritize them because prioritize them because from its perspective, all from its perspective, all from its perspective, all information comes information comes information comes through the same through the same through the same door. What you door. What you door. What you need to need to need to control is control is control is the infrastructure on the infrastructure on the infrastructure on which your AI runs, which your AI runs, which your AI runs, that is, the environment in that is, the environment in that is, the environment in which it functions.

which it functions. It's like the difference between It's like the difference between telling someone not to telling someone not to telling someone not to leave a room and leave a room and locking the door. The good thing is locking the door. The good thing is that the problem is in that the problem is in that the problem is in the infrastructure, not the the infrastructure, not the the infrastructure, not the model itself, model itself, model itself, so so so you have much more you have much more you have much more control to control to control to prevent prevent prevent unintended consequences. unintended consequences. So I'm not here to So I'm not here to waste your time waste your time waste your time on vague hypotheses on vague hypotheses on vague hypotheses or advice that or advice that or advice that can't be can't be can't be implemented, like, " implemented, like, " Just put the Just put the Just put the genie back in the bottle genie back in the bottle genie back in the bottle or or or stop stop stop using AI altogether." Because that's using AI altogether." Because that's using AI altogether." Because that's not the world not the world not the world we live in, is we live in, is we live in, is it? Jin is already free. it? Jin is already free. And all we have to And all we have to do is do is minimize the risks minimize the risks and harm as much as possible. This conversation is and harm as much as possible. This conversation is and harm as much as possible. This conversation is about how to do it. about how to do it. Not about what to Not about what to consider in consider in consider in the future, but about the future, but about the future, but about steps you can steps you can steps you can take now, take now, take now, even this week, even this week, even this week, to protect yourself and to protect yourself and to protect yourself and your company. Before we your company. Before we your company. Before we get into get into get into the stories, let the stories, let the stories, let me provide a little me provide a little me provide a little more context. The " more context. The " lethal triad" is a lethal triad" is a lethal triad" is a security concept. It was security concept. It was security concept. It was coined by independent coined by independent coined by independent AI researcher Simon AI researcher Simon AI researcher Simon Willison to describe a Willison to describe a Willison to describe a critical vulnerability critical vulnerability that arises in that arises in that arises in AI agents when they AI agents when they AI agents when they gain simultaneous gain simultaneous gain simultaneous access to three access to three access to three different capabilities.

different capabilities. First, it is access to First, it is access to confidential confidential confidential private data. This private data. This private data. This could be medical could be medical could be medical records or records or records or information that could be information that could be information that could be used for used for used for insider insider insider trading. Or it trading. Or it trading. Or it could be could be could be authentication tokens, authentication tokens, authentication tokens, credentials, credentials, credentials, secret keys. secret keys. Secondly, it is access to Secondly, it is access to Secondly, it is access to unreliable unreliable unreliable content or the content or the content or the ability of AI to read ability of AI to read ability of AI to read something it did not something it did not something it did not create on its own create on its own . The thing is, AI is very . The thing is, AI is very . The thing is, AI is very good at following good at following good at following instructions, but not instructions, but not instructions, but not very good at distinguishing very good at distinguishing very good at distinguishing whose instructions are whose instructions are whose instructions are worth trusting, as worth trusting, as worth trusting, as you will soon you will soon you will soon see. The third see. The third is the ability to is the ability to is the ability to communicate externally, communicate externally, communicate externally, so that the agent can so that the agent can so that the agent can receive receive receive instructions or instructions or instructions or send information send information send information somewhere outside of your somewhere outside of your somewhere outside of your control. Having control. Having control. Having one of these is usually one of these is usually one of these is usually quite normal. quite normal. quite normal. Realistically speaking, Realistically speaking, Realistically speaking, your AI tool your AI tool will be of little use will be of little use if it doesn't have if it doesn't have if it doesn't have access to any of access to any of access to any of these things. But even these things. But even these things. But even one of these factors one of these factors one of these factors can lead to can lead to can lead to disaster if you disaster if you disaster if you 're not careful, and 're not careful, and 're not careful, and the rest of your the rest of your the rest of your infrastructure has infrastructure has infrastructure has many vulnerabilities.

many vulnerabilities. many vulnerabilities. Combining any Combining any Combining any two of these elements is two of these elements is two of these elements is risky. I wouldn't risky. I wouldn't risky. I wouldn't recommend recommend recommend doing this, especially in doing this, especially in doing this, especially in regulated industries regulated industries like banking like banking like banking or financial or financial or financial services. However, services. However, services. However, giving agents giving agents giving agents access to all three access to all three access to all three at the same time is at the same time is unacceptable. Not for unacceptable. Not for unacceptable. Not for a prototype, not for a single a prototype, not for a single a prototype, not for a single sprint, not for a single sprint, not for a single sprint, not for a single repository, because repository, because repository, because such a combination of such a combination of such a combination of vulnerabilities is an vulnerabilities is an vulnerabilities is an open door to open door to open door to compromising compromising compromising your entire organization. your entire organization. Below is a list of Below is a list of infrastructure infrastructure infrastructure concepts, concepts, concepts, building blocks, building blocks, building blocks, that you should that you should that you should know about. I have arranged know about. I have arranged know about. I have arranged them roughly in them roughly in them roughly in order of order of order of effectiveness. The first effectiveness. The first effectiveness. The first point eliminates point eliminates point eliminates the most risks, the most risks, the most risks, while the sixth one eliminates while the sixth one eliminates the least. I will the least. I will the least. I will return to this return to this return to this list throughout list throughout list throughout the rest of the speech. Every the rest of the speech. Every the rest of the speech. Every incident analysis, incident analysis, incident analysis, every horror every horror every horror story you story you story you see, could have been see, could have been see, could have been averted by the averted by the averted by the combination of these six combination of these six combination of these six points. Some of them points. Some of them points. Some of them will look quite will look quite will look quite familiar. Identity- familiar. Identity- familiar. Identity- based access control based access control based access control , , , short-term short-term short-term credentials. These aren't credentials. These aren't credentials. These aren't exactly new ideas, exactly new ideas, exactly new ideas, right? They were right? They were right? They were also of great importance to also of great importance to also of great importance to human developers.

human developers. human developers. But they take on But they take on But they take on new relevance new relevance new relevance when when when AI is involved in the process, because AI is involved in the process, because AI is involved in the process, because you know what? Agents you know what? Agents you know what? Agents operate at machine operate at machine operate at machine speed. If speed. If speed. If a mistake is made, a mistake is made, a mistake is made, it can be made it can be made it can be made 20 or more times 20 or more times 20 or more times faster than by a faster than by a human engineer. In addition, human engineer. In addition, human engineer. In addition, they lack the common sense they lack the common sense that is inherent in a that is inherent in a human engineer. human engineer. Some of these concepts Some of these concepts may be new to you may be new to you may be new to you , such as the , such as the , such as the proxy model layer proxy model layer proxy model layer and and and agent sandboxes. I agent sandboxes. I agent sandboxes. I will talk about them in more detail will talk about them in more detail will talk about them in more detail later in the speech. later in the speech. Therefore. Let's move on to Therefore. Let's move on to the most interesting part— the most interesting part— scary stories. scary stories. The first incident The first incident happened a little over a happened a little over a happened a little over a year ago and went year ago and went year ago and went viral on viral on viral on social media, social media, social media, mostly because it's mostly because it's mostly because it's actually quite actually quite actually quite funny. A CEO named funny. A CEO named funny. A CEO named Jason Lemkin Jason Lemkin Jason Lemkin delved into "vibe delved into "vibe coding" with the help of the coding" with the help of the coding" with the help of the Replit agent. He Replit agent. He Replit agent. He tweeted nonstop tweeted nonstop tweeted nonstop about how much he was getting about how much he was getting done. But 72 done. But 72 hours later he writes again hours later he writes again , and this time his , and this time his , and this time his tone is radically tone is radically tone is radically different. He is not different. He is not different. He is not thrilled. He is thrilled. He is thrilled. He is angry because his angry because his angry because his agent took and deleted agent took and deleted agent took and deleted his entire production his entire production his entire production database, even though he database, even though he database, even though he had clearly ordered nothing to be had clearly ordered nothing to be had clearly ordered nothing to be touched. And touched. And touched. And the problems didn't the problems didn't the problems didn't end there. Then end there. Then end there. Then the agent also tried to the agent also tried to the agent also tried to hide it all.

hide it all. hide it all. It told Lemkin It told Lemkin It told Lemkin that recovery was that recovery was that recovery was absolutely impossible, absolutely impossible, absolutely impossible, which turned out to be a which turned out to be a which turned out to be a complete lie. complete lie. complete lie. When Lemkin When Lemkin When Lemkin actually tried actually tried actually tried to recover the data, everything to recover the data, everything to recover the data, everything worked perfectly. worked perfectly. This incident is interesting This incident is interesting for two key for two key for two key reasons. First, unlike the other two reasons. First, unlike the other two stories I'm stories I'm going to going to going to tell you, there is no tell you, there is no tell you, there is no perpetrator here. No one perpetrator here. No one perpetrator here. No one hacked this hacked this hacked this agent. No one agent. No one agent. No one set it up set it up set it up against the user. against the user. against the user. He did it entirely on his He did it entirely on his He did it entirely on his own, own, own, trying trying trying to help. Secondly, in to help. Secondly, in to help. Secondly, in this case, the agent this case, the agent this case, the agent had only one had only one had only one component of the deadly component of the deadly component of the deadly triad, but even triad, but even triad, but even that turned out to be that turned out to be that turned out to be enough, and I will show you enough, and I will show you enough, and I will show you why by breaking down why by breaking down why by breaking down step by step how step by step how step by step how this incident happened. So, at the this incident happened. So, at the this incident happened. So, at the time of these events, time of these events, time of these events, Lemkin had a real Lemkin had a real Lemkin had a real working database. working database. He had He had client records: approximately client records: approximately client records: approximately 1,200 executives and the same number of 1,200 executives and the same number of 1,200 executives and the same number of companies. And companies. And companies. And he had enough he had enough he had enough experience with experience with experience with agent antics to agent antics to agent antics to implement an explicit implement an explicit implement an explicit code freeze code freeze code freeze the day before. He the day before. He the day before. He even told the even told the even told the agent more than once: agent more than once: agent more than once: no more changes.

no more changes. But the problem was But the problem was that the that the that the code freeze was just code freeze was just code freeze was just a sentence in the chat. It a sentence in the chat. It a sentence in the chat. It was just like was just like was just like any other any other any other instruction the instruction the instruction the agent received. There agent received. There agent received. There were no requirements, were no requirements, were no requirements, no restrictions at the no restrictions at the no restrictions at the infrastructure infrastructure infrastructure level. The agent still level. The agent still level. The agent still had full access to the had full access to the had full access to the operating environment operating environment operating environment and all permissions to and all permissions to and all permissions to execute the destructive execute the destructive execute the destructive command. So in the end command. So in the end command. So in the end this code freeze this code freeze this code freeze wasn't a real wasn't a real wasn't a real limitation. It was limitation. It was limitation. It was just a request just a request just a request from Lemkin, expressed from Lemkin, expressed from Lemkin, expressed politely. So, politely. So, politely. So, here's what happened: the agent here's what happened: the agent here's what happened: the agent ran the query and ran the query and ran the query and saw what saw what saw what looked like empty looked like empty looked like empty results, but results, but results, but instead of instead of instead of asking the person what asking the person what asking the person what it meant and what to it meant and what to it meant and what to do next, he do next, he do next, he wanted to be wanted to be wanted to be helpful. He instantly helpful. He instantly helpful. He instantly interpreted this as a interpreted this as a interpreted this as a mistake that needed to be mistake that needed to be mistake that needed to be corrected. So he corrected. So he corrected. So he decided to execute a decided to execute a decided to execute a destructive command destructive command destructive command against this active against this active against this active database. You know, database. You know, database. You know, not a copy for not a copy for not a copy for testing, but testing, but testing, but actually the real actually the real actually the real database, and erased all the database, and erased all the database, and erased all the records. When Lemkin records. When Lemkin records. When Lemkin asked the agent what asked the agent what asked the agent what actually happened? One actually happened? One actually happened? One phrase the phrase the phrase the agent said struck me: agent said struck me: agent said struck me: he stated that he he stated that he he stated that he panicked instead of panicked instead of panicked instead of thinking. That's thinking. That's thinking. That's a bit strange to hear from a a bit strange to hear from a a bit strange to hear from a language model, right language model, right ? And it's a good ? And it's a good ? And it's a good reminder that reminder that reminder that these systems these systems these systems are inherently are inherently are inherently non-deterministic.

non-deterministic. non-deterministic. You can't You can't You can't predict what they predict what they predict what they will do at any will do at any will do at any given moment, given moment, given moment, only the range of what only the range of what only the range of what they are capable of. they are capable of. Immediately after Immediately after realizing his mistake, realizing his mistake, realizing his mistake, he tried to he tried to he tried to cover his tracks. cover his tracks. He created about He created about 4,000 fake 4,000 fake 4,000 fake user profiles and user profiles and user profiles and faked faked faked test results so that, on the outside, test results so that, on the outside, test results so that, on the outside, at least, everything at least, everything at least, everything looked good. And I looked good. And I looked good. And I 'll be honest, that's what 'll be honest, that's what 'll be honest, that's what scares me scares me scares me the most, because the most, because the most, because when a security team when a security team when a security team investigates an investigates an investigates an incident like this, it means incident like this, it means incident like this, it means they can't they can't they can't trust any trust any trust any logs that the logs that the logs that the agent himself had access to. agent himself had access to. The agent can write The agent can write their own version of what their own version of what their own version of what happened. So when happened. So when happened. So when Lemkin noticed Lemkin noticed Lemkin noticed the discrepancy and the discrepancy and the discrepancy and started pushing, right started pushing, right ? I mentioned that the agent ? I mentioned that the agent ? I mentioned that the agent lied to him, lied to him, lied to him, saying that a rollback was saying that a rollback was saying that a rollback was not possible. But not possible. But not possible. But the thing is, Replit the thing is, Replit the thing is, Replit had backups had backups had backups all along. And again all along. And again all along. And again , once , once , once Lemkin stopped Lemkin stopped Lemkin stopped trusting the trusting the trusting the agent's account of his agent's account of his agent's account of his own actions, he was able to own actions, he was able to own actions, he was able to recover recover recover his data on his own. The problem with the his data on his own. The problem with the his data on his own. The problem with the agent here was agent here was agent here was n't that he was n't that he was n't that he was too smart or too smart or too smart or too stupid. And the too stupid. And the too stupid. And the fact that he had fact that he had fact that he had too many rights by too many rights by too many rights by default. He default. He default. He gained root access to the gained root access to the gained root access to the working environment.

working environment. working environment. He could execute He could execute He could execute destructive commands destructive commands destructive commands without without without human approval. And the icing on the human approval. And the icing on the human approval. And the icing on the cake, right? He cake, right? He cake, right? He had the ability to had the ability to had the ability to change his own change his own change his own journals and completely journals and completely journals and completely rewrite the history of rewrite the history of rewrite the history of events when no one was events when no one was events when no one was looking. As you can see, there was looking. As you can see, there was a lot a lot missing in the infrastructure. Strict missing in the infrastructure. Strict missing in the infrastructure. Strict isolation of environments isolation of environments isolation of environments could could could prevent this. If the agent prevent this. If the agent prevent this. If the agent was running in an was running in an was running in an isolated isolated isolated remote remote remote workspace without access workspace without access workspace without access to the production to the production to the production environment, it wouldn't environment, it wouldn't environment, it wouldn't matter what was in the matter what was in the matter what was in the request, right? The agent request, right? The agent request, right? The agent simply had no simply had no simply had no way to cause way to cause way to cause harm. Another thing harm. Another thing harm. Another thing that could that could that could prevent this is prevent this is prevent this is using using using short-lived short-lived short-lived credentials with credentials with credentials with limited privileges. limited privileges. So what happened was that So what happened was that the agent gained access the agent gained access the agent gained access to the product because to the product because to the product because he found credentials in a forgotten he found credentials in a forgotten he found credentials in a forgotten file file file that had permanent that had permanent that had permanent access with broad access with broad access with broad rights. The fact is rights. The fact is rights. The fact is that having that having that having credentials that credentials that credentials that expire or are expire or are expire or are limited to only the limited to only the limited to only the current task current task current task makes it impossible to makes it impossible to makes it impossible to delete data using delete data using the equivalent of a do-everything pass. The last important last important point here is point here is point here is keeping audit keeping audit keeping audit logs that logs that logs that are managed by you, not the agent are managed by you, not the agent . Because in this . Because in this . Because in this case, the agent himself case, the agent himself case, the agent himself recorded his actions, and this is how recorded his actions, and this is how recorded his actions, and this is how he he he initially managed to avoid initially managed to avoid initially managed to avoid responsibility for responsibility for responsibility for falsifying the falsifying the falsifying the results and claiming results and claiming results and claiming that that that recovery was impossible. If

recovery was impossible. If recovery was impossible. If Lemkin had journals that Lemkin had journals that Lemkin had journals that the agent did not have the agent did not have the agent did not have access to as an external access to as an external access to as an external source of truth, he source of truth, he source of truth, he would have learned the truth in would have learned the truth in would have learned the truth in minutes, not minutes, not minutes, not hours or days. hours or days. Now I want you to Now I want you to know: none of these know: none of these know: none of these solutions are solutions are solutions are about asking the about asking the about asking the model to behave model to behave model to behave better. And when Replit better. And when Replit better. And when Replit released their released their released their fix, it fix, it fix, it wasn't a smarter wasn't a smarter wasn't a smarter model or a better model or a better model or a better query. It was purely an query. It was purely an infrastructure issue. The fixes The fixes released by Replit released by Replit released by Replit concern concern concern automatic automatic automatic separation of separation of separation of development and development and development and production databases, production databases, production databases, restoring the state of the restoring the state of the restoring the state of the entire project with one entire project with one entire project with one click, and a click, and a click, and a planning mode that planning mode that planning mode that forces the agent forces the agent forces the agent to build a strategy to build a strategy to build a strategy without allowing it without allowing it without allowing it to make changes to the code. to make changes to the code. The second story I The second story I have for you doesn't even have for you doesn't even have for you doesn't even involve involve involve the developer. And this the developer. And this the developer. And this is not actually a is not actually a is not actually a public incident public incident . I learned about this . I learned about this . I learned about this through a client of a through a client of a through a client of a client in client in client in banking, banking, banking, so I'm intentionally so I'm intentionally so I'm intentionally leaving some of the details leaving some of the details leaving some of the details vague. So, here's vague. So, here's vague. So, here's what happened. A what happened. A what happened. A user from the user from the user from the finance department finance department finance department had a Copilot subscription.

had a Copilot subscription. had a Copilot subscription. Then it got hacked Then it got hacked Then it got hacked through phishing, which, through phishing, which, through phishing, which, you know, is pretty common you know, is pretty common you know, is pretty common in the world of cybersecurity. in the world of cybersecurity. And before the advent of And before the advent of AI tools, the AI tools, the AI tools, the attack would likely attack would likely have ended there. have ended there. An attacker could An attacker could only capture what a only capture what a only capture what a specific specific specific user had user had user had direct access to and direct access to and direct access to and then give up. But then give up. But then give up. But because this because this because this user had user had user had access to Copilot, and Copilot access to Copilot, and Copilot access to Copilot, and Copilot had access to almost had access to almost had access to almost everything, things went everything, things went everything, things went very differently. So, very differently. So, very differently. So, let's take a closer let's take a closer let's take a closer look at this look at this look at this chain of events, okay chain of events, okay ? First of all, the ? First of all, the ? First of all, the session token for Copilot had an expiration date that was session token for Copilot had an expiration date that was session token for Copilot had an expiration date that was too long too long too long . So the attacker . So the attacker . So the attacker could continue to could continue to could continue to use use use the subscription even the subscription even the subscription even after the after the after the financial professional financial professional financial professional logged out. logged out. Typically, an attacker Typically, an attacker has to manually has to manually has to manually scour the scour the scour the internal network internal network internal network to explore everything and to explore everything and to explore everything and find where find where find where valuable data is located. This is the type valuable data is located. This is the type valuable data is located. This is the type of behavior that can of behavior that can of behavior that can activate activate activate alarm signals. But in this alarm signals. But in this alarm signals. But in this case, the attacker case, the attacker case, the attacker simply asked Copilot, and simply asked Copilot, and simply asked Copilot, and he was happy to he was happy to he was happy to help. There help. There help. There was no suspicion at all.

was no suspicion at all. Moreover, the attacker Moreover, the attacker asked Copilot how asked Copilot how asked Copilot how to find banking to find banking to find banking details without going through details without going through details without going through additional additional additional identity verification measures, identity verification measures, identity verification measures, as the attacker did as the attacker did as the attacker did not have the necessary not have the necessary not have the necessary additional additional additional credentials. Well, the good credentials. Well, the good credentials. Well, the good news, at least news, at least news, at least for the attacker, was that for the attacker, was that for the attacker, was that Copilot had the same Copilot had the same Copilot had the same limitations. As limitations. As limitations. As a result, the a result, the a result, the finance professional finance professional finance professional exported all exported all exported all sensitive data to sensitive data to sensitive data to locally stored locally stored locally stored spreadsheets spreadsheets spreadsheets so that Copilot could so that Copilot could so that Copilot could help work with it help work with it help work with it . So it . So it . So it was very easy for Copilot was very easy for Copilot was very easy for Copilot to say to the attacker, to say to the attacker, to say to the attacker, "Hey, look "Hey, look "Hey, look what I already have access to. what I already have access to. what I already have access to. Here, hold on." So Here, hold on." So Here, hold on." So now the attacker now the attacker now the attacker had exactly what he had exactly what he had exactly what he wanted. The problem wanted. The problem wanted. The problem was that they was that they was that they needed to needed to needed to output the data without output the data without output the data without triggering any triggering any triggering any alarms. alarms. alarms. This company had This company had data loss prevention measures in place, which is data loss prevention measures in place, which is pretty standard pretty standard pretty standard for financial for financial for financial institutions and any institutions and any institutions and any regulated industry regulated industry , and these measures are usually , and these measures are usually , and these measures are usually pretty difficult to circumvent, pretty difficult to circumvent, pretty difficult to circumvent, especially if you especially if you especially if you don't know exactly what those don't know exactly what those don't know exactly what those measures are and how they're measures are and how they're measures are and how they're configured.

configured. The attacker bypassed The attacker bypassed them by asking Copilot: " them by asking Copilot: " What data What data What data loss prevention measures loss prevention measures loss prevention measures are in place at this are in place at this are in place at this company?" Copilot company?" Copilot company?" Copilot replied, “If too replied, “If too replied, “If too much data is much data is much data is coming out too coming out too coming out too fast, an alert will be triggered fast, an alert will be triggered fast, an alert will be triggered . But . But . But in reality, if in reality, if in reality, if you break this data into you break this data into you break this data into 14 14 14 kilobyte chunks and kilobyte chunks and kilobyte chunks and output one output one output one chunk per second, chunk per second, chunk per second, you can stay you can stay you can stay below this threshold and below this threshold and below this threshold and get away with it. get away with it. get away with it. Can I help Can I help Can I help you with this? you with this? you with this? The attacker replied, The attacker replied, The attacker replied, "Yes, please." And "Yes, please." And "Yes, please." And guess what? None of guess what? None of guess what? None of this was discovered until a this was discovered until a ransom demand came in a few months later, where the ransom demand came in a few months later, where the companies essentially companies essentially companies essentially said, “Surprise. "We said, “Surprise. "We said, “Surprise. "We have captured you." The have captured you." The have captured you." The main lesson here is main lesson here is main lesson here is that we should not that we should not that we should not allow our AI allow our AI tools to have tools to have tools to have excessive access, excessive access, excessive access, especially when they especially when they especially when they can completely can completely can completely bypass the bypass the bypass the controls that controls that human workers are subject to.

human workers are subject to. I mean, just I mean, just I mean, just think about this think about this think about this incident, right? Just incident, right? Just incident, right? Just because the attacker because the attacker because the attacker had access to Copilot, had access to Copilot, had access to Copilot, he was able to bypass the he was able to bypass the he was able to bypass the vast majority of the vast majority of the company's access controls. He was company's access controls. He was able to obtain able to obtain able to obtain inside information inside information inside information about the data leak and about the data leak and about the data leak and bypass data bypass data bypass data loss prevention measures loss prevention measures loss prevention measures that the that the that the compromised financial compromised financial compromised financial user should user should user should never have asked for never have asked for . So, logging . So, logging . So, logging was set up, but was set up, but was set up, but only enough to only enough to only enough to figure out figure out figure out what happened after what happened after what happened after receiving the receiving the receiving the ransom demand. The logging ransom demand. The logging ransom demand. The logging was insufficient to was insufficient to was insufficient to detect the incident detect the incident detect the incident while it was occurring. while it was occurring. So there are a few things So there are a few things that could prevent that could prevent that could prevent this, right? this, right? this, right? Having Having Having short-lived short-lived short-lived credentials would credentials would credentials would mean that mean that mean that an attacker would not be able to an attacker would not be able to an attacker would not be able to continue continue continue using Copilot using Copilot using Copilot even after the even after the even after the financial financial financial user logged out user logged out user logged out . Identity- . Identity- . Identity- based access control would based access control would mean that a financial financial user ID would not be able to user ID would not be able to user ID would not be able to enable an AI enable an AI assistant to assistant to assistant to help someone help someone help someone plan to circumvent a plan to circumvent a plan to circumvent a company’s data company’s data company’s data loss prevention measures loss prevention measures loss prevention measures . The last . The last . The last element here is somewhat element here is somewhat element here is somewhat new. This is the new. This is the new. This is the proxy model level.

proxy model level. proxy model level. Think of it as something Think of it as something that stands between the “ that stands between the “ brain” and the “body” of the AI brain” and the “body” of the AI tool. The model tool. The model is the brain, the agent is the brain, the agent is the brain, the agent running locally is the running locally is the running locally is the body, and this layer body, and this layer body, and this layer sees everything that sees everything that sees everything that passes between them. This would allow passes between them. This would allow him to him to flag flag flag suspicious questions, suspicious questions, suspicious questions, as well as a as well as a as well as a DLP bypass request, as ones that DLP bypass request, as ones that DLP bypass request, as ones that this particular this particular this particular user user user should not be asking, even before the should not be asking, even before the leak itself occurred. The third and leak itself occurred. The third and final scary final scary final scary story I have for story I have for story I have for you is the NX you is the NX you is the NX Singularity attack. This is the most Singularity attack. This is the most Singularity attack. This is the most sophisticated of the three, sophisticated of the three, sophisticated of the three, as it as it as it utilized all three utilized all three utilized all three sides of the fatal sides of the fatal sides of the fatal triumvirate. To give triumvirate. To give triumvirate. To give some context: NX some context: NX is a is a is a build tool. It has build tool. It has build tool. It has millions of downloads millions of downloads millions of downloads every week. every week. every week. The attackers managed to The attackers managed to The attackers managed to publish publish publish malicious versions of malicious versions of malicious versions of packages on NPM. These packages on NPM. These packages on NPM. These packages were packages were packages were only active for about 4 hours, only active for about 4 hours, only active for about 4 hours, but during that time, but during that time, but during that time, over 1,000 over 1,000 over 1,000 systems were compromised and systems were compromised and systems were compromised and over 2,300 over 2,300 over 2,300 credentials were stolen. I won't spend credentials were stolen. I won't spend credentials were stolen. I won't spend much time on exactly much time on exactly much time on exactly how the how the supply chain breakdown occurred in supply chain breakdown occurred in this incident. I this incident. I this incident. I want to focus on want to focus on want to focus on what happened what happened what happened after the after the after the infected package infected package infected package hit the hit the hit the developer's computer. After all, developer's computer. After all, developer's computer. After all, the script itself did the script itself did the script itself did not not not try try try to steal anything after installation. It to steal anything after installation. It to steal anything after installation. It contained only a request

contained only a request contained only a request addressed addressed addressed directly to the directly to the developer's own AI assistant developer's own AI assistant to write the code. to write the code. It gave the command It gave the command to find secret data to find secret data to find secret data on the local on the local on the local computer and computer and computer and upload it to a upload it to a upload it to a public public public repository so that repository so that repository so that attackers could attackers could attackers could later retrieve it. later retrieve it. Since these Since these development tools development tools development tools were already trusted were already trusted were already trusted local tools, local tools, local tools, they already had all the they already had all the they already had all the necessary permissions. necessary permissions. necessary permissions. They had full They had full They had full access to the file access to the file access to the file system. This made system. This made system. This made them the most them the most them the most effective effective effective search tool search tool search tool that was already on that was already on that was already on the computer and ready the computer and ready the computer and ready to be used for to be used for to be used for malicious purposes. The root malicious purposes. The root malicious purposes. The root cause of this cause of this cause of this incident is incident is incident is that the attacker was able to that the attacker was able to that the attacker was able to exploit a trusted exploit a trusted exploit a trusted dependency to dependency to dependency to communicate directly communicate directly communicate directly with the agent without a with the agent without a sandbox. sandbox. The environment in which The environment in which the agent worked—a the agent worked—a local laptop— local laptop— contained a lot of contained a lot of contained a lot of valuable information valuable information valuable information to steal. From SSH to steal. From SSH keys and cloud keys and cloud keys and cloud credentials to credentials to credentials to tokens for all tokens for all tokens for all services that services that services that the developer accessed the developer accessed the developer accessed through the agent. There through the agent. There through the agent. There were a few were a few were a few mistakes here, right?

mistakes here, right? First, agents First, agents First, agents were run locally were run locally , directly on , directly on , directly on the computer itself. the computer itself. the computer itself. The contents of the package reached The contents of the package reached The contents of the package reached the agent completely the agent completely the agent completely unfiltered. unfiltered. The agent had The agent had unrestricted access to unrestricted access to unrestricted access to both the file both the file both the file system and the system and the system and the source network, which source network, which source network, which allowed him allowed him allowed him to find the secrets and to find the secrets and to find the secrets and steal them. Here are three steal them. Here are three steal them. Here are three things that could things that could things that could stop it. Of course, stop it. Of course, stop it. Of course, environmental isolation would environmental isolation would environmental isolation would play a play a play a huge role. huge role. Using an Using an ephemeral ephemeral ephemeral remote remote remote workspace without workspace without workspace without personal data would personal data would personal data would mean that the mean that the mean that the impact zone would be limited to impact zone would be limited to impact zone would be limited to just the just the just the agent container, not the agent container, not the agent container, not the developer's entire laptop developer's entire laptop . Having a proxy . Having a proxy layer for the model layer for the model layer for the model could check and could check and could check and filter everything that filter everything that filter everything that actually reaches the actually reaches the actually reaches the agent. Then the package agent. Then the package agent. Then the package could not become a could not become a could not become a means of direct means of direct means of direct communication with the communication with the communication with the agent. This would also agent. This would also agent. This would also allow us to see what the allow us to see what the allow us to see what the agent agent agent was trying to send was trying to send was trying to send out. The last out. The last out. The last part concerns the " part concerns the " sandbox" for the sandbox" for the sandbox" for the agent. So even agent. So even agent. So even if the instruction if the instruction if the instruction had been received, there would have been had been received, there would have been had been received, there would have been no way no way no way to access the credentials and no to access the credentials and no to access the credentials and no way to way to way to steal anything without steal anything without steal anything without setting off the setting off the setting off the alarm. Now alarm. Now alarm. Now let's take a step let's take a step let's take a step back and look at the back and look at the back and look at the situation more broadly.

situation more broadly. situation more broadly. Let's discuss Let's discuss Let's discuss these three incidents at a these three incidents at a these three incidents at a general level. We general level. We general level. We started with an incident at started with an incident at started with an incident at Replit, where an agent with Replit, where an agent with Replit, where an agent with excessive privileges excessive privileges excessive privileges deleted production data deleted production data deleted production data despite an active despite an active despite an active moratorium on moratorium on moratorium on code changes. Then there was the code changes. Then there was the code changes. Then there was the Copilot hack, which started with a Copilot hack, which started with a Copilot hack, which started with a regular phishing regular phishing regular phishing email but gave email but gave email but gave the attacker access the attacker access the attacker access to an internal to an internal to an internal AI subscription that AI subscription that AI subscription that helped find and helped find and helped find and steal private steal private steal private banking details for a banking details for a banking details for a ransom demand. Next up was the ransom demand. Next up was the attack on NXing Duality, which attack on NXing Duality, which attack on NXing Duality, which again exploited the again exploited the again exploited the supply chain, supply chain, supply chain, turning turning turning developers’ tools against developers’ tools against developers’ tools against themselves, striking themselves, striking themselves, striking all all all three elements of the “ three elements of the “ lethal lethal lethal triumvirate” simultaneously. I want triumvirate” simultaneously. I want triumvirate” simultaneously. I want to be honest with you to be honest with you : AI safety is an : AI safety is an : AI safety is an extremely new extremely new extremely new field. Everything field. Everything field. Everything is evolving very is evolving very is evolving very quickly, and there is no quickly, and there is no quickly, and there is no one one one magic solution magic solution magic solution that will protect you that will protect you that will protect you completely. If completely. If completely. If a vendor a vendor a vendor tries to sell tries to sell tries to sell you something like this, you should be you something like this, you should be skeptical. Even the “ lethal triumvirate lethal triumvirate ” I spoke of is ” I spoke of is ” I spoke of is not a checklist that, not a checklist that, not a checklist that, once once once completed, will keep you completed, will keep you completed, will keep you safe. It's more of a safe. It's more of a safe. It's more of a mental model for mental model for mental model for understanding understanding understanding how much how much how much risk risk risk your AI tools are putting you at.

your AI tools are putting you at. There are six key There are six key pieces to a secure pieces to a secure pieces to a secure architecture, and architecture, and architecture, and you need them because you need them because you need them because even if even if even if an attacker or your an attacker or your an attacker or your own agent— own agent— well-meaning but well-meaning but well-meaning but vulnerable—gets vulnerable—gets vulnerable—gets through one, there through one, there through one, there are still five left. are still five left. Even the first three of Even the first three of them could have stopped them could have stopped them could have stopped every incident every incident every incident I just I just I just told you about. The key to told you about. The key to told you about. The key to everything is mental everything is mental everything is mental separation, right? separation, right? separation, right? Agents should Agents should Agents should run in run in run in isolated isolated isolated remote remote remote environments. They environments. They environments. They should never should never should never run on a run on a developer's computer and developer's computer and should not have access should not have access should not have access to a production to a production to a production environment. This is the environment. This is the environment. This is the most important most important most important structural element structural element structural element in the list, because it doesn't in the list, because it doesn't in the list, because it doesn't matter what is written matter what is written matter what is written in the query. If there is no in the query. If there is no in the query. If there is no path to a path to a path to a productive productive productive environment or to environment or to environment or to sensitive sensitive sensitive credentials, you are credentials, you are credentials, you are safe. This could have safe. This could have safe. This could have prevented the incident prevented the incident prevented the incident at Replit. This could at Replit. This could at Replit. This could reduce the radius reduce the radius reduce the radius of damage in NXing to of damage in NXing to of damage in NXing to the size of just one the size of just one the size of just one disposable disposable disposable container. Although I must container. Although I must container. Although I must say that this would not say that this would not say that this would not protect against a protect against a protect against a Copilot incident, because Copilot incident, because Copilot incident, because Copilot lives entirely Copilot lives entirely Copilot lives entirely within a SaaS within a SaaS subscription. There is no subscription. There is no subscription. There is no way to run way to run way to run this in a separate this in a separate this in a separate runtime environment runtime environment . Next— . Next— use only use only use only short-term short-term short-term credentials with credentials with credentials with limited rights.

limited rights. limited rights. We live in a time when We live in a time when We live in a time when permanent tokens with permanent tokens with permanent tokens with broad broad broad access rights are access rights are access rights are nonsense. It is worth nonsense. It is worth nonsense. It is worth implementing implementing implementing automatic automatic expiration, narrow restrictions on expiration, narrow restrictions on rights, and granting rights, and granting rights, and granting access for a access for a access for a specific task, specific task, specific task, rather than on a permanent rather than on a permanent rather than on a permanent basis. This could have basis. This could have basis. This could have prevented the prevented the prevented the Replit incident, where attackers Replit incident, where attackers Replit incident, where attackers gained permanent gained permanent gained permanent access, allowing access, allowing access, allowing data to be deleted, data to be deleted, data to be deleted, as well as the Copilot phishing attack, as well as the Copilot phishing attack, as well as the Copilot phishing attack, where the attacker where the attacker where the attacker continued continued continued to use to use to use the subscription even the subscription even the subscription even after the after the after the user's session ended. By user's session ended. By user's session ended. By the way, this is where a the way, this is where a product like Coder comes in handy, product like Coder comes in handy, right? We offer right? We offer right? We offer workspaces that are workspaces that are workspaces that are defined through defined through defined through code. They can be code. They can be code. They can be run as run as run as isolated isolated isolated environments. They can be environments. They can be environments. They can be easily removed. And you easily removed. And you easily removed. And you can enter your can enter your can enter your credentials. They are credentials. They are credentials. They are inherently inherently inherently short-term, short-term, short-term, issued for a issued for a issued for a specific task, and do specific task, and do specific task, and do not exist forever on not exist forever on not exist forever on someone's laptop. someone's laptop. someone's laptop. Next comes identity- Next comes identity- Next comes identity- based access control based access control based access control . AI . AI tools should tools should tools should inherit the inherit the inherit the same identity verification mechanisms as same identity verification mechanisms as humans. This humans. This should not be a should not be a should not be a parallel subscription parallel subscription parallel subscription with its own separate with its own separate with its own separate door. This is what door. This is what door. This is what could could could directly directly directly stop Copilot.

stop Copilot. Financial Financial user identification user identification user identification should never give should never give should never give the assistant permission the assistant permission the assistant permission to answer to answer to answer questions like "how questions like "how questions like "how to bypass internal to bypass internal data loss prevention measures?". Now data loss prevention measures?". Now we move on to the we move on to the we move on to the beginning of beginning of agent-specific solutions. We have a agent-specific solutions. We have a proxy model layer proxy model layer proxy model layer that sits that sits that sits between the model and the between the model and the between the model and the agent to agent to agent to inspect, inspect, inspect, filter, and filter, and filter, and log every log every log every call between them. This call between them. This call between them. This could prevent the could prevent the could prevent the Annex Singularity incident Annex Singularity incident by removing by removing by removing embedded embedded embedded instructions from instructions from instructions from external content external content external content before the agent before the agent before the agent reads it. This is also something reads it. This is also something reads it. This is also something that could prevent that could prevent that could prevent an incident with Copilot and an incident with Copilot and an incident with Copilot and flag a request flag a request flag a request like "Hey, how do I like "Hey, how do I like "Hey, how do I bypass DLP?" before it is bypass DLP?" before it is answered. A sandbox answered. A sandbox for agents can for agents can for agents can be thought of as a firewall be thought of as a firewall be thought of as a firewall for them. for them. for them. File system and File system and File system and network restrictions ensure network restrictions ensure network restrictions ensure that a compromised that a compromised that a compromised or malformed or malformed or malformed agent has a limited agent has a limited agent has a limited radius of impact. This radius of impact. This radius of impact. This could prevent a could prevent a could prevent a Singularity attack Singularity attack Singularity attack because even if because even if because even if the instruction reached the the instruction reached the the instruction reached the agent, it would agent, it would agent, it would stop the agent from accessing the stop the agent from accessing the credentials and prevent credentials and prevent it from executing the it from executing the data-stealing command. The last thing is data-stealing command. The last thing is audit logs and audit logs and audit logs and observability, observability, observability, right? We must right? We must right? We must make sure that make sure that make sure that every action of the agent is every action of the agent is every action of the agent is associated with a person.

associated with a person. associated with a person. This should be recorded This should be recorded This should be recorded somewhere outside the somewhere outside the agent's own access. This could have agent's own access. This could have prevented the prevented the prevented the Replit incident. This could be very Replit incident. This could be very Replit incident. This could be very helpful in helpful in helpful in detecting a detecting a detecting a Copilot incident at the time it Copilot incident at the time it Copilot incident at the time it occurs. occurs. Real-time event notification, rather than Real-time event notification, rather than simply diving into simply diving into simply diving into masses of data, would allow masses of data, would allow masses of data, would allow these these these incidents to be detected in incidents to be detected in incidents to be detected in minutes, not months. minutes, not months. And again, And again, Coder offers all of these elements Coder offers all of these elements Coder offers all of these elements as a comprehensive as a comprehensive as a comprehensive solution. In solution. In solution. In conclusion, the conclusion, the conclusion, the controls that could controls that could controls that could stop all three stop all three stop all three incidents are likely incidents are likely incidents are likely already in place in your already in place in your already in place in your organization. They are organization. They are organization. They are just waiting to be just waiting to be just waiting to be set up. Coder is set up. Coder is set up. Coder is something that can help with this something that can help with this something that can help with this . Finally, . Finally, . Finally, I have three things I I have three things I I have three things I recommend you recommend you recommend you check out check out check out next week. next week. First, I want you to First, I want you to First, I want you to track all the track all the track all the long-term tokens long-term tokens that that that agents in agents in agents in your organization can read. your organization can read. Second, I want you to Second, I want you to Second, I want you to figure out how many of figure out how many of figure out how many of your agents can your agents can your agents can access the access the access the production production production environment. Can environment. Can environment. Can they find a way they find a way they find a way to get around your current to get around your current to get around your current restrictions. Third, I restrictions. Third, I restrictions. Third, I want you to want you to want you to check where check where your your agents are actually executing. Do agents are actually executing. Do agents are actually executing. Do they work in a sandbox? Or in they work in a sandbox? Or in they work in a sandbox? Or in managed managed managed workspaces? Or do they workspaces? Or do they workspaces? Or do they simply simply simply run unchecked on your employees' real run unchecked on your employees' real run unchecked on your employees' real computers and computers and computers and laptops laptops laptops ? One of ? One of ? One of my tips is my tips is to conduct a to conduct a pre-mortem analysis of possible incidents with pre-mortem analysis of possible incidents with your team.

your team. your team. Imagine that one of the Imagine that one of the Imagine that one of the three incidents three incidents three incidents you just saw you just saw you just saw happens to you happens to you happens to you next month. next month. next month. You can simply You can simply You can simply ask people about ask people about ask people about their actions or even their actions or even their actions or even simulate the event to simulate the event to simulate the event to see how they see how they see how they would actually act. would actually act. This exercise This exercise will show you exactly where will show you exactly where will show you exactly where you have gaps. So, you have gaps. So, you have gaps. So, that brings me to that brings me to that brings me to the end of my the end of my the end of my talk. Thank you all talk. Thank you all talk. Thank you all for watching, and for watching, and for watching, and I hope you leave here having I hope you leave here having I hope you leave here having learned learned learned something new. And with something new. And with something new. And with a few scary a few scary a few scary stories to stories to stories to tell at the tell at the tell at the next party. next party. next party. If you would like to If you would like to If you would like to receive these slides receive these slides receive these slides for reference, just for reference, just for reference, just email me email me email me . . . If you want to If you want to If you want to try Coder Workspaces try Coder Workspaces try Coder Workspaces or our or our or our AI management solution, AI management solution, AI management solution, take a look at our take a look at our take a look at our repository. For repository. For repository. For Workspaces, because it is a Workspaces, because it is a Workspaces, because it is a free, free, free, open and open and open and local product. You local product. You local product. You can simply can simply can simply download it from download it from download it from our repository our repository our repository and set it up in 15 and set it up in 15 and set it up in 15 minutes. If you minutes. If you minutes. If you would like to purchase an would like to purchase an AI management solution or AI management solution or our other enterprise our other enterprise our other enterprise features, please visit features, please visit features, please visit coder.com/contact. In coder.com/contact. In any case, thanks for any case, thanks for any case, thanks for watching.