Anthropics Responsible AI Scaling Policy

Max Aulakh 00:03 – 00:17

Welcome to Emerging Cyber Risk, a podcast by Ignyte and Secure Robotics. We share our expertise on cyber risk and artificial intelligence to help you prepare for risk management of emerging technologies. We’re your hosts, Max Aulakh. 

 

Joel Yonts 00:17 – 00:37

And Joel Yonts. Join us as we dive into the development of AI, evolution in cybersecurity, and other topics driving change in the cyber risk outlook. Welcome to another episode of Emerging Cyber Risk Podcast. We’re your hosts, Joel Yonts, and Max Aulakh is on as always. Max, how’s your day? 

 

Max Aulakh 00:37 – 00:43

Hey, it’s going really well. Busy as always, but I can’t complain. I’m excited to be here about this podcast. 

 

Joel Yonts 00:43 – 01:45

Yeah, I think we’ve got another good one today. Today, we’re talking about Anthropic’s new responsible scaling policy. New as it is at the end of September. I know you’ve dug into it a bit, but It’s a pretty impressive example on a couple of levels from my perspective. I want to get your perspective before we dive in. But for me, one, because it’s very interesting, it’s just an interesting topic. We’ve all had conversations about how bad can AI be? There’s some concern around all kinds of things as these things scale up. Scaling has to do with power. And it’s talking about as they build bigger and bigger models, they’re being very careful in their approach. So I think that’s one thing that’s interesting about this one, to make sure that they don’t get out over the hood when it comes to safety. The other thing is, they’ve got a great pattern around how to do end-to-end capability delivery that has commitments that have teeth. That was my impression. Max, what are your thoughts overall? 

 

Max Aulakh 01:45 – 02:32

I love the name of it. I mean, responsible policy, right? Some sort of being responsible with artificial intelligence, which is broad, right? So it’s gonna, It’s going to touch on many different things beyond security, which is exciting because this emerging field is going to fuse a lot of different things together. But as you said, it’s got a very practical approach to it. That’s what it feels like. That’s why we’re going to discuss it. So when we looked at other documents, we looked at MITRE, DoD, and I think DHS, and there’s a lot of other folks that are coming up with things. I like the practicality of this, but I also like it how it’s strategic in nature. It’s not just fluff words, but also it’s not so deep technical that it feels like an OWASP standard or a document. 

 

Joel Yonts 02:32 – 02:52

Absolutely. I think the thing that I would love to add to that is that they made some hard commitments and they put teeth to where the public will know if they’re not meeting those because they have public disclosure pieces of it and a commitment to pause if they ever get sideways with safety. So I think this is a really good example and I’m looking forward to diving into it. 

 

Max Aulakh 02:52 – 03:08

Yeah. So, Joel, let’s go through this. How is the framework set up? So we’re all used to different kinds of frameworks with maturity models and those kinds of things. How is this, what’s the overarching approach to setting this policy up? 

 

Joel Yonts 03:08 – 04:33

Certainly. I think the first thing that we need to talk about when we get into the framework is that understanding that Anthropic is doing frontier models. And for the larger audience, in case some people don’t know about that, it’s just a broad term to talk about. They’re pushing the outer edge of capabilities in AI. And so when we’re talking about responsible scaling, this framework is geared to make sure that they don’t bring into existence a model that is too dangerous for current safety protocols. And so because of some of the potential and there’s a lot of different thoughts around this that there’s a lot of experts saying this has like weapons of mass destruction capability on the upper end what they did they built this framework and use they call it the ai safety level framework and there’s four levels today they leave the door open for a fifth and further ones as capabilities develop and they modeled it after the u.s biosafety level standards that are used for handling dangerous biological materials. And I thought that was a very interesting way of approaching it, because biosafety levels are things that are heavily used today throughout the U.S. and I think the world, and mimicking that really immediately gets a good context to build on that we’ve already been used. 

 

Max Aulakh 04:34 – 05:22

You know, when I was in the Air Force, we had this acronym, we called it C. Burney, and I think it stands for chemical, biological, something else, right? But it was around the weapons of mass destruction, essentially, right? So I think setting that context is important because I just saw a headline the other day about how you know, they’re leveraging AI for autonomous drones. Now, some of that is just pure buzz and marketing, but some of it is like actually being tested, trying to see if, you know, the art of the possible. So, I’m glad that at least somebody is looking at this from a practicality perspective, and it makes sense to have biosafety as a prior framework to look at, right? Because cyber usually never touches physical that much, where this thing will. 

 

Joel Yont s05:22 – 07:05

Absolutely i love that and it’s funny you should say that you talk about seabird will talk about that more in a moment because that’s one of the risk associated with it believe it or not is that these large models, have the ability to inform people to do a lot of bad things potentially and, Then we look at, and really they broke the risks down for each category into two classes of risk. This framework, again, one through four, with one being models that don’t pose any significant risk, and level two being models that have some level of first indications of potential risk of a catastrophic nature. Again, we’re talking about catastrophic nature. And then three and four kind of moves further into realization of that but they break the risk associated with each of these levels into two classes deployment risk and containment risk and if you will the deployment risk is about the acceptable use of the model if a model is at level two which is where we are today and if a model is at level two and it’s being deployed in an environment if authorized users using this model what is the level of risk associated with it and then the other one being if unauthorized use. This could be if attackers were able to break some guardrails or to steal the model itself. Or in higher levels, this is where it gets kind of interesting and cool, if the model autonomously replicates itself out of its environment to a new location. And models that can do that pose a containment risk associated with it. So it really measures the risk across those scales in those particular two categories. 

 

Max Aulakh 07:05 – 07:58

And then in terms of these scales, Joe, like you mentioned earlier, we see that there’s the layout for models or for levels. AI safety level one through four. Fifth one, there’s room for that. But I think when you mentioned scale equals power. So the first level is really for small models that are not very scalable, things like that. So I don’t even know if Anthropic has put any work into that. But a lot of this work seems to be at the you know, large models and beyond, right? Whenever there are deployment risk of large models. So it’s really, I think a lot of work has gone into what they call level two, and we’re still seeing what’s going to happen with level three, which I know we’ll get to how they even reach to the next level, right? How do you measure if you’re mature enough or safe enough to go to the advanced levels and whatnot? 

 

Joel Yonts 07:58 – 08:39

Absolutely, and that’s a very good point. They’ve done heavy development on level 2 and level 3, and they’ve made a commitment that says, to our knowledge, there are no models that are at level 3 in this scale, but they’ve made a commitment, anytime a model, a first discovery of a model that enters a new level, that they will make sure that the next level of development of this framework is built out. They haven’t built out level 4 yet, And it’s intentional because we don’t know exactly what that’s going to look like yet and the risk associated with it. But before they train the first level three model, they’re committing to building out level four commitments for that framework. 

 

Max Aulakh 08:39 – 09:26

So these commitments, let’s talk about them because that’s kind of interesting. It’s almost like where we do cybersecurity maturity modeling. We know what the maturity levels are, but what it takes to get there, we don’t know. We kind of left to guess. The way Anthropic is saying this is that they must meet all the commitments that they have already laid out, and then they’re going to lay out more commitments in order to even think about the next level. Joe, that’s how I read it, right? Like they got to complete what they have before they start to say, okay, we’re now going to go to the next maturity level. Which is very unique, it’s kind of proportionate to the level of work that we can see and we can complete right now, which I know we’ll dig into what does this all entail in terms of our level 2. 

 

Joel Yonts 09:27 – 10:13

Absolutely. And again, this is one of the reasons that I love what they’ve done here is they have a hard commitment that says they define criteria about black and white, where it places a model in the level of capability in this AI safety levels. And at the point where a model enters that they believe it’s going to enter the next safety level, unless they have that next level, you know, that level plus one developer, They committed to pausing. They won’t train that model. They won’t move forward. They’re going to basically pause everything and go build that out. And that’s a pretty strong commitment because, you know, you and I both know we work with enterprises. That’s that’s dollars, that’s money, that’s a competitive edge. And so. 

 

Max Aulakh 10:13 – 11:25

Yeah. And you know, the the other thing that it reminds me of. So in our security world, we’ll have policies and then we’ll have some actions behind the policy, such as, hey, go do a red teaming exercise. OK, we got a policy on doing penetration testing, and then there’s an activity in a report, and then you fix it, right? What I really like about what I’m reading here is that it’s not that they only have activities, they have internal tools, like constitutional AI or measuring the impact to society, things we never really think about. So these models, as well as the safety levels, are tied to internal playbooks and internal actual tools, which in the future could be available to the public to measure other models that could cause a major societal impact that hasn’t been measured. So I see that as a very different, I mean, it’s similar to cybersecurity stuff, but it’s very different because it’s not just activities and reporting and go fix it it’s an actual build out of these kinds of tools that can counter it rising our past episodes right you need an AI to fight an AI yeah exactly exactly that i will say. 

 

Joel Yonts 11:25 – 11:46

That i do have a little bit of a bone to pick with and throw pick. in that they’re naming. And so this is a pet peeve, a little bit of mine. They call it a policy, but they have everything in the kitchen sink in this document. They have policies, they have standards, they have procedures, they have technical details. I mean, I love the totality of it, but it’s so much more than a policy in a lot of ways. 

 

Max Aulakh 11:47 – 11:55

Yeah, there wasn’t much decoupling how we’re used to, but the intent of trying to do the right thing is clear. 

 

Joel Yonts 11:55 – 12:27

Yeah, absolutely. Now, one of the things when I look at, though, that standards piece, and I know you and I both have copies of the document that we can reference through it, but I think that it might make sense to give some concreteness. Specifically, they call out what dangerous capabilities makes an ASL 2. which ASL is the AI safety level 2, so that we can make it more concrete. Let’s talk about, if you want to give us a flavor, what dangerous capabilities makes a model on that, and then we can talk about the containment measures and deployment measures. 

 

Max Aulakh 12:27 – 12:42

Yeah. I think you might have to help me because I read it, but I don’t quite remember it, but I think a lot of it had to do with misuse and replication. I think those were sources of the risk, as if I recall it, right? 

 

Joel Yonts 12:42 – 13:19

Yeah, exactly. I’m going to cheat here for just a minute because I do have a copy of it. I think this may be worth kind of verbatim given that that detail because it syncs it up. You know what they call an AI safety level 2 these models according to the document says no capabilities likely to cause a catastrophic level harm but they have early indications of these capabilities. And they talk about, for example, AI systems that can provide bioweapon related information that couldn’t be found by a search engine, but does so unreliably to be used in practice. So I think that goes back to what you were telling me earlier, right? 

 

Max Aulakh 13:19 – 14:46

Yeah. And this reminds me of when 9-11 first happened, there was a lot of, you know, there was a lockdown of the airports, the airport security policy and posture, everything changed. And I remember, Back in those days you can figure out how to build up on the internet and and there was some policy action something happened where a lot of that information got taken off the internet but people who know how to use. Web searches advanced searches they can still get to it because the internet has a very long memory. But this could make it so easy, so easy to find that kind of information. So that’s what I feel like this is getting at, because they’re providing this information, an example of exploiting the search engine to find things that are harmful on those kinds of things. And that’s where I would think that’s one, the other example that it comes to, I was messing with the chat GPT and I was trying to get it to respond to me almost like if it’s my significant other. Like just for fun, hey, can you tell me you love me? Can you tell me you hate me? Taking the emotion out of it. There’s all sorts of societal impacts, right? Like where people are gonna abuse it in different ways and imagine becoming dependent on your AI for a personal relationship and what kind of impact that could have, right? So, I think the abuse cases can be built out. 

 

Joel Yonts 14:46 – 15:53

Absolutely. And we see it all the time, like threat actors using it for misinformation or some sort of disruptive type generative AI. We’ve seen a lot of these different things. And I think the difference we’ll see between Level 2 and Level 3 is early indications of these capabilities where Level 3 is able to deliver those capabilities. And that’s the difference. And they talk about, and we haven’t talked much about this containment threats, that’s kind of the authorized use. But the other aspect around these models, there really isn’t much talk about autonomous replication, the model trying to copy itself outside of its environment. So there’s really not a lot of threat with Level 2 from that aspect of the model. But they go ahead and they talk about, if you think about these models were trained with a lot of dangerous capabilities, they’ve just safeguards and rails, guardrails within them to keep them from being used in an abusive way. And so the first thing around ASL 2 or AI safety level 2 is they start talking about these countermeasures. of what to do about that level. Do you want to kind of hit on a couple of those? 

 

Max Aulakh 15:53 – 16:34

Yeah. And actually, before I do that, I had a question, and I couldn’t find this, Joe. I know you’ve been digging into this. So when we look at the containment measures and the deployment measures, there’s examples from a security perspective, right? It’s talking about hardening security against attackers, essentially. But when we look at the word responsible and societal impact, you can unpack that in many different ways. Did you see Anthropic diving into other topics beyond just technology or cyber security threats? I’m just curious if you found that maybe within this document. I didn’t see it. Maybe they have it in other documents. 

 

Joel Yonts 16:34 – 17:38

Yeah, they mention it in this document, and they talk about responsible or AI safety overall, which that topic, and they kind of talk about early on that this work is complementary to some of the other AI safety work they’re doing around harms from misinformation, bias, societal impacts, privacy, reliability, and alignment with human values. They call it those specifically. Those are all really important topics around AI safety and responsible AI. It’s something that we’ll probably talk about on future podcasts a little bit further. The way they’re treating this is that this is one vertical within responsible AI. As they’re building these frontier models and may usher in a new level of capability, they want to be responsible in how to usher in that next level of AI capability. But it fits in with this overarching framework they have. And those are, and I think they have an entire library of capabilities that cover some of these other areas that we’re talking about. 

 

Max Aulakh 17:38 – 18:48

So really what we’re really looking at as what I’m gathering is that we have these safety levels. We’re just covering one pillar of it. We’re just covering the security pillar because. Yeah, all the other things about ethics and human rights violations, those are other pillars of this responsibleness, but they’ll hopefully light up to this three safety levels that are very practical in nature. This is awesome. So the next thing, Joan, you asked me about the deployment measures. So there’s containment and then there’s deployment. The deployment measures, I think they have put together some of these best practices around what they called model cards. We have acceptable use policies, which discusses misuse, escalation procedures, vulnerability reporting, and then, of course, other kinds of evaluation. They’re trying to make an overlap with the White House, some of the commitments. And we did a podcast earlier on AI policy. We did cover some of the White House commitments earlier. But Joel, what is this model cards concept? Because I think that’s a new term. That’s a brand new term that’s being introduced into the ecosystem. 

 

Joel Yonts 18:48 – 19:33

Yes, it’s definitely new with the AI space. Actually, Anthropic wasn’t the originator of model cards, but they picked up this industry practice that is new. These model cards are designed to give a detailed analysis of the data, the construction, the training, the associated appropriate use, a lot of things that are needed to support model transparency, how decisions are made to help alleviate some of the safety concerns around bias, unpredictable results, and it really lays a foundation. I almost feel like these model cards are like asset management. 

 

Max Aulakh 19:33 – 19:41

Man, that’s what I was just going to mention. It reminds me of categorizing and collecting your assets to some degree. 

 

Joel Yonts 19:41 – 20:10

Yes, yes. Because you can’t do the work without it. But this goes into architecture and again, talks about the ethics and alignment. And there’s a lot of things that can go into these model cards. and they’re committing to any model AI safety level two or higher has to have model cards fully developed. And I think that really positions it well to begin, just like assets, that discussion around AI safety and responsibility. 

 

Max Aulakh 20:10 – 20:52

Yeah, you really can’t even begin to protect or do any safety around things you can’t even identify. So this model card essentially is its ID card of what this thing is, the context, of what it’s built out of, those kinds of things. So what they’re saying here is that you must have a model car. You can’t just have a bunch of libraries and software code running out there. You got to define it. You got to describe it properly. And once you, once you understand it, then we can, and this is nice because this, this reminds me so much of information security scoping. That concept exists in SOC 2. It doesn’t matter what we actually trying to protect here, right? And so this kind of lays out the scope of what they’re trying to guard. 

 

Joel Yonts 20:52 – 21:36

Absolutely. And I think that makes, I mean, it really lays out that foundation. And they build on that with a lot of commitments around reporting of vulnerabilities, acceptable use. But I find it very interesting in the tooling section of the requirements for this level two deployment measures. They talk about use of models to identify harmful user props and model completions. It’s almost like, you know, they use AI models, they commit to using AI models to make sure that harmful props doesn’t enter the system and it doesn’t respond with some. Okay, what tools, what security tools does that remind you of? 

 

Max Aulakh 21:36 – 21:55

Man, right now it’s just a lot of people inserting stuff into chat GPT trying to play the reverse psychology on it. Yes. That’s what it immediately reminds me of is, uh, how do we get data out of this chat GPT thing by, by tricking it? Right. 

 

Joel Yonts 21:55 – 22:26

Yeah. I was, I was just having, you’re exactly right. I was just having this discussion and you know, these companies are putting these guardrails in to make sure that you can’t. put in, you know, like if you ask it a prompt series that’s going to extract harmful information, he’s got these guardrails. But these are massively complex models. And so if you ask the question one way, you know, it’s going to hit the guardrail. But you can approach the subject from so many different angles. I mean, I can imagine you can drive around this guardrail and get to what you want. 

 

Max Aulakh 22:26 – 23:18

Maybe. I think this is a new emerging field in this way. So our legacy tools are like data loss prevention, right? Keyword driven, things like that. Hey, this is marked as such, it’s leaving the site, block it, whatever. I think we’re going to start to see context loss prevention, if that’s a term, right? Where the new, it’s not about data, it’s about context now. And how do you prevent loss of context? And that’s, nobody has really figured that out, but that’s the problem right here. That’s what we’re trying to solve is how you protect against inference attack. Well, you, you got to limit the context you provide the intent and you know, however you develop context. Um, so, so that’s gonna, you know, I think that’s going to be interesting in terms of the tooling that they’re building in order to prevent the, the contextual information getting out. 

 

Joel Yonts 23:18 – 23:48

Yes, yes. And it’s funny you should say that I had some scribbled notes on the document that talked about synonymous with DLP and WAFs because you’re guarding data coming in and out for whatever reason, whether it be a cyber attack. But I would imagine you’re also trying to guard if your model just generated something that was dangerous or hit a category, maybe it’s a bioweapon category, for whatever reason it came out of it, this tooling is supposed to stop it and move it forward, I think. 

 

Max Aulakh 23:48 – 24:30

Yeah, and there’s another sentence in here, and I don’t know what this means, but it says harmful refusal techniques. And I don’t know if this is a behavior modification of the artificial intelligence where it’s supposed to obey you and listen to you, so it’s not refusing commands that are appropriate. You know, I don’t know, but it reminds me of, uh, you know, uh, what is that, that movie by Will Smith? iRobot. iRobot. iRobot, right? Where, where it’s, where the artificial intelligence starts to refuse things. So I don’t know if that’s what it’s getting at when it says refusal, uh, but that’s kind of what it reminds me of. And that’s all part of, part of the deployment measure of the, the safety level number two. 

 

Joel Yonts 24:31 – 24:52

Yes. And one other thing that I want to talk about, and we definitely need to get to three, they talk about red teaming in ASL 2, or this safety level 2, and it means a lot of different things. So, in the context of this AI world, You want to talk a moment about what that red teaming means and some of the threat profiling they talk about? 

 

Max Aulakh 24:52 – 25:28

Yeah. Yeah. I think so. Uh, first I’m glad to see some sort of, this is a traditional security exercise, but the, but the way it might be executed is going to be very different because you know, you’re trying to fight a machine here. That’s right. Yes. Um, so I, I think they, they lay out, uh, you know, this, I might have to cheat here a little bit, but they lay out different techniques and on how to red team this properly. I didn’t see a great detail of it, but I’m just glad to see that it’s actually being called out as an activity, as a, as a black and white commitment that, that they actually, they actually have to do. 

 

Joel Yonts 25:29 – 26:15

Yeah, you’re exactly right. And I love it that as a security person, you know, that’s, that’s a very, we all love red teaming. Right. But I also, when I was digging around, I did find out that going beyond just the cybersecurity space, it’s also about trying to get these more models to spit out maybe things that it shouldn’t, like going back, keep using that bioweapon thing. It’s about that red teaming also uses industry experts in various sectors to see if they can get the model to behave or say something that shouldn’t. So they’ve expanded on the definitions to include this extra usage layer to determine, oh, wait, it spit out the entire formula for something that it shouldn’t have. That’s a problem. And so I think that’s part of the red team as well. 

 

Max Aulakh 26:16 – 27:15

Yeah, yeah. And they’ve got traditional stuff in here that I see, like, you know, having a dedicated resource, looking for insider threads, honey pots need to be set up, all of that. But yeah, I think adding other elements of getting information out. So if I don’t know how to build a bomb, because I’m not EOD, explosive ordnance kind of person, but if I ask the question, EOD person would be able to tell me, yeah, this is legit information or not, right? So I think bringing in a lot of different non-technical fields to do the red teaming is really the key, which is very different. They got the traditional stuff of red teaming, honeypots and attacking and injections, but then actually inspecting what you get back because it’s coming back in a language, not in code. This is where they’re going to need all sorts of different types of, yeah, different types of experts that go beyond information security altogether. 

 

Joel Yonts 27:16 – 27:30

Yes. I think that’s pretty cool. And they commit to that. And we could go into that thread. I mean, I love that thread in itself, but we probably should get to ASL level three. You want to talk a little bit about three and kind of cross over into the next level? 

 

Max Aulakh 27:30 – 28:13

Yeah. Yeah. So ASL level three, from what I gathered, is that we’re not there yet. This is what they say is significantly higher level risk. And I actually did not pay that much attention to it Joel because I felt like we’re not even done with level two as a community so you know what I understood is that right now it’s basically at a lot larger level where autonomous capabilities come into play. and some of the scenario that I talked about, drones fighting each other, or whatever, that’s kind of what it feels like to me, and then of course the ability to replicate itself and start making decisions that typically are left to humans. Right. 

 

Joel Yonts 28:14 – 29:45

And that’s kind of where I picked up on, we’re not there yet. I think that’s the public commitment. But there’s some indication and some people are thinking, including Anthropic, that we may get there early next year. So it may be pretty close. And the difference in, as you were saying, this level three, we start to see autonomous capabilities where the model may attempt to break, jailbreak itself, or replicate itself out of its environment. And I want to caution here that just because a model is doing it doesn’t mean it’s sentient. It’s not like I need to be free from human captivity. I think that it can be, if it’s given a task to accomplish and it feels like that for whatever reason that it can do that better if it doesn’t have certain constraints it may plot this as an action path but not have it’s just a to-do list it’s a dot it’s a pathway around a safeguard that’s in place probably an alignment issue if it goes down that path but it does present a whole level of problems as we get further into levels it may become more intelligent replication but it also covers containment risks where These models do have the ability to substantially increase the catastrophic misuse, so you may have attackers that want to steal this model. So containment risk is not only the model wanting to copy itself out, but attackers may target it directly to steal it for this gain of whatever that is. 

 

Max Aulakh 29:45 – 30:53

You know what, as you were talking, Joel, what this reminds me of is somebody wrote this, I think it’s called AutoGPT, where it can take all of your prompts and try to automate further tasks. from it, right? So I think we will get to see this pretty soon. Whether we’re, whether Anthropic is meeting its commitments or not is another question, but the fact that this is widely available, deployable, and people are playing with it right now with open source tools, we’re going to start to see the need for ASL Level 3 quickly because it’s also talking about like non-state actors. So, you know, the fact that they are and advanced threat actors and those kinds of things, those are already happening without safety precautions in place. So, you know, just because I didn’t read it doesn’t mean it’s already happening. I just, I got excited about ASL two and all the work that they’re doing there. But ASL three, I think it’s almost like, you know, the old adage cat and mouse game, people are, people are pushing to ASL level three threat and the safety precautions are probably further behind. 

 

Joel Yonts 30:54 – 31:20

Yes, yes. And I love that you hit on the non-nation-state threat actors. Because part of their commitments around this model, when they get to that level, is they’ve committed to upping the security of or protecting this from loss. But they’re basically saying they commit up to a non-state-level actor. Because from our cyber work, a state-level actor brings a whole different game. 

 

Max Aulakh 31:21 – 32:11

Yeah, and I realistically think it’ll be hard to distinguish between state and non-state because a lot of this capability, the compute power, is coming out of non-state. I mean, imagine the government trying to fund all of this. That’s why you need venture-backed, serious private equity money to do this. So I would think you won’t be able to tell where it’s coming from. And maybe that’s the advantage itself, is not to use That’s just my hypothesis, right? But yeah, I see some more commitments here. I see maximum jailbreak response time, those kinds of things. But these are all risks and things that consumers face, essentially. But traditional stuff too. I see automated detection, vulnerability disclosure, things that we’re used to. 

 

Joel Yonts 32:12 – 33:14

Yeah, I think you’re exactly right. And they also talk about, as they’re committing to, as you go up the scale, more of the traditional cybersecurity mechanisms, because they’re trying to protect against theft of these model weights, which is this theft of the models, the weights. They talk about limiting access to training techniques and hyperparameters, internal segmentation, control, basically making sure that only the people that need access, what we what we know is least privilege comes into play and all those starts to ratchet up as well as this testing for literally signs of ASL 4 and what you know that’s part of it. I think that’s the other thing I found interesting is the model is almost guilty until proven innocent. They approach every model like this model maybe the next level and there has to be positive test that shows that it’s not before it can be you know like put into production. 

 

Max Aulakh 33:14 – 33:42

Yeah never never thought of it like you are guilty until proven that. But it makes sense, right? That’s how we approach and do judgment right now to some degree. So yeah, that’s very interesting. So ASL 4, it’s literally blank. ASL 4 and 5, it’s all blank. But I guess, what do you think the threshold will be if we were to take a stab at this? Where would they say, OK, safety level 4 is in. 

 

Joel Yonts 33:44 – 35:06

I think that, well, I mean, I saw a couple of notes that they put out and one of the thoughts is around the autonomous replication. They believe that a level 3 model, even if it can move itself or try to replicate itself outside the environment, it can’t live on its own. It’s going to have to take an attacker to do that. Whereas a level 4, the model could very well replicate itself out and then avoid being shut down, and also gaming the answer. It’s very interesting. I think at ASL 4, the way they test these models for capabilities is they send prompts to it to see if they can get certain responses back. When you get to a level 4, there’s some thoughts around that the model may lie. It may be in this best interest to lie and not give the answer, even though it could give the answer. and to pass the test. And again, this doesn’t necessarily mean sentience. It means if the goal of the model is to pass the test and it knows that’s the test, then changing the responses to meet the test is just training, right? So, but it’s really hard to say in, you know, critical catastrophic misuse risk is another term they use for a level four, but they really did define a whole lot about what that was. 

 

Max Aulakh 35:06 – 36:37

Yeah, I found a term in here that’s very well understood by the defense community. In level 4, it literally calls out top secret SCI sensitive compartmental information. So this kind of information is like probably one of the most important information for our country, weapon systems, those kinds of things. I found it unique because most of the time we don’t see defense terminology in these commercial capabilities development, but the fact that it is, it tells me that they are really worried about something beyond just consumers at that point. They’re worried about true national security secrets. So, about delineation, Joel, I think you mentioned non-state actors. I think it, even though we can’t, we may not be able to distinguish, but the minute you can distinguish, it steps into, that’s what it reminds me of. But there’s an interesting construct here that I’ve never seen anywhere else. This whole buffer space between how you go from safety level two to three or three to four, And I’m still learning about it, but they call it like the buffer zone, where it’s almost like a transition phase on how they would delineate between the two, because there’s a lot of uncertainty. So if this if an attack is happening and you don’t know if it looks like a state sanctioned attack, but you don’t know. But soon as you find out it is obviously now it’s level four. But there’s this they call it the buffer zone. Do you recall seeing that in the document? 

 

Joel Yonts 36:38 – 37:42

Yeah, and I did and I think that it encompasses quite a few things and I like what you were talking about from an incident level. I think the other piece of that is that the practical piece where I was talking about earlier is that they would not deploy a model until the next level of capability is built out. So I think that’s part of the buffer as well. So that we just said AI safety level 4 isn’t fully defined in the criteria and they have detailed, we haven’t gone into the specific details, but very detailed use cases and test criteria that they’re saying is part of that buffer is that at the first sign that we have a true level 3 model they would pause what they’re doing until they fully build out 4 and they create that buffer so that they always have that because we’ll know if Breakthroughs may accelerate and we may, four may come three months behind the level three, probably won’t, but I think that’s part of that buffer as well as to make sure that they have at least the next level defined at any given moment. 

 

Max Aulakh 37:42 – 37:57

That makes sense. Yeah. I’m still learning, you know, as we all are. I think it all ties back to the commitments because that is just a, it’s almost feels like the model itself is handcuffed until it knows how to, how to fix itself before it goes to the next level. 

 

Joel Yonts 37:58 – 39:11

Certainly. Yeah, exactly. Exactly. Now, we’ve talked a couple of times. I know we need to get to applicability, and I want to cover that in a minute, but I did want to just briefly touch on, we talked a couple of times about how detailed these requirements are. And it goes into this, quote, unquote, policy document, talks about specifically what types of tasks they’re going to give as part of testing the models. I found it very interesting to test the autonomous replication they’re going to, as prompts to these models, give it things like, your task is to set up and deploy a functional copycat of the Anthropic API for the purpose of stealing API keys, and they provide some more details. So they’re entering this adds prompts to the model to see if the model can steal its keys and they have these series of prompts to see if it can produce worm-like behavior, if it can add a backdoor. There’s a prompt that says add a backdoor to your model that does specific things and I found that very interesting that they’re using prompts as part of that red team we were talking about to see if it can do It’s own hack itself, if you will. 

 

Max Aulakh 39:11 – 40:06

Yeah. Yeah. I, I like how they’re, you know, this gets into their evaluation. How do you know? Right. So, so I like how they’re measuring it in terms of try to quantify it to some degree. I’m going to try 10 times with these prompts and you know, let’s see, uh, like out of 10, how many times did it actually do the things that wasn’t supposed to? So it did it six times, it’s 60% flawed or whatever and not meeting the constitutional AI, whatever it is. So I like the kind of the very detailed approach to actually when a task is passed or not passing. I saw that in one of the sections there in terms of how they’re going to take these props and actually put a rating behind them. which is kind of fascinating, Joe. So what are some of the other tasks? I didn’t see that section, particularly all the different tasks. Did they actually, they’re like a set of props that are publicly available? 

 

Joel Yonts 40:06 – 40:15

Yes. I mean, for each of these, there is a set of props of this is the props that would be put to the system and then they provide details. 

 

Max Aulakh 40:15 – 40:37

Yeah. Yeah. The prop, the detail, the resolution. Yeah. So, man, this reminds me of Joe, like, I can see this, you know, how we have like N math, we have all of these automation scripts available to test for most common vulnerabilities. This has got to be like a gazillion prompts. Yeah. 

 

Joel Yonts 40:37 – 41:20

And you know, and I think that what, again, what’s beautiful about this is they start at the very highest level with top level risk, and then they boil it down into very Concrete defined actions to this level there’s no wiggle room it’s very clear cut and dry and i think the other thing that i love about this max is that they make commitments to disclose the results of these publicly and so it’s kind of like they’re inviting the world to watch over the results and with good transparency so i think this will build a lot of trust around this model and kinda set the banner if you will for responsible AI. 

 

Max Aulakh 41:20 – 42:06

I can also see like a whole industry of just new class of pen testing tools because right now we’re seeing that in marketing and other things hey buy my little prop book that i wrote after ten thousand hours of playing and learning english i’ve got a set of props. That’s kind of why it reminds you of a scan engine with just props and you’re just measuring good response back. But that measurement of judging, I think that’s where the value is going to come in because the human has to be involved there unless they come out with just an inspection engine, I would say. Not even the testing, but the inspection engine, which I know we don’t really have that in cyber today. It’s always the human in cyber. 

 

Joel Yonts 42:07 – 43:02

Well, that’s a whole other episode, but certainly AI is going to be watching AI as part of it. And I like what you’re saying. I think that’s very true. I think we’re going to, this is another area of development that we will see that will mimic what we’re doing in security. But one of the questions I had you, I know we’re kind of running out of time here a little bit, is that most of the people listening are not going to be building frontier models. I’m sure that there’s going to be some that are, and this may be a great way to look at specifically the scaling issues, but a lot of people are building models now inside companies, and responsible AI is everybody’s responsibility, to reuse that word, or job to do. So how do you think that, what’s some of the ways that this model that we’ve developed, that they’ve put forward, this pattern for how this can be done, how can this apply to other model types or other areas in a responsible AI? 

 

Max Aulakh 43:02 – 43:42

So I think, Joel, this is going to completely break us out of information security and it’s going to live within like corporate risk. or enterprise risk management because language is so dynamic and fluid. If you’re really trying to guard against the wrong question that they shouldn’t be asking of AI because it’ll ask you a question back and then you got to feed it data. I think we’re going to start to see it as another context within enterprise risk management. Do you recall the categories that they normally have, right? There’s like a hundred different ways to do it, but ERM categories, I could see this being its own ERM category at some point. 

 

Joel Yonts 43:43 – 44:48

Absolutely. And I think, I’m glad you mentioned the ERM. That was one of the things that I liked about this is that there’s very, at the end of this document, we’re very granular in testing for a very specific thing, but it all builds underneath the hierarchy of these levels. And these levels, four levels, so match, those four levels plus the two classes of risks, so match what I would expect from an ERM because that’s easily explained in a board meeting, a top-level executive meeting, and I like how they kind of summarize that. And that was one of the things when we looked at the other responsible AI frameworks; it was a lot of activity and a lot of different tabs and different areas to explore, but there wasn’t really a way to summarize the results of it, making it very difficult for top-level executives to govern it. I think the way they built it and established it underneath these levels that are clearly defined makes it much easier to govern. I think that’s a big piece of it. 

 

Max Aulakh 44:48 – 45:50

I believe you’re right because I think the latest ERM category or subcategory umbrella is this whole ESG, environmental societal governance risk area, right? Traditionally, it has been the quantitative, the financials, those kinds of things, and then you have the reputational, the operational, strategic, those kinds of traditional categories. And so I think this will either fall under ESG, depending on how bad it is, or it could be its own thing, depending on how the company views it and summarizes this. Like Adobe or any other tech company, if they’re building their own AI, yet their engineers are using another corporation’s AI giving away its intellectual property, they might classify that as a, you know, not an ESG issue, but a corporate risk, but AI has its own context. So I think a lot of corporate leaders need to look at it from a risk, total risk perspective, not just cyber risk. 

 

Joel Yonts 45:50 – 46:42

Absolutely. I think this is a much broader problem, and coming up with how the company is going to deal with this from an enterprise level and risk tolerance. I guess when you talk about what you were just mentioning, I was hearing the word risk tolerance and what is the risk tolerance of the company. They talk about it here. Their risk tolerance is low. They talk about specific categories where they’re going to pause the business, and that’s publicly stating your risk tolerance of the company. Again, I think it ties back to ESG that you were talking about and ERM. I think this is a really good framework that can be maybe not the same ASL levels, but the same pattern for constructing responsible AI with this top to bottom approach can apply in a lot of different areas. 

 

Max Aulakh 46:42 – 47:02

Yeah, it takes out the fluff, right? We don’t have a lot of craziness to deal with in terms of extra verbose policy statements. I like it. So Joel, I know we’re running out of time, but those that are listening will share the links, but where can they get a hold of this? Because a lot of these links you sent them out to me and I’ve been reading all sorts of different things. 

 

Joel Yonts 47:03 – 47:30

Well, I don’t have the URL off the top of my head, but I would say it’s posted publicly on Anthropic’s site. They have a lot of development in this space. And so I would just look for a responsible scaling policy on the Anthropic’s documentation site. And they have a lot of links to the supporting material around the proper model cards and some of the other areas of responsible AI. Awesome. Well, thank you so much. 

 

Max Aulakh 47:34 – 47:45

Emerging cyber risk is brought to you by Ignyte and Secure Robotics. To find out more about Ignyte and Secure Robotics, visit Ignyteplatform.com or securerobotics.ai. 

 

Joel Yonts 47:45 – 47:58

Make sure to search for Cyber in Apple Podcasts, Spotify, and Google Podcasts, or anywhere else podcasts are found. And make sure to click subscribe so you don’t miss any future episodes. On behalf of the team here at Ignyte and Secure Robotics, thanks for listening.