Remember where we were a year ago.<\/p>\n
World leaders gathered for the world\u2019s first AI Safety Summit and\u00a0concluded<\/a>\u00a0that powerful AI could have the \u201cpotential for serious, even catastrophic, harm.\u201d<\/p>\n The UK\u2019s cyber agency\u00a0warned<\/a>\u00a0that \u201cAI will almost certainly increase the volume and heighten the impact of cyber-attacks over the next two years.\u201d<\/p>\n Demis Hassabis, at this point yet to\u00a0receive<\/a>\u00a0his Nobel Prize,\u00a0called<\/a>\u00a0for government to treat \u201cthe risks of AI as seriously as other major global challenges, like climate change.\u201d<\/p>\n And the UK Government responded by\u00a0launching<\/a>\u00a0the world\u2019s first AI Safety Institute.<\/p>\n But there was no playbook for what an AI Safety Institute should be.<\/p>\n A year later, we have built a world leading government organisation for understanding AI threats. We run state of the art evaluations to measure potentially dangerous AI capabilities. We use these insights to create an international approach to defining and measuring these risks. And we conduct world-leading research to understand how these harms will impact our society.<\/p>\n On our first anniversary, we explain what we have built and where we are going next.<\/p>\n \u200d<\/p>\n AI safety isn\u2019t sci-fi. If we\u2019re concerned that powerful AI models might be used for cyber attacks, or might resist human control, or might be used to persuade and manipulate populations, then let\u2019s\u00a0measure<\/em>\u00a0that risk. Let\u2019s introduce empiricism into debates about AI safety.<\/p>\n That\u2019s the goal of the AI Safety Institute.<\/p>\n Our mission is to equip governments with an empirical understanding of the safety of advanced AI systems.<\/p>\n We work with some of the most innovative, fast-moving tech companies in the world. We particularly focus on the gravest of risks which are the sober responsibility of governments. We have bottled the agility, innovation and ambition of the former with the expertise and gravitas of the latter, to create a cutting-edge start-up in government.<\/p>\n Over the last year, we\u2019ve channelled our efforts into three lines of work: becoming the leading authority on the science of assessing model safety; galvanising the AI safety research field; and working with other governments and AI companies to develop a set of global standards.<\/p>\n <\/p>\n Are the most powerful AI models dangerous?<\/p>\n We wanted to build the tools to answer that question, and to refresh that answer as the models get more powerful.<\/p>\n So we\u2019ve built\u00a0evaluation suites<\/a>: sets of tests which we can run on the most powerful AI models to understand their capabilities and safety. Our evaluations target those faculties which would cause us most concern: cyber-attacks, chemical and biological misuses,\u00a0autonomous agent capabilities,<\/a>\u00a0safeguards, and the impacts on society.<\/p>\n Building these evaluation suites has taken thousands of hours from a dedicated and brilliant team. It has paid off: we are now world leading. We now have the expertise, proprietary tools, and infrastructure in place to run multiple testing exercises at once, quickly and rigorously, for a range of frontier model families.<\/p>\n That\u2019s why OpenAI, Google DeepMind and Anthropic are among the companies that have worked with AISI to test their most advanced models. And we use these tools to regularly evaluate publicly available models to give us a snapshot of the frontier of AI capabilities.<\/p>\n In our first year, we\u2019ve conducted evaluations on 16 models.<\/p>\n We call these \u201csuites\u201d because there is no one test which is\u00a0the\u00a0<\/em>test. This is still very much an open science, and we\u2019re finding that we need to combine a range of different methods to get a comprehensive view of a given model\u2019s risk profile. Our\u00a0automated benchmarks<\/a>\u00a0fire hundreds of questions at a model to get a quick read on its capabilities. Our red-teaming puts threat-experts in front of the model for in-depth, scenario-based assessments. And we run human uplift studies \u2013 akin to randomised control trials \u2013 in which \u2018representative users\u2019 (e.g. novices; or computer science undergrads) use AI and we measure whether this gives them a boost in completing potentially harmful tasks.<\/p>\n These tests are not \u201cgovernment safety certificates.\u201d We never endorse the safety of a particular AI model. Nor can we release all our results into the public domain, since they often contain sensitive material.<\/p>\n But these evaluations give governments an insight into the risks developing at the frontier of AI, and an empirical basis to decide if, when and how to intervene.<\/p>\n Our work here is not done. Evaluations are a new science. We\u2019ve\u00a0published<\/a>\u00a0some of our lessons from our first year; in the next year, we want to hone our skill, add new threats to our evaluations suite, and work with more companies to test the world\u2019s most powerful AI models.<\/p>\n <\/p>\n Will the world be ready to respond if the risks from frontier AI increases?<\/p>\n AI systems may be developed in individual countries but once released, they\u2019re used all over the world. There\u2019s no route to safe AI from one country acting alone.<\/p>\n It\u2019s crucial that if AI risks increase, the international community has a developed reflex for responding. That reflex would rely on agreeing, in advance, what harms we are concerned about and the valid methods of detection.<\/p>\n That\u2019s what we, with other government partners, are building.<\/p>\n We started by launching an independent, international review of the science of AI safety. Yoshua Bengio chairs a drafting team which draws on the expertise of 30 countries alongside the UN and EU. The UK AISI provides the Secretariat. The aim is to create a shared, international baseline for understanding AI safety. We\u00a0published<\/a>\u00a0the interim report in May. The final report will be published before the Third AI Safety Summit in Paris next year.<\/p>\n That gives us today\u2019s baseline. Next, we need to define tomorrow\u2019s red lines. What\u00a0don\u2019t\u00a0<\/em>we want AI models to be able to do?<\/p>\n The UK co-hosted the second AI Safety Summit in Seoul in May, in which 28 international\u00a0partners<\/a>\u00a0agreed that there are some capabilities that frontier AI models should not exhibit without appropriate mitigations. The negotiated text specifically called out offensive capabilities relating to chemical and biological weapons, and the potential ability of models to evade human oversight. Now we are working with industry, academia and international partners to flesh out specific thresholds.<\/p>\n If we understand the baseline, and we can define the red lines, then the remaining gap is to agree how we track the moving frontier, and how we respond as we approach or cross these red lines.<\/p>\n A lot of this response is about building the muscle of working together as an international community \u2013 particularly among governments who are taking these risks seriously.<\/p>\n While the UK was first to establish an AI Safety Institute, we were not the last. There are now 10 AISI equivalents coordinated into the\u00a0International Network of AISIs<\/a>, which will meet\u00a0for the first time next week<\/a>. We\u2019ll be discussing how we measure and mitigate AI risks and start building the institutional muscle of coordination and cooperation on the science of AI safety.<\/p>\n We\u2019ve already shown that it is possible for governments to cooperate on technical safety work. In April, we signed an\u00a0agreement<\/a>\u00a0with the US AISI to create an \u2018interoperable\u2019 capability. We\u2019ve since conducted several joint testing exercises with the US government and are working together to taxonomise risks and define best practices for managing them.<\/p>\n These international efforts require contributions from the AI developers. It is these companies, after all, who are ultimately responsible for the safety of their models. At the summit in Seoul, we\u00a0secured commitments from 16 AI companies<\/a>\u00a0to identify, assess and manage AI risks. We\u2019re advancing these Frontier AI Safety Commitments at our\u00a0upcoming conference<\/a>, in which AI developers can compare their approaches to developing their safety policies.<\/p>\n \u200d<\/p>\n There\u2019s a lot of raw science left to be done to guarantee the safety of AI models. So, our third and final effort focuses on galvanising orders of magnitude more research by others on urgent and important questions.<\/p>\n Some of this is about working directly with talented researchers. We recently launched an\u00a0open call<\/a>\u00a0for collaborators on\u00a0key research areas<\/a>\u00a0like safeguards, science of evaluations, and safety cases. We\u2019ve launched\u00a0a bounty<\/a>\u00a0to develop novel evaluations and agent scaffolding. And we\u2019re funding research into the\u00a0systemic risks<\/a>\u00a0associated with AI.<\/p>\n But we also want to put tools in the hands of AI safety researchers everywhere. That\u2019s why we open sourced our evaluations platform,\u00a0Inspect<\/a>. Inspect helps us run evaluations quickly and easily: now it is used by other governments and some of the leading AI developers.<\/p>\n Today we\u2019re going further, and\u00a0releasing dozens of evaluations in Inspect<\/a>. If we can make AI safety research easier, we can make the science go faster.<\/p>\n \u200d<\/p>\n We\u2019ve achieved a lot in a year. More than anything else, this is a testament to the extraordinary people who have come to work at AISI.<\/p>\n We\u2019ve had to move fast because AI moves fast. Our evaluations show AI models getting more powerful quarter by quarter.<\/p>\n By creating a technical authority on AI safety, driving global AI safety standards, and supporting wider research efforts, we’re working to ensure that AI\u2019s harms don\u2019t foreclose its potential.<\/p>\n The government has now set out its long-term vision for AISI\u2019s future and its plans to move the organisation onto a statutory footing. This will give AI developers long-term clarity on how they should work with us.\u202f It is a critical step towards giving us the power and independence required to enhance the safety of AI over the long term.<\/p>\n We are only a year into this journey. There is so much more work to be done.<\/p>\n If you want to do that work with us, we\u2019re\u00a0hiring<\/a>.<\/p>\n Originally posted here<\/a><\/p>\nEmpirical mission<\/h3>\n
A technical authority on AI safety<\/h3>\n
Shaping global AI governance<\/h3>\n
Promoting AI research<\/h3>\n
Conclusion<\/h3>\n
\n