Paper deep dive
AI Systems of Concern
Kayla Matteucci, Shahar Avin, Fazl Barez, SeĂĄn Ă hĂigeartaigh
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/12/2026, 5:28:45 PM
Summary
The paper introduces 'Property X', a cluster of characteristics in AI systemsâincluding agent-like behavior, strategic awareness, and long-range planningâthat are hypothesized to be intrinsically dangerous. The authors argue that as AI capabilities grow, systems high in Property X pose significant safety and control risks. They propose a governance framework that uses capability indicators to identify systems requiring assessment and suggests steering AI development toward 'low-Property X' configurations to ensure safety while maintaining the benefits of advanced AI.
Entities (5)
Relation Signals (3)
Alignment Research Centre â evaluated â GPT-4
confidence 100% · prior to OpenAIâs release of GPT-4, an external evaluation by the Alignment Research Centre tested for risky emergent behaviours.
Anthropic â reportsobservationof â Power-seeking behaviour
confidence 95% · Anthropic, who have reported observing AI systemsâs sycophantic behaviour and stated preference to not be shutdown.
Property X â islinkedto â Intrinsic Danger
confidence 90% · Property X, which is strongly linked to potential intrinsic danger from advanced AI systems.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Concerns around future dangers from advanced AI often centre on systems hypothesised to have intrinsic characteristics such as agent-like behaviour, strategic awareness, and long-range planning. We label this cluster of characteristics as "Property X". Most present AI systems are low in "Property X"; however, in the absence of deliberate steering, current research directions may rapidly lead to the emergence of highly capable AI systems that are also high in "Property X". We argue that "Property X" characteristics are intrinsically dangerous, and when combined with greater capabilities will result in AI systems for which safety and control is difficult to guarantee. Drawing on several scholars' alternative frameworks for possible AI research trajectories, we argue that most of the proposed benefits of advanced AI can be obtained by systems designed to minimise this property. We then propose indicators and governance interventions to identify and limit the development of systems with risky "Property X" characteristics.
Tags
Links
- Source: https://arxiv.org/abs/2310.05876
- Canonical: https://arxiv.org/abs/2310.05876
Trouble viewing inline? Open PDF directly â
Full Text
40,300 characters extracted from source content.
Expand or collapse full text
___________________________________________________ AISystemsofConcern KaylaMatteucci 1 ShaharAvin 1,2 FazlBarez 1,3 SeĂĄnĂhĂigeartaigh 1 ABSTRACT ConcernsaroundfuturedangersfromadvancedAIoften centreonsystemshypothesisedtohaveintrinsic characteristicssuchasagent-likebehaviour,strategic awareness,andlong-rangeplanning.Welabelthiscluster ofcharacteristicsasâPropertyXâ.MostpresentAIsystems arelowinâPropertyXâ;howeverintheabsenceof deliberatesteering,currentresearchdirectionsmayrapidly leadtotheemergenceofhighlycapableAIsystemsthatare alsohighinâPropertyXâ.WearguethatâPropertyXâ characteristicsareintrinsicallydangerous,andwhen combinedwithgreatercapabilitieswillresultinAIsystems forwhichsafetyandcontrolisdifficulttoguarantee.Drawing onseveralscholarsâalternativeframeworksforpossibleAI researchtrajectories,wearguethatmostoftheproposed benefitsofadvancedAIcanbeobtainedbysystems designedsoastominimisethisproperty.Wethenpropose indicatorsandgovernanceinterventionstoidentifyandlimit thedevelopmentofsystemswithriskyâPropertyXâ characteristics. KEYWORDS Safety;Risk;Futures;Governance 1 Introduction Forcenturies,humanshavebeenimaginingmachinesthat areasintelligentormoreintelligentthanhumans.[1]The possibilityofcreatingsuchmachinesisdifficulttoascertain, buttechnologicalinnovationsâincludingthecreationof remarkableartefactssuchasYOLO,[2]AlphaGo,[3] GPT-3,[4]DALL-E,[5]andAlphaFold[6]âhavecaused bothexcitementandconcernaboutthefutureofartificial intelligence(AI).[7][8][9][10][11]ThesuccessoftheDeep Learning,[12]whichunderpinstheaforementioned artefacts,hascontributedsignificantlytothisgrowthin attention,ashastheincreasingapplicationofAIsystemsin agrowingrangeofdomains. [13][14] Moreover,inlargepartbecauseofthediscoveryof scalinglaws[15][16]andtheemergenceofsurprising performancefromlargescale"foundationmodels",[17][18] moreexpertsarenowpredictingtransformativeAI capabilities[19]-capabilitiesthatcouldbringabout transformationassignificantastheindustrialrevolution- withindecades.[20][21][22]Ofcourse,thereremains ampledisagreementaboutwhatleveloftransformationwill becausedbyfutureAIsystems,andwhetheritispossible tocreateAIsystemswithsuperhumancapabilitiesacross domains.Weworkontheassumptionthatthereisa plausiblechanceofdoingsoand,giventhis,webelievethat itisimportanttoconsiderthepotentialimplicationsofsuch technologies. Specialattentionhasfocusedonthepossibilityofsuch advancedsystemsbeingdangerousâperhapseven catastrophicallyso. Whilethereisyetnoconsensusonwhatmightmake advancedAIsystemsdangerous, severalworkspointtoa clusterofpropertiesaround"agent-likebehaviour",[23][24] "consequentialreasoning",[25]"planning",and"strategic awareness".[26][27]Letuslabeltheproperty,orsetof properties,thatmakeanadvancedAIsystemdangerous "PropertyX".LetusalsoexpectthatpropertyX ismadeup of,orissimilarto,thosepropertieslistedabove. 1 CENTREFORTHESTUDYOFEXISTENTIALRISK,UniversityofCambridge,3UniversityofOxford 2ShaharAvinissecondedtotheUK'sDepartmentforScience,Innovation,andTechnology.Thispaper,written beforethesecondment,doesn'trepresenttheUKgovernment'sviews. Correspondingauthor:so348@cam.ac.uk Figure1:SimplifiedlandscapeoffutureAIsystemsand risk. SystemspossessingrelativelyhighlevelsofpropertyX requireustosolvethe"alignmentproblem"[28][29]before theyaredeveloped,orotherwiseriskcatastrophic consequences(Fig.1).Unfortunately,therearesome compellingargumentsastowhydevelopersmightbe incentivisedtodevelopsystemsthathavemoreofproperty X.[30][31] Wecanmakethispictureclearerwithananalogy. InsteadofAI,wecanthinkofpowergeneration.Inthis case,theverticalaxiswillbethetotalinstalledgeneration capacity,andthehorizontalaxis("propertyX")willbe carbonemissions.Weenterintoadangerouszoneifwe havebothahightotalgenerationcapacity,andalarge enoughamountofhigh-carbonenergygeneration â together,thetworesultinenoughcarbonbeingemitted todrivethegreenhouseeffectandadverselychangethe globalclimate.Ifwehadtechnologiesthatallowedusto gaintheenergybenefitoffossilfuelswhileavoidingthe emissions(e.g.perfectcarboncapturetechnology),that wouldbeanalogousto"solvingalignment".Absentsucha solution,ifwewanttoavoidtheharmsofclimatechange, weeitherneedtocaptotalenergyuse,orcreateincentive structurestosteerpowergenerationtowardslow-carbon energysources.Giventhemagnitudeofriskand uncertaintyassociatedwitheachstrategy,weshouldideally pursueallthree:carboncapturetechnology,renewableand low-emissionpowergeneration,andreductionin consumption. Similarly,ifvariousAIR&Dprojectsareinfactoccupying pointsonthesimplifiedlandscapedepictedinFig.1,this suggestsafewstrategiesforavoidingcatastrophic outcomesresultingfromadvancedAI: 1.Placeandenforceacaponhowadvancedeach project'sAIcapabilityshouldbe 2.Placeandenforceacaponhowmuchof"propertyX" eachAIsystemshouldhave 3.MaintaininvestmentinAIalignmentresearchwhile 1.slowingdownthepaceofcapabilityprogress toallowmoretimefordevelopingAIalignment solutions, 1.directingprojectstowardslow"propertyX" configurationstoallowmoretimefor developingAIalignmentsolutions. Inthispaperweexploreoption3b.Westartwithan explorationof"propertyX",basedontheconceptof "agents"andrelatedideasintheliteraturethatconsiders dangerous,advancedAIsystems.Then,weexplorea rangeofpolicyleversthatmightsteerAIdevelopment towardssystemsthathavelessof"propertyX". 2 PropertyX Scopingsystemsofconcerncanbeimaginedasafunction ofboththeproblematicattributeswithinacontainedsystem andthedangersthatmayarisewhenthatsystemis introducedintopotentiallydangeroussettings.Here,we drawuponadefinitionofriskasafunctionofhazards, exposure,andvulnerabilities.Hazardsrefertotheconcrete attributesofanAIsystemthatmightmakeitintrinsically dangerous.ExposurereferstothecontextinwhichtheAI systemisdeployed,ortheextentoftheareainwhichthe hazardcouldhaveanimpact.Vulnerabilityreferstothe levelofpreparednesstoavoidandmitigatethehazard. Givenourfocusontheintrinsicdangerofsystemsthat arehighinpropertyX,ourprimaryemphasisisonhazards. (Tonarrowouranalysis,weexcludesystemsthatarelowin propertyXâeveniftheycouldbehazardous,forexample, whenintroducedintodangerousenvironmentswithhigh vulnerabilityorexposure.)Still,wewilllaterproposethata regulatorâsjobextendsbeyondscrutinisingtheisolated characteristicsofAIsystems;theymustalsoconsiderthe potentialusesofsuchsystems,aswellasrelevant organisationsâpreparednesstoidentifyandrespondto signsofdanger. Buildinguponthisdefinitionofrisk, wecanfurther compartmentaliseAIrisksintoaccidents,misuse,and structure.[32]Inaccidents,anAIsystemisnotworkingas intended,andthiscausesharm;inmisuse,theoperatorof anAIsystemisusingittocauseharmtoanother;in structuralharms,theAIsystemisoperatingasintended, andtheoperatordoesnotseektocauseharm,butthe interactionbetweenthesystemandtheworldresultsin unintendedharm(e.g.perpetuatingexistinginequalitiesor contributingtounemployment). Thisbreakdownseparatesthreefactors:theAIsystem, theoperator,andthecontext.Foroperatorandcontext, somefactorsincreaserisk.Forinstance,someoperators aremotivatedtocauseharm(e.gterrorists,oppressive governments),oratleastinsufficientlymotivatedto anticipateandmitigateharms(e.g.profit-maximising companies,narrowly-focusedbureaucracies). Furthermore, somecontextshavemorepotentialforharm(e.g.defence, criticalinfrastructure,medicine)oraremoresensitiveto pre-existingstructuralproblems(e.g.finance,education). CanwefindsimilarpropertiesthatmakespecificAIsystems moredangerous,whenconsideredindependentlyfromthe operatorandthecontext? Severalauthorshavearguedthatindeed,therearesuch propertiesthatwouldmakefutureAIsystemsmore intrinsicallydangerous.Earlyargumentsfocusedon InstrumentalRationality:theselectionofactionsthatare assessedtohavethehighestlikelihoodofbringingabouta certaingoal.ViewsofAIsystemsasoptimising goal-seekersfitwellwithcommonframeworkssuchas reinforcementlearning,[33]andhavegainedtractionin thinkingaboutmoreadvancedAIsystemsthrough frameworkssuchasAIXI.[34]Thisframingraisesthe concernofconvergentinstrumentalgoals,[35][36][37] suchasshutdown-avoidanceandresourceacquisition.In theabsenceofsufficientrestrictionsontheseinstrumental goals,asystemcouldleadtocatastrophicoutcomes.[38] ViewingAIsystemsasdisplayinginstrumentalrationalityis linkedtoseeingthemasagents,andseveralauthorshave recentlyarguedthatpropertieslinkedtoagencyarerelated tointrinsicdanger.[39] AnothercommonframingofintrinsicdangerfromAI systemsfocusesontheemergenceofpower-seeking behaviour.Thisinturnhasbeenlinkedtopropertiesof agenticplanning,anabilitytoformulateachainofactionsin pursuitofagoal,andstrategicawareness,an understandingoftheworldthroughalensofpowerand capacitytoinfluenceoutcomes.[40]Powerseekinghas beenshowntoarisenaturallyunderreinforcementlearning. [41][42]Thisisnowarecognisedconcernwithinleading AGIlabs,suchasAnthropic,[43]whohavereported observingAIsystemsâsycophanticbehaviourandstated preferencetonotbeshutdown.[44] ThecombinationofincreasedAIcapabilities,andthe emergentpower-seekingbehaviour,couldcreatea dangerousescalationdynamic,asdepictedinFig.1.By definition,power-seekingbehaviourisalwaysintrinsically motivatedtoincreasethesystem'scapabilitieswithinits accessibledomain.Ontheotherhand,morecapable systemsarelikelytoincludemorecomprehensivemodels oftheworld,andabettermodellingofcausalrelations withinit;thismightcontributetostrategicawareness,which canboostpower-seekingtendencies.Takentogether,we areconcernedwiththegrowingprevalenceofsystemsthat arebothextremelycapableandhavecontinuedtoseek powerâtrendswhichcouldthereforeleadtosystem behaviourthatonlyfurtherdevelopsthesetraits,potentially leadingtoexistentialrisks. Webelievethereissufficientcommonalityamongstthe clusterofinstrumentalrationality,agency,themixofagentic planningandstrategicawareness,andsimilarproperties suchasconsequentialism,tomarkitasPropertyX,whichis stronglylinkedtopotentialintrinsicdangerfromadvanced AIsystems,suchasthepursuitofconvergentinstrumental goalsandtheemergenceofpowerseekingbehaviour.Very likelythisisnotasingleproperty,butratheraclusterof linkedcharacteristics,whichmayevolveintime.Asyet,this clusterofcharacteristicsdoesnothaveaconsensus definition,norisitoperationalisedinawaythatcanbe directlyevaluatedandmeasured. Nonetheless,webelievethatsubjectiveexpert assignmentof"PropertyX-ness"todifferentAIsystemswill showahighdegreeofinter-rateragreement,andfor illustrationhavelistedtheauthors'assessmentsofseveral well-knownartefactsinTable1.Furtherspecificationand operationalisationof"PropertyX"isanactiveresearchtask, boththeoreticalandempirical.However,evenavague clusterofpropertiescanbereliedontodrivesafety-oriented policies,especiallyontheexpectationthatinthefuturewe willhaveabetterunderstandingofthisclusterandtoolsto evaluatespecificsystemswithregardstotheseproperties. Thelackofdefinitionalconsensusshouldnothinderpolicy interventions,whichthemselveswillservetoiterativelytest whichmetricsandindicatorsareusefulforthoseseekingto limitthedevelopmentofdangerousAIsystems. Table1:AuthorsâassessmentofdegreetowhichvariousAI artefactsexhibitPropertyX Technology Authorsâ PropertyX rating Rationale AlphaFold None/LowNoagency,nolong-term planning,narrowdomainof application. Facial Recognitio n None/LowNoagency,nostrategic planning,potentialforlimited strategicawarenessthrough modellingofhumanemotions. Teslaâs self-driving car software None/LowShort-termplanningwithina narrowdomain,potentialfor limitedstrategicawareness throughmodellingofother road-users'behavioursand intentions. A simulated, evolving nematode inan artificial environmen t LowAgentialplanningwithvery limitedstrategicawarenessin arestricteddomain. ChatGPT Low/MediumNoagency,nostrategic planning,potentialforstrategic awarenessthroughmodelling ofuserpreferencesand behaviours. CICEROMediumAgentialandstrategicplanning inarestricteddomain. GATOHighAgentialplanningandstrategic awarenesswithineachdomain ofapplication,acrossarange ofdomains. 3 Positivefutureswithlow-propertyX systems Severalauthorshavewarnedthat,despitePropertyX'slink todanger,therearestrongincentivestodevelopsystems withhighPropertyX:agenticplanningandstrategic awarenessallowsystemstohavehigherautonomy,more generality,andgreaterimpact,allcontributingtotheir economic,military,andR&Dpotential,especiallywhen facingcompetitionfromotherAIsystems.[45][46][47]The pursuitofsystemsthatcombine"humancompetition"(the goalofachievinghuman-levelcompetenciesinAIsystems, includingthoselinkedtopropertyX)and"autonomy"(which isdirectlylinkedtotheagenticaspectofpropertyX)has beendescribedasthedominanttechnologyparadigmof "actuallyexistingAI".[48] Nonetheless,therearepositivevisionsforsystemsthat remainlowonpropertyX(beyondtheobviousadvantageof avoidingextremerisks).Theauthorsthatdescribethe currentparadigmas"actuallyexistingAI"offerinsteada CollectiveIntelligencevisionofAIthatfocuseson "complementarity"betweenAIsystemsandhumans,as opposedtocompetitionandthereplacementofhuman intelligence,increased"participation"ofbothhumansandAI systemsincollectivedecisions,asopposedtoautonomy, and"mutualism",avisionofdecentralisationand heterogeneityasopposedtocentralisationofdecision makinginadvancedAIsystems.[49]Whiletheauthorsdo notprovideafuturisticvisionforthiskindoftechnology paradigm,theypointatWikipediaandTaiwan'sdigital democracyascurrenttechnology-enabledcollective intelligenceplatformsthatembodytheseprinciples,andthat onourreadingarebothpromisingandlowonpropertyX. Anothervisionforlow-propertyXsystemscomesfrom HumanCentred-AI.[50]IncontrasttopursuingArtificial GeneralIntelligence,whichischaracterisedasthepursuit of"machinecognition,autonomousagentsand commonsensereasoning"(highonPropertyX),Human CentredAIuses"designprocesseswithhumanstakeholder participationtocreatepowerfulAI-infusedsupertools, tele-bots,activeappliances,andcontrolcenters,which ensurehumancontrolofevermorepotenttechnologies" (lowonPropertyX).WhileHCAIisnotopposedto autonomy,andevenhighdegreesofautonomy,this autonomyisalwayscoupledwithhighlevelsofeffective humancontrol;onourreadingthismeansthatinpractice, thisdoesrequiregivingupontechnologicalartefactsthat wouldhavehighPropertyX.Thisstillleavesawiderange ofsmarttoolsandintelligentsupportsystemsthatempower theusertoachievemuchmorethantheycouldbefore. Finally,wewishtonoteworksthatarguethatthecurrent AIdevelopmenttrajectoryinfactpointsmoreinthedirection oflowpropertyXsystems.Onesuchvisionisof ComprehensiveAIServices,whichseesincreasing generalityandautonomyinthecreationofnovelAIartefacts (intheAIR&Dpipeline),butlimitedautonomyand generalityintheartefactsproducedbythispipeline.[51]On thisvision,ahighlygeneralandheavilyautomatedR&D pipeline,thatcouldforexampletrainawiderangeof modelsandautonomouslymakeefficienttrainingdecisions, isusedtogenerateaverywiderangeofservices,whether theyaredomain-specificlanguagemodels,protein-structure predictors,codingassistants,orautonomousdrivingagents. WhiletheAIR&Dpipelineitselfedgestowardshigher propertyX,itisoneormorestepsremovedfromdirect contactwiththeworld,whereastheartefactsproducedare loweronpropertyX.Amorerecentvisionisthatofopen agencies,whichpaintsasimilarpicturebutnow incorporatesgenerally-applicablefoundationmodels,which arethemselvesnon-agentic(andthereforeloweron propertyX),ascommoninterfacestointeractwiththe ecosystemofAIservicesandagents,eachofwhichis tailoredtoaspecificdomainandthereforeloweronproperty X.[52] Whatcouldthislooklikeinpractice?Foranytaskthat collectivehumanintelligencehasalreadyshownanability tomakeprogresson,weexpectthatinprincipleitshouldbe possibletomakegreaterandmorerapidprogressin combinationusingthetoolsofAIsystemslowonProperty X.WeexpectthistoincludeR&Dchallengessuchas materialanddrugdiscovery,diseasediagnosticsandpublic healthmonitoring;engineeringchallengessuchas sustainableenergyproduction(includingfusionpower generation),robustandsustainablefoodproduction,and spaceexploration;andcreativedomains,including assistancetothegenerationofvisualart,music,textand video.RestrictionsonpropertyXwouldmostlikelybefeltin domainsthatbenefitfromveryhighautonomyand long-termplanning,includinglong-termstrategicplanning (includingfinancialplanning),nationalsecuritystrategyand militaryoperationsinhostileenvironments,andautonomous scientificdiscovery.Wewouldalsobelimitedinourabilityto studyhuman(and"general")intelligencethroughthestudy ofartefacts. 4 Policyinterventionstosteerawayfrom highpropertyXsystems 4.1 CanWeLimittheDevelopmentof âSystemsofConcernâ? Ingeneral,existingresearchhasnotrigorouslyconsidered thepossibilitythatgovernmentsmightplayasubstantial roleinlimitingthecreationof AIsystemswithhighproperty Xâanemergingclusterofcharacteristicsbywhicha systemmightsimulateagency,reasoning,planning,and awarenessofitsbroaderenvironment.Todate,such characteristicshavebeenachievedonlytoaverylimited extentinAIsystems;andthushavehadalimitedcapacity toleadtoharm.However,presentadvancessuggestthat suchcharacteristicsmaybeseentoamuchmore significantdegreeinfrontierAIsystemsincomingyears. Workonconcretemechanismsandpolicyleversto implementsafeandethicalAI[53]hastendedtofocuson contextualrisks(e.g.bias,fairness,security)asopposedto intrinsicrisks(e.g.relatingtopropertyX).[54][55]The focusofproposedregulationisoftenatthepointofthe appliedAIproduct,orproductwithinwhichAIisused,rather thanatthestageofAIresearchanddevelopment.However, thepresentpaceofprogresssuggestsagreaterrolefor governancetoplayinthedevelopmentprocessofAI, particularlyforthefrontierAIsystemsthatmightbemost likelytoexhibitpropertyXcharacteristics. Asoutlinedinprevioussections,thedegreeofintrinsic dangerposedbyanadvancedAIsystemislinkedbothtoits generaldegreeofcapability(competenceandgenerality) andtoitspropertyX-ness.Here,wefirstintroduce indicators(Table.2)thatmightalertregulatorstothe developmentofsystemswithsignificantcapabilitieswithin theirjurisdictions.Weproposethatanevaluationofproperty Xbecarriedoutwithrespecttosystemsindicatedashigh capability,allowinginterventionandmitigationofrisks. Wethenproceedtodiscusspotentialframeworksfor limitingsystemsofconcern.Owingtoalackofempirical evidenceaboutthemosteffectivepoliciesforregulatingthe developmentofintrinsicallydangerousAIsystems,and giventhateachsiteofpolicymakingrequiresatailored approach,ourgoalistodiscussarangeofoptionswithout deferencetoanyoneofthem. Futureresearchwouldbenefitfromdrawingupon insightsfromotherhigh-stakespolicyareastoidentifyideal institutionaldesigns,incentivesanddisincentives,methods ofimposingoversight,andresistancethatmayarisefrom powerfulinterestgroups.Suchresearchwouldbe immediatelyapplicabletoensuringasaferandmoreethical R&DenvironmentforadvancedAIsystems. Wenotethatanumberofleadingorganisations developinglargemodelsarealreadyundertakingalignment research,riskassessmentsandred-teaminginternallyand withexternalcollaborators,aimedatmakingtheirAI systemssafebeforerelease.Someoftheseprocesses focusonconcernsthatmapcloselytotheâPropertyXâ characteristicswedescribeabove.Forexample,priorto OpenAIâsreleaseofGPT-4,anexternalevaluationbythe AlignmentResearchCentretestedforriskyemergent behaviourssuchaspower-seekingbehaviour.[56]There arealsocallsforexternalauditingfromwithinsomeofthese companies,suggestingaroleforgovernmentstoestablish suchauditingbodies.Forexample,Anthropicnotethatthey âplantomakeexternallylegiblecommitments[...]toallow anindependent,externalorganisationtoevaluatebothour modelâscapabilitiesandsafetyâ.[57] 4.2 IdentifyingandDetectingâSystemsof Concernâ Toimaginepolicyinterventions,itisfirstnecessaryto considerthechallengeofidentifyingintrinsicallydangerous systemsinapolicycontext.Foragovernancebody attemptingtolimitthespreadofhazardousAIsystems,it willberelevanttoconsiderthetell-talesignsofconcerning researchwithanapproachthatallowsforearlydetection andappliesequalscrutinythroughoutthelifecycleofa system. Asoutlinedinprevioussections,weexpectintrinsic dangerfromsystemsthathavehighcapabilitiesandare alsohighonpropertyX.DirectevaluationofpropertyXis thereforelikelytobeanimportantpartofdetectingAI systemsofconcern.Forexample,thiscouldtaketheform oftestsduringasystem'sdevelopmentorafteritis deployed;asystemâsperformancecanrevealdistinctrisks andsurprisingbehaviours,suchasdeception,long-term strategicplanningbeyondthesystemâsintendedscope,or subversionofsafeguardsandguardrails.Therisks presentedbysuchbehaviourareevenmoreacutewithin systemswhoseoutputsareuninterpretable.Weexpectthat elicitationanddetectionofsuchbehaviourswouldrequire novelassessmenttechniquesandcontinualupdate,tobe developedandmaintainedbydomainspecialists. BecauseevaluationofpropertyXislikelytoberelatively intrusiveandcostly,webelieveitwouldbepragmatictoonly subjectasmallsubsetofallAIsystemstosuchevaluations. Theoverallcapabilityofthesystemcouldbeusedtodecide whichsystemsundergofurtherscrutiny,asitisthe combinationofcapabilityandpropertyXthatleadsto intrinsicdanger.Atpresent,wecantentativelypointat severalproxiesthatcouldindicatethatanAIsystemwill displayhighcapabilities,outlinedbrieflyinTable2.These havebeenadaptedfrompriorworkthatconsidersamore high-dimensionalcharacterisationoftheParetofrontofAI improvements,beyondbenchmarksandperformance metrics.[58]Althoughthemarkersproposedhereare deliberatelygearedtowardpracticalapplications,itis importanttonotethatentitiesseekingtomitigatethe developmentofsystemsofconcernwouldlikelystillneedto continuouslyadapt,expand,andtailortheirownindicators. Table2:Potentialindicatorsfortheneedtoassessproperty X Indicator Description Compute Amountofcomputationalpower(measured, e.g.,infloatingpointoperations)requiredto trainanddeploytheAIsystem. Load DimensionsoftheAIsystem,e.g.numberof parameters. Software Algorithmsusedtotrainandrunthesystem, andsupportingsoftwareinfrastructuresuch asmachinelearningframeworks. Physical components Datacentres,laboratories,physical hardware,andenergyconsumption associatedwiththesystem. Time Amountoftimerequiredfortrainingmodels andrunningthem. Degreeofhuman supervision Extentofhumaninvolvement,whetherin trainingmodelsorvettingdecisionsreached bysystems. Data Amountandtypeofdatausedinmodels. Behaviour Extenttowhichtheactionstakenbya systemcomportwiththeexpectationsof systemoperators. Interpretability and Explainability respectively,theabilitytoapproximate understandingofanopaquesystemandthe abilitytoinherentlyunderstandasystem.[59] Wecanconsiderthesignificanceoftheabovefactorsby imagininghowaregulatormightemploythemtodetecta systemthatshouldundergoevaluationforpropertyX.For instance,todayâsmostadvancedAIsystemsare characterisedbytheneedforverylargetrainingcompute (althoughthepreviouslyrapidgrowthincomputemaysoon taper[60])andhighload(parametercount),whichare directlylinked(viascalinglaws)tohighercapabilities,and thereforetoahigherpotentialforharm.Atpresent,thevery largeamountsofcomputerequiredtotrainfrontierAI systemshavepredominantlybeenwieldedonlybyalimited numberofactors,providinganintuitivestartingpointto detectpotentialsystemsofconcern.Relatedly,thephysical hardwareandinfrastructurerequiredtobuildandoperate supercomputerscanserveasamarkerofadvancedAI research,ascantherelativelyhighenergyconsumptionof advancedcomputing. Similarly,theconditionssurroundingthecreationof models,aswellassafeguardsinvolvedintheirdeployment, willaffecttheirpotentialforharm.Forexample,regulators mightdeemthatthegreaterthequantityofdataandtime requiredtotrainamodel,thegreatertheriskspresented. TheymightalsodeemthatcertainMLtechniquespresenta heightenedrisk.Inotherinstances,dangerscouldarise fromcertaintypesofresearchthatbypasstheneedfor largecomputingpowerbutstillachievehighlysophisticated systems.Regardlessoftheamountofcompute,advanced systemstrainedandoperatedwithalowdegreeofhuman supervisionandlackofguardrailswouldalsobeasourceof concern. ItshouldbenotedthattheindicatorslistedinTable2aim toaddressasystemâsintrinsicdangeranddonotdealwith thecontextinwhichitisdeployed.Still,othercontextual factorsareusefultoregulatorsandareworthmentioning althoughtheyarenotthefocusofthispaper.Forexample,if usinglargevolumesoftrainingdataisapossibletripwire, regulatorswouldbenefitfromkeepingaclosewatchover effortstogatherdataatamassscale.Datagleanedfrom crowdsourcingcampaignsorfromindividualsâonlineactivity canprovideasignificantsourceoftrainingdataforlarge andpotentiallydangerousmodels,whosedevelopment mightbedetectedsoonerifregulatorsareprivyto suspiciousdatacollectionactivities.Inthiscase,contextual detailscanstrengthentheproposedtripwire. Anadditionalcategoryofcontextualfactorsisformedby thosethataugmentasystemâsintrinsicrisks.Suchfactors mightincludeasystemâsdegreeofnetworkconnectivityand theresultingpossibilitythatthehazardsitcreatescanbe experiencedmorewidely.Anotherfactoristhesafety cultureoftheinstitutioninwhichthesystemisdeveloped; whereasconcerningbehavioursexperiencedwithina risk-averseenvironmentwouldcorrectlybenotedasared flag,thesameincidentsmightgounreportedbyaless awareworkforce. 4.3 DesigningPolicyInterventions Toconceptualisetherelevanceoftripwiresfordetectingand limitingpropertyX,weconsiderhowahypothetical nation-statemightutilisethem.Thenation-stateimaginedis relativelylargeandpowerful,andithousesadvancedAI researchcapacity,includingwithinitsprivatesector, academicinstitutions,andgovernmententities.Itpossesses sufficientpowertoreignindangerousAIresearchthrough domesticpolicyinterventionsandcanalsoeffectivelymake useofexportcontrols.Ithassignificantinfluencewithinthe internationalcommunityandcanmobilisethisinfluenceto impactattitudestowardtheregulationofAI. Thenation-stateisanidealunitofanalysisforthree reasons.First,althoughthereisvalueindiscussing internationalframeworkstoregulatethedevelopmentof dangerousAI,internationallawislimitedinitsabilityto constrainsystemsofconcernwithoutsignificant interventionsatthenationallevel.[61] Second,nation-statescandirectlyregulatecorporations, whicharedecisiveactorsinthedevelopmentofsystemsof concern;whilenationalpolicyhasoftenproventobean imperfectinstrumentforchangingthebehaviourof increasinglypowerfulcorporations,[62]itremainsthemost potenttoolfordoingsoandcanhaverippleeffectswithina systemofglobalisedcommerce. Finally,numerousinternationalframeworkshavearisen fromtheadvocacyeffortsofindividualnation-stateswhose domesticpoliciesandvaluesystemscanshapeapproaches toglobalproblems.[63]Forexample,theUnitedStatesâ AtomsforPeacecampaigncreatedinternationalmomentum thatultimatelyledtothenegotiationoftheNuclear Non-ProliferationTreaty[63].Thus,byconfiningour analysistonation-states,wehopetoshowwhatbroad changesmaylooklikeonasmallerscale,aswellashow thebenefitsofrobustnationalpoliciescanextendbeyond thestatesthatenactthem. Whilethereisnoone-size-fits-allpolicysolution,any successfulinterventionmustpossessafewkeyattributesto detectandmitigatepropertyX.First,independentoversight canensurethatpowerfulinterestgroupsdonotobfuscate effortstodetectandlimitsystemshighinpropertyX. Typically,themosteffectiveindependentbodiesare democraticinnatureandthuscanbecontestedbycitizens andelectedofficials.[65]Withtheirmandatetoservepublic interests,suchbodieshavestrongincentivestodisplay expertiseandprofessionalism,impartiality,andscrutiny.[66] Followingthedesignationofanindependentagencyto performoversight,thatagencywouldrequireadirectlineof communicationwiththeentitiesitregulates,perhaps throughinternalcomplianceofficers. Second,regulatorsmustdevelopguidelinesforthe detectionofpropertyX,inadditiontocultivatingthe technicalexpertisenecessarytodiscernifsuchguidelines areupheld.Regulatorsmightdeveloprubricsforexamining thelevelofpropertyXwithinasystem.[67]Ifasystem scoresaboveacertainlevelonthatrubric,theregulator mightthenrequirethatcertainredteamingactivities, benchmarks,andauditlogsbecompletedbeforethe institutioncanproceedwiththatsystem.Importantly,the roleofexpertiseâbothforregulatorsandthosebeing regulatedâisessentialfordetectingsystemsofconcern. Withoutanadequateunderstanding,forexample,of unwantedsystembehavioursandtheirpotentialdangers,it isdifficultforsystemownersandregulatorstomakegood useofrubrics. Third,giventherelativelylargenumberofactorswho mightbesubjecttoscrutiny,theregulatormustbeableto effectivelyoperateunderconditionsofuncertaintyandwith limitedresources.Whilenotcarriedoutbyasingle nation-state,theenforcementofnuclearsafeguardsbythe InternationalAtomicEnergyAgency(IAEA)providesa relevantexample.Eachyearonalimitedbudget,a relativelysmallnumberofIAEAinspectorsarechallenged toinspectalargenumberofnuclearfacilitiesworldwide; sincetheIAEAcannotreasonablyinspecteverysingle nuclearreactororspentfuelrod,theagencyhasembraced statisticalapproachescentredaroundrandomsamplingand remotesurveillance.[68]Evenifimperfectandultimately unabletopreventillicitactivitiesoutright,theIAEAâssystem ofinspectionshasstilloftensucceededindetecting violations.[69]Similarly,aregulatorybodyseekingtolimit propertyXmightaimtogatherasmuchinformationas possiblewithaprimaryemphasisonearlydetectionfor suspiciousorconcerningactivities;doingsomightinvolve theuseofsimilarmethodsforhandlinglargequantitiesof datatoeasetheburdenonregulators. Finally,regulatorsrequireenforcementcapacity, includingsufficientaccesstosystemdesignersand operatorstoexplorethepotentialpresenceofpropertyX.To ensureadherencetoguidelines,regulatorsmustbegranted accesstorelevantemployees(forinterviews)andfacilities (forinspection)ifconcernsarise.Anumberofarrangements couldresultinraisingconcerns:Whileentitiesmightbe requiredtoprovecompliancetobeginresearchand/or remaininoperation,theycouldalsobesubjectedtoregular and/orrandominspections.Amorelooselycrafted regulatoryframeworkmightrelyonwhistle-blowerstocome forward,onlyinvestigatingconcernsiftheywereraised internally.Alternatively,theinteractionswiththeregulator mightbedictatedbycertainmilestonesinasystem lifecycle,withregularinspectionsscheduledaccordingto thespeedandscopeofAIresearch.Anyoftheabove formulationswoulddependuponaregulatorâsabilityto speakdirectlywithsystemowners,potentiallyinterviewing themaboutconcerningbehaviourdisplayedbythesystem. Asacomponentofenforcement,regulatorsmightalso requiresystemownersandoperatorstoundergomandatory trainingstoincreasethelevelofknowledgeaboutproperty Xandthemeansofavoidingit. 5 Conclusion Wehaveoutlinedasetofpotentialintrinsiccharacteristics ofadvancedAIsystemsthattogetherwelabelâPropertyXâ. TheseincludecharacteristicsdiscussedintheAIsafety literaturesuchaspower-seekingbehaviour,instrumental rationality,andstrategicawarenessthatmayleadtorisky behaviourssuchasthepursuitofconvergentinstrumental goals.SystemshighinPropertyXmayprovetobe exceptionallycapableandsomaybeanappealinggoalof research;however,wehavearguedthatsuchsystemsare highlylikelytobedangerousanddifficulttoalign.The researchvisionsoutlinedinCollectiveIntelligence, Human-CentredAI,andComprehensiveAIservicesprovide compellingalternativesthatdemonstratethatprogress acrossscientific,economicandsocietaldomainscanbe supportedbyAIsystemslowinPropertyX.Whilethesetof characteristicsthatmakeupPropertyXarenotyetfully specified,itisnonethelesspossibletobeginenvisioning governanceregimesthatidentifyandaddressPropertyX behavioursintheAIdevelopmentprocess.Ourproposals areintendedasastartingsketchfordoingso. REFERENCES [1]StephenCave,KantaDihal,andSarahDillon.2020. AI Narratives.OxfordUniversityPress. [2]JosephRedmon,SantoshDivvala,RossGirshick,andAli Farhadi.2016.YouOnlyLookOnce:Unified,Real-TimeObject Detection. 2016IEEEConferenceonComputerVisionandPattern Recognition(CVPR) (2016).DOI:https://doi.org/10.1109/cvpr.2016.91 [3]DavidSilver,AjaHuang,ChrisJ.Maddison,ArthurGuez,Laurent Sifre,GeorgeVandenDriessche,JulianSchrittwieser,IoannisAntonoglou, VedaPanneershelvam,MarcLanctot,SanderDieleman,DominikGrewe, JohnNham,NalKalchbrenner,IlyaSutskever,TimothyLillicrap,Madeleine Leach,KorayKavukcuoglu,ThoreGraepel,andDemisHassabis.2016. MasteringthegameofGowithdeepneuralnetworksandtree search. Nature 529,7587(2016),484-489. DOI:https://doi.org/10.1038/nature16961 [4]TomBrown,BenjaminMann,NickRyder,MelanieSubbiah,Jared Kaplan,PrafullaDhariwal,ArvindNeelakantan,PranavShyam,GirishSastry, AmandaAskell,SandhiniAgarwal,ArielHerbert-Voss,GretchennKrueger, TomHenighan,RewonChild,AdityaRamesh,DanielZiegler,JeffreyWu, ClemensWinter,ChristopherHesse,MarkChen,EricSigler,MateuszLitwin, ScottGray,BenjaminChess,JackClark,ChristopherBerner,Sam, McCandlish,AlecRadford,IlyaSutskever,andDarioAmodei.2020. LanguageModelsareFew-ShotLearners.34 th ConferenceonNeural InformationProcessingSystems.Vancouver,Canada. [5]AdityaRamesh,MikhailPavlov,GabrielGoh,ScottGray,Chelsea Voss,AlecRadford,MarkChen,andIlyaSutskever.2021.Zero-Shot Text-to-ImageGeneration. Proceedingsofthe38thInternationalConference onMachineLearning (2021). [6]JumperJ,EvansR,PritzelA,GreenT,FigurnovM,Ronneberger O,TunyasuvunakoolK,BatesR,ĆœĂdekA,PotapenkoA,BridglandA.Highly accurateproteinstructurepredictionwithAlphaFold.Nature.2021Aug 26;596(7873):583-9. [7]MachineryC.Computingmachineryandintelligence-AMTuring. Mind.1950;59(236):433. [8]GoodIJ.Speculationsconcerningthefirstultraintelligent machine.InAdvancesincomputers1966Jan1(Vol.6,p.31-88).Elsevier. [9]LandisG,BaileyS.Vision-21:InterdisciplinaryScienceand EngineeringintheEraofCyberspace.Proceedings,NASALewisResearch Center.1993Mar30. [10]Bostrom,N.(2014)Superintelligence.Oxford:OxfordUniversity Press,Incorporated. [11]Boden,M.A.(2016)Ai.Oxford:OxfordUniversityPress, Incorporated. [12]BengioY,LecunY,HintonG.DeeplearningforAI. CommunicationsoftheACM.2021Jun21;64(7):58-65. [13]StanfordUniversityHuman-CentredArtificialIntelligence.Artificial IntelligenceIndexReport2022. [14]NathanBenaichandIanHogarth.StateofAIReport2022. [15]KaplanJ,McCandlishS,HenighanT,BrownTB,ChessB,Child R,GrayS,RadfordA,WuJ,AmodeiD.Scalinglawsforneurallanguage models.arXivpreprintarXiv:2001.08361.2020Jan23. [16]HoffmannJ,BorgeaudS,MenschA,BuchatskayaE,CaiT, RutherfordE,CasasDD,HendricksLA,WelblJ,ClarkA,HenniganT. Trainingcompute-optimallargelanguagemodels.arXivpreprint arXiv:2203.15556.2022Mar29. [17]BommasaniR,HudsonDA,AdeliE,AltmanR,AroraS,vonArx S,BernsteinMS,BohgJ,BosselutA,BrunskillE,BrynjolfssonE.Onthe opportunitiesandrisksoffoundationmodels.arXivpreprint arXiv:2108.07258.2021Aug16. [18]GanguliD,HernandezD,LovittL,AskellA,BaiY,ChenA, ConerlyT,DassarmaN,DrainD,ElhageN,ElShowkS.Predictabilityand surpriseinlargegenerativemodels.In2022ACMConferenceonFairness, Accountability,andTransparency2022Jun21(p.1747-1764). [19]GruetzemacherR,WhittlestoneJ.Thetransformativepotentialof artificialintelligence.Futures.2022Jan1;135:102884. [20]GruetzemacherR,ParadiceD,LeeKB.Forecasting transformativeAI:Anexpertsurvey.arXivpreprintarXiv:1901.08579.2019 Jan24. [21]Stein-PerlmanZ,Weinstein-RaunB,GraceK.expertsurveyon progressinAI.AIImpacts.Availableonlineat:https://aiimpacts. org/2022-expert-survey-on-progress-in-ai(accessedDecember7,2022). 2022Aug3. [22]AjeyaCotra.Two-yearupdateonmypersonalAItimelines.AI AlignmentForum. [23]OmohundroSM.ThebasicAIdrives.InAGI2008Feb18(Vol. 171,p.483-492). [24]BostromN.Thesuperintelligentwill:Motivationandinstrumental rationalityinadvancedartificialagents.MindsandMachines.2012 May;22:71-85. [25]EliezerYudkowsky.AGIRuin:Alistoflethalities.AIAlignment Forum. [26]JosephCarlsmith.IsPower-SeekingAIanExistentialRisk?. arXivpreprintarXiv:2206.13353.2022Jun16. [27]AjeyaCotra.Withoutspecificcountermeasures,theeasiestpath totransformativeAIlikelyleadstoAItakeover.AIAlignmentForum. [28]Christian,B.(2021)Thealignmentproblem:howcanmachines learnhumanvalues?/BrianChristian.Main. [29]Russell,S.J.(2019)Humancompatible:artificialintelligenceand theproblemofcontrol. [30]Gwern.WhytoolAIswanttobeagentAIs.2016. [31]KatjaGrace.IncentivestocreateAIsystemsknowntopose extinctionrisks.AIImpacts. [32]ZwetslootR,DafoeA.ThinkingaboutrisksfromAI:Accidents, misuseandstructure.Lawfare.February.2019Feb;11:2019. [33]SuttonR.S.andBartoA.G.(1998)Reinforcementlearning:an introduction.Cambridge,MA:MITPress(Adaptivecomputationandmachine learning). [34]HutterM.Atheoryofuniversalartificialintelligencebasedon algorithmiccomplexity.arXivpreprintcs/0004001.2000Apr3. [35]BostromN.Thesuperintelligentwill:Motivationandinstrumental rationalityinadvancedartificialagents.MindsandMachines.2012 May;22:71-85. [36]OmohundroSM.ThebasicAIdrives.InAGI2008Feb18(Vol. 171,p.483-492). [37]Benson-TilsenT,SoaresN.FormalizingConvergentInstrumental Goals.InAAAIWorkshop:AI,Ethics,andSociety2016Mar29. [38]Bostrom,N.(2014)Superintelligence.Oxford:OxfordUniversityPress, Incorporated. [39]ChanA,SalganikR,MarkeliusA,PangC,RajkumarN, KrasheninnikovD,LangoscoL,HeZ,DuanY,CarrollM,LinM.Harmsfrom IncreasinglyAgenticAlgorithmicSystems.arXivpreprintarXiv:2302.10329. 2023Feb20. [40]CarlsmithJ.IsPower-SeekingAIanExistentialRisk?.arXiv preprintarXiv:2206.13353.2022Jun16. [41]TurnerAM,SmithL,ShahR,CritchA,TadepalliP.Optimal policiestendtoseekpower.arXivpreprintarXiv:1912.01683.2019Dec3. [42]TurnerAM,TadepalliP.ParametricallyRetargetable Decision-MakersTendToSeekPower.arXivpreprintarXiv:2206.13477.2022 Jun27. [43]Anthropic.CoreviewsonAIsafety:when,why,what,andhow. 2023. [44]PerezE,RingerS,LukoĆĄiĆ«tÄK,NguyenK,ChenE,HeinerS, PettitC,OlssonC,KunduS,KadavathS,JonesA.DiscoveringLanguage ModelBehaviorswithModel-WrittenEvaluations.arXivpreprint arXiv:2212.09251.2022Dec19. [45]CarlsmithJ.IsPower-SeekingAIanExistentialRisk?.arXiv preprintarXiv:2206.13353.2022Jun16. [46]Gwern.WhytoolAIswanttobeagentAIs.2016. [47]ChanA,SalganikR,MarkeliusA,PangC,RajkumarN, KrasheninnikovD,LangoscoL,HeZ,DuanY,CarrollM,LinM.Harmsfrom IncreasinglyAgenticAlgorithmicSystems.arXivpreprintarXiv:2302.10329. 2023Feb20. [48]SiddarthD,AcemogluD,AllenD,CrawfordK,EvansJ,Jordan M,WeylE.HowAIfailsus.arXivpreprintarXiv:2201.04200.2021Dec1. [49]SiddarthD,AcemogluD,AllenD,CrawfordK,EvansJ,Jordan M,WeylE.HowAIfailsus.arXivpreprintarXiv:2201.04200.2021Dec1. [50]Shneiderman,B.(2022)Human-CenteredAI.Oxford:Oxford UniversityPress. [51]DrexlerKE.Reframingsuperintelligence:ComprehensiveAI servicesasgeneralintelligence.FutureofHumanityInstitute,Universityof Oxford.2019Jan. [52]EricDrexler.(2023)Theopenagencymodel:notesonAifor complex,consequentialproblems.AIAlignmentForum. [53]JobinA,IencaM,VayenaE.ThegloballandscapeofAIethics guidelines.NatureMachineIntelligence.2019Sep;1(9):389-99. [54]FischerSC,LeungJ,AnderljungM,OâkeefeC,TorgesS,Khan SM,GarfinkelB,DafoeA.AIPolicyLevers:AReviewoftheUS GovernmentâsToolstoShapeAIResearch,Development,andDeployment. [55]FalcoG,ShneidermanB,BadgerJ,CarrierR,DahburaA,Danks D,ElingM,GoodloeA,GuptaJ,HartC,JirotkaM.GoverningAIsafety throughindependentaudits.NatureMachineIntelligence.2021 Jul;3(7):566-71. [56]OpenAI.(2023)GPT-4SystemCard. [57]Anthropic.CoreviewsonAIsafety:when,why,what,andhow. 2023. [58]MartĂnez-PlumedF,AvinS,BrundageM,DafoeA,hĂigeartaigh SĂ,HernĂĄndez-OralloJ.BetweenProgressandPotentialImpactofAI:the NeglectedDimensions.arxXivpreprint.2018. [59]RudinC,ChenC,ChenZ,HuangH,SemenovaL,ZhongC. Interpretablemachinelearning:Fundamentalprinciplesand10grand challenges.StatisticSurveys.2022;16:1-85. [60]LohnA,MusserM.Howmuchlongercancomputingpower,drive artificialintelligenceprogress?.2022-10-08].https://cset.georgetown. edu/wp-content/uploads/AI-and-Compute-How-Much-Longer-Can-Computing -Power-Drive-Artificial-Intelligence-Progress_v2.pdf.2022. [61]MaasMM.Internationallawdoesnotcompute:Artificial intelligenceandthedevelopment,displacementordestructionoftheglobal legalorder.MelbourneJournalofInternationalLaw.2019Jul1;20(1):29-57. [62]BabicM,FichtnerJ,HeemskerkEM.Statesversuscorporations: Rethinkingthepowerofbusinessininternationalpolitics.Theinternational spectator.2017Oct2;52(4):20-43. [63]Khagram,S.,Riker,J.V.andSikkink,K.(2002)Restructuring WorldPolitics.NED-New.Basel:UniversityofMinnesotaPress(Social Movements,ProtestandContention). [64]KrigeJ.Atomsforpeace,scientificinternationalism,andscientific intelligence.Osiris.2006Jan;21(1):161-81. [65]Pettit,P.(2012)Onthepeople'sterms:arepublicantheoryand modelofdemocracy.Cambridge;NewYork:CambridgeUniversityPress (JohnRobertSeeleylectures;8). [66]Pettit,P.(2012)Onthepeople'sterms:arepublicantheoryand modelofdemocracy.Cambridge;NewYork:CambridgeUniversityPress (JohnRobertSeeleylectures;8). [67]CatherineAiken.(2021)ClassifyingAISystems.CSET. [68]InternationalAtomicEnergyAgency.(2010)Preparingforfuture verificationchallenges:summaryofaninternationalsafeguardssymposium. [69]HibbsM.IranandtheEvolutionofSafeguards:Verificationand Implementation.CarnegieEndowmentforInternationalPeace.2015.