CIGIPapersNo.279—August2023AI-RelatedRiskTheMeritsofanESG-BasedApproachtoOversightMardiWitzelandNirajBhargavaCIGIPapersNo.279—August2023AI-RelatedRiskTheMeritsofanESG-BasedApproachtoOversightMardiWitzelandNirajBhargavaCopyright©2023bytheCentreforInternationalGovernanceInnovationTheopinionsexpressedinthispublicationarethoseoftheauthorsanddonotnecessarilyreflecttheviewsoftheCentreforInternationalGovernanceInnovationoritsBoardofDirectors.Forpublicationsenquiries,pleasecontactpublications@cigionline.org.ThisworkislicensedunderaCreativeCommonsAttribution—Non-commercial—NoDerivativesLicense.Toviewthislicense,visit(www.creativecommons.org/licenses/by-nc-nd/3.0/).Forre-useordistribution,pleaseincludethiscopyrightnotice.CentreforInternationalGovernanceInnovationandCIGIareregisteredtrademarks.67ErbStreetWestWaterloo,ON,CanadaN2L6C2www.cigionline.orgAboutCIGITheCentreforInternationalGovernanceInnovation(CIGI)isanindependent,non-partisanthinktankwhosepeer-reviewedresearchandtrustedanalysisinfluencepolicymakerstoinnovate.Ourglobalnetworkofmultidisciplinaryresearchersandstrategicpartnershipsprovidepolicysolutionsforthedigitalerawithonegoal:toimprovepeople’sliveseverywhere.HeadquarteredinWaterloo,Canada,CIGIhasreceivedsupportfromtheGovernmentofCanada,theGovernmentofOntarioandfounderJimBalsillie.ÀproposduCIGILeCentrepourl’innovationdanslagouvernanceinternationale(CIGI)estungroupederéflexionindépendantetnonpartisandontlesrecherchesévaluéespardespairsetlesanalysesfiablesincitentlesdécideursàinnover.Grâceàsonréseaumondialdechercheurspluridisciplinairesetdepartenariatsstratégiques,leCIGIoffredessolutionspolitiquesadaptéesàl’èrenumériquedansleseulbutd’améliorerlaviedesgensdumondeentier.LeCIGI,dontlesiègesetrouveàWaterloo,auCanada,bénéficiedusoutiendugouvernementduCanada,dugouvernementdel’Ontarioetdesonfondateur,JimBalsillie.CreditsManagingDirectorofDigitalEconomyRobertFayDirector,ProgramManagementDiannaEnglishProjectManagerJennyThielSeniorPublicationsEditorJenniferGoyderPublicationsEditorSusanBubakGraphicDesignerBrooklynnSchwartzTableofContentsviAbouttheAuthorsviAcronymsandAbbreviations1ExecutiveSummary1Introduction3WhatIsESG?4WhyGovernAIUsethroughanESGApproach?5TheWorldofAIGovernance8TheNatureofAI-RelatedRisk11TheMaterialityofAI-RelatedRisk16IntegratingMaterialAI-RelatedRisksintoESG17ConsiderationsforaPathForward19WorksCited20AppendixviCIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaAbouttheAuthorsMardiWitzelisanassociatewithNuEnergy.aiandisfocusedonESG(environmental,socialandcorporategovernance)andAIgovernance,andthespecialchallengesfacinghigh-growthfirms.SheservesontheboardofPolyML,aprivatefirmspecializinginmachinelearningandadvancedanalytics,andhas20yearsofexperienceinnot-for-profitboardgovernance,stakeholderengagementandstrategicplanning.NirajBhargavaistheCEOandleadfacultyatNuEnergy.aiandanexpertonartificialintelligence(AI)governance.HeleadsateamofexpertsspecializedinAIgovernanceeducation,inthecreationoforganization-levelAIgovernanceframeworksandintheintegrationofNuEnergy’sAItrustmeasurementsoftware,theMachineTrustPlatform.AcronymsandAbbreviationsAGIartificialgeneralintelligenceAIartificialintelligenceAIAAlgorithmicImpactAssessmentAIDAArtificialIntelligenceandDataActAIRMFAIRiskManagementFrameworkCDSBClimateDisclosureStandardsBoardESGenvironmental,social,governanceIFRSInternationalFinancialReportingStandardsIIRCInternationalIntegratedReportingCouncilIPintellectualpropertyISOInternationalStandardsOrganizationISSBInternationalSustainabilityStandardsBoardITinformationtechnologyNGOsnon-governmentalorganizationsNISTNationalInstituteofStandardsandTechnologySASBSustainabilityAccountingStandardsBoardTCFDTaskForceonClimate-RelatedFinancialDisclosureVRFValueReportingFoundation1AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightExecutiveSummaryArtificialintelligence(AI)technologyhasbeenfoundtogeneratevalueformanyfirms;however,italsohasunintendedandundesirableconsequences.TherealityofAI-relatedriskhasledtothedevelopmentofAIgovernanceframeworksandcallsforgreateroversightoftheuseofAI.ThemeritsofanESG(environmental,social,governance)-basedapproachtooversightofAI-relatedriskareconsideredinthispaper,withafocusonthecurrenttrajectoryofinternationalsustainabilitystandardsdevelopment.Despitetheirdifferences,AIgovernanceandESGreportingbothseektoaddressriskinthebroadestsense,withproactiveandtransparentapproachestoitsmanagementandmitigation.RecognizingthatreadersmaybefamiliarwitheitherAIgovernanceorESGbutnotlikelyboth,thepaperisconstructedsoastoprovideanoverviewofeach.ThepaperexamineswhatisdifferentaboutAI-relatedriskandidentifiesfourfactors:speedandscale,AIempowerment,AIlifecycleandAIethics.Theanalysisfindspossiblegapsand/ormaterialtopicsthatarenotcoveredbytheSustainabilityAccountingStandardsBoard(SASB),includingAI-relatedrisksthatmaydifferonanindustrybasisandonthebasisofanenterprise’sroleintheAIvaluechain.ApreliminarysetofrecommendationsforincorporatingmaterialAI-relatedriskintoESGreporting,coveringbothgeneralorcontext-settingdisclosuresandindustry-specificdisclosures,isprovided.IntroductionAItechnologyisbeingappliedbroadlyinbusiness.Accordingtoonestudy(McKinsey&Company2022),adoptionhasmorethandoubledsince2017,buttheproportionoforganizationsusingAIhasplateauedinrecentyearsat50–60percent.Anotherglobalsurvey(IBM2022)revealedacontinuingupwardtrendinadoption,with35percentofcompaniesreportedlyusingAItoday,42percentexploringitsuseandAIadoptionupfourpercentfrom2021to2022.AndwhileAIhasbeenfoundtogeneratevalueformanyfirms,itisalsorecognizedasadouble-edgedsword,givingrisetoahostofunintendedandundesirableconsequences.ReflectingtherealityofAI-relatedrisk,therehasbeenaproliferationinthedevelopmentofAIgovernanceframeworksandagrowingbodyofliteraturecallingforgreateroversightofitsuse.ArecentarticleinMITSloanManagementReview(Silverman2020)identifiedseverallensesthroughwhichboardsmayapproachthemanagementofAI-relatedrisk:compliance,strategicplanning,legalorbusinessrisk,andESG.ThispaperexaminesthemeritsofanESG-basedapproachtoboardoversightofAI-relatedrisk.Thisisapracticalandtimelydiscussion.FollowingavanguardofearlyAIadoptioninthelate2000s,thehardrealityofthetechnology’srisksandnegativeimpactsbecameapparent.Inresponse,regulatoryauthoritiestooknotice,andvariouslegalandpolicyinstrumentshavebeenproposed,passedorareunderreview.ThistakesnothingawayfromtheenormityofthepotentialforAItodogood,topromotegrowththroughinnovationandincreasedproductivity,and,withanyluck,tohelphumansaddresssomeofthebiggestchallengesourplanetandsocietyface.Butitisfairtosaythatalongsidethesegreatexpectations,isanowequallyrootedbeliefintheimperativeofresponsibleortrustworthyAI.Inrecentyears,therehavebeendevelopmentsinbothAIgovernanceandsustainabilityreportingthatsharecommonthemes.Theemphasisonthetwinobjectivesofleveragingopportunitywhilemanagingriskisevidentfromboth,albeittodifferingdegrees.ThefocusofAIgovernanceismorepronouncedonrisk,whilethedrivingforcebehindsustainabilityreportingisabitofadancebetweenthetwo.Mostcritically,AIgovernanceandsustainabilityreportinghavebothadvocatedstronglyforgreatertransparencyanddisclosureacrossabroadersetofrisks.Eachdiscipline,iftheycanbelabelledassuch,isconcernedwiththeidentification,managementandmitigationofrisksthatextendbeyondshort-termfinancialimpacts,includinglegal,regulatoryandreputationalconcerns,forexample.Inparticular,ESGhaspushedtheagendaofevaluatingopportunityandriskaccordingtoalonger-termview.Wherehistoricallythesetypesofrisksmayhavebeenseenasnon-financial,theproponentsofESGholdthattheseshouldbeseenaspre-financialrisksthataredestinedtocomehome2CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavatoroostinenterprisevaluationandfinancialsatsomepoint—andoftensoonerthanlater.ThediscussionrelatingtoAIasanESGriskhasthepotentialtobecomecomplex,andthispaperhasintentionallysetsomeboundaries.ThefirstisanacknowledgementthatwhileAIispartofthebroaderconstellationofdigitalgovernanceconcerns,andmanyoftheargumentsitcontainscouldbeappliedtodataanddigitalmorebroadly,thispaperremainsfocusedonAI.ThesecondboundarypertainstotheliteratureonESGandsustainability:thesereferencebothsinglemateriality(i.e.,theissuesthatposematerialfinancialriskstoafirm)anddoublemateriality(i.e.,theconsequencesofafirm’soperationsthatposematerialimpactstotheenvironmentandsociety);thispaperisfocusedonlyonadiscussionofthefinancialmaterialityofAIrisktofirms.Despitetheirdifferenthistories,theworldsofAIgovernanceandESGreportingshareacommonmotivation:toaddressriskinthebroadestsense,withproactiveandtransparentapproachestoitsmanagement.Evidenceofthisagendacanbeseeninrecentdevelopmentsfromeachdiscipline.ThepublicationoftheEuropeanUnion’sproposedArtificialIntelligenceAct(AIAct)inApril2021,representsthefirstexampleofacomprehensiveregulatoryapproachtoAI,carryingwithitabroadsuiteofobligations,includingtransparencyanddisclosurerelatingtoAI,and,inparticular,theenterprisesystemssupportingitsresponsibledevelopmentanduse.InNovember2021,theInternationalFinancialReportingStandards(IFRS)FoundationannouncedthecreationofanInternationalSustainabilityStandardsBoard(ISSB),signallingtheadventofamoreunifiedglobalapproachtovoluntaryreportinganddisclosureonESGorsustainabilitystandards.1TherootofthecaseforanESG-basedapproachtoassessingriskandopportunityisfoundinstakeholderorientation,thetimehorizonitappliestobothriskandopportunityandtheroleofreportinganddisclosure.StakeholderconcernaboutAI-relatedbusinessimpacts,ingeneral,andhowdataissourced,securedandused,inparticular,isatanall-timehigh.ThecallforappropriatecorporatemanagementanddisclosureofAIuseisemergingasbothapublicexpectation1ThetermsESGandsustainabilityareusedinterchangeablyinthispaperinrelationtothereportinganddisclosureinitiatives.and,inmanyjurisdictions,alegalorregulatoryrequirement.Whereasconventionalaccountingmethodsarelimitedtodayintermsoftheirabilitytoincorporatemanyfinanciallymaterialissues,ESGframeworksprovideacomplementaryapproach.TheliteratureonESGandAIgovernancehasbeenevolving,butthereareonlyafewpapersspecificallyexploringtheutilityofESGasaframeworkforunderstanding,reportinganddisclosingAI-relatedrisk.JamesBrusseau(2023)findsthecurrentsuiteofESGratingsframeworkslackingforevaluatingAIimpactsandproposesamodelbasedoncommonlyheldprinciplesforethicalAI,ratherthanadaptationofanexistingESGframework.HenrikSkaugSætra(2021)proposesaframeworkforevaluatingESG-relatedimpactsofAIaccordingtotheUnitedNationsSustainableDevelopmentGoals.Sætra(2022)buildsonearlierwork,presentinganAIESGprotocol—aframeworkforevaluatingtheESGimplicationsofAIcapabilities,assetsandactivitiesaccordingtothreescopesofimpactsandwheretheseareexperiencedinthesupplychain.ThispaperbuildsonthisresearchinanexaminationofthevalueofapplyinganESGlenstothechallengeofAIgovernance,andspecificallythemanagementofAI-relatedrisk.RecognizingthepracticalimplicationsofthenewlyformedISSBforglobalreporting,thecontributionofthispaperistoreviewtherangeofAIgovernancetoolsavailabletoday,explorethenatureofAI-relatedriskandsetforthanapproachtohowthereportinganddisclosureofAI-relatedriskcouldbeintegratedintotheworkoftheISSB,towarddevelopmentofasingleglobalbaselineforsustainabilityreporting.ThehopeandexpectationisafullyimplementedESGframework,incorporatingAIand,ultimately,thefullrealmofdigitalgovernance,willresultinsystems,controlsandaccountabilityformonitoringandreportingonthepartofchieffinancialofficers.ThefirstsectionprovidesanoverviewofwhatismeantbyESG,becauseanyevaluationofitsvalueasalensrequiresabasicappreciationofwhatESGis.Withthisinhand,thesecondsectionconceptuallyexploresthequestionofwhyAIshouldbegovernedthroughanESGapproach.Inordertodigintothisquestionmoredeeply,thethirdandfourthsectionsexaminethestateofAIgovernanceapproachesandthenatureofAI-relatedrisk,respectively.HavingafoundationalunderstandingofAI-relatedriskandAIgovernancefacilitatesthediscussionthatisthemeatofthis3AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightresearch,foundinthelasttwosectionsofthepaper.ThefifthsectionaddressestheideaofmaterialityinthecontextofAI-relatedriskandthesixthsectionprovidespreliminaryideasabouthowtoaccommodatethesematerialconcernsintothestructureofESGstandards,reportinganddisclosurethatexiststoday.Specifically,thepaperproposesquestionsandideasreflectingthestateoftheISSB’sguidanceascapturedinitsExposureDraft,GeneralRequirementsforDisclosureofSustainability-relatedFinancialInformation.2WhatIsESG?ItishardtofindagoodsingledefinitionofESG,becauseitdependsontheapplication.ESGmightbeseenasasetofenvironmental,socialandgovernancecriteriathatinvestorsusetoscreeninvestments.Itcouldbeseentorepresentthescoreofafirm’scollectiveconsciousnessforESGfactors.Alternatively,acompanymightviewESGasasetofstandardsforcorporatebehaviourtobeusedinformulatingstrategiesforlong-termvaluecreation.ESGisaframeworkforthinkingmorecomprehensivelyand,therefore,moreaccuratelyabouttherisksandopportunitiesthatfirmsfaceovershort,mediumandlongertimehorizonsandhowthesemayimpactfirmperformance.Regardlessofwhatitiscalled,thereisastronglikelihoodthatimportantrisksandopportunitiesarenotbeingmanagedifafirmhasnotadoptedanESGframework.InsteadofcallingitESG,thisapproachtothinking,planning,doingandreportingcouldinsteadbecalled“integratedfinancialandnon-financialgovernance”becauseitincorporatesabroaderrangeoffactorsimpactingcompanyperformanceandvaluationthantraditionalrulesoffinancialdisclosurerequiretoday.Practicallyspeaking,ESGfororganizationsmanifestsitselfasamanagement,reportinganddisclosureapproachthatmaybefacilitatedbyoneormoreofanumberofESGframeworks.LargepubliccompaniesareespeciallylikelytobedoingESGreportingtoday,withevidencethatmorethan90percentoftheS&P500publishsustainability2Seewww.ifrs.org/projects/work-plan/general-sustainability-related-disclosures/exposure-draft-and-comment-letters/.reports(GovernanceandAccountabilityInstitute2021),althoughthetransparencyandqualityoftheirdatavaryconsiderably.Itisnotonlypubliccompanies—privatecompanies,governmentagenciesandnon-governmentalorganizations(NGOs)alsopublishESGdata.Organizationsmaychoosetouseoneoftheleadingglobalframeworks,suchasthoseprovidedbytheGlobalReportingInitiativeortheValueReportingFoundation(VRF)3toidentify,measureandreportontheissuesthataremostmaterialtotheirbusinesses.Andbeyondtheinformationprovideddirectlybyanorganization,investorsandotherstakeholdersmaylooktosustainabilityinformationthatispublishedbythird-partyESGRatingsAgencies,suchasSustainalyticsorMCSIESGResearch.ThenumberofcompaniesthatpublishESGreportswillcontinuetogrow,aswillthedepthandqualityofdata,forafewreasons.Stakeholders,includinginvestors,consumers,employeesandregulators,aredemandingaccesstothisinformation.Additionally,thereisevidenceofESG’svalueasadriveroffirmperformanceandoftheroleintangibleassetsplayinenterprisevaluation(ofteneclipsingtangibleassets).AndwhilethecurrenthodgepodgeofglobalESGstandardsandframeworkshasundermineduptakeacrossmanysectors,thatisabouttochange,withtheannouncementofaglobalinitiativetocreateuniversalstandardsforsustainabilityorESGreporting.In2021,theIFRSFoundationestablishedanISSBasaparallelorganizationtotheInternationalAccountingStandardsBoard,whosefinancialreportingstandardsareusedinover140countries.Thisneworganizationwillspearheadconvergenceonasetofharmonizedglobalsustainabilitystandards,incollaborationwiththeworld’sleadingESGreportingframeworks.ThenewISSBwillconsolidateleadinginvestor-focusedsustainabilitydisclosureorganizationsincludingtheClimateDisclosureStandardsBoard(CDSB)andtheVRF,whichitselfisanamalgamoftheformerSASBandtheIIRC.Thisconsolidation,intoabodycapableofdevelopingandoverseeingasinglesetofglobalsustainabilitystandards,isexpectedtobeagamechangerforsustainabilityreporting.IfacompanyistakingaproactiveapproachtoESG,itmeanstheboardandseniormanagementteamare,one,consciouslyaskingquestions,getting3TheSustainabilityAccountingStandardsBoard(SASB)andtheInternationalIntegratedReportingCouncil(IIRC)cametogethertoformtheVRFin2021.4CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaeducatedandbuildingplansthatincorporateESGfactorsalongsidetraditionalproduct-marketmixplanning;and,two,makingcapitalexpendituresinbothareas.Intheenvironmental,or“E,”space,thatmightinvolveplanningaroundthingssuchasclimateresilience,energymanagement,wastereductionorecologicalimpacts.Inthesocial,or“S”space,itistheorganization’spracticesandimpactsinrelationtohumanandsocialcapitalthatareconsidered—humanrights,healthandsafety,diversityandinclusion,customerprivacy,anddatasecurityareexamples.Finally,inthegovernance,or“G,”space,theissuesattheforefrontincludebusinessmodelandinnovation,businessethics,managementofthelegislativeandregulatoryenvironment,andsystemicriskmanagement.WhatisanESGfactorthatacompanymaynotbethinking,planningordoinganythingabouttodaybutwhereitmatters?Takeplanningfornet-zero.IntheEspace,thereisallkindsoftalkabouttheUN2030AgendaforSustainableDevelopmentandthepushfornet-zeroemissionsby2050,buthasthistranslatedintoconcreteplanswithinmostfirms?OrintheSspace,howaboutanorganization-widedatastrategy,focusedonthecollection,security,applicationandgovernanceofdata?Andifafirmdoesinfacthaveawell-constructedplanforESG-typefactors,withdeliverables,timeframesandalignedsystemssuchascompensationpolicy,isthereavaluetothat?Whywouldarobustnet-zeroplanprovidingcompetitivedifferentiationandgrowthpotentialnotbeworthsomething—anetpositivecontributortofirmvalue?Despitethecontroversyandtheskeptics,itseemsinevitablethattheglobalenergysectorwillgothroughamajortransitionawayfromcarbon-intensiveproductsandtowardmoresustainablesolutions,representingamajoreconomicopportunity.Ontheflipside,ifanorganizationisnotpractisinggooddatagovernance,isitthennotonlyfailingtocapitalizeonanopportunity,butalsoexposingitselftofuturerisk?Itseemslogicalthatfirmsthatareearlytothepartywithwell-conceivedplansandcapitalexpenditurestrategieswillbebeneficiaries,andlaggardswhofocusonlyontraditionalfinancialanalysiswillfallbehind.Concreteplanningapproachesinareassuchasclimateresilienceanddatagovernancecanbeseenasintangibleassets,inthesamewayapieceofintellectualproperty(IP)is.ThepointofallthisistohighlightthematerialityofESGfactorsandtheirrelationshiptoboththenon-financialandfinancialperformanceoffirms.Acrossthespectrumofintangibleassettypes,AIcouldbeviewedasasortof“levered”driveroffirmvaluation,helpingfirmsmakethebestuseoftheirothertangibleandintangibleassets.IthasbeenarguedthatAIisupendingtheindustrialageanddestroyingtraditionalbusinessthinking(Davenport2019).Newcommercialbehemothsaregrowingupwithoutthelegacyanchorsofinflexiblephysicalassets,andmanyofthemostvaluablecompaniestodayaresoftware,networkandplatform-based,andhavelittleinthewayofphysicalassets.Forbothnewandtraditionalbusinessmodels,AIrepresentsapowerfultoolforvaluecreation.WhyGovernAIUsethroughanESGApproach?WhyapproachthegovernanceofAIthroughESG?Toanswerthisquestion,ithelpstounderstandwhatisbehindthemomentumofESGtoday,andspecificallytheroleofESGinsupportinglong-termvaluecreation.ESGorsustainabilityapproacheshaveemergedatthefrontierofcorporatepurposeandstrategy,enablingbothvaluecreationandriskmitigationforfirms.TheriseinESGisasmuchexplainedbythechangingcompositionofmarketvaluationsasbysomesenseofmoralimperative.Takingstockofacompany’sESGscorecardmapscloselytoitsnon-financialperformance,andthatisaverybigdealtoday.Why?Becausethenon-financialelementsoffirmsaregrowinginbothsizeandproportion,andexertingmajorinfluenceonfirmvaluation.Therehasbeenadramaticshiftintheproportionofcorporatevaluationthatisattributabletointangibleversustangibleassets.Assetssuchashumancapital,IP,companyreputationandcustomerloyaltyrepresentgreatervaluethantheyusedto.Accordingtoonerecentstudy,only17percentofthevalueofS&P500marketvaluewasattributedtointangiblesin1975,whereasby2015thatnumberhadgrownto84percent(OceanTomo2021).Thematerialityofintangibleassetsisreinforcedbothbythescaleoftheireconomic5AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightpresenceandempiricalevidenceoftheirpositivecorrelationwithhigherprice-to-bookratios.WhatdoesthishavetodowithAI?AIisincreasinglyseenasanasset—adriveroforganizationalvalue.AImaybeemployedasalevertohelpfirmsoperatemoreefficientlyandeffectivelyand,inthissense,itmaybeviewedasaproductivitytool,justanothertechnologyforinformationtechnology(IT)tomanage,butitismorethanthat.AIisapervasivetechnology,potentiallywithapplicationsineverydepartment,entangledwithdataandenterprisesystemarchitecture.AIiscapablenotjustofdoingthingsfasterandbetterthanhumans,butofdoingthingsonascalethatpreviouslywouldhavebeenimpracticalforhumans.Itisalreadyandwillcontinuetobeagamechangerinmanyindustries.Inthissense,AIhasthepotentialtobeamonganyfirm’smostvaluableintangibleassets.Atthesametime,AIisnotwithoutrisk.TherearerisksrelatingdirectlytotheimplementationofAI,includingthepossibilityofdatamismanagement,algorithmicbias,erroranddrift.TherearerisksassociatedwiththecomplexityofAIandthechallengeofexplainingoutcomes.Therearefirst-ordereffectsfromAIusethatmayimpactindividualsororganizations.Peoplemaybediscriminatedagainstorwronglydirected.Companiesandindustriesmayexperiencejoblosses.Therearealsosecond-orderconsequencesassociatedwithAIuse,includingdynamicssuchasskillsatrophy,withthepotentialtoimpactindividuals,organizationsandsociety.Inpracticalterms,theESGelevationofbroadlybasedreportinganddisclosurepracticesservestoinforminvestors,instilldisciplineandshapecapitalinvestmentdecisionsinconsiderationofbothnear-termandlonger-termmaterialfactors.Therealityisthatregulationandstandardsneverentirelykeepupwithinnovation,andthereisarequisiteforgoodfirm-levelgovernancetoreachbeyondthesufficiencyofcompliancerequirements.Butitisalsotruethatregulationandstandardscanprovidemeaningfulguidanceandgoalpoststoorganizations.Andtoday,withthelaunchoftheISSBandthepromiseofaforthcomingsetofglobalstandardsforsustainabilityreporting,thepracticeofidentifying,measuring,managingandreportingonawiderangeoffinancialandnon-financialinformationislikelytobecomeinstitutionalized.ThebottomlineisAIuserepresentsbothanopportunityandarisk—itsuseisamaterialconcernforfirms,withthepotentialtoinfluencefinancialperformance,firmreputationandtoimpactstakeholders.ThewidespreadrecognitionofAIasadouble-edgedswordhasspawnedanindustryaroundthecallforitsresponsibleuse.Theseinitiativesmaybereferredtoas“responsibleAI”or“ethicalAI”andwhiletheyacknowledgetheenormityofthetechnology’sbusinesspotential,thefocusisgenerallymorearoundhowAIisgovernedfromariskmanagementstandpoint.TheWorldofAIGovernanceAIgovernanceisanoverarchingtermthatisusedtorefertoawiderangeofapproachesthathaveemergedinresponsetotheperceivedrisksandimpactsassociatedwithAI.ThepotentialforAI-relatedrisksandimpactsisbroadlyacknowledgedtoday,withthepolicyandpracticesarounditcontinuingtoevolveattheinternational,nationalandsubnationallevelsofgovernmentandamongmyriadindustryandNGOactors.ThissectionprovidesanoverviewofsomeofthehighlightsinAIgovernancedevelopments—strategies,policy,guidance,regulations,standardsandpractices—relatingtotherecognitionofAI-relatedrisk.NationalStrategiesTheOrganisationforEconomicCo-operationandDevelopmentcatalogueshundredsofnationalAIpolicyinitiativesfrom69countries,territoriesandtheEuropeanUnion.4Thesearebroadlycategorizedintofourgroups:governance-related(564),financialsupport(294),AIenablersandotherincentives(423)andguidanceandregulation(301).Theseinitiativesarebeingundertakenbygovernmententities,researchandeducationorganizations,privatecompanies,socialgroups,individualeconomicactors(forexample,entrepreneursandprivateinvestors)andintermediaries(forexample,incubators,industryassociationsandtechnologytransferoffices).4Seehttps://oecd.ai/en/dashboards/overview.6CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaEthicalAIFrameworksNumerousorganizationshaveproposedethicalprinciplesforthedevelopmentanddeploymentofAItechnologies.Theseincludewell-knownframeworkssuchastheEuropeanUnion’sEthicsGuidelinesforTrustworthyAI,theInstituteofElectricalandElectronicsEngineers’GlobalInitiativeforEthicalConsiderationsinAIandAutonomousSystems,thePartnershiponAI’sPrinciplesforAIandtheSingaporeModelAIFramework.Theseprinciples-basedframeworksareeachuniquebuttheytendtosharecommonthreads.MostoftheleadingethicalAIframeworksincludeprinciplesrelatingtotransparencyandexplainability,fairnessandbias,accountability,privacyandhuman-centricity.LegalandRegulatoryInstrumentsUntilrecently,legislationrelatingtothedevelopmentanduseofAIwaslimitedtosubnational,sector-specificcases.ThatchangedwiththeintroductionofseveraloverarchingAIlawsproposedatthenationallevel.AI-relatedlegislationwillevolveovertimegiventhedynamicnatureofthetechnologyitself.Iftherewasanyquestionofthis,theinevitabilitywaslaidbarewiththeriseofgenerativeAIasapopulartool,andtheconsequencesforhowAIshouldbedefined.Theplayersinthevaluechainandwhoshouldbeheldaccountablefordifferentcontrolsandresponsibilitiesareimportanttoconsider.→EUProposedAIAct:ProposedinApril2021,theEUAIActrepresentsthefirstexampleofacomprehensiveregulatoryinstrumentforAIoversightatthenational—or,inthiscase,supranational—levelofgovernment.TheEUAIActtakesarisk-basedapproachtoregulatingAI,outliningfourdifferentriskcategories:unacceptablerisk,high-risk,limitedriskandminimalrisk.Thebulkoftheproposedregulationaddressesrequirementsforhigh-risksystems,whichincluderobustapproachestoriskmanagement,datagovernance,technicaldocumentation,record-keeping,transparencyandprovisionofinformationtousers,humanoversightandaccuracy,robustnessandcybersecurity.TheburdenofresponsibilityismainlyplacedonthedevelopersofAI,whohaveanobligationtoimplementaqualitymanagementsystemandotherstipulations,accordingtotheguidanceoftheproposedregulation.Thisgraduatedtieringofrequirementsbasedonriskisoftenreferredtoas“proportionality,”markingeffortsbyregulatorstofocustheregulatoryburdenwhereitismostneededandfreelessriskyapplicationspacestopursueinnovationunfettered.NumerousstandardsarecontemplatedbytheEuropeanUniontosupporttheirlegislativeobjectives.ArecentstandardizationrequestfromtheEuropeanUniontoCEN-CENELEC(theEuropeanCommitteeforStandardizationandtheEuropeanCommitteeforElectrotechnicalStandardization)proposesstandardsin10areascoveringAI,organizationalsystems,dataqualityanddataaccess.→USAlgorithmicAccountabilityAct:In2022,USlawmakersintroducedtheAlgorithmicAccountabilityActinboththeHouseandSenate.TheAlgorithmicAccountabilityActisfocusedonautomatedprocessesandsystemsdeployedtorender“criticaldecisions.”Withintwoyearsofenactment,theproposedactwillrequiretheFederalTradeCommissiontopromoteregulationsthatrequireimpactassessments.Basedontheoutcomesoftheseimpactassessments,coveredentitieswillberequiredtoundertakeactionstoeliminateormitigateimpactsthatdemonstrateamaterialnegativeimpactthatisexpectedtohavealegalorothersignificanteffectonaconsumer’slife.→Canada’sArtificialIntelligenceandDataAct(AIDA):In2022,theCanadianfederalgovernmentintroducedBillC-27intheHouseofCommonsforfirstreading.AsapartofCanada’sDigitalCharter,BillC-27containsthreeseparateactsrelatingtodataprivacyandAI,includingtheproposedAIDA.AIDAisfocusedontheprovisionofnewrulesfortheresponsibledevelopmentanduseofAI.AkeycomponentofthebilloutlinesthatcompaniesmustassesswhethertheirAIsystemsare“highimpact”(tobefullydefinedintheregulations)and,ifso,theymustmeetasetofobligationsaroundriskassessmentandmitigationofbias,systemmonitoring,transparencyandrecordkeeping,noticeandtheuseofanonymizeddata.TheproposedAIDAisacomponentofCanada’sNationalAIStrategy,whichwaslaunchedin2017andupdatedin2020.ThestrategyaimstopromotethedevelopmentandadoptionofAIinCanadawhilealsoaddressingissuessuchasethicalconsiderations,diversityandinclusion,andtheimpactofAIontheworkforce.7AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversight→Canada’sAlgorithmicImpactAssessment(AIA):TheAIAisamandatoryriskassessmenttoolforusewithinthefederalgovernment,insupportoftheTreasuryBoard’sDirectiveonAutomatedDecision-Making.Animpactassessmentsectionisattheheartofthetool,queryingmatterssuchasthelevelofhumaninvolvementinthedirective,reversibilityofadecisionanddurationofimpact.TheoutputoftheAIAisthedeterminationoftheimpactlevelofanautomateddecision-makingapplicationasLevelI(littletonoimpact),LevelII(moderateimpact),LevelIII(highimpact)orLevelIV(veryhighimpact).Basedontheimpactlevel,therearedifferentrequirementsforpeerreview,notice,human-in-the-loopfordecisions,explanationrequirement,training,contingencyplanningandapprovalforthesystemtooperate.AIStandardsNumerousstandardsorganizationsaredevelopingstandardsinsupportoftheresponsibleuseofAI.Belowareexamplesfromtwointernationalleaders,theInternationalStandardsOrganization(ISO)andtheNationalInstituteofStandardsandTechnology(NIST).→ISO42001—AIManagementSystem:ISO42001providesrequirementsforestablishing,implementing,maintainingandcontinuallyimprovinganAImanagementsystem.OrganizationsareexpectedtofocusapplicationofrequirementsonfeaturesthatareuniquetoAI.ISO42001outlinestheneedfororganizationstounderstandtheirinternalandexternalcontextandtheneedsofinterestedparties,andtoestablishthescopeofthemanagementsystemonthisbasis.Thestandardsetsoutrequirementsinsixdifferentareas:–leadership(forexample,AIpolicy,roles,responsibilitiesandauthorities);–planning(forexample,riskcriteriaandriskassessment,systemimpactassessment,AIobjectives);–support(forexample,resources,competence,awareness,communication,documentation);–operation(forexample,processesforoperationalplanningandcontrol);–performanceevaluation(forexample,monitoring,measurement,evaluation,internalaudit);and–improvement(forexample,continualimprovementofAImanagementsystem).→NISTAIRiskManagementFramework(AIRMF):TheAIRMFacknowledgestheuniquerisksassociatedwithAIsystemsandpromotesriskmanagementasakeycomponentintheresponsibledevelopmentanduseofAI.AccordingtoNIST,AIriskmanagementcandriveresponsibleusesandpracticesbypromptingorganizationsandtheirinternalteamsthatdesign,developanddeployAItothinkmorecriticallyaboutcontextandpotentialorunexpectednegativeandpositiveimpacts.TheNISTAIRMFincludesfourfunctionstohelporganizationsaddresstherisksofAIsystemsinpractice:govern,map,measureandmanage.While“govern”appliestoallstagesoforganizations’AIriskmanagementprocessesandprocedures,the“map,measureandmanage”functionscanbeappliedinAI-system-specificcontextsandatspecificstagesoftheAIlifecycle.TheNISTAIRMFcharacterizestheprinciplesfortrustworthyAIasaccurate,validandreliable,safe,explainableandinterpretable,privacy-enhanced,fair,andaccountableandtransparent.NISThasalsoissuedaspecialpublicationtitledTowardsaStandardforIdentifyingandManagingBiasinAI,inwhichitdiscussesthreecategoriesofbias(systemicbias,humanbias,andstatisticalandcomputationalbias)andprovidesupdatedlifecycleguidance(pre-design,design,deployment).Enterprise-LevelAIGovernanceThereviewabovehighlightsthesignificantlevelofactivityintheguidancesideoftheAIgovernancespacetoday.AdifferentquestioniswhatareorganizationsactuallydoinginregardtoAIopportunityandrisk?Anecdotalevidencesuggeststhattheproliferationofprinciples-basedframeworksandguidancehasseepedintothecorporateethos,withmanycompanies,organizationsandgovernmentdepartmentsespousingtheirownsetofethicalAIprinciples.Wherethechallengenowliesisintheoperationalizationofthisguidance.KnowingwhatethicalAIlookslikeisaverydifferentmattertoknowinghowtodoit,muchless8CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaactuallyimplementingtheprocesses,proceduresandaccountabilityregimestoachieveit.AIgovernanceleadersrecognizetherealitythatonesizedoesnotfitall.Best-in-classapproachestoenterprise-levelgovernancetakestockofthearrayofAIgovernanceguidanceandinvolvethecreationoforganization-specificframeworksforethicalAIandthedevelopmentofplansandpracticesforoperationalizingtheseframeworks.ThecalltooperationalizeAIgovernanceprinciplesandframeworksrepresentsasignificantchallengeforenterpriseAI.ThechallengeofoperationalizingethicalAIprincipleswithpracticalapproachesisatthecoreofenterprise-levelAIgovernance.Movingfrom“principlestopractice”isacommonrefraininAIgovernancetoday,andafundamentalweaknessofthearrayofregulatoryapproachesisthefacttheyhavenotstimulatedmuchinthewayofpracticalimplementation.Inmanyrespects,theworldofESGandsustainabilityreportingisgoingthroughasimilarevolution,butalittlefurtherahead.TheworkoftheTaskForceonClimate-RelatedFinancialDisclosure(TCFD)forexample,representsadeterminedintenttoconstructaframeworkthatwouldfacilitateimplementationaswellasdisclosure.TheTCFDrecommendationsarenotnarrowlyfocusedonanorganization’semissionsmetrics,butincludedisclosurerequirementsrelatingtogovernance,strategy,riskmanagementandtargets.TheISSB’sdraftguidanceisbuildingonthesesamedisclosurepillarsandisexpectedtopropagatemoreuptakeandbettergovernanceofsustainabilitytopics.TheprogressthatisbeingmadeonenhancedsustainabilityreportingshouldbehelpfultothecauseofAIgovernanceontwofronts,operationalizationandstandardization.ThedirectionofESGreportingprovidesanexampleofhowtonudgethingsalongthespectrumfromprinciples-basedframeworkstoguidancethatincorporatesexpressionsofaccountability,operationalrequirementsandmetrics.Inaddition,ESGreportingisonthevergeofhavingsomethingelsetoofferinstructionally,throughthestandardizationofsustainabilityframeworksacrossglobaljurisdictions.Thedriveforglobalbenchmarkingofsustainability-relatedrisksandopportunitiestohelpinvestorsmakeinformeddecisionshashighlightedthemeritsofinstitutionalizinganapproachthatbothsatisfiesinvestors’informationneedsandpromotesthedisciplineofgoodgovernance.The“globalsustainabilitystandardstrain”isleavingthestationandtherewillsoonbeglobalbenchmarksandstandardsforawiderangeofsustainability-relatedrisksandopportunities.ItseemsimprudentthatAI,withitspromiseofmassiveeconomicandsocialimpactgoingforward,shouldbeleftout.AIgovernancecanlearnsomethingfromtheprogressionofsustainabilityreportinganddisclosureandadvancethesepracticesoperationally.Forthistohappen,thereneedstobearobustdiscussionaroundthenatureandmaterialityofAIriskandopportunityandtheimplicationsforreportinganddisclosure.ThenextsectionfocusesonthenatureofAI-relatedrisk.TheNatureofAI-RelatedRiskContemporarydictionariesdefineAIasasubfieldwithincomputersciencewherethepracticeisoneofmachinesdevelopingthecapabilitiesofhumans,includingcognition,patternrecognition,reasoninganddecisionmaking,andperformingtasksinahuman-likeway.ThetermsAI,machinelearning,deeplearningandneuralnetworksareoftenusedinterchangeably,buttheprocessesbywhichtheyoperatearedifferentandcreatedifferentlevelsofriskfororganizations.Formanycompanies,itisasubsetofAIknownasmachinelearningthatholdsthemostinterestandwherethebulkofAIdevelopmentistakingplace.Naturallanguageprocessing,neuralnetworksanddeeplearningareallsubsetsofmachinelearning.GenerativeAIisalsoaformofmachinelearning.Thearrayoftoolsandtechniquesthatcomprisemachinelearningareoftenreferredtoasartificialnarrowintelligence.Beyondmachinelearninganddeeplearningisartificialgeneralintelligence(AGI).AGIisabiggerconceptthatinvolvesthecreationofintelligentmachinesthatcansimulatehumanthoughtandbehaviour.LittleprogresshasbeenmadedevelopinghigherformsofAIthatwouldapproachdecisionsmuchinthesamewayhumansdo,understandingorrememberingemotionsandinteractingwithpeople.EventheadventofwidelyavailablegenerativeAIapplications,suchasthe9AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightOpenAIChatGPTapplication,areexamplesofnarrowratherthangeneralAI.ChatGPTdoesnotthinkorfeellikeahuman—itistrainedtoscanandsynthesizeinformationfromanunfathomablyvastonlinedatabase,producingcollatedmaterialfromexistingsources,morelikeasmartdigitallibrarianthanaphilosopherorinventor.Inthispaper,thetermsmachinelearningandAIareusedinterchangeably,asiscolloquialinsocietytoday.Inallcases,theyrefertoartificialnarrowintelligence.BusinessmanagersdonotneedtounderstandthetechnicaldetailsofAI,buttheydoneedtobearmedwithenoughknowledgetoknowwhatishappeningintheirorganizations.ThisincludesknowingwhattypeofAIisbeingusedandforwhatpurposes,whatriskeachapplicationcarriesandwhatapproachthefirmistakingtomanagethatrisk.ThisshouldbeaproactiveexerciseandshouldstartwithanunderstandingofthenatureofAI-relatedrisk.WhatIsDifferentaboutAI-RelatedRiskfromaTechnologyStandpoint?Organizationshavelookedtotechnologyfordecadestosupporttheirbusinessoperations.Financialservices,inparticular,haveusedmodelstoreducerisk,supportregulatorycomplianceandrealizestrategicbusinessneeds.Traditionalmodels,forexample,havebeenusedinbankingtosupportdecisionsandpredictionsinareassuchascapitalprovisioning,strategicplanning,pricing,assetliquidity,customerrelationshipmanagement,moneylaunderingandfrauddetection.WhatisitaboutAIthatamplifiestheriskscenariofromtraditionaldecisionmodels?Machine-learningmodelshavedeliveredahostofbenefitsthroughtheirabilitytoscaleupdecisionmakingorpredictionsandaddresstasksthatarebeyondtraditionalmodelsduetotheamountofdatainvolvedandtheconstraintsoftime.Somewhatironically,theavailabilityofbigdataisoneofthefactorsthathasfacilitatedgrowthinmachinelearning,alongwithaccesstogreatercomputingpowerandprogressinalgorithmicdevelopment.Butwiththesedevelopmentscometrade-offs.Incontrasttotraditionalmodels,machine-learningmodelsaredynamicandnon-deterministic.Theircapacitytolearnisthefoundationoftheirabilitytooperateatscale,butthisalsointroducesvariabilityintotheprocess.Machine-learningmodelscangooff-track.Inmachinelearning,themodelistypicallydesignedtodescribe,predictorprescribesomething—theseareknownastargetvariablesoroutcomes.Howdomachineslearntodothis?Machinelearningstartswithdata,anditmayincludespreadsheets,text,pictures,financialtransactions,readingsfromsensorsortheinformationinannualreports.Thebulkofthedataavailableisusedtotrainthemachine-learningmodelandasmallportionofitisreservedtotesttheperformanceofthemodel’saccuracy.Therisksandunintendedconsequencesthatemergefromthecoremachine-learningapplicationitselfderivefromthreeareas:thehandlingofdata,characteristicsofthemodelanddeploymentofthemodelovertime.→Data-relatedrisk:LeveragingdataforAIisabusiness-drivencall,aboutharnessingthelatentpowerindatatogetatstrategicallyvaluableinformation.Thiscreatesaneedtomakedatabroadlyavailableacrosstheenterprise,andintroducesdistinctrisksintermsofdatastorage,datasecurityanddataprivacy.Additionally,themachine-learningprocesshasitsownchallenges,withdatapotentiallymigratingacrossmultiplepartners,organizationsand/orcountries.Thepresenceofdifferentlaws,policiesandethicsateachstagemayimpactmodellearning.Somethingthatisconsideredethicalinonejurisdictionmaybeconsideredunethicalinanother.→Characteristicsofthemachine-learningmodel:Theriskderivingfromthemachine-learningmodelmayrelatetomodelquality,modelcomplexityortheapproachtomodellearningandvalidation.Modelqualityiscompromisedbythemisapplicationofmachine-learningalgorithmstofindpatternsindatawherenoneexist.Modelcomplexityincreasesasdesignpassesfrommachinelearningtodeeplearning.Theflipsideofthepowerofneuralnetworksistheiropaqueness,generatingoutcomesthatmaynotbeeasilyexplainable.Model-learningstyle,andspecificallytheuseofsupervisedversusunsupervisedlearning,underliestheinherentriskinmachinelearning.Withunsupervisedlearningthereisnooutcomevariableonwhichtotrainthemodel,andthealgorithmisonitsowntosortthroughdataforpatternsandstructures.Thiscreatesadifferentmanagementandoversightchallenge.10CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargava→Lifecycleofthemodelandmodeldrift:OneofthesourcesofAI-relatedriskderivesfromtheAIlifecycle—theprogressionofstepsthatincludesdataactivities,modeldesign,modeldevelopment,modeldeployment,monitoringformodeldriftandmodeldisposal.Therearedifferentriskswitheachstageofthelifecycleandtheinevitabilityofdatadriftpresentsacontinuouschallenge.Machine-learningmodelsaredynamicandthisiswhatreallydistinguishesmachine-learningrisk.Datachangesbecausetheworldchanges.Amodeltrainedonwinterseasonaltemperatureswillmakedifferentpredictionsthanonetrainedonsummertemperatures.Becauseofdrift,somemodelsneedretraining,evenonaregularbasis.Inmachinelearning,modeldriftisnormal;itissomethingtobeexpectedandmanaged,butitprovidesamovingtarget.Therisksassociatedwithmachine-learningtechnologyhaveimplicationsforitsuseinoperationalsettingsandthisdemandsnewgovernanceapproaches.Machine-learningriskstodayarereal,butthepotentialforreallybadthingstohappenismorerelatedtomanagementandoversight,thananymysticalcapacityofthetechnologyitself.Thereisnothingsuper-humanabouttoday’sAItechnologythatshouldenableittoescapehumancontrolanddirection.ThegenerativeAIapplicationspresentatrickierchallengebecauseofthevastnessofthedataonwhichthemodelsaretrainedandonwhichtheyrunandcontinuetolearn.Butevenhere,intheory,humandevelopershavecontrolofwhatdatathemodelsarefedandtheability,withlabelling,toparseoutundesirabledataelements.Machinelearningisatool,andlikeotherpowerfultoolsthroughouthumanhistory,itcanbeusedforgoodorbad,andevenwithoutintention,itsusecanhavenegativeconsequences.ThisunderliesthecallfortrustworthyAI.WhatIsDifferentaboutAI-RelatedRiskfromanOrganizationalStandpoint?Inadditiontothetechnology-relatedrisksofAI,therearerisksandimplicationstoitsdevelopmentanduseinanorganizationalcontext,asatoolforpracticalapplication.TherearefourfactorscharacterizingtherisksassociatedwithAIinthecontextofitspracticaluse:→Speedandscale:ThespeedandscalewithwhichAIcandriveoutputsmightbeviewedaswhatunderliestheopportunityforthistechnology,butalsopresentsanexacerbatingconditionforrisk.AIdonewellyieldsenormousupside.Conversely,AIdonebadlycanbeverybadforbusiness—nottomentionpeopleandtheplanet.ThisrealityisthemotivatingforcebothtoembracetheopportunityofAIandcultivaterobustAIgovernance.→AIasempowering:AIisnotapassivetechnology.Itharnessesthelatentpowerofdatatowardsomeendthatwillhaveaneffect.Placingthistoolintothepipelineofanentity’soperationmeansthereisgoingtobeaneffectonsomething—anindividual,agroup,theplanet,thecompany,astakeholder.ThefactAIis,bydefinition,evidencebased,islikelytoempowerthosewhohaveaccesstoit,andpromotemoreautomateddecisionmaking,forgoodorbad.→TheAIlifecycle:ThelifecycleassociatedwithAI(i.e.,fromdatathroughdesign,development,deploymentandongoingmonitoringfordrift)constitutesanewchallengeforgovernanceandoversight.Thefactthatrisksvaryatdifferentstagesofthelifecycleisoneriskbutthebiggerchallengeistheinevitabilityofdriftandthepersistenceofchangeafterdeployment.Thishasimplicationsforthetypesofpolicies,processesandorganizingstructuresthatfirmsneedtohaveinplacetoeffectivelygovernAI.→EthicalAI:ThereareuniquechallengesinansweringthecallforAIthatishuman-centric,fairandnon-discriminatory,transparentandexplainable,highqualityandaccurate,safeandsecure,compliantwithprivacyrulesandsubjecttoclearaccountability.Thishasimplicationsforpolicymakersinsideorganizations,butalsoforemployeesmorebroadly,astheroleofdataanddataanalyticsislikelytopermeateallaspectsofenterpriseactivity.TheimportantthingtorecognizeaboutAI-relatedriskisthefactthatitderivesbothfromuniqueaspectsofthetechnologyandfromthepracticalrealitiesofhowAIisdeveloped,usedandmanagedbyindividualsandorganizations.AlongsidetheopportunitysideofAI,theserisksmaypresentandevolveinvaryingcombinations,givingrisetopotentiallegal,regulatory,reputationalandfinancialimpacts.11AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightTheMaterialityofAI-RelatedRiskInFebruary2023,Googlesharesdroppedbyninepercentinasingleday—astunningUS$100billionlossinmarketcap—inthewakeofrevelationsthatBard,itsrecentlyreleasedchatbot,hadproducedafactualerror.ThiswasnotthefirsttimeAIhadgonebadlywithacost,butitwasthefirsttimetheworldsawagiantstumbleandfallhardwithAI.TherearehugeexpectationsforgenerativeAIandAIingeneral.Thecapitalmarket’sresponsetoBard’sshortcomingsistheliteralembodimentofthematerialityofAI-relatedrisk.TheideaofmaterialityisatthecoreofESGandsustainabilityreporting.ThematerialityofAI-relatedrisk(andopportunity)underliestheargumentforincorporatingAIintosustainabilityandESGframeworks.Initsearlywork,theISSBhasaligneditsdescriptionofmaterialitywithIFRSAccountingStandards,stating“theIFRSFoundation’sfocusisonmeetingtheinformationneedsofinvestors.Therefore,theISSBusesthesamedefinitionof‘material’thatisusedinIFRSAccountingStandards—thatis,informationismaterialifomitting,obscuringormisstatingitcouldbereasonablyexpectedtoinfluenceinvestordecisions.”5TheGooglesharepriceslideprovidesconcreteevidenceofthefinancialimpactofAI-relatedrisk,butitisjustoneisolatedexample(AIlacksaccuracy),withoneparticularapplication(ChatGPT/generativeAI).ConsideringthebroaderecosystemofAItechnologiesandusecases,whatcanwesayconstitutesmaterialinformation?BuildingonthepaththeISSBiscarvingtowardaglobalbaselineofsustainability-relatedreportinganddisclosure,thequestionthatlogicallyfollowsis:WhatisitabouttheAIthatfirmsaredeveloping,procuring,deployingandusing,thatcouldreasonablybeexpectedtoinfluenceinvestors’decisions?And,morespecificallywithregardtoAI-relatedrisk:WhatAI-relatedriskismaterialinasustainabilitycontext?AcceptingthattherearenovelrisksassociatedwithAIintermsofboththetechnologyandits5Seewww.ifrs.org/groups/international-sustainability-standards-board/issb-frequently-asked-questions/.practicalapplication,isitreasonabletosuggestthattheverypresenceofAIinanorganization’svaluechainismaterial?Morelikely,therewillbesomethresholdabovewhichinformationpertainingtoAI-relatedriskwillbematerialbut,critically,thisinformationwillincludeinsightintoboththetechnologyandtheorganizationalstrategy,policiesandprocessesforaddressingtherisk.Andbecauseanorganization’sambitionsandstrategiesforAIaretypicallyexpressedattheenterpriselevel,whilethetechnologyismostoftenappliedtosolveaparticularbusinessproblemoropportunity,therewillbematerialinformationatbothlevels.Theauthorssuggesttherearethreedifferenttypesofdisclosureinformationthatmaybeconsideredmaterial:i)enterprise-levelinformationaboutsystemicstrategy,policies,processesandprocedures;ii)usecase-specificinformationaboutpolicies,processesandprocedures;andiii)measuresandmetricsofperformanceforperformanceevaluationofi)andii).DisclosureonPolicies,ProcessesandProcedures(EnterpriseLevelandUse-CaseLevel)Whileanorganization’sgrandstrategyforAIandpoliciesarounditsdevelopment,procurementandusewill(hopefully)liveattheenterpriselevel,therealityismanyAIprojectstakeplacewithinanentity’sbusinessunits,whereAIisappliedtosolvespecificbusinessproblems.TheAIusecasesthatinterestbusinessunitsmaybetotallydifferent,withdifferenttechnologyandgovernanceimplications.Thismeanstherewillbeinformationthatismaterialtoreportinganddisclosureatboththeenterprise-levelandtheuse-caselevel.Howanentityisorganizedintermsofaccountability,rolesandresponsibilitiesandaccordingtowhatpolicies,processesandproceduresismaterialtothetopicofAI-relatedrisk.PerformanceMeasuresandMetricsInformationabouttheperformanceoftheenterprisewritlarge,andpertainingtoitsspecificAIuse-casesystemsismaterial.Foreachlevel,thereshouldbeassociatedmeasuresandmetricsforevaluationofperformanceascomparedtobenchmarksandgoals.12CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaThesemetricswillneedtocaptureinformationabouttheperformanceofthetechnologyinthetraditionalsenseofmodelaccuracy,butalsoacrossarangeofmeasuresfortrustworthyAI,includingtopicssuchastransparency,explainability,fairnessandbias.Informationaboutbothanorganization’sgoalaspirationsanditsperformanceversusgoalswillbematerialtoenterprisevalueand,therefore,toinvestors.MaterialityofAI-RelatedRiskandDirectionoftheISSBTheideaofscopingthematerialityofAI-relatedriskaccordingtoorganizationalsystemsandprocesses,ontheonehand,andperformancemetrics,ontheother,alignswiththeISSB’sguidanceinitsExposureDraft(S1)“GeneralRequirementsforDisclosureofSustainability-relatedFinancialInformation.”6TheISSB’sdraftguidancedescribessustainability-relatedfinancialinformationasbroaderthantheinformationreportedinfinancialstatements,andpotentiallyincludinganentity’sgovernanceofsustainability-relatedrisksandopportunities:thestrategyforaddressingthem,theexpectedimpactofrelateddecisionsoncashinflowsandoutflows,theentity’sreputation,performanceandprospectsasaconsequenceofrelatedactionsanditsdevelopmentofknowledge-basedassets.TheISSB’sdraftguidanceidentifiesfourareasofcoredisclosurecontent:governance,strategy,riskmanagement,andmetricsandtargetsastheyeachrelatetoanentity’sapproachtoidentifying,addressing,managingandmonitoringsignificantsustainability-relatedrisksandopportunities.TheExposureDraft(S1)representstheISSB’soverarchingsetofgeneraldraftguidance,anditisunderstoodthatmorespecificstandardswillfollow,addressingdiscretesustainabilitytopicsand,possibly,addressingtheseonanindustry-basisasappropriate.Inthemeantime,theISSB’sguidanceinstructsentitiestousethedraftguidanceand“considerthedisclosuretopicsintheindustry-basedSASBStandards,theISSB’snon-mandatoryguidance(suchastheCDSB6TheISSB’sExposureDraft(S1)“GeneralRequirementsforDisclosureofSustainability-relatedFinancialInformation”wasreleasedinMarch2022forpubliccomment.ItisdesignedtobethesustainabilityequivalentofIAS1“PresentationofFinancialStatements”thatdefinesacompletesetoffinancialstatementsandIAS8“AccountingPolicies,ChangesinAccountingEstimatesandErrors”thatprovidesguidanceontheestablishmentandimplementationofaccountingpolicies.Frameworkapplicationguidanceforwater-andbiodiversity-relateddisclosures),themostrecentpronouncementsofotherstandard-settingbodieswhoserequirementsaredesignedtomeettheneedsofusersofgeneralpurposefinancialreporting,andsustainability-relatedrisksandopportunitiesidentifiedbyentitiesthatoperateinthesameindustriesorgeographies”(IFRS2022).SASBStandardsinanAIContextMaterialityisaboutpreparingbusinessestoaddressrisksandopportunities,anditfollowsthattheissuesofgreatestimpactinminingpreciousmetalsarenotthesameasthoseinhealthcareorbanking.Likewise,therisksforAIinlawenforcement,thejudicialsystemandhealthcarearenotthesameasinentertainmentormanufacturing.Forthisreason,itislikelythatanindustry-specificapproachtoAI-relatedsustainabilitystandardswillbeappropriatetocomplementtheISSB’sfourareasofcoredisclosurecontent.TheSASBframeworklendsitselftoindustry-andissue-specificdisclosures,asindividualstandardsareoutlinedacross77industriesinsixsectors,andarecharacterizedaccordingtofivedimensions:environment,socialcapital,humancapital,businessmodelandinnovation,andleadershipandgovernance.ForcompaniesworkingwithAI,therearepotentialimpactsineachSASBdimension.Thisraisestwoobviousquestions:→DotheSASBstandardscapturedisclosurerequirementsforanythingmaterialtoAIthatisnotcoveredbytheISSB’sdraftguidance?→DotheSASBstandards,astheyexisttoday,adequatelycoverthesustainability-relatedrisksandopportunitiesassociatedwithAI?ItisdifficulttoanswerthefirstquestionwithoutaclearersenseofhoworganizationswillusethenewISSBdraftguidance,but,onbalance,thecharacteroftheISSBdisclosurerequirementslooksdifferenttoSASB.Thedisclosurerequirementsinthedraftguidancereadlikerequestsforinformationaboutmacro,enterprise-levelsystemsandprocessesforoversight.Incontrast,SASBqueriesmorespecifictopicssuchasemissions,humanrightsimpactandconsumerprivacy,anddoesthisonanindustry-by-industrybasis.Therewillbesomeoverlapforsure,especiallyonquestionsrelatingtobusinessmodelimpactandgovernancetopics,whichmakeanappearancein13AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightbothframeworks.Nonetheless,therearereasonablegroundstobelieveSASBwillfunctionasanicecomplementtotheISSGgeneralrequirements.Turningtothesecondquestion,then,aretheSASBstandardsoftodayadequate?Thisquestioncanbebrokendownintoseveralsub-questions,eachofwhichwillneedtobeaddressedindividually:a)AretherematerialtopicsrelatingtoAIrisk(andopportunity)thatarenotcurrentlycapturedbytheSASBframework?b)IsthematerialityofAI-relatedrisk(andopportunity)differentindustry-to-industry?c)IsthereadividebetweenthematerialissuesfacinginherentlydigitalAIcompaniesand/orusecases(forexample,frauddetection)versusAIproductionapplications(forexample,advancedroboticsinmanufacturing)?d)Isthereadifferenceinthematerialityofrisks(andopportunities)forthedevelopersversusprocurersversususersversusplatformprovidersofAI?Thefirstthreequestions,capturedinpointsa),b)andc),arereallyaskingthequestionshouldtherebeanew(horizontal)topic(s)intheSASBframeworkpertainingtoAI?Thefinalquestion,posedinpointd),ismorenuanced:IfitisdecidedthatAIdevelopersfacedifferentmaterialrisks,doesthatpointtoadiscretesetofquestionsina(nother)newhorizontaltopicor,alternatively,areAIdeveloperssufficientlydifferenttoothersoftwarecompanies,suchthatanewvertical“AIdeveloperindustry”mightbewarranted?Pointa)—SufficiencyofCurrentSASBStandardsforAITheauthorscontendthattheSASBstandardsdonotadequatelycaptureAI-relatedriskandopportunitytoday.TheSASBrequirementsforsoftwareandITservicesbestillustratethisbecause,arguably,thisistheindustrywiththemostoverlapwithAI.TheSASBstandardsaskfirmsinthesoftwareandITservicesindustrytomeasureandreportonsixmetricsinfourissuecategories:environmentalfootprintofhardwareinfrastructure(environment);customerprivacyanddatasecurity(socialcapital);employeeengagement,diversityandinclusion(humancapital);andcompetitivebehaviourandsystemriskmanagement(leadershipandgovernance).Letusfocusonthecustomerprivacyanddatasecurity,again,offeringlotsofoverlapwithAI.Figure1showsthattheSASBstandardsforsoftwareandITservicesincludeasetofquantitativemetricsandarequestfordescriptionofpoliciesandpractices,forbothcustomerprivacyanddatasecurity.Currently,therequestisforpolicies,practicesandperformanceonkeymetricsrelatingtodataprivacyanddatasecurityinagenericway.InthecontextofAI,andassumingapplicationswherethelevelofriskorimpactcrossesthethresholdtowarrantreporting,therewouldbedifferentquestions.Thesequestionswouldpertaintodata,modelsandoutcomes,specificallyinthecontextofanAIusecase,andpotentiallyincorporatingqueriesrelatingtothedifferentstagesoftheAIlifecycle.ThesemightincludedisclosurerequirementspertainingtothetrustworthinessofAI,includingexplainability,fairness,biasandaccuracy.Additionally,therewouldbequestionstoextractinformationaboutanentity’senterprise-levelsystemsforgoverningAIincludingqualitymanagement,riskmanagementandprivacymanagementapproaches.Thesetypesofquestionsandmeasures,astheyapplytoAI,areabsentfromthecurrentSASBframework.Furtheranalysisislikelytouncovernumeroustopicsforconsiderationacrossmultipledimensions.AIisapervasivetechnologyandcanbeexpectedtohaveimpactsonindividuals,society,theplanet,businessmodelsandgovernance.Theseimpactswillbebothpositiveandnegative—therewillbeopportunitiesandrisks.OncetheISSBdecidestoevaluateAI-relatedrisksandimpacts,newandexpandedchallengesforframingbothriskandopportunityarelikelytobeuncovered.Pointsb)andc)—DifferencesinMaterialityofAIbyIndustryItseemsintuitivethattherisksandopportunitiesassociatedwithAIwillvary,butaninterestingquestioniswhethertheyvarybyindustryorusecase,andhowSASB(oranyotherframework)wouldhandlethat.Superficially,itmightappearthatabankwouldhavematerialissuesrelatingtoAIwhereasamanufacturingcompanywouldnot.ButwhatifthemanufacturingcompanyisnotjustusingAIinrobotics,butisalsousingittoscreenresumes,orforaprogramrelatingtophysicalsafetyontheplantfloor?Andhowquickshouldwebeto14CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavadismissthematerialityofAIinrobotics?Thehighprobabilityisthatwithinthenextdecade,mostindustriesandeverycompanyofacertainsizewillbeusingAIandfacingassociatedmaterialriskandopportunity.ThisreinforcestherequirementforAI-relatedtopicsashorizontaladditionstotheframework,andsuggeststhevariabilitywithwhichtheframeworkmayapplythemacrossindustries,whichSASBiswell-suitedtoaccommodate.Pointd)—DifferencesinMaterialityofAIbyTypeofAIActorForstarters,letusmakeitsimpleandconsideronlywhetherthematerialityofrisksandopportunitiesmaybedifferentfordevelopersversususersofAI.ThisquestionmightpointtothemeritsofhorizontaltopicadditionstotheSASBframework,toaccommodatetheparticularrisksandopportunitiesthatAIdevelopersface.Alternatively,therecouldbeanargumentthatAIdevelopment,atleastforthoseorganizationsthatareintenselyinvolvedinpioneeringAIdevelopment,warrantsitsownnewvertical.ThiswouldbethecaseifthishypotheticalverticalofintenseAIdeveloperswasfoundtohaveasufficientlyuniquesetofmaterialdisclosuretopicsthatotherindustries—principallysoftwareandIT,butalsogenerallyanyindustrythatisusingAIanddoinglittlein-housedevelopment—donot.Thisquestionwarrantsfurtherinvestigativeanalysisandis,inthefullestsense,beyondtheremitofthispaper;however,weofferpreliminarythoughts.AppreciatingthattheSASBframeworkconsidersmaterialityaccordingtoindustrysectorandcapitaldimension,therecouldbeananalysisofthedistinctrisksthattheuseofAIdevelopmentposeswithineachofthefivedimensionsandhowthesearedistincttothoseofAIdeployersandusers.Thetablebelowprovidesasampleofthetypesofquestionsthatstakeholders,includingregulators,investorsandmembersofthegeneralpublic,maywantinsightinto.Sowheredoesallthisleaveus?Wehavewide-rangingguidancefromtheworldofAIgovernancewherethereisnostandardizationbutcommonthemesaroundprinciplesandriskhaveemerged.TherisktopicsthatAIgovernancetheoristsespousealignwellwiththegeneralframingofESGrisk,butAIgovernanceapproachesare—withsomeexceptions—stillprettylightonoperationalapproachesincludingguardrails,measuresandFigure1:SoftwareandITServicesSustainabilityDisclosureTopicsandAccountingMetrics,DataPrivacyandFreedomofExpression,DataSecurityDataPrivacy&FreedomofExpressionDescriptionofpoliciesandpracticesrelatingtobehavioraladvertisinganduserprivacyDiscussion/AnalysisNumberofuserswhoseinformationisusedforsecondarypurposesQuantitativeTotalamountofmonetarylossesasaresultoflegalproceedingsassociatedwithuserprivacyQuantitative(1)Numberoflawenforcementrequestsforuserinformation,(2)numberofuserswhoseinformationwasrequested,(3)percentageresultingindisclosureQuantitativeListofcountrieswherecoreproductsorservicesaresubjecttogovernment-requiredmonitoring,blocking,contentfiltering,orcensoringDiscussion/AnalysisDataSecurity(1)Numberofdatabreaches,(2)percentageinvolvingpersonallyidentifiableinformation(PII),(3)numberofusersaffectedQuantitativeDescriptionofapproachtoidentifyingandaddressingdatasecurityrisks,includinguseofthird-partycybersecuritystandardsDiscussion/AnalysisSource:SASB(2018).15AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightmetrics.WehavedraftguidancefromtheISSBonhowaglobalstandardwillbebuiltfordisclosureofsustainability-relatedrisksandopportunities,wherethereisinadequateconsiderationofAI-relatedrisks.Basedonthegeneralrequirementsinthedraftguidanceandtheindustry-specificguidanceintheSASBstandards,wehaveatemplatefromwhichtoworktointegrateAIrisk-relatedconcernsintoESGreportinganddisclosure.ThesixthsectionofthepaperexploreshowwhatweknowaboutAI-relatedriskandAIgovernancecanbelacedintotheevolvingguidanceonsustainability-relatedreportingTable1:AI-RelatedRiskPosedbyDevelopersversusDeployers(ExampleQuestions)AI/Machine-LearningDevelopersAI/Machine-LearningDeployers/Users1.Whatweretheethicalandlegalconsiderationsthatguidedalgorithmicdevelopment?2.Whatlevelofeducationandexperiencedothemachine-learningprogrammers/developershave?3.Whatdatawasselectedtotrainthealgorithmandwhy?4.Whatapproachtoalgorithmictrainingwasused?5.Howhavehumansbeeninvolvedinevaluatingandconfirmingthemachine-learningmodel?6.Howarecustomerstrainedintheuseandongoingdeploymentofthemachine-learningmodel?7.Towhatextentcanthefirmexplainhowthemachine-learningmodelmakesthedecisionsthatitdoes?8.Hastheorganizationeverbeenfoundtobenon-compliantwithlegislationand/orregulationsrelatingtotheuseofAIinanyjurisdictionwhereitoperates?9.Hasanycustomeroftheorganizationbeenfoundtobenon-compliantand/orsubjecttoalawsuitinrelationtoitsuseoftheAIsuppliedbytheorganization,wherethesourceoftheproblemhasbeenidentifiedasthepurchasedproduct?10.Howmuchenergydoestheorganizationuseannuallytopowerthecomputersthattrainthemodels?11.ShouldtherebeanewAI/machinelearning-specificstandardwithinsocialcapitaladdressinganindividual’srighttoknowwhethertheyaresubjecttoahuman-ledversusmachine-drivendecision?12.ArethecurrentdisclosurerequirementsrelatingtodatasecurityanddataprivacysufficientinanAI/machine-learningworldwherebroadergovernanceconsiderationslikesource,quality,accuracy,consentandtheintegrityofwhole-processmanagementplayarole?13.Doexistingdisclosuretopicssuchashumanrights,datasecurityanddataprivacyneednewaccountingmetricsthatarespecifictoAIandmachinelearning(i.e.,thereareconsiderationsfordatamanagementwithAIthatareincrementaltootherdataapplicationsbasedontheneedtotrain,finalizeandthenrunmodels)?14.Shouldanorganizationthatemploysmachinelearningreportitsapproachtomonitoringmodeldriftandincidenceofdrift?15.Shouldorganizationsusingmachinelearningreportwhereitisbeingusedandhowtheyhavemodifiedtheirgovernanceapproachesasaresult?16.DomembersofthepublicalwayshavearighttoknowwhenAI/machinelearningisbeingusedandwhentheyaresubjecttoadecisionmadebyamachine?Source:Authors.1.2.3.4.5.6.16CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaanddisclosureinsupportofinvestorneedsandlong-termenterprisevaluecreation.IntegratingMaterialAI-RelatedRisksintoESGThispaperaddressesthechallengeofintegratingAI-relatedrisksintoESGreportingbytakingtheISSB’sdraftguidanceasatemplate(includingtheSASBstandards)andexploringwhatsalientpiecesofAIgovernancepracticemightbelayeredinasinputs,andhowthismightbeaccomplished.ThepurposeofthisexerciseisthepromiseofamorestructuredandstandardizedglobalapproachtoAIgovernance,bothprinciplesandpractice.ThisglobalbaselineshouldbeconstructedtofacilitatetheoperationalizationofarobustAIgovernancepracticethrough:→theestablishmentofenterprise-levelandusecase-levelpolicies,processesandprocedures,withassociatedmetrics/targets;and→theestablishmentofanaccountabilityregime,includingclearrolesandresponsibilitiesforAIgovernanceprocesseswithintheentity.ThediscussionbelowsuggestsanapproachtodevelopingstandardsforAI-related,sustainability-relateddisclosuresthatembracesbothgeneraldisclosurerequirements(i.e.,basedontheISSBdraftguidance)andspecificdisclosurerequirements(i.e.,basedontheformatofindustryandtopic-specificdisclosurerequirementsasfoundintheSASBstandards).ThedecisiontoworkwithbothgeneralandspecificdisclosurerequirementsreflectstheguidancefromtheISSB,butalsoreflectstherealitythatdifferentcompaniesandindustrieshavevaryingroles,engagementandintensitywithAIsystems.AseriesofpreliminaryrecommendationsaremadeforanapproachtointegratingmaterialAI-risksintoESGreportingframeworks.TheserecommendationswillbeofinteresttoanyonewithaninterestintrustworthyAI,AIgovernanceandESGreporting,butarespecificallyaimedattheworkinggroupsoftheISSB,withaviewtobuildingouttheevolvingglobalsustainabilitystandardstoincludeAI-relatedriskandopportunity.WhilethefocusofthispaperhasbeenontheintegrationofAI-relatedriskintoESGreporting,theauthorsacknowledgethatbothAI-relatedriskandopportunityissuesarematerialtoinvestorsandstakeholders.Inmakingtheserecommendations,theauthorsthereforeparentheticallyacknowledgeAI-related“opportunity”despitenothavingdelvedintothenatureofthatopportunityinanyfulsomeway.→Recommendation1:Sustainability-relatedrisks(andopportunities)relatingtoorganizations’development,procurementand/oruseofAIshouldbeincludedintheeffortsoftheISSBtodeliveracomprehensiveglobalbaselineofsustainability-relateddisclosurestandardsthatprovideinvestorsandothercapitalmarketparticipantswithinformationaboutcompanies’sustainability-relatedrisksandopportunitiestohelpthemmakeinformeddecisions.→Recommendation2:InapproachingtheintegrationofAI-relatedsustainability-relatedrisks(andopportunities)intotheglobalbaselineforsustainabilityreportinganddisclosure,considerationshouldbegiventobothgeneraldisclosurerequirementsandindustry-specific/topic-specificrisks(andopportunities).→Recommendation3:AspartoftheprocessofundertakingintegrationofAI-relatedsustainability-risks(andopportunities)intotheglobalbaselineforsustainabilityreportinganddisclosure,adefinitionofAIshouldbeconstructedthroughconsultationwithstakeholders.GeneralDisclosureRequirementsAstartingpointforthedevelopmentofrequirementsforgeneraldisclosure,istoconsidertherisks(andopportunities)ofAIforeachofthefourcorecontentareasprovidedintheISSBdraftguidance:governance,strategy,riskmanagement,andmetricsandtargets.Thereisachallenge,however,inablanketapproachtogeneraldisclosureonAI,intheabsenceoforganizationalcontext.Forthisreason,anyassessmentofthefourcorecontentareasshouldbeaccompaniedbyanOrganizationalAIStatementofContext,clarifyingtheorganization’sroleandintensitywithrespecttoAIsystems.ThecontentofthestatementofcontextmightborrowfromorevenreferencetheISO42001ContextoftheOrganizationstandard.Asanexample,thematerialtopicsfordisclosuremaybeexpected17AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversighttovarywiththerole(s)oftheorganizationwithrespecttoAIsystems,includingAIdevelopmentforownuse,AIdevelopmentforusebyothers,AIprocurementforownuse,AIprocurementforusebyothers,platformhostingAItoolsforownuseandplatformhostingAItoolsforusebyothers.→Recommendation4:AI-related,sustainability-relatedgeneraldisclosureshouldincludeanOrganizationalAIStatementofContextregardinganorganization’srolewithrespecttoAIsystems.→Recommendation5:ConsiderationshouldbegiventotheconstructionofguidancesuchthatanOrganization’sStatementofContextwillinfluencetypeofdisclosurerequiredrelatingAI-related,sustainability-relatedrisk(andopportunity).TheextenttowhichcorecontentquestionsarerelevantislikelytohingeontheroleandintensityoftheorganizationwithrespecttoAIsystemsandtheanticipatedopportunities,risksandimpactsoftheirdeployment.InorganizationsthatarelightusersofAI,forexample,itmaymakesensetoreportanddiscloseonAI-relatedrisksandopportunitiesinthegeneralbasketoftheorganization’ssustainability-relateddisclosures.Ontheotherhand,anorganizationthatisintenselyactiveinAIdevelopmentand/ordeployment,orinvolvedwithhigh-risk,high-impactAIimplementations,shouldbeencouragedtoundertakeAI-specificsustainability-relateddisclosuresduetothenoveltyandpotentialimpactofthetechnology.→Recommendation6:Considerationshouldbegiventotheestablishmentofatier-orthreshold-basedapproachtogeneraldisclosurerequirementsinrelationtoAI-relatedrisksandopportunities,reflectingthemeritsofproportionalityandthedesirabilityofavoidingundulyburdensomereportingonorganizationswhouseonlylow-riskorlow-impactAI.WiththeOrganizationalStatementofContextinhand,thetypesofsustainability-relatedinformationthatmightbeconsideredmaterialfordisclosureongovernance,strategy,riskmanagement,andmetricsandtargetscanbefleshedout.BorrowingfromtheISSBdraftguidance,arangeofquestionsqueryingAI-related,sustainability-relatedmaterialinformationshouldbedeveloped.ExamplesofthetypesofquestionsthatmayberelevantforeachcoreareaarefoundintheAppendix,andreflecttherealitythatAIusecasesareuniquefromonetoanother,andtendtounfoldatthesub-enterpriselevel,withindiscretebusinessunits.→Recommendation7:TheISSBDraftGuidanceforGeneralRequirementsforDisclosureofSustainability-relatedFinancialInformationshouldbeevaluatedwithaviewtoadaptingandexpandingonthequestionsandtopicstoreflectthebreadthofmaterialAI-related,sustainability-relatedrisks(andopportunities).Industry-SpecificRequirementsInrecognitionofthefactthattheopportunity,riskandapplicationofAIwillvaryacrossindustries,andthattheremaybematerialdifferencesinparticularbetweenusersanddevelopersofAI,itisrecommendedthatthedevelopmentoffutureglobalstandardsforAI-relateddisclosurecontemplatetheindustry-specificapproachoftheSASBstandards.→Recommendation8:TheSASBstandardsshouldbeevaluatedwithaviewtoestablishingtheiradequacyincoveringthequestionsandtopicsthatreflectthebreadthofmaterialAI-related,sustainability-relatedrisks(andopportunities)totheextentthattheseare:–notlikelytobewell-coveredthroughgeneraldisclosurerequirements;–likelyorsuspectedtoincludedifferentmaterialrisks(andopportunities)acrossdifferentindustries;and–likelyorsuspectedtoincludedifferentmaterialrisks(andopportunities)dependingonwhethertheorganization’sroleinAIispredominantlyasadeveloperorauserorboth.ConsiderationsforaPathForwardFororganizationsthataredevelopingorusingAIandmachinelearning,ordoingboth,therearegoodreasonstoconsiderthespecialgovernancechallengesthatcomewithit.Therequirement18CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaforaccountability,transparencyandfairnessrelatingtoAIisagrowingpublicexpectation,andisbecomingalegalrequirementinsomejurisdictions.Thesearematerialquestionsforfirmsandtheirstakeholders,includinginvestors.ThequestionofhoworganizationsshouldreportanddiscloseonmaterialinformationpertainingtotheirAI-relatedrisksandopportunitiesiscomplex,tappingregulatory,legal,compliance,ethicalandpublicrelationsconsiderations.ESGreportingprovidesathoughtfulapproachforhowtoassessmaterialrisksandopportunitiesrelatingtoAIandmachinelearning,butdoesnotyetopenthedoorpracticallyfororganizationstomeasure,monitorandreportonitsuse.NoneoftheprevalentESGframeworkstodayincorporatestandardsspecificallydesignedfordisclosurerelatingtoAI.Thispaperaimstostimulatethatpossibility,withspecificintenttoengagetheISSBanditsworkinggroupsinconsiderationoftherequirementsforAI-related,sustainability-relatedreportinganddisclosure.TherecommendationssetforthinthispaperhavebeendevelopedinrecognitionoftheISSB’smomentuminestablishingaglobalsetofstandardsaroundreportingontheESGimpactsoffirms,andtherelevanceoftheseimpactstotheinvestorsoffirmsthatdevelopanddeployAI.TheemphasisonESGdisclosurethatisfinanciallymaterialtoinvestorsalignswiththecurrentdirectionoftheISSBbutdoesnotdenythemeritsofESGdisclosuremorebroadly.Financialmaterialityspeakstothematerialityofanindividualfirm’sESGinformationintermsofimpactonfuturecashflowsand,therefore,thevalueoftheenterprisetoaninvestor.Inaddition,thereisincreasingrecognitionintheroleofbetainformation,orhowafirm’sESGpracticesimpactthecoststhatafirmexternalizestotheeconomy,which,inturn,affectsoverallsecuritiesmarketreturns(Alexander2022).Withoutbeta-relatedinformation,disclosuresfailtocapturetheextenttowhichonefirm’spracticesimpactthereturnsofothercompaniesinaninvestor’sportfolioandacrosstheeconomyasawhole.Beyondthis,thereisacallforESG-relateddisclosurethatdoesnotaffectinvestors,butisrelevanttotheimpactoffirmsonotherstakeholders.TheadditionalvectorsofinformationreferencedaboverepresentopportunitiestobuildoutAI-relatedsustainabilityreporting,buttheauthorsarguethereisvalueinstartingwiththeinitialguidanceoftheISSBanddisclosurebasedonfinancialmateriality.ThereisanopportunitytoadvanceAI-relatedmeasurement,reportinganddisclosurebymergingselectcontentfromtheworldofAIguidelinesandgovernanceintoexistingESGreportingframeworks.Todaythereisnostandardizedapproachtoreportinganddisclosingonanorganization’sAI-relatedstrategy,activity,performance,risksand/orimpacts.Whilesignificantgrowthinthenumberofprinciple-basedethicalAIframeworkshasbeenwitnessedoverthelastfive–sixyears,themovementtooperationalizethemwithpracticalgovernanceapproachesisneitherrobustnorwidespread.Recognizingthecriticaljuncturethatsustainabilityreportinghasarrivedat—onthevergeofrealizingasinglesetofglobalstandards—thispaperisfocusedonthemeritsofintegratingtopicsofAIgovernanceintothosestandards.ThemeritsofthisintegrationmaybefoundbothfromtheestablishmentofaglobalsetofstandardsaroundAIgovernanceandinthepromiseofthosestandardscontainingoperationallyfocusedcontent.19AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightWorksCitedAlexander,Frederick.2022.“OneSmallStepfromFinancialMaterialitytoSesquimateriality:ACriticalConceptualLeapfortheISSB.”HarvardLawSchoolForumonCorporateGovernance,May4.https://corpgov.law.harvard.edu/2022/05/04/one-small-step-from-financial-materiality-to-sesquimateriality-a-critical-conceptual-leap-for-the-issb/.Brusseau,James.2023.“AIhumanimpact:TowardamodelforethicalinvestinginAI-intensivecompanies.”JournalofSustainableFinance&Investment13(2):1030–57.https://doi.org/10.1080/20430795.2021.1874212.Davenport,Tim.2019.“AIIsDestroyingTraditionalBusinessThinking.”Forbes,April5.www.forbes.com/sites/tomdavenport/2019/04/05/ai-is-destroying-traditional-business-thinking/?sh=56fb27993205.GovernanceandAccountabilityInstitute.2021.SustainabilityReportinginFocus:S&P500&Russell1000Companies.www.ga-institute.com/index.php?id=9128.IBM.2022.“IBMGlobalAIAdoptionIndex2022.”www.ibm.com/watson/resources/ai-adoption.IFRS.2022.“IFRSS1GeneralRequirementsforDisclosureofSustainability-relatedFinancialInformation.”www.ifrs.org/issued-standards/ifrs-sustainability-standards-navigator/ifrs-s1-general-requirements/#about.McKinsey&Company.2022.“ThestateofAIin2022—andahalfdecadeinreview.”December6.www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2022-and-a-half-decade-in-review.OceanTomo.2021.“IntangibleAssetMarketValueStudy.”www.oceantomo.com/intangible-asset-market-value-study/.Sætra,HenrikSkaug.2021.“AFrameworkforEvaluatingandDisclosingtheESGRelatedImpactsofAIwiththeSDGs.”Sustainability13(15):8503.https://doi.org/10.3390/su13158503.———.2022.“TheAIESGprotocol:Evaluatinganddisclosingtheenvironment,social,andgovernanceimplicationsofartificialintelligencecapabilities,assets,andactivities.”SustainableDevelopment31(2):1027–37.https://doi.org/10.1002/sd.2438.SASB.2018.“Software&ITServices.”SustainabilityAccountingStandard.Silverman,Karen.2020.“WhyYourBoardNeedsaPlanforAIOversight.”MITSloanManagementReview,October19.https://sloanreview.mit.edu/article/why-your-board-needs-a-plan-for-ai-oversight/.20CIGIPapersNo.279—August2023•MardiWitzelandNirajBhargavaAppendixExampleQuestionsforDisclosureTopicsofAIRisk-RelatedInformation:BasedonDraftGuidanceContainedintheISSB’sExposureDraft(S1)GeneralRequirementsforDisclosureofSustainability-RelatedFinancialInformationGovernance-relateddisclosure(speakstoaccountabilityandprocesswithintheentityforsustainability-relatedrisksandopportunities):→IsoneoverarchingbodyresponsibleforoversightofAI-relatedrisksandopportunitieswithintheentity?Ifso,pleaseidentifythebody.→Howaretheresponsibilitiesofthebody(ies)capturedintermsofreferenceandpolicy?→Howdo(es)theaccountablebody(ies)ensureappropriateskillsandcompetencies?→Howdo(es)thebody(ies)considerAI-relatedriskandopportunityinthecontextofstrategyandriskmanagementincludingtrade-offs?→Howdo(es)thebody(ies)establishtargetsforAIperformanceincludingonethicalgroundsandmonitoringofprogress?→Whatis(are)thebody’s(ies’)descriptionoftheroleofmanagementinallthis?→Howdo(es)thebody(ies)responsibleforoversightofAI-relatedrisksattheenterpriselevel,delegateortrickledownresponsibilitiesandaccountabilityforAIprogramsandprojectsattheuse-caselevel?→Howdo(es)thebody(ies)chargedwithoversightofdatagovernanceandAI-relatedrisksandopportunitiesfunctiontogether?Strategy-relateddisclosure(speakstowhatsustainability-relatedinformationmaybematerialinrelationtoanimpactonbusinessmodelandfinancials):→WhatAI-relatedrisksandopportunitiesfacetheentitythatitreasonablyexpectscouldaffectitsbusinessmodel,strategyandcashflows,itsaccesstofinanceanditscostofcapital,overtheshort,mediumandlongterm?Andhowdoestheentitydefineshort,mediumandlongterm?→Givenanentity’spositionontheAI-relatedrisksandopportunitiesitfaces,whatmightbetheeffectoftheseonitsbusinessmodelandspecificallyitsvaluechain?→Givenanentity’spositionontheAI-relatedrisksandopportunitiesitfaces,whatmightbetheeffectoftheseonstrategyanddecisionmaking?→Givenanentity’spositionontheAI-relatedrisksandopportunitiesitfaces,whatmightbetheeffectoftheseonfinancialposition,financialperformanceandcashflows?→GiventheunderstoodAI-relatedrisksandopportunities,whatistheresilienceoftheentity,oritscapacitytoadjusttotheuncertaintiesarisingfromtheserisks?Riskmanagementdisclosure(speakstodetailsofhowthesustainability-relatedrisksandopportunitiesareidentified,assessedandmanagedinordertoenableevaluationoftheentity’sriskprofileandriskmanagementprocesses):→WhatistheprocessbywhichtheentityidentifiesAI-relatedrisksandopportunitiesforriskmanagementpurposes?Istheentity’sapproachtoAI-relatedriskmanagementimplementedattheenterpriselevelortheuse-caselevelorboth?→Whatistheprocessbywhichtheentityassesses,prioritizesandmonitorsAI-relatedrisksandopportunities?Istheentity’sapproachtotheassessment,prioritizationandmonitoringofAI-relatedriskimplementedattheenterpriselevelortheuse-caselevelorboth?→TowhatextentistheAI-relatedriskmanagementprocessintegratedintotheenterprise’soverallriskmanagementprocess?21AI-RelatedRisk:TheMeritsofanESG-BasedApproachtoOversightMetricsandtargetsdisclosure(speakstohowtheentitymeasures,monitorsandmanagesitssignificantsustainability-relatedrisksandopportunitiesinordertounderstandhowtheentityassessesitsperformance,includingprogresstowardtargetsithasset):→Theentityistoincludemetrics(enterpriselevelanduse-caselevel)thatapplytotheactivitiesinlinewithitsbusinessmodelandinrelationtoAI-relatedrisksandopportunities.Theremaybedifferentsustainability-relatedrisksandopportunitiesthataredrivenbyAIacrossdifferentbusinesses/industriesinwhichtheentityisengaged.→Theentityistodisclosethemetrics(enterpriselevelanduse-caselevel)itusestomanageandmonitorAI-relatedrisksandopportunitiesandperformance,includingprogressagainstestablishedgoalsandtargets.→Theentityistodisclosedetailsrelatingtothedevelopmentofthemetrics(enterpriselevelanduse-caselevel)relatingtoAI-relatedrisksandopportunities,includinghowitwasdefined,whetherathirdpartyvalidateditandwhatmethodswereusedtocalculatethetargets.→Specialconsiderationshouldbegiventotheestablishmentofmetricsthatpertaintoenterprise-levelrisksandopportunities(forexample,transparencyofAIgovernanceandprocesses,qualityoftalent,accesstotalent,qualitymanagementsystemperformance,riskmanagementsystemperformance,compliancerecord,mediacomment,stakeholderfeedback)versusmetricsthatpertaintotheuse-caselevel,which,inturn,couldbeaggregated(forexample,AIsystemperformance,AIsystemfairness,AIsystemexplainability,AIsystemsecurity,datasecurityandprivacyinthecontextoftheAIsystem).67ErbStreetWestWaterloo,ON,CanadaN2L6C2www.cigionline.org@cigionline