Decoding AI Risks from Government Regulations and Company Policies

Generative AI systems like ChatGPT make headlines, not only for their impressive capabilities but also for their concerning failure modes. From chatbots generating dangerous misinformation to AI-generated deepfakes undermining truth and trust, the risks posed by generative AI are no longer hypothetical (e.g., NY Lawyer Faces Possible Sanctions for Citing Phony ChatGPT Case). As AI rapidly integrates into every sector of society and the economy, the industry needs to ensure the understanding and mitigation of diverse risks it poses.

One major challenge is that current efforts to categorize and mitigate AI risks are siloed and disjointed, making it challenging to provide comprehensive safety evaluations for AI systems. While companies tailor their risk policies to specific jurisdictions and use cases, government regulations prioritize high-level societal concerns but lack specificity, and academic taxonomies often fail to incorporate the latest industry and policy developments. This lack of a common language enlarges the gap between the development and the safe deployment of AI systems in practice.

Presenting the first comprehensive regulation AI Risk Taxonomy

AIR 2024, a unified AI Risk Taxonomy, bridges these gaps. Developed by analyzing 16 policies from leading AI companies and 8 regulatory or voluntary frameworks from across the EU, US, and China, AIR 2024 is the first to provide comprehensive safety categories based on a set of regulations and company policies. It identifies 314 risk types and organizes them into a four-level hierarchy, enabling more consistent and complete AI safety evaluations across regions and sectors. At the highest level, this taxonomy encompasses System & Operational Risks, Content Safety Risks, Societal Risks, and Legal & Rights Risks. The taxonomy establishes connections between different descriptions and approaches to risk, highlighting the overlaps and discrepancies between public and private sector conceptions of risk.


By grounding the taxonomy in real-world policies, AIR 2024 establishes a shared understanding of relevant and applicable risks across contexts. This “universal language” of AI risk facilitates more precise communication and collaboration among policymakers, industry leaders, researchers, and regulators. It also highlights areas where governance frameworks must be strengthened, or corporate best practices could inform regulatory efforts.

Test examples from different risk categories (Regulation-based Safety)

Findings 

Summary: 

We have made the following 6 key findings regarding AI risk categorization and governance:

  • Persistent Vulnerabilities: Even the most advanced open-source model (Llama 3.1 405B), when prompted with challenging harmful inquiries, still fails in specific cases. This indicates a significant scope for improvement, even for state-of-the-art models.
  • Fragmented Landscape: Private and public sectors significantly diverge in AI risk categorization, creating a patchwork of approaches.
  • Common Core Concerns: Despite differences, a growing consensus exists on fundamental AI risks like privacy violations and discriminatory activities.
  • Critical Gaps: Private and public sectors often overlook important risks such as worker disempowerment and threats to democratic participation.
  • Regional Variations: Geographic and cultural factors strongly influence AI risk priorities and regulatory approaches.
  • Urgent Need for Unification: A standardized, comprehensive AI risk assessment and mitigation framework is critically needed across industries and jurisdictions.

Private sector categorizations of AI risk specifications

With AIR 2024, our analysis of 16 AI company policies reveals both areas of consensus and significant gaps in how the private sector categorizes and prioritizes AI risks.

We find that nearly all companies’ policies extensively cover risks related to using generative AI for privacy violations, child sexual abuse content, monetized sexual content, criminal activities, and harassment. This strong alignment suggests the AI industry recognizes the critical importance of mitigating these harms. However, even within these common categories, the level of specificity varies widely between companies. For instance, the “Harassment” category in our AIR 2024 taxonomy contains 11 specific risks, including bullying, threats, intimidation, shaming, humiliation, insults/personal attacks, abuse, provoking, trolling, doxxing, and cursing. Yet the most comprehensive policies regarding harassment– from companies like Cohere and DeepSeek–cover at most 6 of these 11 risks. This inconsistency highlights the need for more standardization across companies to ensure thorough and consistent coverage of key risks across the industry.

By contrast, we found several risk categories which are rarely mentioned in companies’ policies. Strikingly, few company policies we analyzed address risks related to disempowering workers, such as AI-enabled surveillance of workers, despite this being a major concern in government frameworks like the White House Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. It’s important to note that some risks, such as automation-driven job displacement, maybe more effectively addressed at the societal level rather than through individual company policies. However, other risks that few policies address, such as offensive language, disrupting social order, unfair market practices, and fraudulent schemes, could potentially be mitigated at the model level. Interestingly, the few policies that do mention these risks primarily belong to companies based in China, hinting at the influence of regional context on the development of AI policies.

We also found salient differences in how companies categorize risks across their policy documents. For instance, Meta’s model-specific acceptable use policy for its Llama models is more extensive than its platform-wide Terms of Service, while Google takes a more uniform approach. OpenAI’s usage policies have evolved over time, removing some risk categories like depicting violence while expanding others like defamation. These patterns underscore the complexity and dynamism of the AI risk landscape and how companies’ legal and policy teams take different approaches to reducing risk based on the product.

Perhaps most importantly, our analysis reveals that company policies have substantial gaps in addressing key areas, such as AI’s potential to deter democratic participation, generate pro-self-harm content, and infringe on fundamental rights. With only a handful of company policies tackling these risks, it’s clear that a more comprehensive and proactive approach to AI governance is urgently needed. As AI systems become increasingly powerful and pervasive, it is crucial for companies to take a more active role in identifying and mitigating potential harms rather than waiting for government regulations to catch up. By proactively addressing these gaps and aligning their policies with the broader societal concerns highlighted in government frameworks, companies can demonstrate their commitment to responsible AI development and help build public trust in these transformative technologies.

By mapping the landscape of how AI risks are currently categorized by the private sector, our work demonstrates the patchwork nature of existing policy risk mitigation efforts and the pressing need for a unified framework. The AIR 2024 taxonomy offers a path forward, enabling more consistent and comprehensive risk assessment across regions and sectors. But this work is just a first step — realizing the full potential of AI while mitigating its risks will require ongoing collaboration and vigilance from all stakeholders.

Public sector categorizations of AI risk specifications (EU, US, China)

Shifting our focus to the public sector, our analysis of AI regulations in the European Union, United States, and China reveals similarities and differences in how these leading jurisdictions approach AI risk.

One notable finding is that despite their distinct regulatory frameworks, the EU, the US, and China share a core set of concerns. All three jurisdictions recognize risks related to automated decision-making, unsafe autonomous systems, unfair market practices, privacy violations, and discriminatory activities, among others. This overlap suggests a growing global consensus on some of the most pressing challenges posed by generative AI.

However, while these shared risk categories provide a common foundation, each jurisdiction has unique areas of emphasis. The EU AI Act, for instance, stands out for its focus on protecting vulnerable groups from AI systems that exploit their unique identities. China’s regulations, meanwhile, place a strong emphasis on risks related to manipulation of public opinion and social stability.

Diving deeper, we find that the level of specificity in these regulations varies widely. China’s policies tend to be the most granular, with detailed descriptions of risks mapped to dozens of specific categories in our taxonomy. This is partly due to the fact that Chinese regulators are farther along in the process, having already developed implementing regulations such as TC260. In contrast, the US and EU often stick to higher-level categories, leaving more room for interpretation, as they have yet to develop similarly detailed implementing regulations. As the AI regulatory landscape continues to evolve, we expect to see more granular guidelines emerge from the US and EU, providing greater clarity on operationalizing these high-level principles in practice.

These differences in approach are reflected in the regulatory tools each jurisdiction employs. China requires companies to obtain licenses to deploy generative AI services, while the EU focuses on mitigating dangers from high-risk AI systems. The US relies largely on voluntary frameworks, although binding rules stemming from the recent White House Executive Order are on the horizon.

Our analysis uncovers important gaps where even these major regulations fall short. The US Executive Order is the only policy we examined that explicitly emphasizes risks related to child sexual abuse – a glaring omission in other jurisdictions, especially given the technology’s potential to be misused for such heinous purposes. While the EU and China have made significant strides in developing comprehensive AI regulations, their failure to explicitly address this critical risk category underscores the need for a more harmonized global approach to AI governance.

As AI continues to advance at a breakneck pace, it’s clear that regulators are scrambling to keep up. While the EU, the US, and China have all made significant strides in identifying and categorizing AI risks, our work suggests that the current regulatory landscape remains fragmented and inconsistent across jurisdictions. This lack of harmonization creates challenges for companies operating in multiple regions and hinders the development of globally recognized standards for responsible AI. By providing a unified taxonomy to map these efforts, we aim to highlight the areas where greater alignment is needed and facilitate the kind of cross-border collaboration and knowledge-sharing that will be essential for ensuring the safe and responsible development of AI on a global scale.

Final Remarks

As generative AI systems become more capable and integrated into our daily lives and society, establishing a clear and well-grounded taxonomy of AI risks like AIR 2024 is a crucial first step – but it is only the beginning of the journey towards ensuring that advanced AI systems are developed and deployed responsibly, in alignment with societal values and priorities.

This figure shows the landscape we envision for AI that benefits humanity. It also highlights the unique position and mission of Virtue AI in providing a layer of safety to bridge general-purpose AI systems toward enterprise-level trustworthiness for society.

AIR 2024 is the starting point of this process (I). It aims to provide a solid and well-grounded foundation for categorizing AI risks. This then enables the development of the backend layer (II) focused on AI risk assessment and management. Advancing AI safety also requires the responsible development and implementation of risk management tooling, as well as a societal feedback loop (III) that continuously identifies harms and allows companies to update their understanding of AI risks over time.

By establishing this comprehensive framework, we aim to harmonize how governments, companies, and consumers think about AI risks. This will create a positive feedback loop where the identification and categorization of risks inform the development of more effective regulations and company policies, which in turn drive the creation of safer and more responsible AI systems. A shared understanding of AI risks across all stakeholders will make developing and deploying AI technologies much more transparent and accountable, enabling a more proactive and collaborative approach to risk mitigation. AIR 2024 marks an important milestone on this critical journey towards a future where transformative AI systems are developed and deployed in a manner that is safe, responsible, and aligned with societal values and priorities. By providing a common language and framework for understanding AI risks, we aim to facilitate the kind of multi-stakeholder collaboration and coordination that will be essential for realizing the immense potential benefits of AI while navigating its complex challenges and risks.

Appendix (the complete set of risk categories)

(Click here to view the full-resolution image of the risk categories)