Problem

The increasing integration of AI into critical services and products exposes it to sophisticated cyber threats that traditional defenses are ill-equipped to handle. While cyber attacks like ransomware and phishing are escalating, AI systems introduce new attack surfaces and vulnerabilities that existing cyber threat intelligence (CTI) frameworks do not adequately cover. Empirical studies highlight a significant number of adversarial attacks targeting AI models at data, model, and deployment levels. This evolving threat landscape necessitates new CTI approaches that define AI-specific indicators of compromise (IoCs) and attack patterns to maintain effective defenses.

Method

The authors conducted a systematic literature review to understand how CTI practices must adapt for AI systems. The review involved: 1. Search Strategy: Identifying keywords such as "Cyber threat intelligence," "CTI for AI," "AI incidents," "AI incidents database," "AI cyber threats," and "AI vulnerabilities." These were used to search academic databases like Google Scholar and Scopus, with additional exploration of references from identified papers. 2. Analysis and Extraction: Each paper was thoroughly read to extract information related to CTI in AI, including proposed frameworks, data sources, IoCs, and applications in security tools. 3. Synthesis: Insights were structured and organized to highlight necessary adaptations of CTI practices for AI-specific threats, addressing four research questions regarding the differences between classical and AI CTI, relevant knowledge sources, benefits to AI protection tools, and methods for measuring IoC similarity.

Results

The study revealed that classical CTI, focused on traditional IT assets, differs significantly from CTI for AI, which must account for unique assets like training data, model parameters, and inference pipelines. AI-specific vulnerabilities include data poisoning, model backdoors, and adversarial examples. The research identified three main categories of sources for building an AI CTI knowledge base: 1. Vulnerability-oriented sources: Frameworks like AVID, OWASP AI Security and Privacy Guide, ENISA, and SAIF provide guidance and structured knowledge about AI weaknesses. AVID, for example, is an open-source database of AI/ML vulnerabilities. 2. Incident-oriented sources: Databases like the AI Incident Database (AIID) and taxonomies such as CSET AI Harm Taxonomy and GMF taxonomy document real-world AI failures, misuses, and harms, providing empirical evidence. 3. Adversary-oriented sources: Frameworks such as MITRE ATLAS map attack vectors against AI/ML systems, analogous to MITRE ATT&CK for traditional IT, describing attacker TTPs specific to AI lifecycle phases (e.g., reconnaissance of ML artifacts, model-specific attacks like data poisoning, prompt injection). Additionally, specialized datasets for prompt injection attacks and malicious model repositories were identified, though their quality and coverage vary. While resources like MITRE ATLAS and AIID are mature, many specialized datasets have quality issues, and malicious model repositories have limited documented cases. The paper also outlines how an AI CTI knowledge base would define IoCs beyond traditional ones, including suspicious model weights or unusual dataset patterns. It proposes methods for measuring similarity between collected IoCs and potentially malicious AI models or datasets. These include deep hashing, which creates compact binary fingerprints of AI assets for fast comparison, and similarity hashing algorithms like TLSH and LZJD, adapted from malware analysis. More robust methods like semantic consistency hashing (SCH) and fuzzy hashing are also discussed to enhance detection accuracy and handle modified assets.

Implications

A comprehensive CTI for AI knowledge base would significantly enhance AI protection tools. By storing signatures of known malicious models, datasets, and attack techniques, security tools could proactively scan AI systems before deployment and investigate suspicious behaviors against a history of incidents. This enables faster identification of causes and application of proven fixes, leveraging resources like AIID and CSET. Furthermore, encoding adversary TTPs from MITRE ATLAS would allow for monitoring against AI-specific reconnaissance, data poisoning, or evasion attempts, akin to how EDR tools use ATT&CK. The integration of deep hashing and fuzzy matching techniques would enable the detection of similar, even previously unseen, malicious models. Structured taxonomies (AVID, CSET, GMF) would aid in automating incident sorting, severity assessment, and response guidance. The primary challenge lies in balancing detection speed with accuracy. Future research should focus on defining novel AI-specific IoCs and developing frameworks for monitoring and responding to AI threats in practice.