Problem

The responsible development and deployment of AI systems necessitates thorough documentation of all constituent artifacts, including datasets, model weights, training pipelines, and runtime dependencies. While standards like SPDX 3.0 offer support for AI and dataset profiles, a significant gap exists in practical tooling capable of automatically generating standards-compliant AI Bills of Materials (AIBoMs) in a scalable and extensible manner. Traditional Software Bills of Materials (SBoMs) are insufficient for AI, which requires tracking elements such as training data provenance and model architecture. The lack of comprehensive AIBoMs can lead to severe issues like regulatory non-compliance, legal disputes, and significant financial losses, as exemplified by a fictional German health-tech firm facing substantial penalties due to inadequate AI documentation.

Method

AIGen is a modular AIBoM generator designed to produce machine-readable, SPDX 3.0 AI profile-compliant inventories. Built on the MLflow MLOps framework, AIGen employs a hybrid extraction approach. For structured data like hyperparameters and metrics, it leverages MLflow's APIs. For unstructured data such as use cases and logs, it combines data mining heuristics with Large Language Models (LLMs) to analyze source code and documentation. The tool operates through a five-phase pipeline: Configuration and Resource Initialization, Data Retrieval (where 'Builders' gather metadata), Serialization, and Validation. Key architectural components include the Java-based PipeManager, which orchestrates execution and employs Java reflection for extensibility. 'Builders' are specialized units for extracting metadata from diverse sources (e.g., MLflow, GitHub, Kaggle, Hugging Face Hub). 'Teams' group Builders to consolidate data for a single SPDX component, and 'Goals' assign SPDX fields to Builders with a priority-based fallback mechanism for robustness. An 'LLMClient' provides provider-agnostic generative capabilities for natural-language fields. Configuration is managed through a two-tier YAML system, allowing flexible pipeline definition. The 'Serializer' transforms extracted key-value pairs into standardized AIBoM documents, designed with delegation and composition to adapt to evolving standards. AIGen also includes features for Notebook Optimization, Sliding Window Extraction for LLM context, License Resolution, and non-blocking Execution Safety.

Results

A preliminary evaluation of AIGen was conducted on eight open-source ML projects from GitHub and Kaggle, covering Computer Vision, Tabular Data, and Recommendation Systems. MLflow was integrated into training scripts to track metrics and artifacts. For LLM-assisted fields, DeepSeek-r1 8B was used via Ollama with specific temperature and top_p settings. The generated AIBoM fields were qualitatively assessed on a four-point scale (Perfect, Good, Lackluster, Bad), and instances of Meta-commentary were noted. Out of 124 generated AIBoM fields across the projects, 91% were rated as Perfect (59%) or Good (32%). Fields handled by structured Builders, such as MLflow and Kaggle, consistently received Perfect ratings for deterministic metadata like hyperparameters and metrics. The most challenging aspect was synthesizing descriptive fields from source code and READMEs using LLMs; while often accurate, the model occasionally hallucinated or suffered context loss due to the sliding-window strategy. The study also noted that domain-specific fields, such as limitations, often require external context beyond the analyzed artifacts.

Implications

AIGen offers a concrete, reusable foundation for transparent and accountable AI supply chain governance, directly addressing the operational void in current AIBoM generation. By automating the creation of SPDX 3.0-compliant AIBoMs, AIGen facilitates compliance with regulatory frameworks such as the European Union AI Act, the NIST AI Risk Management Framework, and ISO/IEC 42001. The tool's modular architecture supports extensibility, adaptability to evolving standards, and integration with heterogeneous MLOps pipelines. Future work will include extending the open-source evaluation with industrial validation in collaboration with a partner company, assessing AIGen's configurability, robustness against incomplete metadata, and its utility in supporting human-in-the-loop auditing for compliance-relevant AI documentation in real-world MLOps pipelines.