Problem

Autonomous agents often operate in environments governed by policies, but prior work largely focuses on ensuring strict compliance. There's a critical need for agents that can reason about non-compliance, particularly in high-stakes situations where policy deviation might be necessary to achieve a goal while minimizing negative consequences. Additionally, understanding non-compliant behavior can help policymakers refine policies by simulating realistic human decision-making. Existing frameworks for non-compliance often only consider plan length, overlooking the severity of rule violations or potential harm.

Method

The researchers developed a framework that extends Gelfond and Lobo's Authorization and Obligation Policy Language (AOPL) to AOPL-P, incorporating explicit penalty definitions for policy violations. They created an automated Python-based translator to convert AOPL-P policies into Answer Set Programming (ASP). This allows for reasoning about penalties during plan generation. They refined ASP-based planning algorithms and introduced new "emergency" and "non-emergency" behavior modes that prioritize either minimizing plan execution time or accumulated penalties, respectively. Crucially, the framework includes mechanisms to prevent harm to humans by assigning high penalties and enforcing strict constraints for relevant policy rules (e.g., stopping for pedestrians). The framework integrates ASP encodings of the dynamic domain, policies with penalties, a general policy reasoning module, problem specifications, a general planning module, and a module for ranking and selecting optimal plans.

Results

Experiments in two dynamic domains (Rooms Domain and Traffic Norms Domain) demonstrated that the new framework generates higher-quality plans compared to previous approaches (Harders and Inclezan, HI framework). In the Rooms Domain, their framework showed significant improvements in time efficiency, completing experiments in under 0.5 seconds compared to 2-4 seconds for the HI framework. While the HI framework was sometimes faster in the Traffic Norms Domain, this came at the expense of plan quality, as it did not refine driving speeds or consider human harm. The new framework's emergency mode, unlike the HI framework's "Risky" mode, explicitly includes stopping for pedestrians, highlighting its ability to prevent human harm. It also generates plans with more appropriate driving speeds, adhering closer to speed limits in non-emergency scenarios, which the HI framework often overlooked. Performance was found to be sensitive to the number of distinct speed values considered in the Traffic Norms domain, but optimizations like checking action executability improved efficiency.

Implications

This framework significantly enhances autonomous decision-making by enabling agents to reason about policy compliance and non-compliance with a nuanced understanding of penalties and consequences, especially in critical situations. It provides a valuable tool for policymakers to refine policies by simulating diverse agent behaviors, including realistic human-like deviations. Future work includes collaborating with ethics experts to refine the penalty scheme, identifying more relevant behavior modes for policymakers, and further enhancing the framework's computational efficiency and scalability for real-world applications.