Problem

The Indian judicial system faces a significant backlog of cases, with millions unresolved across various court levels. Artificial intelligence offers potential solutions, particularly for Legal Judgment Prediction (LJP) and Appellate Judgment Prediction (AJP). However, existing LJP systems often lack the interpretability required for high-stakes appellate cases, where understanding the legal reasoning behind predictions is crucial for transparency and trustworthiness. The complexity of legal language and domain-specific reasoning presents a substantial challenge to building effective and trustworthy AJP systems.

Method

Vichara, a novel framework tailored for the Indian judicial system, addresses these challenges through a six-stage pipeline. It begins with rhetorical role classification, identifying the function of each sentence (e.g., fact, argument, ruling). Next, case context is constructed from identified facts, capturing key information like parties, legal issues, and stances. The core innovation lies in extracting structured "decision points" which are discrete legal determinations including the issue, decision-maker, outcome, and reasoning. These decision points are filtered to represent only the present court's determinations. Subsequently, the present court ruling is generated, and a binary judgment outcome (Appeal Granted or Dismissed) is predicted by comparing this ruling with the appellant's stance. Finally, Vichara generates structured explanations inspired by the IRAC (Issue-Rule-Application-Conclusion) framework, providing sections like Facts of the Case, Legal Issues Presented, Applicable Law and Precedents, Analysis/Reasoning, and Predicted Conclusion. This structured approach, leveraging large language models (LLMs) without fine-tuning, aims to provide both accurate predictions and highly interpretable legal explanations.

Results

Vichara was evaluated on two Indian legal datasets, PredEx and ILDC_expert, using four LLMs: GPT-4o mini, Llama-3.1-8B, Mistral-7B, and Qwen2.5-7B. In judgment prediction, GPT-4o mini achieved the highest F1 scores (81.5 on PredEx and 80.3 on ILDC_expert), closely followed by Llama-3.1-8B. Notably, Vichara, when combined with these LLMs, surpassed existing judgment prediction benchmarks on both datasets. For explanation quality, human evaluations by legal experts across Clarity, Linking, and Usefulness metrics showed GPT-4o mini to be superior, with Mistral-7B also performing strongly. An ablation study further confirmed that each stage of the Vichara pipeline significantly contributes to both predictive performance and explanation quality, with decision point extraction showing the largest impact on predictive performance.

Implications

Vichara represents a significant advancement in explainable AI for judicial applications within the Indian legal system. By providing accurate predictions alongside structured, interpretable explanations, it enhances transparency and trustworthiness, allowing legal professionals to efficiently assess the soundness of AI-generated legal insights. This framework can assist in prioritizing appeals and evaluating legal reasoning, potentially alleviating the burden of massive case backlogs. Despite its strengths, Vichara has limitations, including its current restriction to the Indian judiciary, variability introduced by prompt-based LLM querying, and the limited scope of human evaluation. Future work aims to reduce computational overhead, adapt the framework to other case types and legal jurisdictions, and expand the evaluation to include a more diverse pool of legal professionals and datasets.