This paper exposes widespread annotation errors in leading text-to-SQL benchmarks, BIRD and Spider 2.0-Snow, and demonstrates how these inaccuracies severely distort model performance evaluations and leaderboard rankings. It also introduces SAR-Agent and SAPAR, an AI-powered toolkit designed to effectively detect and correct these pervasive errors, advocating for higher quality benchmark development.

This paper introduces LLMSQL, a systematically revised and transformed version of the WikiSQL dataset specifically designed for the LLM era of Text-to-SQL. It addresses and rectifies various structural and annotation issues in the original WikiSQL, providing a clean and reliable benchmark for evaluating large language models.

This paper introduces CoddLLM, a 12-billion-parameter foundation model specifically designed for data analytics applications. It leverages a novel data recipe and post-training approach, enabling superior performance in tasks like data discovery, schema creation, and natural language to SQL conversion, outperforming existing state-of-the-art models on various benchmarks.