Best Financial LLM Benchmarks in 2026
Patronus AI TeamNovember 20267 min read
The main benchmarks for financial LLMs in 2026 are FinanceBench, FinQA, TAT-QA, ConvFinQA, FinBen, DocFinQA, the Finance Agent Benchmark, BizBench and FinanceQA. They test different things. Some ask questions about a company’s filings, some test arithmetic over tables, and the newest give an agent tools and an analyst’s research task. Pick the one that matches the work your model will do, check whether the test set is open, and read how answers are scored.
Finance is hard for language models because answers depend on exact numbers in long documents, and a near miss is still wrong. Benchmarks show where a model breaks before a customer finds out, and some now serve as training environments as well as tests. This guide examines nine, starting with FinanceBench from Patronus AI, a behavioral research lab that publishes at NeurIPS, ICML and ACL. FinanceBench has 350+ citations and 81,000+ Hugging Face downloads of its open sample as of November 2026.
The nine fall into three groups. FinanceBench, FinQA and DocFinQA ask questions about public company filings, at different document lengths. TAT-QA, ConvFinQA, BizBench and FinanceQA test numerical reasoning over tables, conversations and analyst conventions. FinBen and the Finance Agent Benchmark are broad suites, the first across many financial language tasks and the second with an agent that researches filings using tools.
Last updated
Key takeaways
- Give the model the documents. Analysts work from filings, so the most realistic tests are open book. FinanceBench pairs each answer with an evidence string from the filing, and DocFinQA puts the full filing in context.
- Long context is still a weak spot. DocFinQA extends FinQA’s contexts from under 700 words to an average of about 123,000, and its authors report it challenges state-of-the-art systems.
- Simple questions still find failures. FinanceBench was built as a minimum standard of clear-cut questions, yet in its paper a leading model with retrieval answered 81% of sampled questions wrong or refused them.
- Agentic tests are the newest layer. The Finance Agent Benchmark gives a model tools and 900+ expert-reviewed questions in its current version, and its top leaderboard score was 67.85% at the October 29, 2026 update.
- Check what is open. FinanceBench publishes a 150-case sample, FinanceQA a 148-question test split, and the Finance Agent Benchmark a 27-question public set and a hosted leaderboard.
- Citations show what the field uses. FinQA has 940+ citations, TAT-QA 760+, FinanceBench 350+ and ConvFinQA 310+ on Semantic Scholar.
Comparison table
All nine benchmarks at a glance, in the order of this guide. Citations are Semantic Scholar counts as of November 2026. Sort by any column marked with arrows.
| # | Benchmark | Best for | What it tests | Size | Open data | Citations |
|---|---|---|---|---|---|---|
| 1 | FinanceBench | Answers from real filings, with evidence | Open-book QA over 10-K, 10-Q and 8-K filings | 10,231 questions | 150-case sample, CC BY-NC 4.0 | 350+ |
| 2 | FinQA | Multi-step arithmetic over reports | Numerical reasoning with reasoning programs | 8,281 QA pairs | MIT license | 940+ |
| 3 | TAT-QA | Tables and text together | Hybrid table and text QA | 16,552 questions | CC BY 4.0 | 760+ |
| 4 | ConvFinQA | Follow-up questions | Conversational numerical reasoning | 3,892 conversations | MIT license | 310+ |
| 5 | FinBen | A broad first pass | 24 tasks in seven categories | 36 datasets | Open on GitHub | 230+ |
| 6 | DocFinQA | Long documents | FinQA over full filings | 7,437 questions | MIT license | 100+ |
| 7 | Finance Agent Benchmark | Agents doing analyst research | Research with tools over SEC filings | 900+ questions in v2.1 | 27-question public set, hosted leaderboard | 75+ |
| 8 | BizBench | Financial answers computed in code | Eight quantitative reasoning tasks | Eight tasks | Apache 2.0 license | 40+ |
| 9 | FinanceQA | Analyst conventions and assumptions | Investment analysis tasks | 148-question test split | Apache 2.0 license | 20+ |
1. FinanceBench
FinanceBench is an open-book question answering benchmark about publicly traded companies, built by Patronus AI and released in 2023 (paper).
How does FinanceBench work?
- Questions about public companies, answered from their 10-K, 10-Q and 8-K filings, covering information extraction, numerical reasoning and logical reasoning.
- 10,231 questions, each with an answer and an evidence string. A 150-case open sample is on Hugging Face under CC BY-NC 4.0, with the source documents on GitHub.
- In the paper, people reviewed 2,400 answers from 16 model configurations by hand.
Where it fits
FinanceBench is widely used, with 350+ citations and 81,000+ Hugging Face downloads of its open sample. Patronus AI reports it is now used as an environment by post-training teams at all major labs.
Best for: Testing whether a model can answer clear questions from real filings, with evidence to check each answer against.
2. FinQA
FinQA is a dataset of question and answer pairs over financial reports, written by financial experts and published at EMNLP 2021 (paper).
How does FinQA work?
- Questions need numerical reasoning over the text and tables of financial reports.
- Each answer comes with an annotated reasoning program, so the steps can be checked.
- Data and code are on GitHub under the MIT license, with a private test set.
Where it fits
FinQA is the most cited benchmark here, with 940+ citations, and ConvFinQA and DocFinQA build on it.
Best for: Measuring multi-step arithmetic over report tables and text.
3. TAT-QA
TAT-QA is a question answering benchmark built from real financial reports that mix tables and text, published at ACL 2021 (paper).
How does TAT-QA work?
- Answers often need addition, subtraction, counting, comparison or sorting across a table and the text around it.
- The paper also introduced TAGOP, a model that picks table cells and text spans and reasons over them.
- Data is on GitHub under CC BY 4.0, and the code under the MIT license.
Where it fits
TAT-QA has 760+ citations and is a standard test for reasoning over mixed tables and text.
Best for: Testing reasoning across a table and its surrounding text together.
4. ConvFinQA
ConvFinQA extends FinQA into conversations, where each question builds on earlier turns. It was published at EMNLP 2022 (paper).
How does ConvFinQA work?
- Tests chains of numerical reasoning across a conversation.
- Released at conversation level and turn level.
- Data and code are on GitHub under the MIT license.
Where it fits
ConvFinQA has 310+ citations. It suits assistants that answer a run of questions about one report.
Best for: Follow-up questions about the same financial report.
5. FinBen
FinBen is an open benchmark suite from The Fin AI community, published at NeurIPS 2024 in the Datasets and Benchmarks track (paper).
How does FinBen work?
- 36 datasets across 24 tasks.
- Seven categories, information extraction, textual analysis, question answering, text generation, risk management, forecasting and decision-making.
- Code is in The Fin AI’s PIXIU repository on GitHub.
Where it fits
FinBen has 230+ citations. Its breadth makes it a first pass rather than a deep test of one skill.
Best for: A broad first pass across many financial language tasks.
6. DocFinQA
DocFinQA takes FinQA questions and gives the model the full filing instead of a short excerpt. It was published at ACL 2024 and released by Kensho on Hugging Face (paper).
How does DocFinQA work?
- 7,437 questions from FinQA with their full documents.
- An average context of about 123,000 words, against under 700 in FinQA.
- Built to test retrieval pipelines and long-context models. MIT license.
Where it fits
DocFinQA has 100+ citations. It isolates the long-document problem that short excerpts hide.
Best for: Long-document retrieval and reasoning over full filings.
7. Finance Agent Benchmark
The Finance Agent Benchmark from Vals AI tests agents on finance research problems that require analysis of recent SEC filings (paper).
How does the Finance Agent Benchmark work?
- In the paper, 537 expert-authored questions in nine task categories, designed with experts from banks, hedge funds and private equity firms. Version 2.1 has 900+ expert-reviewed questions, still in nine categories.
- An agentic harness gives the model the tools it needs to research each answer.
- A live leaderboard for version 2.1. The top score was 67.85% at the October 29, 2026 update, and scores are not comparable with earlier versions.
Where it fits
The Finance Agent Benchmark has 75+ citations. It tests the agent as a researcher rather than a reader of a given document.
Best for: Agents that do analyst research with tools.
8. BizBench
BizBench is a benchmark for quantitative reasoning in business and finance, published at ACL 2024 and released by Kensho on Hugging Face (paper).
How does BizBench work?
- Eight quantitative reasoning tasks, including three financial code-generation tasks that answer questions through program synthesis.
- Other tasks isolate reading values from financial text and tables, and knowing the financial formulas needed for an answer.
- Apache 2.0 license.
Where it fits
BizBench has 40+ citations. It separates the skills a financial answer needs, so a team can see which one fails.
Best for: Testing whether a model can compute financial answers in code.
9. FinanceQA
FinanceQA tests the numerical analysis done at hedge funds, private equity firms and investment banks. It was released in 2025 (paper).
How does FinanceQA work?
- Tasks include hand-spreading metrics, following accounting and valuation conventions, and making assumptions when information is missing.
- A 148-question test split on Hugging Face under Apache 2.0, typed as basic, assumption and conceptual questions.
- Its authors report that models fail about 60% of the tasks.
Where it fits
FinanceQA has 20+ citations. It is closest to the judgment calls of analyst work.
Best for: Analyst-style work where the model has to make and state assumptions.
Why FinanceBench and Patronus AI stand out
Research depth
Patronus AI is a research lab first. Its 19 papers include work at NeurIPS, ICML and ACL. FinanceBench has 350+ citations and 81,000+ Hugging Face downloads of its open sample, and TRAIL, on errors in agent traces, has 110+ citations.
Evidence with every answer
Every FinanceBench answer comes with the evidence string from the filing, so a team can see why an answer is right, not only whether it matched.
From benchmark to training environment
Patronus AI reports that post-training teams at all major labs now use FinanceBench as an environment. It builds RL environments for finance and other knowledge work, with problems written and checked by experts.
Enterprise-ready security
Patronus AI lists SOC 2, HIPAA and TISAX, with data ownership set in each agreement.
For teams training models on financial work, Patronus AI builds RL environments in finance and other expert domains, starting from research like FinanceBench. Talk to Patronus AI about finance environments.
Frequently asked questions
What is the best benchmark for financial LLMs?
It depends on the work. FinanceBench and DocFinQA test answers from filings, FinQA, TAT-QA and BizBench test numerical reasoning, and the Finance Agent Benchmark tests research with tools. Many teams run more than one.
What is FinanceBench?
FinanceBench is Patronus AI’s open-book benchmark of 10,231 questions about public companies, answered from their filings with evidence strings. It has 350+ citations as of November 2026.
How do I get the FinanceBench dataset?
A 150-case open sample is on Hugging Face as PatronusAI/financebench, and the source documents are on GitHub. Our FinanceBench page links the paper.
Which financial benchmarks test AI agents?
The Finance Agent Benchmark gives an agent tools to research SEC filings, and FinBen includes forecasting and decision-making tasks.
Which financial benchmarks test long documents?
DocFinQA gives the model the full filing, an average of about 123,000 words, instead of a short excerpt. FinanceBench is open book over 10-K, 10-Q and 8-K filings.
Can a financial benchmark be used for training?
Yes. A benchmark with clear answers and evidence can become an RL environment, with the answer check as the reward. Patronus AI reports FinanceBench is used this way by post-training teams.
Last updated