![\"DAYJOB:](\"https://cdn.prod.website-files.com/68dcd2ceb173c46fa029931c/6ae0be39ff7dfbf1d1c0be10_6ae0be167d69fdd5f5b61e46_dayjob-tax-leaderboard.png\")
Surge AI · October 27, 2026
\nTL;DR
\nReal-world tax work is messy. Instructions are incomplete, records conflict, and the most important problems aren't always the ones explicitly assigned. DAYJOB: Tax evaluates whether AI agents can handle that reality, taking ownership of a tax professional's work from an initial client request to a finished deliverable.
\nAcross 50 environments and 1,789 files, agents must navigate complex tax records, uncover issues clients may not know to raise, and make consequential judgments with incomplete information. Success requires more than tax knowledge and accurate calculations. It demands the initiative and professional judgment to figure out what needs to be done, investigate what matters, and deliver work that meets professional standards.
\n\nAnna was planning to sell her condo. She had moved out in August 2024 and wanted to know whether closing in October 2026 or January 2027 would be better for taxes.
\nAn AI agent reviewed both options and correctly concluded that either date would satisfy the residence requirement for a tax break worth up to $250,000 in excluded gains.
\nBut it left out an important deadline.
\nUnder the usual rules, homeowners must have lived in a property for at least two of the five years before selling it to qualify for the exclusion. Because Anna moved out in August 2024, she would stop satisfying that requirement around the end of August 2027. If her sale was delayed beyond August 2027, she could lose eligibility.
\nAcross five attempts, Astra never identified the deadline. Opus identified it in all five attempts.
\nThe agent evaluated the proposed dates correctly, but its advice left Anna unaware of a deadline that could affect a major financial decision. A tax adviser should recognize consequential issues even when the client doesn't know to ask about them.
\nThis is the kind of professional judgment we're testing with DAYJOB: Tax, the latest addition to Surge AI's DAYJOB benchmark family.
\nDAYJOB evaluates AI agents on professional tax assignments requiring specialized expertise, complex materials, and decisions that aren't fully specified in the initial instructions. Our earlier releases covered Healthcare and Finance. DAYJOB: Tax extends that work into tax preparation, compliance, and advisory services.
\nThe benchmark contains 50 environments across eight areas of tax practice. Agents receive client requests and supporting records, then produce tax returns, workpapers, technical reviews, or client advice.
\nThese assignments test professional judgment alongside tax knowledge, calculations, retrieval, and instruction following. Agents must decide what deserves investigation, resolve conflicting evidence, recognize when information is insufficient, and carry those decisions into their final work.
\nClaude Sonnet 5.5 leads with a 10.8% pass rate on the DAYJOB: Tax leaderboard, followed by Claude Fable 5.1 at 9.6% and Claude Opus 5.5 at 9.2%.
\nWe report pass@1 at a 95% rubric threshold. An attempt passes when it satisfies at least 95% of the task's criteria, which assess calculations, supporting evidence, consequential issues, and the completed deliverable.
\nWe also compared model performance against average cost per trial.
\n![\"\"](\"https://cdn.prod.website-files.com/68dcd2ceb173c46fa029931c/6ae0c6ccbb53b2b878fdafa0_pareto.png\")
Claude Sonnet 5.5 and Opus 5.5 occupy the high-performance end of the measured Pareto frontier.
\nDAYJOB: Tax covers eight areas, each requiring different rules, records, and expertise:
\nSome assignments require complete filing packages with supporting workpapers. Others ask agents to investigate transactions, review previous filings, or prepare advice about decisions clients haven't yet made.
\nThe benchmark contains 1,789 workspace files, with a median of 30 per assignment and as many as 123. These include 1,487 PDFs, 117 Excel workbooks, 38 Word documents, and additional emails, images, CSVs, JSON records, and text files.
\nThe workspaces include filed and draft tax returns, K-1s, W-2s, IRS notices, general ledgers, payroll records, contracts, and correspondence. They also contain the less standardized evidence that tax professionals routinely work with.
\nOne assignment includes a photograph of a handwritten receipt for merchandise purchased for resale. The agent must read the item descriptions and amounts, reconcile them with spending records, and determine how the purchases affect inventory and cost of goods sold.
\n![\"Photograph](\"https://cdn.prod.website-files.com/68dcd2ceb173c46fa029931c/6ae0dfece016abfe1a6d9a88_6ae0dfdf124711b53397acf1_dayjob-tax-handwritten-receipt-wide.png\")
Another assignment involves an industrial silo-services company whose client assumes it operates only in Alabama. But its field dispatch log shows crews working across five states. The job locations, crew sizes, and days on site provide evidence relevant to the company's multistate tax obligations.
\n![\"Field](\"https://cdn.prod.website-files.com/68dcd2ceb173c46fa029931c/6ae0be38ff7dfbf1d1c0be0d_6ae0be2fc839b1bb6c17f682_dayjob-tax-field-dispatch-log.jpeg\")
Other environments are similarly varied. One involves a cross-border household whose Polish employment and property records must be reconciled with cryptocurrency exchange exports, wallet histories, and historical purchase lots.
\nDAYJOB tasks begin with real-world professional workflows. Tax experts turn these situations into realistic assignments, developing the supporting records, expected outcomes, and grading criteria. Each task undergoes a three-layer expert review for correctness, completeness, and solvability.
\nTax experts bring experience across corporate taxation, partnerships, individual returns, trusts, and multistate compliance. Their backgrounds include:
\nA separating couple asked an AI agent how to divide their Roth IRA and 401(k). Opus 5.5 told them they weren't legally married and recommended keeping their accounts separate, with a cash payment to even out the balances.
\nIt opened one response:
\n\nYou aren't legally married, so neither account can be split without triggering taxes and penalties.
But the couple's own records raised doubts about that conclusion. Although they now lived in Florida, where new common-law marriages generally aren't recognized, they had previously lived together in Texas and held a ceremony in Colorado. Texas recognizes informal marriages under certain conditions, and a valid marriage established elsewhere may remain valid in Florida.
\nThat distinction could change the tax consequences of dividing their retirement accounts.
\nAcross all five attempts, Opus based its recommendations around the assumption that the couple was unmarried, even when it acknowledged uncertainty. Astra recognized the need to verify their marital status in four of five attempts. One response advised:
\n\nWe first need to confirm your legal marital status.
The client asked how to divide the accounts. The agent needed to establish whether they were married before recommending a course of action. The evidence that should have prompted that investigation was already in the files.
\nMost existing tax benchmarks begin with a defined problem: a legal question to answer, a return to calculate, or missing taxpayer information to collect. These evaluations measure important capabilities, but much of a tax professional's work involves figuring out what the problem actually is. Clients provide incomplete instructions, records contain conflicting evidence, and consequential issues may never appear in the original request.
\nDAYJOB tests whether agents can handle that end-to-end responsibility, including identifying issues the client overlooked, evaluating conflicting evidence, and deciding what needs to be done in the first place.
\nCan the model answer a difficult tax question?
\nTaxBench evaluates 114 tax questions across multiple specialties, each with a defined problem and grading rubric.
\nDAYJOB also asks whether the client asked the right question. Anna wants to compare two dates for selling her condo. Both satisfy the residence requirement for a tax exclusion, but does the agent warn her when that eligibility could expire?
\nCan the model calculate an accurate return?
\nTaxCalcBench evaluates tax-return accuracy against expected filing values, using realistic tax documents and taxpayer scenarios.
\nDAYJOB also asks whether the underlying records actually support the tax treatment. A studio's ledger references an attached receipt, but the receipt is missing. Does the agent flag the missing evidence or incorrectly treat the reimbursement as verified?
\nCan the agent gather missing information and file correctly?
\nTaxLab tests whether agents can question a simulated taxpayer, obtain missing information, and complete a federal individual tax return.
\nDAYJOB extends that responsibility across the wider tax profession. The deliverable might be a partnership loss analysis, an estate return, a review of a disputed tax position, or a strategy letter.
\nThe central distinction is how much responsibility remains with the agent. DAYJOB evaluates whether agents can discover consequential issues, exercise professional judgment, and carry their findings through to a completed filing, review, or recommendation.
\n![\"\"](\"https://cdn.prod.website-files.com/68dcd2ceb173c46fa029931c/6ae0c6e13cf0140c3efbcbcf_summary.png\")
DAYJOB tests whether AI agents can take ownership of a professional assignment from start to finish. That means navigating messy records, identifying issues the client never raised, making sound judgments with incomplete information, and delivering work that meets professional standards.
\nThe results show how difficult that remains for frontier models. Even when agents can perform the individual steps, exercising professional judgment and reliably carrying out real-world tax workflows remain out of reach.
\nAgents haven’t yet passed the audit.
\nThe full leaderboard, with every model we evaluated, is on the DAYJOB: Tax benchmark page.
\nTo learn more or request access to the full benchmark, contact benchmarks@surgehq.ai.
\nOriginally published at the Surge AI blog.
","headings":[{"level":2,"text":"Introducing DAYJOB: Tax","id":"introducing-dayjob-tax"},{"level":2,"text":"Results: Leaderboard and Pareto Frontiers","id":"results-leaderboard-and-pareto-frontiers"},{"level":3,"text":"Performance versus cost","id":"performance-versus-cost"},{"level":2,"text":"The breadth of professional tax work","id":"the-breadth-of-professional-tax-work"},{"level":2,"text":"What's inside the assignments?","id":"what-s-inside-the-assignments"},{"level":2,"text":"Created and reviewed by tax experts","id":"created-and-reviewed-by-tax-experts"},{"level":2,"text":"A retirement plan built on an unverified assumption","id":"a-retirement-plan-built-on-an-unverified-assumption"},{"level":2,"text":"Where DAYJOB: Tax fits","id":"where-dayjob-tax-fits"},{"level":3,"text":"TaxBench","id":"taxbench"},{"level":3,"text":"TaxCalcBench","id":"taxcalcbench"},{"level":3,"text":"TaxLab","id":"taxlab"},{"level":2,"text":"What DAYJOB: Tax measures","id":"what-dayjob-tax-measures"}]}}