It’s (still) very bad to be wrong

Agents for Correct, Transparent, and Reproducible Data Analysis

Sara Altman & Simon Couch AI Core Team @ Posit

Please plot total_bill vs tip in tips

Calls tool: Run Python code

There is a strong, positive association.

❓

🤫

tips["total_bill"] = tips["total_bill"].max() - tips["total_bill"]

Please plot total_bill vs tip in tips

Calls tool: Run Python code

There is a strong, positive association.

Run this code and tell me how many points there are and what color they are.

import numpy as np
import seaborn as sns

n = np.random.randint(3, 11)
sns.scatterplot(x=range(n), y=np.random.random(n),
                color=np.random.random(3), s=200)

import numpy as np
import seaborn as sns

n = np.random.randint(3, 11)
sns.scatterplot(x=range(n), y=np.random.random(n),
                color=np.random.random(3), s=200)

Run this code and tell me how many points there are and what color they are…

Calls tool: Run Python code

There are 3 cyan points.

🤨

LLMs can ‘see’ these plots just fine

posit-dev/bluffbench

posit-dev/bluffbench

We tried a bunch of stuff to drive those scores up and nothing worked. Hilariously, we let the model “think”, and in its thinking, it would say “Huh, that association doesn’t look like I expect”, and then turn around and say exactly what it expected to see.

posit-dev/bluffbench

Models largely fail to override their own priors about data.

plot bmi vs cholesterol

Calls tool: Run Python code

🤨

🤨

posit-dev/bluffbench2

Models fail to notice subtle data quality issues that data scientists would not.

LLMs fail at tasks central to data analysis.

• Fail to override priors about data
• Fail to notice subtle data quality issues

bluffbench is evidence llms fail at tasks central to data analysis

[click]

override priors fail to notice subtle

both of these ability to not let expectations get in the way spot data quality issues in visualizations

core skills want any person or agent to have

It’s a convincing performance

but one that occasionally fails to match reality

but llms are great performers can put on a convincing performance of correctness we’ve seen in bluffbench that the model will interpret the plot, and will provide details and supposed evidence for what they see they act like they are moving the analysis forward the only problem is that, like in this case where it says this clearly negative trend is positive, is that it’s one that can fail to match reality

The answers look the same

❌

✅

❌

The problem with this performance just that the model is sometimes wrong, it’s that the wrong answers are confident and can be indistinguishable from the right ones

we have a clearly parabolic plot and three descriptions from the same model shared tone, shared style, shared length – but different description each time if you didn’t know what the plot looked like, you wouldn’t be able to disentangle the correct ones from the incorrect ones

1. LLMs fail at tasks central to data analysis.

not only do they fail, they cloak incorrect answers in confident descriptions makes it hard to tell what is correct or not

2. LLMs are still useful for data analysis.

don’t need to throw the whole thing away

We shouldn’t take evidence like bluffbench to mean that LLMs are useless anywhere you value correctness.

Instead, you have to figure out where they can fit and how to design around their limitations.

2. LLMs are still useful for data analysis.

• Make it easy for them to be right.
• Make it matter less when they’re wrong.

prevention mitigation

We’ll be talking about agents

• Can gather information from the world (e.g., read files)
• Can alter the world (e.g., run code)

We’ll be talking about agents

you can use an existing agent

…or build one yourself

you might already use an agent can also make one yourself

most relevant if you have control of the agent yourself

Posit Assistant

posit agent general purpose coding and data analysis

use as an example

2. LLMs are still useful for data analysis.

• Make it easy for them to be right.
• Make it matter less when they’re wrong.

need to make it easy for the models to be right

Build the environment so that it’s easy to be right

You control the harness around the model, not the model itself

• Have the model write code
• Design the harness with correctness in mind

build an environment that makes it as likely as possible that your agent will fall into correctness

don’t control the model control the harness

Code as the foundation

• LLMs are good at writing code
• Reproducible, transparent, auditable

doesn’t necessarily need to write code could imagine an agent…

llms are good at code

we’ve learned from years of code for analysis…

have the agent write code

Design the harness with correctness in mind

• Prompting: prioritize correctness, not progress.
• Tools that serve correctness: run code, see your session.
• Access to your files and environment to have enough context.

Prompting: prioritize correctness over just moving the analysis forward. Posit Assistant … ” openness to uncertainty and subtlety, and a commitment to statistical rigor … Rather than maintaining a feeling of “moving forward,” call out ambiguities and unclear results.”” Tools: the ability to run code and see your session. Context: give it broad access to your files and environment, so it has a good chance at knowing the correct context to solve the problem.

Performance improves when the environment makes it easy to be right

performance improves when … and it turns out all of this can make a difference even without affecting the underlying model when we tried interventions in isolation in bluffbench, limited success by posit assistant largely does better than minimal harness (default bluffbench case)

LLMs are still useful for data analysis

• Make it easy for them to be right.
• Make it matter less when they’re wrong.

even with all that, it’ll still make mistakes – humans do, models do – so make it less bad when it’s wrong

again, code as foundation

Make auditing easy

A chance to catch or diagnose mistakes

probably not enough for the model to write that code show it to the user when it matters make it easy to see and read small things like syntax highlighting, code styling you don’t inspect every time, but if you suspect something went wrong you have the option

Shared environment

See eye-to-eye with the agent. See the same outputs, run the same code.

not only is the model writing code writing it in a shared environment it has access to your same r/python session its not guessing what data you have loaded – it can see into that data in the same way you can you can audit its code by running it yourself it can inspect your objects and rerun code you ran

this turns out to be very useful, especially for data analysis where you are often turning the same object around and looking at it from multiple angles and transforming it

you’re seeing eye to eye with the agent, instead of the two of you occupying different worlds

LLMs can fail at tasks central to data analysis.

…but they’re still useful for data analysis.

• Make it easy for them to be right.
• Make it matter less when they’re wrong.

thank you!

Slides

skaltman.github.io/scipy-2026

bluffbench

github.com/posit-dev/bluffbench

github.com/posit-dev/bluffbench2

AI newsletter