This blog post is Human-Centered Content: Written by humans for humans.

Data Analysis questions are among the most common questions I see from clients on the Assist desk. Now that AI has entered the chat, so to speak, I get a lot of questions about using AI to analyze, clean and work with data to make the crunching of numbers, especially numbers contained in large and complex tables, a bit easier. Recently I had a client come to me with the following question:

“I’ve got a large data set I’m trying to get some preliminary analysis on, and I’ve asked Claude to analyze it to get our average transaction value and find any major outliers. The numbers look pretty good, but they’re off from the calculations and checks I’ve done on my own. What am I missing?”

Out of the box with a plain text or reasoning-only prompt, Claude performs data and document analysis tasks the same way it handles any other question. It reads and reasons over the text and files you give it, then generates answers based on patterns and estimation rather than crunching the actual numbers.

A reasoning-only prompt might look like this:

“My sales data is [pasted/attached]. Act as a data analyst, take a look, tell me the average order value and point out any orders that seem unusually high or low.”

Prompted this way, Claude estimates rather than calculates. Sums, averages and percentages come from pattern matching on what it reads rather than math, and nothing checks the work. There is no built-in mechanism to catch a misread value, a missed row or a wrong assumption.

For small, simple data tasks where you can paste in the data and eyeball it yourself, this level of analysis is usually fine. It gets risky when you have larger and more complex data sets, multi-step aggregation, and in any situation where the number has to be right.

When precision matters, code-based prompting is the way to go.

A code-based prompt might look like:

“My sales data is [pasted/attached]. Act as a data analyst and write and run Python (pandas) to calculate the exact average order value, then flag any orders more than two standard deviations from the mean as outliers.”

This solves most of those problems encountered with a reasoning-only prompt with just a sentence change. We ask Claude specifically to write and run code to analyze the data instead of reasoning its way to an answer on a quick look-see and vibes. The math is math instead of a well-informed guess, and you can read the code to see exactly what Claude did.

So, How Do We Check the Work?

Getting the math right is only half the battle. Code-based analysis can and will still make mistakes. So what do we do? We add one more piece to this prompt (a validation pass) and ask Claude to check its work and validate the analysis before delivering it.

Our code-based prompt with the validation step added might look like:

“My sales data is [pasted/attached]. Act as a data analyst and write and run Python (pandas) to calculate the exact average order value, then flag any orders more than two standard deviations from the mean as outliers. Before giving me the final results, validate the analysis: Check for double-counting or duplicate rows, confirm the aggregation isn’t skewed by missing or null values, and recompute the average using a second method to cross-check it. Show me the validation results along with the final numbers.”

We’ve kept the same computation step as before, but with a built-in sanity check and visibility into both the initial analysis and the verification step.

What we end up with is three layers, each one covering for the layer before it. Code execution with our code-based prompt gets the actual math. The verification step confirms that the analysis answered the question(s) we actually asked. And the visible code lets us do a final human audit before the numbers go anywhere that matters.

For an added layer of verification, we can take the analysis that’s delivered and ask Claude in a new session to verify the numbers one more time. That might look like pasting in your analysis from the previous prompt or adding the file output along with the original data and framing the new prompt as an audit request.

That audit request prompt might look like:

“I ran an analysis earlier and got these results: [paste results/code/summary]. Here’s the original data again: [attach]. Write and run Python to independently recompute the average order value and outlier detection from scratch. Don’t just review the logic, execute the calculations. Show your work and check for double-counting, incorrect aggregation, or survivorship bias, and tell me if the original numbers hold up.”

The aim is to get analysis worth checking, not to cut the human out of the loop. Using code-based prompts and validation for data analysis has made a noticeable difference in the usefulness of Claude’s output for analysis tasks for this client, especially with larger data sets and instances where multiple data sets need to be considered together.

Trust takes verification. If you want help building that into your own workflow, we’ve got a solution to help you.