Using Claude to critique your experimental design

A quick primer before the walkthrough:

In plain English: Use Claude as a "first reviewer" for your experimental design before you burn through reagents and instrument time.

Key terms:

• Experimental design — the plan for how many samples, controls, and replicates you'll run.
• Power / sample size — how many samples you need to reliably detect a real effect.
• Confounder — a hidden variable that could explain your result and mislead the conclusion.

What you'll need: A Claude account and a written draft of your protocol or study design.

Comfort level: You should be able to describe your experiment in a paragraph or two. Some stats familiarity helps but isn't required.

New to this? You are using the model as a sparring partner: you describe the experiment you plan to run in six short fields, and it tries to poke holes in it before the reagents are used. It is not a statistician and not a PI — everything it flags still needs your judgement.

Claude as a design partner, not an oracle

We have found Claude to be good at structured critique: when asked explicitly, it tends to surface confounders and alternative explanations rather than simply agreeing with a plan. That makes it a useful second reviewer for experimental design — not a replacement for your PI or a statistician, but a fast, patient sparring partner.

This tutorial shows a workflow you can use before substantive experiments: a 15-minute Claude critique that can surface common design flaws such as missing controls, underpowered arms, and ambiguous readouts.

The setup

Open Claude and paste a design brief, not a protocol. The brief has six fields:

1. Question. The specific biological question, in one sentence.
2. Hypothesis. What you expect to see, and the direction.
3. System. Model organism, cell line, reagents, key vendors.
4. Manipulation. What you are perturbing, how, and at what dose/duration.
5. Readout. What you will measure, how, and the expected effect size.
6. Analysis. Statistical test, n per arm, and how you are powering it.

Then a critique prompt:

You are a rigorous peer reviewer. Critique the experimental design below.

For each of these, either confirm it is adequate or flag a problem:
- Positive control (is there one, and does it demonstrate the assay works?)
- Negative control (is there one, and does it demonstrate specificity?)
- Vehicle / sham control (matched for solvent, handling, timing?)
- Blinding (who is blinded, at what step?)
- Randomization (how are subjects/wells assigned?)
- Confounders (batch, sex, passage number, time of day, edge effects)
- Statistical power (is n justified? are you multiple-testing?)
- Readout specificity (could a non-specific effect produce the same signal?)
- Effect size (is the expected effect within the assay dynamic range?)
- Alternative explanations (list three plausible ones beyond the hypothesis)

For each flag, propose a concrete fix.

Why this beats "check my design"

A generic "check my design" prompt gets you generic advice. The structured checklist forces Claude to touch every failure mode explicitly, which surfaces the ones you did not think to ask about.

A worked example

Below is a complete, realistic design brief for a compound screen in THP-1 macrophages. It is intentionally imperfect — the kind of first draft a team might write before running the assay. The projected Claude output is illustrative guidance, not a validated protocol; a qualified methods or statistics reviewer should verify any design before it is used in the lab.

1. The design brief

Question Does compound X reduce IL-6 secretion in LPS-stimulated THP-1 macrophages compared with vehicle-treated cells?

Hypothesis Compound X will dose-dependently reduce IL-6 release, with the highest dose producing at least a 50% reduction relative to the LPS-only vehicle control.

System THP-1 monocytes (ATCC) differentiated to macrophages with 100 nM PMA for 48 h, then rested 24 h in fresh RPMI + 10% FBS. Cells are plated at 1 × 10^5 cells/well in 96-well plates.

Manipulation

• LPS stimulation: 100 ng/mL E. coli O111:B4 LPS for 6 h.
• Compound X: 0.1, 1, and 10 μM, added 1 h before LPS.
• Vehicle: 0.1% DMSO, matched volume across all wells.

Readout Supernatant IL-6 measured by sandwich ELISA (pg/mL). Each plate includes a recombinant IL-6 standard curve.

Analysis One-way ANOVA with Dunnett's post-hoc test versus the LPS + vehicle group. n = 3 wells per condition, pooled across three independent experiments.

2. The prompt pasted into Claude

Privacy and confidentiality warning: Do not paste confidential, proprietary, patient, or unpublished data into Claude or any other third-party AI service. Remove internal compound names, patient identifiers, proprietary protocols, and any information you are not authorized to share before using this workflow.

You are a rigorous peer reviewer. Critique the experimental design below.

For each of these, either confirm it is adequate or flag a problem:
- Positive control (is there one, and does it demonstrate the assay works?)
- Negative control (is there one, and does it demonstrate specificity?)
- Vehicle / sham control (matched for solvent, handling, timing?)
- Blinding (who is blinded, at what step?)
- Randomization (how are subjects/wells assigned?)
- Confounders (batch, sex, passage number, time of day, edge effects)
- Statistical power (is n justified? are you multiple-testing?)
- Readout specificity (could a non-specific effect produce the same signal?)
- Effect size (is the expected effect within the assay dynamic range?)
- Alternative explanations (list three plausible ones beyond the hypothesis)

For each flag, propose a concrete fix.

DESIGN BRIEF:
[ paste the six-field brief above ]

3. Projected Claude output

Confirmed - Positive control: The LPS + vehicle arm is present and should produce a robust IL-6 signal; the standard curve validates the ELISA readout. - Vehicle control: 0.1% DMSO is included and matched by volume.

>

Flagged — with fixes - Negative control missing. There is no unstimulated vehicle arm (no LPS, no compound) to establish baseline IL-6 secretion. Fix: add an unstimulated vehicle control to every plate. - Cytotoxicity readout missing. If compound X reduces viability, IL-6 will drop for reasons unrelated to anti-inflammatory activity. Fix: add a cell-viability assay (e.g., resazurin or LDH) for every dose in the LPS-stimulated arms. - Vehicle mismatch risk. If compound X stock concentrations differ, final DMSO may vary across doses even if volume is constant. Fix: back-dilute all stocks to the same solvent concentration so every well receives the same final DMSO percentage. - Randomization not stated. The design brief does not say whether wells are randomized across plate positions. Fix: randomize treatment assignment by plate map to avoid edge and positional effects. - Confounder: PMA batch and passage. THP-1 differentiation efficiency varies between PMA lots and passage numbers. Fix: use a single PMA aliquot for the entire study, keep passage number narrow (e.g., P4–P10), and include experiment as a blocking factor in the analysis. - Pseudoreplication. n = 3 wells × 3 experiments gives nine measurements, but ANOVA treats wells as independent. Wells within one experiment are technical replicates. Fix: average the three wells per experiment to obtain one value per condition per experiment, then analyze with experiment as the biological replicate. With only three independent experiments, consult a statistician before fitting a mixed model. - Readout specificity. A reduced IL-6 signal could be caused by cytotoxicity, ELISA matrix interference, or reduced cell density. Fix: pair the ELISA with a cell-viability assay and a spike-recovery control. - Alternative explanations. (1) Compound X inhibits TLR4 signaling directly rather than the proposed downstream target. (2) Compound X is cytotoxic at 10 μM. (3) Compound X interferes with the ELISA detection antibody or standard curve.

4. What changed because of the critique

Getting more out of the conversation

After the initial critique, follow up with:

• "Rank the flags by how likely they are to change the conclusion."
• "For the top flag, draft the added experiment in one paragraph."
• "What is the smallest change to the design that addresses all critical flags?"

Claude handles multi-turn refinement well; use it.

Where Claude falls short

• Statistical power calculations. It will give you a formula and often a number, but treat the number as directional. Run G*Power or a proper simulation for anything you will publish.
• Domain-specific artifacts. Claude does not know that your particular antibody clone cross-reacts with a paralog. Feed it the reagent details if you want that kind of catch.
• Regulatory contexts. Do not use Claude to critique a GLP or IND-enabling study design without an expert in the loop.

Bottom line

Fifteen minutes with Claude and a structured brief will not replace a good collaborator. But it will catch the boring failure modes — the missed control, the mismatched vehicle, the underpowered arm — that are the most common reason experiments have to be repeated.

If something goes wrong

• The critique is generic ("add more replicates"). Your brief was thin. Give real numbers — plate layout, n per group, concentrations, timepoints — and it will get specific.
• It flags things that do not apply to your system. Say so and ask it to re-rank the remaining flags by how likely each one is to change your conclusion.
• It proposes a statistical test with confidence. Treat that as a prompt to talk to a statistician, not as an answer.
• You end up with twenty action items. Ask for the three that most threaten the main claim, and handle those first.

Plain-English glossary

• Positive / negative control — conditions that should and should not produce the effect, proving the assay works.
• Vehicle — the solvent the compound arrives in; it must be matched across conditions.
• Confounder — something that changes alongside your variable and could explain the result instead.
• Technical vs biological replicate — repeat measurements of one sample vs independent samples.
• Power — the chance of detecting a real effect of a given size with your sample number.