Cell counting and segmentation with ImageJ + ML

A quick primer before the walkthrough:

In plain English: Combine classic ImageJ pipelines with modern segmentation models (Cellpose, StarDist) to count and measure cells reproducibly.

Key terms:

• ImageJ / Fiji — the standard open-source image-analysis tool for microscopy.
• Segmentation — outlining each object (cell, nucleus) in an image.
• Cellpose / StarDist — pre-trained AI models that segment cells much better than hand-tuned thresholds.

What you'll need: Fiji installed, a small set of test images, and 1–2 hours to run through the pipeline.

Comfort level: You've used ImageJ before. Python knowledge helps but isn't strictly required — you can drive Cellpose from a GUI.

New to this? You are replacing manual clicking with a pre-trained model that outlines nuclei for you, then letting ImageJ measure the outlines. You do not train anything and you do not write code until the macro step — which is just ImageJ recording your clicks.

What you'll build

A reproducible cell-counting workflow in Fiji/ImageJ using a pre-trained StarDist model, with an LLM helping you tune parameters and document the pipeline.

1. Install

• Fiji (Image J with plugins bundled): https://imagej.net/software/fiji/
• Enable update sites: Help → Update → Manage update sites → check StarDist and CSBDeep.
• Restart Fiji.

2. Segment nuclei with StarDist

Open a DAPI image, then run:

Plugins → StarDist → StarDist 2D

Recommended defaults for standard fluorescence nuclei:

• Model: Versatile (fluorescent nuclei)
• Probability threshold: 0.5
• NMS threshold: 0.4
• Output type: Label Image (for downstream measurement)

3. Measure

Analyze → Set Measurements → check Area, Mean, Integrated density, Shape descriptors. Analyze → Analyze Particles on the label image → Results table.

4. Record it as a macro

Plugins → Macros → Record, redo the steps once, save as count_nuclei.ijm. Now any image drops into:

run("StarDist 2D", "input=[] modelChoice=[Versatile (fluorescent nuclei)] "
  + "normalizeInput=true percentileBottom=1.0 percentileTop=99.8 "
  + "probThresh=0.5 nmsThresh=0.4 outputType=[Label Image] nTiles=1");
run("Analyze Particles...", "size=20-Infinity display clear");
saveAs("Results", getDirectory("image") + getTitle() + ".csv");

Batch it: Process → Batch → Macro.

5. LLM-assisted tuning

Paste 10 rows of your Results CSV and a low-res thumbnail description ("many merged doublets in dense regions") and ask:

"Given nuclei often touch in this image, what StarDist parameters or post-processing would reduce merged detections? Explain trade-offs."

Typical suggestions: lower nmsThresh to 0.3, add a watershed pass, or retrain with a few annotated tiles using the StarDist Python API.

6. Sanity check

Always count 3–5 fields by hand and compare. If the automated count is within 10% and errors aren't biased (all under- or over-counting one condition), you're done. Otherwise iterate.

Pitfalls

• StarDist is nucleus-specific — for whole-cell segmentation use Cellpose.
• Auto-threshold on brightfield rarely works; use fluorescence channels for detection and measure intensity elsewhere.
• Save the model version and parameter set with every dataset.

If something goes wrong

• StarDist finds far too many objects. Raise the probability threshold slightly, and check you are running it on the nuclear channel alone rather than a merged image.
• Touching cells get merged into one blob. Lower the overlap threshold, and confirm the image scale is set correctly so size filters mean what you think.
• Counts differ between images that look identical. Uneven illumination or focus is the usual cause. Apply the same background correction to every image and keep acquisition settings fixed.
• The macro works on one image but fails in batch. Batch mode needs explicit file paths and a close("*") at the end of each loop; record it once, then read through the recorded steps.
• The automated count disagrees with your manual count. Counting 3–5 fields by hand and comparing is a useful illustrative QC check, not a validation. A consistent offset is easier to interpret and report than a random one, but any decision about fitness for purpose belongs to your own validation process. Nothing here is suitable for clinical or diagnostic use.

• Before pasting results into an AI tool. Remove sample identifiers, patient metadata, and any other confidential information first, and follow your organization's policy on third-party tools.

Plain-English glossary

• Segmentation — deciding which pixels belong to which cell.
• Threshold — the cut-off that separates signal from background.
• ROI — a region of interest; here, one outlined nucleus.
• Scale / calibration — how many microns one pixel represents.
• Macro — a recorded list of ImageJ steps you can replay on a whole folder.