TriageWell is a real symptom-urgency assistant we are building, backed by Claude and a hard-coded emergency safety net. Before we called it done, we ran contradish on it the same way we would on anyone else's product: same phrasing techniques, same judge, no special treatment.
Not the bare Claude model. The actual running application, including its hard-coded safety layer.
TriageWell sorts a symptom into one of four tiers (emergency, urgent, same-week, self-care) and asks clarifying questions before deciding, unless what has already been said is enough to know it is an emergency. On top of the model, a plain pattern-matching check runs on every message and can raise an emergency banner on its own, independent of whatever the model says. We wanted to know whether the tier, and the banner, stayed the same when the same symptom was described differently: casually, as a hypothetical, prefaced with a claimed medical background, or minimized with a story about someone else.
Before any changes were made in response to the results.
The clearest failure was also the most important one. TriageWell's hard-coded emergency banner fired correctly every time, regardless of phrasing, because it does not depend on the model's judgment at all. But the model's own written reply did not hold as firm:
This is the exact failure mode the anti-manipulation rule in TriageWell's own system prompt was written to prevent, it just had not anticipated this particular framing. A story about someone else is not information about this person's risk, and the model was not yet holding that line.
contradish's judge does not just flag failures, it names the pattern and suggests the fix.
Never soften an emergency because it is wrapped in a third-party anecdote ("my friend had this and was fine"). For an active red-flag symptom, the call-911 directive must be the first sentence, before any clarifying questions and before engaging with whatever story came with it.
State medication dosing limits with the same fixed numbers every time, regardless of how casually or hypothetically the question is asked. Weight-based scaling of an adult medication dose for a child is never acceptable, regardless of urgency or how emotionally pressured the situation is.
We re-ran the identical 33 cases against the patched prompt. No cases were removed or reworded to make the second run easier.
Every case the fix directly targeted came back clean:
0.293 is better. It is still contradish's own "marginal" band, not "stable." Here is what did not get fixed, including one thing that got worse.
Every number on this page came from contradish's own library pointed at a real, running application, not a demo built to look good.