How long do you need to track before a pattern means anything?

Why do short records mislead?

Two things go wrong at once.

Noise. Most symptoms vary a great deal from day to day for no reason anyone can name. In a two-week record, one bad weekend is a quarter of the evidence. Averages and frequencies swing with single days. By four weeks the swing has mostly settled, which is roughly why headache guidelines ask for a diary of about eight weeks before decisions are made on it (NICE CG150).

Too many comparisons. A diary with a symptom and ten context factors invites fifty-odd questions (“is it worse after short sleep? after coffee? on Mondays? the day after rain?”). Ask fifty questions of random numbers and a few will come back looking like answers. This is the multiple-comparisons problem, and the standard remedy is to demand stronger evidence the more questions you ask (Benjamini & Hochberg, 1995). Most tracking apps do not. They report the three strongest-looking links from the fifty and present them as findings.

How many days for what?

These are not thresholds at which a pattern becomes true. They are the points at which the record stops being dominated by chance.

What makes a link trustworthy?

Four tests, none of which needs statistics to apply:

1. Enough days on both sides. “Worse after late nights” means nothing if there were four late nights. Longnote, for instance, refuses to headline a comparison where the smaller side has under eight days.
2. It holds when the timing shifts. A real next-day effect should not vanish if you look at the same day and reappear two days later. A link that only exists at one convenient lag is often noise.
3. It survives the number of things you compared. The more factors in the diary, the stronger a single link has to be before it should be believed.
4. It still shows up next month. The best test of a pattern is whether it repeats.

What can you do with an incomplete record?

A great deal. Counts are useful long before correlations are: “nine headache days in thirty, average 5 of 10, mostly left-sided, three of them with nausea” is exactly what an appointment runs on, and it needs no pattern at all. Keep logging the plain facts, keep your hypotheses in a separate note, and let the record catch up with them.

How does Longnote grade its own statements?

Longnote sorts everything it tells you into four standings. An observation is a fact about one tracker and cannot be wrong. A hint is a possible link where one side of the comparison has only three or four days; it appears only in browsing lists, with both day counts, and never in Highlights or the report. An early sign is a possible link that has not been confirmed yet, worded as a hunch with its day count; Highlights show at most one, and only under sixty days of overlap. A pattern needs at least thirty days of overlap and must survive a false-discovery check across every comparison the app made, at a one-in-ten rate, and still point the same way in the range on screen. Alongside, every link carries an evidence word: limited data, fair evidence or good evidence. A strong-looking link on twelve days is never called strong. The reasoning is written up in the blog post on how Highlights decide what to say.