One number decided everything
A public booking portal releases slots without warning. They are taken within minutes. People set alarms for six in the morning and refresh, for weeks.
We measured one. Five checks in a row reported nothing. The sixth reported four openings. The seventh reported nothing again. The gap was ninety two seconds.
Every decision in the product that followed is downstream of that number, and that is the useful lesson: find the one measurement that constrains the system, and most of the architecture stops being a matter of taste.
What ninety seconds rules out
Polling measured in minutes. A five minute interval misses most of them. The interval had to be under the window with room to spare, which is where ninety seconds came from.
A human in the loop. Any design where somebody reviews before the alert goes out is already too slow. So the review moved to where it belongs: what counts as a change worth sending is decided in advance, in the rules, not in the moment.
One check per subscriber. Twenty people watching the same office would mean twenty requests against a public service that never asked for our traffic. Shared checks mean one request per procedure, and everyone watching that procedure gets the result. That is also the reason the product can be sold for a one time fee rather than a monthly one.
The parts that are easy to forget
Confirm before you send. A false alert costs more trust than a late one. A change is verified before anything goes out, because the fastest way to lose a subscriber is to make them refresh a page that has nothing on it.
Two channels, not one. Email and a messaging app, both included, because an alert that sits unread in an inbox for an hour did not happen. Charging extra for the second channel would have been the obvious business decision and the wrong product decision.
Watch the watcher. A second process checks that the monitor is still running. A monitoring tool that dies quietly is worse than no monitoring tool, because the silence is indistinguishable from good news.
Write down why an alert went out. Every check, every detection, every send, logged. When somebody asks why they were told at 04:12, the answer has to be readable without opening code.
Always on is a different kind of build
Most projects are finished when they ship. This one is only useful while it is running, and that changes every decision: retries, rate limits, health checks, and a separate process whose only job is to notice when the main one stops.
It also changes what failure costs. An hour of downtime on a marketing site is an inconvenience. An hour of downtime here is a refund, because the product promised to be awake.
The transferable part
If you are thinking about a system that watches something, find your ninety seconds first. How long does the thing you care about stay true? Everything else - the interval, the channels, the alerting rules, the cost of the whole operation - falls out of that one answer, and arguing about the rest before you have it is wasted time.
The product this is written from is CitaCatch. The alerting and retry side of it is the same work described under automation and integrations.
