Reading paths: Start below for the plain-language explanation, or jump to the technical deep dive for implementation details and verification.
If a product promises an hourly summary, people expect about one summary per hour. They do not expect a new message every time the system finishes another calculation.
That gap appeared in a long job-matching discussion. The diagnosis connected repeated messages to digest creation happening when analysis finished. Several completions could reuse the same hourly boundary while still creating separate deliveries.
The later design proposed one logical digest per UTC hour, bounded results, and rules for late arrivals. That proposal is the source of this article. It is not evidence that the complete redesign shipped or that duplicate notifications disappeared in production.
A timer tells you when to look
Imagine a teacher sending a weekly class update. A calendar reminder tells the teacher when to write it. It does not prevent someone else from sending a second update for the same week. It also does not decide what to do with news that arrives after publication.
A timer has the same limitation. It can wake a service, but it cannot define the meaning of a digest. The product needs rules about which period the digest represents, which items belong in it, and whether that period has already been handled.
Without those rules, the timer and the rest of the application can disagree. One part thinks it is sending the hourly update. Another thinks it is announcing newly finished work. Both can be operating exactly as written while breaking the user's expectation.
The first design question is therefore a product question: what counts as one update?
Give the period a stable name
A useful identity might combine the recipient, the digest type, and the hour it covers. For example, a particular user's job summary for the hour beginning at 14:00 UTC is one logical object. Retrying its preparation should find that object again.
This is different from assigning a fresh random ID whenever a worker starts. A fresh ID distinguishes attempts. It does not establish that the attempts concern the same promised delivery.
Both identities can be useful. The logical identity says what the user should receive. The attempt identity helps investigate individual preparation or delivery attempts.
A uniqueness rule in storage can prevent two workers from independently creating the same logical digest. It cannot, by itself, guarantee that an external messaging service will display exactly one message. That boundary needs its own delivery reasoning.
Keeping these two problems separate makes the system easier to explain and test.
Decide what happens to late results
Suppose a digest closes at 14:00. One analysis finishes at 14:01, although the underlying item was discovered earlier. Should the system reopen the previous digest?
Reopening sounds helpful, but it makes the meaning of a completed period unstable. People might receive an extra message, a silent edit, or an update with a different ranking from the one they already read.
A simpler policy is to include the late result in the next eligible digest. The product can explain that summaries include results ready at the cutoff, rather than everything that happened in the outside world during an exact wall-clock interval.
That policy trades a little delay for predictable delivery. An urgent alert product may choose a different tradeoff. The important part is choosing deliberately rather than letting the timing of background calculations make the decision accidentally.
Selection is different from consumption
The job-matching plan also proposed a strict limit on the number of selected jobs. A bounded digest respects attention, but it creates another question: what happens to eligible items that did not fit?
Imagine 37 useful results are ready and the digest can include 20. Marking all 37 as delivered would silently lose 17. Leaving all 37 eligible without recording the selected set could send the same top 20 again next hour.
The design needs to record exactly which items were selected and what happens next. Unselected items might remain pending, expire after a defined period, or fall out when better results arrive. Each choice affects the product people experience.
Once preparation begins, saving the chosen set also makes retries more predictable. A delivery retry should not quietly become a different digest because the ranking changed while the messaging provider was unavailable.
A failed send and an uncertain send differ
Some delivery failures are clear. A provider may reject a request before accepting it. Other failures leave uncertainty. The application may time out after the provider has accepted the message but before the response reaches the application.
Immediately retrying an uncertain send can produce a duplicate. Never retrying it can leave someone without a message. There is no magic local database flag that removes this ambiguity from every external service.
If the provider supports a reliable deduplication key, reuse the logical delivery key. Otherwise, preserve an uncertain state and use whatever reconciliation evidence the provider offers. Some products may choose an explicit manual recovery path.
The historical plan treated ambiguous sends cautiously. The transferable lesson is to represent uncertainty honestly instead of converting every timeout into "not sent."
Separate different promises
A job digest and a questionnaire are different product messages. One summarizes new opportunities. The other asks for information that may improve future recommendations.
Combining them behind one loosely defined delivery mechanism can create confusing behavior. If job-card preparation fails but questions continue, the user may see a question-only interaction and reasonably conclude that the matching feature is broken.
Separate preparation and delivery records make those outcomes visible. The product can then apply different limits and retry rules to each message type.
Separation does not require building a large collection of services. A small application can use clear records and functions within one codebase. The useful boundary is in the meaning and ownership of the messages, not in the number of deployed processes.
Test a timeline, not just a function
A test that calls a digest function once will miss most of the interesting failures. The important examples involve time and competing events:
| Timeline | Expected product behavior |
|---|---|
| Two workers claim the same hour | One logical digest exists |
| Analysis finishes after the cutoff | It follows the late-arrival policy |
| More items qualify than fit | Only selected items count as included |
| Delivery clearly fails | A retry keeps the same selected content |
| Delivery outcome is uncertain | The system preserves uncertainty |
| The service restarts after several hours | Catch-up follows a bounded policy |
These cases test promises someone can understand. They also reveal where the system needs durable state: information that survives a restart, rather than disappearing with the process that happened to be running.
Catch-up should respect attention
After an outage, a scheduler may discover several missed periods. Sending every missed digest immediately can turn a recovery success into a bad user experience.
The product might send one combined catch-up summary, prepare only the newest period, or offer older results inside the application. The correct choice depends on how quickly the information loses value and what the user was promised.
Record the skipped or combined periods so the next restart does not rediscover them as unfinished work. Recovery needs a durable explanation of what happened, even when that explanation is "intentionally combined into a later summary."
Measure the promise people experience
A healthy timer is not enough to prove a healthy notification feature. Useful checks include duplicate logical periods, late-result delay, selected items never delivered, uncertain sends, and the number of messages a recipient actually receives.
Those measurements connect infrastructure behavior to the product promise. They also help distinguish a ranking problem from a scheduling problem or a delivery problem.
An hourly digest is a small distributed-systems problem hiding inside an ordinary product feature. Giving the hour a stable identity makes its rules visible. The rest of the design becomes a series of understandable choices about timing, content, and trust.
Technical deep dive
Plain-language bridge: The hour needs a permanent identity, and retries need to reuse it. Otherwise, each retry can look like a brand-new promise to the user.
Diagram: Freeze one digest per logical hour
[results ready before cutoff]
|
v
[recipient + kind + UTC hour] <---- unique logical identity
|
v
[commit selected item IDs]
|
v
[send]
/ | \
v v v
confirmed rejected uncertain
| |
v v
retry reconcile
[result ready after cutoff] ----> next eligible digest
Retries reuse the digest identity and selection. An uncertain send requires evidence about delivery; a timeout alone does not establish that nothing was sent.
Define the bucket and uniqueness rule
For an hourly UTC policy, the bucket start is floor(epochSeconds / 3600) * 3600. An illustrative uniqueness key is:
(recipient_id, digest_kind, bucket_start_utc)
A database uniqueness constraint can prevent two workers from creating separate logical digests for that key. It does not prevent two attempts to send the same digest externally.
Choose the cutoff separately from the bucket label. For example, a digest prepared for 14:00 may select results ready before 14:00 and defer later completions. Specify whether the interval is closed or open at each endpoint so a result exactly on the boundary has one unambiguous destination.
Commit a stable selection
Inside the preparation transaction, create or obtain the digest, select eligible items, and record its selected item IDs. If items may appear only once across all digests of a kind, enforce that ownership in the data model as well as in the selection query.
Use a deterministic tie-breaker for equal scores. Sorting only by score can produce different top results across retries. Save the selected payload version or the required source versions so later delivery does not silently recalculate a different digest.
Mark an item included only when it belongs to the committed selection. Eligibility, selection, and confirmed delivery remain separate facts.
Model delivery uncertainty explicitly
A useful delivery state machine distinguishes prepared, sending, confirmed, rejected, and uncertain. A timeout after sending is not automatically a rejected delivery.
prepared -> sending -> confirmed
-> rejected -> eligible retry
-> uncertain -> reconcile
On retry, reuse the provider's idempotency key if that provider supports the required semantics. Without such support or a way to query the accepted delivery, exactly-once visible delivery may be unattainable. The product needs a deliberate choice between duplicate risk, missed-message risk, and manual resolution.
Test time and competing workers
Use a controllable clock for bucket boundaries and barriers for concurrent preparation. Test the instant before and at the cutoff, a late analysis, equal-ranked items, two claimants, and a restart after an uncertain send. Verify both digest rows and selected item ownership.
For catch-up, place an explicit limit on missed buckets and record combined or skipped periods durably. Otherwise, every restart may rediscover the same missed hours.
The historical discussion proposed this kind of redesign. The contract here makes its requirements concrete without presenting the plan as a verified production rollout.