Two ways a recurring task scheduler breaks

"Every Monday at 9am, create a task called 'Prepare weekly report.'" The date math for that is genuinely simple. What's not simple is what happens when you run this scheduler on more than one server, and what happens when one specific recurrence rule is somehow broken — both of which turn a straightforward feature into either duplicate tasks piling up every week or a background job stuck retrying the same failure forever.

Failure mode one: two replicas, one Monday

If every backend replica runs its own ticker checking "is anything due?", every replica finds the same due recurrence at the same time and every replica creates a task for it. This is the same shape of problem as the weekly digest email — see [Why your weekly digest never sends twice](/blog/weekly-digest-dedupe) — but WKFGo solves it differently here, because task creation isn't a single idempotent insert the way a (org, week) unique row is.

The fix is leader election instead of a database constraint. WKFGo elects one replica as leader using a PostgreSQL session-level advisory lock, and the recurrence ticker checks that status before doing anything:

for range ticker.C {
    // Singleton job: only the elected leader materializes recurrences so
    // a task is not created once per running replica.
    if !cluster.IsLeader() {
        continue
    }
    s.RunDue()
}

Every replica runs the ticker; only the one holding the advisory lock actually acts. If the leader's process dies, its database session drops, Postgres releases the lock, and another replica picks it up on its next tick — failover with no extra coordination infrastructure, and on a single-instance deployment the lock is acquired immediately so nothing behaves differently.

Failure mode two: a broken rule that hot-loops

Say a recurrence references a column that got deleted, and Materialize starts failing every time the scheduler tries it. The naive implementation retries that same rule every tick, forever, logging an error every minute and never making progress. WKFGo's scheduler advances next_run_at regardless of whether materialization succeeded:

if err := s.Materialize(&due[i]); err != nil {
    log.Printf("[recurrence] rule %d (%q) failed: %v", due[i].ID, due[i].Title, err)
    // Push next_run_at forward anyway so a broken rule can't hot-loop.
}
now := time.Now()
s.db.Model(&Recurrence{}).Where("id = ?", due[i].ID).Updates(map[string]interface{}{
    "next_run_at": due[i].ComputeNext(now),
    ...
})

A failing rule gets exactly one attempt per scheduled occurrence, not an unbounded retry storm — it fails, logs, and the schedule moves on to the next occurrence rather than immediately retrying the one that just broke. The tradeoff is explicit: a broken rule silently skips a task instead of blocking the entire scheduler pass behind a retry loop. Given the alternative is a scheduler that can get stuck making zero progress on every other recurrence because one rule is broken, skipping forward is the safer failure.

The month-end edge case

ComputeNext for a monthly recurrence clamps the day of month to 28:

day := r.MonthDay
if day < 1 {
    day = 1
}
if day > 28 {
    day = 28 // keep it valid in February too
}

A rule set for "the 31st of every month" would either error out in February or silently drift depending on the calendar library's overflow behavior — clamping to 28 trades a small amount of date precision (it's always the 28th, never the 30th or 31st) for a schedule that behaves identically in every month, with no special-casing needed for February or 30-day months.

What "materialize" actually creates

Creating the task is not just an INSERT — it also assigns the configured users, sends each an in-app notification, and broadcasts the same task.created realtime event a manually created task would, so a recurring task shows up live on the board exactly like one a person just typed:

realtime.Broadcast(r.ProjectID, "task.created", 0, map[string]interface{}{
    "id": task.ID, "recurrenceId": r.ID,
})

Nothing downstream — the board, webhooks, notifications — needs to know a task came from a recurrence rather than a person; it's the same event either way, tagged with recurrenceId for anyone who wants to filter on it.

Common mistakes

Retrying a failed scheduled job indefinitely instead of advancing past it. It feels safer ("we'll get it next time") but a genuinely broken rule then consumes scheduler cycles forever instead of failing once and surfacing in logs for someone to fix.

Running singleton background jobs on every replica without coordination. It works fine at one instance and silently starts duplicating output the moment you scale horizontally — exactly the kind of bug that only shows up in production, under load, right when you least want a new bug.

Not clamping calendar edge cases (Feb 30th, month lengths). Recurring-task and recurring-billing code both hit this; unclamped day-of-month arithmetic either throws or drifts depending on the language's date library.

FAQ

Can I trigger a recurrence manually, outside its schedule?

Yes — there's a "run now" endpoint that calls Materialize directly, useful for testing a new rule without waiting for its next scheduled occurrence.

What happens to already-created tasks if I delete the recurrence rule?

They're untouched — a recurrence only creates tasks going forward; deleting the rule stops future creation and has no retroactive effect on past ones.

Does a paused (inactive) recurrence still get checked every tick?

The due-query filters on active = true at the database level, so an inactive recurrence is never even fetched, let alone materialized.

Summary

A recurring task feature isn't really about the calendar math — that part's a day of work. It's about making sure the thing that runs the calendar math runs exactly once per occurrence, cluster-wide, and that one broken rule can't take the rest of your scheduled work down with it.

• Set up a recurring task
• Explore features
• Pricing