Heartbeat
How Telescope receives heartbeat signals and detects missed or failed runs.
Heartbeat monitors are passive checks for scheduled jobs and background workers. Your system sends signals to a unique URL, and Telescope evaluates whether signals are on time, late, missed, or explicitly failed.
How it works
- Your job sends
GET,POST, orHEADrequests to the monitor's heartbeat URL. - The API validates the heartbeat ID, signal type, and per-monitor rate limit.
- The API records signal metadata in monitor
config(latest ping/signal/fail timestamps and request metadata) and inserts aneventsrow. - The worker runs
HeartbeatChecker, reads the latest config, and evaluates monitor state using expected interval plus grace period.
Supported signals
You can send signal state using a path segment (/:signal) or query (?signal= / ?state= / ?status=):
| Signal | Meaning |
|---|---|
success | Successful run/ping (default if omitted) |
start | Run started (informational) |
fail | Explicit failure signal |
Optional message text can be passed with ?msg= / ?message= or JSON body { "message": "..." }.
Status evaluation
HeartbeatChecker evaluates these states:
| State | Monitor status | Meaning |
|---|---|---|
waiting_for_first_ping | Up | Monitor has not received any ping yet |
in_progress | Up | A start signal was received and is still within grace |
on_time | Up | Last success ping is within expected interval |
in_grace | Up | Ping is late but still inside grace period |
missed | Down | No success ping before grace deadline |
failed_signal | Down | Most recent explicit signal is a failure |
Configuration options
Stored in monitor config (camelCase or snake_case aliases accepted):
| Setting | Default | Description |
|---|---|---|
expectedIntervalSeconds | falls back to monitor frequency | Expected time between successful pings |
gracePeriodSeconds | 300 | Extra delay allowed before marking down |
Result data highlights
Heartbeat checks write structured result_data including:
stateexpected_interval_ms,grace_period_mslast_ping_at,last_signal_at,last_fail_at,last_start_at,last_signal_stateexpected_by,grace_deadlinems_since_last_ping,ms_until_down,missed_beats- signal metadata (
signal_message,signal_ip,signal_method,signal_user_agent)
Operational notes
- Heartbeat ingestion enforces per-monitor API rate limiting (
60requests per minute). failsignals mark monitor state down immediately.- A later
successsignal clears failure state and returns monitor to up.