Telescope

Heartbeat

How Telescope receives heartbeat signals and detects missed or failed runs.

Heartbeat monitors are passive checks for scheduled jobs and background workers. Your system sends signals to a unique URL, and Telescope evaluates whether signals are on time, late, missed, or explicitly failed.

How it works

  1. Your job sends GET, POST, or HEAD requests to the monitor's heartbeat URL.
  2. The API validates the heartbeat ID, signal type, and per-monitor rate limit.
  3. The API records signal metadata in monitor config (latest ping/signal/fail timestamps and request metadata) and inserts an events row.
  4. The worker runs HeartbeatChecker, reads the latest config, and evaluates monitor state using expected interval plus grace period.

Supported signals

You can send signal state using a path segment (/:signal) or query (?signal= / ?state= / ?status=):

SignalMeaning
successSuccessful run/ping (default if omitted)
startRun started (informational)
failExplicit failure signal

Optional message text can be passed with ?msg= / ?message= or JSON body { "message": "..." }.

Status evaluation

HeartbeatChecker evaluates these states:

StateMonitor statusMeaning
waiting_for_first_pingUpMonitor has not received any ping yet
in_progressUpA start signal was received and is still within grace
on_timeUpLast success ping is within expected interval
in_graceUpPing is late but still inside grace period
missedDownNo success ping before grace deadline
failed_signalDownMost recent explicit signal is a failure

Configuration options

Stored in monitor config (camelCase or snake_case aliases accepted):

SettingDefaultDescription
expectedIntervalSecondsfalls back to monitor frequencyExpected time between successful pings
gracePeriodSeconds300Extra delay allowed before marking down

Result data highlights

Heartbeat checks write structured result_data including:

  • state
  • expected_interval_ms, grace_period_ms
  • last_ping_at, last_signal_at, last_fail_at, last_start_at, last_signal_state
  • expected_by, grace_deadline
  • ms_since_last_ping, ms_until_down, missed_beats
  • signal metadata (signal_message, signal_ip, signal_method, signal_user_agent)

Operational notes

  • Heartbeat ingestion enforces per-monitor API rate limiting (60 requests per minute).
  • fail signals mark monitor state down immediately.
  • A later success signal clears failure state and returns monitor to up.

On this page