Duvaxa

DevOps & Cloud · Alert noise reduction

QuietPager

Quieter nights for on-call.

Find the alerts that wake people for nothing, see what on-call really costs, and get the exact configuration change that fixes each one.

SRE and platform leadsEngineering managers with a tired rotaMSPs running 24x7 support
alert-tuner · example
$ alert-tuner review --pagerduty --days 30 --tz +05:30
240 pages from 6 alerts (56 a week)
200 pages (83%) came from alerts flagged as noise; about 83 of 95 on-call hours
AT003 HighCPU: pages, then clears by itself (145 pages)
AT002 DiskAlmostFull: flapping (40 pages)
AT006 APIErrorRate: fires together with PostgresTooManyConnections
$ alert-tuner review ... --fixes fixes.yml
Wrote fixes.yml
Real run on the built-in demo: 30 days of generated history for a typical platform team.
ReadsPagerDuty, Opsgenie, Prometheus
Finds10 kinds of noise
AccessRead-only
OutputReport and a fixes file

The problem

On-call teams are paged dozens of times a week, mostly by alerts that clear themselves, flap around a threshold or fire alongside another alert. People stop trusting the pager, sleep badly and eventually leave, and nobody can say which alerts to fix first.

What it does

01

Reads a month of history from PagerDuty, Opsgenie or Prometheus, or a CSV or JSON export from any paging tool

02

Finds ten kinds of noise: self-resolving, flapping, ignored, always-firing, duplicate, storming, slow-to-acknowledge, low-severity and night-heavy alerts

03

Shows the loudest alerts, who carried the pages and how many came at night or at weekends

04

Turns noise into on-call hours, so the cost is a number a manager can act on

05

Writes the fixes: for: and keep_firing_for on Prometheus rules, Alertmanager inhibit rules, grouping and routes

06

Lints Prometheus rule files and PrometheusRule objects before they page anyone

07

Night and weekend in your own time zone; filter by service or team

How it works

  1. Connect PagerDuty, Opsgenie or Prometheus, or export a month of alerts
  2. Review the noise report and the fixes file with the team
  3. Apply the changes and re-run a month later to see pages fall

Works with

PagerDutyOpsgeniePrometheusAlertmanagerThanosMimirGrafana CloudCSV / JSON export

Runs in your CI

GitHub Actions workflow step
# Check alerting rules on every change
alert-tuner lint rules/
# Monthly on-call review
alert-tuner review --pagerduty --days 30 --format html --out review.html
Licensed customers get the action or command-line tool installed in their own organisation; it runs on your runners, so your code and plans never leave them. The command-line tools run in GitLab CI the same way.

Your data

  • Read-only API keys; it never changes routing or silences anything.
  • Suggested changes are written to a file for you to review and apply.
  • Runs on your machine or in CI.

Who it's for

SRE and platform leadsEngineering managers with a tired rotaMSPs running 24x7 support

Questions

What counts as noise?

Alerts that clear before anyone can act, flap, are rarely acknowledged, always fire, fire alongside another alert, storm across many instances, or page at warning level.

How do we prove it worked?

Run it again a month later: pages per week, night pages and the share of noise should all fall.

Related

Often used with QuietPager

See QuietPager on your own data

A 30-minute call. We run the product on your export, repository or document and walk you through the results.

Book a demo