An on-call agent for your own systems

By the time you're paged, the evidence is already in Slack.

Rootsight investigates every alert the moment it fires, read-only: metrics, logs, dependencies, infrastructure, recent deploys and their diff. It names the cause, or hands the incident to a person when it doesn't recognise it, and posts the evidence to Slack. It changes nothing in your systems.

Read-only everywhere16 known failure typesAWS · Google Cloud · Azure · Kubernetes · ComposeGitHub · Bitbucket · JenkinsAny LLM, or none

See it work

The same incident, two ways

A real incident recorded on our test stack: a deploy broke the sentiment service. Press Replay to watch who does what.

warningHighErrorRate · ai-appHTTP 5xx error rate > 5%fired 21:09:25 UTC
Stepall steps

On-call engineer, alone

10 of 10 steps by a person
  1. Gets paged
    phone, 21:09 UTC
  2. Opens a laptop and the VPN
  3. Finds the right Grafana dashboard
    ai-app · errors
  4. Reads the error rate
    5xx climbing past 5%
  5. Works out app errors vs downstream
    greps logs
  6. Checks what was deployed
    git log / CI history
  7. Opens the latest diff
    f102047, app/server.js
  8. Decides it was the deploy
  9. Writes it up in Slack
    #incidents
  10. Rolls back
    a person, as always

With Rootsight

5 by the agent · 2 by a person
  1. Reads metrics
    errors 40.3%, all app 5xx (none downstream), p95 93 ms, traffic x1.0
  2. Reads deploy history
    1 in 24h: f102047 "refactor: lexicon-based sentiment classifier", right before onset
  3. Reads the diff
    app/server.js, 16 lines: new classify(); LEXICON[w].valueOf() throws on unknown words
  4. Names the cause
    bad code deploy; fix: roll back f102047
  5. Posts to Slack and tags on-call
    evidence attached, with Acknowledge and Check again
  6. On-call reads the page
    cause, evidence and fix in one message
  7. Rolls back
    a person decides; Rootsight never changes anything
Rootsight APP #incidents
HighErrorRate on ai-app: bad code deploy. 40.3% of requests failing, all from the app itself. Deploy f102047 "refactor: lexicon-based sentiment classifier" went out right before it; its diff changes app/server.js. Suggested fix: roll back f102047. @oncall
AcknowledgeCheck again

Evidence lines are the agent's own readings from that incident. The person's steps are the usual manual path for the same alert; nothing here claims how many minutes each takes.

What changes for the team

An SRE plus Rootsight, against an SRE alone

SRE alone
  • Gets paged, then starts gathering evidence from scratch
  • Knows the dashboards and queries by memory
  • Each incident's lessons live in one person's head
  • Infrastructure advice depends on who is looking
SRE + rootsight
  • Gets paged with the evidence and a likely cause already in Slack
  • Reviews and decides; never has to hunt for the first clue
  • Every reviewed incident trains the next model
  • Design suggestions come from measurements, with every option
What stays with people
  • Every change to your systems: it is read-only
  • Approving infrastructure changes, with the admin password
  • Incidents it doesn't recognise: it hands them over and says why
  • The final call on every verdict

What does what

Nine parts, one job each

Pick a part to see what it reads, what it does, what it never does, and where you see it.

alert → investigator → novelty guard → explanation → Slack → your review → retraining

Measured, not promised

How well it diagnoses

950 simulated incidents it never trained on: 800 from the 16 known failure types, and 150 from failure types it has never seen, which it should hand to a person.

Hand-written rulesthe usual runbook logic
86.9%
RL agent (DQN) + guardmean of 3 training seeds
90.5%
Supervised cascade + guardthe best so far
94.4%

Share of the 800 known-failure incidents given the right cause.

It starts before you do

The investigation runs when the alert fires, not when someone opens a laptop. The page in Slack arrives with evidence attached.

Attention goes where it's needed

Incidents it recognises come with a cause and a fix to review. Ones it doesn't are handed over with the reason, so nobody trusts a guess.

Nothing to undo

Every check is a read. Changes to infrastructure are drafts until approved with the admin password, and they leave as a pull request.

Knowledge stays

Every reviewed incident and every fix confirmed by checking again becomes training data. A new model replaces the old one only if it scores at least as well.

No secrets kept

Tokens live in your CI's secret store or this machine's keychain, never in a file. They're redacted from logs, Slack and anything sent to an LLM.

Set up

Install it from anywhere

One npm command installs Rootsight wherever Node and Python run; then run setup in your repository. Setup reads your code and monitoring first, asks only what it can't find, and writes nothing until you agree.

npm install -g rootsight
rootsight init
  1. Look around. Where you run, your services, your CI, your Prometheus and Loki.
  2. Ask. Features, Slack, the LLM, the admin password; only what it couldn't find.
  3. Fit to your stack. Writes queries for the metric and log conventions you actually use.
  4. Review. Every file, package, download and secret, before anything happens.
  5. Set up. Progress bars with time left; files written whole or not at all.

Where it stands today

  • Trained on simulated incidents. On 5 real incidents recorded on our test stack and kept aside, the model in use handed all 5 to a person: none wrong, none solved.
  • Cloud discovery is tested against stand-ins shaped like the AWS, Google Cloud and Azure APIs, not real accounts.
  • The fact check verifies numbers, names and sources in explanations; it can't judge statements like "within the normal range".
Rootsight · read-only on-call agentNumbers from the project's test split; see the README for the full table.

Command palette

Jump to a page or run an action