Skip to content
NBS Workshop

Confidence Levels

Overview

Every root cause identified during synthesis receives a confidence rating based on how many independent evidence sources corroborate it. This rating drives how prominently the cause appears in the final report and whether it requires human review.

The IOT Manager assigns confidence during its synthesis step — no other agent assigns or overrides confidence ratings.

The Three Levels

HIGH — 3+ sources agree

Strong convergence across data trends, maintenance records, manual documentation, and/or historical cases. Treated as the probable root cause and presented prominently in the report.

Colour: Green (#16a34a)

MEDIUM — 2 sources agree

Plausible cause with partial corroboration. Included in the report with caveats noting which evidence is missing. Engineering team should validate before acting.

Colour: Amber (#f59e0b)

LOW — 1 source or conflicting evidence

Speculative — mentioned only if no higher-rated causes exist. Flagged for human review with a prominent “Requires Validation” marker.

Colour: Gray (#6b7280)

Evidence Sources

Confidence is built from the outputs of four specialist agents, each providing an independent line of evidence.

SourceAgentWhat It ProvidesExample Evidence
Sensor TrendsData AgentStatistical analysis of parameter behaviour over timeUV intensity declining at -0.53 W/m² per hour across 3 sessions
Maintenance StatusPMS AgentWork order history, overdue components, part replacements13 of 16 UV lamps exceed 9,000-hour rated life
Manual CausesManual AgentDocumented causes for the alarm code from equipment manualAlarm W152: primary cause is “UV lamp aging” (p.228)
Past IncidentsCasefile AgentHistorical incidents on same equipment or alarm codeSimilar decline on MV Atlantic Runner resolved by lamp replacement

How Evidence Feeds Into Confidence

graph TB
    subgraph "Phase 1 — Independent Evidence"
        DA["Data Agent\nSensor Trends\nDecline rate, anomaly timestamps,\nparameter correlations"]
        PMS["PMS Agent\nMaintenance Status\nOverdue components, service gaps,\npart replacement history"]
        MA["Manual Agent\nAlarm Causes\nDocumented causes for alarm code,\nmanual section references"]
        CF["Casefile Agent\nPast Incidents\nHistorical matches, fleet patterns,\nprior resolutions"]
    end


    subgraph "Synthesis"
        MATRIX["Cross-Reference Matrix\nRow = candidate cause\nColumn = evidence source"]
        COUNT["Count Corroborating\nSources per Cause"]
        ASSIGN["Assign Confidence\n3+ = HIGH\n2 = MEDIUM\n1 = LOW"]
    end


    DA --> MATRIX
    PMS --> MATRIX
    MA --> MATRIX
    CF --> MATRIX
    MATRIX --> COUNT --> ASSIGN


    ASSIGN --> HIGH["HIGH Causes\nProminent in report"]
    ASSIGN --> MED["MEDIUM Causes\nIncluded with caveats"]
    ASSIGN --> LOW["LOW Causes\nFlagged for human review"]


    style DA fill:#0d1b2a,stroke:#1b9aaa,stroke-width:2px,color:#fff
    style PMS fill:#0d1b2a,stroke:#1b9aaa,stroke-width:2px,color:#fff
    style MA fill:#0d1b2a,stroke:#1b9aaa,stroke-width:2px,color:#fff
    style CF fill:#0d1b2a,stroke:#1b9aaa,stroke-width:2px,color:#fff
    style MATRIX fill:#1a1a2e,stroke:#e94560,stroke-width:2px,color:#fff
    style COUNT fill:#1a1a2e,stroke:#e94560,stroke-width:2px,color:#fff
    style ASSIGN fill:#1a1a2e,stroke:#e94560,stroke-width:3px,color:#fff
    style HIGH fill:#16a34a,stroke:#15803d,stroke-width:2px,color:#fff
    style MED fill:#f59e0b,stroke:#d97706,stroke-width:2px,color:#000
    style LOW fill:#6b7280,stroke:#4b5563,stroke-width:2px,color:#fff

Worked Example

This example illustrates how the IOT Manager assigns HIGH confidence to a root cause using evidence from three independent agents.

  1. Alert Received

    Alert ai_abc123 fires: UV intensity on BWTS-UV-003 has dropped below the 25 W/m² compliance threshold. Current reading: 18.2 W/m².

  2. Data Agent Findings

    Reports a steady decline of -0.53 W/m² per hour across 3 operating sessions. The decline accelerates as lamp hours exceed rated life. Strong positive correlation between lamp_hours and intensity drop.

    Evidence: Quantitative trend confirms degradation pattern.

  3. PMS Agent Findings

    Reports that 13 of 16 UV lamps have exceeded their rated 9,000-hour operational life. No lamp replacement work orders have been raised in the last 6 months. Last full lamp replacement was 14 months ago.

    Evidence: Maintenance gap confirms lamps are overdue for replacement.

  4. Manual Agent Findings

    Finds alarm code W152 in the PureBallast 3.1 alarm list (p.228). The primary documented cause is “UV lamp aging or failure”. The troubleshooting section (p.275) confirms that gradual intensity decline is the expected symptom of lamp degradation.

    Evidence: Manual documentation matches the observed alarm and symptom pattern.

  5. Casefile Agent Findings

    Finds no prior incidents for BWTS-UV-003 but identifies a similar case on BWTS-UV-001 aboard MV Atlantic Runner, where the same alarm was resolved by full lamp replacement.

    Evidence: Historical precedent supports, but on different equipment — treated as supporting rather than primary evidence.

  6. Synthesis — Confidence Assignment

    Candidate CauseData AgentPMS AgentManual AgentCasefile AgentCount
    UV lamp degradationDecline matches degradation pattern13/16 lamps past rated lifeW152 lists lamp aging as primary causeSimilar case resolved by replacement4
    Sensor calibration driftDecline too consistent for driftNo calibration work orders overdueNot listed for W152No prior calibration incidents0
    Quartz sleeve foulingNo fouling signature in dataNo sleeve cleaning overdueListed as secondary cause for W152No prior fouling cases1

    Result: “UV lamp degradation” receives HIGH confidence (4 corroborating sources). “Quartz sleeve fouling” receives LOW confidence (1 source, secondary). “Sensor calibration drift” is excluded (0 sources).

Handling Edge Cases

When no candidate cause reaches MEDIUM or HIGH confidence, the Manager marks the synthesis as INCONCLUSIVE. The report is still generated but includes a prominent “Requires Manual Review” banner. The Manager also writes a note to Para Memory for the weekly synthesis review.