Auto-load detection library from S1 API, improve coverage map accuracy

- Fetch detection library rules from platform-rules API at startup (falls
  back to extracted.json); adds Sync Detection Library button for refresh
- Parser column simplified to ✓ Parsed / ✗ Not Parsed
- Detection counts now use library rules only (exclude custom STAR rules)
- Add close-match suggestions for dataSource.name mismatches (e.g. CloudTrail
  → AWS CloudTrail, Microsoft 365 Collaboration → Microsoft O365)
- Exclude SentinelOne Ranger AD from coverage map (native S1 source)
- Add success feedback banners to Load SDL Parsers and Sync Library buttons
- Remove rule_counts.json manual override; extracted.json is source of truth
- Remove Load Detections button; rules auto-import on backend startup
- Add get_account_id() and get_platform_rules() to s1_client

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Mick
2026-05-20 15:14:10 -04:00
parent 6e137438b1
commit 6cd9da82da
8 changed files with 580 additions and 90 deletions
+35 -1
View File
@@ -1,6 +1,6 @@
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from db import engine, Base
from db import engine, Base, get_db, ParsedRule
from routers import coverage, ingest, settings, quality
Base.metadata.create_all(bind=engine)
@@ -15,6 +15,40 @@ with engine.connect() as _conn:
app = FastAPI(title="SIEM Toolkit", version="1.0.0")
@app.on_event("startup")
async def auto_load_detections():
"""
Auto-load detection library rules on startup.
Tries the live S1 API first (accurate 'sources' field); falls back to extracted.json.
Skips if rules are already loaded — use the 'Sync Library' button to force a refresh.
"""
import os
from sqlalchemy.orm import Session
from services import s1_client
db: Session = next(get_db())
try:
existing = db.query(ParsedRule).filter_by(rule_type="library").count()
if existing > 0:
return # Already loaded — skip until user manually refreshes
# Try live API first
try:
rules = await s1_client.get_platform_rules()
if rules:
coverage._import_from_api_rules(db, rules)
return
except Exception:
pass
# Fall back to local file
detections_file = os.environ.get("DETECTIONS_FILE", "/app/data/detections.json")
if os.path.exists(detections_file):
coverage._import_detections(db, detections_file)
finally:
db.close()
app.add_middleware(
CORSMiddleware,
allow_origins=["http://localhost:3001"],