Mirrors/marcredhat-siem-toolkit-patched
1
0
Fork 0
mirror of https://github.com/marcredhat/SIEM-toolkit-patched synced 2026-06-08 20:37:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Ingest Dashboard: optional background cache pre-warmer

Browse Source 
Adds an asyncio background task that re-runs the heavy Ingest Dashboard
queries every ~4 min (just under the 5 min TTL) so the in-process cache
is always populated. First user hit on any dashboard widget then returns
from cache (single-digit ms) instead of waiting 30-60s for SDL.

Components:
  - backend/services/prewarmer.py: standalone module, opt-in via
    INGEST_PREWARM=1; configurable windows via INGEST_PREWARM_HOURS /
    INGEST_PREWARM_DAYS / INGEST_PREWARM_DAILY_VOLUME_DAYS and interval
    via INGEST_PREWARM_INTERVAL_SECONDS. Logs through the uvicorn logger
    so cycles are visible in 'docker logs'.
  - backend/main.py: spawn the task on FastAPI startup.
  - docker-compose.yml: forward INGEST_PREWARM* env vars to the
    backend service (default off).

Observed on a busy tenant with INGEST_PREWARM=1, default windows:
  top-sources?days=7 first hit after restart: ~39s -> ~8ms (cache warm).

Defaults to disabled (INGEST_PREWARM=0) so existing users see no
behaviour change.
This commit is contained in:
marc
2026-05-22 20:41:36 +02:00
parent bfff0eeec0
commit 7d19c57a5d
3 changed files with 110 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/main.py
+8
View File
@@ -45,6 +45,14 @@ with engine.connect() as _conn:
app = FastAPI(title="SIEM Toolkit", version="1.0.0")
@app.on_event("startup")
async def start_ingest_prewarmer():
"""Start optional background pre-warmer for the Ingest Dashboard cache.
Opt-in via INGEST_PREWARM=1. See backend/services/prewarmer.py."""
from services import prewarmer
prewarmer.start_if_enabled()
@app.on_event("startup")
async def auto_load_detections():
"""