Dashboard reloads on multi-day windows could take 30-60s and sometimes
returned HTTP 502 ("internal Scalyr error") when the SDL window was
expressed in days. Two-part fix:
1. In-process async TTL cache (services/async_cache.py)
- 5 min TTL on top-sources, by-event-type, daily-volume.
- Single-flight lock per cache key (no thundering herd).
- Optional ?nocache=1 query param to force a refresh.
- New endpoints: GET /api/ingest/cache-stats, DELETE /api/ingest/cache.
2. Normalise days -> hours upstream of the PowerQuery
- SDL is unstable on day-scale windows for large group-by counts on
busy tenants but stable on the equivalent hour-scale window.
- top-sources?days=1 used to 502; now works.
Observed timings on a busy tenant:
top-sources?days=7 cold ~55s -> warm ~13ms (~4300x)
top-sources?days=1 was 502 -> ~4ms (cold) / ~1.4ms (warm)
- s1_client: configurable PowerQuery timeout via SDL_PQ_TIMEOUT env var
(default 600s, was hardcoded 120s) with separate connect/read timeouts
via httpx.Timeout; retry on ReadTimeout via SDL_PQ_TIMEOUT_RETRIES;
better error messages include query snippet and parse non-JSON responses
- ingest: fix simulate-filter SDL syntax (== → =, drop leading | on base
expression, surface PowerQuery error field, cleaner empty-filter fallback)
- docker-compose: pass SDL_PQ_TIMEOUT and SDL_PQ_TIMEOUT_RETRIES through
to backend container with sensible defaults
Not taken from PR #2:
- .gitignore parsers/* change — would untrack the 7 committed parser files
- s1_client/quality/coverage changes already present in main from prior work
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Coverage map: replace filename fuzzy-match with exact dataSource.name
lookup read directly from parser file attributes; grok/dottedJson parsers
now flagged as "parser_needed" with format type shown in the UI
- Bar chart: SVG linearGradient (light purple → deep violet) replaces flat fill
- Ingest dashboard: add 1h button (first option) backed by new backend
hours= query param on /api/ingest/top-sources; daily-volume chart shows
informational message when in 1h mode
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- daily-volume: run per-day PowerQueries in parallel with asyncio.gather
instead of sequentially with sleeps — 3 days now completes in ~16s vs 140s+
- Default view changed from 7d to 3d; day buttons updated to [3, 5, 7]
- igLoad: fire daily-volume and top-sources simultaneously with Promise.allSettled
so both panels load in parallel rather than one after the other
- Each panel shows "Querying data lake…" spinner while loading
- Each panel renders independently — one failure doesn't block the other
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
marc