release: v1.0.5

release: v1.0.4
2026-06-13 10:21:21 +00:00 · 2026-05-06 14:49:45 +02:00 · 2026-04-22 16:53:20 +02:00 · 2026-04-22 15:49:51 +02:00 · 2026-04-15 12:22:05 +02:00 · 2026-04-14 22:21:06 +02:00
61 changed files with 1361 additions and 874 deletions
@@ -22,7 +22,7 @@ jobs:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
-        run: pip install pytest pytest-asyncio aiohttp pydantic colorama rich fpdf2 dnspython phonenumbers stem cloudscraper
+        run: pip install -r requirements.txt pytest pytest-asyncio
      - name: Run tests
        run: python -m pytest tests/ -v
@@ -8,6 +8,8 @@ on:
 jobs:
  release:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v4
@@ -18,7 +20,7 @@ jobs:
          python-version: "3.11"
      - name: Install dependencies
-        run: pip install pytest pytest-asyncio aiohttp pydantic colorama rich fpdf2 dnspython phonenumbers stem cloudscraper
+        run: pip install -r requirements.txt pytest pytest-asyncio
      - name: Run tests
        run: python -m pytest tests/ -v
@@ -28,6 +30,9 @@ jobs:
          sudo apt-get install -y ruby ruby-dev build-essential
          sudo gem install fpm
      - name: Build source plugins
        run: python build_sources.py
      - name: Build .deb
        run: bash build_deb.sh
@@ -39,6 +39,3 @@ Thumbs.db
 # Env files
 .env
 .env.*
 # Internal development notes — not for distribution
 IMPROVEMENTS.md
@@ -2,6 +2,142 @@
 All notable changes to NOX are documented here.
 ## [1.0.5] — 2026-05-06
 ### Sources
 - **Fixed:** `pypi_user` — endpoint was querying the PyPI package API (`/pypi/{target}/json`) instead of user profiles, returning package metadata for any username that matched a package name and 404 for all others. Source now uses the PyPI XML-RPC `user_packages` method, returning the actual list of packages owned by the target username.
 ### Engine
 - **Fixed:** `Detect.qtype` — dotted-quad strings with out-of-range octets (e.g. `255.255.255.256`) were classified as `phone` instead of `username`. Invalid IPs now fall through to `username`.
 - **Fixed:** `Record.dedup_key` — records with no identity fields (email, username, password) from different sources all produced the same SHA-256 hash, causing all but the first to be silently dropped from the DB. Source name is now included in the key when identity fields are absent.
 - **Added:** `raw_payload` support in `NoxSourceProvider` — POST sources can now send raw string bodies (e.g. XML-RPC) in addition to JSON and form-encoded payloads.
 - **Fixed:** `FileSystemProvider._by_regex` — regex-extracted values that are not email addresses (e.g. package names, usernames) are now correctly placed in the `username` field instead of `email`.
 ## [1.0.4] — 2026-04-22
 ### Engine
 - **Fixed:** `_build_ssl_context` — custom TLS context had zero CA certificates loaded, causing `SSLCertVerificationError` on all HTTPS connections. CA bundle now loaded via `certifi` with `load_default_certs()` fallback.
 - **Fixed:** `_NOISE_RE` in reporting — `ssl.`, `aiohttp.`, `asyncio.` were substring-matched, silently zeroing legitimate emails and domains (e.g. `user@ssl.example.com`). Patterns now anchored to start-of-string or whitespace.
 - **Fixed:** `COMBO_RE` in `ScrapeEngine` — multiline greedy match produced credentials with embedded newlines in email/password fields. Pattern now excludes newlines from both capture groups.
 - **Fixed:** `_in_flight` dict in `AvalancheScanner` — entries were never removed after processing, causing unbounded memory growth on deep scans. Entry is now popped in the `finally` block after the future resolves.
 - **Fixed:** `DorkingEngine.__init__` — `ProxyManager.get_proxies()` was called eagerly on every `Orchestrator` instantiation, triggering a proxy fetch and OPSEC warning even for local-only commands (`--crack`, `--list-sources`, `--analyze`). Proxy fetch is now lazy.
 - **Fixed:** `ScrapeEngine._fetch_content` — IntelX paste content fetch used `DB.get_key("intelx")` which does not read `apikeys.json`. Replaced with `Vault.get("INTELX_API_KEY")` for consistent key resolution.
 - **Fixed:** `AsyncSource._rec` and `RiskEngine.score` — plugin `confidence` values were stored in `NoxSourceProvider._confidence` but never transferred to `Record.source_confidence`. All 124 plugin records received a flat `0.5` confidence regardless of their declared value. `_rec()` now injects `self._confidence` into the record; `RiskEngine.score()` uses `record.source_confidence` as fallback when the source is not in `_SRC_CONFIDENCE`.
 - **Fixed:** `ConfigManager.get_key` — `None` results were cached, preventing env vars set after the first lookup from being detected in the same session. Only positive values are now cached.
 - **Fixed:** Recursive Avalanche Engine — identifiers extracted from paste content (`paste["patterns"]`) were not being harvested as pivot seeds. All `scrape_res["pastes"]` pattern matches are now fed into `_extract_ids_from_text`.
 ### Sources
 - **Fixed:** `circl_hashlookup` — endpoint hardcoded to `/lookup/md5/{target}` but `input_type=hash` accepts SHA1/SHA256. SHA1 and SHA256 backup endpoints added; engine now routes each hash type to the correct path.
 - **Updated:** `crt_sh` — `reliability_score` lowered from 5 to 3, `is_volatile=true` added to reflect documented intermittent availability.
 ## [1.0.3] — 2026-04-15
 ### Engine
 - **Fixed:** Recursive Avalanche Engine — identifiers extracted from paste content (`paste["patterns"]`) were not being harvested as pivot seeds. Bare emails and other identifiers found in IntelX paste bodies that lacked a `:password` separator were silently dropped from the pivot queue. All `scrape_res["pastes"]` pattern matches are now fed into `_extract_ids_from_text` and reinjected consistently with `credentials`, `telegram`, and `dork_misconfigs`.
 ## [1.0.2] — 2026-04-14
 ### Sources
 - **Fixed:** `misp_search` — `MISP_URL` added to `api_key_slots` so the instance base URL is resolved at runtime; `health_check_url` corrected from unresolvable placeholder to `https://misp.local`
 - **Fixed:** `threatconnect_search` — removed unresolvable `{TC_SIGNATURE}` HMAC placeholder from the `Authorization` header; `reliability_score` lowered to `2`, `is_volatile` set to `true`
 - **Fixed:** `spycloud_breach` — endpoint corrected from `breach/data/emails` to `breach/catalog/emails` (standard breach lookup tier)
 - **Fixed:** `duckduckgo_api` — primary instance updated to `search.sapti.me`; 5 backup SearXNG instances added to `backup_endpoints` (now consumed by the engine)
 - **Fixed:** `gravatar` — endpoint now MD5-hashes the email before URL substitution via new `query_transform: md5_lower` field; raw email was returning 404 on every query
 - **Replaced:** `bgpview_ip` → `ripestat_ip` (RIPE Stat prefix-overview API) — BGPView free API decommissioned January 2025; RIPE Stat is free, keyless, and stable (`reliability_score: 5`)
 - **Fixed:** `twitter_v2` — marked `is_volatile=true`, `confidence` lowered to `0.1`; free-tier bearer tokens receive HTTP 403 since February 2024
 - **Fixed:** `fofa_info` — `qbase64` parameter now receives `base64(domain="<target>")` via `query_transform: fofa_domain`; raw domain was producing malformed queries
 - **Fixed:** `pipl_search` — Pipl shut down public REST API in Q3 2024; `reliability_score` lowered to `2`, `confidence` to `0.3`, `is_volatile=true`
 - **Fixed:** `spyonweb` — API confirmed unreachable; `reliability_score` lowered to `1`, `confidence` to `0.1`, `is_volatile=true`
 - **Fixed:** `hudsonrock_osint` — `is_volatile=true`; `rate_limit` raised from `5.0` to `30.0` to respect Cavalier API throttling (~10 req/hour free tier)
 - **Fixed:** `mailboxlayer`, `numverify`, `ipstack`, `ipinfodb` — endpoints and `health_check_url` migrated from `http://` to `https://`; API keys were being transmitted in cleartext before the server-side redirect
 - **Added:** `xposedornot` plugin (free, public breach analytics)
 - **Added:** `MISP_URL` to service registry and `apikeys.json` — back-filled automatically on first run after upgrade
 - Source count: 123 → 124
 ### Config
 - **Fixed:** Duplicate `xposedornot` entry removed from `SERVICE_REGISTRY` in `config_handler.py`
 ### Engine
 - **Fixed:** `_parse_retry_after` helper added — `int()` on an HTTP-date `Retry-After` header raised `ValueError`, causing the retry loop to abort as a hard failure; all 5 call sites in `_get`, `_post`, `Session.get`, and `Session.post` updated
 - **Fixed:** `_random_headers` — `Sec-CH-UA` Client Hints were emitted even when a Firefox UA was passed via the `extra` override; guard now evaluates the final `User-Agent` after overrides are applied
 - **Fixed:** `HashEngine._hashmob` — Hashmob API v2 changed request field from `"hash"` to `"hashes"` (array) and response schema from `{found, result}` to `{data: [{plaintext}]}`
 - **Fixed:** `DeHashEngine` — both `_lookup` and the sync fallback were calling the deprecated `/search` (v1) endpoint; updated to `/v2/search`
 - **Fixed:** `DorkEngine.run` — results were labelled with the requested engine name (`google`/`bing`/`ddg`) instead of `SearXNG` which is the actual backend; the 3× request multiplication (one pass per engine name, all hitting the same SearXNG pool) is eliminated
 - **Fixed:** `DB.close()` — background event loop was stopped but never closed, leaving the loop object open on process exit
 - **Fixed:** `NoxSourceProvider._fetch` — `backup_endpoints` defined in source plugins were parsed but never consumed; primary endpoint failure now falls through to backups in order
 - **Fixed:** `_local_crack_sync_blocking` — `hashlib.md5/sha1` now called with `usedforsecurity=False` to prevent hard crash on FIPS-enabled systems (RHEL 9, hardened Kali); Python 3.8 compat guard included
 ### Codebase
 - All internal tracking comments replaced with clean prose throughout `nox.py`, `build_sources.py`, and all helper modules
 ### Build
 - `BUILD_DATE` updated to `2026-04-14`
 - `pyproject.toml` version bumped to `1.0.2`; `requests` minimum pin aligned to `>=2.32.3`
 ## [1.0.1] — 2026-04-13
 ### Sources
 - **Removed:** `cit0day` (HTML fingerprint challenge, no JSON response), `vigilante_pw` (redirects to dehashed.com), `scylla_sh_search` (domain parked, permanently unreachable)
 - **Restored:** `proxynova_comb` (live, returns valid JSON — was incorrectly removed)
 - **Fixed:** `leak_lookup` now requires API key (`LEAK_LOOKUP_API_KEY`) — provider removed unauthenticated access
 - **Fixed:** `intelx_search` two-phase poll implemented — plugin previously submitted the search job but never polled for results, returning 0 records for all queries
 - **Fixed:** `hudsonrock_osint` missing `User-Agent` header added — endpoint returns 403 without a browser UA
 - **Removed:** `HASHES_API_KEY` registry entry — hashes.org shut down in 2023; `HASHES_COM_API_KEY` is the correct active slot
 - **Added:** `LEAK_LOOKUP_API_KEY` to service registry
 ### Engine
 - **Fixed:** `bypass_required` field in source plugins now enforced at runtime — sources declaring `["cloudflare"]` bypass are skipped when `curl_cffi` is absent (previously the field was parsed but never read)
 - **Fixed:** Guardian proxy auto-fetch updated to ProxyScrape v3 API (v2 deprecated Q1 2026); `proxy-list.download` replaced with `proxifly` free list
 ### Dependencies
 - **Updated:** `requests>=2.32.3` (CVE fixes)
 ### README
 - Source count updated: 123 active plugins
 ### Dependencies
 - **Updated:** `aiohttp` minimum pin raised to `>=3.13.5` (connection-pool stability fixes under high concurrency)
 - **Added:** `zstandard>=0.23.0` — enables native zstd decompression in aiohttp for Cloudflare/Fastly CDN responses
 ### Engine
 - **Updated:** `Accept-Encoding` header now includes `zstd` (`gzip, deflate, br, zstd`) to match Chrome 124+ behaviour
 ## [1.0.1] — 2026-04-11
 ### Sources
 - **Added 9 new sources:** `proxynova_comb` (COMB breach search, free), `shodan_internetdb` (IP intel, free), `circl_hashlookup` (NSRL hash lookup, free), `ipapi_is` (IP geolocation, free), `threatfox` (abuse.ch IOC database), `urlhaus` (abuse.ch malware URLs), `malwarebazaar` (abuse.ch hash lookup), `fullhunt_subdomains` (attack surface), `netlas_search` (internet scanner)
 - **Removed 7 dead sources:** `threatcrowd_email`, `threatcrowd_domain` (DNS dead), `spyse_domain`, `spyse_ip` (API shut down), `hashes_org` (DNS dead), `leakstats_pw` (DNS dead), `checkleaked` (endpoint gone)
 - **Fixed:** `dehashed` endpoint migrated from `/search` to `/v2/search`
 - **Fixed:** `hudsonrock_osint` endpoint corrected to `search-by-email` with `input_type: email` (was `search-by-login` with `input_type: username`)
 - **Fixed:** `scylla_sh_search` migrated from dead `scylla.sh` to active `scylla.so`
 - **Fixed:** `emailrep_io` now requires API key (`EMAILREP_API_KEY`) — free unauthenticated tier removed by provider
 - **Fixed:** `duckduckgo_api` repurposed from dead DDG Instant Answer API to SearXNG JSON search
 ### Engine
 - **Fixed:** POST 429 `Retry-After` cap was 4s (should be 30s, matching GET path)
 - **Fixed:** Linear retry backoff replaced with exponential backoff + jitter in all 4 retry paths (`_get`, `_post`, `Session.get`, `Session.post`)
 - **Fixed:** `--reset-sources` now removes orphaned plugins from `~/.nox/sources/` in addition to copying new ones
 - **Fixed:** DDG HTML scraper replaced with SearXNG JSON API across all call sites — DDG HTML endpoint bot-blocked since 2025
 - **Fixed:** SearXNG instance pool updated: `searx.be` (403), `search.bus-hit.me` (DNS dead), `searxng.site` (SSL error) replaced with 6 active instances; pool extracted to module-level `_SEARX_INSTANCES` constant
 - **Fixed:** All 11 dead paste site APIs removed from `ScrapeEngine.PASTE_SITES`; paste intelligence now routed through SearXNG dorks and IntelX
 ### WAF Resilience
 - **Updated:** User-Agent pool updated to Chrome/135, Firefox/136, Edge/135 (was Chrome/131, Firefox/133)
 - **Added:** `Sec-CH-UA`, `Sec-CH-UA-Mobile`, `Sec-CH-UA-Platform` Client Hints headers for Chromium-based UAs
 - **Fixed:** `_CH_UA_MAP` ordering — Edge UA now correctly gets `"Microsoft Edge"` brand (was getting `"Google Chrome"` due to dict iteration order)
 - **Fixed:** `_search()` sync method no longer passes `use_cloudscraper=True` to SearXNG JSON API calls
 ### Hash Cracking
 - **Removed 6 dead/paywalled cracker APIs:** nitrxgen (DNS dead), hash.help (DNS dead), hashkiller (403), hashes.com free path (404), md5decrypt (403), cmd5 (paywalled — returns `CMD5-ERROR:-1` for all hashes)
 - **Added:** Local rockyou wordlist as primary crack path (no external calls, no rate limits, no data leakage)
 - **Added:** `hashes.com` keyed API as external fallback (`HASHES_COM_API_KEY`)
 ### Dependencies
 - **Added:** `brotli>=1.1.0` — required for aiohttp to decompress `br`-encoded responses
 ### Config
 - **Added 7 new API key slots:** `EMAILREP_API_KEY`, `HASHES_COM_API_KEY`, `THREATFOX_API_KEY`, `URLHAUS_API_KEY`, `MALWAREBAZAAR_API_KEY`, `FULLHUNT_API_KEY`, `NETLAS_API_KEY`
 ## [1.0.0] — 2026-04-02
 ### Initial Release
@@ -11,10 +11,11 @@
 **Cyber Threat Intelligence Framework**
-[![Status](https://img.shields.io/badge/Status-v1.0.0-success)](https://github.com/nox-project/nox-framework/releases/tag/v1.0.0)
+[![Status](https://img.shields.io/badge/Status-v1.0.5-success)](https://github.com/nox-project/nox-framework/releases/tag/v1.0.5)
 [![Python](https://img.shields.io/badge/Python-3.8%2B-blue?logo=python&logoColor=white)](https://www.python.org/)
 [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE.txt)
 [![Kali Linux](https://img.shields.io/badge/Kali%20Linux-Ready-557C94?logo=kalilinux&logoColor=white)](https://www.kali.org/)
 [![BlackArch](https://img.shields.io/badge/BlackArch-Available-1E1E2E?logo=archlinux&logoColor=white)](https://blackarch.org/)
 [![Platform](https://img.shields.io/badge/Platform-Linux%20%7C%20macOS%20%7C%20Windows-lightgrey)](https://github.com/nox-project/nox-framework)
 [![Sources](https://img.shields.io/badge/Sources-124-red)](https://github.com/nox-project/nox-framework)
@@ -44,7 +45,7 @@ NOX is a purpose-built cyber threat intelligence engine designed for operators w
 |-|-|
 | **124 JSON Plugin Sources** | Every intelligence source is a JSON plugin. The execution engine contains zero hardcoded source logic. |
 | **Async Core** | Full `asyncio` event loop with JA3 fingerprinting, SSL session management, per-request jitter, and configurable concurrency. |
-| **Autoscan Pipeline** | `--autoscan` triggers: breach scan → recursive pivot → Google/Bing/DDG dorking → paste/Telegram scraping — all in one command. |
+| **Autoscan Pipeline** | `--autoscan` triggers: breach scan → recursive pivot → Google/Bing/SearXNG dorking → paste/Telegram scraping — all in one command. |
 | **Recursive Avalanche Engine** | Every identifier discovered — from breach records, dork hits, or scraped paste/Telegram content — is re-injected as a new seed. Per-asset pipeline is sequential (breach → crack → dork → scrape); child assets run concurrently via `asyncio.gather`. A global `seen_assets` set prevents infinite loops. Concurrency and depth are fully configurable at runtime via `--threads` and `--depth`. |
 | **Hash Pivoting** | Hashes found in breach data are automatically identified (MD5/SHA1/SHA256/NTLM/bcrypt) and cracked via concurrent background API queries. Cracked plaintexts are injected into the pivot queue as password-recycling seeds. Failures are logged silently — the scan never stops. |
 | **Guardian Proxy Engine** | Zero-config OPSEC layer: reads `proxies.txt` if present; otherwise auto-fetches and validates a high-anonymity proxy pool in-memory. Full SOCKS5/HTTP/S and Tor support. |
@@ -52,7 +53,7 @@ NOX is a purpose-built cyber threat intelligence engine designed for operators w
 | **Identity Graphing** | Union-Find correlation engine unifies breach records into identity clusters across all sources, using type-aware pivot classification. |
 | **Enterprise Forensic Reports** | Professional PDF/HTML/JSON/CSV/Markdown reports with Executive Summary dashboard (Total Time, Nodes Discovered, Cleartext Passwords, Pivot Depth), interactive Pivot Chain Visualization, and strict data sanitization — no technical noise in output. JSON exports are self-describing with a full metadata block. |
 | **HVT Detection** | Auto-flags C-level, Admin, DevOps, and government domain accounts as High-Value Targets. |
-| **Dorking Engine** | Passive document discovery via Google/Bing/DDG dorks with PDF/Office metadata extraction. |
+| **Dorking Engine** | Passive document discovery via Google/Bing/SearXNG dorks with PDF/Office metadata extraction. |
 | **Scraping Engine** | Paste site indexing, Telegram CTI channel monitoring, credential extraction, and misconfiguration discovery. Each autoscan asset gets a dedicated scrape session — no shared state. |
 | **Proxy / Tor** | SOCKS5, HTTP/S proxy, full Tor routing via `stem`, and automatic Guardian fallback. SOCKS5 proxies are validated and routed correctly via `aiohttp-socks`. |
 | **Secure Key Store** | API keys managed via `~/.config/nox-cli/apikeys.json` (chmod 0600). Unconfigured keys are silently skipped. Keys set via environment variable are picked up automatically without restarting. |
@@ -114,7 +115,7 @@ For each asset (seed + every discovered identifier):
  │     → password-recycling breach scan
  │
  ├─ Phase 3 — Dorking
-  │     Google/Bing/DDG dorks → leaked docs, .env files, SQL dumps
+  │     Google/Bing/SearXNG dorks → leaked docs, .env files, SQL dumps
  │     → new identifiers extracted and re-injected
  │
  └─ Phase 4 — Scraping
@@ -373,7 +374,7 @@ usage: nox-cli [-h] [-t TARGET] [-i] [--version]
  --fullscan              Breach + pivot only (no dork/scrape)
  --no-pivot              Disable recursive pivot enrichment
  --depth N               Avalanche pivot depth (default: 2)
-  --dork TARGET           Google/Bing/DDG dorking for leaked documents
+  --dork TARGET           Google/Bing/SearXNG dorking for leaked documents
  --scrape TARGET         Paste site + Telegram scraping
  --crack HASH            Identify and crack a hash
  --no-online-crack       Local wordlist only — no data sent to third-party APIs
@@ -406,7 +407,7 @@ Command        Description
 -----------    ---------------------------------------------------------------
 autoscan       Full pipeline: breach + pivot + dork + scrape
 scan           Breach intelligence scan only
-dork           Google/Bing/DDG dorking for leaked documents
+dork           Google/Bing/SearXNG dorking for leaked documents
 scrape         Paste site + Telegram scraping
 crack          Identify and crack a hash
 analyze        Deep password strength analysis
@@ -1,10 +1,10 @@
 #!/usr/bin/env bash
 set -e
-# NOX v1.0.0 — .deb build script (FPM)
+# NOX v1.0.5 — .deb build script (FPM)
 # Requires: fpm  →  gem install fpm
-VERSION="1.0.0"
+VERSION="1.0.5"
 PKG_NAME="nox-cli"
 ARCH="all"
 OUT_DIR="dist"
@@ -53,6 +53,8 @@ class SourceConfig(BaseModel):
    rate_limit:       float                    = 1.0
    headers:          Dict[str, str]           = Field(default_factory=dict)
    payload_template: Optional[Dict[str, Any]] = None
    raw_payload:      Optional[str]            = None
    regex_pattern:    Optional[str]            = None
    api_key_slots:    List[str]                = Field(default_factory=list)
    # ── Typing & pivoting ───────────────────────────────────────────────────
@@ -69,8 +71,13 @@ class SourceConfig(BaseModel):
    bypass_required:   Optional[List[str]]     = None          # omitted when empty
    user_agent_type:   Optional[str]           = None          # omitted when absent
    backup_endpoints:  List[str]               = Field(default_factory=list)
    # H2: optional confidence override — when set, takes precedence over formula
    confidence:        Optional[float]         = None
    query_transform:   Optional[str]           = None          # e.g. "md5_lower"
    # Two-phase poll support (e.g. IntelX: POST → job_id → GET results)
    poll_endpoint:     Optional[str]           = None
    poll_id_field:     Optional[str]           = None
    poll_id_param:     Optional[str]           = None
    poll_json_root:    Optional[str]           = None
    @field_validator("reliability_score")
    @classmethod
@@ -81,12 +88,10 @@ class SourceConfig(BaseModel):
    @model_validator(mode="after")
    def _validate_source(self) -> "SourceConfig":
        # H1: GET endpoints must contain {target} placeholder
        if self.method.upper() == "GET" and "{target}" not in self.endpoint:
            raise ValueError(
                f"'{self.name}': GET endpoint must contain {{target}} placeholder: {self.endpoint!r}"
            )
        # L3: volatile sources must have reliability_score ≤ 4 (was > 3, now > 4)
        if self.is_volatile and self.reliability_score > 4:
            raise ValueError(
                f"'{self.name}': is_volatile sources must have reliability_score ≤ 4"
@@ -95,11 +100,10 @@ class SourceConfig(BaseModel):
    def to_json(self) -> str:
        data = self.model_dump(exclude_none=True)
-        # Drop is_volatile / bypass_required / user_agent_type when falsy
+        # Drop falsy optional fields
-        for key in ("is_volatile", "bypass_required", "user_agent_type"):
+        for key in ("is_volatile", "bypass_required", "user_agent_type", "query_transform"):
            if not data.get(key):
                data.pop(key, None)
        # H2: use explicit confidence if set, otherwise derive from reliability_score
        data["confidence"] = (
            round(self.confidence, 2)
            if self.confidence is not None
@@ -119,6 +123,8 @@ def _mk(
    rate_limit: float = 1.0,
    headers: Optional[Dict[str, str]] = None,
    payload_template: Optional[Dict[str, Any]] = None,
    raw_payload: Optional[str] = None,
    regex_pattern: Optional[str] = None,
    api_key_slots: Optional[List[str]] = None,
    input_type: InputType = "any",
    output_type: Optional[List[str]] = None,
@@ -131,6 +137,12 @@ def _mk(
    bypass_required: Optional[List[str]] = None,
    user_agent_type: Optional[str] = None,
    backup_endpoints: Optional[List[str]] = None,
    poll_endpoint: Optional[str] = None,
    poll_id_field: Optional[str] = None,
    poll_id_param: Optional[str] = None,
    poll_json_root: Optional[str] = None,
    confidence: Optional[float] = None,
    query_transform: Optional[str] = None,
 ) -> SourceConfig:
    return SourceConfig(
        name=name, category=category, endpoint=endpoint, method=method,
@@ -138,6 +150,8 @@ def _mk(
        rate_limit=rate_limit,
        headers=headers or {},
        payload_template=payload_template,
        raw_payload=raw_payload,
        regex_pattern=regex_pattern,
        api_key_slots=api_key_slots or [],
        input_type=input_type,
        output_type=output_type or [],
@@ -150,6 +164,12 @@ def _mk(
        bypass_required=bypass_required or None,
        user_agent_type=user_agent_type,
        backup_endpoints=backup_endpoints or [],
        poll_endpoint=poll_endpoint,
        poll_id_field=poll_id_field,
        poll_id_param=poll_id_param,
        poll_json_root=poll_json_root,
        confidence=confidence,
        query_transform=query_transform or None,
    )
@@ -174,7 +194,7 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          input_type="domain", output_type=["domain"],
          normalization_map={"name_value": "domain"},
          tags=["passive", "fast"],
-          health_check_url="https://crt.sh", reliability_score=5),
+          health_check_url="https://crt.sh", reliability_score=3, is_volatile=True),
    _base("hackertarget_hostsearch", "dns_recon",
          "https://api.hackertarget.com/hostsearch/?q={target}", "GET",
@@ -240,24 +260,6 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          tags=["passive"],
          health_check_url="https://urlscan.io", reliability_score=5),
    _base("threatcrowd_email", "threat_intel",
          "https://www.threatcrowd.org/searchApi/v2/email/report/?email={target}", "GET",
          {"domains": "$.domains"},
          rate_limit=5.0,
          input_type="email", output_type=["domain"],
          tags=["passive", "threat"],
          health_check_url="https://www.threatcrowd.org", reliability_score=3,
          is_volatile=True, bypass_required=["cloudflare"], user_agent_type="browser"),
    _base("threatcrowd_domain", "threat_intel",
          "https://www.threatcrowd.org/searchApi/v2/domain/report/?domain={target}", "GET",
          {"ips": "$.resolutions[*].ip_address"},
          rate_limit=5.0,
          input_type="domain", output_type=["ip"],
          tags=["passive", "threat"],
          health_check_url="https://www.threatcrowd.org", reliability_score=3,
          is_volatile=True, bypass_required=["cloudflare"], user_agent_type="browser"),
    _base("pulsedive", "threat_intel",
          "https://pulsedive.com/api/info.php?indicator={target}", "GET",
          {"risk": "$.risk", "threats": "$.threats"},
@@ -266,13 +268,27 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          tags=["passive", "threat"],
          health_check_url="https://pulsedive.com", reliability_score=4),
    _base("xposedornot", "breach_data",
          "https://api.xposedornot.com/v1/breach-analytics?email={target}", "GET",
          {"breaches": "$.ExposedBreaches.breaches_details"},
          rate_limit=2.0,
          headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36"},
          input_type="email", output_type=["email", "domain"],
          normalization_map={"breach": "breach_name", "domain": "domain",
                             "xposed_date": "breach_date", "xposed_data": "data_types",
                             "password_risk": "password_risk"},
          tags=["passive", "stealth"],
          health_check_url="https://api.xposedornot.com", reliability_score=4, confidence=0.75),
    _base("hudsonrock_osint", "breach_data",
-          "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-login?username={target}", "GET",
+          "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-email?email={target}", "GET",
          {"stealers": "$.stealers"},
-          input_type="username", output_type=["email", "domain"],
+          rate_limit=30.0,
          headers={"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36"},
          input_type="email", output_type=["email", "domain", "username"],
          normalization_map={"stealers": "breach_record"},
          tags=["passive", "stealth"],
-          health_check_url="https://cavalier.hudsonrock.com", reliability_score=4),
+          health_check_url="https://cavalier.hudsonrock.com", reliability_score=3, is_volatile=True),
    _base("ipinfo_io", "geolocation",
          "https://ipinfo.io/{target}/json", "GET",
@@ -291,17 +307,20 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          tags=["passive", "fast"],
          health_check_url="https://ipapi.co", reliability_score=4),
-    _base("bgpview_ip", "network",
+    _base("ripestat_ip", "network",
-          "https://api.bgpview.io/ip/{target}", "GET",
+          "https://stat.ripe.net/data/prefix-overview/data.json?resource={target}", "GET",
-          {"prefixes": "$.data.prefixes[*].prefix"},
+          {"asns": "$.data.asns[*].asn", "holder": "$.data.asns[0].holder"},
          input_type="ip", output_type=["ip"],
          normalization_map={"asn": "asn_number", "holder": "asn_org"},
          tags=["passive", "infrastructure"],
-          health_check_url="https://api.bgpview.io", reliability_score=4),
+          health_check_url="https://stat.ripe.net", reliability_score=5),
-    _base("emailrep_io", "email_rep",
+    _auth("emailrep_io", "email_rep",
          "https://emailrep.io/{target}", "GET",
          {"reputation": "$.reputation"},
          rate_limit=2.0,
          headers={"Key": "{EMAILREP_API_KEY}"},
          api_key_slots=["{EMAILREP_API_KEY}"],
          input_type="email", output_type=["email"],
          normalization_map={"reputation": "email_reputation"},
          tags=["passive", "fast"],
@@ -328,6 +347,7 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          {"name": "$.entry[0].displayName"},
          rate_limit=2.0,
          input_type="email", output_type=["username"],
          query_transform="md5_lower",
          tags=["passive"],
          health_check_url="https://www.gravatar.com", reliability_score=4),
@@ -402,11 +422,15 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          health_check_url="https://ipvigilante.com", reliability_score=3, is_volatile=True),
    _base("pypi_user", "social",
-          "https://pypi.org/pypi/{target}/json", "GET",
+          "https://pypi.org/pypi", "POST",
-          {"info": "$.info"},
+          {},
          headers={"Content-Type": "text/xml"},
          input_type="username", output_type=["username"],
          normalization_map={},
          regex_pattern=r"<string>Owner</string></value>\s*<value><string>([^<]+)</string>",
          tags=["passive"],
-          health_check_url="https://pypi.org", reliability_score=5),
+          health_check_url="https://pypi.org", reliability_score=4,
          raw_payload="<?xml version='1.0'?><methodCall><methodName>user_packages</methodName><params><param><value>{target}</value></param></params></methodCall>"),
    _base("npm_user", "social",
          "https://registry.npmjs.org/-/v1/search?text=maintainer:{target}", "GET",
@@ -446,11 +470,19 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          health_check_url="https://checkurl.phishtank.com", reliability_score=4),
    _base("duckduckgo_api", "search",
-          "https://api.duckduckgo.com/?q={target}&format=json", "GET",
+          "https://search.sapti.me/search?q={target}&format=json&categories=general", "GET",
-          {"abstract": "$.Abstract"},
+          {"results": "$.results"},
          input_type="any", output_type=["url"],
          normalization_map={"url": "url", "title": "title"},
          tags=["passive", "fast"],
-          health_check_url="https://api.duckduckgo.com", reliability_score=5),
+          backup_endpoints=[
              "https://searx.tiekoetter.com/search?q={target}&format=json&categories=general",
              "https://searx.perennialte.ch/search?q={target}&format=json&categories=general",
              "https://search.mdosch.de/search?q={target}&format=json&categories=general",
              "https://paulgo.io/search?q={target}&format=json&categories=general",
              "https://priv.au/search?q={target}&format=json&categories=general",
          ],
          health_check_url="https://search.sapti.me", reliability_score=3, is_volatile=True),
    _base("cve_search", "vulns",
          "https://cve.circl.lu/api/cve/{target}", "GET",
@@ -474,28 +506,45 @@ FREE_PUBLIC_SOURCES: List[SourceConfig] = [
          tags=["passive"],
          health_check_url="https://packetstormsecurity.com", reliability_score=4),
-    _base("checkleaked", "breaches",
+    _base("proxynova_comb", "breaches",
-          "https://api.checkleaked.cc/check/{target}", "GET",
+          "https://api.proxynova.com/comb?query={target}", "GET",
-          {"found": "$.found"},
+          {"lines": "$.lines"},
          input_type="email", output_type=["email"],
          normalization_map={"lines": "credential_line"},
          tags=["passive", "stealth"],
-          health_check_url="https://api.checkleaked.cc", reliability_score=2, is_volatile=True,
+          health_check_url="https://api.proxynova.com", reliability_score=3, is_volatile=True),
          backup_endpoints=["https://checkleaked.cc/api/check/{target}"]),
    _base("scylla_sh_search", "breaches",
          "https://scylla.sh/search?q={target}", "GET",
          {"results": "$.*"},
          input_type="email", output_type=["email", "domain"],
          tags=["passive", "stealth"],
          health_check_url="https://scylla.sh", reliability_score=2, is_volatile=True,
          backup_endpoints=["https://scylla.sh/api/search?q={target}"]),
-    _base("vigilante_pw", "breaches",
+    _base("shodan_internetdb", "scanners",
-          "https://vigilante.pw/api/search?q={target}", "GET",
+          "https://internetdb.shodan.io/{target}", "GET",
-          {"results": "$.results"},
+          {"hostnames": "$.hostnames", "ports": "$.ports", "vulns": "$.vulns"},
-          input_type="email", output_type=["email"],
+          input_type="ip", output_type=["domain", "ip"],
-          tags=["passive", "stealth"],
+          normalization_map={"hostnames": "domain", "vulns": "cve"},
-          health_check_url="https://vigilante.pw", reliability_score=2, is_volatile=True),
+          tags=["passive", "fast", "infrastructure"],
          health_check_url="https://internetdb.shodan.io",
          reliability_score=5),
    _base("circl_hashlookup", "hashes",
          "https://hashlookup.circl.lu/lookup/md5/{target}", "GET",
          {"filename": "$.FileName", "known_malicious": "$.KnownMalicious"},
          input_type="hash", output_type=["hash"],
          normalization_map={"FileName": "filename", "MD5": "hash_md5"},
          tags=["passive", "fast"],
          health_check_url="https://hashlookup.circl.lu",
          reliability_score=5,
          backup_endpoints=[
              "https://hashlookup.circl.lu/lookup/sha1/{target}",
              "https://hashlookup.circl.lu/lookup/sha256/{target}",
          ]),
    _base("ipapi_is", "geolocation",
          "https://api.ipapi.is/?q={target}", "GET",
          {"org": "$.org", "asn": "$.asn.asn", "abuse": "$.abuse.email"},
          input_type="ip", output_type=["domain"],
          normalization_map={"org": "asn_org", "asn": "asn_number", "abuse": "abuse_contact"},
          tags=["passive", "fast"],
          health_check_url="https://api.ipapi.is",
          reliability_score=4),
 ]
 # ---------------------------------------------------------------------------
@@ -583,27 +632,10 @@ AUTHENTICATED_PREMIUM_SOURCES: List[SourceConfig] = [
          {"results": "$.results"},
          api_key_slots=["{FOFA_API_KEY}", "{FOFA_EMAIL}"],
          input_type="domain", output_type=["ip", "domain"],
          query_transform="fofa_domain",
          tags=["passive", "infrastructure"],
          health_check_url="https://fofa.info", reliability_score=4),
    _auth("spyse_domain", "scanners",
          "https://api.spyse.com/v1/domain/details/{target}", "GET",
          {"asn": "$.data.asn"},
          headers={"Authorization": "Bearer {SPYSE_API_KEY}"},
          api_key_slots=["{SPYSE_API_KEY}"],
          input_type="domain", output_type=["ip"],
          tags=["passive"],
          health_check_url="https://api.spyse.com", reliability_score=3),
    _auth("spyse_ip", "scanners",
          "https://api.spyse.com/v1/ip/details/{target}", "GET",
          {"geo": "$.data.geo"},
          headers={"Authorization": "Bearer {SPYSE_API_KEY}"},
          api_key_slots=["{SPYSE_API_KEY}"],
          input_type="ip", output_type=["ip"],
          tags=["passive"],
          health_check_url="https://api.spyse.com", reliability_score=3),
    _auth("onyphe_datascan", "scanners",
          "https://www.onyphe.io/api/v2/simple/datascan/{target}", "GET",
          {"results": "$.results"},
@@ -736,11 +768,11 @@ AUTHENTICATED_PREMIUM_SOURCES: List[SourceConfig] = [
    _auth("threatconnect_search", "threat_intel",
          "https://api.threatconnect.com/v2/indicators/{target}", "GET",
          {"data": "$.data"},
-          headers={"Authorization": "TC {TC_API_KEY}:{TC_SIGNATURE}"},
+          headers={"Authorization": "TC {TC_API_KEY}"},
          api_key_slots=["{TC_API_KEY}"],
          input_type="any", output_type=["ip", "domain"],
          tags=["passive", "threat"],
-          health_check_url="https://api.threatconnect.com", reliability_score=4),
+          health_check_url="https://api.threatconnect.com", reliability_score=2, is_volatile=True),
    _auth("threatportal", "threat_intel",
          "https://threatportal.io/api/v1/search?q={target}", "GET",
@@ -801,11 +833,11 @@ AUTHENTICATED_PREMIUM_SOURCES: List[SourceConfig] = [
          "{MISP_URL}/attributes/restSearch", "POST",
          {"attributes": "$.Attribute[*].value"},
          headers={"Authorization": "{MISP_API_KEY}", "Content-Type": "application/json"},
-          api_key_slots=["{MISP_API_KEY}"],
+          api_key_slots=["{MISP_API_KEY}", "{MISP_URL}"],
          input_type="any", output_type=["ip", "domain", "hash"],
          payload_template={"returnFormat": "json", "value": "{target}"},
          tags=["passive", "threat"],
-          health_check_url="{MISP_URL}", reliability_score=4),
+          health_check_url="https://misp.local", reliability_score=4),
 ]
 AUTHENTICATED_PREMIUM_SOURCES += [
@@ -824,7 +856,7 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          health_check_url="https://haveibeenpwned.com", reliability_score=5),
    _auth("dehashed", "breaches",
-          "https://api.dehashed.com/search?query={target}", "GET",
+          "https://api.dehashed.com/v2/search?query={target}", "GET",
          {"entries": "$.entries"},
          headers={"Authorization": "Basic {DEHASHED_AUTH_BASE64}", "Accept": "application/json"},
          api_key_slots=["{DEHASHED_AUTH_BASE64}"],
@@ -856,6 +888,10 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          payload_template={"term": "{target}", "buckets": [], "lookuplevel": 0,
                            "maxresults": 100, "timeout": 0, "datefrom": "", "dateto": "",
                            "sort": 4, "media": 0, "terminate": []},
          poll_endpoint="https://2.intelx.io/intelligent/search/result",
          poll_id_field="id",
          poll_id_param="id",
          poll_json_root="records",
          tags=["passive", "stealth"],
          health_check_url="https://2.intelx.io", reliability_score=5),
@@ -879,7 +915,7 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          health_check_url="https://leakcheck.io", reliability_score=4),
    _auth("spycloud_breach", "breaches",
-          "https://api.spycloud.io/enterprise-v2/breach/data/emails/{target}", "GET",
+          "https://api.spycloud.io/enterprise-v2/breach/catalog/emails/{target}", "GET",
          {"results": "$.results"},
          headers={"X-API-Key": "{SPYCLOUD_API_KEY}"},
          api_key_slots=["{SPYCLOUD_API_KEY}"],
@@ -915,41 +951,28 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          tags=["passive", "stealth"],
          health_check_url="https://api.breachaware.com", reliability_score=3, is_volatile=True),
-    _auth("tines_breach", "breaches",
+    _auth("flare_leaksdb", "breaches",
-          "https://api.tines.com/breaches/{target}", "GET",
+          "https://api.flare.io/leaksdb/v2/credentials/email/{target}", "GET",
-          {"breaches": "$.breaches"},
+          {"items": "$.items"},
-          headers={"Authorization": "Bearer {TINES_API_KEY}"},
+          headers={"Authorization": "Bearer {FLARE_API_KEY}"},
-          api_key_slots=["{TINES_API_KEY}"],
+          api_key_slots=["{FLARE_API_KEY}"],
-          input_type="email", output_type=["email"],
+          input_type="email", output_type=["email", "username"],
-          tags=["passive"],
+          normalization_map={"email": "email_address", "username": "username",
-          health_check_url="https://api.tines.com", reliability_score=3),
+                             "password": "plaintext_password", "hash": "password_hash",
                             "source": "breach_name"},
          tags=["passive", "stealth"],
          health_check_url="https://api.flare.io", reliability_score=4),
-    _auth("leakstats_pw", "breaches",
+    _auth("leak_lookup", "breaches",
          "https://leakstats.net/api/password/{target}", "GET",
          {"count": "$.count"},
          headers={"api-key": "{LEAKSTATS_API_KEY}"},
          api_key_slots=["{LEAKSTATS_API_KEY}"],
          input_type="hash", output_type=["hash"],
          tags=["passive"],
          health_check_url="https://leakstats.net", reliability_score=3, is_volatile=True),
    _base("leak_lookup", "breaches",
          "https://leak-lookup.com/api/search", "POST",
          {"results": "$.message"},
          headers={"X-API-Key": "{LEAK_LOOKUP_API_KEY}"},
          api_key_slots=["{LEAK_LOOKUP_API_KEY}"],
          input_type="email", output_type=["email"],
          payload_template={"query": "{target}", "type": "email_address"},
          tags=["passive", "stealth"],
          health_check_url="https://leak-lookup.com", reliability_score=3, is_volatile=True),
    _auth("cit0day", "breaches",
          "https://cit0day.in/api/v1/search?query={target}", "GET",
          {"results": "$.results"},
          headers={"Authorization": "Bearer {CIT0DAY_API_KEY}"},
          api_key_slots=["{CIT0DAY_API_KEY}"],
          input_type="email", output_type=["email"],
          tags=["passive", "stealth"],
          health_check_url="https://cit0day.in", reliability_score=2, is_volatile=True),
    # ── DNS Recon ─────────────────────────────────────────────────────────────
    _auth("securitytrails_sub", "dns_recon",
@@ -1003,7 +1026,8 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          api_key_slots=["{SPYONWEB_API_KEY}"],
          input_type="domain", output_type=["domain"],
          tags=["passive"],
-          health_check_url="https://api.spyonweb.com", reliability_score=3),
+          health_check_url="https://api.spyonweb.com", reliability_score=1,
          is_volatile=True, confidence=0.1),
    # ── WHOIS ─────────────────────────────────────────────────────────────────
@@ -1053,15 +1077,17 @@ AUTHENTICATED_PREMIUM_SOURCES += [
    # ── Enrichment ────────────────────────────────────────────────────────────
-    _auth("clearbit_enrich", "enrichment",
+    _auth("seon_email", "enrichment",
-          "https://person.clearbit.com/v2/people/find?email={target}", "GET",
+          "https://api.seon.io/SeonRestService/email-api/v3", "POST",
-          {"full_name": "$.name.fullName"},
+          {"data": "$.data"},
-          headers={"Authorization": "Bearer {CLEARBIT_API_KEY}"},
+          headers={"X-API-KEY": "{SEON_API_KEY}", "Content-Type": "application/json"},
-          api_key_slots=["{CLEARBIT_API_KEY}"],
+          api_key_slots=["{SEON_API_KEY}"],
-          input_type="email", output_type=["username", "domain"],
+          input_type="email", output_type=["email", "domain", "username"],
-          normalization_map={"fullName": "full_name"},
+          payload_template={"email": "{target}"},
          normalization_map={"email": "email_address", "domain": "domain",
                             "full_name": "full_name", "phone_number": "phone"},
          tags=["passive"],
-          health_check_url="https://person.clearbit.com", reliability_score=4),
+          health_check_url="https://api.seon.io", reliability_score=4),
    _auth("fullcontact", "enrichment",
          "https://api.fullcontact.com/v3/person.enrich", "POST",
@@ -1088,7 +1114,8 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          api_key_slots=["{PIPL_API_KEY}"],
          input_type="email", output_type=["username", "domain", "phone"],
          tags=["passive"],
-          health_check_url="https://api.pipl.com", reliability_score=4),
+          health_check_url="https://api.pipl.com", reliability_score=2,
          is_volatile=True, confidence=0.3),
    # ── Email Reputation ──────────────────────────────────────────────────────
@@ -1127,12 +1154,12 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          health_check_url="https://api.hunter.io", reliability_score=4),
    _auth("mailboxlayer", "email_rep",
-          "http://apilayer.net/api/check?access_key={MAILBOX_API_KEY}&email={target}", "GET",
+          "https://apilayer.net/api/check?access_key={MAILBOX_API_KEY}&email={target}", "GET",
          {"score": "$.score"},
          api_key_slots=["{MAILBOX_API_KEY}"],
          input_type="email", output_type=["email"],
          tags=["passive"],
-          health_check_url="http://apilayer.net", reliability_score=3),
+          health_check_url="https://apilayer.net", reliability_score=3),
    _auth("abstract_email", "email_rep",
          "https://emailvalidation.abstractapi.com/v1/?api_key={ABSTRACT_API_KEY}&email={target}", "GET",
@@ -1160,7 +1187,8 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          api_key_slots=["{TWITTER_BEARER_TOKEN}"],
          input_type="username", output_type=["username"],
          tags=["passive"],
-          health_check_url="https://api.twitter.com", reliability_score=4),
+          health_check_url="https://api.twitter.com", reliability_score=1,
          is_volatile=True, confidence=0.1),
    _auth("github_code_search", "code",
          "https://api.github.com/search/code?q={target}", "GET",
@@ -1183,13 +1211,13 @@ AUTHENTICATED_PREMIUM_SOURCES += [
    # ── Geolocation ───────────────────────────────────────────────────────────
    _auth("ipstack", "geolocation",
-          "http://api.ipstack.com/{target}?access_key={IPSTACK_API_KEY}", "GET",
+          "https://api.ipstack.com/{target}?access_key={IPSTACK_API_KEY}", "GET",
          {"country": "$.country_name"},
          api_key_slots=["{IPSTACK_API_KEY}"],
          input_type="ip", output_type=["ip"],
          normalization_map={"country_name": "geo_country"},
          tags=["passive", "fast"],
-          health_check_url="http://api.ipstack.com", reliability_score=4),
+          health_check_url="https://api.ipstack.com", reliability_score=4),
    _auth("ipgeolocation_io", "geolocation",
          "https://api.ipgeolocation.io/ipgeo?apiKey={IPGEO_API_KEY}&ip={target}", "GET",
@@ -1218,34 +1246,24 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          health_check_url="https://extreme-ip-lookup.com", reliability_score=3),
    _auth("ipinfodb", "geolocation",
-          "http://api.ipinfodb.com/v3/ip-city/?key={IPINFODB_API_KEY}&ip={target}&format=json", "GET",
+          "https://api.ipinfodb.com/v3/ip-city/?key={IPINFODB_API_KEY}&ip={target}&format=json", "GET",
          {"city": "$.cityName"},
          api_key_slots=["{IPINFODB_API_KEY}"],
          input_type="ip", output_type=["ip"],
          normalization_map={"cityName": "geo_city"},
          tags=["passive"],
-          health_check_url="http://api.ipinfodb.com", reliability_score=3),
+          health_check_url="https://api.ipinfodb.com", reliability_score=3),
    # ── Phone ─────────────────────────────────────────────────────────────────
    _auth("numverify", "phone",
-          "http://apilayer.net/api/validate?access_key={NUMVERIFY_API_KEY}&number={target}", "GET",
+          "https://apilayer.net/api/validate?access_key={NUMVERIFY_API_KEY}&number={target}", "GET",
          {"valid": "$.valid", "carrier": "$.carrier"},
          api_key_slots=["{NUMVERIFY_API_KEY}"],
          input_type="phone", output_type=["phone"],
          normalization_map={"valid": "phone_valid", "carrier": "phone_carrier"},
          tags=["passive"],
-          health_check_url="http://apilayer.net", reliability_score=4),
+          health_check_url="https://apilayer.net", reliability_score=4),
    # ── Hashes ────────────────────────────────────────────────────────────────
    _auth("hashes_org", "hashes",
          "https://hashes.org/api.php?key={HASHES_API_KEY}&query={target}", "GET",
          {"found": "$.results"},
          api_key_slots=["{HASHES_API_KEY}"],
          input_type="hash", output_type=["hash"],
          tags=["passive"],
          health_check_url="https://hashes.org", reliability_score=3),
    # ── Search ────────────────────────────────────────────────────────────────
@@ -1265,6 +1283,65 @@ AUTHENTICATED_PREMIUM_SOURCES += [
          input_type="any", output_type=["url"],
          tags=["passive"],
          health_check_url="https://api.bing.microsoft.com", reliability_score=5),
    _auth("threatfox", "threat_intel",
          "https://threatfox-api.abuse.ch/api/v1/", "POST",
          {"results": "$.data"},
          headers={"API-KEY": "{THREATFOX_API_KEY}", "Content-Type": "application/json"},
          payload_template={"query": "search_ioc", "search_term": "{target}"},
          api_key_slots=["{THREATFOX_API_KEY}"],
          input_type="any", output_type=["ip", "domain", "hash"],
          normalization_map={"ioc": "indicator", "malware": "malware_family"},
          tags=["passive", "threat"],
          health_check_url="https://threatfox-api.abuse.ch",
          reliability_score=5),
    _auth("urlhaus", "threat_intel",
          "https://urlhaus-api.abuse.ch/v1/host/", "POST",
          {"urls": "$.urls"},
          headers={"Auth-Key": "{URLHAUS_API_KEY}"},
          payload_template={"host": "{target}"},
          api_key_slots=["{URLHAUS_API_KEY}"],
          input_type="domain", output_type=["url", "domain"],
          normalization_map={"url": "malware_url", "threat": "threat_type"},
          tags=["passive", "threat"],
          health_check_url="https://urlhaus-api.abuse.ch",
          reliability_score=5),
    _auth("malwarebazaar", "hashes",
          "https://mb-api.abuse.ch/api/v1/", "POST",
          {"data": "$.data"},
          headers={"API-KEY": "{MALWAREBAZAAR_API_KEY}"},
          payload_template={"query": "get_info", "hash": "{target}"},
          api_key_slots=["{MALWAREBAZAAR_API_KEY}"],
          input_type="hash", output_type=["hash"],
          normalization_map={"file_name": "filename", "tags": "tags"},
          tags=["passive", "threat"],
          health_check_url="https://mb-api.abuse.ch",
          reliability_score=5),
    _auth("fullhunt_subdomains", "dns_recon",
          "https://fullhunt.io/api/v1/domain/{target}/subdomains", "GET",
          {"hosts": "$.hosts"},
          headers={"X-API-KEY": "{FULLHUNT_API_KEY}"},
          api_key_slots=["{FULLHUNT_API_KEY}"],
          input_type="domain", output_type=["domain", "ip"],
          normalization_map={"host": "domain"},
          tags=["passive", "infrastructure"],
          health_check_url="https://fullhunt.io",
          reliability_score=4),
    _auth("netlas_search", "scanners",
          "https://app.netlas.io/api/responses/?q={target}&source_type=include&start=0&fields=", "GET",
          {"items": "$.items"},
          headers={"X-API-Key": "{NETLAS_API_KEY}"},
          api_key_slots=["{NETLAS_API_KEY}"],
          input_type="ip", output_type=["ip", "domain"],
          normalization_map={"data.ip": "ip_address", "data.domain": "domain"},
          tags=["passive", "infrastructure"],
          health_check_url="https://app.netlas.io",
          reliability_score=4),
 ]
@@ -1273,9 +1350,9 @@ AUTHENTICATED_PREMIUM_SOURCES += [
 # ---------------------------------------------------------------------------
 def build_nox_sources(output_dir: str = None) -> None:
-    # H3: resolve output_dir relative to this script's location, not CWD.
+    # Resolve output_dir relative to this script's location so the command
-    # This ensures `python /opt/nox-cli/build_sources.py` from any directory
+    # `python /opt/nox-cli/build_sources.py` always writes to the correct
-    # always writes to /opt/nox-cli/sources/ instead of ./sources/.
+    # package sources/ directory regardless of the working directory.
    if output_dir is None:
        output_dir = str(Path(__file__).resolve().parent / "sources")
    os.makedirs(output_dir, exist_ok=True)
@@ -1,4 +1,4 @@
-.TH NOX\-CLI 1 "2026-03-30" "1.0.0" "NOX Framework"
+.TH NOX\-CLI 1 "2026-05-06" "1.0.5" "NOX Framework"
 .SH NAME
 nox-cli \- Advanced Asynchronous Cyber Threat Intelligence Framework
 .SH SYNOPSIS
@@ -4,18 +4,20 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "nox-cli"
-version = "1.0.0"
+version = "1.0.5"
 description = "Advanced Asynchronous Cyber Threat Intelligence Framework"
 readme = { file = "README.md", content-type = "text/markdown" }
 license = { text = "Apache-2.0" }
 authors = [{ name = "nox-project" }]
 requires-python = ">=3.8"
 dependencies = [
-    "aiohttp>=3.9.0",
+    "aiohttp>=3.13.5",
    "aiohttp-socks>=0.8.4",
    "aiosqlite>=0.20.0",
    "httpx[http2]>=0.27.0",
-    "requests>=2.31.0",
+    "brotli>=1.1.0",
    "zstandard>=0.23.0",
    "requests>=2.32.3",
    "certifi>=2024.2.2",
    "cloudscraper>=1.2.71",
    "beautifulsoup4>=4.12.3",
@@ -2,13 +2,15 @@
 # Python 3.8+  |  pip install -r requirements.txt
 # ── Core (Async) ───────────────────────────────────────────────────────
-aiohttp>=3.9.0
+aiohttp>=3.13.5
 aiohttp-socks>=0.8.4          # SOCKS4/5 proxy support for aiohttp
 aiosqlite>=0.20.0             # async SQLite (forensic persistence DB)
 httpx[http2]>=0.27.0          # Guardian Engine: dynamic proxy fetch + HTTP/2
 brotli>=1.1.0                 # brotli decompression for aiohttp br responses
 zstandard>=0.23.0             # zstd decompression for aiohttp zstd responses (Cloudflare/Fastly CDNs)
 # ── Intelligence & Scraping ────────────────────────────────────────────
-requests>=2.31.0
+requests>=2.32.3
 certifi>=2024.2.2             # up-to-date CA bundle for SSL verification
 cloudscraper>=1.2.71          # Cloudflare-protected endpoint bypass
 beautifulsoup4>=4.12.3
@@ -9,7 +9,7 @@ requirements = [
 setup(
    name="nox-cli",
-    version="1.0.0",
+    version="1.0.5",
    author="nox-project",
    description="Advanced Asynchronous Cyber Threat Intelligence Framework",
    long_description=Path("README.md").read_text(),
@@ -1,27 +0,0 @@
 {
    "name": "bgpview_ip",
    "category": "network",
    "endpoint": "https://api.bgpview.io/ip/{target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "prefixes": "$.data.prefixes[*].prefix"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "ip",
    "output_type": [
        "ip"
    ],
    "normalization_map": {},
    "tags": [
        "passive",
        "infrastructure"
    ],
    "health_check_url": "https://api.bgpview.io",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.85
 }
@@ -1,30 +0,0 @@
 {
    "name": "checkleaked",
    "category": "breaches",
    "endpoint": "https://api.checkleaked.cc/check/{target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "found": "$.found"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "email",
    "output_type": [
        "email"
    ],
    "normalization_map": {},
    "tags": [
        "passive",
        "stealth"
    ],
    "health_check_url": "https://api.checkleaked.cc",
    "expected_status": 200,
    "reliability_score": 2,
    "is_volatile": true,
    "backup_endpoints": [
        "https://checkleaked.cc/api/check/{target}"
    ],
    "confidence": 0.55
 }
@@ -0,0 +1,34 @@
 {
    "name": "circl_hashlookup",
    "category": "hashes",
    "endpoint": "https://hashlookup.circl.lu/lookup/md5/{target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "filename": "$.FileName",
        "known_malicious": "$.KnownMalicious"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "hash",
    "output_type": [
        "hash"
    ],
    "normalization_map": {
        "FileName": "filename",
        "MD5": "hash_md5"
    },
    "tags": [
        "passive",
        "fast"
    ],
    "health_check_url": "https://hashlookup.circl.lu",
    "expected_status": 200,
    "reliability_score": 5,
    "backup_endpoints": [
        "https://hashlookup.circl.lu/lookup/sha1/{target}",
        "https://hashlookup.circl.lu/lookup/sha256/{target}"
    ],
    "confidence": 1.0
 }
@@ -1,32 +0,0 @@
 {
    "name": "cit0day",
    "category": "breaches",
    "endpoint": "https://cit0day.in/api/v1/search?query={target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "results": "$.results"
    },
    "rate_limit": 1.0,
    "headers": {
        "Authorization": "Bearer {CIT0DAY_API_KEY}"
    },
    "api_key_slots": [
        "{CIT0DAY_API_KEY}"
    ],
    "input_type": "email",
    "output_type": [
        "email"
    ],
    "normalization_map": {},
    "tags": [
        "passive",
        "stealth"
    ],
    "health_check_url": "https://cit0day.in",
    "expected_status": 200,
    "reliability_score": 2,
    "is_volatile": true,
    "backup_endpoints": [],
    "confidence": 0.55
 }
@@ -1,33 +0,0 @@
 {
    "name": "clearbit_enrich",
    "category": "enrichment",
    "endpoint": "https://person.clearbit.com/v2/people/find?email={target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "full_name": "$.name.fullName"
    },
    "rate_limit": 1.0,
    "headers": {
        "Authorization": "Bearer {CLEARBIT_API_KEY}"
    },
    "api_key_slots": [
        "{CLEARBIT_API_KEY}"
    ],
    "input_type": "email",
    "output_type": [
        "username",
        "domain"
    ],
    "normalization_map": {
        "fullName": "full_name"
    },
    "tags": [
        "passive"
    ],
    "health_check_url": "https://person.clearbit.com",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.85
 }
@@ -25,7 +25,8 @@
    ],
    "health_check_url": "https://crt.sh",
    "expected_status": 200,
-    "reliability_score": 5,
+    "reliability_score": 3,
    "is_volatile": true,
    "backup_endpoints": [],
-    "confidence": 1.0
+    "confidence": 0.7
 }
@@ -1,7 +1,7 @@
 {
    "name": "dehashed",
    "category": "breaches",
-    "endpoint": "https://api.dehashed.com/search?query={target}",
+    "endpoint": "https://api.dehashed.com/v2/search?query={target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
@@ -1,11 +1,11 @@
 {
    "name": "duckduckgo_api",
    "category": "search",
-    "endpoint": "https://api.duckduckgo.com/?q={target}&format=json",
+    "endpoint": "https://search.sapti.me/search?q={target}&format=json&categories=general",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
-        "abstract": "$.Abstract"
+        "results": "$.results"
    },
    "rate_limit": 1.0,
    "headers": {},
@@ -14,14 +14,24 @@
    "output_type": [
        "url"
    ],
-    "normalization_map": {},
+    "normalization_map": {
        "url": "url",
        "title": "title"
    },
    "tags": [
        "passive",
        "fast"
    ],
-    "health_check_url": "https://api.duckduckgo.com",
+    "health_check_url": "https://search.sapti.me",
    "expected_status": 200,
-    "reliability_score": 5,
+    "reliability_score": 3,
-    "backup_endpoints": [],
+    "is_volatile": true,
-    "confidence": 1.0
+    "backup_endpoints": [
        "https://searx.tiekoetter.com/search?q={target}&format=json&categories=general",
        "https://searx.perennialte.ch/search?q={target}&format=json&categories=general",
        "https://search.mdosch.de/search?q={target}&format=json&categories=general",
        "https://paulgo.io/search?q={target}&format=json&categories=general",
        "https://priv.au/search?q={target}&format=json&categories=general"
    ],
    "confidence": 0.7
 }
@@ -3,13 +3,17 @@
    "category": "email_rep",
    "endpoint": "https://emailrep.io/{target}",
    "method": "GET",
-    "requires_auth": false,
+    "requires_auth": true,
    "selectors": {
        "reputation": "$.reputation"
    },
    "rate_limit": 2.0,
-    "headers": {},
+    "headers": {
-    "api_key_slots": [],
+        "Key": "{EMAILREP_API_KEY}"
    },
    "api_key_slots": [
        "{EMAILREP_API_KEY}"
    ],
    "input_type": "email",
    "output_type": [
        "email"
@@ -0,0 +1,38 @@
 {
    "name": "flare_leaksdb",
    "category": "breaches",
    "endpoint": "https://api.flare.io/leaksdb/v2/credentials/email/{target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "items": "$.items"
    },
    "rate_limit": 1.0,
    "headers": {
        "Authorization": "Bearer {FLARE_API_KEY}"
    },
    "api_key_slots": [
        "{FLARE_API_KEY}"
    ],
    "input_type": "email",
    "output_type": [
        "email",
        "username"
    ],
    "normalization_map": {
        "email": "email_address",
        "username": "username",
        "password": "plaintext_password",
        "hash": "password_hash",
        "source": "breach_name"
    },
    "tags": [
        "passive",
        "stealth"
    ],
    "health_check_url": "https://api.flare.io",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.85
 }
@@ -27,5 +27,6 @@
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "query_transform": "fofa_domain",
    "confidence": 0.85
 }
@@ -0,0 +1,34 @@
 {
    "name": "fullhunt_subdomains",
    "category": "dns_recon",
    "endpoint": "https://fullhunt.io/api/v1/domain/{target}/subdomains",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "hosts": "$.hosts"
    },
    "rate_limit": 1.0,
    "headers": {
        "X-API-KEY": "{FULLHUNT_API_KEY}"
    },
    "api_key_slots": [
        "{FULLHUNT_API_KEY}"
    ],
    "input_type": "domain",
    "output_type": [
        "domain",
        "ip"
    ],
    "normalization_map": {
        "host": "domain"
    },
    "tags": [
        "passive",
        "infrastructure"
    ],
    "health_check_url": "https://fullhunt.io",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.85
 }
@@ -22,5 +22,6 @@
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "query_transform": "md5_lower",
    "confidence": 0.85
 }
@@ -1,28 +0,0 @@
 {
    "name": "hashes_org",
    "category": "hashes",
    "endpoint": "https://hashes.org/api.php?key={HASHES_API_KEY}&query={target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "found": "$.results"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [
        "{HASHES_API_KEY}"
    ],
    "input_type": "hash",
    "output_type": [
        "hash"
    ],
    "normalization_map": {},
    "tags": [
        "passive"
    ],
    "health_check_url": "https://hashes.org",
    "expected_status": 200,
    "reliability_score": 3,
    "backup_endpoints": [],
    "confidence": 0.7
 }
@@ -29,13 +29,13 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "alienvault_otx_malware":  {"display": "AlienVault OTX (Malware)",   "public": True},
    "alienvault_otx_user":     {"display": "AlienVault OTX (User)",      "public": True},
    "anubis_subdomains":       {"display": "Anubis Subdomains",          "public": True},
-    "bgpview_ip":              {"display": "BGPView IP",                 "public": True},
+    "ripestat_ip":             {"display": "RIPE Stat IP",               "public": True},
-    "checkleaked":             {"display": "CheckLeaked",                "public": True},
+    "xposedornot":             {"display": "XposedOrNot",                  "public": True},
    "crt_sh":                  {"display": "crt.sh",                     "public": True},
    "cve_search":              {"display": "CVE Search",                 "public": True},
    "cxsecurity":              {"display": "CXSecurity",                 "public": True},
    "duckduckgo_api":          {"display": "Google / DDG Dorks",         "public": True},
-    "emailrep_io":             {"display": "EmailRep.io",                "public": True},
+    "emailrep_io":             {"display": "EmailRep.io",                "public": False},
    "github_users":            {"display": "GitHub Users",               "public": True},
    "gitlab_search":           {"display": "GitLab Search",              "public": True},
    "gravatar":                {"display": "Gravatar",                   "public": True},
@@ -44,7 +44,10 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "hackertarget_hostsearch": {"display": "HackerTarget Host Search",   "public": True},
    "hackertarget_reverseip":  {"display": "HackerTarget Reverse IP",    "public": True},
    "hackertarget_whois":      {"display": "WHOIS (HackerTarget)",       "public": True},
-    "hudsonrock_osint":        {"display": "HudsonRock OSINT",           "public": True},
+    "ipapi_is":                {"display": "ipapi.is",                      "public": True},
    "circl_hashlookup":        {"display": "CIRCL Hash Lookup",             "public": True},
    "proxynova_comb":          {"display": "ProxyNova COMB",                "public": True},
    "shodan_internetdb":       {"display": "Shodan InternetDB",             "public": True},
    "ipapi_co":                {"display": "ipapi.co",                   "public": True},
    "ipinfo_io":               {"display": "IPInfo.io",                  "public": True},
    "ipvigilante":             {"display": "IPVigilante",                "public": True},
@@ -59,14 +62,10 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "reddit_user":             {"display": "Reddit User",                "public": True},
    "robtex_ip":               {"display": "Robtex IP",                  "public": True},
    "scamwatcher":             {"display": "ScamWatcher",                "public": True},
    "social_scan":             {"display": "Social Scan",                "public": True},
    "sublist3r_api":           {"display": "Sublist3r API",              "public": True},
    "threatcrowd_domain":      {"display": "ThreatCrowd (Domain)",       "public": True},
    "threatcrowd_email":       {"display": "ThreatCrowd (Email)",        "public": True},
    "threatminer_domain":      {"display": "ThreatMiner (Domain)",       "public": True},
    "threatminer_ip":          {"display": "ThreatMiner (IP)",           "public": True},
    "urlscan_search":          {"display": "URLScan.io",                 "public": True},
    "vigilante_pw":            {"display": "Vigilante.pw",               "public": True},
    "wayback_machine":         {"display": "Wayback Machine",            "public": True},
    # ── Private / key-required ────────────────────────────────────────
    "ABSTRACT_API_KEY":         {"display": "Abstract Email Validation", "public": False},
@@ -78,8 +77,7 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "BING_API_KEY":             {"display": "Bing Search API",           "public": False},
    "CENSYS_AUTH_BASE64":       {"display": "Censys",                    "public": False},
    "CIRCL_AUTH_BASE64":        {"display": "CIRCL.lu PDNS",             "public": False},
-    "CIT0DAY_API_KEY":          {"display": "Cit0day",                   "public": False},
+    "SEON_API_KEY":             {"display": "SEON Email Intelligence",   "public": False},
    "CLEARBIT_API_KEY":         {"display": "Clearbit Enrich",           "public": False},
    "CRIMINALIP_API_KEY":       {"display": "CriminalIP",                "public": False},
    "DEHASHED_AUTH_BASE64":     {"display": "Dehashed",                  "public": False},
    "DNSDB_API_KEY":            {"display": "DNSDB Passive DNS",         "public": False},
@@ -94,7 +92,6 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "GOOGLE_CX_KEY":            {"display": "Google Custom Search (API key)", "public": False},
    "GOOGLE_CX_ID":            {"display": "Google Custom Search (CX ID)",   "public": False},
    "GREYNOISE_API_KEY":        {"display": "GreyNoise",                 "public": False},
    "HASHES_API_KEY":           {"display": "Hashes.org",                "public": False},
    "HIBP_API_KEY":             {"display": "HaveIBeenPwned",            "public": False},
    "HIPPO_API_KEY":            {"display": "EmailHippo",                "public": False},
    "HUNTER_API_KEY":           {"display": "Hunter.io",                 "public": False},
@@ -109,7 +106,6 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "JOE_API_KEY":              {"display": "Joe Sandbox",               "public": False},
    "LEAKCHECK_API_KEY":        {"display": "LeakCheck",                 "public": False},
    "LEAKIX_API_KEY":           {"display": "LeakIX",                    "public": False},
    "LEAKSTATS_API_KEY":        {"display": "LeakStats.pw",              "public": False},
    "MAILBOX_API_KEY":          {"display": "Mailboxlayer",              "public": False},
    "MALSHARE_API_KEY":         {"display": "MalShare",                  "public": False},
    "METADEFENDER_API_KEY":     {"display": "MetaDefender",              "public": False},
@@ -125,9 +121,8 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "SNUSBASE_API_KEY":         {"display": "Snusbase",                  "public": False},
    "SPYCLOUD_API_KEY":         {"display": "SpyCloud",                  "public": False},
    "SPYONWEB_API_KEY":         {"display": "SpyOnWeb",                  "public": False},
    "SPYSE_API_KEY":            {"display": "Spyse",                     "public": False},
    "TC_API_KEY":               {"display": "ThreatConnect",             "public": False},
-    "TINES_API_KEY":            {"display": "Tines Breach",              "public": False},
+    "FLARE_API_KEY":            {"display": "Flare LeaksDB",              "public": False},
    "TP_API_KEY":               {"display": "ThreatPortal",              "public": False},
    "TWITTER_BEARER_TOKEN":     {"display": "Twitter / X API v2",        "public": False},
    "URLVOID_API_KEY":          {"display": "URLVoid",                   "public": False},
@@ -139,6 +134,15 @@ SERVICE_REGISTRY: Dict[str, Dict] = {
    "WHOXY_API_KEY":            {"display": "Whoxy WHOIS",               "public": False},
    "ZEROBOUNCE_API_KEY":       {"display": "ZeroBounce",                "public": False},
    "ZOOMEYE_API_KEY":          {"display": "ZoomEye",                   "public": False},
    "EMAILREP_API_KEY":         {"display": "EmailRep.io",               "public": False},
    "HASHES_COM_API_KEY":       {"display": "Hashes.com (crack API)",    "public": False},
    "THREATFOX_API_KEY":        {"display": "ThreatFox (abuse.ch)",      "public": False},
    "URLHAUS_API_KEY":          {"display": "URLhaus (abuse.ch)",        "public": False},
    "MALWAREBAZAAR_API_KEY":    {"display": "MalwareBazaar (abuse.ch)",  "public": False},
    "FULLHUNT_API_KEY":         {"display": "FullHunt (attack surface)", "public": False},
    "NETLAS_API_KEY":           {"display": "Netlas.io (internet scanner)", "public": False},
    "LEAK_LOOKUP_API_KEY":      {"display": "Leak-Lookup",               "public": False},
    "MISP_URL":                 {"display": "MISP Instance URL",         "public": False},
 }
 _PRIVATE_KEYS = {k: v for k, v in SERVICE_REGISTRY.items() if not v["public"]}
@@ -224,7 +228,8 @@ class ConfigManager:
            return cls._cache[key_name]
        val = os.environ.get(key_name, "") or cls._get_store().get(key_name, "")
        result = None if (not val or val == UNIVERSAL_PLACEHOLDER) else val
-        cls._cache[key_name] = result
+        if result is not None:
            cls._cache[key_name] = result
        return result
    # Backward-compatible alias used by nox.py internals
@@ -12,9 +12,9 @@ import logging
 import re
 from typing import List, Optional, Tuple
-# C2: MD5 and NTLM share the same 32-char hex pattern.
+# MD5 and NTLM share the same 32-char hex pattern. MD5 is listed first as it
-# We list md5 first (most common in breach data) but also accept ntlm
+# is the most common type in breach data. async_crack queries both md5 and
-# so callers can query NTLM-specific APIs when needed.
+# ntlm-specific APIs for any 32-char hash.
 _PATTERNS: List[Tuple[str, re.Pattern]] = [
    ("bcrypt",  re.compile(r"^\$2[aby]?\$\d{2}\$.{53}$")),
    ("sha256",  re.compile(r"^[a-f0-9]{64}$", re.I)),
@@ -67,43 +67,47 @@ async def _query_api(session, url: str, fmt: str) -> Optional[str]:
 async def async_crack(session, hash_value: str, hash_type: str) -> Optional[str]:
    """
-    Query multiple rainbow-table APIs concurrently.
+    Attempt to recover the plaintext for a given hash.
    Returns first plaintext found, or None. bcrypt is skipped.
-    C1: create tasks upfront for cancellation, but await each via asyncio.shield
+    Strategy:
-    inside as_completed — no double wait_for wrapping.
+    1. Local rockyou wordlist (no external calls, no rate limits).
-    C2: for 32-char hex (md5/ntlm ambiguity), also query NTLM-specific APIs.
+    2. hashes.com API if HASHES_COM_API_KEY is configured.
-    Per-API timeout: 8s. Global budget: 20s (CRACK_TIMEOUT).
+    bcrypt is skipped — computationally infeasible for online cracking.
    All tasks are cancelled as soon as the first result is found.
    """
    if hash_type == "bcrypt":
        return None
    h = hash_value.strip().lower()
    apis = [
        (f"https://www.nitrxgen.net/md5db/{h}",                                    "text"),
        (f"https://hashes.com/en/api/hash?hash={h}",                               "json"),
        (f"https://hash.help/api/lookup/{h}",                                       "json"),
        (f"https://hashkiller.io/api/search.php?hash={h}",                         "json"),
        (f"https://md5decrypt.net/Api/api.php?hash={h}&hash_type={hash_type}&email=&code=", "text"),
        (f"https://www.cmd5.org/api.ashx?hash={h}",                                "text"),
    ]
    # C2: for 32-char hashes (md5/ntlm ambiguous), add NTLM-specific endpoint
    if hash_type == "md5" and len(h) == 32:
        apis.append((f"https://hashes.com/en/api/hash?hash={h}&type=ntlm", "json"))
-    # C1: create tasks so we can cancel them; shield each before passing to wait_for
+    # 1. Local wordlist first — fast, zero external exposure
-    # so cancellation of the shield future does not cancel the underlying task prematurely.
+    import concurrent.futures as _cf
    loop = asyncio.get_running_loop()
    with _cf.ThreadPoolExecutor(max_workers=1) as _ex:
        local = await loop.run_in_executor(_ex, _local_crack_sync_blocking, hash_value, hash_type)
    if local:
        return local
    # 2. hashes.com if API key is configured
    apis = []
    try:
        from sources.helpers.config_handler import ConfigManager  # type: ignore
        hashes_com_key = ConfigManager.get_key("HASHES_COM_API_KEY")
        if hashes_com_key:
            apis.append((f"https://hashes.com/en/api/search?hash={h}&key={hashes_com_key}", "json"))
    except Exception:
        pass
    if not apis:
        return None
    tasks = [asyncio.create_task(_query_api(session, url, fmt)) for url, fmt in apis]
    result: Optional[str] = None
    try:
        for fut in asyncio.as_completed(tasks):
            try:
                res = await asyncio.wait_for(asyncio.shield(fut), timeout=_API_TIMEOUT)
-            except (asyncio.TimeoutError, asyncio.CancelledError):
+            except (asyncio.TimeoutError, asyncio.CancelledError, Exception):
                continue
            except Exception:
                continue
            if res:
                result = res
@@ -111,9 +115,49 @@ async def async_crack(session, hash_value: str, hash_type: str) -> Optional[str]
    except Exception:
        pass
    finally:
        # Cancel all remaining tasks and await to suppress pending-task warnings
        for t in tasks:
            if not t.done():
                t.cancel()
        await asyncio.gather(*[t for t in tasks if not t.done()], return_exceptions=True)
    return result
 def _local_crack_sync_blocking(hash_value: str, hash_type: str) -> Optional[str]:
    """Pure-sync version for ThreadPoolExecutor."""
    import hashlib as _hl
    from pathlib import Path as _Path
    wordlist = _Path.home() / ".nox" / "wordlists" / "rockyou.txt"
    if not wordlist.exists():
        return None
    h = hash_value.strip().lower()
    # usedforsecurity=False is required on FIPS-enabled systems (Python 3.9+).
    # On Python 3.8 the kwarg does not exist, so we fall back gracefully.
    def _md5(w):
        try:
            return _hl.md5(w, usedforsecurity=False).hexdigest()
        except TypeError:
            return _hl.md5(w).hexdigest()
    def _sha1(w):
        try:
            return _hl.sha1(w, usedforsecurity=False).hexdigest()
        except TypeError:
            return _hl.sha1(w).hexdigest()
    _hashers = {
        "md5":    _md5,
        "sha1":   _sha1,
        "sha256": lambda w: _hl.sha256(w).hexdigest(),
    }
    hasher = _hashers.get(hash_type)
    if not hasher:
        return None
    try:
        with wordlist.open("rb") as f:
            for line in f:
                word = line.rstrip(b"\n\r")
                if hasher(word) == h:
                    return word.decode("utf-8", errors="replace")
    except Exception:
        pass
    return None
@@ -14,9 +14,9 @@ from typing import Any, Dict, List
 # ── Noise patterns stripped from all report output ────────────────────
 _NOISE_RE = re.compile(
    r"(Traceback \(most recent|File \".*\.py\"|TimeoutError|ProxyError"
-    r"|ConnectionError|aiohttp\.|ClientConnector|ssl\.|asyncio\."
+    r"|ConnectionError|ClientConnector|Task exception|NoneType|Object of type"
-    r"|Task exception|NoneType|Object of type)",
+    r"|(?:^|[\s(])aiohttp\.|(?:^|[\s(])asyncio\.|(?:^|[\s(])ssl\.)",
-    re.I,
+    re.I | re.MULTILINE,
 )
 _CTRL_RE = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]")
@@ -48,11 +48,11 @@ def _raw(v: Any, maxlen: int = 200) -> str:
 def _pdf_safe(s: str, maxlen: int = 180) -> str:
-    # D4: sanitize for fpdf2 core fonts (latin-1 subset).
+    # Sanitise for fpdf2 core fonts (latin-1 subset).
    # NFKD normalization decomposes accented chars (é→e + combining accent)
    # so common accented Latin characters survive as their base letter.
-    # Truly non-latin-1 chars (Cyrillic, CJK, etc.) become '?' — intentional:
+    # Truly non-latin-1 chars (Cyrillic, CJK, etc.) become '?' — fpdf2 core
-    # fpdf2 core fonts cannot render them and would raise UnicodeEncodeError.
+    # fonts cannot render them and would raise UnicodeEncodeError.
    s = _raw(s, maxlen)
    try:
        import unicodedata
@@ -114,7 +114,7 @@ def render_pivot_chain(data: dict) -> List[str]:
    chain  = data.get("pivot_chain") or []
    target = _raw(data.get("target", "?"))
-    # D2: if pivot_log is available, build chain from it (accurate tree)
+    # Build chain from pivot_log when available — it carries the full tree with depth and provenance.
    pivot_log = data.get("pivot_log") or []
    if pivot_log:
        lines: List[str] = []
@@ -195,14 +195,12 @@ def to_json(data: dict, path: str) -> None:
    dork_results   = data.get("dork_results", []) or []
    scrape_results = data.get("scrape_results", {}) or {}
    # D3: apply consistent cap (1000) — same as HTML
    _RECORD_CAP = 1000
    out_data = {
        "framework":       f"NOX v{_NOX_VERSION}",
        "generated":       datetime.now().isoformat(),
        "target":          data.get("target", ""),
        # J3: self-describing metadata block
        "_meta": {
            "scan_id":        hashlib.sha256(
                f"{data.get('target','')}{datetime.now().isoformat()}".encode()
@@ -387,7 +385,6 @@ def to_html(data: dict, path: str) -> None:
 # ── PDF report (fpdf2) ────────────────────────────────────────────────
 def to_pdf(data: dict, path: str, investigator_id: str = "NOX-AUTO") -> None:
    # D1: raise a clear error with install hint if fpdf2 is absent — never silently return.
    try:
        from fpdf import FPDF  # type: ignore
    except ImportError:
@@ -31,7 +31,6 @@ _PIVOT_TYPES = {"email", "username", "phone", "name", "ip", "domain"}
 def _cfg_depth(orc=None) -> int:
    # A7/A10: read from orchestrator config if available
    if orc is not None:
        cfg = getattr(orc, "config", None)
        if cfg is not None:
@@ -46,7 +45,6 @@ def _cfg_depth(orc=None) -> int:
 def _cfg_concurrency(orc=None) -> int:
    # A7: read from orchestrator config if available
    if orc is not None:
        cfg = getattr(orc, "config", None)
        if cfg is not None:
@@ -137,29 +135,24 @@ class AvalancheScanner:
    def __init__(self, orchestrator: "Orchestrator") -> None:
        self._orc             = orchestrator
        self.seen_assets: Set[str]  = set()
        # A2: single semaphore for the entire run, created lazily inside the event loop
        self._sem: Optional[asyncio.Semaphore] = None
        self._all_records: List     = []
        self._dork_hits:   List[dict] = []
        self._seen_dork_urls: Set[str] = set()
        # A6: scrape_hits merged atomically per _do_process call
        self._scrape_hits: Dict     = {"pastes": [], "credentials": [], "hashes": [],
                                       "telegram": [], "dork_misconfigs": []}
        self._max_depth: int        = 0
        self._in_flight: Dict[str, asyncio.Future] = {}
        self.pivot_log: List[dict]  = []
        # A8: global set to prevent duplicate entries in discovered_assets
        self._seen_discovered: Set[str] = set()
        self.discovered_assets: List[dict] = []
    def _get_sem(self) -> asyncio.Semaphore:
        # A2: semaphore created once per run, shared across all coroutines
        if self._sem is None:
            self._sem = asyncio.Semaphore(_cfg_concurrency(self._orc))
        return self._sem
    async def run(self, target: str) -> tuple:
        # A9: respect no_pivot flag from config
        cfg = getattr(self._orc, "config", None)
        no_pivot = getattr(cfg, "no_pivot", False) if cfg else False
        if no_pivot:
@@ -196,7 +189,6 @@ class AvalancheScanner:
    async def _process(self, asset: str, depth: int,
                       parent: Optional[str], found_in: str) -> None:
        """Dedup gate: ensures each asset is processed exactly once."""
        # A10: use per-run depth from orchestrator config
        if depth > _cfg_depth(self._orc):
            _syslog.debug("avalanche depth cap reached for %s", asset)
            return
@@ -205,7 +197,7 @@ class AvalancheScanner:
        if not key:
            return
-        # A1: add to seen_assets FIRST (atomic gate) before any other check.
+        # Add to seen_assets before any await to prevent concurrent duplicates.
        # If already present, wait on the in-flight future if one exists, then return.
        if key in self.seen_assets:
            if key in self._in_flight:
@@ -235,6 +227,7 @@ class AvalancheScanner:
        finally:
            if not fut.done():
                fut.set_result(None)
            self._in_flight.pop(key, None)
    # ── Core pipeline ─────────────────────────────────────────────────
@@ -326,7 +319,8 @@ class AvalancheScanner:
            _syslog.warning("SCRAPE_FAIL asset=%s err=%s", asset, exc)
            scrape_res = {}
-        # A6: collect scrape results locally, then merge atomically
+        # Collect scrape results locally then merge into the shared dict.
        # The event loop is single-threaded so the merge is safe without a lock.
        scrape_count = 0
        local_scrape: Dict = {k: [] for k in self._scrape_hits}
        for k in self._scrape_hits:
@@ -336,7 +330,7 @@ class AvalancheScanner:
                    item["pivot_depth"] = depth
                local_scrape[k].append(item)
                scrape_count += 1
-        # Atomic merge into shared dict (single-threaded event loop — safe)
+        # Merge into shared dict — safe within the single-threaded event loop.
        for k, items in local_scrape.items():
            self._scrape_hits[k].extend(items)
        _out("ok" if scrape_count else "dim",
@@ -369,6 +363,13 @@ class AvalancheScanner:
            for val, vqtype in _extract_ids_from_text(raw, exclude=asset):
                if vqtype in _PIVOT_TYPES:
                    new_ids.append((val, vqtype, "scrape", ref))
        for paste in (scrape_res or {}).get("pastes", []):
            ref = f"paste:{paste.get('id', paste.get('site', 'paste'))}"
            for matches in (paste.get("patterns") or {}).values():
                for m in (matches or []):
                    for val, vqtype in _extract_ids_from_text(str(m), exclude=asset):
                        if vqtype in _PIVOT_TYPES:
                            new_ids.append((val, vqtype, "scrape", ref))
        for tg in (scrape_res or {}).get("telegram", []):
            ref = f"t.me/{tg.get('channel','')}"
            for val, vqtype in _extract_ids_from_text(tg.get("text", ""), exclude=asset):
@@ -393,7 +394,6 @@ class AvalancheScanner:
            queued.add(child_key)
            child_entry = {"asset": val, "qtype": vqtype, "found_in": phase, "ref": ref}
            children.append(child_entry)
            # A8: prevent duplicate entries in discovered_assets across parallel parents
            if child_key not in self._seen_discovered:
                self._seen_discovered.add(child_key)
                self.discovered_assets.append({
@@ -412,12 +412,12 @@ class AvalancheScanner:
                self._process(val, depth + 1, parent=asset, found_in=phase)
            )
-        # A5: run child tasks FIRST, then append pivot_log so the log reflects actual outcomes
+        # Run child tasks before appending to pivot_log so the log reflects actual outcomes.
        if child_tasks:
            _out("info", f"{indent}  → reinjecting {len(child_tasks)} new asset(s)…")
            await asyncio.gather(*child_tasks, return_exceptions=True)
-        # ── Log this node (after children complete — A5) ──────────────
+        # ── Log this node ─────────────────────────────────────────────
        self.pivot_log.append({
            "asset":    asset,
            "qtype":    qtype,
@@ -461,8 +461,8 @@ class AvalancheScanner:
    # ── Scrape dispatcher ─────────────────────────────────────────────
    async def _async_scrape(self, asset: str) -> dict:
-        # A3: instantiate a fresh Session + ScrapeEngine per call to avoid sharing
+        # Instantiate a fresh Session and ScrapeEngine per call — requests.Session
-        # a non-thread-safe requests.Session / cloudscraper across concurrent coroutines.
+        # and cloudscraper are not safe to share across concurrent coroutines.
        _empty: dict = {"pastes": [], "credentials": [], "hashes": [],
                        "telegram": [], "dork_misconfigs": []}
        try:
@@ -517,8 +517,7 @@ async def _crack_and_inject(session, hash_value: str, record_ref,
    _out("ok", f"  [crack] {hash_value[:16]}… → {plaintext}  (from {parent_asset})")
    cracked_out.append(plaintext)
-    # A4: inject cracked plaintext as qtype="password" — NOT as username.
+    # Inject the cracked plaintext as a password-recycling pivot seed.
    # Only pivot on it if sources support password-recycling queries.
    key = plaintext.lower()
    if key not in seen_assets and depth + 1 <= _cfg_depth(scanner._orc):
        await scanner._process(plaintext, depth + 1,
@@ -1,19 +1,22 @@
 {
    "name": "hudsonrock_osint",
    "category": "breach_data",
-    "endpoint": "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-login?username={target}",
+    "endpoint": "https://cavalier.hudsonrock.com/api/json/v2/osint-tools/search-by-email?email={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "stealers": "$.stealers"
    },
-    "rate_limit": 1.0,
+    "rate_limit": 30.0,
-    "headers": {},
+    "headers": {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36"
    },
    "api_key_slots": [],
-    "input_type": "username",
+    "input_type": "email",
    "output_type": [
        "email",
-        "domain"
+        "domain",
        "username"
    ],
    "normalization_map": {
        "stealers": "breach_record"
@@ -24,7 +27,8 @@
    ],
    "health_check_url": "https://cavalier.hudsonrock.com",
    "expected_status": 200,
-    "reliability_score": 4,
+    "reliability_score": 3,
    "is_volatile": true,
    "backup_endpoints": [],
-    "confidence": 0.85
+    "confidence": 0.7
 }
@@ -40,5 +40,9 @@
    "expected_status": 200,
    "reliability_score": 5,
    "backup_endpoints": [],
    "poll_endpoint": "https://2.intelx.io/intelligent/search/result",
    "poll_id_field": "id",
    "poll_id_param": "id",
    "poll_json_root": "records",
    "confidence": 1.0
 }
@@ -0,0 +1,33 @@
 {
    "name": "ipapi_is",
    "category": "geolocation",
    "endpoint": "https://api.ipapi.is/?q={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "org": "$.org",
        "asn": "$.asn.asn",
        "abuse": "$.abuse.email"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "ip",
    "output_type": [
        "domain"
    ],
    "normalization_map": {
        "org": "asn_org",
        "asn": "asn_number",
        "abuse": "abuse_contact"
    },
    "tags": [
        "passive",
        "fast"
    ],
    "health_check_url": "https://api.ipapi.is",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.85
 }
@@ -1,7 +1,7 @@
 {
    "name": "ipinfodb",
    "category": "geolocation",
-    "endpoint": "http://api.ipinfodb.com/v3/ip-city/?key={IPINFODB_API_KEY}&ip={target}&format=json",
+    "endpoint": "https://api.ipinfodb.com/v3/ip-city/?key={IPINFODB_API_KEY}&ip={target}&format=json",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
@@ -22,7 +22,7 @@
    "tags": [
        "passive"
    ],
-    "health_check_url": "http://api.ipinfodb.com",
+    "health_check_url": "https://api.ipinfodb.com",
    "expected_status": 200,
    "reliability_score": 3,
    "backup_endpoints": [],
@@ -1,7 +1,7 @@
 {
    "name": "ipstack",
    "category": "geolocation",
-    "endpoint": "http://api.ipstack.com/{target}?access_key={IPSTACK_API_KEY}",
+    "endpoint": "https://api.ipstack.com/{target}?access_key={IPSTACK_API_KEY}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
@@ -23,7 +23,7 @@
        "passive",
        "fast"
    ],
-    "health_check_url": "http://api.ipstack.com",
+    "health_check_url": "https://api.ipstack.com",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
@@ -3,17 +3,21 @@
    "category": "breaches",
    "endpoint": "https://leak-lookup.com/api/search",
    "method": "POST",
-    "requires_auth": false,
+    "requires_auth": true,
    "selectors": {
        "results": "$.message"
    },
    "rate_limit": 1.0,
-    "headers": {},
+    "headers": {
        "X-API-Key": "{LEAK_LOOKUP_API_KEY}"
    },
    "payload_template": {
        "query": "{target}",
        "type": "email_address"
    },
-    "api_key_slots": [],
+    "api_key_slots": [
        "{LEAK_LOOKUP_API_KEY}"
    ],
    "input_type": "email",
    "output_type": [
        "email"
@@ -1,31 +0,0 @@
 {
    "name": "leakstats_pw",
    "category": "breaches",
    "endpoint": "https://leakstats.net/api/password/{target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "count": "$.count"
    },
    "rate_limit": 1.0,
    "headers": {
        "api-key": "{LEAKSTATS_API_KEY}"
    },
    "api_key_slots": [
        "{LEAKSTATS_API_KEY}"
    ],
    "input_type": "hash",
    "output_type": [
        "hash"
    ],
    "normalization_map": {},
    "tags": [
        "passive"
    ],
    "health_check_url": "https://leakstats.net",
    "expected_status": 200,
    "reliability_score": 3,
    "is_volatile": true,
    "backup_endpoints": [],
    "confidence": 0.7
 }
@@ -1,7 +1,7 @@
 {
    "name": "mailboxlayer",
    "category": "email_rep",
-    "endpoint": "http://apilayer.net/api/check?access_key={MAILBOX_API_KEY}&email={target}",
+    "endpoint": "https://apilayer.net/api/check?access_key={MAILBOX_API_KEY}&email={target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
@@ -20,7 +20,7 @@
    "tags": [
        "passive"
    ],
-    "health_check_url": "http://apilayer.net",
+    "health_check_url": "https://apilayer.net",
    "expected_status": 200,
    "reliability_score": 3,
    "backup_endpoints": [],
@@ -0,0 +1,38 @@
 {
    "name": "malwarebazaar",
    "category": "hashes",
    "endpoint": "https://mb-api.abuse.ch/api/v1/",
    "method": "POST",
    "requires_auth": true,
    "selectors": {
        "data": "$.data"
    },
    "rate_limit": 1.0,
    "headers": {
        "API-KEY": "{MALWAREBAZAAR_API_KEY}"
    },
    "payload_template": {
        "query": "get_info",
        "hash": "{target}"
    },
    "api_key_slots": [
        "{MALWAREBAZAAR_API_KEY}"
    ],
    "input_type": "hash",
    "output_type": [
        "hash"
    ],
    "normalization_map": {
        "file_name": "filename",
        "tags": "tags"
    },
    "tags": [
        "passive",
        "threat"
    ],
    "health_check_url": "https://mb-api.abuse.ch",
    "expected_status": 200,
    "reliability_score": 5,
    "backup_endpoints": [],
    "confidence": 1.0
 }
@@ -17,7 +17,8 @@
        "value": "{target}"
    },
    "api_key_slots": [
-        "{MISP_API_KEY}"
+        "{MISP_API_KEY}",
        "{MISP_URL}"
    ],
    "input_type": "any",
    "output_type": [
@@ -30,7 +31,7 @@
        "passive",
        "threat"
    ],
-    "health_check_url": "{MISP_URL}",
+    "health_check_url": "https://misp.local",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
@@ -0,0 +1,35 @@
 {
    "name": "netlas_search",
    "category": "scanners",
    "endpoint": "https://app.netlas.io/api/responses/?q={target}&source_type=include&start=0&fields=",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "items": "$.items"
    },
    "rate_limit": 1.0,
    "headers": {
        "X-API-Key": "{NETLAS_API_KEY}"
    },
    "api_key_slots": [
        "{NETLAS_API_KEY}"
    ],
    "input_type": "ip",
    "output_type": [
        "ip",
        "domain"
    ],
    "normalization_map": {
        "data.ip": "ip_address",
        "data.domain": "domain"
    },
    "tags": [
        "passive",
        "infrastructure"
    ],
    "health_check_url": "https://app.netlas.io",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.85
 }
@@ -1,7 +1,7 @@
 {
    "name": "numverify",
    "category": "phone",
-    "endpoint": "http://apilayer.net/api/validate?access_key={NUMVERIFY_API_KEY}&number={target}",
+    "endpoint": "https://apilayer.net/api/validate?access_key={NUMVERIFY_API_KEY}&number={target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
@@ -24,7 +24,7 @@
    "tags": [
        "passive"
    ],
-    "health_check_url": "http://apilayer.net",
+    "health_check_url": "https://apilayer.net",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
@@ -24,7 +24,8 @@
    ],
    "health_check_url": "https://api.pipl.com",
    "expected_status": 200,
-    "reliability_score": 4,
+    "reliability_score": 2,
    "is_volatile": true,
    "backup_endpoints": [],
-    "confidence": 0.85
+    "confidence": 0.3
 }
@@ -1,11 +1,11 @@
 {
-    "name": "vigilante_pw",
+    "name": "proxynova_comb",
    "category": "breaches",
-    "endpoint": "https://vigilante.pw/api/search?q={target}",
+    "endpoint": "https://api.proxynova.com/comb?query={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
-        "results": "$.results"
+        "lines": "$.lines"
    },
    "rate_limit": 1.0,
    "headers": {},
@@ -14,15 +14,17 @@
    "output_type": [
        "email"
    ],
-    "normalization_map": {},
+    "normalization_map": {
        "lines": "credential_line"
    },
    "tags": [
        "passive",
        "stealth"
    ],
-    "health_check_url": "https://vigilante.pw",
+    "health_check_url": "https://api.proxynova.com",
    "expected_status": 200,
-    "reliability_score": 2,
+    "reliability_score": 3,
    "is_volatile": true,
    "backup_endpoints": [],
-    "confidence": 0.55
+    "confidence": 0.7
 }
@@ -1,14 +1,16 @@
 {
    "name": "pypi_user",
    "category": "social",
-    "endpoint": "https://pypi.org/pypi/{target}/json",
+    "endpoint": "https://pypi.org/pypi",
-    "method": "GET",
+    "method": "POST",
    "requires_auth": false,
-    "selectors": {
+    "selectors": {},
        "info": "$.info"
    },
    "rate_limit": 1.0,
-    "headers": {},
+    "headers": {
        "Content-Type": "text/xml"
    },
    "raw_payload": "<?xml version='1.0'?><methodCall><methodName>user_packages</methodName><params><param><value>{target}</value></param></params></methodCall>",
    "regex_pattern": "<string>Owner</string></value>\\s*<value><string>([^<]+)</string>",
    "api_key_slots": [],
    "input_type": "username",
    "output_type": [
@@ -20,7 +22,7 @@
    ],
    "health_check_url": "https://pypi.org",
    "expected_status": 200,
-    "reliability_score": 5,
+    "reliability_score": 4,
    "backup_endpoints": [],
-    "confidence": 1.0
+    "confidence": 0.85
 }
@@ -0,0 +1,31 @@
 {
    "name": "ripestat_ip",
    "category": "network",
    "endpoint": "https://stat.ripe.net/data/prefix-overview/data.json?resource={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "asns": "$.data.asns[*].asn",
        "holder": "$.data.asns[0].holder"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "ip",
    "output_type": [
        "ip"
    ],
    "normalization_map": {
        "asn": "asn_number",
        "holder": "asn_org"
    },
    "tags": [
        "passive",
        "infrastructure"
    ],
    "health_check_url": "https://stat.ripe.net",
    "expected_status": 200,
    "reliability_score": 5,
    "backup_endpoints": [],
    "confidence": 1.0
 }
@@ -1,31 +0,0 @@
 {
    "name": "scylla_sh_search",
    "category": "breaches",
    "endpoint": "https://scylla.sh/search?q={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "results": "$.*"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "email",
    "output_type": [
        "email",
        "domain"
    ],
    "normalization_map": {},
    "tags": [
        "passive",
        "stealth"
    ],
    "health_check_url": "https://scylla.sh",
    "expected_status": 200,
    "reliability_score": 2,
    "is_volatile": true,
    "backup_endpoints": [
        "https://scylla.sh/api/search?q={target}"
    ],
    "confidence": 0.55
 }
@@ -0,0 +1,41 @@
 {
    "name": "seon_email",
    "category": "enrichment",
    "endpoint": "https://api.seon.io/SeonRestService/email-api/v3",
    "method": "POST",
    "requires_auth": true,
    "selectors": {
        "data": "$.data"
    },
    "rate_limit": 1.0,
    "headers": {
        "X-API-KEY": "{SEON_API_KEY}",
        "Content-Type": "application/json"
    },
    "payload_template": {
        "email": "{target}"
    },
    "api_key_slots": [
        "{SEON_API_KEY}"
    ],
    "input_type": "email",
    "output_type": [
        "email",
        "domain",
        "username"
    ],
    "normalization_map": {
        "email": "email_address",
        "domain": "domain",
        "full_name": "full_name",
        "phone_number": "phone"
    },
    "tags": [
        "passive"
    ],
    "health_check_url": "https://api.seon.io",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.85
 }
@@ -0,0 +1,34 @@
 {
    "name": "shodan_internetdb",
    "category": "scanners",
    "endpoint": "https://internetdb.shodan.io/{target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "hostnames": "$.hostnames",
        "ports": "$.ports",
        "vulns": "$.vulns"
    },
    "rate_limit": 1.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "ip",
    "output_type": [
        "domain",
        "ip"
    ],
    "normalization_map": {
        "hostnames": "domain",
        "vulns": "cve"
    },
    "tags": [
        "passive",
        "fast",
        "infrastructure"
    ],
    "health_check_url": "https://internetdb.shodan.io",
    "expected_status": 200,
    "reliability_score": 5,
    "backup_endpoints": [],
    "confidence": 1.0
 }
@@ -1,7 +1,7 @@
 {
    "name": "spycloud_breach",
    "category": "breaches",
-    "endpoint": "https://api.spycloud.io/enterprise-v2/breach/data/emails/{target}",
+    "endpoint": "https://api.spycloud.io/enterprise-v2/breach/catalog/emails/{target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
@@ -22,7 +22,8 @@
    ],
    "health_check_url": "https://api.spyonweb.com",
    "expected_status": 200,
-    "reliability_score": 3,
+    "reliability_score": 1,
    "is_volatile": true,
    "backup_endpoints": [],
-    "confidence": 0.7
+    "confidence": 0.1
 }
@@ -1,30 +0,0 @@
 {
    "name": "spyse_domain",
    "category": "scanners",
    "endpoint": "https://api.spyse.com/v1/domain/details/{target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "asn": "$.data.asn"
    },
    "rate_limit": 1.0,
    "headers": {
        "Authorization": "Bearer {SPYSE_API_KEY}"
    },
    "api_key_slots": [
        "{SPYSE_API_KEY}"
    ],
    "input_type": "domain",
    "output_type": [
        "ip"
    ],
    "normalization_map": {},
    "tags": [
        "passive"
    ],
    "health_check_url": "https://api.spyse.com",
    "expected_status": 200,
    "reliability_score": 3,
    "backup_endpoints": [],
    "confidence": 0.7
 }
@@ -1,30 +0,0 @@
 {
    "name": "spyse_ip",
    "category": "scanners",
    "endpoint": "https://api.spyse.com/v1/ip/details/{target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "geo": "$.data.geo"
    },
    "rate_limit": 1.0,
    "headers": {
        "Authorization": "Bearer {SPYSE_API_KEY}"
    },
    "api_key_slots": [
        "{SPYSE_API_KEY}"
    ],
    "input_type": "ip",
    "output_type": [
        "ip"
    ],
    "normalization_map": {},
    "tags": [
        "passive"
    ],
    "health_check_url": "https://api.spyse.com",
    "expected_status": 200,
    "reliability_score": 3,
    "backup_endpoints": [],
    "confidence": 0.7
 }
@@ -9,7 +9,7 @@
    },
    "rate_limit": 1.0,
    "headers": {
-        "Authorization": "TC {TC_API_KEY}:{TC_SIGNATURE}"
+        "Authorization": "TC {TC_API_KEY}"
    },
    "api_key_slots": [
        "{TC_API_KEY}"
@@ -26,7 +26,8 @@
    ],
    "health_check_url": "https://api.threatconnect.com",
    "expected_status": 200,
-    "reliability_score": 4,
+    "reliability_score": 2,
    "is_volatile": true,
    "backup_endpoints": [],
-    "confidence": 0.85
+    "confidence": 0.55
 }
@@ -1,32 +0,0 @@
 {
    "name": "threatcrowd_domain",
    "category": "threat_intel",
    "endpoint": "https://www.threatcrowd.org/searchApi/v2/domain/report/?domain={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "ips": "$.resolutions[*].ip_address"
    },
    "rate_limit": 5.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "domain",
    "output_type": [
        "ip"
    ],
    "normalization_map": {},
    "tags": [
        "passive",
        "threat"
    ],
    "health_check_url": "https://www.threatcrowd.org",
    "expected_status": 200,
    "reliability_score": 3,
    "is_volatile": true,
    "bypass_required": [
        "cloudflare"
    ],
    "user_agent_type": "browser",
    "backup_endpoints": [],
    "confidence": 0.7
 }
@@ -1,32 +0,0 @@
 {
    "name": "threatcrowd_email",
    "category": "threat_intel",
    "endpoint": "https://www.threatcrowd.org/searchApi/v2/email/report/?email={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "domains": "$.domains"
    },
    "rate_limit": 5.0,
    "headers": {},
    "api_key_slots": [],
    "input_type": "email",
    "output_type": [
        "domain"
    ],
    "normalization_map": {},
    "tags": [
        "passive",
        "threat"
    ],
    "health_check_url": "https://www.threatcrowd.org",
    "expected_status": 200,
    "reliability_score": 3,
    "is_volatile": true,
    "bypass_required": [
        "cloudflare"
    ],
    "user_agent_type": "browser",
    "backup_endpoints": [],
    "confidence": 0.7
 }
@@ -0,0 +1,41 @@
 {
    "name": "threatfox",
    "category": "threat_intel",
    "endpoint": "https://threatfox-api.abuse.ch/api/v1/",
    "method": "POST",
    "requires_auth": true,
    "selectors": {
        "results": "$.data"
    },
    "rate_limit": 1.0,
    "headers": {
        "API-KEY": "{THREATFOX_API_KEY}",
        "Content-Type": "application/json"
    },
    "payload_template": {
        "query": "search_ioc",
        "search_term": "{target}"
    },
    "api_key_slots": [
        "{THREATFOX_API_KEY}"
    ],
    "input_type": "any",
    "output_type": [
        "ip",
        "domain",
        "hash"
    ],
    "normalization_map": {
        "ioc": "indicator",
        "malware": "malware_family"
    },
    "tags": [
        "passive",
        "threat"
    ],
    "health_check_url": "https://threatfox-api.abuse.ch",
    "expected_status": 200,
    "reliability_score": 5,
    "backup_endpoints": [],
    "confidence": 1.0
 }
@@ -1,30 +0,0 @@
 {
    "name": "tines_breach",
    "category": "breaches",
    "endpoint": "https://api.tines.com/breaches/{target}",
    "method": "GET",
    "requires_auth": true,
    "selectors": {
        "breaches": "$.breaches"
    },
    "rate_limit": 1.0,
    "headers": {
        "Authorization": "Bearer {TINES_API_KEY}"
    },
    "api_key_slots": [
        "{TINES_API_KEY}"
    ],
    "input_type": "email",
    "output_type": [
        "email"
    ],
    "normalization_map": {},
    "tags": [
        "passive"
    ],
    "health_check_url": "https://api.tines.com",
    "expected_status": 200,
    "reliability_score": 3,
    "backup_endpoints": [],
    "confidence": 0.7
 }
@@ -24,7 +24,8 @@
    ],
    "health_check_url": "https://api.twitter.com",
    "expected_status": 200,
-    "reliability_score": 4,
+    "reliability_score": 1,
    "is_volatile": true,
    "backup_endpoints": [],
-    "confidence": 0.85
+    "confidence": 0.1
 }
@@ -0,0 +1,38 @@
 {
    "name": "urlhaus",
    "category": "threat_intel",
    "endpoint": "https://urlhaus-api.abuse.ch/v1/host/",
    "method": "POST",
    "requires_auth": true,
    "selectors": {
        "urls": "$.urls"
    },
    "rate_limit": 1.0,
    "headers": {
        "Auth-Key": "{URLHAUS_API_KEY}"
    },
    "payload_template": {
        "host": "{target}"
    },
    "api_key_slots": [
        "{URLHAUS_API_KEY}"
    ],
    "input_type": "domain",
    "output_type": [
        "url",
        "domain"
    ],
    "normalization_map": {
        "url": "malware_url",
        "threat": "threat_type"
    },
    "tags": [
        "passive",
        "threat"
    ],
    "health_check_url": "https://urlhaus-api.abuse.ch",
    "expected_status": 200,
    "reliability_score": 5,
    "backup_endpoints": [],
    "confidence": 1.0
 }
@@ -0,0 +1,36 @@
 {
    "name": "xposedornot",
    "category": "breach_data",
    "endpoint": "https://api.xposedornot.com/v1/breach-analytics?email={target}",
    "method": "GET",
    "requires_auth": false,
    "selectors": {
        "breaches": "$.ExposedBreaches.breaches_details"
    },
    "rate_limit": 2.0,
    "headers": {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36"
    },
    "api_key_slots": [],
    "input_type": "email",
    "output_type": [
        "email",
        "domain"
    ],
    "normalization_map": {
        "breach": "breach_name",
        "domain": "domain",
        "xposed_date": "breach_date",
        "xposed_data": "data_types",
        "password_risk": "password_risk"
    },
    "tags": [
        "passive",
        "stealth"
    ],
    "health_check_url": "https://api.xposedornot.com",
    "expected_status": 200,
    "reliability_score": 4,
    "backup_endpoints": [],
    "confidence": 0.75
 }
Author	SHA1	Message	Date
nox-project	0814bfed2e	release: v1.0.5	2026-05-06 14:49:45 +02:00
nox-project	edcbf99fa7	release: v1.0.4	2026-04-22 16:53:20 +02:00
nox-project	61bd9b5555	release: v1.0.4	2026-04-22 15:49:51 +02:00
nox-project	294c7cbf84	release: v1.0.3 - Fix Avalanche Engine: paste pattern identifiers now harvested as pivot seeds - Bump version to 1.0.3	2026-04-15 12:22:05 +02:00
nox-project	d72163d4b6	fix: bump build_deb.sh version to 1.0.2	2026-04-14 22:21:06 +02:00
nox-project	9bf66d3e50	release: v1.0.2 - 124 sources (+1 xposedornot, bgpview replaced with ripestat) - Fix gravatar MD5 transform, fofa base64 query encoding - Fix misp_search URL resolution, threatconnect HMAC placeholder - Fix spycloud, duckduckgo, mailboxlayer/numverify/ipstack/ipinfodb endpoints - Fix DeHashEngine v1→v2, DorkEngine engine label, backup_endpoints consumed - Fix Retry-After HTTP-date parsing, Hashmob API schema, FIPS hashlib crash - Fix DB.close() event loop leak, _random_headers CH-UA override - Add query_transform mechanism (md5_lower, fofa_domain) - Lower scores: spyonweb, pipl_search, twitter_v2, hudsonrock rate_limit - Clean all internal tracking comments, fix Italian docstring	2026-04-14 21:18:30 +02:00
nox-project	cf4428329e	docs: update CHANGELOG for v1.0.1 maintenance patch	2026-04-13 21:44:33 +02:00
nox-project	1612a7ef48	maint: patch sources, engine hardening, proxy update for v1.0.1	2026-04-13 21:42:08 +02:00
nox-project	a166ce411d	fix(sources): replace clearbit_enrich and tines_breach with seon_email and flare_leaksdb	2026-04-13 13:24:27 +02:00
nox-project	ee1c2257d4	fix: version 1.0.1 in pyproject.toml	2026-04-13 10:58:31 +02:00
nox-project	c7dbc1baac	chore(deps): aiohttp>=3.13.5, add zstandard, zstd Accept-Encoding	2026-04-13 10:30:23 +02:00
nox-project	245d0f1714	fix: grant contents:write for release workflow	2026-04-11 15:14:08 +02:00
nox-project	200a21bf49	fix: use requirements.txt in CI and release workflows	2026-04-11 15:11:16 +02:00
nox-project	47bd936b6d	fix: version fallback reads from dpkg when metadata unavailable	2026-04-11 15:01:17 +02:00
nox-project	911e814080	fix: add build_sources step to release workflow	2026-04-11 14:31:43 +02:00
nox-project	ad45400e84	bump build_deb.sh version to 1.0.1	2026-04-11 14:22:32 +02:00
nox-project	7febdc60f5	v1.0.1 — quarterly maintenance: source audit, engine fixes, WAF hardening	2026-04-11 14:12:06 +02:00