From 09961f6feb26c4bde6268bfbb53db9e96ebfd434 Mon Sep 17 00:00:00 2001 From: Mick Grove Date: Thu, 16 Apr 2026 13:34:44 -0700 Subject: [PATCH] performance improvements and access map viewer improvements --- CHANGELOG.md | 2 +- README.md | 2 +- docs-site/docs/changelog.md | 2 +- docs-site/docs/features/report-viewer.md | 40 +++++++++++++++ docs-site/docs/getting-started/quick-start.md | 2 + docs-site/docs/usage/basic-scanning.md | 4 +- docs-site/mkdocs.yml | 1 + docs-site/scripts/prepare-docs.py | 49 ++++++++++++++++++- docs/USAGE.md | 2 + 9 files changed, 99 insertions(+), 5 deletions(-) create mode 100644 docs-site/docs/features/report-viewer.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 8aaf103..a3e0fca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,7 @@ All notable changes to this project will be documented in this file. - Added revocation support for 7 rules across 6 providers: Discord webhooks (single-step DELETE), DigitalOcean PATs (self-revoke via OAuth), and multi-step HttpMultiStep revocation for LaunchDarkly, Resend, Linode, and Netlify (2 rules). Built-in revocation coverage is now 34 provider families with 53 revocation-enabled rules. - Expanded Alibaba Cloud coverage with STS temporary credential detection for STS access key IDs, STS security tokens, and STS access key secrets. Built-in rule coverage is now 923 rules total. - **Access Map:** Alibaba Cloud long-lived and STS access key pairs (validated `kingfisher.alibabacloud.2` and `kingfisher.alibabacloud.5`): caller identity via STS GetCallerIdentity; standalone `kingfisher access-map alibaba` (alias `aliyun`). -- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer; imported findings deduplicate by secret identity; TruffleHog uses detector short names. See `docs/USAGE.md`. +- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer with deduplication for repeated imported findings, and publish a static upload-based viewer on the docs site for GitHub Pages hosting. See `docs/USAGE.md`. - Fixed parser-based context gating so assignment-style contextual secrets still scan in raw text when parser verification is unavailable, instead of being dropped. - Corrected several newly added SaaS rules and validators, including LiveKit (with dependent API secret validation), Tinybird, Inngest, Tolgee, Unkey, Composio, Hex.pm, Trigger.dev, Voiceflow, WorkOS, and Infisical. - Added 61 new detection rules across 46 providers: Axiom (API token + PAT), Trigger.dev (secret key + PAT), Dub.co, Svix webhook signing secret, Liveblocks, Inngest (signing key + event key), Seam, Courier, Cal.com, Arcjet, WarpStream, Mem0, Mintlify, Pirsch, Tinybird, Tolgee (project key + PAT), Ory (API key + session + OAuth2 tokens), Xendit, Xata, Crossmint (server + client keys), DeepL (Free + Pro), Flagsmith, E2B, Infisical, WooCommerce (consumer key + secret), Nightfall AI, Ramp (client ID + secret), Hex.pm (personal + workspace tokens), Convex deploy key, MiniMax, Mappedin (key + secret), Pollinations (secret + publishable), Fal.ai, Aikido, Hack Club, GuardSquare, Browser Use, Composio, Gamma, Hex.tech, Mastra, redirect.pizza, Upstash, and WorkOS. Also added new prefixed-token rules for Netlify (`nfp_`), Cloudflare (`cfut_`), and Supabase (`sb_publishable_`). Added live HTTP validation for 30 of these rules. diff --git a/README.md b/README.md index ff742ee..1594801 100644 --- a/README.md +++ b/README.md @@ -156,7 +156,7 @@ kingfisher scan /path/to/code kingfisher scan /path/to/code --view-report ``` -You can also open existing Kingfisher, Gitleaks, or TruffleHog JSON reports with `kingfisher view `. +You can also open existing Kingfisher, Gitleaks, or TruffleHog JSON reports with `kingfisher view `. For a shareable upload-based experience, the docs site also hosts the report viewer as a static page. ### 4: Show only validated (live) secrets diff --git a/docs-site/docs/changelog.md b/docs-site/docs/changelog.md index 040b8f3..4cf5118 100644 --- a/docs-site/docs/changelog.md +++ b/docs-site/docs/changelog.md @@ -12,7 +12,7 @@ All notable changes to this project will be documented in this file. - Added revocation support for 7 rules across 6 providers: Discord webhooks (single-step DELETE), DigitalOcean PATs (self-revoke via OAuth), and multi-step HttpMultiStep revocation for LaunchDarkly, Resend, Linode, and Netlify (2 rules). Built-in revocation coverage is now 34 provider families with 53 revocation-enabled rules. - Expanded Alibaba Cloud coverage with STS temporary credential detection for STS access key IDs, STS security tokens, and STS access key secrets. Built-in rule coverage is now 923 rules total. - **Access Map:** Alibaba Cloud long-lived and STS access key pairs (validated `kingfisher.alibabacloud.2` and `kingfisher.alibabacloud.5`): caller identity via STS GetCallerIdentity; standalone `kingfisher access-map alibaba` (alias `aliyun`). -- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer; imported findings deduplicate by secret identity; TruffleHog uses detector short names. See `docs/USAGE.md`. +- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer with deduplication for repeated imported findings, and publish a static upload-based viewer on the docs site for GitHub Pages hosting. See `docs/USAGE.md`. - Fixed parser-based context gating so assignment-style contextual secrets still scan in raw text when parser verification is unavailable, instead of being dropped. - Corrected several newly added SaaS rules and validators, including LiveKit (with dependent API secret validation), Tinybird, Inngest, Tolgee, Unkey, Composio, Hex.pm, Trigger.dev, Voiceflow, WorkOS, and Infisical. - Added 61 new detection rules across 46 providers: Axiom (API token + PAT), Trigger.dev (secret key + PAT), Dub.co, Svix webhook signing secret, Liveblocks, Inngest (signing key + event key), Seam, Courier, Cal.com, Arcjet, WarpStream, Mem0, Mintlify, Pirsch, Tinybird, Tolgee (project key + PAT), Ory (API key + session + OAuth2 tokens), Xendit, Xata, Crossmint (server + client keys), DeepL (Free + Pro), Flagsmith, E2B, Infisical, WooCommerce (consumer key + secret), Nightfall AI, Ramp (client ID + secret), Hex.pm (personal + workspace tokens), Convex deploy key, MiniMax, Mappedin (key + secret), Pollinations (secret + publishable), Fal.ai, Aikido, Hack Club, GuardSquare, Browser Use, Composio, Gamma, Hex.tech, Mastra, redirect.pizza, Upstash, and WorkOS. Also added new prefixed-token rules for Netlify (`nfp_`), Cloudflare (`cfut_`), and Supabase (`sb_publishable_`). Added live HTTP validation for 30 of these rules. diff --git a/docs-site/docs/features/report-viewer.md b/docs-site/docs/features/report-viewer.md new file mode 100644 index 0000000..68418e2 --- /dev/null +++ b/docs-site/docs/features/report-viewer.md @@ -0,0 +1,40 @@ +--- +title: "Hosted Report Viewer" +description: "Open the Kingfisher report viewer from the docs site and upload Kingfisher, Gitleaks, or TruffleHog JSON reports directly in your browser." +--- + +Kingfisher ships a browser-based report viewer that can also be hosted from the documentation site as a static page. + +[Open the hosted report viewer](../access-map-viewer/index.html) + +## What it supports + +- Upload local `Kingfisher` JSON and JSONL reports +- Upload local `Gitleaks` JSON reports +- Upload local `TruffleHog` JSON and JSONL reports +- Merge multiple uploaded reports in one browser session +- Explore findings, detector breakdowns, and access-map data when present + +## Hosted vs local viewer + +The hosted docs-site version is upload-based. It does not use the CLI-only local `/report` endpoint that powers `kingfisher view`. + +Use the hosted version when you want a shareable static viewer on GitHub Pages. + +Use the local CLI viewer when you want Kingfisher to open a report directly from disk: + +```bash +kingfisher view report.json +``` + +## Sample data + +You can test the hosted page with a bundled sample report: + +- [Open sample report JSON](../access-map-viewer/sample-report.json) + +## Notes + +- Everything runs client-side in the browser. +- Imported third-party reports are normalized for viewing and deduplicated by fingerprint logic in the viewer. +- Native-only CLI conveniences such as auto-loading `/report` remain part of the local `kingfisher view` workflow. diff --git a/docs-site/docs/getting-started/quick-start.md b/docs-site/docs/getting-started/quick-start.md index 2928599..856b9be 100644 --- a/docs-site/docs/getting-started/quick-start.md +++ b/docs-site/docs/getting-started/quick-start.md @@ -59,6 +59,8 @@ kingfisher scan /path/to/code --view-report You can also open existing Kingfisher, Gitleaks, or TruffleHog JSON reports with `kingfisher view `. +If you want a shareable upload-based version, the docs site also hosts the [report viewer](../features/report-viewer.md). + ## 4. Show Only Live Secrets Filter to only secrets confirmed active by provider APIs: diff --git a/docs-site/docs/usage/basic-scanning.md b/docs-site/docs/usage/basic-scanning.md index 4501eed..554f68f 100644 --- a/docs-site/docs/usage/basic-scanning.md +++ b/docs-site/docs/usage/basic-scanning.md @@ -135,7 +135,9 @@ The browser-based viewer also supports loading multiple files via drag-and-drop The local viewer also accepts Gitleaks JSON and TruffleHog JSON/JSONL as imported report formats. Imported findings are normalized into the viewer for triage, filtering, and export, which makes the viewer useful as a shared local workbench even when the original scan came from another tool. -Imported reports are display-oriented. They do not include Kingfisher-native `access_map` data, `validate` / `revoke` commands, or the same fingerprint semantics as a native Kingfisher report. Imported TruffleHog and Gitleaks findings deduplicate by secret identity. TruffleHog findings marked as verified are shown as active credentials; all other imported findings are treated as not attempted rather than inactive. For full validation context and blast-radius mapping, re-scan with Kingfisher and add `--access-map` when appropriate. +A static upload-based copy of the viewer can also be hosted from the docs site for GitHub Pages deployments. The hosted version keeps the same client-side report browsing flow, but it does not use the local CLI `/report` endpoint that powers `kingfisher view`. + +Imported reports are display-oriented. They do not include Kingfisher-native `access_map` data, `validate` / `revoke` commands, or the same fingerprint semantics as a native Kingfisher report. TruffleHog findings marked as verified are shown as active credentials; all other imported findings are treated as not attempted rather than inactive. For full validation context and blast-radius mapping, re-scan with Kingfisher and add `--access-map` when appropriate. ### Pipe any text directly into Kingfisher by passing `-` diff --git a/docs-site/mkdocs.yml b/docs-site/mkdocs.yml index 12da3ee..504a22d 100644 --- a/docs-site/mkdocs.yml +++ b/docs-site/mkdocs.yml @@ -83,6 +83,7 @@ nav: - Deployment: usage/deployment.md - Features: - Access Map (Blast Radius): features/access-map.md + - Hosted Report Viewer: features/report-viewer.md - Secret Revocation: features/revocation.md - Source Code Parsing: features/parsing.md - Finding Fingerprints: features/fingerprints.md diff --git a/docs-site/scripts/prepare-docs.py b/docs-site/scripts/prepare-docs.py index 5300a0c..5960f4e 100644 --- a/docs-site/scripts/prepare-docs.py +++ b/docs-site/scripts/prepare-docs.py @@ -8,10 +8,17 @@ Copies documentation from /docs/ into docs-site/docs/ with transformations: import os import re +import shutil REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", "..")) DOCS_SRC = os.path.join(REPO_ROOT, "docs") DOCS_DST = os.path.join(REPO_ROOT, "docs-site", "docs") +VIEWER_SRC_DIR = os.path.join(DOCS_SRC, "access-map-viewer") +VIEWER_DST_DIR = os.path.join(DOCS_DST, "access-map-viewer") +VIEWER_CLI_BOOTSTRAP = " loadCliReport();\n" +VIEWER_STATIC_BOOTSTRAP = ( + " // Static docs-site build: skip the CLI-only /report bootstrap.\n" +) # Mapping: source filename -> (destination path, title, description) DOC_MAP = { @@ -209,7 +216,46 @@ def copy_changelog(): ) with open(dst, "w", encoding="utf-8") as f: f.write(content) - print(f" CHANGELOG.md -> changelog.md") + print(" CHANGELOG.md -> changelog.md") + + +def transform_viewer_for_docs_site(content: str) -> str: + """Disable the CLI-only embedded report bootstrap in the hosted viewer.""" + if VIEWER_CLI_BOOTSTRAP not in content: + raise RuntimeError( + "Could not find CLI bootstrap marker in access-map viewer" + ) + return content.replace(VIEWER_CLI_BOOTSTRAP, VIEWER_STATIC_BOOTSTRAP, 1) + + +def copy_access_map_viewer(): + """Publish a static-hosted copy of the access-map viewer into docs-site/docs.""" + src_index = os.path.join(VIEWER_SRC_DIR, "index.html") + dst_index = os.path.join(VIEWER_DST_DIR, "index.html") + if not os.path.exists(src_index): + print( + " WARNING: docs/access-map-viewer/index.html not found, " + "skipping viewer publish" + ) + return + + os.makedirs(VIEWER_DST_DIR, exist_ok=True) + + with open(src_index, "r", encoding="utf-8") as f: + content = f.read() + transformed = transform_viewer_for_docs_site(content) + with open(dst_index, "w", encoding="utf-8") as f: + f.write(transformed) + print(" access-map-viewer/index.html -> access-map-viewer/index.html") + + sample_src = os.path.join(VIEWER_SRC_DIR, "sample-report.json") + sample_dst = os.path.join(VIEWER_DST_DIR, "sample-report.json") + if os.path.exists(sample_src): + shutil.copy2(sample_src, sample_dst) + print( + " access-map-viewer/sample-report.json -> " + "access-map-viewer/sample-report.json" + ) def main(): @@ -223,6 +269,7 @@ def main(): print(f" WARNING: {src_name} not found, skipping") copy_changelog() + copy_access_map_viewer() print("Done.") diff --git a/docs/USAGE.md b/docs/USAGE.md index 5af86f6..274feb4 100644 --- a/docs/USAGE.md +++ b/docs/USAGE.md @@ -130,6 +130,8 @@ The browser-based viewer also supports loading multiple files via drag-and-drop The local viewer also accepts Gitleaks JSON and TruffleHog JSON/JSONL as imported report formats. Imported findings are normalized into the viewer for triage, filtering, and export, which makes the viewer useful as a shared local workbench even when the original scan came from another tool. +A static upload-based copy of the viewer can also be hosted from the docs site for GitHub Pages deployments. The hosted version keeps the same client-side report browsing flow, but it does not use the local CLI `/report` endpoint that powers `kingfisher view`. + Imported reports are display-oriented. They do not include Kingfisher-native `access_map` data, `validate` / `revoke` commands, or the same fingerprint semantics as a native Kingfisher report. TruffleHog findings marked as verified are shown as active credentials; all other imported findings are treated as not attempted rather than inactive. For full validation context and blast-radius mapping, re-scan with Kingfisher and add `--access-map` when appropriate. ### Pipe any text directly into Kingfisher by passing `-`