forked from mirrors/kingfisher
performance improvements and access map viewer improvements
This commit is contained in:
parent
68dd1266a6
commit
09961f6feb
9 changed files with 99 additions and 5 deletions
|
|
@ -7,7 +7,7 @@ All notable changes to this project will be documented in this file.
|
|||
- Added revocation support for 7 rules across 6 providers: Discord webhooks (single-step DELETE), DigitalOcean PATs (self-revoke via OAuth), and multi-step HttpMultiStep revocation for LaunchDarkly, Resend, Linode, and Netlify (2 rules). Built-in revocation coverage is now 34 provider families with 53 revocation-enabled rules.
|
||||
- Expanded Alibaba Cloud coverage with STS temporary credential detection for STS access key IDs, STS security tokens, and STS access key secrets. Built-in rule coverage is now 923 rules total.
|
||||
- **Access Map:** Alibaba Cloud long-lived and STS access key pairs (validated `kingfisher.alibabacloud.2` and `kingfisher.alibabacloud.5`): caller identity via STS GetCallerIdentity; standalone `kingfisher access-map alibaba` (alias `aliyun`).
|
||||
- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer; imported findings deduplicate by secret identity; TruffleHog uses detector short names. See `docs/USAGE.md`.
|
||||
- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer with deduplication for repeated imported findings, and publish a static upload-based viewer on the docs site for GitHub Pages hosting. See `docs/USAGE.md`.
|
||||
- Fixed parser-based context gating so assignment-style contextual secrets still scan in raw text when parser verification is unavailable, instead of being dropped.
|
||||
- Corrected several newly added SaaS rules and validators, including LiveKit (with dependent API secret validation), Tinybird, Inngest, Tolgee, Unkey, Composio, Hex.pm, Trigger.dev, Voiceflow, WorkOS, and Infisical.
|
||||
- Added 61 new detection rules across 46 providers: Axiom (API token + PAT), Trigger.dev (secret key + PAT), Dub.co, Svix webhook signing secret, Liveblocks, Inngest (signing key + event key), Seam, Courier, Cal.com, Arcjet, WarpStream, Mem0, Mintlify, Pirsch, Tinybird, Tolgee (project key + PAT), Ory (API key + session + OAuth2 tokens), Xendit, Xata, Crossmint (server + client keys), DeepL (Free + Pro), Flagsmith, E2B, Infisical, WooCommerce (consumer key + secret), Nightfall AI, Ramp (client ID + secret), Hex.pm (personal + workspace tokens), Convex deploy key, MiniMax, Mappedin (key + secret), Pollinations (secret + publishable), Fal.ai, Aikido, Hack Club, GuardSquare, Browser Use, Composio, Gamma, Hex.tech, Mastra, redirect.pizza, Upstash, and WorkOS. Also added new prefixed-token rules for Netlify (`nfp_`), Cloudflare (`cfut_`), and Supabase (`sb_publishable_`). Added live HTTP validation for 30 of these rules.
|
||||
|
|
|
|||
|
|
@ -156,7 +156,7 @@ kingfisher scan /path/to/code
|
|||
kingfisher scan /path/to/code --view-report
|
||||
```
|
||||
|
||||
You can also open existing Kingfisher, Gitleaks, or TruffleHog JSON reports with `kingfisher view <report.json>`.
|
||||
You can also open existing Kingfisher, Gitleaks, or TruffleHog JSON reports with `kingfisher view <report.json>`. For a shareable upload-based experience, the docs site also hosts the report viewer as a static page.
|
||||
|
||||
### 4: Show only validated (live) secrets
|
||||
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@ All notable changes to this project will be documented in this file.
|
|||
- Added revocation support for 7 rules across 6 providers: Discord webhooks (single-step DELETE), DigitalOcean PATs (self-revoke via OAuth), and multi-step HttpMultiStep revocation for LaunchDarkly, Resend, Linode, and Netlify (2 rules). Built-in revocation coverage is now 34 provider families with 53 revocation-enabled rules.
|
||||
- Expanded Alibaba Cloud coverage with STS temporary credential detection for STS access key IDs, STS security tokens, and STS access key secrets. Built-in rule coverage is now 923 rules total.
|
||||
- **Access Map:** Alibaba Cloud long-lived and STS access key pairs (validated `kingfisher.alibabacloud.2` and `kingfisher.alibabacloud.5`): caller identity via STS GetCallerIdentity; standalone `kingfisher access-map alibaba` (alias `aliyun`).
|
||||
- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer; imported findings deduplicate by secret identity; TruffleHog uses detector short names. See `docs/USAGE.md`.
|
||||
- **Report viewer:** Import Gitleaks and TruffleHog JSON into the bundled local viewer with deduplication for repeated imported findings, and publish a static upload-based viewer on the docs site for GitHub Pages hosting. See `docs/USAGE.md`.
|
||||
- Fixed parser-based context gating so assignment-style contextual secrets still scan in raw text when parser verification is unavailable, instead of being dropped.
|
||||
- Corrected several newly added SaaS rules and validators, including LiveKit (with dependent API secret validation), Tinybird, Inngest, Tolgee, Unkey, Composio, Hex.pm, Trigger.dev, Voiceflow, WorkOS, and Infisical.
|
||||
- Added 61 new detection rules across 46 providers: Axiom (API token + PAT), Trigger.dev (secret key + PAT), Dub.co, Svix webhook signing secret, Liveblocks, Inngest (signing key + event key), Seam, Courier, Cal.com, Arcjet, WarpStream, Mem0, Mintlify, Pirsch, Tinybird, Tolgee (project key + PAT), Ory (API key + session + OAuth2 tokens), Xendit, Xata, Crossmint (server + client keys), DeepL (Free + Pro), Flagsmith, E2B, Infisical, WooCommerce (consumer key + secret), Nightfall AI, Ramp (client ID + secret), Hex.pm (personal + workspace tokens), Convex deploy key, MiniMax, Mappedin (key + secret), Pollinations (secret + publishable), Fal.ai, Aikido, Hack Club, GuardSquare, Browser Use, Composio, Gamma, Hex.tech, Mastra, redirect.pizza, Upstash, and WorkOS. Also added new prefixed-token rules for Netlify (`nfp_`), Cloudflare (`cfut_`), and Supabase (`sb_publishable_`). Added live HTTP validation for 30 of these rules.
|
||||
|
|
|
|||
40
docs-site/docs/features/report-viewer.md
Normal file
40
docs-site/docs/features/report-viewer.md
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
title: "Hosted Report Viewer"
|
||||
description: "Open the Kingfisher report viewer from the docs site and upload Kingfisher, Gitleaks, or TruffleHog JSON reports directly in your browser."
|
||||
---
|
||||
|
||||
Kingfisher ships a browser-based report viewer that can also be hosted from the documentation site as a static page.
|
||||
|
||||
[Open the hosted report viewer](../access-map-viewer/index.html)
|
||||
|
||||
## What it supports
|
||||
|
||||
- Upload local `Kingfisher` JSON and JSONL reports
|
||||
- Upload local `Gitleaks` JSON reports
|
||||
- Upload local `TruffleHog` JSON and JSONL reports
|
||||
- Merge multiple uploaded reports in one browser session
|
||||
- Explore findings, detector breakdowns, and access-map data when present
|
||||
|
||||
## Hosted vs local viewer
|
||||
|
||||
The hosted docs-site version is upload-based. It does not use the CLI-only local `/report` endpoint that powers `kingfisher view`.
|
||||
|
||||
Use the hosted version when you want a shareable static viewer on GitHub Pages.
|
||||
|
||||
Use the local CLI viewer when you want Kingfisher to open a report directly from disk:
|
||||
|
||||
```bash
|
||||
kingfisher view report.json
|
||||
```
|
||||
|
||||
## Sample data
|
||||
|
||||
You can test the hosted page with a bundled sample report:
|
||||
|
||||
- [Open sample report JSON](../access-map-viewer/sample-report.json)
|
||||
|
||||
## Notes
|
||||
|
||||
- Everything runs client-side in the browser.
|
||||
- Imported third-party reports are normalized for viewing and deduplicated by fingerprint logic in the viewer.
|
||||
- Native-only CLI conveniences such as auto-loading `/report` remain part of the local `kingfisher view` workflow.
|
||||
|
|
@ -59,6 +59,8 @@ kingfisher scan /path/to/code --view-report
|
|||
|
||||
You can also open existing Kingfisher, Gitleaks, or TruffleHog JSON reports with `kingfisher view <report.json>`.
|
||||
|
||||
If you want a shareable upload-based version, the docs site also hosts the [report viewer](../features/report-viewer.md).
|
||||
|
||||
## 4. Show Only Live Secrets
|
||||
|
||||
Filter to only secrets confirmed active by provider APIs:
|
||||
|
|
|
|||
|
|
@ -135,7 +135,9 @@ The browser-based viewer also supports loading multiple files via drag-and-drop
|
|||
|
||||
The local viewer also accepts Gitleaks JSON and TruffleHog JSON/JSONL as imported report formats. Imported findings are normalized into the viewer for triage, filtering, and export, which makes the viewer useful as a shared local workbench even when the original scan came from another tool.
|
||||
|
||||
Imported reports are display-oriented. They do not include Kingfisher-native `access_map` data, `validate` / `revoke` commands, or the same fingerprint semantics as a native Kingfisher report. Imported TruffleHog and Gitleaks findings deduplicate by secret identity. TruffleHog findings marked as verified are shown as active credentials; all other imported findings are treated as not attempted rather than inactive. For full validation context and blast-radius mapping, re-scan with Kingfisher and add `--access-map` when appropriate.
|
||||
A static upload-based copy of the viewer can also be hosted from the docs site for GitHub Pages deployments. The hosted version keeps the same client-side report browsing flow, but it does not use the local CLI `/report` endpoint that powers `kingfisher view`.
|
||||
|
||||
Imported reports are display-oriented. They do not include Kingfisher-native `access_map` data, `validate` / `revoke` commands, or the same fingerprint semantics as a native Kingfisher report. TruffleHog findings marked as verified are shown as active credentials; all other imported findings are treated as not attempted rather than inactive. For full validation context and blast-radius mapping, re-scan with Kingfisher and add `--access-map` when appropriate.
|
||||
|
||||
### Pipe any text directly into Kingfisher by passing `-`
|
||||
|
||||
|
|
|
|||
|
|
@ -83,6 +83,7 @@ nav:
|
|||
- Deployment: usage/deployment.md
|
||||
- Features:
|
||||
- Access Map (Blast Radius): features/access-map.md
|
||||
- Hosted Report Viewer: features/report-viewer.md
|
||||
- Secret Revocation: features/revocation.md
|
||||
- Source Code Parsing: features/parsing.md
|
||||
- Finding Fingerprints: features/fingerprints.md
|
||||
|
|
|
|||
|
|
@ -8,10 +8,17 @@ Copies documentation from /docs/ into docs-site/docs/ with transformations:
|
|||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
|
||||
REPO_ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..", ".."))
|
||||
DOCS_SRC = os.path.join(REPO_ROOT, "docs")
|
||||
DOCS_DST = os.path.join(REPO_ROOT, "docs-site", "docs")
|
||||
VIEWER_SRC_DIR = os.path.join(DOCS_SRC, "access-map-viewer")
|
||||
VIEWER_DST_DIR = os.path.join(DOCS_DST, "access-map-viewer")
|
||||
VIEWER_CLI_BOOTSTRAP = " loadCliReport();\n"
|
||||
VIEWER_STATIC_BOOTSTRAP = (
|
||||
" // Static docs-site build: skip the CLI-only /report bootstrap.\n"
|
||||
)
|
||||
|
||||
# Mapping: source filename -> (destination path, title, description)
|
||||
DOC_MAP = {
|
||||
|
|
@ -209,7 +216,46 @@ def copy_changelog():
|
|||
)
|
||||
with open(dst, "w", encoding="utf-8") as f:
|
||||
f.write(content)
|
||||
print(f" CHANGELOG.md -> changelog.md")
|
||||
print(" CHANGELOG.md -> changelog.md")
|
||||
|
||||
|
||||
def transform_viewer_for_docs_site(content: str) -> str:
|
||||
"""Disable the CLI-only embedded report bootstrap in the hosted viewer."""
|
||||
if VIEWER_CLI_BOOTSTRAP not in content:
|
||||
raise RuntimeError(
|
||||
"Could not find CLI bootstrap marker in access-map viewer"
|
||||
)
|
||||
return content.replace(VIEWER_CLI_BOOTSTRAP, VIEWER_STATIC_BOOTSTRAP, 1)
|
||||
|
||||
|
||||
def copy_access_map_viewer():
|
||||
"""Publish a static-hosted copy of the access-map viewer into docs-site/docs."""
|
||||
src_index = os.path.join(VIEWER_SRC_DIR, "index.html")
|
||||
dst_index = os.path.join(VIEWER_DST_DIR, "index.html")
|
||||
if not os.path.exists(src_index):
|
||||
print(
|
||||
" WARNING: docs/access-map-viewer/index.html not found, "
|
||||
"skipping viewer publish"
|
||||
)
|
||||
return
|
||||
|
||||
os.makedirs(VIEWER_DST_DIR, exist_ok=True)
|
||||
|
||||
with open(src_index, "r", encoding="utf-8") as f:
|
||||
content = f.read()
|
||||
transformed = transform_viewer_for_docs_site(content)
|
||||
with open(dst_index, "w", encoding="utf-8") as f:
|
||||
f.write(transformed)
|
||||
print(" access-map-viewer/index.html -> access-map-viewer/index.html")
|
||||
|
||||
sample_src = os.path.join(VIEWER_SRC_DIR, "sample-report.json")
|
||||
sample_dst = os.path.join(VIEWER_DST_DIR, "sample-report.json")
|
||||
if os.path.exists(sample_src):
|
||||
shutil.copy2(sample_src, sample_dst)
|
||||
print(
|
||||
" access-map-viewer/sample-report.json -> "
|
||||
"access-map-viewer/sample-report.json"
|
||||
)
|
||||
|
||||
|
||||
def main():
|
||||
|
|
@ -223,6 +269,7 @@ def main():
|
|||
print(f" WARNING: {src_name} not found, skipping")
|
||||
|
||||
copy_changelog()
|
||||
copy_access_map_viewer()
|
||||
print("Done.")
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -130,6 +130,8 @@ The browser-based viewer also supports loading multiple files via drag-and-drop
|
|||
|
||||
The local viewer also accepts Gitleaks JSON and TruffleHog JSON/JSONL as imported report formats. Imported findings are normalized into the viewer for triage, filtering, and export, which makes the viewer useful as a shared local workbench even when the original scan came from another tool.
|
||||
|
||||
A static upload-based copy of the viewer can also be hosted from the docs site for GitHub Pages deployments. The hosted version keeps the same client-side report browsing flow, but it does not use the local CLI `/report` endpoint that powers `kingfisher view`.
|
||||
|
||||
Imported reports are display-oriented. They do not include Kingfisher-native `access_map` data, `validate` / `revoke` commands, or the same fingerprint semantics as a native Kingfisher report. TruffleHog findings marked as verified are shown as active credentials; all other imported findings are treated as not attempted rather than inactive. For full validation context and blast-radius mapping, re-scan with Kingfisher and add `--access-map` when appropriate.
|
||||
|
||||
### Pipe any text directly into Kingfisher by passing `-`
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue