performance improvements and rule improvements

This commit is contained in:
Mick Grove 2026-04-24 00:23:50 -07:00
commit ceff3ab1c5
10 changed files with 18 additions and 18 deletions

View file

@ -6,7 +6,7 @@ All notable changes to this project will be documented in this file.
- **Report viewer cross-tool triage:** when a Kingfisher report is loaded alongside a Gitleaks or TruffleHog report, matching imported findings are enriched with Kingfisher's validation verdict, validation response, validate command, and revoke command. Matching is keyed on `commit + file + line` with a `file + line` fallback, and enriched rows show an "Enriched by Kingfisher" callout in the detail panel plus an "Enriched" chip in the findings table. Added a **Source** column to the findings table; a new **Duplicates Removed by Tool** dashboard panel showing per-tool cards for Kingfisher / TruffleHog / Gitleaks; and an upload-time **Deduplicate findings** toggle (on by default) so users can inspect the raw rows before fingerprint dedup when needed.
- Fixed the HTML report viewer dark mode so charts redraw correctly on theme changes and follow the system color scheme until manually overridden.
- Fixed [#344](https://github.com/mongodb/kingfisher/issues/344): baseline fingerprints no longer have to be hexadecimal. The fingerprint value emitted by scan output (JSON, JSONL, pretty, SARIF) can now be copied directly into a baseline file and will match on the next scan. `--manage-baseline` now writes fingerprints in decimal to match scan output, and legacy 16-char hex (and `0x`-prefixed hex) entries continue to be accepted, so existing baseline files keep working unchanged.
- Expanded the bundled ruleset to **938 rules** (817 standalone detectors + 121 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
- Expanded the bundled ruleset to **942 rules** (820 standalone detectors + 122 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
- Documentation: expanded coverage of the **Report Viewer & Triager** across `README.md`, `docs/USAGE.md`, and the docs site (`docs-site/docs/features/report-viewer.md`, `docs-site/docs/usage/basic-scanning.md`). The same viewer is available locally via `kingfisher view <report.json>` and as a hosted static upload-based page at [https://mongodb.github.io/kingfisher/viewer/](https://mongodb.github.io/kingfisher/viewer/). Both forms import Kingfisher, Gitleaks, and TruffleHog JSON/JSONL for cross-tool triage with fingerprint-based deduplication and blast-radius rendering.
## [v1.96.0]

View file

@ -7,7 +7,7 @@
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License" style="height: 24px;" />
</a>
<a href="https://github.com/mongodb/kingfisher">
<img src="https://img.shields.io/badge/Detection%20Rules-938-2ea043.svg" alt="Detection Rules" style="height: 24px;" />
<img src="https://img.shields.io/badge/Detection%20Rules-942-2ea043.svg" alt="Detection Rules" style="height: 24px;" />
</a>
<br>
<a href="https://github.com/mongodb/kingfisher/pkgs/container/kingfisher">
@ -17,7 +17,7 @@
Kingfisher is an open source secret scanner and **live secret validation** tool built in Rust.
It combines Intel's SIMD-accelerated regex engine (Hyperscan) with language-aware parsing to achieve high accuracy at massive scale, and ships with [938 built-in rules](https://mongodb.github.io/kingfisher/rules/builtin-rules/) to detect, **validate**, and triage leaked API keys, tokens, and credentials before they ever reach production.
It combines Intel's SIMD-accelerated regex engine (Hyperscan) with language-aware parsing to achieve high accuracy at massive scale, and ships with [942 built-in rules](https://mongodb.github.io/kingfisher/rules/builtin-rules/) to detect, **validate**, and triage leaked API keys, tokens, and credentials before they ever reach production.
Kingfisher also ships a **browser-based report viewer** that visualizes and triages findings from Kingfisher **and** from Gitleaks and TruffleHog JSON reports — so you can import scans from other tools and triage them in the same UI. A [hosted copy of the viewer](https://mongodb.github.io/kingfisher/viewer/) is published on the Kingfisher docs site.
@ -51,9 +51,9 @@ Kingfisher is a high-performance, open source secret detection tool for source c
</div>
### Performance, Accuracy, and 938 Rules
### Performance, Accuracy, and 942 Rules
- **Performance**: multithreaded, Hyperscanpowered scanning built for huge codebases
- **Extensible rules**: 938 built-in rules (484 with live validation) plus YAML-defined custom rules ([docs/RULES.md](/docs/RULES.md))
- **Extensible rules**: 942 built-in rules (484 with live validation) plus YAML-defined custom rules ([docs/RULES.md](/docs/RULES.md))
- **Validate & Revoke**: live validation of discovered secrets, plus direct revocation for supported platforms (GitHub, GitLab, Slack, AWS, GCP, and more) ([docs/USAGE.md](/docs/USAGE.md))
- **Revocation support matrix**: current built-in revocation coverage across providers and rule IDs ([docs/REVOCATION_PROVIDERS.md](/docs/REVOCATION_PROVIDERS.md))
- **Blast Radius Mapping**: instantly map leaked keys to their effective cloud identities and exposed resources with `--access-map`. Supports 42 providers (see table below).
@ -375,7 +375,7 @@ kingfisher scan /path/to/scan --access-map --view-report
# Detection Rules
Kingfisher ships with [938 built-in rules](crates/kingfisher-rules/data/rules/) covering cloud keys, AI tokens, CI/CD secrets, database credentials, and SaaS API keys. Below is an overview — see the full list in [crates/kingfisher-rules/data/rules/](crates/kingfisher-rules/data/rules/):
Kingfisher ships with [942 built-in rules](crates/kingfisher-rules/data/rules/) covering cloud keys, AI tokens, CI/CD secrets, database credentials, and SaaS API keys. Below is an overview — see the full list in [crates/kingfisher-rules/data/rules/](crates/kingfisher-rules/data/rules/):
| Category | What we catch |
|----------|---------------|

View file

@ -8,7 +8,9 @@ rules:
(?:.|[\n\r]){0,16}?
(?:client|id)
(?:.|[\n\r]){0,16}?
([a-z0-9]{30,40})
(
[a-z0-9]{30,40}
)
\b
pattern_requirements:
min_digits: 2
@ -41,8 +43,6 @@ rules:
\b
bitbucket
(?:.|[\n\r]){0,32}?
(?:SECRET|PRIVATE|ACCESS|KEY|TOKEN)
(?:.|[\n\r]){0,32}?
\b
(
[a-z0-9+_\-+]{44}

View file

@ -11,7 +11,7 @@ All notable changes to this project will be documented in this file.
- **Report viewer cross-tool triage:** when a Kingfisher report is loaded alongside a Gitleaks or TruffleHog report, matching imported findings are enriched with Kingfisher's validation verdict, validation response, validate command, and revoke command. Matching is keyed on `commit + file + line` with a `file + line` fallback, and enriched rows show an "Enriched by Kingfisher" callout in the detail panel plus an "Enriched" chip in the findings table. Added a **Source** column to the findings table; a new **Duplicates Removed by Tool** dashboard panel showing per-tool cards for Kingfisher / TruffleHog / Gitleaks; and an upload-time **Deduplicate findings** toggle (on by default) so users can inspect the raw rows before fingerprint dedup when needed.
- Fixed the HTML report viewer dark mode so charts redraw correctly on theme changes and follow the system color scheme until manually overridden.
- Fixed [#344](https://github.com/mongodb/kingfisher/issues/344): baseline fingerprints no longer have to be hexadecimal. The fingerprint value emitted by scan output (JSON, JSONL, pretty, SARIF) can now be copied directly into a baseline file and will match on the next scan. `--manage-baseline` now writes fingerprints in decimal to match scan output, and legacy 16-char hex (and `0x`-prefixed hex) entries continue to be accepted, so existing baseline files keep working unchanged.
- Expanded the bundled ruleset to **938 rules** (817 standalone detectors + 121 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
- Expanded the bundled ruleset to **942 rules** (820 standalone detectors + 122 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
- Documentation: expanded coverage of the **Report Viewer & Triager** across `README.md`, `docs/USAGE.md`, and the docs site (`docs-site/docs/features/report-viewer.md`, `docs-site/docs/usage/basic-scanning.md`). The same viewer is available locally via `kingfisher view <report.json>` and as a hosted static upload-based page at [https://mongodb.github.io/kingfisher/viewer/](https://mongodb.github.io/kingfisher/viewer/). Both forms import Kingfisher, Gitleaks, and TruffleHog JSON/JSONL for cross-tool triage with fingerprint-based deduplication and blast-radius rendering.
## [v1.96.0]

View file

@ -2,7 +2,7 @@
title: Kingfisher — Open Source Secret Scanner with Live Validation
description: >-
Kingfisher is an open source secret scanner with live validation, blast radius
mapping, and credential revocation. 938 detection rules (484 with live validation),
mapping, and credential revocation. 942 detection rules (484 with live validation),
plus a browser-based report viewer that also triages Gitleaks and TruffleHog output.
Built in Rust by MongoDB.
template: home.html

View file

@ -268,7 +268,7 @@ flowchart TD
### Loading Builtin Rules
Kingfisher currently ships with 938 built-in rules for common secret types:
Kingfisher currently ships with 942 built-in rules for common secret types:
```rust
use kingfisher_rules::{get_builtin_rules, Confidence};

View file

@ -1,12 +1,12 @@
---
title: "Built-in Rules List"
description: "Complete list of all 938 built-in secret detection rules in Kingfisher. Searchable and filterable by provider, confidence level, and validation support."
description: "Complete list of all 942 built-in secret detection rules in Kingfisher. Searchable and filterable by provider, confidence level, and validation support."
---
# Built-in Rules
Kingfisher ships with **938 detection rules** across **580 providers**
(817 detectors + 121 dependent rules).
Kingfisher ships with **942 detection rules** across **580 providers**
(820 detectors + 122 dependent rules).
Of these, **605** include live validation and **57** support direct revocation.
!!! tip "Search"

View file

@ -300,7 +300,7 @@ kingfisher scan ./my-project \
## Custom Rules
Kingfisher currently ships with 938 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
Kingfisher currently ships with 942 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
First, review [RULES.md](../rules/overview.md) to learn how to create custom Kingfisher rules.

View file

@ -297,7 +297,7 @@ kingfisher scan ./my-project \
## Custom Rules
Kingfisher currently ships with 938 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
Kingfisher currently ships with 942 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
First, review [RULES.md](RULES.md) to learn how to create custom Kingfisher rules.

View file

@ -265,7 +265,7 @@ flowchart TD
### Loading Builtin Rules
Kingfisher currently ships with 938 built-in rules for common secret types:
Kingfisher currently ships with 942 built-in rules for common secret types:
```rust
use kingfisher_rules::{get_builtin_rules, Confidence};