forked from mirrors/kingfisher
performance improvements and rule improvements
This commit is contained in:
parent
a4e8117c8e
commit
ceff3ab1c5
10 changed files with 18 additions and 18 deletions
|
|
@ -6,7 +6,7 @@ All notable changes to this project will be documented in this file.
|
|||
- **Report viewer cross-tool triage:** when a Kingfisher report is loaded alongside a Gitleaks or TruffleHog report, matching imported findings are enriched with Kingfisher's validation verdict, validation response, validate command, and revoke command. Matching is keyed on `commit + file + line` with a `file + line` fallback, and enriched rows show an "Enriched by Kingfisher" callout in the detail panel plus an "Enriched" chip in the findings table. Added a **Source** column to the findings table; a new **Duplicates Removed by Tool** dashboard panel showing per-tool cards for Kingfisher / TruffleHog / Gitleaks; and an upload-time **Deduplicate findings** toggle (on by default) so users can inspect the raw rows before fingerprint dedup when needed.
|
||||
- Fixed the HTML report viewer dark mode so charts redraw correctly on theme changes and follow the system color scheme until manually overridden.
|
||||
- Fixed [#344](https://github.com/mongodb/kingfisher/issues/344): baseline fingerprints no longer have to be hexadecimal. The fingerprint value emitted by scan output (JSON, JSONL, pretty, SARIF) can now be copied directly into a baseline file and will match on the next scan. `--manage-baseline` now writes fingerprints in decimal to match scan output, and legacy 16-char hex (and `0x`-prefixed hex) entries continue to be accepted, so existing baseline files keep working unchanged.
|
||||
- Expanded the bundled ruleset to **938 rules** (817 standalone detectors + 121 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
|
||||
- Expanded the bundled ruleset to **942 rules** (820 standalone detectors + 122 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
|
||||
- Documentation: expanded coverage of the **Report Viewer & Triager** across `README.md`, `docs/USAGE.md`, and the docs site (`docs-site/docs/features/report-viewer.md`, `docs-site/docs/usage/basic-scanning.md`). The same viewer is available locally via `kingfisher view <report.json>` and as a hosted static upload-based page at [https://mongodb.github.io/kingfisher/viewer/](https://mongodb.github.io/kingfisher/viewer/). Both forms import Kingfisher, Gitleaks, and TruffleHog JSON/JSONL for cross-tool triage with fingerprint-based deduplication and blast-radius rendering.
|
||||
|
||||
## [v1.96.0]
|
||||
|
|
|
|||
10
README.md
10
README.md
|
|
@ -7,7 +7,7 @@
|
|||
<img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License" style="height: 24px;" />
|
||||
</a>
|
||||
<a href="https://github.com/mongodb/kingfisher">
|
||||
<img src="https://img.shields.io/badge/Detection%20Rules-938-2ea043.svg" alt="Detection Rules" style="height: 24px;" />
|
||||
<img src="https://img.shields.io/badge/Detection%20Rules-942-2ea043.svg" alt="Detection Rules" style="height: 24px;" />
|
||||
</a>
|
||||
<br>
|
||||
<a href="https://github.com/mongodb/kingfisher/pkgs/container/kingfisher">
|
||||
|
|
@ -17,7 +17,7 @@
|
|||
|
||||
Kingfisher is an open source secret scanner and **live secret validation** tool built in Rust.
|
||||
|
||||
It combines Intel's SIMD-accelerated regex engine (Hyperscan) with language-aware parsing to achieve high accuracy at massive scale, and ships with [938 built-in rules](https://mongodb.github.io/kingfisher/rules/builtin-rules/) to detect, **validate**, and triage leaked API keys, tokens, and credentials before they ever reach production.
|
||||
It combines Intel's SIMD-accelerated regex engine (Hyperscan) with language-aware parsing to achieve high accuracy at massive scale, and ships with [942 built-in rules](https://mongodb.github.io/kingfisher/rules/builtin-rules/) to detect, **validate**, and triage leaked API keys, tokens, and credentials before they ever reach production.
|
||||
|
||||
Kingfisher also ships a **browser-based report viewer** that visualizes and triages findings from Kingfisher **and** from Gitleaks and TruffleHog JSON reports — so you can import scans from other tools and triage them in the same UI. A [hosted copy of the viewer](https://mongodb.github.io/kingfisher/viewer/) is published on the Kingfisher docs site.
|
||||
|
||||
|
|
@ -51,9 +51,9 @@ Kingfisher is a high-performance, open source secret detection tool for source c
|
|||
|
||||
</div>
|
||||
|
||||
### Performance, Accuracy, and 938 Rules
|
||||
### Performance, Accuracy, and 942 Rules
|
||||
- **Performance**: multithreaded, Hyperscan‑powered scanning built for huge codebases
|
||||
- **Extensible rules**: 938 built-in rules (484 with live validation) plus YAML-defined custom rules ([docs/RULES.md](/docs/RULES.md))
|
||||
- **Extensible rules**: 942 built-in rules (484 with live validation) plus YAML-defined custom rules ([docs/RULES.md](/docs/RULES.md))
|
||||
- **Validate & Revoke**: live validation of discovered secrets, plus direct revocation for supported platforms (GitHub, GitLab, Slack, AWS, GCP, and more) ([docs/USAGE.md](/docs/USAGE.md))
|
||||
- **Revocation support matrix**: current built-in revocation coverage across providers and rule IDs ([docs/REVOCATION_PROVIDERS.md](/docs/REVOCATION_PROVIDERS.md))
|
||||
- **Blast Radius Mapping**: instantly map leaked keys to their effective cloud identities and exposed resources with `--access-map`. Supports 42 providers (see table below).
|
||||
|
|
@ -375,7 +375,7 @@ kingfisher scan /path/to/scan --access-map --view-report
|
|||
|
||||
# Detection Rules
|
||||
|
||||
Kingfisher ships with [938 built-in rules](crates/kingfisher-rules/data/rules/) covering cloud keys, AI tokens, CI/CD secrets, database credentials, and SaaS API keys. Below is an overview — see the full list in [crates/kingfisher-rules/data/rules/](crates/kingfisher-rules/data/rules/):
|
||||
Kingfisher ships with [942 built-in rules](crates/kingfisher-rules/data/rules/) covering cloud keys, AI tokens, CI/CD secrets, database credentials, and SaaS API keys. Below is an overview — see the full list in [crates/kingfisher-rules/data/rules/](crates/kingfisher-rules/data/rules/):
|
||||
|
||||
| Category | What we catch |
|
||||
|----------|---------------|
|
||||
|
|
|
|||
|
|
@ -8,7 +8,9 @@ rules:
|
|||
(?:.|[\n\r]){0,16}?
|
||||
(?:client|id)
|
||||
(?:.|[\n\r]){0,16}?
|
||||
([a-z0-9]{30,40})
|
||||
(
|
||||
[a-z0-9]{30,40}
|
||||
)
|
||||
\b
|
||||
pattern_requirements:
|
||||
min_digits: 2
|
||||
|
|
@ -41,8 +43,6 @@ rules:
|
|||
\b
|
||||
bitbucket
|
||||
(?:.|[\n\r]){0,32}?
|
||||
(?:SECRET|PRIVATE|ACCESS|KEY|TOKEN)
|
||||
(?:.|[\n\r]){0,32}?
|
||||
\b
|
||||
(
|
||||
[a-z0-9+_\-+]{44}
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ All notable changes to this project will be documented in this file.
|
|||
- **Report viewer cross-tool triage:** when a Kingfisher report is loaded alongside a Gitleaks or TruffleHog report, matching imported findings are enriched with Kingfisher's validation verdict, validation response, validate command, and revoke command. Matching is keyed on `commit + file + line` with a `file + line` fallback, and enriched rows show an "Enriched by Kingfisher" callout in the detail panel plus an "Enriched" chip in the findings table. Added a **Source** column to the findings table; a new **Duplicates Removed by Tool** dashboard panel showing per-tool cards for Kingfisher / TruffleHog / Gitleaks; and an upload-time **Deduplicate findings** toggle (on by default) so users can inspect the raw rows before fingerprint dedup when needed.
|
||||
- Fixed the HTML report viewer dark mode so charts redraw correctly on theme changes and follow the system color scheme until manually overridden.
|
||||
- Fixed [#344](https://github.com/mongodb/kingfisher/issues/344): baseline fingerprints no longer have to be hexadecimal. The fingerprint value emitted by scan output (JSON, JSONL, pretty, SARIF) can now be copied directly into a baseline file and will match on the next scan. `--manage-baseline` now writes fingerprints in decimal to match scan output, and legacy 16-char hex (and `0x`-prefixed hex) entries continue to be accepted, so existing baseline files keep working unchanged.
|
||||
- Expanded the bundled ruleset to **938 rules** (817 standalone detectors + 121 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
|
||||
- Expanded the bundled ruleset to **942 rules** (820 standalone detectors + 122 dependent rules), with **484 standalone detectors** now including live HTTP / service-specific validation.
|
||||
- Documentation: expanded coverage of the **Report Viewer & Triager** across `README.md`, `docs/USAGE.md`, and the docs site (`docs-site/docs/features/report-viewer.md`, `docs-site/docs/usage/basic-scanning.md`). The same viewer is available locally via `kingfisher view <report.json>` and as a hosted static upload-based page at [https://mongodb.github.io/kingfisher/viewer/](https://mongodb.github.io/kingfisher/viewer/). Both forms import Kingfisher, Gitleaks, and TruffleHog JSON/JSONL for cross-tool triage with fingerprint-based deduplication and blast-radius rendering.
|
||||
|
||||
## [v1.96.0]
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
title: Kingfisher — Open Source Secret Scanner with Live Validation
|
||||
description: >-
|
||||
Kingfisher is an open source secret scanner with live validation, blast radius
|
||||
mapping, and credential revocation. 938 detection rules (484 with live validation),
|
||||
mapping, and credential revocation. 942 detection rules (484 with live validation),
|
||||
plus a browser-based report viewer that also triages Gitleaks and TruffleHog output.
|
||||
Built in Rust by MongoDB.
|
||||
template: home.html
|
||||
|
|
|
|||
|
|
@ -268,7 +268,7 @@ flowchart TD
|
|||
|
||||
### Loading Builtin Rules
|
||||
|
||||
Kingfisher currently ships with 938 built-in rules for common secret types:
|
||||
Kingfisher currently ships with 942 built-in rules for common secret types:
|
||||
|
||||
```rust
|
||||
use kingfisher_rules::{get_builtin_rules, Confidence};
|
||||
|
|
|
|||
|
|
@ -1,12 +1,12 @@
|
|||
---
|
||||
title: "Built-in Rules List"
|
||||
description: "Complete list of all 938 built-in secret detection rules in Kingfisher. Searchable and filterable by provider, confidence level, and validation support."
|
||||
description: "Complete list of all 942 built-in secret detection rules in Kingfisher. Searchable and filterable by provider, confidence level, and validation support."
|
||||
---
|
||||
|
||||
# Built-in Rules
|
||||
|
||||
Kingfisher ships with **938 detection rules** across **580 providers**
|
||||
(817 detectors + 121 dependent rules).
|
||||
Kingfisher ships with **942 detection rules** across **580 providers**
|
||||
(820 detectors + 122 dependent rules).
|
||||
Of these, **605** include live validation and **57** support direct revocation.
|
||||
|
||||
!!! tip "Search"
|
||||
|
|
|
|||
|
|
@ -300,7 +300,7 @@ kingfisher scan ./my-project \
|
|||
|
||||
## Custom Rules
|
||||
|
||||
Kingfisher currently ships with 938 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
|
||||
Kingfisher currently ships with 942 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
|
||||
|
||||
First, review [RULES.md](../rules/overview.md) to learn how to create custom Kingfisher rules.
|
||||
|
||||
|
|
|
|||
|
|
@ -297,7 +297,7 @@ kingfisher scan ./my-project \
|
|||
|
||||
## Custom Rules
|
||||
|
||||
Kingfisher currently ships with 938 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
|
||||
Kingfisher currently ships with 942 built-in rules, but you may want to add your own custom rules or modify existing detection to better suit your needs.
|
||||
|
||||
First, review [RULES.md](RULES.md) to learn how to create custom Kingfisher rules.
|
||||
|
||||
|
|
|
|||
|
|
@ -265,7 +265,7 @@ flowchart TD
|
|||
|
||||
### Loading Builtin Rules
|
||||
|
||||
Kingfisher currently ships with 938 built-in rules for common secret types:
|
||||
Kingfisher currently ships with 942 built-in rules for common secret types:
|
||||
|
||||
```rust
|
||||
use kingfisher_rules::{get_builtin_rules, Confidence};
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue