forked from mirrors/kingfisher
added kingfisher.github.9 to detect the new ~520-character stateless GitHub App installation token format (ghs_<APP_ID>_<JWT>). The legacy 36-character ghs_ rule
This commit is contained in:
Mick Grove
parent
2320a7ff72
commit
5465d903cf
17 changed files with 676 additions and 20 deletions
|
|
@ -2,6 +2,9 @@
|
|||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## [v1.98.0]
|
||||
- Fixed [#359](https://github.com/mongodb/kingfisher/issues/359): added `kingfisher.github.9` to detect the new ~520-character stateless GitHub App installation token format (`ghs_<APP_ID>_<JWT>`). The legacy 36-character `ghs_` rule (`kingfisher.github.5`) is retained for older / GHES-issued tokens that are still in circulation.
|
||||
|
||||
## [v1.97.0]
|
||||
- **Report viewer cross-tool triage:** when a Kingfisher report is loaded alongside a Gitleaks or TruffleHog report, matching imported findings are enriched with Kingfisher's validation verdict, validation response, validate command, and revoke command. Matching is keyed on `commit + file + line` with a `file + line` fallback, and enriched rows show an "Enriched by Kingfisher" callout in the detail panel plus an "Enriched" chip in the findings table. Added a **Source** column to the findings table; a new **Duplicates Removed by Tool** dashboard panel showing per-tool cards for Kingfisher / TruffleHog / Gitleaks; and an upload-time **Deduplicate findings** toggle (on by default) so users can inspect the raw rows before fingerprint dedup when needed.
|
||||
- Fixed the HTML report viewer dark mode so charts redraw correctly on theme changes and follow the system color scheme until manually overridden.
|
||||
|
|
|
|||
153
CONTRIBUTING.md
Normal file
153
CONTRIBUTING.md
Normal file
|
|
@ -0,0 +1,153 @@
|
|||
# Contributing to Kingfisher
|
||||
|
||||
Thank you for your interest in contributing to Kingfisher.
|
||||
|
||||
Kingfisher is an open-source project owned by MongoDB and licensed under the
|
||||
[Apache License 2.0](LICENSE). We welcome bug reports, feature requests,
|
||||
documentation improvements, rule additions, validation improvements, and code
|
||||
contributions.
|
||||
|
||||
## Before You Start
|
||||
|
||||
- Be respectful and collaborative. Participation in this project is covered by
|
||||
the [MongoDB Community Code of Conduct](https://www.mongodb.com/community-code-of-conduct).
|
||||
- If you plan to submit a pull request, sign the
|
||||
[MongoDB Contributor Agreement](https://www.mongodb.com/legal/contributor-agreement)
|
||||
first.
|
||||
- For security vulnerabilities, do not open a public issue. Follow
|
||||
[SECURITY.md](SECURITY.md) instead.
|
||||
|
||||
## Ways to Contribute
|
||||
|
||||
- Report bugs with clear reproduction steps, environment details, and logs when
|
||||
possible.
|
||||
- Propose features or usability improvements through GitHub issues.
|
||||
- Improve documentation in `README.md`, `docs/`, or `docs-site/`.
|
||||
- Add or refine detection rules under
|
||||
`crates/kingfisher-rules/data/rules/`.
|
||||
- Improve validation, revocation, scanning performance, output formats, or
|
||||
integrations.
|
||||
|
||||
## Reporting Bugs and Requesting Features
|
||||
|
||||
Before opening a new issue:
|
||||
|
||||
- Check whether an existing issue already covers the problem or request.
|
||||
- Confirm the issue still reproduces on a recent `main` checkout or current
|
||||
release when practical.
|
||||
- Include the smallest reproducible example you can provide.
|
||||
|
||||
Use the repository issue templates when they fit your case.
|
||||
|
||||
## Development Setup
|
||||
|
||||
Kingfisher is a Rust workspace. The workspace minimum Rust version is `1.94`,
|
||||
and CI currently uses Rust `1.94.1`.
|
||||
|
||||
Helpful commands:
|
||||
|
||||
```bash
|
||||
cargo build
|
||||
make tests
|
||||
cargo test --workspace --all-targets
|
||||
cargo fmt --all
|
||||
cargo clippy --workspace --all-targets -- -D warnings
|
||||
```
|
||||
|
||||
For repository layout and project-specific guidance, see:
|
||||
|
||||
- [README.md](README.md)
|
||||
- [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md)
|
||||
- [docs/USAGE.md](docs/USAGE.md)
|
||||
- [docs/RULES.md](docs/RULES.md)
|
||||
|
||||
## Contribution Guidelines
|
||||
|
||||
### Keep changes focused
|
||||
|
||||
- Prefer small, reviewable pull requests over large mixed changes.
|
||||
- Avoid unrelated refactors in the same PR unless they are necessary for the
|
||||
fix.
|
||||
- Update tests and docs when behavior changes.
|
||||
|
||||
### Do not commit real secrets
|
||||
|
||||
Kingfisher is a secret scanner. Never add live credentials, customer data, or
|
||||
real tokens anywhere in the repository, including:
|
||||
|
||||
- tests
|
||||
- fixtures
|
||||
- examples
|
||||
- docs
|
||||
- screenshots
|
||||
- benchmark artifacts
|
||||
|
||||
Use clearly fake placeholders or provider-documented example values only.
|
||||
|
||||
### Rule contributions
|
||||
|
||||
If you are adding or updating a rule:
|
||||
|
||||
- Follow the schema and authoring guidance in [docs/RULES.md](docs/RULES.md).
|
||||
- Prefer YAML-defined validation and revocation when the provider API supports
|
||||
it.
|
||||
- Keep patterns specific and efficient.
|
||||
- Add realistic examples and relevant tests.
|
||||
- Set rule confidence to `medium`.
|
||||
|
||||
Useful validation commands:
|
||||
|
||||
```bash
|
||||
cargo test -p kingfisher-rules
|
||||
cargo test --workspace --all-targets
|
||||
kingfisher scan ./testdata --rule <rule-family-or-id> --rule-stats
|
||||
kingfisher validate --rule <rule-id> <token-or-secret>
|
||||
```
|
||||
|
||||
## Testing Expectations
|
||||
|
||||
Run the narrowest relevant checks for your change before opening a PR, then run
|
||||
broader checks when practical.
|
||||
|
||||
Examples:
|
||||
|
||||
- Rule-only changes: `cargo test -p kingfisher-rules`
|
||||
- General Rust changes: `make tests`
|
||||
- Formatting: `cargo fmt --all`
|
||||
- Linting: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
|
||||
If you cannot run a relevant check locally, say so in the pull request and
|
||||
explain why.
|
||||
|
||||
## Documentation Changes
|
||||
|
||||
- Keep examples consistent with current CLI behavior.
|
||||
- Update related docs when flags, outputs, or workflows change.
|
||||
- After changing `docs-site/` sources, rebuild the site when practical:
|
||||
|
||||
```bash
|
||||
docs-site/.venv/bin/mkdocs build -f docs-site/mkdocs.yml
|
||||
```
|
||||
|
||||
## Pull Request Checklist
|
||||
|
||||
Before opening a PR, make sure you have:
|
||||
|
||||
- signed the MongoDB Contributor Agreement
|
||||
- kept the change focused
|
||||
- added or updated tests where needed
|
||||
- updated docs where needed
|
||||
- run the relevant local checks
|
||||
- avoided adding any real secrets or sensitive data
|
||||
|
||||
In the PR description, include:
|
||||
|
||||
- what changed
|
||||
- why it changed
|
||||
- how you tested it
|
||||
- any follow-up work or known limitations
|
||||
|
||||
## Questions
|
||||
|
||||
If you are unsure whether a change is in scope, open an issue first so the
|
||||
approach can be discussed before you spend time on implementation.
|
||||
2
Cargo.lock
generated
2
Cargo.lock
generated
|
|
@ -5030,7 +5030,7 @@ dependencies = [
|
|||
|
||||
[[package]]
|
||||
name = "kingfisher"
|
||||
version = "1.97.0"
|
||||
version = "1.98.0"
|
||||
dependencies = [
|
||||
"anyhow",
|
||||
"asar",
|
||||
|
|
|
|||
|
|
@ -48,7 +48,7 @@ http = "1.4"
|
|||
|
||||
[package]
|
||||
name = "kingfisher"
|
||||
version = "1.97.0"
|
||||
version = "1.98.0"
|
||||
description = "MongoDB's blazingly fast and accurate secret scanning and validation tool"
|
||||
edition.workspace = true
|
||||
rust-version.workspace = true
|
||||
|
|
|
|||
|
|
@ -13,6 +13,9 @@
|
|||
<a href="https://github.com/mongodb/kingfisher/pkgs/container/kingfisher">
|
||||
<img src="https://ghcr-badge.elias.eu.org/shield/mongodb/kingfisher/kingfisher" alt="ghcr downloads" />
|
||||
</a>
|
||||
<a href="https://github.com/mongodb/kingfisher/releases">
|
||||
<img src="https://img.shields.io/github/downloads/mongodb/kingfisher/total" alt="GitHub Downloads" style="height: 24px;" />
|
||||
</a>
|
||||
<br>
|
||||
|
||||
Kingfisher is an open source secret scanner and **live secret validation** tool built in Rust.
|
||||
|
|
@ -392,7 +395,7 @@ Kingfisher ships with [942 built-in rules](crates/kingfisher-rules/data/rules/)
|
|||
|
||||
## Write Custom Rules
|
||||
|
||||
Kingfisher ships with 484 built-in rules that include HTTP and service-specific validation checks (AWS, Azure, GCP, etc.) to confirm if a detected string is a live credential.
|
||||
Of Kingfisher's 942 built-in rules, 484 include HTTP and service-specific validation checks (AWS, Azure, GCP, etc.) to confirm if a detected string is a live credential.
|
||||
|
||||
However, you may want to add your own custom rules, or modify a detection to better suit your needs / environment.
|
||||
|
||||
|
|
|
|||
|
|
@ -362,3 +362,48 @@ rules:
|
|||
- |
|
||||
GITHUB_CLIENT_ID=ac58d6da7d7a84c039b7
|
||||
GITHUB_SECRET=37d02377a3e9d849e18704c3ec883f9c5787d857
|
||||
|
||||
- name: GitHub App Server-to-Server Token (stateless JWT format)
|
||||
id: kingfisher.github.9
|
||||
pattern: |
|
||||
(?x)
|
||||
(
|
||||
ghs_[0-9]+_
|
||||
[A-Za-z0-9_-]+ \. [A-Za-z0-9_-]+ \. [A-Za-z0-9_-]+
|
||||
)
|
||||
\b
|
||||
min_entropy: 3.5
|
||||
examples:
|
||||
- 'ghs_12345_eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3NDU1NjgwMDAsImV4cCI6MTc0NTU2ODM2MCwiaXNzIjoiMTIzNDUiLCJzdWIiOiJnaXRodWJ8MTIzNDUifQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c'
|
||||
references:
|
||||
- https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/about-authentication-with-a-github-app
|
||||
- https://github.com/mongodb/kingfisher/issues/359
|
||||
validation:
|
||||
type: Http
|
||||
content:
|
||||
request:
|
||||
method: GET
|
||||
url: https://api.github.com/user
|
||||
headers:
|
||||
Authorization: token {{ TOKEN }}
|
||||
Accept: application/vnd.github+json
|
||||
response_matcher:
|
||||
- report_response: true
|
||||
- match_all_words: true
|
||||
type: WordMatch
|
||||
words:
|
||||
- '"login"'
|
||||
- '"id"'
|
||||
revocation:
|
||||
type: Http
|
||||
content:
|
||||
request:
|
||||
method: DELETE
|
||||
url: https://api.github.com/installation/token
|
||||
headers:
|
||||
Authorization: token {{ TOKEN }}
|
||||
Accept: application/vnd.github+json
|
||||
response_matcher:
|
||||
- report_response: true
|
||||
- type: StatusMatch
|
||||
status: [204]
|
||||
11
docs-site/docs/blog/index.md
Normal file
11
docs-site/docs/blog/index.md
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
---
|
||||
title: Kingfisher Blog
|
||||
description: >
|
||||
News, tutorials, and deep-dives on secret detection, credential validation,
|
||||
and supply-chain security from the Kingfisher team at MongoDB.
|
||||
---
|
||||
|
||||
# Kingfisher Blog
|
||||
|
||||
Tutorials, release notes, and deep-dives on secret scanning, credential
|
||||
validation, and supply-chain security.
|
||||
|
|
@ -0,0 +1,201 @@
|
|||
---
|
||||
date: 2026-04-26
|
||||
title: "Beyond Detection: Live Validation, Blast Radius, and One-Command Revocation"
|
||||
description: >
|
||||
Detection alone is noise. Kingfisher answers the three questions that
|
||||
actually matter when a secret leaks — is it live, what does it reach,
|
||||
and can we revoke it now — across AWS, GCP, GitHub, GitLab, Slack,
|
||||
and dozens of other providers.
|
||||
categories:
|
||||
- Features
|
||||
tags:
|
||||
- validation
|
||||
- blast-radius
|
||||
- revocation
|
||||
- secret-scanning
|
||||
---
|
||||
|
||||
# Beyond Detection: Live Validation, Blast Radius, and One-Command Revocation
|
||||
|
||||
A regex match on `AKIA[0-9A-Z]{16}` is the easy part. Every secret scanner
|
||||
finds those. The hard part — and the part that decides whether your Tuesday
|
||||
afternoon turns into an incident — is what happens **after** the match.
|
||||
|
||||
Kingfisher answers the three questions that actually matter:
|
||||
|
||||
1. **Is this credential alive right now?**
|
||||
2. **What can it reach?**
|
||||
3. **Can we kill it from here?**
|
||||
|
||||
<!-- more -->
|
||||
|
||||
## 1. Live validation, not just pattern matching
|
||||
|
||||
Out of Kingfisher's 820 standalone detectors, **484 include live validation
|
||||
logic**. Every one of those calls the provider's own API and reports the
|
||||
credential as `Active`, `Inactive`, or `NotAttempted` — so a 4,000-finding
|
||||
scan collapses to the dozen findings that are actually live.
|
||||
|
||||
Validation runs automatically when you scan:
|
||||
|
||||
```bash
|
||||
kingfisher scan github --organization my-org
|
||||
```
|
||||
|
||||
Or you can run it standalone when you've already pulled a suspicious value
|
||||
out of a paste, a log, or a customer ticket:
|
||||
|
||||
```bash
|
||||
# Hit GitHub's user API to confirm the token works
|
||||
kingfisher validate --rule github "$GITHUB_TOKEN"
|
||||
|
||||
# AWS needs both halves of the keypair
|
||||
kingfisher validate --rule aws \
|
||||
--arg "$AWS_ACCESS_KEY_ID" \
|
||||
"$AWS_SECRET_ACCESS_KEY"
|
||||
|
||||
# A GCP service account JSON, straight from the file
|
||||
kingfisher validate --rule gcp "$(cat service-account.json)"
|
||||
|
||||
# A Postgres connection URI — does it actually authenticate?
|
||||
kingfisher validate --rule postgres "$POSTGRES_URI"
|
||||
```
|
||||
|
||||
Validation logic lives in the rule YAML, not in compiled Rust, which is
|
||||
why coverage is high and growing — every new detector ships with a
|
||||
validation block whenever the provider exposes a safe check call.
|
||||
|
||||
## 2. Blast radius mapping — what does this token actually reach?
|
||||
|
||||
A leaked AWS key bound to a single read-only S3 bucket and a leaked AWS key
|
||||
bound to organization-wide `AdministratorAccess` are not the same incident.
|
||||
The first is a Friday afternoon ticket. The second is a war room.
|
||||
|
||||
Add `--access-map` to a scan and Kingfisher authenticates each live
|
||||
credential, enumerates what it can do, and writes the result alongside
|
||||
the finding:
|
||||
|
||||
```bash
|
||||
kingfisher scan github --organization my-org \
|
||||
--access-map \
|
||||
--format json \
|
||||
--output findings.json
|
||||
```
|
||||
|
||||
Each cloud finding gets an `access_map` block with the identity, the
|
||||
permissions, and the concrete resources reachable. Today this is supported
|
||||
for **AWS, GCP, Azure Storage, Azure DevOps, GitHub, GitLab, Slack, and
|
||||
Microsoft Teams.**
|
||||
|
||||
You can also run it standalone — useful when triaging a single credential
|
||||
you've fished out of a paste or a customer report:
|
||||
|
||||
```bash
|
||||
# What does this AWS keypair actually own?
|
||||
kingfisher access-map aws ./aws.json --json-out aws.access-map.json
|
||||
|
||||
# Same for a GitHub token
|
||||
kingfisher access-map github ./github.token --json-out github.access-map.json
|
||||
|
||||
# Or a GCP service account
|
||||
kingfisher access-map gcp ./service-account.json --json-out gcp.access-map.json
|
||||
```
|
||||
|
||||
The HTML report viewer (`--format html`) renders the access map as a
|
||||
clickable tree — identity at the root, then services, then individual
|
||||
resources and permissions. It's the fastest way to get a non-engineer
|
||||
stakeholder to grasp severity in five seconds rather than five minutes.
|
||||
|
||||
## 3. Revocation — kill the token from where you found it
|
||||
|
||||
Validation tells you a credential is live. Blast radius tells you why it's
|
||||
urgent. Revocation tells you it's done.
|
||||
|
||||
For every rule whose provider exposes a safe revocation API, Kingfisher
|
||||
ships the revocation call as part of the rule definition. One command,
|
||||
no console:
|
||||
|
||||
```bash
|
||||
# Revoke a GitHub PAT
|
||||
kingfisher revoke --rule github "$GITHUB_TOKEN"
|
||||
|
||||
# Revoke a GitLab token
|
||||
kingfisher revoke --rule gitlab "$GITLAB_TOKEN"
|
||||
|
||||
# Revoke a Slack bot token
|
||||
kingfisher revoke --rule slack "$SLACK_TOKEN"
|
||||
|
||||
# Deactivate an AWS access key
|
||||
kingfisher revoke --rule aws \
|
||||
--arg "$AWS_ACCESS_KEY_ID" \
|
||||
"$AWS_SECRET_ACCESS_KEY"
|
||||
|
||||
# Disable a GCP service account key
|
||||
kingfisher revoke --rule gcp "$(cat service-account.json)"
|
||||
```
|
||||
|
||||
The same Liquid templating that powers the validation request handles
|
||||
revocation — including multi-step flows for providers that need a separate
|
||||
key-id lookup before disabling. (See
|
||||
[`docs/RULES.md`](https://github.com/mongodb/kingfisher/blob/main/docs/RULES.md#multi-step-revocation)
|
||||
for the schema.)
|
||||
|
||||
This matters in two scenarios:
|
||||
|
||||
- **Mass revocation after a leak.** A laptop or a CI runner gets popped and
|
||||
you have a list of fingerprints. `kingfisher revoke` walks the list, no
|
||||
human pivoting between five provider consoles.
|
||||
- **Automated response.** Wire `kingfisher revoke` into the same job that
|
||||
scanned and validated, gated by an allow-list of rule IDs you've decided
|
||||
are safe to auto-revoke (typically: short-lived CI tokens, dev-environment
|
||||
secrets). The credential is dead before the on-call gets paged.
|
||||
|
||||
## The combined workflow
|
||||
|
||||
In practice these three primitives chain into a single pipeline:
|
||||
|
||||
```bash
|
||||
# 1. Scan + validate + map blast radius in one call
|
||||
kingfisher scan github --organization my-org \
|
||||
--access-map \
|
||||
--format json \
|
||||
--output findings.json
|
||||
|
||||
# 2. Pull just the live, high-blast-radius findings
|
||||
jq '[.[] | select(.validation.status == "Active")
|
||||
| select(.access_map.permissions
|
||||
| any(. == "*" or contains("Admin")))]' \
|
||||
findings.json > urgent.json
|
||||
|
||||
# 3. Triage in the HTML viewer (or revoke programmatically)
|
||||
kingfisher view findings.json
|
||||
```
|
||||
|
||||
Three commands, full incident workflow — find, prioritize, kill.
|
||||
|
||||
## Why this is the right shape
|
||||
|
||||
Most scanners stop at step one because going further is expensive: every
|
||||
provider has its own auth flow, its own permission model, its own
|
||||
revocation API. Kingfisher gets to a high-coverage version of all three by
|
||||
keeping the logic in YAML rule files (the same place the detection regex
|
||||
lives), reusing typed validators for the common families (AWS, GCP, JWT,
|
||||
Postgres, MongoDB, MySQL, JDBC, Azure Storage, Coinbase), and letting rule
|
||||
authors drop down to a `Raw` validator only for genuinely odd providers.
|
||||
|
||||
The upshot for users: when a new detector lands, you almost always get
|
||||
validation, blast radius, and revocation along with it — not three
|
||||
separate roadmaps.
|
||||
|
||||
## Next up
|
||||
|
||||
- **Catching secrets in pull requests with GitHub Actions** — pre-merge
|
||||
scanning so leaked credentials never reach `main`.
|
||||
- **Top leaked credential types we see in the wild** — what validation
|
||||
telemetry says about the credential leak landscape.
|
||||
- **Docker image scanning** — pulling and scanning every layer for
|
||||
embedded secrets.
|
||||
|
||||
Got a provider you'd love to see validation or revocation support for?
|
||||
Open an issue at
|
||||
[mongodb/kingfisher](https://github.com/mongodb/kingfisher/issues).
|
||||
|
|
@ -0,0 +1,178 @@
|
|||
---
|
||||
date: 2026-04-26
|
||||
title: "Scanning an Entire GitHub Organization for Leaked Secrets"
|
||||
description: >
|
||||
Step-by-step guide to scanning every repository in a GitHub organization
|
||||
for leaked credentials with Kingfisher — including history, issues, wikis,
|
||||
and gists — and validating which secrets are still live.
|
||||
categories:
|
||||
- Tutorials
|
||||
tags:
|
||||
- github
|
||||
- secret-scanning
|
||||
- validation
|
||||
- tutorial
|
||||
---
|
||||
|
||||
# Scanning an Entire GitHub Organization for Leaked Secrets
|
||||
|
||||
Most organizations have hundreds of repositories — some abandoned, some active,
|
||||
plenty inherited from acquisitions. A leaked AWS key in a five-year-old archived
|
||||
repo is just as dangerous as one in `main` today. Kingfisher can enumerate every
|
||||
repo in a GitHub organization, scan the full git history, and then **validate
|
||||
which credentials are still live** so you know what to rotate first.
|
||||
|
||||
<!-- more -->
|
||||
|
||||
## What you need
|
||||
|
||||
- Kingfisher installed (`brew install mongodb/brew/kingfisher`, or grab a
|
||||
release from [GitHub](https://github.com/mongodb/kingfisher/releases)).
|
||||
- A GitHub personal access token exported as `KF_GITHUB_TOKEN`. A classic
|
||||
token with `repo` and `read:org` scopes is enough for private repos; for
|
||||
public-only scans, even an unscoped token raises your rate limit and
|
||||
is strongly recommended.
|
||||
- About 5 GB of free disk for clones (varies by org size — use
|
||||
`--git-clone-dir /path/to/big/disk` if your home volume is small).
|
||||
|
||||
## The one-liner
|
||||
|
||||
```bash
|
||||
export KF_GITHUB_TOKEN=ghp_yourTokenHere
|
||||
kingfisher scan github --organization my-org
|
||||
```
|
||||
|
||||
That's it — Kingfisher enumerates every repo, clones each one, scans the full
|
||||
commit history, runs all 942 detection rules, and validates findings against
|
||||
provider APIs.
|
||||
|
||||
## Tuning for real-world orgs
|
||||
|
||||
Real orgs have huge monorepos, archived junk, and forks you don't care about.
|
||||
Three flags do most of the work:
|
||||
|
||||
```bash
|
||||
kingfisher scan github --organization my-org \
|
||||
--repo-clone-limit 500 \
|
||||
--github-exclude 'my-org/*-archive' \
|
||||
--github-exclude 'my-org/legacy-monorepo' \
|
||||
--git-clone-dir /var/tmp/kf-clones \
|
||||
--format sarif \
|
||||
--output kf-findings.sarif
|
||||
```
|
||||
|
||||
- **`--repo-clone-limit`** caps the number of clones per scan. Useful for
|
||||
staged rollouts ("first 500 repos by stars") or to stay under disk budget.
|
||||
- **`--github-exclude`** accepts exact `OWNER/REPO` strings or gitignore-style
|
||||
globs (`my-org/*-archive`). Repeat the flag for each pattern. Matching is
|
||||
case-insensitive.
|
||||
- **`--git-clone-dir`** moves clones off your home volume. Combine with
|
||||
`--keep-clones` if you want to re-scan later without re-cloning.
|
||||
|
||||
## Pulling in issues, wikis, and gists
|
||||
|
||||
Secrets don't only live in code. Issues and pull request descriptions are a
|
||||
common leak source — someone pastes a stack trace with a JWT, or an
|
||||
"oncall handoff" issue with a temporary token that never got rotated. Add
|
||||
`--repo-artifacts` to fetch these:
|
||||
|
||||
```bash
|
||||
kingfisher scan github --organization my-org --repo-artifacts
|
||||
```
|
||||
|
||||
This pulls each repo's issues (including PRs), wiki, and any **public** gists
|
||||
owned by the repo owner, and scans them all. It does cost API calls, so plan
|
||||
accordingly if you're near a rate limit.
|
||||
|
||||
## Following the people, not just the org
|
||||
|
||||
This is the trick that catches what every other scanner misses. Developers
|
||||
leak secrets in *personal* repositories — side projects, dotfiles, throwaway
|
||||
forks. If a contributor to your org has a public personal repo with an active
|
||||
token that grants access to org infrastructure, that's a real incident.
|
||||
|
||||
Pass a single repo URL with `--include-contributors` and Kingfisher will
|
||||
enumerate the contributors, then clone and scan **every public repo they own**:
|
||||
|
||||
```bash
|
||||
kingfisher scan https://github.com/my-org/critical-service \
|
||||
--include-contributors \
|
||||
--repo-clone-limit 200
|
||||
```
|
||||
|
||||
This is a noisy operation — start with one or two critical repos rather than
|
||||
the whole org. GitHub will rate-limit aggressive enumeration, so a token
|
||||
(`KF_GITHUB_TOKEN`) is required in practice.
|
||||
|
||||
## Reading the output
|
||||
|
||||
The default `pretty` output is human-friendly for terminals. For automation,
|
||||
pick the format that matches your downstream tool:
|
||||
|
||||
```bash
|
||||
# JSON for custom tooling
|
||||
kingfisher scan github --organization my-org --format json --output findings.json
|
||||
|
||||
# SARIF for GitHub code scanning, GitLab, or any SARIF-aware UI
|
||||
kingfisher scan github --organization my-org --format sarif --output findings.sarif
|
||||
|
||||
# TOON for piping to an LLM or agent
|
||||
kingfisher scan github --organization my-org --format toon
|
||||
```
|
||||
|
||||
The interactive HTML report is often the fastest way to triage a large scan —
|
||||
filter by rule, by validation status, or by repository, and click through to
|
||||
the exact commit and line:
|
||||
|
||||
```bash
|
||||
kingfisher scan github --organization my-org --format html --output kf-report.html
|
||||
```
|
||||
|
||||
## Triage by validation status
|
||||
|
||||
The single most important column in the output is **validation**. A live
|
||||
credential is a fire — a never-was-valid one is noise. Filter to live findings
|
||||
first:
|
||||
|
||||
```bash
|
||||
jq '.[] | select(.validation.status == "Active")' findings.json
|
||||
```
|
||||
|
||||
Then walk those credentials in order of blast radius. For AWS, GCP, GitHub,
|
||||
GitLab, and Slack tokens, Kingfisher already maps what each one can access —
|
||||
look at the `access_map` field in the JSON output, or the **Blast Radius**
|
||||
panel in the HTML report.
|
||||
|
||||
## Revoke from the CLI
|
||||
|
||||
For supported providers, you don't need to log into a console — Kingfisher can
|
||||
revoke directly:
|
||||
|
||||
```bash
|
||||
kingfisher revoke --rule kingfisher.aws.access_key.1 AKIAEXAMPLE...
|
||||
```
|
||||
|
||||
Each rule that supports revocation declares the API call in its YAML. Today
|
||||
this works for AWS, GitHub, GitLab, Slack, and a growing list of SaaS
|
||||
providers — see [`docs/RULES.md`](https://github.com/mongodb/kingfisher/blob/main/docs/RULES.md)
|
||||
for the current list and how to add revocation to a custom rule.
|
||||
|
||||
## Wiring it into a recurring job
|
||||
|
||||
A first scan is the one-shot baseline. The real value is recurring scans
|
||||
catching new leaks within hours, not months. The simplest pattern is a nightly
|
||||
GitHub Action or scheduled CI job that runs the org scan, diffs against
|
||||
yesterday's findings, and pages on net-new live credentials. We'll cover that
|
||||
end-to-end in the next post.
|
||||
|
||||
## What's next
|
||||
|
||||
- **Catching secrets in pull requests with GitHub Actions** — pre-merge
|
||||
scanning so leaks never reach `main`.
|
||||
- **The most common credential types we see leaked in the wild** — what
|
||||
Kingfisher's validation telemetry says about the credential leak landscape.
|
||||
- **Docker image scanning** — pulling images directly and scanning every
|
||||
layer for embedded secrets.
|
||||
|
||||
If there's a workflow you'd like us to cover, open an issue at
|
||||
[mongodb/kingfisher](https://github.com/mongodb/kingfisher/issues).
|
||||
|
|
@ -7,6 +7,9 @@ description: "Kingfisher release history: new features, rules, bug fixes, and im
|
|||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## [v1.98.0]
|
||||
- Fixed [#359](https://github.com/mongodb/kingfisher/issues/359): added `kingfisher.github.9` to detect the new ~520-character stateless GitHub App installation token format (`ghs_<APP_ID>_<JWT>`). The legacy 36-character `ghs_` rule (`kingfisher.github.5`) is retained for older / GHES-issued tokens that are still in circulation. Bundled ruleset is now **943 rules** (821 standalone detectors + 122 dependent rules), with **485 standalone detectors** offering live validation.
|
||||
|
||||
## [v1.97.0]
|
||||
- **Report viewer cross-tool triage:** when a Kingfisher report is loaded alongside a Gitleaks or TruffleHog report, matching imported findings are enriched with Kingfisher's validation verdict, validation response, validate command, and revoke command. Matching is keyed on `commit + file + line` with a `file + line` fallback, and enriched rows show an "Enriched by Kingfisher" callout in the detail panel plus an "Enriched" chip in the findings table. Added a **Source** column to the findings table; a new **Duplicates Removed by Tool** dashboard panel showing per-tool cards for Kingfisher / TruffleHog / Gitleaks; and an upload-time **Deduplicate findings** toggle (on by default) so users can inspect the raw rows before fingerprint dedup when needed.
|
||||
- Fixed the HTML report viewer dark mode so charts redraw correctly on theme changes and follow the system color scheme until manually overridden.
|
||||
|
|
|
|||
|
|
@ -2,7 +2,7 @@
|
|||
title: Kingfisher — Open Source Secret Scanner with Live Validation
|
||||
description: >-
|
||||
Kingfisher is an open source secret scanner with live validation, blast radius
|
||||
mapping, and credential revocation. 942 detection rules (484 with live validation),
|
||||
mapping, and credential revocation. 943 detection rules (485 with live validation),
|
||||
plus a browser-based report viewer that also triages Gitleaks and TruffleHog output.
|
||||
Built in Rust by MongoDB.
|
||||
template: home.html
|
||||
|
|
|
|||
|
|
@ -1,13 +1,13 @@
|
|||
---
|
||||
title: "Built-in Rules List"
|
||||
description: "Complete list of all 942 built-in secret detection rules in Kingfisher. Searchable and filterable by provider, confidence level, and validation support."
|
||||
description: "Complete list of all 943 built-in secret detection rules in Kingfisher. Searchable and filterable by provider, confidence level, and validation support."
|
||||
---
|
||||
|
||||
# Built-in Rules
|
||||
|
||||
Kingfisher ships with **942 detection rules** across **580 providers**
|
||||
(820 detectors + 122 dependent rules).
|
||||
Of these, **605** include live validation and **57** support direct revocation.
|
||||
Kingfisher ships with **943 detection rules** across **581 providers**
|
||||
(821 detectors + 122 dependent rules).
|
||||
Of these, **485** include live validation and **50** support direct revocation.
|
||||
|
||||
!!! tip "Search"
|
||||
Use the search box below to filter rules by provider name, rule ID, or confidence level.
|
||||
|
|
@ -127,7 +127,7 @@ Of these, **605** include live validation and **57** support direct revocation.
|
|||
<td>Agora</td>
|
||||
<td>Agora App ID</td>
|
||||
<td><code>kingfisher.agora.1</code></td>
|
||||
<td>Low</td>
|
||||
<td>Medium</td>
|
||||
<td></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
|
|
@ -1572,6 +1572,22 @@ Of these, **605** include live validation and **57** support direct revocation.
|
|||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Confluence</td>
|
||||
<td>Confluence Data Center Personal Access Token</td>
|
||||
<td><code>kingfisher.confluence.1</code></td>
|
||||
<td>Medium</td>
|
||||
<td>Yes</td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Confluence</td>
|
||||
<td>Confluence Data Center Domain</td>
|
||||
<td><code>kingfisher.confluence.2</code></td>
|
||||
<td>Medium</td>
|
||||
<td></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Confluent</td>
|
||||
<td>Confluent Client ID</td>
|
||||
<td><code>kingfisher.confluent.1</code></td>
|
||||
|
|
@ -2096,7 +2112,7 @@ Of these, **605** include live validation and **57** support direct revocation.
|
|||
<td>DocuSign API Secret Key</td>
|
||||
<td><code>kingfisher.docusign.1</code></td>
|
||||
<td>Medium</td>
|
||||
<td>Yes</td>
|
||||
<td></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
|
|
@ -2940,6 +2956,14 @@ Of these, **605** include live validation and **57** support direct revocation.
|
|||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Github</td>
|
||||
<td>GitHub App Server-to-Server Token (stateless JWT format)</td>
|
||||
<td><code>kingfisher.github.9</code></td>
|
||||
<td>Medium</td>
|
||||
<td>Yes</td>
|
||||
<td>Yes</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Gitlab</td>
|
||||
<td>GitLab Private Token</td>
|
||||
<td><code>kingfisher.gitlab.1</code></td>
|
||||
|
|
@ -3463,7 +3487,7 @@ Of these, **605** include live validation and **57** support direct revocation.
|
|||
<td>Huawei</td>
|
||||
<td>Huawei Open Platform Client ID</td>
|
||||
<td><code>kingfisher.huawei.1</code></td>
|
||||
<td>Low</td>
|
||||
<td>Medium</td>
|
||||
<td></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
|
|
@ -3700,6 +3724,22 @@ Of these, **605** include live validation and **57** support direct revocation.
|
|||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Jira</td>
|
||||
<td>Jira Data Center Personal Access Token</td>
|
||||
<td><code>kingfisher.jira.3</code></td>
|
||||
<td>Medium</td>
|
||||
<td>Yes</td>
|
||||
<td>Yes</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Jira</td>
|
||||
<td>Jira Data Center Domain</td>
|
||||
<td><code>kingfisher.jira.4</code></td>
|
||||
<td>Medium</td>
|
||||
<td></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>Jotform</td>
|
||||
<td>Jotform API Key</td>
|
||||
<td><code>kingfisher.jotform.1</code></td>
|
||||
|
|
@ -7215,7 +7255,7 @@ Of these, **605** include live validation and **57** support direct revocation.
|
|||
<td>Webex</td>
|
||||
<td>Webex Integration Client ID</td>
|
||||
<td><code>kingfisher.webex.1</code></td>
|
||||
<td>Low</td>
|
||||
<td>Medium</td>
|
||||
<td></td>
|
||||
<td></td>
|
||||
</tr>
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
site_name: Kingfisher
|
||||
site_url: https://mongodb.github.io/kingfisher
|
||||
site_description: >-
|
||||
Open source secret scanner with live validation. 942 detection rules,
|
||||
Open source secret scanner with live validation. 943 detection rules,
|
||||
blast radius mapping, credential revocation, and a browser-based
|
||||
report viewer that also imports Gitleaks and TruffleHog output.
|
||||
Built in Rust by MongoDB.
|
||||
|
|
@ -46,6 +46,22 @@ theme:
|
|||
|
||||
plugins:
|
||||
- search
|
||||
- blog:
|
||||
blog_dir: blog
|
||||
post_date_format: long
|
||||
post_url_format: "{date}/{slug}"
|
||||
post_excerpt: required
|
||||
archive: true
|
||||
categories: true
|
||||
pagination_per_page: 10
|
||||
authors: false
|
||||
- rss:
|
||||
match_path: blog/posts/.*
|
||||
date_from_meta:
|
||||
as_creation: date
|
||||
categories:
|
||||
- categories
|
||||
- tags
|
||||
- minify:
|
||||
minify_html: true
|
||||
|
||||
|
|
@ -99,6 +115,8 @@ nav:
|
|||
- Python Bindings: reference/python-bindings.md
|
||||
- Benchmarks & Comparison: reference/comparison.md
|
||||
- Report Viewer: https://mongodb.github.io/kingfisher/viewer/
|
||||
- Blog:
|
||||
- blog/index.md
|
||||
- Changelog: changelog.md
|
||||
|
||||
extra:
|
||||
|
|
|
|||
|
|
@ -36,7 +36,7 @@
|
|||
<section class="kf-stats">
|
||||
<div class="kf-stats__inner md-grid">
|
||||
<div class="kf-stats__item">
|
||||
<span class="kf-stats__number">942</span>
|
||||
<span class="kf-stats__number">943</span>
|
||||
<span class="kf-stats__label">Detection Rules</span>
|
||||
</div>
|
||||
<div class="kf-stats__item">
|
||||
|
|
@ -48,8 +48,8 @@
|
|||
<span class="kf-stats__label">Scan Targets</span>
|
||||
</div>
|
||||
<div class="kf-stats__item">
|
||||
<span class="kf-stats__number">34</span>
|
||||
<span class="kf-stats__label">Revocation Providers</span>
|
||||
<span class="kf-stats__number">49</span>
|
||||
<span class="kf-stats__label">Rules with Revocation</span>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
|
@ -90,7 +90,7 @@
|
|||
<div class="kf-feature">
|
||||
<h3>Direct Revocation</h3>
|
||||
<p>
|
||||
Revoke compromised credentials directly from the CLI for 34 provider families
|
||||
Revoke compromised credentials directly from the CLI for 29 provider families
|
||||
including GitHub, GitLab, Slack, AWS, GCP, Heroku, and Cloudflare.
|
||||
</p>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -7,7 +7,7 @@
|
|||
"@context": "https://schema.org",
|
||||
"@type": "SoftwareApplication",
|
||||
"name": "Kingfisher",
|
||||
"description": "Open source secret scanner with live validation. 942 detection rules, blast radius mapping, and credential revocation.",
|
||||
"description": "Open source secret scanner with live validation. 943 detection rules, blast radius mapping, and credential revocation.",
|
||||
"applicationCategory": "DeveloperApplication",
|
||||
"operatingSystem": "Linux, macOS, Windows",
|
||||
"license": "https://opensource.org/licenses/Apache-2.0",
|
||||
|
|
|
|||
|
|
@ -1,5 +1,6 @@
|
|||
mkdocs-material>=9.5
|
||||
mkdocs-minify-plugin>=0.8
|
||||
mkdocs-rss-plugin>=1.6
|
||||
pillow>=10.0
|
||||
cairosvg>=2.7
|
||||
pyyaml>=6.0
|
||||
|
|
|
|||
|
|
@ -69,8 +69,8 @@ def generate_markdown(rules):
|
|||
total = len(rules)
|
||||
detectors = sum(1 for r in rules if not r["dependent"])
|
||||
dependent = total - detectors
|
||||
validated = sum(1 for r in rules if r["validates"])
|
||||
revocable = sum(1 for r in rules if r["revokes"])
|
||||
validated = sum(1 for r in rules if r["validates"] and not r["dependent"])
|
||||
revocable = sum(1 for r in rules if r["revokes"] and not r["dependent"])
|
||||
providers = len(set(r["provider"] for r in rules))
|
||||
|
||||
lines = [
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue