From 81574833f7a8a488dd2c25a6497ddda0f545c5c1 Mon Sep 17 00:00:00 2001 From: Mick Grove Date: Sun, 5 Oct 2025 16:37:15 -0700 Subject: [PATCH] Updated README --- README.md | 47 +++++++++++---------- docs/assets/icons/aws-s3.svg | 34 +++++++++++++++ docs/assets/icons/azure-devops.svg | 1 + docs/assets/icons/bitbucket.svg | 15 +++++++ docs/assets/icons/confluence.svg | 1 + docs/assets/icons/docker.svg | 1 + docs/assets/icons/files.svg | 67 ++++++++++++++++++++++++++++++ docs/assets/icons/gitea.svg | 1 + docs/assets/icons/github.svg | 3 ++ docs/assets/icons/gitlab.svg | 1 + docs/assets/icons/jira.svg | 1 + docs/assets/icons/local-git.svg | 1 + docs/assets/icons/slack.svg | 6 +++ 13 files changed, 157 insertions(+), 22 deletions(-) create mode 100644 docs/assets/icons/aws-s3.svg create mode 100644 docs/assets/icons/azure-devops.svg create mode 100644 docs/assets/icons/bitbucket.svg create mode 100644 docs/assets/icons/confluence.svg create mode 100644 docs/assets/icons/docker.svg create mode 100644 docs/assets/icons/files.svg create mode 100644 docs/assets/icons/gitea.svg create mode 100644 docs/assets/icons/github.svg create mode 100644 docs/assets/icons/gitlab.svg create mode 100644 docs/assets/icons/jira.svg create mode 100644 docs/assets/icons/local-git.svg create mode 100644 docs/assets/icons/slack.svg diff --git a/README.md b/README.md index 55707a0..0825506 100644 --- a/README.md +++ b/README.md @@ -5,29 +5,23 @@ [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) -Kingfisher is a blazingly fast secret‑scanning and live validation tool built in Rust. It combines Intel’s hardware‑accelerated Hyperscan regex engine with language‑aware parsing via Tree‑Sitter, and **ships with hundreds of built‑in rules** to detect, validate, and triage secrets before they ever reach production +Kingfisher is a blazingly fast secret‑scanning and live validation tool built in Rust. It combines Intel’s hardware‑accelerated Hyperscan regex engine with language‑aware source code parsing, and **ships with hundreds of built‑in rules** to detect, validate, and triage secrets before they ever reach production

Originally forked from Praetorian’s Nosey Parker, Kingfisher **adds** live cloud-API validation; many more targets (GitLab, BitBucket, Gitea, S3, Docker, Jira, Confluence, Slack); compressed-file extraction and scanning; baseline and allowlist controls; language-aware detection (~20 languages); and a native Windows binary. See [Origins and Divergence](#origins-and-divergence) for details. - ## Key Features -- **Multiple Scan Targets**: -

- Files & Dirs - Local Git - GitHub - GitLab - Bitbucket - Gitea -
- Docker - Jira - Confluence - Slack - AWS S3 -

+### Multiple Scan Targets +| Files / Dirs | Local Git | GitHub | GitLab | Azure DevOps | Bitbucket | Gitea | +|:-------------:|:----------:|:------:|:------:|:-------------:|:----------:|:------:| +| Files / Dirs
Files & Dirs | Local Git
Local Git | GitHub
GitHub | GitLab
GitLab | Azure DevOps
Azure DevOps | Bitbucket
Bitbucket | Gitea
Gitea | + +| Docker | Jira | Confluence | Slack | AWS S3 | +|:------:|:----:|:-----------:|:-----:|:------:| +| Docker
Docker | Jira
Jira | Confluence
Confluence | Slack
Slack | AWS S3
AWS S3 | + +### Performance, Accuracy, and Hundreds of Rules - **Performance**: multithreaded, Hyperscan‑powered scanning built for huge codebases - **Extensible rules**: hundreds of built-in detectors plus YAML-defined custom rules ([docs/RULES.md](/docs/RULES.md)) - **Broad AI SaaS coverage**: finds and validates tokens for OpenAI, Anthropic, Google Gemini, Cohere, Mistral, Stability AI, Replicate, xAI (Grok), Ollama, Langchain, Perplexity, Weights & Biases, Cerebras, Friendli, Fireworks.ai, NVIDIA NIM, Together.ai, Zhipu, and many more @@ -46,6 +40,8 @@ See ([docs/COMPARISON.md](docs/COMPARISON.md)) - [Kingfisher](#kingfisher) - [Key Features](#key-features) + - [Multiple Scan Targets](#multiple-scan-targets) + - [Performance, Accuracy, and Hundreds of Rules](#performance-accuracy-and-hundreds-of-rules) - [Benchmark Results](#benchmark-results) - [Getting Started](#getting-started) - [Installation](#installation) @@ -79,6 +75,11 @@ See ([docs/COMPARISON.md](docs/COMPARISON.md)) - [Skip specific GitLab projects during enumeration](#skip-specific-gitlab-projects-during-enumeration) - [Scan remote GitLab repository by URL](#scan-remote-gitlab-repository-by-url) - [List GitLab repositories](#list-gitlab-repositories) + - [Scanning Azure Repos](#scanning-azure-repos) + - [Scan Azure DevOps organization or collection (requires `KF_AZURE_TOKEN` or `KF_AZURE_PAT`)](#scan-azure-devops-organization-or-collection-requires-kf_azure_token-or-kf_azure_pat) + - [Scan specific Azure DevOps projects](#scan-specific-azure-devops-projects) + - [Skip specific Azure repositories during enumeration](#skip-specific-azure-repositories-during-enumeration) + - [List Azure repositories](#list-azure-repositories) - [Scanning Gitea](#scanning-gitea) - [Scan Gitea organization (requires `KF_GITEA_TOKEN`)](#scan-gitea-organization-requires-kf_gitea_token) - [Scan Gitea user](#scan-gitea-user) @@ -1015,14 +1016,16 @@ kingfisher scan --help Kingfisher began as a fork of Praetorian’s Nosey Parker, as our experiment with adding live validation support and embedding that validation directly inside each rule. Since that initial fork, it has diverged heavily from Nosey Parker: -- Replaced the SQLite datastore with an in-memory store + Bloom filter -- Collapsed the workflow into a single scan-and-report phase with direct JSON/BSON/SARIF outputs -- Added Tree-Sitter parsing on top of Hyperscan for deeper language-aware detection -- Removed datastore-driven reporting/annotations in favor of live validation, baselines, allowlists, and compressed-file extraction +- Added support for live validation of discovered secrets +- Added hundreds of new rules +- Added support for analyzing compressed files +- Added support for building "baselines" to allow for only reporting on newly discovered secrets +- Added Tree-Sitter based source code parsing on top of Hyperscan for deeper language-aware detection - Expanded support for new targets (GitLab, BitBucket, Gitea, Jira, Confluence, Slack, S3, Docker, etc.) +- Replaced the SQLite datastore with an in-memory store + Bloom filter +- Collapsed the workflow into a single scan-and-report phase with direct JSON/BSON/SARIF outputs - Delivered cross-platform builds, including native Windows - # Roadmap - More rules diff --git a/docs/assets/icons/aws-s3.svg b/docs/assets/icons/aws-s3.svg new file mode 100644 index 0000000..3f63be5 --- /dev/null +++ b/docs/assets/icons/aws-s3.svg @@ -0,0 +1,34 @@ + + + + + + + + + + + + + + + + + + diff --git a/docs/assets/icons/azure-devops.svg b/docs/assets/icons/azure-devops.svg new file mode 100644 index 0000000..4e4773e --- /dev/null +++ b/docs/assets/icons/azure-devops.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/icons/bitbucket.svg b/docs/assets/icons/bitbucket.svg new file mode 100644 index 0000000..38af1ce --- /dev/null +++ b/docs/assets/icons/bitbucket.svg @@ -0,0 +1,15 @@ + + + + + + + + Bitbucket-blue + + + + + + + \ No newline at end of file diff --git a/docs/assets/icons/confluence.svg b/docs/assets/icons/confluence.svg new file mode 100644 index 0000000..22249e1 --- /dev/null +++ b/docs/assets/icons/confluence.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/icons/docker.svg b/docs/assets/icons/docker.svg new file mode 100644 index 0000000..0a9c6b0 --- /dev/null +++ b/docs/assets/icons/docker.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/icons/files.svg b/docs/assets/icons/files.svg new file mode 100644 index 0000000..1ebd008 --- /dev/null +++ b/docs/assets/icons/files.svg @@ -0,0 +1,67 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/assets/icons/gitea.svg b/docs/assets/icons/gitea.svg new file mode 100644 index 0000000..7ed0012 --- /dev/null +++ b/docs/assets/icons/gitea.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/icons/github.svg b/docs/assets/icons/github.svg new file mode 100644 index 0000000..a8d1174 --- /dev/null +++ b/docs/assets/icons/github.svg @@ -0,0 +1,3 @@ + + + diff --git a/docs/assets/icons/gitlab.svg b/docs/assets/icons/gitlab.svg new file mode 100644 index 0000000..abe3f37 --- /dev/null +++ b/docs/assets/icons/gitlab.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/icons/jira.svg b/docs/assets/icons/jira.svg new file mode 100644 index 0000000..57a68f0 --- /dev/null +++ b/docs/assets/icons/jira.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/icons/local-git.svg b/docs/assets/icons/local-git.svg new file mode 100644 index 0000000..994fb2c --- /dev/null +++ b/docs/assets/icons/local-git.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/assets/icons/slack.svg b/docs/assets/icons/slack.svg new file mode 100644 index 0000000..fb55f72 --- /dev/null +++ b/docs/assets/icons/slack.svg @@ -0,0 +1,6 @@ + + + + + +