eblume/kingfisher
1
0
Fork 0
forked from mirrors/kingfisher
Code Pull requests Activity Actions
1,308 commits 5 branches 83 tags 53 MiB
Commit graph

50 commits

Author SHA1 Message Date
Mick Grove
f681591ee8 changes in response to PR review 2026-03-18 17:19:30 -07:00
Mick Grove
f0a3bee587 added --max-validation-response-length <BYTES> 2026-03-16 22:25:32 -07:00
Mick Grove
fcac8cf1b7 rules updated 2026-03-03 16:47:59 -08:00
Mick Grove
5882468177 Added optional validation rate limiting via --validation-rps (global) and repeatable --validation-rps-rule <RULE_SELECTOR=RPS> (per-rule override) for both scan and validate. Throttling now applies across built-in validator types (HTTP/gRPC plus AWS, GCP, Coinbase, MongoDB, Postgres, MySQL, JDBC, JWT, and Azure Storage). Rule selectors support the short form (for example, github=2 matches kingfisher.github.*) with longest-prefix precedence when multiple selectors apply. 2026-02-12 12:33:59 -08:00
Mick Grove
2d6abb95c9 fixes in response to pr review 2026-02-11 23:44:09 -08:00
Mick Grove
4ab5932d57 - Added Vercel credential rules for new token formats introduced February 2026: vcp_ (personal access), vci_ (integration), vca_ (app access), vcr_ (app refresh), vck_ (AI Gateway API key). All use CRC32/Base62 checksum validation. Legacy 24-char format retained as kingfisher.vercel.1. 
- Added revocation support for Vercel app tokens (vca_, vcr_) via https://api.vercel.com/login/oauth/token/revoke. Requires VERCEL_APP_CLIENT_ID (or NEXT_PUBLIC_VERCEL_APP_CLIENT_ID) and VERCEL_APP_CLIENT_SECRET.
- Fixed validate/revoke command generation to omit regex named captures (e.g., BODY, CHECKSUM) when they are not used by validation/revocation templates, so rules like Vercel no longer produce unnecessary --var BODY=... arguments.
 2026-02-11 13:56:17 -08:00
Mick Grove
265e569c60 - Fixed validation flakiness under service rate limiting by retrying HTTP validations on 429/408 in addition to transient 5xx failures. 
- Prevented transient HTTP validation failures (429/5xx) from being cached, avoiding cache poisoning that could suppress later successful validations in the same scan.
 2026-02-11 11:38:24 -08:00
Mick Grove
4a74e95756 v1.81.0 2026-02-10 19:43:34 -08:00
Mick Grove
e518fb30f2 v1.81.0 2026-02-10 19:24:19 -08:00
Mick Grove
1a40fb3bfd Fixed AWS access key validation to support temporary/session keys (ASIA prefix) in addition to long-lived keys (AKIA prefix). 2026-02-06 17:05:32 -08:00
Mick Grove
5253204c2a preparing for v1.78.0 2026-02-02 23:22:08 -08:00
Mick Grove
63f1d515ae preparing for v1.78.0 2026-02-02 18:39:24 -08:00
Mick Grove
76be1df60c Refactored into multiple crates. Added the 'validate' subcommand 2026-01-28 10:27:24 -08:00
Mick Grove
38a0dd9e26 Switched compression dependencies to pure-Rust bzip2/lzma implementations and pared zip features to avoid C-based codecs for bz2/xz handling. 2026-01-23 10:45:08 -08:00
Mick Grove
b4feb86f47 - Fixed validation deduplication for rules with nested unnamed captures (e.g. (?<REGEX>...(ABC|DEF)...)) to use the primary capture for grouping, ensuring each unique match triggers a separate validation request. 
- Added trace-level (-vv) logging for internal validation dedup keys and grouping to aid debugging.
 2026-01-21 13:13:43 -08:00
Mick Grove
7237a931d5 v1.73.0 2026-01-01 22:24:57 -08:00
Mick Grove
587dfc5892 - Fixed deduplication for dependency-provider rules so dependent validations run per blob 
- Updated Artifactory rule entropy and added new artifactory rule
 2025-12-21 22:07:45 -08:00
Mick Grove
078fa16e6a - Reduced per-match memory usage by compacting stored source locations and interning repeated capture names. 
- Stored optional validation response bodies as boxed strings to avoid allocating empty payloads and to streamline validator caches.
- Parallelized git cloning based on the configured job count and begin scanning repositories as soon as each clone finishes to reduce end-to-end scan times.
- Combined per-repository results into a single aggregate summary after scans complete.
- Added initial access-map support and report viewer html file. Currently beta features.
 2025-12-04 22:02:30 -08:00
Mick Grove
c6b10f0b47 - Skip reporting MongoDB and Postgres findings when their connection strings cannot be parsed, even when validation is disabled. 
- Improve MySQL detection by broadening URI coverage and adding live validation that skips clearly invalid connection strings.
 2025-11-16 23:25:42 -08:00
Mick Grove
f9d75eaadd - Skip reporting MongoDB and Postgres findings when their connection strings cannot be parsed, even when validation is disabled. 
- Improve MySQL detection by broadening URI coverage and adding live validation that skips clearly invalid connection strings.
 2025-11-15 08:11:25 -08:00
Mick Grove
2ed94f75d7 added jdbc rule and validator 2025-11-12 22:25:33 -08:00
Mick Grove
d6c1dfc9d0 updated allocator 2025-11-11 13:24:06 -08:00
Mick Grove
dca955a95c v1.63.0 2025-11-10 18:47:51 -08:00
Mick Grove
da2fb6700d changes in response to code review 2025-11-09 09:16:50 -08:00
Mick Grove
7eb7be72cd fixing rules 2025-11-08 15:03:53 -08:00
Mick Grove
8aac161603 fixing rules 2025-11-08 10:48:00 -08:00
Mick Grove
ccbbbad5bc Added checksum comparisons to pattern_requirements, new suffix, crc32, and base62 Liquid filters, and verbose logging so mismatched checksums are skipped with context rather than reported as findings. 2025-11-07 16:31:24 -08:00
Mick Grove
cb22388bd1 updated smoke_branch tests 2025-10-26 11:53:29 -07:00
Mick Grove
03d7364888 - Added first-class Hugging Face scanning support, including CLI enumeration, token authentication, and integration with remote scans. 
- Condensed GitError formatting to report the exit status and the first informative lines from stdout/stderr, producing concise git clone failure logs.
- Added support for scanning Google Cloud Storage buckets via --gcs-bucket, including optional prefixes and service-account authentication.
- Added --skip-aws-account (now accepting comma-separated values) and --skip-aws-account-file to bypass live AWS validation for known canary/honey-token account IDs without triggering alerts. Kingfisher now ships with several canary AWS account IDs pre-seeded in the skip list and now reports matching findings as "Not Attempted" with the "Response" containing "(skip list entry)" so its clear that validation was intentionally skipped and why.
 2025-10-15 22:47:40 -07:00
Mick Grove
654f1ef41f Added a new CLI flag, --user-agent-suffix to allow developers to append additional information to the user-agent 2025-09-18 14:11:54 -07:00
Mick Grove
895dac63b8 updated user-agent 2025-09-10 16:13:28 -07:00
Mick Grove
3bfcc074f4 updated user-agent 2025-09-10 16:08:33 -07:00
Mick Grove
58c84d543e - Enabled MongoDB URI validation 
- AWS + GCP validators now respect HTTPS_PROXY and share a consistent user agent across AWS, GCP, and HTTP validation
 2025-09-09 22:35:17 -07:00
Mick Grove
6a1d9e4142 - Enabled MongoDB URI validation 
- AWS + GCP validators now respect HTTPS_PROXY and share a consistent user agent across AWS, GCP, and HTTP validation
 2025-09-09 16:45:02 -07:00
Mick Grove
c3513ea206 Optimized memory usage via string interning and extensive data sharing 2025-09-02 19:54:44 -07:00
Mick Grove
9de355a5c8 Decode Base64 blobs and scan their contents for secrets while skipping short strings for performance 2025-08-30 16:44:55 -07:00
Mick Grove
e54dbe90d0 - Improved rules: github oauth2, diffbot, mailchimp, aws 
- Added validation to SauceLabs rule
- Added rules: shodan, bitly, flickr
 2025-08-29 17:24:26 -07:00
Mick Grove
5b8e83f5e7 refactored rule loading 2025-08-15 13:13:33 -07:00
Mick Grove
46d0ecce3b - New rules: Telegram bot token, OpenWeatherMap, Apify 
- New OpenAI detectors added (@joshlarsen)
- Fixed bug that broke validation when using unnamed group captures
 2025-08-01 16:56:04 -07:00
Mick Grove
f48eeb79e2 Fixed validation caching for HTTP validators to include rendered headers so inactive secrets no longer appear active, in some cases 2025-08-01 09:15:24 -07:00
Mick Grove
e73aec9d70 - Fixed issue when more than 1 named capture group is used in a rule variable 
- Added 2 new liquid template filters: 'b64dec' and 'es256_sign'
- Added custom validator for Coinbase, and a Coinbase rule that uses it
 2025-07-31 16:52:50 -07:00
Mick Grove
9b4856d7d5 Fixed Gitlab support. Added pre-commit and pre-receive installation scripts. 2025-07-23 19:57:33 -07:00
Mick Grove
0830606260 change that hoists the redirect-free reqwest::Client into a single, lazily-initialized static so every call to validate_jwt re-uses the same handle (and therefore the same connection-pool, DNS cache, TLS session cache, etc) 2025-07-14 17:22:51 -07:00
Mick Grove
601ca05fc8 JWT validation performs OpenID Connect discovery using the iss claim and verifies signatures via JWKS 2025-07-14 15:31:44 -07:00
Mick Grove
cd4f626502 Added support for HTTP request bodies in rule validation. Added mistral and perplexity rule 2025-07-08 17:49:12 -07:00
Mick Grove
28af26b23a Introduced flag – skip files/dirs whose path resembles tests (, , , , ), reducing noise. 2025-06-28 09:16:42 -07:00
Mick Grove
87d2a83e3e Fix: HTML detection now requires both HTML content-type and html tag, fixing webhook false negatives 2025-06-27 15:28:34 -07:00
Mick Grove
18e0b3c9b4 Fixed malformed rules. Now validating that response_matcher is present in validation section of all rules 2025-06-25 23:29:46 -07:00
Mick Grove
0d3513b6f9 Fixed malformed rules. Now validating that response_matcher is present in validation section of all rules 2025-06-25 22:17:37 -07:00
Mick Grove
fc4aee9e41 preparing for v1.12 2025-06-24 17:17:16 -07:00