Commit graph

209 commits

Author SHA1 Message Date
Mick Grove
e0587f12ba cleanup' 2025-12-05 10:26:58 -08:00
Mick Grove
078fa16e6a - Reduced per-match memory usage by compacting stored source locations and interning repeated capture names.
- Stored optional validation response bodies as boxed strings to avoid allocating empty payloads and to streamline validator caches.
- Parallelized git cloning based on the configured job count and begin scanning repositories as soon as each clone finishes to reduce end-to-end scan times.
- Combined per-repository results into a single aggregate summary after scans complete.
- Added initial access-map support and report viewer html file. Currently beta features.
2025-12-04 22:02:30 -08:00
Mick Grove
2f69219b5c Fixed Bitbucket authenticated cloning bug 2025-11-24 23:43:11 -08:00
Mick Grove
d15afe7cea updated tests 2025-11-24 12:49:06 -08:00
Mick Grove
eeafe2fe6b updated tests 2025-11-24 11:08:31 -08:00
Mick Grove
0acaaa0680 Updated Summary to include scan date, kingfisher version ran, and latest kingfisher version available 2025-11-24 10:36:58 -08:00
Mick Grove
ec1e3a3432 - Fixed deduplication to consider rule identifiers so overlapping patterns are not merged before validation
- Moved 'URI with Username and Secret' to a low-confidence rule
2025-11-23 11:32:27 -08:00
Mick Grove
ae01a24414 Added checksum to GitLab rule 2025-11-21 12:33:10 -08:00
Mick Grove
116aac2f83 - Updating to support Bitbucket App Passwords
- Improved boundaries for several rules
- Added more rules
2025-11-20 16:34:13 -08:00
Mick Grove
17e0ca3594 - Updating to support Bitbucket App Passwords
- Improved boundaries for several rules
- Added more rules
2025-11-20 16:33:28 -08:00
Mick Grove
c6b10f0b47 - Skip reporting MongoDB and Postgres findings when their connection strings cannot be parsed, even when validation is disabled.
- Improve MySQL detection by broadening URI coverage and adding live validation that skips clearly invalid connection strings.
2025-11-16 23:25:42 -08:00
Mick Grove
377a220f89 - Skip reporting MongoDB and Postgres findings when their connection strings cannot be parsed, even when validation is disabled.
- Improve MySQL detection by broadening URI coverage and adding live validation that skips clearly invalid connection strings.
2025-11-15 08:46:53 -08:00
Mick Grove
8d77f9d298 - Skip reporting MongoDB and Postgres findings when their connection strings cannot be parsed, even when validation is disabled.
- Improve MySQL detection by broadening URI coverage and adding live validation that skips clearly invalid connection strings.
2025-11-15 08:43:54 -08:00
Mick Grove
f9d75eaadd - Skip reporting MongoDB and Postgres findings when their connection strings cannot be parsed, even when validation is disabled.
- Improve MySQL detection by broadening URI coverage and adding live validation that skips clearly invalid connection strings.
2025-11-15 08:11:25 -08:00
Mick Grove
426acac892 Filter out empty 'KF_BITBUCKET_*' environment values when constructing the Bitbucket authentication configuration so blank variables no longer override valid credentials 2025-11-13 14:30:27 -08:00
Mick Grove
c399a7a9d1 added jdbc rule and validator 2025-11-12 22:58:31 -08:00
Mick Grove
2ed94f75d7 added jdbc rule and validator 2025-11-12 22:25:33 -08:00
Mick Grove
d6c1dfc9d0 updated allocator 2025-11-11 13:24:06 -08:00
Mick Grove
97fbcef1ba updated allocator 2025-11-10 21:32:52 -08:00
Mick Grove
12eda3141a updated allocator 2025-11-10 21:24:48 -08:00
Mick Grove
dca955a95c v1.63.0 2025-11-10 18:47:51 -08:00
Mick Grove
da2fb6700d changes in response to code review 2025-11-09 09:16:50 -08:00
Mick Grove
b5d964d5b0 updated confluent rule with a checksum. Added zuplo rule with a checksum 2025-11-09 08:47:19 -08:00
Mick Grove
4efdcc35e1 updated confluent rule with a checksum. Added zuplo rule with a checksum 2025-11-09 08:42:16 -08:00
Mick Grove
94a51c3d04 updated confluent rule with a checksum. Added zuplo rule with a checksum 2025-11-08 16:01:58 -08:00
Mick Grove
7eb7be72cd fixing rules 2025-11-08 15:03:53 -08:00
Mick Grove
8aac161603 fixing rules 2025-11-08 10:48:00 -08:00
Mick Grove
ccbbbad5bc Added checksum comparisons to pattern_requirements, new suffix, crc32, and base62 Liquid filters, and verbose logging so mismatched checksums are skipped with context rather than reported as findings. 2025-11-07 16:31:24 -08:00
Mick Grove
f606f59f93 Added an optional exclude_words list to PatternRequirements so matches containing case-insensitive placeholder words are filtered out, with accompanying tests to cover the new behavior. 2025-11-05 17:19:11 -08:00
Mick Grove
6a5a62a5b5 Added an optional exclude_words list to PatternRequirements so matches containing case-insensitive placeholder words are filtered out, with accompanying tests to cover the new behavior. 2025-11-04 13:07:24 -08:00
Mick Grove
bd8bc09d0e Added an optional exclude_words list to PatternRequirements so matches containing case-insensitive placeholder words are filtered out, with accompanying tests to cover the new behavior. 2025-11-04 14:15:04 -05:00
Mick Grove
a3e426e6ee Automatically set --no-dedup whenever --manage-baseline is supplied so baseline management retains every occurrence of a finding 2025-11-04 14:06:35 -05:00
Mick Grove
0f953f59a5 pattern_requirements for rules — Post-regex character-class gating to cut false positives without lookarounds. Authors can now require minimum counts of digits, uppercase, lowercase, and special characters, with an optional custom special-char set.
Why: Hyperscan doesn’t support lookaheads/behinds, so many “must contain X and Y” checks had to be baked into the regex (hurting readability) or were impossible. pattern_requirements applies lightweight, in-memory checks after a match is found, keeping patterns fast and clean.
2025-11-04 13:55:31 -05:00
Mick Grove
ca3f175427 updated for v1.61.0 2025-10-30 22:50:41 -07:00
Mick Grove
cb22388bd1 updated smoke_branch tests 2025-10-26 11:53:29 -07:00
Mick Grove
4ee15b9363 - Fixed local filesystem scans to keep open_path_as_is enabled when opening Git repositories and only disable it for diff-based scans.
- Created Linux and Windows specific installer script
- Updated diff-focused scanning so --branch-root-commit can be provided alongside --branch, letting you diff from a chosen commit while targeting a specific branch tip (still defaulting back to the --branch ref when the commit is omitted).
2025-10-25 17:13:16 -07:00
Mick Grove
7d9d3be132 - Fixed local filesystem scans to keep open_path_as_is enabled when opening Git repositories and only disable it for diff-based scans.
- Created Linux and Windows specific installer script
- Updated diff-focused scanning so --branch-root-commit can be provided alongside --branch, letting you diff from a chosen commit while targeting a specific branch tip (still defaulting back to the --branch ref when the commit is omitted).
2025-10-25 17:12:51 -07:00
Mick Grove
1dcaa21bb5 Fixed bug in test when run on Windows 2025-10-23 22:04:01 -07:00
Mick Grove
9f91cbdab6 added tests for --branch and --since-commit feature 2025-10-23 17:02:31 -07:00
Mick Grove
98333a4bda updated anthropic rule 2025-10-23 15:02:30 -07:00
Mick Grove
c3b675c132 - Added provider-specific kingfisher scan subcommands (for example kingfisher scan github …) that translate into the legacy flags under the hood. The new layout keeps backwards compatibility while removing the wall of provider options from kingfisher scan --help.
- Updated the README so every provider example (GitHub, GitLab, Bitbucket, Azure Repos, Gitea, Hugging Face, Slack, Jira, Confluence, S3, GCS, Docker) uses the new subcommand style.
- Restored the direct kingfisher scan /path/to/dir flow for local filesystem scans while adding a --list-only switch to each provider subcommand so repository enumeration no longer requires the standalone github repos, gitlab repos, etc. commands.
- Removed the legacy top-level provider commands (kingfisher github, kingfisher gitlab, kingfisher gitea, kingfisher bitbucket, kingfisher azure, kingfisher huggingface) now that enumeration lives under kingfisher scan <provider> --list-only.
- Fixed kingfisher scan github … (and other provider-specific subcommands) so they no longer demand placeholder path arguments before the CLI accepts the request.
- Removed the --bitbucket-username, --bitbucket-token, and --bitbucket-oauth-token flags in favour of KF_BITBUCKET_* environment variables when authenticating to Bitbucket.
2025-10-22 16:30:32 -07:00
Mick Grove
1b181a368a - Added provider-specific kingfisher scan subcommands (for example kingfisher scan github …) that translate into the legacy flags under the hood. The new layout keeps backwards compatibility while removing the wall of provider options from kingfisher scan --help.
- Updated the README so every provider example (GitHub, GitLab, Bitbucket, Azure Repos, Gitea, Hugging Face, Slack, Jira, Confluence, S3, GCS, Docker) uses the new subcommand style.
- Restored the direct kingfisher scan /path/to/dir flow for local filesystem scans while adding a --list-only switch to each provider subcommand so repository enumeration no longer requires the standalone github repos, gitlab repos, etc. commands.
- Removed the legacy top-level provider commands (kingfisher github, kingfisher gitlab, kingfisher gitea, kingfisher bitbucket, kingfisher azure, kingfisher huggingface) now that enumeration lives under kingfisher scan <provider> --list-only.
- Fixed kingfisher scan github … (and other provider-specific subcommands) so they no longer demand placeholder path arguments before the CLI accepts the request.
- Removed the --bitbucket-username, --bitbucket-token, and --bitbucket-oauth-token flags in favour of KF_BITBUCKET_* environment variables when authenticating to Bitbucket.
2025-10-22 16:24:09 -07:00
amone
47027d2ce5 update: just sort and dedup once 2025-10-21 16:36:05 +08:00
Mick Grove
122885199d - Fixed kingfisher scan so that providing --branch without --since-commit now diffs the branch against the empty tree and scans every commit reachable from that branch.
- Added rules for meraki, duffel, finnhub, frameio, freshbooks, gitter, infracost, launchdarkly, lob, maxmind, messagebird, nytimes, prefect, salingo, sendinblue, sentry, shippo, twitch, typeform
2025-10-20 18:23:12 -07:00
Mick Grove
65a00e2705 change in response to code review 2025-10-16 10:20:18 -07:00
Mick Grove
03d7364888 - Added first-class Hugging Face scanning support, including CLI enumeration, token authentication, and integration with remote scans.
- Condensed GitError formatting to report the exit status and the first informative lines from stdout/stderr, producing concise git clone failure logs.
- Added support for scanning Google Cloud Storage buckets via --gcs-bucket, including optional prefixes and service-account authentication.
- Added --skip-aws-account (now accepting comma-separated values) and --skip-aws-account-file to bypass live AWS validation for known canary/honey-token account IDs without triggering alerts. Kingfisher now ships with several canary AWS account IDs pre-seeded in the skip list and now reports matching findings as "Not Attempted" with the "Response" containing "(skip list entry)" so its clear that validation was intentionally skipped and why.
2025-10-15 22:47:40 -07:00
Mick Grove
b4073855f2 kingfisher:ignore is only directive built-in 2025-10-11 18:04:00 -07:00
Mick Grove
9f13727666 kingfisher:ignore is only directive built-in 2025-10-11 15:27:21 -07:00
Mick Grove
4c952bf1bf Respect user color settings in update messages by using the same color helper as the main reporter, ensuring consistent output and no ANSI codes on update check, when color is disabled 2025-10-11 12:36:35 -07:00
Mick Grove
3647d759a3 - Added a --no-ignore CLI flag to disable inline directives when you need every potential secret reported
- Added: repeatable --ignore-comment <TOKEN> flag to reuse inline directives from other scanners (for example NOSONAR,
  kics-scan ignore, gitleaks:allow, etc)
2025-10-10 16:23:41 -07:00