eblume/kingfisher
1
0
Fork 0
forked from mirrors/kingfisher
Code Pull requests Activity Actions
1,572 commits 5 branches 83 tags 53 MiB
Commit graph

271 commits

Author SHA1 Message Date
Mick Grove
7fc01e5aca fixing bugs 2026-05-28 18:39:45 -07:00
Mick Grove
7d4719c7c5 add docker --archive support 2026-05-28 15:50:15 -07:00
Mick Grove
816a75e3e4 add docker --archive support 2026-05-28 13:54:59 -07:00
Mick Grove
0552d67df7 Authored Devin / Cognition rule 2026-05-27 17:55:32 -07:00
Mick Grove
207174e1a8 merged 2 PRs and updated changelog 2026-05-22 12:37:37 -04:00
Mick Grove
138eefe2b9 Fixed failed to spawn thread: Os { code: 11, kind: WouldBlock } panics during validation-heavy scans. Kingfisher built two Tokio runtimes (main + artifact-fetcher) that each defaulted to 512 blocking threads, which combined with Rayon pools and per-call spawns could exceed the OS per-user thread limit (RLIMIT_NPROC, default 8000 on macOS). Both runtimes now cap their blocking pools at max(num_jobs * 8, 32), and on Unix the soft RLIMIT_NPROC is raised to the hard limit at startup so users don't need to tune ulimit -u manually. 2026-05-22 11:50:47 -04:00
Mick Grove
1636b07810 preparing for v1.100.0 2026-05-18 09:42:04 -07:00
Mick Grove
31663b03b5 Release binary trimmed from 34 MB to 26 MB (~24% smaller). Switched jsonwebtoken to its rust_crypto backend (eliminates our scanner's pull on aws-lc-rs), bumped workspace hmac 0.12→0.13, sha1 0.10→0.11, sha2 0.10→0.11 to deduplicate our internal crypto code with the AWS sigv4 side, and migrated affected call sites in kingfisher-core, kingfisher-rules, and kingfisher-scanner to the digest-0.11 API (hex::encode for hex digests, explicit KeyInit import for HMAC). 2026-05-07 13:46:17 -07:00
Mick Grove
34b5c48888 - Archive scanning now reaches inside Android/iOS app packages: added apk, aab, and ipa to the recognized ZIP-based archive formats so secrets embedded in APK/AAB/IPA contents (e.g. classes*.dex, res/values/strings.xml) are extracted and matched. -- 
- Git repository scans now extract archive blobs encountered in the object database, not just on the filesystem. Previously a .zip/.jar/.apk/.tar.gz committed to a repo was scanned as raw compressed bytes, so secrets inside it were invisible. The git enumerator fans each archive entry out as a synthetic blob with the original commit metadata. Honors --no-extract-archives for opt-out.
- Performance: ZIP-based git blobs ≤ 64 MB extract entirely in memory (no temp-file round trip), beating the v1.99.0 baseline by ~15% on a 80 GiB monorepo despite scanning ~300K additional archive-content blobs. Larger archives auto-fall-back to a disk-streaming extractor.
- Memory safety: hard caps on archive extraction — 64 MB compressed pre-flight, 256 MB aggregate decompressed per archive (in-memory and disk paths), 512 MB per entry, plus a PK\x03\x04 magic-byte gate. Worst-case footprint is bounded at ~num_jobs * 320 MB.
 2026-05-06 17:50:35 -07:00
Mick Grove
394d05dd4d preparing for v1.99.0 2026-05-04 23:10:16 -07:00
Mick Grove
e30a7539b2 preparing for v1.99.0 2026-05-04 17:22:21 -07:00
Mick Grove
f6e05f0211 preparing for v1.99.0 2026-05-04 13:26:11 -07:00
Mick Grove
b2287c99ee --self-update (alias --update) on a scan or other command now **re-execs into the freshly installed binary** so the current invocation completes with the new code and the latest detection rules. Previously the on-disk binary was replaced but the running process kept using the old in-memory version, requiring a second invocation to pick up the changes. On Unix this is a true exec() (same PID); on Windows the new binary is spawned and the parent exits with its status code. The explicit kingfisher self-update subcommand still updates and exits without re-execing. Self-update now also covers Windows arm64 (the asset was already published; the runtime cfg map gained the missing arm). See docs/ADVANCED.md → *Update Checks*. 2026-05-01 20:14:27 -07:00
Mick Grove
1619737e2c improved access map viewer 2026-04-30 18:11:10 -07:00
Mick Grove
20e08105cf improved github organization scanning 2026-04-30 16:40:43 -07:00
Mick Grove
30b9eba427 copilot fixes 2026-04-29 22:50:31 -07:00
Mick Grove
8d9f5bed40 Added first-class **Postman** scanning target: new kingfisher scan postman subcommand (and equivalent --postman-* flags) fetches workspaces, collections, and environments via the Postman API and scans them for hard-coded credentials in request auth blocks, pre-request/test scripts, saved example responses, and — notably — secret-typed environment variables, which the API returns in plaintext despite the UI mask. Selectors: --workspace, --collection, --environment, --all, with optional --include-mocks-monitors and --api-url for self-hosted endpoints. Authenticates via KF_POSTMAN_TOKEN (or POSTMAN_API_KEY) sent as X-Api-Key; honors X-RateLimit-RetryAfter on 429s. Findings link back to https://go.postman.co/... URLs in reports. 2026-04-29 08:58:11 -07:00
Mick Grove
997480ffc7 Added first-class **Postman** scanning target: new kingfisher scan postman subcommand (and equivalent --postman-* flags) fetches workspaces, collections, and environments via the Postman API and scans them for hard-coded credentials in request auth blocks, pre-request/test scripts, saved example responses, and — notably — secret-typed environment variables, which the API returns in plaintext despite the UI mask. Selectors: --workspace, --collection, --environment, --all, with optional --include-mocks-monitors and --api-url for self-hosted endpoints. Authenticates via KF_POSTMAN_TOKEN (or POSTMAN_API_KEY) sent as X-Api-Key; honors X-RateLimit-RetryAfter on 429s. Findings link back to https://go.postman.co/... URLs in reports. 2026-04-29 08:12:08 -07:00
Mick Grove
bf6c7da4a4 added blog posts 2026-04-28 15:28:48 -07:00
Mick Grove
5465d903cf added kingfisher.github.9 to detect the new ~520-character stateless GitHub App installation token format (ghs_<APP_ID>_<JWT>). The legacy 36-character ghs_ rule 2026-04-26 16:56:44 -07:00
Mick Grove
ceff3ab1c5 performance improvements and rule improvements 2026-04-24 00:23:50 -07:00
Mick Grove
d8e0a41fe8 performance improvements and rule improvements 2026-04-23 14:42:10 -07:00
Mick Grove
7ee1fd5163 performance improvements and rule improvements 2026-04-22 23:39:19 -07:00
Mick Grove
79139e49b8 - Fixed the HTML access-map viewer dark mode so charts redraw correctly on theme changes and follow the system color scheme until manually overridden. 
- Fixed [#344](https://github.com/mongodb/kingfisher/issues/344): baseline fingerprints no longer have to be hexadecimal. The fingerprint value emitted by scan output (JSON, JSONL, pretty, SARIF) can now be copied directly into a baseline file and will match on the next scan. --manage-baseline now writes fingerprints in decimal to match scan output, and legacy 16-char hex (and 0x-prefixed hex) entries continue to be accepted, so existing baseline files keep working unchanged.
 2026-04-20 17:54:51 -07:00
Mick Grove
c50b3ba292 performance improvements and rule improvements 2026-04-19 16:33:13 -07:00
Mick Grove
a13b175fc5 performance improvements and rule improvements 2026-04-19 14:50:11 -07:00
Mick Grove
74cad26aed performance improvements and rule improvements 2026-04-17 11:01:46 -07:00
Mick Grove
a27f90d619 performance improvements and rule improvements 2026-04-16 16:57:31 -07:00
Mick Grove
09961f6feb performance improvements and access map viewer improvements 2026-04-16 13:34:44 -07:00
Mick Grove
c89e527053 bug fix 2026-04-16 06:44:12 -07:00
Mick Grove
efa47ba140 updates to new rules 2026-04-15 14:37:26 -07:00
Mick Grove
6100eeb6b5 updated docs 2026-04-14 22:56:19 -07:00
Mick Grove
45e3933dfa added more rules + validators 2026-04-14 13:46:08 -07:00
Mick Grove
c7f7adb223 added more rules + validators 2026-04-14 12:52:27 -07:00
Mick Grove
4b89cd0606 cleaned up dependency tree 2026-04-13 21:44:45 -07:00
Mick Grove
d2008dc3b7 cleaned up dependency tree 2026-04-13 20:43:09 -07:00
Mick Grove
365422a819 fixed performance regression 2026-04-09 22:21:02 -07:00
Mick Grove
2de703105f fixed performance regression 2026-04-09 21:06:51 -07:00
Mick Grove
0cb854872b Replaced tree-sitter with a lighter parser-based context verifier built from handwritten lexers plus tl/cssparser, preserving context-dependent matching while cutting about 19 MB from the release binary. 2026-04-07 23:20:17 -07:00
Mick Grove
afee0b7181 updated rules 2026-04-07 10:42:44 -07:00
Mick Grove
45a565fa6e added more rules 2026-04-06 22:18:58 -07:00
Mick Grove
372b0e579e more rules 2026-04-03 21:35:28 -07:00
Mick Grove
c171704884 updated vectorscan 2026-04-02 19:35:30 -07:00
Mick Grove
d42620919f updated dependencies 2026-04-01 14:58:08 -07:00
Mick Grove
13bad3f172 added more access-maps 2026-04-01 13:39:24 -07:00
Mick Grove
19fe52a9bf added more access-maps 2026-04-01 10:20:52 -07:00
Mick Grove
fc542afa99 fixed github actions 2026-03-29 17:08:58 -07:00
Mick Grove
482a60bb9d fixed github actions 2026-03-29 10:41:54 -07:00
Mick Grove
b9da8e2829 added more rules 2026-03-29 08:19:34 -07:00
Mick Grove
e0a403607f updated in response to ossf scorecard 2026-03-27 22:26:35 -07:00