eblume/kingfisher
1
0
Fork 0
forked from mirrors/kingfisher
Code Pull requests Activity Actions
1,572 commits 5 branches 83 tags 53 MiB
Commit graph

421 commits

Author SHA1 Message Date
Mick Grove
31af4b4f6f updated docs 2026-05-28 21:01:44 -07:00
Mick Grove
7fc01e5aca fixing bugs 2026-05-28 18:39:45 -07:00
Mick Grove
7d4719c7c5 add docker --archive support 2026-05-28 15:50:15 -07:00
Mick Grove
816a75e3e4 add docker --archive support 2026-05-28 13:54:59 -07:00
Mick Grove
a0d2fa3611 merged 2 PRs and updated changelog 2026-05-22 13:15:59 -07:00
Mick Grove
bb7fea155e merged 2 PRs and updated changelog 2026-05-22 14:17:59 -04:00
Mick Grove
b1fb90b4c9
 		Merge pull request #387 from AgentEnder/codex/contain-validator-panics 
fix(validation): contain validator panics
 2026-05-22 12:34:17 -04:00
Mick Grove
138eefe2b9 Fixed failed to spawn thread: Os { code: 11, kind: WouldBlock } panics during validation-heavy scans. Kingfisher built two Tokio runtimes (main + artifact-fetcher) that each defaulted to 512 blocking threads, which combined with Rayon pools and per-call spawns could exceed the OS per-user thread limit (RLIMIT_NPROC, default 8000 on macOS). Both runtimes now cap their blocking pools at max(num_jobs * 8, 32), and on Unix the soft RLIMIT_NPROC is raised to the hard limit at startup so users don't need to tune ulimit -u manually. 2026-05-22 11:50:47 -04:00
Craigory Coppola
fd13f268f0 fix(validation): redact panic payloads and clarify panic handling 
Addresses review feedback on the validator panic-containment change:

- Keep raw panic payloads out of the cached and user-visible
  `validation_response_body`, since a panic message can embed secret
  material (e.g. a token captured in a debug string). The visible body
  now reports only the stable rule id, and the detailed payload is
  emitted via truncated structured logging.
- Replace the nested `Result<Result<(), String>, Elapsed>` with a
  self-describing `ValidationOutcome` enum (`Completed` / `Panicked` /
  `TimedOut`) so call sites and signatures read clearly.
- Document why the `AssertUnwindSafe` panic boundary is sound: the
  recovery path deterministically resets the match's validation fields,
  and the shared counters/cache are only mutated after the boundary
  returns, so an unwind cannot leave them inconsistent.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-05-21 23:02:57 -04:00
Craigory Coppola
d2e4e2f737 fix(validation): contain validator panics 2026-05-21 21:56:21 -04:00
Mick Grove
c67dcc049d preparing for v1.100.0 2026-05-18 23:39:05 -04:00
Mick Grove
1830a140d8 preparing for v1.100.0 2026-05-18 22:28:19 -04:00
Mick Grove
a8e01c4a6e preparing for v1.100.0 2026-05-18 18:33:42 -07:00
Mick Grove
a148a153ac preparing for v1.100.0 2026-05-18 18:12:27 -07:00
Mick Grove
d125d68e88 preparing for v1.100.0 2026-05-18 16:11:15 -07:00
Mick Grove
f1c6f50d9a preparing for v1.100.0 2026-05-18 15:51:16 -07:00
Mick Grove
b58eed2696 preparing for v1.100.0 2026-05-18 15:19:11 -07:00
Mick Grove
91d9f431c5 preparing for v1.100.0 2026-05-18 14:27:01 -07:00
Mick Grove
514832b533 preparing for v1.100.0 2026-05-18 14:13:30 -07:00
Mick Grove
0dedcef95f preparing for v1.100.0 2026-05-18 13:25:13 -07:00
Mick Grove
54d9fc7ecd preparing for v1.100.0 2026-05-18 13:03:16 -07:00
Mick Grove
1636b07810 preparing for v1.100.0 2026-05-18 09:42:04 -07:00
Mick Grove
31663b03b5 Release binary trimmed from 34 MB to 26 MB (~24% smaller). Switched jsonwebtoken to its rust_crypto backend (eliminates our scanner's pull on aws-lc-rs), bumped workspace hmac 0.12→0.13, sha1 0.10→0.11, sha2 0.10→0.11 to deduplicate our internal crypto code with the AWS sigv4 side, and migrated affected call sites in kingfisher-core, kingfisher-rules, and kingfisher-scanner to the digest-0.11 API (hex::encode for hex digests, explicit KeyInit import for HMAC). 2026-05-07 13:46:17 -07:00
Mick Grove
34b5c48888 - Archive scanning now reaches inside Android/iOS app packages: added apk, aab, and ipa to the recognized ZIP-based archive formats so secrets embedded in APK/AAB/IPA contents (e.g. classes*.dex, res/values/strings.xml) are extracted and matched. -- 
- Git repository scans now extract archive blobs encountered in the object database, not just on the filesystem. Previously a .zip/.jar/.apk/.tar.gz committed to a repo was scanned as raw compressed bytes, so secrets inside it were invisible. The git enumerator fans each archive entry out as a synthetic blob with the original commit metadata. Honors --no-extract-archives for opt-out.
- Performance: ZIP-based git blobs ≤ 64 MB extract entirely in memory (no temp-file round trip), beating the v1.99.0 baseline by ~15% on a 80 GiB monorepo despite scanning ~300K additional archive-content blobs. Larger archives auto-fall-back to a disk-streaming extractor.
- Memory safety: hard caps on archive extraction — 64 MB compressed pre-flight, 256 MB aggregate decompressed per archive (in-memory and disk paths), 512 MB per entry, plus a PK\x03\x04 magic-byte gate. Worst-case footprint is bounded at ~num_jobs * 320 MB.
 2026-05-06 17:50:35 -07:00
Mick Grove
12c141bfac preparing for v1.99.0 2026-05-05 07:08:40 -07:00
Mick Grove
aca11be36d preparing for v1.99.0 2026-05-04 23:47:48 -07:00
Mick Grove
394d05dd4d preparing for v1.99.0 2026-05-04 23:10:16 -07:00
Mick Grove
bacdca6a52 preparing for v1.99.0 2026-05-04 19:00:45 -07:00
Mick Grove
b28f15252c preparing for v1.99.0 2026-05-04 18:03:29 -07:00
Mick Grove
e30a7539b2 preparing for v1.99.0 2026-05-04 17:22:21 -07:00
Mick Grove
a9cdaea6cd preparing for v1.99.0 2026-05-04 14:48:41 -07:00
Mick Grove
f6e05f0211 preparing for v1.99.0 2026-05-04 13:26:11 -07:00
Mick Grove
0e1fe0cede webhook support and kingfisher configuration yaml support 2026-05-03 23:10:45 -07:00
Mick Grove
a4cf3990a5 webhook support and kingfisher configuration yaml support 2026-05-03 22:11:26 -07:00
Mick Grove
44d67cea1b added SLSA provenance 2026-05-02 00:14:31 -07:00
Mick Grove
b2287c99ee --self-update (alias --update) on a scan or other command now **re-execs into the freshly installed binary** so the current invocation completes with the new code and the latest detection rules. Previously the on-disk binary was replaced but the running process kept using the old in-memory version, requiring a second invocation to pick up the changes. On Unix this is a true exec() (same PID); on Windows the new binary is spawned and the parent exits with its status code. The explicit kingfisher self-update subcommand still updates and exits without re-execing. Self-update now also covers Windows arm64 (the asset was already published; the runtime cfg map gained the missing arm). See docs/ADVANCED.md → *Update Checks*. 2026-05-01 20:14:27 -07:00
Mick Grove
1619737e2c improved access map viewer 2026-04-30 18:11:10 -07:00
Mick Grove
20e08105cf improved github organization scanning 2026-04-30 16:40:43 -07:00
Mick Grove
87f6bd818f copilot fixes 2026-04-30 11:40:22 -07:00
Mick Grove
b89c952043 copilot fixes 2026-04-30 11:28:45 -07:00
Mick Grove
cceab35ec1 copilot fixes 2026-04-30 10:56:35 -07:00
Mick Grove
90737f098c copilot fixes 2026-04-30 09:29:23 -07:00
Mick Grove
b7b6dfdeb2 copilot fixes 2026-04-30 09:02:49 -07:00
Mick Grove
06f72ec9f0 copilot fixes 2026-04-30 08:38:14 -07:00
Mick Grove
2c08659563 copilot fixes 2026-04-30 00:32:49 -07:00
Mick Grove
c94bd89195 copilot fixes 2026-04-29 23:42:33 -07:00
Mick Grove
30b9eba427 copilot fixes 2026-04-29 22:50:31 -07:00
Mick Grove
1337588c7b Added first-class **Postman** scanning target: new kingfisher scan postman subcommand (and equivalent --postman-* flags) fetches workspaces, collections, and environments via the Postman API and scans them for hard-coded credentials in request auth blocks, pre-request/test scripts, saved example responses, and — notably — secret-typed environment variables, which the API returns in plaintext despite the UI mask. Selectors: --workspace, --collection, --environment, --all, with optional --include-mocks-monitors and --api-url for self-hosted endpoints. Authenticates via KF_POSTMAN_TOKEN (or POSTMAN_API_KEY) sent as X-Api-Key; honors X-RateLimit-RetryAfter on 429s. Findings link back to https://go.postman.co/... URLs in reports. 2026-04-29 11:46:17 -07:00
Mick Grove
8d9f5bed40 Added first-class **Postman** scanning target: new kingfisher scan postman subcommand (and equivalent --postman-* flags) fetches workspaces, collections, and environments via the Postman API and scans them for hard-coded credentials in request auth blocks, pre-request/test scripts, saved example responses, and — notably — secret-typed environment variables, which the API returns in plaintext despite the UI mask. Selectors: --workspace, --collection, --environment, --all, with optional --include-mocks-monitors and --api-url for self-hosted endpoints. Authenticates via KF_POSTMAN_TOKEN (or POSTMAN_API_KEY) sent as X-Api-Key; honors X-RateLimit-RetryAfter on 429s. Findings link back to https://go.postman.co/... URLs in reports. 2026-04-29 08:58:11 -07:00
Mick Grove
997480ffc7 Added first-class **Postman** scanning target: new kingfisher scan postman subcommand (and equivalent --postman-* flags) fetches workspaces, collections, and environments via the Postman API and scans them for hard-coded credentials in request auth blocks, pre-request/test scripts, saved example responses, and — notably — secret-typed environment variables, which the API returns in plaintext despite the UI mask. Selectors: --workspace, --collection, --environment, --all, with optional --include-mocks-monitors and --api-url for self-hosted endpoints. Authenticates via KF_POSTMAN_TOKEN (or POSTMAN_API_KEY) sent as X-Api-Key; honors X-RateLimit-RetryAfter on 429s. Findings link back to https://go.postman.co/... URLs in reports. 2026-04-29 08:12:08 -07:00