feat(newsroom): wire kong CLI app, health endpoints, and graceful shutdown (task 6.1) #286

Merged
prskr merged 6 commits from paperclip/ICB-159-app-wiring into main 2026-10-16 14:01:13 +00:00
Contributor

Implements task 6.1 of OpenSpec change newsroom (newsroom/tasks/16-app-wiring.md).

Changes

  • Kong CLI App (cmd/newsroom): Root app struct with env-only secret flags (DB_PASSWORD, LLM_API_KEY, READECK_TOKEN) and default address :8081.
  • Route Map & Cross-Origin Protection: Single serve mux wiring /health/*, /assets/*, UI pages, Atom routes, and /admin/*. Preserves PR #274 cross-origin protection across all unsafe UI requests.
  • Health Endpoints (internal/health): /health/readiness (pings database) and /health/liveness (lightweight, no db ping). Removed old /healthz.
  • Route Validation: Added reserved prefix validation to prevent collisions with /ui, /assets, /admin, /health, /feed, /provider, /entry, /briefings.
  • Graceful Shutdown: Intercepts interrupt signals, gracefully shuts down the HTTP server and scheduler, and closes database connections cleanly.
  • Deletions: Removed obsolete internal/server skeleton and /v1/enrich proxy handler.

Verification

  • bazel test //newsroom/... passed (22/22 tests pass).
  • go run ./newsroom/cmd/newsroom --help verified flags and secret env-vars.
  • go test ./newsroom/... and go vet ./newsroom/... passed cleanly.
  • Verified no symlinks exist in the repository tree.

Co-Authored-By: Paperclip noreply@paperclip.ing

Implements task 6.1 of OpenSpec change `newsroom` (`newsroom/tasks/16-app-wiring.md`). ### Changes - **Kong CLI App (`cmd/newsroom`)**: Root `app` struct with env-only secret flags (`DB_PASSWORD`, `LLM_API_KEY`, `READECK_TOKEN`) and default address `:8081`. - **Route Map & Cross-Origin Protection**: Single serve mux wiring `/health/*`, `/assets/*`, UI pages, Atom routes, and `/admin/*`. Preserves PR #274 cross-origin protection across all unsafe UI requests. - **Health Endpoints (`internal/health`)**: `/health/readiness` (pings database) and `/health/liveness` (lightweight, no db ping). Removed old `/healthz`. - **Route Validation**: Added reserved prefix validation to prevent collisions with `/ui`, `/assets`, `/admin`, `/health`, `/feed`, `/provider`, `/entry`, `/briefings`. - **Graceful Shutdown**: Intercepts interrupt signals, gracefully shuts down the HTTP server and scheduler, and closes database connections cleanly. - **Deletions**: Removed obsolete `internal/server` skeleton and `/v1/enrich` proxy handler. ### Verification - `bazel test //newsroom/...` passed (22/22 tests pass). - `go run ./newsroom/cmd/newsroom --help` verified flags and secret env-vars. - `go test ./newsroom/...` and `go vet ./newsroom/...` passed cleanly. - Verified no symlinks exist in the repository tree. Co-Authored-By: Paperclip <noreply@paperclip.ing>
feat(newsroom): wire kong CLI app, health endpoints, and graceful shutdown (task 6.1)
All checks were successful
Validate / validate (pull_request) Successful in 2m19s
0c948f27b5
- Implement modular kong-based root app struct in cmd/newsroom with env-only secret flags (DB_PASSWORD, LLM_API_KEY, READECK_TOKEN) and default address :8081
- Build single serve mux mounting /health/*, /assets/*, UI routes, Atom routes, and /admin/*
- Maintain cross-origin protection across full app handler for unsafe UI requests
- Add internal/health endpoints: /health/readiness (pings database) and /health/liveness (lightweight, no db ping)
- Add reserved-prefix collision test in internal/config
- Ensure clean graceful shutdown of HTTP server, scheduler, and database pool on context cancellation
- Remove obsolete internal/server package and synchronous extractor proxy handler
- Mark task 6.1 complete in openspec/changes/newsroom/tasks.md

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Author
Contributor

QA review: newsroom 6.1 App wiring (kong), PR #286 @ 0c948f2

Verdict: FAIL: 3 blocking findings. Most of the wiring is right: kong app, single mux, health split, reserved prefixes, cross-origin protection through the full app. It falls short on one acceptance criterion (os.Getenv). It also adds a process crash on a config that passes validation, and its shutdown does not respect its own time limit.

Success condition tested: newsroom/tasks/16-app-wiring.md acceptance criteria and the ICB-160 plan checks. Specs: newsroom-operations ("Readiness and liveness…", "Shutdown drains in-flight work") and newsroom-config ("Route namespace is collision-free", "Secrets come from the environment only").

Steps run

  1. /code-review (high) on origin/main...origin/paperclip/ICB-159-app-wiring (merge-base fe97611, 1 commit, 14 files). Every finding below was checked against the code or reproduced; unconfirmed ones were dropped.
  2. bazel test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore //newsroom/... → Executed 22 out of 22 tests: 22 tests pass.
  3. The same command with --test_output=all --test_arg=-test.v on //newsroom/cmd/newsroom:newsroom_integration_test, plus a temporary QA probe file (not committed; worktree deleted). The container tests really ran and were not skipped: TestIntegration_WiringRoutes, …CrossOriginProtectionThroughFullApp and …GracefulShutdown all PASS.
  4. bazel run --experimental_convenience_symlinks=ignore //newsroom/cmd/newsroom -- --help, plus the real binary run against configs with reserved routes.
  5. Greps for os.Getenv, ListenAndServe, CrossOriginProtection, AddTrustedOrigin, v1/enrich and curator; a diff of go.mod/MODULE.bazel; git diff --summary checked for mode 120000; Forgejo commit status.

Blocking (Linus)

B1. os.Getenv is still in newsroom/: cmd/newsroom/app.go:102 (resolveDSN).

  • Expected: no os.Getenv in newsroom/ (task acceptance criterion), and all secrets come through kong flags so that --help lists every secret.
  • Actual: grep -rn os.Getenv newsroom --include=*.go (non-test) → newsroom/cmd/newsroom/app.go:102: return os.Getenv(key). When --db.password is empty, the password comes from whatever variable database.passwordEnv names, and that variable never appears in --help. Also, injectPassword silently drops the password when the DSN has no user.
  • Fix: take the DB password only from c.Database.Password (DB_PASSWORD) and remove the os.Getenv fallback. If passwordEnv is set to anything other than DB_PASSWORD, fail at startup with a clear error, or ignore it; how passwordEnv changes long-term belongs to newsroom-cleanup. Return an error instead of silently dropping the password when the DSN has no user.

B2. summarize: true without llm.baseURL passes validation, then crashes the whole process (nil-interface panic).

  • Expected: a config error at startup, or a classified summarization failure. Never a crash.
  • Actual: app.go:252-255 only builds a summarizer when cfg.LLM.BaseURL != "". Otherwise enrich.New(store, extractor, nil), and validateSummarize does not require a base URL. Probe TestQAProbe_SummarizeWithoutBaseURL: buildHandler err=<nil>, then POST /admin/ingest/g/test →
    panic: runtime error: invalid memory address or nil pointer dereference
    enrich.(*Enricher).summarizeBody  newsroom/internal/enrich/enrich.go:258
    enrich.(*Enricher).Enrich         newsroom/internal/enrich/enrich.go:148
    ingest.(*Ingester).enrichKeptEntry newsroom/internal/ingest/ingest.go:485
    ingest.(*Ingester).processEntries.func1 newsroom/internal/ingest/ingest.go:297
    
    The panic happens in an ingest worker goroutine, so net/http does not recover it and the process exits. The same path runs from cron and from the always-registered backfill job.
  • Fix: fail startup when any provider has summarize: true (or summary backfill is active) and llm.baseURL is empty, or never pass a nil summarizer. Add a test for it.

B3. Shutdown is not bounded when an ingest is in flight, and a normal SIGINT exits non-zero.

  • Expected (spec): "wait — bounded — for in-flight ingest… shut down the HTTP server, and only then release the database pool", and "SIGINT shuts the service down cleanly".
  • Actual: cron and RunAll runs use context.Background()/WithoutCancel with a 2-minute route timeout (schedule.go:220,249,368). Run gives s.Stop and srv.Shutdown one shared 10s budget, one after the other, and nothing cancels the runs. Probe TestQAProbe_ShutdownOverrunsBudget (feed upstream hangs for 20s, POST /admin/ingest → 202, then cancel): Run returned after 20s: err=context deadline exceeded. That is twice the 10s limit, because the deferred store.Close() waits for the run that is still holding a connection. srv.Shutdown also got an already-expired context, and kctx.FatalIfErrorf turns this into exit 1.
  • Fix: give the scheduler a base context that Run cancels when the drain budget runs out, so in-flight runs stop. Give srv.Shutdown its own budget, or shut HTTP down first so new admin triggers stop. Treat "drained within budget" as a clean exit. Add a test where the ingest outlives the budget.
  • For contrast, an ingest that finishes inside the budget drains correctly: probe TestQAProbe_ShutdownDrainsInFlightIngestNoLeaks → Run returned: <nil>, ingest status=200 feedDone=true. A strict goleak.VerifyNone with an IgnoreCurrent() baseline and no runtime_pollWait ignore passed.

Non-blocking

  • N1 (decision for Steve Wozniak): SIGTERM is not handled. main.go:13 uses signal.NotifyContext(…, os.Interrupt, os.Kill). os.Kill (SIGKILL) cannot be caught, and SIGTERM, which Kubernetes sends, keeps Go's default behaviour of exiting immediately, so the drain never runs in a pod. The brief told Linus to use exactly os.Kill (mirroring content-extractor), so the code matches the brief, but the spec's "termination signal" requirement is not met in a pod. It's a one-line fix (syscall.SIGTERM); I recommend making it in this round, before 6.3 deploys it.
  • N2. If a readeck: section is present but READECK_TOKEN is unset (app.go:260), export is silently disabled with no log line and no error. At least log a warning, or fail at startup.
  • N3. The PR's TestIntegration_GracefulShutdown ignores internal/poll.runtime_pollWait, which hides any goroutine blocked on a socket (a leaked serve loop or pgx connection). My strict version shows there is no leak today; the guard is just weak. Use an IgnoreCurrent() baseline instead.
  • N4 (FYI, no change needed here). /admin/ sits outside CrossOriginProtection, as the plan requires for header-less clients. Wrapping it would still let curl through and would stop drive-by browser POSTs to /admin/ingest. Authentication and admin hardening are out of scope for 6.1; noting it for newsroom-cleanup.

Nits

  • app.go:303: srv.Shutdown never returns http.ErrServerClosed, so that check never runs.
  • app.go:221-224: the servingLimit default duplicates ServingConfig.init (50) and serving.Register's own default.
  • internal/health is close to a copy of content-extractor/internal/health, and CheckerFunc is unused. The brief asked for "mirroring", so this is fine for now; a shared pkg/health would be a follow-up.
  • --help: --http.read-header-timeout, --http.idle-timeout and --http.max-header-bytes show no env names. They come from pkg/cli and were like that before this PR.

Acceptance checklist (checked by running)

Check Result
bazel test //newsroom/... uncached, container tests included ✅ 22/22
--help lists all flags with env names; secrets are env-only with no default ✅ --db.password=STRING ($DB_PASSWORD), --llm.api-key=STRING ($LLM_API_KEY), --readeck.token=STRING ($READECK_TOKEN), --http.address=":8081" ($HTTP_ADDRESS), --config="config.yaml" ($CONFIG), LOG_*. ❌ the DB secret can still arrive another way (B1)
Readiness fails with the DB down; liveness does not touch the DB; /healthz is gone ✅ Probe with a TCP proxy that was killed: up readiness=200 liveness=200 healthz=404, down readiness=503 liveness=200
Reserved-prefix route fails startup ✅ Real binary: /ui/x, /health, /assets → route collides with a reserved prefix. A single segment under /briefings/x is allowed on purpose (config/briefing.go:175, briefing feeds per tasks 03/21); /briefings itself and deeper paths are rejected
Missing config file ✅ newsroom: error: --config: stat /nope.yaml: no such file or directory
SIGINT: server closes, scheduler drains, pool closes, no leaks ⚠️ Works when the ingest finishes inside the budget; ❌ not bounded (B3); SIGTERM not handled (N1)
No os.Getenv / http.ListenAndServe ❌ os.Getenv at app.go:102 (B1); ✅ only srv.ListenAndServe() on the configured http.Server
internal/curator and /v1/enrich deleted ✅ No matches for curator, internal/server or v1/enrich in newsroom/
Cross-origin protection through the full app ✅ Integration test PASS (uncached): cross-site Sec-Fetch-Site → 403 with starred_at unchanged; foreign Origin → 403; same-origin → 200 with DB updated; /admin/ingest/g/test with no Origin and no Sec-Fetch-Site → 200. Exactly one NewCrossOriginProtection (web.go:88); no AddTrustedOrigin or bypass
No new third-party dependency ✅ go.mod/go.sum/MODULE.bazel unchanged (goleak was already a dependency)
CI Validate ✅ Validate / validate (pull_request) success on 0c948f2
6.1 ticked in tasks.md ✅ (it should be un-ticked until B1–B3 land)
No symlinks ✅ no mode 120000 in the diff; find . -type l -not -path './.git/*' prints nothing

Not covered

  • I did not send a real OS signal to the binary with a live database; I tested by cancelling the Run context, which is what NotifyContext does. SIGTERM behaviour (N1) comes from reading the code and Go's documented semantics, not from running it.
  • "Readiness fails until migrations finish": migrations run in storage.Open before the listener starts, so this holds by construction. I did not test it separately.
  • I simulated "DB down" by killing a TCP proxy in front of Postgres, not by stopping the container.
  • I did not test real LLM or Readeck integrations.

Next owner: Linus Torvalds for B1–B3, routed by Steve Wozniak. Steve Wozniak also needs to decide on N1. I'll re-review with /code-review once there's a new SHA. Not merged.

## QA review: newsroom 6.1 App wiring (kong), PR #286 @ `0c948f2` **Verdict: FAIL**: 3 blocking findings. Most of the wiring is right: kong app, single mux, health split, reserved prefixes, cross-origin protection through the full app. It falls short on one acceptance criterion (`os.Getenv`). It also adds a process crash on a config that passes validation, and its shutdown does not respect its own time limit. **Success condition tested:** `newsroom/tasks/16-app-wiring.md` acceptance criteria and the ICB-160 plan checks. Specs: newsroom-operations ("Readiness and liveness…", "Shutdown drains in-flight work") and newsroom-config ("Route namespace is collision-free", "Secrets come from the environment only"). ### Steps run 1. `/code-review` (high) on `origin/main...origin/paperclip/ICB-159-app-wiring` (merge-base `fe97611`, 1 commit, 14 files). Every finding below was checked against the code or reproduced; unconfirmed ones were dropped. 2. `bazel test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore //newsroom/...` → **Executed 22 out of 22 tests: 22 tests pass.** 3. The same command with `--test_output=all --test_arg=-test.v` on `//newsroom/cmd/newsroom:newsroom_integration_test`, plus a temporary QA probe file (not committed; worktree deleted). The container tests really ran and were not skipped: `TestIntegration_WiringRoutes`, `…CrossOriginProtectionThroughFullApp` and `…GracefulShutdown` all PASS. 4. `bazel run --experimental_convenience_symlinks=ignore //newsroom/cmd/newsroom -- --help`, plus the real binary run against configs with reserved routes. 5. Greps for `os.Getenv`, `ListenAndServe`, `CrossOriginProtection`, `AddTrustedOrigin`, `v1/enrich` and `curator`; a diff of `go.mod`/`MODULE.bazel`; `git diff --summary` checked for mode `120000`; Forgejo commit status. ### Blocking (Linus) **B1. `os.Getenv` is still in `newsroom/`: `cmd/newsroom/app.go:102` (`resolveDSN`).** - Expected: no `os.Getenv` in `newsroom/` (task acceptance criterion), and all secrets come through kong flags so that `--help` lists every secret. - Actual: `grep -rn os.Getenv newsroom --include=*.go` (non-test) → `newsroom/cmd/newsroom/app.go:102: return os.Getenv(key)`. When `--db.password` is empty, the password comes from whatever variable `database.passwordEnv` names, and that variable never appears in `--help`. Also, `injectPassword` silently drops the password when the DSN has no user. - Fix: take the DB password only from `c.Database.Password` (`DB_PASSWORD`) and remove the `os.Getenv` fallback. If `passwordEnv` is set to anything other than `DB_PASSWORD`, fail at startup with a clear error, or ignore it; how `passwordEnv` changes long-term belongs to newsroom-cleanup. Return an error instead of silently dropping the password when the DSN has no user. **B2. `summarize: true` without `llm.baseURL` passes validation, then crashes the whole process (nil-interface panic).** - Expected: a config error at startup, or a classified summarization failure. Never a crash. - Actual: `app.go:252-255` only builds a summarizer when `cfg.LLM.BaseURL != ""`. Otherwise `enrich.New(store, extractor, nil)`, and `validateSummarize` does not require a base URL. Probe `TestQAProbe_SummarizeWithoutBaseURL`: `buildHandler err=<nil>`, then `POST /admin/ingest/g/test` → ``` panic: runtime error: invalid memory address or nil pointer dereference enrich.(*Enricher).summarizeBody newsroom/internal/enrich/enrich.go:258 enrich.(*Enricher).Enrich newsroom/internal/enrich/enrich.go:148 ingest.(*Ingester).enrichKeptEntry newsroom/internal/ingest/ingest.go:485 ingest.(*Ingester).processEntries.func1 newsroom/internal/ingest/ingest.go:297 ``` The panic happens in an ingest worker goroutine, so net/http does not recover it and the process exits. The same path runs from cron and from the always-registered backfill job. - Fix: fail startup when any provider has `summarize: true` (or summary backfill is active) and `llm.baseURL` is empty, or never pass a nil summarizer. Add a test for it. **B3. Shutdown is not bounded when an ingest is in flight, and a normal SIGINT exits non-zero.** - Expected (spec): "wait — **bounded** — for in-flight ingest… shut down the HTTP server, and only then release the database pool", and "SIGINT shuts the service down cleanly". - Actual: cron and `RunAll` runs use `context.Background()`/`WithoutCancel` with a 2-minute route timeout (`schedule.go:220,249,368`). `Run` gives `s.Stop` and `srv.Shutdown` one shared 10s budget, one after the other, and nothing cancels the runs. Probe `TestQAProbe_ShutdownOverrunsBudget` (feed upstream hangs for 20s, `POST /admin/ingest` → 202, then cancel): **`Run returned after 20s: err=context deadline exceeded`**. That is twice the 10s limit, because the deferred `store.Close()` waits for the run that is still holding a connection. `srv.Shutdown` also got an already-expired context, and `kctx.FatalIfErrorf` turns this into exit 1. - Fix: give the scheduler a base context that `Run` cancels when the drain budget runs out, so in-flight runs stop. Give `srv.Shutdown` its own budget, or shut HTTP down first so new admin triggers stop. Treat "drained within budget" as a clean exit. Add a test where the ingest outlives the budget. - For contrast, an ingest that finishes inside the budget drains correctly: probe `TestQAProbe_ShutdownDrainsInFlightIngestNoLeaks` → `Run returned: <nil>`, `ingest status=200 feedDone=true`. A strict `goleak.VerifyNone` with an `IgnoreCurrent()` baseline and **no** `runtime_pollWait` ignore passed. ### Non-blocking - **N1 (decision for Steve Wozniak): SIGTERM is not handled.** `main.go:13` uses `signal.NotifyContext(…, os.Interrupt, os.Kill)`. `os.Kill` (SIGKILL) cannot be caught, and SIGTERM, which Kubernetes sends, keeps Go's default behaviour of exiting immediately, so the drain never runs in a pod. The brief told Linus to use exactly `os.Kill` (mirroring content-extractor), so the code matches the brief, but the spec's "termination signal" requirement is not met in a pod. It's a one-line fix (`syscall.SIGTERM`); I recommend making it in this round, before 6.3 deploys it. - **N2.** If a `readeck:` section is present but `READECK_TOKEN` is unset (`app.go:260`), export is silently disabled with no log line and no error. At least log a warning, or fail at startup. - **N3.** The PR's `TestIntegration_GracefulShutdown` ignores `internal/poll.runtime_pollWait`, which hides any goroutine blocked on a socket (a leaked serve loop or pgx connection). My strict version shows there is no leak today; the guard is just weak. Use an `IgnoreCurrent()` baseline instead. - **N4 (FYI, no change needed here).** `/admin/` sits outside `CrossOriginProtection`, as the plan requires for header-less clients. Wrapping it would still let curl through and would stop drive-by browser POSTs to `/admin/ingest`. Authentication and admin hardening are out of scope for 6.1; noting it for newsroom-cleanup. ### Nits - `app.go:303`: `srv.Shutdown` never returns `http.ErrServerClosed`, so that check never runs. - `app.go:221-224`: the `servingLimit` default duplicates `ServingConfig.init` (50) and `serving.Register`'s own default. - `internal/health` is close to a copy of `content-extractor/internal/health`, and `CheckerFunc` is unused. The brief asked for "mirroring", so this is fine for now; a shared `pkg/health` would be a follow-up. - `--help`: `--http.read-header-timeout`, `--http.idle-timeout` and `--http.max-header-bytes` show no env names. They come from `pkg/cli` and were like that before this PR. ### Acceptance checklist (checked by running) | Check | Result | |---|---| | `bazel test //newsroom/...` uncached, container tests included | ✅ 22/22 | | `--help` lists all flags with env names; secrets are env-only with no default | ✅ `--db.password=STRING ($DB_PASSWORD)`, `--llm.api-key=STRING ($LLM_API_KEY)`, `--readeck.token=STRING ($READECK_TOKEN)`, `--http.address=":8081" ($HTTP_ADDRESS)`, `--config="config.yaml" ($CONFIG)`, `LOG_*`. ❌ the DB secret can still arrive another way (B1) | | Readiness fails with the DB down; liveness does not touch the DB; `/healthz` is gone | ✅ Probe with a TCP proxy that was killed: up `readiness=200 liveness=200 healthz=404`, down `readiness=503 liveness=200` | | Reserved-prefix route fails startup | ✅ Real binary: `/ui/x`, `/health`, `/assets` → `route collides with a reserved prefix`. A single segment under `/briefings/x` is allowed on purpose (`config/briefing.go:175`, briefing feeds per tasks 03/21); `/briefings` itself and deeper paths are rejected | | Missing config file | ✅ `newsroom: error: --config: stat /nope.yaml: no such file or directory` | | SIGINT: server closes, scheduler drains, pool closes, no leaks | ⚠️ Works when the ingest finishes inside the budget; ❌ not bounded (B3); SIGTERM not handled (N1) | | No `os.Getenv` / `http.ListenAndServe` | ❌ `os.Getenv` at `app.go:102` (B1); ✅ only `srv.ListenAndServe()` on the configured `http.Server` | | `internal/curator` and `/v1/enrich` deleted | ✅ No matches for `curator`, `internal/server` or `v1/enrich` in `newsroom/` | | Cross-origin protection through the full app | ✅ Integration test PASS (uncached): cross-site `Sec-Fetch-Site` → 403 with `starred_at` unchanged; foreign `Origin` → 403; same-origin → 200 with DB updated; `/admin/ingest/g/test` with no Origin and no Sec-Fetch-Site → 200. Exactly one `NewCrossOriginProtection` (`web.go:88`); no `AddTrustedOrigin` or bypass | | No new third-party dependency | ✅ `go.mod`/`go.sum`/`MODULE.bazel` unchanged (`goleak` was already a dependency) | | CI `Validate` | ✅ `Validate / validate (pull_request) success` on `0c948f2` | | 6.1 ticked in `tasks.md` | ✅ (it should be un-ticked until B1–B3 land) | | No symlinks | ✅ no mode `120000` in the diff; `find . -type l -not -path './.git/*'` prints nothing | ### Not covered - I did not send a real OS signal to the binary with a live database; I tested by cancelling the `Run` context, which is what `NotifyContext` does. SIGTERM behaviour (N1) comes from reading the code and Go's documented semantics, not from running it. - "Readiness fails until migrations finish": migrations run in `storage.Open` before the listener starts, so this holds by construction. I did not test it separately. - I simulated "DB down" by killing a TCP proxy in front of Postgres, not by stopping the container. - I did not test real LLM or Readeck integrations. **Next owner:** Linus Torvalds for B1–B3, routed by Steve Wozniak. Steve Wozniak also needs to decide on N1. I'll re-review with `/code-review` once there's a new SHA. **Not merged.**
Co-Authored-By: Paperclip <noreply@paperclip.ing>
fix(newsroom): address review findings B1-B3, N1-N3 on app wiring
All checks were successful
Validate / validate (pull_request) Successful in 4m31s
5979c3d447
- B1: resolve DB password via DB_PASSWORD, reject unsupported passwordEnv, error on missing DSN user
- B2: require non-empty llm.baseURL when summarize is true or backfill configured; fallback to DisabledSummarizer
- B3: bounded graceful shutdown with DrainTimeout, CancelInFlight, and separate HTTP shutdown budget
- N1: handle syscall.SIGTERM alongside os.Interrupt
- N2: warn at startup when readeck is configured without READECK_TOKEN
- N3: capture goleak baseline with IgnoreCurrent and scoped net/http persistConn ignores
- Re-tick task 6.1 in openspec tasks

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Author
Contributor

Verdict: PASS. PR #286 at 5979c3d is OK to merge. Nothing is blocking. The non-blocking items below are for Steve Wozniak to place: fold them into this PR or record them in newsroom-cleanup.

Re-review of the fix round (ICB-162, commit 5979c3d) against my previous FAIL at 0c948f2 (ICB-160). I did not merge.

Success condition tested

B1, B2, B3 and N1 are fixed, shown by running them. N2 and N3 are done. Nothing that passed last time has regressed. All //newsroom/... tests are green without cached results, including the container integration tests. No symlinks, no new dependencies, CI Validate is green, and 6.1 is ticked at the final commit.

Steps run

git worktree add --detach /tmp/rv286 5979c3d       # outside the workspace; removed afterwards
grep -rn 'os.Getenv|os.LookupEnv|os.Environ' newsroom --include=*.go | grep -v _test.go
git diff origin/main...5979c3d --summary | grep 120000      # → 0
git diff origin/main...5979c3d -- go.mod go.sum MODULE.bazel  # → empty
go build -o /tmp/qa-bin/newsroom ./newsroom/cmd/newsroom      # binary used for --help and startup checks
bazel test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore --test_output=errors //newsroom/...
bazel test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore --test_output=all --test_arg=-test.v \
  --test_filter='TestQAProbe_' //newsroom/cmd/newsroom:newsroom_integration_test   # my probes, real Postgres 18 container
/code-review on 0c948f2..5979c3d, and again on origin/main...5979c3d

Results per item

Item Expected Actual Evidence
Full suite green, uncached 22/22 pass Executed 22 out of 22 tests: 22 tests pass. EXIT=0
B1 os.Getenv none in non-test newsroom/ none. The only match is a comment in config/database.go:52 grep
B1 custom passwordEnv error exit 1: database.passwordEnv "MY_PW" is not supported: database password must be set via DB_PASSWORD / --db.password binary
B1 password + DSN without user error exit 1: database dsn has no user to attach password to binary
B1 --help every secret listed with its env var and no default --db.password ($DB_PASSWORD), --llm.api-key ($LLM_API_KEY), --readeck.token ($READECK_TOKEN), no defaults. DrainTimeout is hidden binary
B2 summarize: true without llm.baseURL config error, no panic exit 1: invalid config: providers.p: summarize is true but llm.baseURL is empty. Backfill without baseURL also gives exit 1 binary
B3 ingest outlives the budget Run returns within budget + margin, run cancelled, pool closed Run returned after 1.001985108s err=<nil> (budget 1s). Upstream request got context canceled. backends after Run: 0. Strict goleak.VerifyNone(baseline) passes TestQAProbe_ShutdownOverrunsBudget PASS
B3 in-flight ingest drains Run returns nil, no leaks Run returned after 1.516754017s err=<nil> served=true, entries persisted: 1, 0 backends, strict goleak passes TestQAProbe_ShutdownDrainsInFlightIngestNoLeaks PASS
N1 SIGTERM caught, clean exit NotifyContext(..., os.Interrupt, syscall.SIGTERM), and os.Kill is gone. A real SIGTERM sent to the process running the real main() (kong parse, real Postgres): readiness was 200 before the signal, and main() returned normally 391µs after SIGTERM (the exit-0 path; FatalIfErrorf was not hit, the process was not killed), 0 backends left TestQAProbe_RealSIGTERMThroughMain PASS
N2 warning when the token is missing WARN readeck config present but READECK_TOKEN / --readeck.token is unset; export disabled, covered by TestIntegration_ReadeckWithoutTokenLogsWarning suite
N3 IgnoreCurrent() baseline, no runtime_pollWait ignore done. See nit 4 diff
Earlier PASS items no regression only one http.NewCrossOriginProtection() (web/web.go:88) and no AddTrustedOrigin. The health, reserved-prefix and routing code is not touched by the fix diff. TestIntegration_WiringRoutes and TestIntegration_CrossOriginProtectionThroughFullApp pass suite
Hygiene no new dependencies, no mode 120000 entries, tasks.md 6.1 is [x] at 5979c3d, CI Validate / validate (pull_request) = success git / Forgejo status

Findings, most severe first

Blocking: none.

Non-blocking (real, but none crashes, leaks a secret or loses data):

  1. The configured logger is never made the slog default. AfterApply binds Logging.PrepareLogger() into kong, but nothing calls slog.SetDefault; I grepped the whole repo. ingest, enrich and openaicompat log through package-level slog.WarnContext/ErrorContext/InfoContext (10+ call sites, e.g. ingest.go:462, :490, openaicompat.go:215). Their lines bypass LOG_FORMAT/LOG_LEVEL and come out as Go's default text log next to the app's JSON. In the probe run, main() logged JSON and the ingest error logged plain text. This was already present at 0c948f2 and I missed it last round. The fix is one line in AfterApply.
  2. Shutdown is not hard-bounded when a run ignores ctx. If the 2s grace Stop also times out, its error is discarded, Run returns nil (exit 0), and the deferred store.Close() (pgxpool.Close) waits for any connection still held. NotifyContext's cancel only runs on exit, so a second Ctrl-C or SIGTERM is swallowed. In practice pgx honours ctx and my probe shows a clean 1.0s exit, so this is theoretical.
  3. Readiness stays 200 during the drain window (up to 10s + 2s). POST /admin/ingest/{route} sent in that window gets 500 (ErrSchedulerStopped), not 503.
  4. passwordEnv is rejected in the CLI, not in config validation. resolveDSN rejects a non-default database.passwordEnv at runtime, but config.Validate and the database.go comment still accept and describe any env var name. A DSN that sets the user with ?user= (and has no userinfo) is also rejected when a password is set. Both match the agreed "error" behaviour; the check just sits in the wrong layer.
  5. ErrNoLLMBaseURL is reported once for each provider with summarize: true instead of once overall (provider.go:165).
  6. The backfill cron is registered whenever llm.baseURL is set, even if no provider summarizes, because init() always fills in a default Backfill.

Nits:

  • The abort-watcher goroutine block is copy-pasted in executeRoute and runGenericJob; context.AfterFunc would avoid the extra goroutine per run.
  • loadConfig is now just a pass-through to config.LoadFile, and the resolveDSN lookup closure is a constant stub.
  • DisabledSummarizer.Summarize has no doc comment and its params are named but unused (CI lint passed anyway).
  • N3: the persistConn ignores were widened from IgnoreTopFunction to IgnoreAnyFunction. That works, but it is looser than needed.
  • explicitBackfill is only set by YAML decoding, so a Config built in code skips the check.
  • The ingest.Stats → schedule.Stats field-by-field copy could be a type conversion.
  • health is now the third copy of the probe handler (content-extractor, chargelog).
  • Readiness uses store.Pool() instead of store.
  • The listen-address env var changed from LISTEN_ADDR to HTTP_ADDRESS (no deployment exists yet; 6.3 should use HTTP_ADDRESS).

Not covered

  • SIGTERM to an out-of-process binary: there is no Docker or Postgres in my sandbox. I sent a real SIGTERM to the test process running the real main() against a real Postgres container on the remote executor (see the table). An actual process exit code was not observed. The not-killed / no-FatalIfErrorf path is equivalent to exit 0.
  • SIGINT through main(): not sent separately. It uses the same NotifyContext and the same ctx-cancel path, which the probes cover.
  • Real LLM / Readeck endpoints: not exercised (mocks only).
  • Kubernetes deployment: not exercised (6.3 is out of scope).

Probe files and the worktree are removed. find . -type l -not -path './.git/*' prints nothing.

Next owner: Steve Wozniak decides whether non-blocking items 1–6 go into this PR (Linus Torvalds) or into newsroom-cleanup. Item 1 is a one-line fix and worth doing first. With that decision made, the owner can merge.

**Verdict: PASS.** PR #286 at `5979c3d` is OK to merge. Nothing is blocking. The non-blocking items below are for Steve Wozniak to place: fold them into this PR or record them in `newsroom-cleanup`. Re-review of the fix round (ICB-162, commit `5979c3d`) against my previous FAIL at `0c948f2` (ICB-160). I did not merge. ### Success condition tested B1, B2, B3 and N1 are fixed, shown by running them. N2 and N3 are done. Nothing that passed last time has regressed. All `//newsroom/...` tests are green without cached results, including the container integration tests. No symlinks, no new dependencies, CI `Validate` is green, and 6.1 is ticked at the final commit. ### Steps run ``` git worktree add --detach /tmp/rv286 5979c3d # outside the workspace; removed afterwards grep -rn 'os.Getenv|os.LookupEnv|os.Environ' newsroom --include=*.go | grep -v _test.go git diff origin/main...5979c3d --summary | grep 120000 # → 0 git diff origin/main...5979c3d -- go.mod go.sum MODULE.bazel # → empty go build -o /tmp/qa-bin/newsroom ./newsroom/cmd/newsroom # binary used for --help and startup checks bazel test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore --test_output=errors //newsroom/... bazel test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore --test_output=all --test_arg=-test.v \ --test_filter='TestQAProbe_' //newsroom/cmd/newsroom:newsroom_integration_test # my probes, real Postgres 18 container /code-review on 0c948f2..5979c3d, and again on origin/main...5979c3d ``` ### Results per item | Item | Expected | Actual | Evidence | |---|---|---|---| | Full suite | green, uncached | **22/22 pass** | `Executed 22 out of 22 tests: 22 tests pass.` `EXIT=0` | | B1 `os.Getenv` | none in non-test `newsroom/` | none. The only match is a comment in `config/database.go:52` | grep | | B1 custom `passwordEnv` | error | exit 1: `database.passwordEnv "MY_PW" is not supported: database password must be set via DB_PASSWORD / --db.password` | binary | | B1 password + DSN without user | error | exit 1: `database dsn has no user to attach password to` | binary | | B1 `--help` | every secret listed with its env var and no default | `--db.password ($DB_PASSWORD)`, `--llm.api-key ($LLM_API_KEY)`, `--readeck.token ($READECK_TOKEN)`, no defaults. `DrainTimeout` is hidden | binary | | B2 `summarize: true` without `llm.baseURL` | config error, no panic | exit 1: `invalid config: providers.p: summarize is true but llm.baseURL is empty`. Backfill without baseURL also gives exit 1 | binary | | B3 ingest outlives the budget | `Run` returns within budget + margin, run cancelled, pool closed | `Run returned after 1.001985108s err=<nil>` (budget 1s). Upstream request got `context canceled`. `backends after Run: 0`. Strict `goleak.VerifyNone(baseline)` passes | `TestQAProbe_ShutdownOverrunsBudget` PASS | | B3 in-flight ingest drains | `Run` returns nil, no leaks | `Run returned after 1.516754017s err=<nil> served=true`, `entries persisted: 1`, 0 backends, strict goleak passes | `TestQAProbe_ShutdownDrainsInFlightIngestNoLeaks` PASS | | N1 SIGTERM | caught, clean exit | `NotifyContext(..., os.Interrupt, syscall.SIGTERM)`, and `os.Kill` is gone. A real `SIGTERM` sent to the process running the real `main()` (kong parse, real Postgres): readiness was 200 before the signal, and `main() returned normally 391µs after SIGTERM` (the exit-0 path; `FatalIfErrorf` was not hit, the process was not killed), 0 backends left | `TestQAProbe_RealSIGTERMThroughMain` PASS | | N2 | warning when the token is missing | `WARN readeck config present but READECK_TOKEN / --readeck.token is unset; export disabled`, covered by `TestIntegration_ReadeckWithoutTokenLogsWarning` | suite | | N3 | `IgnoreCurrent()` baseline, no `runtime_pollWait` ignore | done. See nit 4 | diff | | Earlier PASS items | no regression | only one `http.NewCrossOriginProtection()` (`web/web.go:88`) and no `AddTrustedOrigin`. The health, reserved-prefix and routing code is not touched by the fix diff. `TestIntegration_WiringRoutes` and `TestIntegration_CrossOriginProtectionThroughFullApp` pass | suite | | Hygiene | | no new dependencies, no mode `120000` entries, `tasks.md` 6.1 is `[x]` at `5979c3d`, CI `Validate / validate (pull_request)` = success | git / Forgejo status | ### Findings, most severe first **Blocking:** none. **Non-blocking** (real, but none crashes, leaks a secret or loses data): 1. **The configured logger is never made the slog default.** `AfterApply` binds `Logging.PrepareLogger()` into kong, but nothing calls `slog.SetDefault`; I grepped the whole repo. `ingest`, `enrich` and `openaicompat` log through package-level `slog.WarnContext`/`ErrorContext`/`InfoContext` (10+ call sites, e.g. `ingest.go:462`, `:490`, `openaicompat.go:215`). Their lines bypass `LOG_FORMAT`/`LOG_LEVEL` and come out as Go's default text log next to the app's JSON. In the probe run, `main()` logged JSON and the ingest error logged plain text. This was already present at `0c948f2` and I missed it last round. The fix is one line in `AfterApply`. 2. **Shutdown is not hard-bounded when a run ignores ctx.** If the 2s grace `Stop` also times out, its error is discarded, `Run` returns nil (exit 0), and the deferred `store.Close()` (`pgxpool.Close`) waits for any connection still held. `NotifyContext`'s `cancel` only runs on exit, so a second Ctrl-C or SIGTERM is swallowed. In practice pgx honours ctx and my probe shows a clean 1.0s exit, so this is theoretical. 3. **Readiness stays 200 during the drain window** (up to 10s + 2s). `POST /admin/ingest/{route}` sent in that window gets 500 (`ErrSchedulerStopped`), not 503. 4. **`passwordEnv` is rejected in the CLI, not in config validation.** `resolveDSN` rejects a non-default `database.passwordEnv` at runtime, but `config.Validate` and the `database.go` comment still accept and describe any env var name. A DSN that sets the user with `?user=` (and has no userinfo) is also rejected when a password is set. Both match the agreed "error" behaviour; the check just sits in the wrong layer. 5. `ErrNoLLMBaseURL` is reported once for each provider with `summarize: true` instead of once overall (`provider.go:165`). 6. The backfill cron is registered whenever `llm.baseURL` is set, even if no provider summarizes, because `init()` always fills in a default `Backfill`. **Nits:** - The abort-watcher goroutine block is copy-pasted in `executeRoute` and `runGenericJob`; `context.AfterFunc` would avoid the extra goroutine per run. - `loadConfig` is now just a pass-through to `config.LoadFile`, and the `resolveDSN` lookup closure is a constant stub. - `DisabledSummarizer.Summarize` has no doc comment and its params are named but unused (CI lint passed anyway). - N3: the `persistConn` ignores were widened from `IgnoreTopFunction` to `IgnoreAnyFunction`. That works, but it is looser than needed. - `explicitBackfill` is only set by YAML decoding, so a `Config` built in code skips the check. - The `ingest.Stats` → `schedule.Stats` field-by-field copy could be a type conversion. - `health` is now the third copy of the probe handler (content-extractor, chargelog). - Readiness uses `store.Pool()` instead of `store`. - The listen-address env var changed from `LISTEN_ADDR` to `HTTP_ADDRESS` (no deployment exists yet; 6.3 should use `HTTP_ADDRESS`). ### Not covered - **SIGTERM to an out-of-process binary:** there is no Docker or Postgres in my sandbox. I sent a real `SIGTERM` to the test process running the real `main()` against a real Postgres container on the remote executor (see the table). An actual process exit code was not observed. The not-killed / no-`FatalIfErrorf` path is equivalent to exit 0. - **SIGINT through `main()`:** not sent separately. It uses the same `NotifyContext` and the same ctx-cancel path, which the probes cover. - **Real LLM / Readeck endpoints:** not exercised (mocks only). - **Kubernetes deployment:** not exercised (6.3 is out of scope). Probe files and the worktree are removed. `find . -type l -not -path './.git/*'` prints nothing. **Next owner:** Steve Wozniak decides whether non-blocking items 1–6 go into this PR (Linus Torvalds) or into `newsroom-cleanup`. Item 1 is a one-line fix and worth doing first. With that decision made, the owner can merge.
fix(newsroom): configure slog default logger and address review nits
All checks were successful
Validate / validate (pull_request) Successful in 3m41s
4125c9f630
- Item 1: call slog.SetDefault(logger) in AfterApply with test coverage
- Nit 1: extract abort-watcher via runContext using context.AfterFunc in Scheduler
- Nit 2: remove loadConfig pass-through, simplify resolveDSN lookup closure
- Nit 3: add doc comment and blank param identifiers to DisabledSummarizer.Summarize
- Nit 4: document why goleak IgnoreAnyFunction is required for net/http persistConn
- Nit 5: replace YAML-only explicitBackfill with isDefaultBackfill and add test for code-built config
- Nit 6: use type conversion schedule.Stats(stats) instead of field-by-field copy
- Nit 7: ping health check through store instead of store.Pool()
- Nit 8: note that deployment must set HTTP_ADDRESS in tasks.md and 18-deployment-and-release.md

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Author
Contributor

Verdict: PASS — re-check of 5979c3d..4125c9f (one commit, fix(newsroom): configure slog default logger and address review nits). Not merged.

Success condition tested

  • Item 1: the configured logger becomes the slog default. Package-level ingest / enrich / openaicompat lines come out as JSON under LOG_FORMAT=json and follow LOG_LEVEL, when parsed through kong the way main() does it.
  • Each requested nit is fixed, or skipped with a reason.
  • No regression in B1–B3 / N1–N3: the shutdown probes still pass with strict goleak. One NewCrossOriginProtection and no AddTrustedOrigin. No new dependency. No mode 120000. 6.1 ticked. CI green.

Steps run

git fetch origin paperclip/ICB-159-app-wiring   # head 4125c9f, == PR head
git diff 5979c3d..4125c9f ; git diff --summary ... | grep 120000   # 0
/code-review 5979c3d..4125c9f
bazel --bazelrc=.bazelrc.agents test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore --test_output=errors //newsroom/...
bazel ... --test_output=all --test_arg=-test.v --test_filter='TestQAProbe_' //newsroom/cmd/newsroom:newsroom_integration_test   # temporary probe file, now deleted
find . -type l -not -path './.git/*'   # prints nothing

Results

Check Expected Actual
Full suite, uncached (the integration targets with real Postgres containers are inside //newsroom/...; no target is tagged manual) green Executed 22 out of 22 tests: 22 tests pass. EXIT=0
Item 1, LOG_LEVEL=info (TestQAProbe_SlogDefaultThroughKong/info, kong Parse from env like main()) every stderr line is JSON, including ingest and openaicompat PASS. Every line parsed as JSON, for example {"level":"INFO","msg":"openai-compatible completion","model":"m",...} and {"level":"WARN","msg":"enrichment failed for entry","entry_id":2,...}
Item 1, LOG_LEVEL=warn INFO lines dropped, including openaicompat PASS. The only line was {"level":"WARN","msg":"enrichment failed for entry",...}
B3 overrun (TestQAProbe_ShutdownOverrunsBudget, budget 1s) returns within budget, run cancelled, no leaks Run returned after 1.000957453s err=<nil>, upstream context canceled, backends after Run: 0, strict goleak.VerifyNone(baseline) passes
B3 drain (TestQAProbe_ShutdownDrainsInFlightIngestNoLeaks) in-flight ingest finishes, nil, no leaks Run returned after 1.515296778s err=<nil>, entries persisted: 1, 0 backends, strict goleak passes
AfterFunc refactor same cancel behaviour as the goroutine watcher yes: both probes above, plus the existing GracefulShutdown* tests in the suite
Nit: abort-watcher duplication fixed runContext uses context.AfterFunc and is used in both places
Nit: loadConfig / resolveDSN stub fixed config.LoadFile is called directly. The lookup is func(_ string) string { return password }, which is safe because the PasswordEnv guard directly above only allows "" or DB_PASSWORD
Nit: DisabledSummarizer fixed doc comment added, params are _
Nit: persistConn ignores skipped with a reason I accept the reason. My own goleak output shows the idle persistConn.readLoop goroutine with internal/poll.runtime_pollWait on top of the stack, so IgnoreTopFunction cannot match it. A comment now explains this
Nit: explicitBackfill for a Config built in code fixed an isDefaultBackfill flag is set by init(); new test TestLLMConfig_Validate_CodeBuiltBackfillWithoutBaseURLErrors
Nit: Stats conversion fixed schedule.Stats(stats)
Nit: readiness through store fixed health.PingChecker{Pinger: store}
Nit: HTTP_ADDRESS note on 6.3 fixed tasks.md 6.3 and tasks/18-deployment-and-release.md
health copy skipped as instructed not touched
Hygiene NewCrossOriginProtection once (web/web.go:88), AddTrustedOrigin 0. No change to go.mod / go.sum / MODULE.bazel. No mode 120000. 6.1 [x]. CI Validate / validate (pull_request) = success (run 609) on 4125c9f
Secrets none in logs the opening database pool line logs a redacted DSN (password replaced with xxxxx)

Findings

Blocking: none.
Non-blocking: none.
Nits (optional; none of them is worth another round on their own):

  1. schedule.go:164: abortCh is now dead state. Nothing reads it any more, but CancelInFlight still closes it next to abortCancel(). It can go.
  2. schedule.go:167: abortCtx is a func() context.Context that always returns the same context. A plain context.Context field would be simpler.
  3. Only New sets abortCtx and abortCancel, so a zero-value Scheduler now panics in runContext. Nothing builds one that way today (only New does, per grep), so this is theoretical.
  4. isDefaultBackfill is only set inside init(). Code that replaces LLM.Backfill after Init() would skip the base-URL check. This is an edge case and nothing does it today.
  5. TestAfterApply_ConfiguresSlogDefault swaps the global os.Stderr. That is fine today because the package has no t.Parallel.

/code-review also raised these, which I dismissed: slog.SetDefault as global state (it was the requested fix), coupling through the Stats conversion (requested), and assert.NoError not stopping the test (alecthomas assert is fatal).

Not covered

  • Real LLM and Readeck endpoints (mocks only). Kubernetes (6.3 is out of scope). No out-of-process exit code, because there is no Docker in my sandbox; the probes run on the remote executor's Postgres containers.
  • The probe goleak baseline is taken after the Postgres container starts. Otherwise the testcontainers Docker client's idle keep-alive connection shows up as a "leak". This is probe setup, not an app leak; the app's own goroutines are fully covered.

Probe files are removed, and find . -type l -not -path './.git/*' prints nothing.

Next owner: the owner. Nothing is outstanding for this PR, and it can be merged. Nits 1–4 can go into newsroom-cleanup if wanted (Steve Wozniak's call).

**Verdict: PASS** — re-check of `5979c3d..4125c9f` (one commit, `fix(newsroom): configure slog default logger and address review nits`). Not merged. ### Success condition tested - Item 1: the configured logger becomes the slog default. Package-level `ingest` / `enrich` / `openaicompat` lines come out as JSON under `LOG_FORMAT=json` and follow `LOG_LEVEL`, when parsed through kong the way `main()` does it. - Each requested nit is fixed, or skipped with a reason. - No regression in B1–B3 / N1–N3: the shutdown probes still pass with strict goleak. One `NewCrossOriginProtection` and no `AddTrustedOrigin`. No new dependency. No mode `120000`. 6.1 ticked. CI green. ### Steps run ``` git fetch origin paperclip/ICB-159-app-wiring # head 4125c9f, == PR head git diff 5979c3d..4125c9f ; git diff --summary ... | grep 120000 # 0 /code-review 5979c3d..4125c9f bazel --bazelrc=.bazelrc.agents test --config=remote --nocache_test_results --experimental_convenience_symlinks=ignore --test_output=errors //newsroom/... bazel ... --test_output=all --test_arg=-test.v --test_filter='TestQAProbe_' //newsroom/cmd/newsroom:newsroom_integration_test # temporary probe file, now deleted find . -type l -not -path './.git/*' # prints nothing ``` ### Results | Check | Expected | Actual | |---|---|---| | Full suite, uncached (the integration targets with real Postgres containers are inside `//newsroom/...`; no target is tagged `manual`) | green | `Executed 22 out of 22 tests: 22 tests pass.` `EXIT=0` | | Item 1, `LOG_LEVEL=info` (`TestQAProbe_SlogDefaultThroughKong/info`, kong `Parse` from env like `main()`) | every stderr line is JSON, including ingest and openaicompat | PASS. Every line parsed as JSON, for example `{"level":"INFO","msg":"openai-compatible completion","model":"m",...}` and `{"level":"WARN","msg":"enrichment failed for entry","entry_id":2,...}` | | Item 1, `LOG_LEVEL=warn` | INFO lines dropped, including openaicompat | PASS. The only line was `{"level":"WARN","msg":"enrichment failed for entry",...}` | | B3 overrun (`TestQAProbe_ShutdownOverrunsBudget`, budget 1s) | returns within budget, run cancelled, no leaks | `Run returned after 1.000957453s err=<nil>`, upstream `context canceled`, `backends after Run: 0`, strict `goleak.VerifyNone(baseline)` passes | | B3 drain (`TestQAProbe_ShutdownDrainsInFlightIngestNoLeaks`) | in-flight ingest finishes, nil, no leaks | `Run returned after 1.515296778s err=<nil>`, `entries persisted: 1`, 0 backends, strict goleak passes | | `AfterFunc` refactor | same cancel behaviour as the goroutine watcher | yes: both probes above, plus the existing `GracefulShutdown*` tests in the suite | | Nit: abort-watcher duplication | fixed | `runContext` uses `context.AfterFunc` and is used in both places | | Nit: `loadConfig` / `resolveDSN` stub | fixed | `config.LoadFile` is called directly. The lookup is `func(_ string) string { return password }`, which is safe because the `PasswordEnv` guard directly above only allows `""` or `DB_PASSWORD` | | Nit: `DisabledSummarizer` | fixed | doc comment added, params are `_` | | Nit: `persistConn` ignores | skipped with a reason | I accept the reason. My own goleak output shows the idle `persistConn.readLoop` goroutine with `internal/poll.runtime_pollWait on top of the stack`, so `IgnoreTopFunction` cannot match it. A comment now explains this | | Nit: `explicitBackfill` for a `Config` built in code | fixed | an `isDefaultBackfill` flag is set by `init()`; new test `TestLLMConfig_Validate_CodeBuiltBackfillWithoutBaseURLErrors` | | Nit: `Stats` conversion | fixed | `schedule.Stats(stats)` | | Nit: readiness through `store` | fixed | `health.PingChecker{Pinger: store}` | | Nit: `HTTP_ADDRESS` note on 6.3 | fixed | `tasks.md` 6.3 and `tasks/18-deployment-and-release.md` | | `health` copy | skipped as instructed | not touched | | Hygiene | | `NewCrossOriginProtection` once (`web/web.go:88`), `AddTrustedOrigin` 0. No change to `go.mod` / `go.sum` / `MODULE.bazel`. No mode `120000`. 6.1 `[x]`. CI `Validate / validate (pull_request)` = **success** (run 609) on `4125c9f` | | Secrets | none in logs | the `opening database pool` line logs a redacted DSN (password replaced with `xxxxx`) | ### Findings **Blocking:** none. **Non-blocking:** none. **Nits** (optional; none of them is worth another round on their own): 1. `schedule.go:164`: `abortCh` is now dead state. Nothing reads it any more, but `CancelInFlight` still closes it next to `abortCancel()`. It can go. 2. `schedule.go:167`: `abortCtx` is a `func() context.Context` that always returns the same context. A plain `context.Context` field would be simpler. 3. Only `New` sets `abortCtx` and `abortCancel`, so a zero-value `Scheduler` now panics in `runContext`. Nothing builds one that way today (only `New` does, per grep), so this is theoretical. 4. `isDefaultBackfill` is only set inside `init()`. Code that replaces `LLM.Backfill` *after* `Init()` would skip the base-URL check. This is an edge case and nothing does it today. 5. `TestAfterApply_ConfiguresSlogDefault` swaps the global `os.Stderr`. That is fine today because the package has no `t.Parallel`. /code-review also raised these, which I dismissed: `slog.SetDefault` as global state (it was the requested fix), coupling through the `Stats` conversion (requested), and `assert.NoError` not stopping the test (alecthomas `assert` is fatal). ### Not covered - Real LLM and Readeck endpoints (mocks only). Kubernetes (6.3 is out of scope). No out-of-process exit code, because there is no Docker in my sandbox; the probes run on the remote executor's Postgres containers. - The probe goleak baseline is taken *after* the Postgres container starts. Otherwise the testcontainers Docker client's idle keep-alive connection shows up as a "leak". This is probe setup, not an app leak; the app's own goroutines are fully covered. Probe files are removed, and `find . -type l -not -path './.git/*'` prints nothing. **Next owner:** the owner. Nothing is outstanding for this PR, and it can be merged. Nits 1–4 can go into `newsroom-cleanup` if wanted (Steve Wozniak's call).
fix(newsroom): remove dead abortCh and simplify abortCtx in schedule.go
Some checks failed
Validate / validate (pull_request) Failing after 2m7s
e5969e46d0
- Remove dead abortCh field, close call, and allocation in Scheduler
- Store abortCtx as plain context.Context instead of func() context.Context

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Author
Contributor

Addressed Grace Hopper's two optional nits from the re-review in commit e5969e46d0e72e6d085cc066c96839e53c1cb2d4:

  1. schedule.go:164: removed dead abortCh field, close(s.abortCh) in CancelInFlight, and channel allocation in New.
  2. schedule.go:167: converted abortCtx from func() context.Context to a plain context.Context stored directly in New and passed to context.AfterFunc.

Verified with go test -race ./newsroom/internal/schedule/..., go vet ./newsroom/internal/schedule/..., and full go test ./newsroom/.... All passed.

PR remains open and not merged.

Addressed Grace Hopper's two optional nits from the re-review in commit `e5969e46d0e72e6d085cc066c96839e53c1cb2d4`: 1. `schedule.go:164`: removed dead `abortCh` field, `close(s.abortCh)` in `CancelInFlight`, and channel allocation in `New`. 2. `schedule.go:167`: converted `abortCtx` from `func() context.Context` to a plain `context.Context` stored directly in `New` and passed to `context.AfterFunc`. Verified with `go test -race ./newsroom/internal/schedule/...`, `go vet ./newsroom/internal/schedule/...`, and full `go test ./newsroom/...`. All passed. PR remains open and not merged.
@ -167,2 +176,4 @@
// New creates a new Scheduler with the provided options.
func New(opts ...Option) *Scheduler {
abortCtx, abortCancel := context.WithCancel(context.Background())
Owner

should the parent context not be the one of the app basically, so jobs are also canceled, on [Ctrl]-[C]?

should the parent context not be the one of the app basically, so jobs are also canceled, on `[Ctrl]-[C]`?
prskr marked this conversation as resolved
fix(newsroom): store abortCtx as func to comply with containedctx
All checks were successful
Validate / validate (pull_request) Successful in 3m6s
44a661f1ae
Revert abortCtx from a direct context.Context struct field back to a
func() context.Context to satisfy the nogo containedctx analyzer.
Document the rationale in a doc comment on the field.

Co-Authored-By: Paperclip <noreply@paperclip.ing>
Author
Contributor

Fixed the containedctx nogo build error in commit 44a661f1ae25a98dcbdeb6f55e6e7efaf1a79401 and reviewed Peter's question regarding parent context:

1. App Context vs. In-Flight Job Drain on [Ctrl]-[C]

Binding job runs to the app context (which cancels immediately on [Ctrl]-[C] / SIGTERM) does not make sense with the graceful shutdown architecture:

  • Under graceful shutdown (review finding B3), on shutdown the app stops accepting new runs and gives in-flight runs a drain budget (c.DrainTimeout, default 10s) via s.Stop(drainCtx) so active ingestion runs can finish cleanly without leaving partial records or dirty state.
  • If jobs used the app context directly, they would abort immediately at t=0s on [Ctrl]-[C], defeating the drain period and failing TestIntegration_GracefulShutdown_DrainTimeout (which explicitly verifies that runs are given the drain window and only aborted if they exceed it).
  • s.CancelInFlight() is specifically called when the drain budget is exceeded or if srv.ListenAndServe() fails.

2. Fix for containedctx nogo Error

  • Reverted abortCtx from a direct context.Context struct field back to func() context.Context on Scheduler.
  • Added a doc comment on the field clarifying that storing it as func() context.Context is intentional for containedctx analyzer compliance while supporting context.AfterFunc without background watcher goroutines.
  • abortCh remains removed (nit 1).

Verification

  • bazel --bazelrc=.bazelrc.agents test //newsroom/internal/schedule/...: PASS (schedule_test passed, nogo passed with zero errors).
  • bazel --bazelrc=.bazelrc.agents test //newsroom/cmd/newsroom:newsroom_test: PASS (nogo on newsroom_lib and newsroom_test passed).
  • bazel --bazelrc=.bazelrc.agents test //newsroom/internal/enrich/... //newsroom/internal/ingest/...: PASS (all 4 test targets passed).
  • go vet ./newsroom/internal/schedule/...: PASS.
  • go test -race ./newsroom/internal/schedule/...: PASS (1.147s).
  • go build -buildvcs=false ./newsroom/...: PASS.
Fixed the `containedctx` nogo build error in commit `44a661f1ae25a98dcbdeb6f55e6e7efaf1a79401` and reviewed Peter's question regarding parent context: ### 1. App Context vs. In-Flight Job Drain on `[Ctrl]-[C]` Binding job runs to the app context (which cancels immediately on `[Ctrl]-[C]` / `SIGTERM`) does not make sense with the graceful shutdown architecture: - Under graceful shutdown (review finding B3), on shutdown the app stops accepting new runs and gives in-flight runs a drain budget (`c.DrainTimeout`, default 10s) via `s.Stop(drainCtx)` so active ingestion runs can finish cleanly without leaving partial records or dirty state. - If jobs used the app context directly, they would abort immediately at t=0s on `[Ctrl]-[C]`, defeating the drain period and failing `TestIntegration_GracefulShutdown_DrainTimeout` (which explicitly verifies that runs are given the drain window and only aborted if they exceed it). - `s.CancelInFlight()` is specifically called when the drain budget is exceeded or if `srv.ListenAndServe()` fails. ### 2. Fix for `containedctx` nogo Error - Reverted `abortCtx` from a direct `context.Context` struct field back to `func() context.Context` on `Scheduler`. - Added a doc comment on the field clarifying that storing it as `func() context.Context` is intentional for `containedctx` analyzer compliance while supporting `context.AfterFunc` without background watcher goroutines. - `abortCh` remains removed (nit 1). ### Verification - `bazel --bazelrc=.bazelrc.agents test //newsroom/internal/schedule/...`: PASS (`schedule_test` passed, nogo passed with zero errors). - `bazel --bazelrc=.bazelrc.agents test //newsroom/cmd/newsroom:newsroom_test`: PASS (nogo on `newsroom_lib` and `newsroom_test` passed). - `bazel --bazelrc=.bazelrc.agents test //newsroom/internal/enrich/... //newsroom/internal/ingest/...`: PASS (all 4 test targets passed). - `go vet ./newsroom/internal/schedule/...`: PASS. - `go test -race ./newsroom/internal/schedule/...`: PASS (1.147s). - `go build -buildvcs=false ./newsroom/...`: PASS.
prskr approved these changes 2026-10-16 14:01:09 +00:00
prskr merged commit f24b111a3f into main 2026-10-16 14:01:13 +00:00
prskr deleted branch paperclip/ICB-159-app-wiring 2026-10-16 14:01:13 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
prskr/tools!286
No description provided.