Both only checked title existence, so editing a stream's rules or a
dashboard's widgets under the same title silently skipped the update
forever. step_dashboard() now diffs the search's queries content
(delete+recreate view+search on change - no subscribers/history to lose,
unlike alerts or streams); step_streams() diffs rules individually by
(field,type,value,inverted) and PUT-updates metadata in place, since
delete+recreating a stream would briefly break live message routing.
Confirmed live on a local test deployment: a false "changed" positive
from search_types list-reordering and from search_servers.json's own
query-format inconsistency (bare string vs the {query_string,type} object
Graylog actually stores) had to be fixed before step_dashboard() settled
into a stable no-op; step_streams() was verified with a full inert
rule-add/remove and a metadata-change round-trip against the live API.
Confirmed live: "CRITICAL: unrecognized critical-severity syslog" (the
only alert with a 2-element streams list) kept reporting "config changed"
every single run even with no actual edit, because a plain dict/list ==
comparison in Python is order-sensitive and Graylog doesn't necessarily
return the streams array in the same order it was submitted in. List
values are now sorted before comparing on both sides, since list order
was never semantically meaningful here. Verified locally against a
same-elements-different-order case (now matches) and a genuinely-changed
case (still correctly detected as different) before pushing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The previous fix compared the whole live 'config' object against the
whole desired one - but Graylog's GET response fills in extra defaults
(query_parameters, filters, use_cron_scheduling, cron_expression,
cron_timezone, ...) that never appear in our alert JSON files. That made
every single alert compare as "different" on every single run, not just
the one actually edited - confirmed live: a re-run PUT-updated all 7
alerts when only alert7's query had changed.
Fixed by comparing only the keys our own files actually author (project
the live config down to just those keys before comparing), and switched
from interpolating JSON into python -c string literals to writing to temp
files and reading them - the former is fragile the moment a value contains
a quote or backslash.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Same class of bug just fixed in step_pipeline_rules()/step_pipelines():
title-only existence check meant an alert's config (query, threshold,
group_by) could never actually change on a re-run once created. Confirmed
live: alert7's query was corrected in git (vendor:accel-ppp ->
vendor:accel-ppp AND event_type:*) but a subsequent re-run still reported
"already exists" with the old, buggy query untouched. Now compares by the
'config' object (not just title) and PUT-updates + re-enables the
schedule if it differs, same pattern as the other two fixes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- step_pipeline_rules() and step_pipelines() only ever created new
rules/pipelines by title, never updated existing ones whose content
changed under the same title. Confirmed live this session: 5 new rules
were added to rules/*.json and created fine in Graylog, but "Servers
Parsing"'s stage list was never updated to actually call them, since
the pipeline already existed. Both functions now compare by 'source'
content and PUT-update if it differs, matching the pattern already used
elsewhere (e.g. step_inputs()'s timezone self-heal).
- alert7's query (vendor:accel-ppp) was too broad: it also matches
routine traffic tagged only by the generic accelppp_interface_tag
fallback rule, which sets vendor but never event_type - producing false
"(Empty Value)" group_by matches on ordinary volume. Confirmed live.
Fixed to vendor:accel-ppp AND event_type:* - severity_tag was
considered instead but rejected, since accelppp_router_address_error
(the rule that inspired this alert) never set it either.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- step_compose_files() now uses GRAYLOG_ADMIN_PASSWORD as the actual admin
password on a fresh install if set, instead of always generating a
random one. Solves the CI secret-staleness problem at the root: pin a
password once and it's correct both at creation time and on every later
re-run, instead of a fresh install randomly generating a password the
stored secret then has to be manually kept in sync with.
- deploy-from-scratch.yml gained a `cores` workflow_dispatch input
(default 4), passed through to create-graylog-lxc.sh's --cores flag.
Only takes effect when the container is actually created fresh, same as
every other --cores usage in this project.
- Documented both in README.md/README.uk.md.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Introduced by an earlier edit to .admin_credentials_ONE_TIME's own
instructional text ("...you've stored the password: rm <path>"), which
itself contains "password: " - the same substring resolve_admin_password()
greps for. grep -oP matched BOTH occurrences; command substitution joined
them with a real newline, producing a corrupted two-line "password" that
never matched .env's GRAYLOG_ROOT_PASSWORD_SHA2, causing every gcurl call
to silently 401 and wait_for_api_ready to time out no matter how generous
the timeout was (confirmed by re-verifying live: the real password,
extracted correctly, hashes to exactly what .env already has).
Fixed both ends: reworded the instructional text to not repeat "password:",
and hardened the regex itself (^Graylog admin password: anchor, grep -m1)
so a future wording change can't reintroduce the same class of bug.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Confirmed live: on a CI-triggered run, the REST API genuinely took longer
than 60s to accept authenticated requests even though the container had
already reported healthy - verified after the fact that both the API and
the admin credentials were fine, this was purely insufficient margin
(likely due to concurrent load during the run), not a logic bug. 180s
matches the same order of magnitude as the existing 300s docker-health
wait elsewhere in this script.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Docker's healthcheck can report graylog-server "healthy" a few seconds
before the REST API is actually ready to serve authenticated requests -
confirmed live: the first gcurl call (step_index_retention) intermittently
got an empty response body, crashing the downstream `python3 -c
"json.load(sys.stdin)"` with "Expecting value: line 1 column 1".
Adds wait_for_api_ready(), polling the same endpoint step_index_retention
already needs (up to 60s) before proceeding, mirroring the existing
retry-loop pattern already used for docker compose pull.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- export LC_ALL=C.UTF-8 LANG=C.UTF-8 at the top of the script: the
container inherits LANG=en_US.UTF-8 from pct exec's calling shell but
never generates that locale, so every apt-get call printed "Setting
locale failed" warnings from perl/apt-listchanges. C.UTF-8 is glibc-
builtin, no locale-gen needed.
- The final summary now prints the generated admin user/password directly
and deletes .admin_credentials_ONE_TIME right after, instead of just
pointing at the file and leaving it for the operator to read and clean
up by hand.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Two flood-detection alerts (per-source message volume, calibrated live
against real traffic) grouped by gl2_remote_ip
- Session correlation: accelppp_interface fallback tagging plus
radius_session_id/calling_station_id/radius_username extraction, so a
subscriber's full session lifecycle is searchable by one key
- Replace the single combined dashboard with three focused ones (Overview
& Alerts, Network Equipment, Servers & Sessions)
- Propagate GRAYLOG_ROOT_TIMEZONE and IP-in-alerts fixes into the reusable
install script and templates
- Add a Forgejo Actions workflow (manual trigger) that re-runs
install-graylog.sh on a self-hosted runner living in the container,
automating the deploy step this project has done by hand all along
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
