PMM-15304 Fix node type detection - #5870
Conversation
PMM Server always runs in a container, but its own Inventory Node was created as "generic": the setup fixtures hardcoded GenericNodeType, both for a single server and for every HA replica. The client-side heuristic behind the node-type default was broken too. checkContainer() only looked for "/docker/" or "/lxc/" in /proc/1/cgroup, which under cgroup v2 holds just "0::/" - so Docker, Podman and Kubernetes all read as a plain host. It now checks the runtime marker files, the "container" and KUBERNETES_SERVICE_HOST variables, and a wider set of cgroup v1 paths. pmm-admin register hardcoded "generic" as its default instead of using the detected ${nodeTypeDefault} that pmm-admin config already honors. Existing Nodes keep the type they were registered with; only fresh installs get the corrected fixture. Signed-off-by: Alex Demidoff <a@demidoff.me>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #5870 +/- ##
==========================================
+ Coverage 43.59% 45.90% +2.30%
==========================================
Files 415 417 +2
Lines 43134 43689 +555
==========================================
+ Hits 18804 20054 +1250
+ Misses 22454 21646 -808
- Partials 1876 1989 +113 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (2)
🔗 Linked repositories identifiedCodeRabbit considers these linked repositories for cross-repo context during reviews:
Included review availability: Your plan provides up to 4 included reviews per hour; 2 remain after this review. WalkthroughThe change expands container detection for Docker, Podman, LXC, Kubernetes, environment markers, and cgroup identifiers. Sequence Diagram(s)sequenceDiagram
participant Caller
participant nodeinfo.Get
participant checkContainer
participant Filesystem
participant Environment
Caller->>nodeinfo.Get: Request node information
nodeinfo.Get->>checkContainer: Pass root filesystem
checkContainer->>Filesystem: Read runtime markers and cgroup data
checkContainer->>Environment: Inspect container variables
checkContainer-->>nodeinfo.Get: Return container status
nodeinfo.Get-->>Caller: Return node information
Merge Risk: ⚪ Minimal · up to This PR corrects container node classification and registration defaults without introducing an actionable merge-blocking risk; it is merge-ready after normal checks and review. 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Full details: Description checkExplanation The description includes the ticket number, feature build link, problem statement, root causes, implementation details, scope limitations, and verification results. The optional related-work section is not required, and no API endpoint changes are described. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
systemd sets "container" in PID 1's environment only and strips it from the services it starts, so an agent running as a unit inside LXC or LXD on cgroup v2 saw no signal at all. Check /run/systemd/container, which systemd writes when it boots as PID 1 in a container. Also match /system.slice/docker-<id>.scope, the path form the systemd cgroup driver produces, which "/docker/" cannot. Signed-off-by: Alex Demidoff <a@demidoff.me>
Ticket number: PMM-15304
Feature build: Percona-Lab/pmm-submodules#4555
PMM Server always runs in a container, yet its own Inventory Node showed up as
generic— most visibly in an HA deployment, where everypmm-ha-*replica sat next tocontainerclient nodes with the wrong type.Two independent causes:
managed/models/database.go:1640(HA) andmanaged/models/database.go:1672(single server) hardcodedGenericNodeTypein the setup fixtures. The internal pmm-agent is set up with--skip-registration, so it never goes through the client path where a node type is chosen — pmm-managed writes the row itself.checkContainer()inutils/nodeinfo/nodeinfo.gowas broken on cgroup v2. It only looked for/docker/or/lxc/in/proc/1/cgroup, which under cgroup v2 holds just0::/. Modern Docker, Podman and Kubernetes all read as a plain host. It now also checks the runtime marker files (/.dockerenv,/run/.containerenv), thecontainerandKUBERNETES_SERVICE_HOSTenvironment variables, and a wider set of cgroup v1 paths (/kubepods,containerd,crio-,libpod).Also fixed along the way:
pmm-admin registerhardcodeddefault:"generic"for its node-type argument instead of the${nodeTypeDefault}Kong variable thatadmin/cmd/bootstrap.goalready computes andpmm-admin configalready honors.Existing Nodes are deliberately left alone — no migration. Only fresh installs get the corrected fixture.
Verification
pmm-adminand ran it inside a real cgroup v2 Docker container:[<node-type>] Node type. One of: [generic,container]. Default: container. On a macOS host it still reportsgeneric.TestCheckContainerinutils/nodeinfo/nodeinfo_test.gocovers host/Docker/Podman/LXC/Kubernetes under both cgroup versions against a fake filesystem root. It replaces therequire.False(t, info.Container)assertion inTestGet, which only held because detection was broken and would now fail whenever the suite runs inside a container (make env TARGET=test-common).managed/models/node_helpers_test.go(includingFindNodesByType, where PMM Server now matches thecontainerfilter),managed/services/management/node_test.goandmanaged/services/qan/client_test.go.managed/modelsgreen via DB-backed run;managed/services/{qan,inventory,management,agents}andmanaged/servicesgreen against a live PostgreSQL.