Skip to content

Default EdgeNodeCluster stub to avoid 10-min volumemgr longhorn-wait on EVE-k single-node #1193

Description

@eriknordmark

Eden should publish a default EdgeNodeCluster to avoid 10-min volumemgr wait on EVE-k

Summary

On EVE-k single-node devices brought up by eden, volumemgr defaults
to a 10-minute wait for longhorn at startup. The wait is gated by EVE's
behavior — see lf-edge/eve#6018 — and the device side fix is to flip
waitForLhFlag to false by default. But there is a smaller side that
eden can fix today: publish a minimal EdgeNodeCluster from the
controller so that pillar's parseEdgeNodeClusterConfig produces an
EdgeNodeClusterConfig with Valid=true and a non-ReplicatedStorage
type. With that ENCC published, volumemgr's existing code path skips
the longhorn-readiness sub-wait, avoiding the 10-min stall.

Currently eden never sets EdgeDevConfig.cluster, so pillar always
publishes an empty ENCC (Initialized=true, Valid=false), and
volumemgr's longhorn wait fires on every EVE-k boot.

Proposed fix

(implemented in PR linked below)

  1. Add cluster *config.EdgeNodeCluster to device.Ctx with
    GetCluster() / SetCluster() accessors.
  2. Wire EdgeDevConfig.Cluster = dev.GetCluster() in
    pkg/controller/device.go's GetConfigBytes.
  3. Default every device created via CreateEdgeNode() to a loopback
    stub:
    ClusterName:      "eden-stub"
    ClusterId:        "00000000-0000-0000-0000-000000000001"
    ClusterInterface: "lo"
    ClusterIpPrefix:  "127.0.0.1/32"
    IsWorkerNode:     false
    JoinServerIp:     "127.0.0.1"
    ClusterType:      CLUSTER_TYPE_K3S_BASE
    
    The values are consistent with "this is the only node"
    (joinServerIp == clusterIpPrefix address → BootstrapNode=true
    in pillar). No actual clustering happens; only the
    Valid=true && ClusterType != ReplicatedStorage predicate matters.
  4. Add eden controller edge-node cluster-set --type=... and
    cluster-clear CLI commands for tests that want a different
    ClusterType or pillar's no-cluster branch explicitly.

Why this is safe

  • The stub publishes Valid=true with a benign loopback config. It
    doesn't cause pillar to attempt any actual clustering — that requires
    much more (cluster networking, kube-vip, etc.).
  • Tests that want to verify pillar's behavior under "no cluster config"
    can call cluster-clear to drop the stub.

Verified against

Reproducer observed during a cross-HV upgrade test on
baseos-hv-check-volume-gate (kvm→k upgrade). After eve-k boots:

  • kubectl get nodes Ready
  • kubectl get ns shows no longhorn-system
  • logread | grep volumemgr shows "volumemgr run: wait for kubernetes"
  • /run/volumemgr/ContentTreeStatus/ empty for ~10 min
  • After 10 min, WaitForKubernetes returns timeout, volumemgr proceeds.

This delays every baseos update by ~10 min until volumemgr unblocks.

Linked

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions