Ansible Role: SeaweedFS

Install and configure SeaweedFS on Debian/Ubuntu. Deploys the four components — master, volume, filer, s3 gateway — as separate systemd services. Each component can be enabled independently so a single host can run the full stack or you can split roles across machines.

More information on our deployment on Hetzner Hardware server can be found in this Blog Article.

Requirements

• Ansible ≥ 2.14
• Debian 12 (bookworm) / Ubuntu 22.04 (jammy) / Ubuntu 24.04 (noble) / Ubuntu 26.04
• A user with become privileges
• Outbound HTTPS to github.com for the binary download (or override seaweedfs_download_url with a local mirror)

Quick start

# inventory/hosts.yml
all:
  hosts:
    storage01.example.com:

# group_vars/all.yml (use ansible-vault for secrets)
seaweedfs_version: "4.34"
seaweedfs_s3_admin_access_key: "admin"
seaweedfs_s3_admin_secret_key: !vault |
  $ANSIBLE_VAULT;1.1;AES256
  ...

# playbook.yml
- hosts: storage01.example.com
  become: true
  roles:
    - we-manage.seaweedfs

Run it:

ansible-playbook -i inventory/hosts.yml playbook.yml --ask-vault-pass

After it converges, the four services are reachable on:

Service	Port	Purpose
master	9333	Cluster coordinator + admin web UI
volume	8080	Chunk storage
filer	8888	POSIX-like filesystem on top of weed
s3	8333	S3-compatible HTTP API

Test the S3 gateway:

aws --endpoint-url http://storage01.example.com:8333 \
    --no-verify-ssl s3 ls

Component layout

The role assumes a single-node "everything together" deployment by default. Split deployments are done by toggling the component flags per host:

# master-only hosts
seaweedfs_master_enabled: true
seaweedfs_volume_enabled: false
seaweedfs_filer_enabled: false
seaweedfs_s3_enabled:     false

# volume-only hosts
seaweedfs_master_enabled: false
seaweedfs_volume_enabled: true
seaweedfs_filer_enabled: false
seaweedfs_s3_enabled:     false
seaweedfs_volume_masters: "master1.example.com:9333,master2.example.com:9333"

Variables

Versioning and download

Variable	Default	Notes
seaweedfs_version	"4.34"	Tag from the upstream GitHub releases.
seaweedfs_arch	"amd64"	amd64 or arm64.
seaweedfs_download_url	upstream GitHub release tarball	Override for an internal mirror.
seaweedfs_checksum	""	Format sha256:abc.... Recommended for reproducible installs.

Paths and user

Variable	Default
seaweedfs_bin_dir	/usr/local/bin
seaweedfs_user	seaweedfs
seaweedfs_group	seaweedfs
seaweedfs_data_dir	/var/lib/seaweedfs
seaweedfs_config_dir	/etc/seaweedfs
seaweedfs_log_dir	/var/log/seaweedfs
seaweedfs_bind_ip	ansible_default_ipv4.address

Component toggles

Variable	Default
seaweedfs_master_enabled	true
seaweedfs_volume_enabled	true
seaweedfs_filer_enabled	true
seaweedfs_s3_enabled	true

Master

Variable	Default
seaweedfs_master_port	9333
seaweedfs_master_port_grpc	19333
seaweedfs_master_metrics_port	9321
seaweedfs_master_dir	{{ data_dir }}/master
seaweedfs_master_peers	[] — list of host:port
seaweedfs_default_replication	"000"
seaweedfs_volume_size_limit_mb	30000
seaweedfs_volume_preallocate	false

Volume

Variable	Default
seaweedfs_volume_port	8080
seaweedfs_volume_port_grpc	18080
seaweedfs_volume_metrics_port	9322
seaweedfs_volume_dirs	["{{ data_dir }}/volumes"]
seaweedfs_volume_masters	"localhost:{{ master_port }}"
seaweedfs_volume_max	4000 (0 = auto)
seaweedfs_volume_disk_type	hdd
seaweedfs_volume_data_center	""
seaweedfs_volume_rack	""
seaweedfs_volume_index	memory
seaweedfs_volume_extra_opts	""

Filer

Variable	Default
seaweedfs_filer_port	8888
seaweedfs_filer_port_grpc	18888
seaweedfs_filer_metrics_port	9323
seaweedfs_filer_dir	{{ data_dir }}/filer
seaweedfs_filer_masters	"localhost:{{ master_port }}"
seaweedfs_filer_store	leveldb2

Override templates/filer.toml.j2 for postgres/mysql/redis backends.

S3 gateway

Variable	Default
seaweedfs_s3_port	8333
seaweedfs_s3_metrics_port	9324
seaweedfs_s3_config	{{ config_dir }}/s3.json
seaweedfs_s3_domain_name	""
seaweedfs_s3_allow_empty_folder	false
seaweedfs_s3_admin_access_key	"" — REQUIRED when S3 is enabled
seaweedfs_s3_admin_secret_key	"" — REQUIRED when S3 is enabled

Per-bucket credentials are managed at runtime via weed shell s3.configure, not by this role.

The role writes /etc/seaweedfs/s3.json only when it does not yet exist (force: false). The file is the runtime source of truth for all identities — admin plus everything added later via weed shell s3.configure -apply. Rewriting it from the template would silently wipe per-customer identities. To rotate the admin key on an existing host:

weed shell -master=localhost:9333 -filer=localhost:8888 <<<\
  "s3.configure -access_key=admin -secret_key=NEW -user=admin \
   -actions=Admin,Read,Write,List,Tagging -apply"

Managing buckets

This role only provisions the admin identity. To create per-tenant buckets and credentials, talk to weed shell directly. Example:

ssh storage01.example.com
weed shell -master=localhost:9333 -filer=localhost:8888 <<'EOF'
s3.bucket.create -name tenant-a
s3.configure -access_key=tenant-a -secret_key=$(openssl rand -hex 24) \
             -buckets=tenant-a -user=tenant-a \
             -actions=Read,Write,List,Tagging -apply
EOF

Bucket-level credentials persist in the filer and survive restarts.

The running s3 gateway picks new identities up live via its filer subscription — usually. We have seen that subscription go stale: the filer has the identity, the gateway keeps returning 403 AccessDenied. If a freshly added identity does not work, restart the gateway:

systemctl restart seaweedfs-s3

That rebuilds the subscription from scratch and pulls every current identity from the filer.

HA / multi-master

Run an odd number of masters (3 or 5) on dedicated hosts:

# group_vars/seaweedfs_masters.yml
seaweedfs_master_enabled: true
seaweedfs_volume_enabled: false
seaweedfs_filer_enabled: false
seaweedfs_s3_enabled:     false
seaweedfs_master_peers:
  - "master1.example.com:9333"
  - "master2.example.com:9333"
  - "master3.example.com:9333"

# group_vars/seaweedfs_volumes.yml
seaweedfs_master_enabled: false
seaweedfs_volume_enabled: true
seaweedfs_volume_masters: "master1.example.com:9333,master2.example.com:9333,master3.example.com:9333"
seaweedfs_default_replication: "001"  # or 010/100

The default leveldb2 filer store is not suitable for HA filer deployments — switch to a shared backend (postgres/mysql/redis) and override filer.toml.j2.

Things to be aware of

• No TLS: the role does not configure TLS for the public S3, master, or filer endpoints. Terminate TLS at a reverse proxy (nginx, caddy, haproxy) in front of the s3 gateway if you need it.
• Firewall: the role does not manage ufw/iptables. Open the relevant ports yourself.
• Volume binary upgrade restarts all four services, since the binary is shared. Plan upgrades during a maintenance window.
• seaweedfs_volume_max is a hard cap on the number of volumes per server. The default (4000) suits HDD-heavy hosts; reduce on small disks.

Inspecting the cluster

# master status / topology
curl http://localhost:9333/cluster/status
curl http://localhost:9333/dir/status

# volume server status
curl http://localhost:8080/status

# filer
curl http://localhost:8888/?pretty=y

# metrics (Prometheus)
curl http://localhost:9321/metrics   # master
curl http://localhost:9322/metrics   # volume
curl http://localhost:9323/metrics   # filer
curl http://localhost:9324/metrics   # s3

License

MIT — see LICENSE.