diff --git a/README.md b/README.md index 4c3fb6fcb7..105f069d4b 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ KubeBlocks add-ons. | orchestrator | orchestrator-3.2.6 | Orchestrator is a MySQL high availability and replication management tool, runs as a service and provides command line access, HTTP API and Web interface. | ApeCloud | | orioledb | orioledb-16.4.0 | OrioleDB is a new storage engine for PostgreSQL, bringing a modern approach to database capacity, capabilities and performance to the world's most-loved database platform. | ApeCloud | | polardbx | polardbx-cdc-2.3.0
polardbx-cn-2.3.0
polardbx-dn-2.3.0
polardbx-gms-2.3.0 | PolarDB-X is a cloud native distributed SQL Database designed for high concurrency, massive storage, complex querying scenarios. | ApeCloud Vettal Wu | -| postgresql | postgresql-12.14.0
postgresql-12.14.1
postgresql-12.15.0
postgresql-12.22.0
postgresql-13.23.0
postgresql-14.18.0
postgresql-14.23.0
postgresql-14.7.2
postgresql-14.8.0
postgresql-15.13.0
postgresql-15.18.0
postgresql-15.7.0
postgresql-16.14.0
postgresql-16.4.0
postgresql-16.9.0
postgresql-17.10.0
postgresql-17.5.0
postgresql-18.1.0
postgresql-18.4.0 | A PostgreSQL (with Patroni HA) cluster definition Helm chart for Kubernetes | ldming dingshun-cmss | +| postgresql | pgbouncer-1.25.2
postgresql-12.14.0
postgresql-12.14.1
postgresql-12.15.0
postgresql-12.22.0
postgresql-13.23.0
postgresql-14.18.0
postgresql-14.23.0
postgresql-14.7.2
postgresql-14.8.0
postgresql-15.13.0
postgresql-15.18.0
postgresql-15.7.0
postgresql-16.14.0
postgresql-16.4.0
postgresql-16.9.0
postgresql-17.10.0
postgresql-17.5.0
postgresql-18.1.0
postgresql-18.4.0 | A PostgreSQL (with Patroni HA) cluster definition Helm chart for Kubernetes | ldming dingshun-cmss | | pulsar | pulsar-bookies-recovery-2.11.2
pulsar-bookies-recovery-3.0.2
pulsar-bookies-recovery-4.0.6
pulsar-bookkeeper-2.11.2
pulsar-bookkeeper-3.0.2
pulsar-bookkeeper-4.0.6
pulsar-broker-2.11.2
pulsar-broker-3.0.2
pulsar-broker-4.0.6
pulsar-proxy-2.11.2
pulsar-proxy-3.0.2
pulsar-proxy-4.0.6
pulsar-zookeeper-2.11.2
pulsar-zookeeper-3.0.2 | Apache Pulsar is an open-source, distributed messaging and streaming platform built for the cloud. | cjc7373 caiq1nyu | | qdrant | qdrant-1.10.0
qdrant-1.13.4
qdrant-1.15.4
qdrant-1.15.5
qdrant-1.16.3
qdrant-1.17.1
qdrant-1.18.2
qdrant-1.19.0
qdrant-1.5.0
qdrant-1.7.3
qdrant-1.8.1
qdrant-1.8.4 | High-performance, massive-scale Vector Database for the next generation of AI. | cjc7373 | | rabbitmq | rabbitmq-3.10.25
rabbitmq-3.11.28
rabbitmq-3.12.14
rabbitmq-3.13.7
rabbitmq-3.8.34
rabbitmq-3.9.29
rabbitmq-4.0.9
rabbitmq-4.1.6
rabbitmq-4.2.1
rabbitmq-4.3.1 | RabbitMQ is a reliable and mature messaging and streaming broker. | xuriwuyun | diff --git a/addons-cluster/postgresql/Chart.yaml b/addons-cluster/postgresql/Chart.yaml index 1e3994bda9..0c002f5124 100644 --- a/addons-cluster/postgresql/Chart.yaml +++ b/addons-cluster/postgresql/Chart.yaml @@ -1,7 +1,7 @@ apiVersion: v2 name: postgresql-cluster type: application -version: 1.0.5 +version: 1.0.6 description: A PostgreSQL (with Patroni HA) cluster Helm chart for KubeBlocks. dependencies: diff --git a/addons/postgresql/Chart.yaml b/addons/postgresql/Chart.yaml index 3427e49d38..d7edd0d355 100644 --- a/addons/postgresql/Chart.yaml +++ b/addons/postgresql/Chart.yaml @@ -4,7 +4,7 @@ description: A PostgreSQL (with Patroni HA) cluster definition Helm chart for Ku type: application -version: 1.0.5 +version: 1.0.6 # The helm chart contains multiple kernel versions of PostgreSQL (with Patroni HA), # appVersion should be consistent with the highest PostgreSQL (with Patroni HA) kernel version. diff --git a/addons/postgresql/README.md b/addons/postgresql/README.md index 5b0eecdb34..25e0608e4f 100644 --- a/addons/postgresql/README.md +++ b/addons/postgresql/README.md @@ -10,6 +10,11 @@ PostgreSQL (Postgres) is an open source object-relational database known for rel |------------------|------------------------|-----------------------|-------------------|-----------|------------|-----------|--------|------------| | replication | Yes | Yes | Yes | Yes | Yes | Yes | Yes | Yes | +The `replication` topology includes a managed, stateless PgBouncer component. +An omitted PgBouncer component has zero replicas. At zero replicas, clients +connect directly to PostgreSQL on 5432. One or more replicas activate the +pooled endpoint on 6432 while the Cluster keeps the same topology. + ### Backup and Restore | Feature | Method | Description | @@ -166,6 +171,178 @@ NAME VERSIONS postgresql 18.4.0,18.1.0,17.10.0,17.5.0,16.14.0,16.9.0,16.4.0,15.18.0,15.13.0,15.7.0,14.23.0,14.18.0,14.8.0,14.7.2,13.23.0,12.22.0,12.15.0,12.14.1,12.14.0 Available Xd ``` +### PgBouncer connection pool + +The managed component runs PgBouncer 1.25.2. Zero replicas keep client traffic +on the direct PostgreSQL endpoint. One or more replicas activate the managed +pool. The default PgBouncer resources are 100m CPU and 128Mi memory requests, +with 500m CPU and 512Mi memory limits. + +#### Enable and disable + +To activate PgBouncer when creating a Cluster, add it to the `replication` +topology: + +```yaml +apiVersion: apps.kubeblocks.io/v1 +kind: Cluster +metadata: + name: pg-cluster + namespace: demo +spec: + clusterDef: postgresql + topology: replication + terminationPolicy: Delete + componentSpecs: + - name: postgresql + serviceVersion: "14.23.0" + replicas: 2 + resources: + requests: {cpu: "500m", memory: 512Mi} + limits: {cpu: "500m", memory: 512Mi} + volumeClaimTemplates: + - name: data + spec: + accessModes: [ReadWriteOnce] + resources: + requests: + storage: 20Gi + - name: pgbouncer + replicas: 2 + resources: + requests: {cpu: "100m", memory: 128Mi} + limits: {cpu: "500m", memory: 512Mi} +``` + +For a Cluster that already contains the `pgbouncer` component, activate two +replicas with a HorizontalScaling OpsRequest: + +```yaml +apiVersion: operations.kubeblocks.io/v1alpha1 +kind: OpsRequest +metadata: + name: pg-pgbouncer-enable + namespace: demo +spec: + clusterName: pg-cluster + type: HorizontalScaling + horizontalScaling: + - componentName: pgbouncer + scaleOut: + replicaChanges: 2 +``` + +Return the component to zero replicas with a second HorizontalScaling +OpsRequest: + +```yaml +apiVersion: operations.kubeblocks.io/v1alpha1 +kind: OpsRequest +metadata: + name: pg-pgbouncer-disable + namespace: demo +spec: + clusterName: pg-cluster + type: HorizontalScaling + horizontalScaling: + - componentName: pgbouncer + scaleIn: + replicaChanges: 2 +``` + +Clusters created with Addon 1.0.5 retain their in-Pod PgBouncer sidecar. After +upgrading such a Cluster, add the managed `pgbouncer` component at zero replicas +before using the scaling operations above. + +#### Connect + +The managed endpoint uses the following in-cluster address: + +```text +-pgbouncer..svc:6432 +``` + +For the example Cluster, connect to `pg-cluster-pgbouncer.demo.svc:6432`. +The Service receives endpoints after at least one PgBouncer replica reaches +Ready. Check the endpoint before publishing the address: + +```bash +kubectl get endpoints -n demo pg-cluster-pgbouncer +``` + +#### Configure + +PgBouncer starts with the following per-instance settings: + +| Parameter | Default | Allowed values | Scope | +|-----------|---------|----------------|-------| +| `pool_mode` | `session` | `session`, `transaction`, `statement` | New client connections | +| `max_client_conn` | `500` | `1..999999` | Client connections per PgBouncer instance | +| `default_pool_size` | `20` | `1..999999` | Backend connections per user/database pool | +| `min_pool_size` | `5` | `0..999999` | Retained backend connections per user/database pool | +| `reserve_pool_size` | `5` | `0..999999` | Reserve connections per user/database pool | +| `max_db_connections` | `80` | `0..999999`; `0` means unlimited | Backend connections per database and PgBouncer instance | +| `max_user_connections` | `80` | `0..999999`; `0` means unlimited | Backend connections per user and PgBouncer instance | + +PostgreSQL `max_connections` remains the global connection limit. Review the +PgBouncer settings after changing PostgreSQL capacity, database or user counts, +or the PgBouncer replica count. Also size memory and file descriptors for the +selected client limit. + +Apply parameter changes with a Reconfiguring OpsRequest. A successful +OpsRequest means that KubeBlocks accepted and rendered the requested +configuration. The KubeBlocks config manager then makes an asynchronous, +best-effort SIGHUP attempt for each running PgBouncer instance; the OpsRequest +status does not confirm that every instance has completed its reload. Pods are +not restarted by this reload action: + +```yaml +apiVersion: operations.kubeblocks.io/v1alpha1 +kind: OpsRequest +metadata: + name: pg-pgbouncer-reconfigure + namespace: demo +spec: + type: Reconfiguring + clusterName: pg-cluster + reconfigures: + - componentName: pgbouncer + parameters: + - key: max_client_conn + value: "1000" + - key: default_pool_size + value: "30" + - key: max_db_connections + value: "60" + - key: max_user_connections + value: "40" +``` + +KubeBlocks validates values against the documented ranges before applying +them. + +Configuration propagation and reload complete asynchronously across running +replicas. Confirm that each instance reports the requested values with +PgBouncer `SHOW CONFIG` before relying on new connection limits. + +#### Runtime behavior and requirements + +- Readiness requires a successful connection through PgBouncer to the writable + PostgreSQL primary. The same PgBouncer Pod becomes Ready automatically after + PostgreSQL recovers. +- Dynamic parameter reload uses the KubeBlocks 1.0 config-manager sidecar with + a shared process namespace and UID 0. The target namespace Pod Security policy + must permit this runtime; use a policy less restrictive than the Kubernetes + Restricted profile. The PgBouncer container continues to run as UID/GID 70 + with privilege escalation disabled and all capabilities dropped. +- Applications reconnect with backoff after a PostgreSQL primary change and + retry any interrupted transaction. Transaction and statement pooling modes + also require application compatibility testing. +- TLS-enforced Clusters use the direct PostgreSQL endpoint on 5432 in this + release. +- Clusters using the managed PgBouncer component stay on PostgreSQL Addon 1.0.6 + or later. + ### Horizontal scaling #### Scale-out diff --git a/addons/postgresql/config/pgbouncer-config-constraint.cue b/addons/postgresql/config/pgbouncer-config-constraint.cue new file mode 100644 index 0000000000..38e79dabe6 --- /dev/null +++ b/addons/postgresql/config/pgbouncer-config-constraint.cue @@ -0,0 +1,52 @@ +#PgBouncerParameter: { + // Pooling mode used for new client connections. + pool_mode?: "session" | "transaction" | "statement" | *"session" + + // Maximum number of client connections accepted by one PgBouncer instance. + max_client_conn?: int & >=1 & <=999999 | *500 + + // Default number of backend connections for each user/database pool. + default_pool_size?: int & >=1 & <=999999 | *20 + + // Minimum number of backend connections retained for each user/database pool. + min_pool_size?: int & >=0 & <=999999 | *5 + + // Additional backend connections allowed when a pool is exhausted. + reserve_pool_size?: int & >=0 & <=999999 | *5 + + // Maximum number of backend connections to one database per PgBouncer instance. Zero means unlimited. + max_db_connections?: int & >=0 & <=999999 | *80 + + // Maximum number of backend connections for one user per PgBouncer instance. Zero means unlimited. + max_user_connections?: int & >=0 & <=999999 | *80 +} + +#PgBouncerConfig: { + #PgBouncerParameter + + listen_addr: "*" + listen_port: 6432 + unix_socket_dir: "/tmp" + unix_socket_mode: "0770" + auth_file: "/etc/pgbouncer/userlist.txt" + auth_type: "md5" + auth_user: "postgres" + auth_query: "SELECT rolname, CASE WHEN rolvaliduntil IS NOT NULL AND rolvaliduntil < pg_catalog.now() THEN NULL ELSE rolpassword END FROM pg_catalog.pg_authid WHERE rolname=$1 AND rolcanlogin" + auth_dbname: "postgres" + admin_users: "postgres" + stats_users: "postgres" + client_tls_sslmode: "disable" + server_tls_sslmode: "disable" + ignore_startup_parameters: "extra_float_digits" + reserve_pool_timeout: 5 + server_idle_timeout: 600 + server_lifetime: 3600 + query_wait_timeout: 120 + client_idle_timeout: 0 +} + +#PgBouncerConfiguration: { + pgbouncer: #PgBouncerConfig +} + +configuration: #PgBouncerConfiguration diff --git a/addons/postgresql/config/pgbouncer-ini.tpl b/addons/postgresql/config/pgbouncer-ini.tpl index 3e5e9a2e09..3f9fa1cc19 100644 --- a/addons/postgresql/config/pgbouncer-ini.tpl +++ b/addons/postgresql/config/pgbouncer-ini.tpl @@ -1,21 +1,27 @@ [pgbouncer] listen_addr = * listen_port = 6432 -unix_socket_dir = /tmp/ -unix_socket_mode = 0777 -auth_file = /opt/bitnami/pgbouncer/conf/userlist.txt -auth_user = postgres -auth_query = SELECT usename, passwd FROM pg_shadow WHERE usename=$1 -pidfile =/opt/bitnami/pgbouncer/tmp/pgbouncer.pid -logfile =/opt/bitnami/pgbouncer/logs/pgbouncer.log +unix_socket_dir = /tmp +unix_socket_mode = 0770 +auth_file = /etc/pgbouncer/userlist.txt auth_type = md5 +auth_user = postgres +auth_query = SELECT rolname, CASE WHEN rolvaliduntil IS NOT NULL AND rolvaliduntil < pg_catalog.now() THEN NULL ELSE rolpassword END FROM pg_catalog.pg_authid WHERE rolname=$1 AND rolcanlogin +auth_dbname = postgres +admin_users = postgres +stats_users = postgres pool_mode = session +client_tls_sslmode = disable +server_tls_sslmode = disable ignore_startup_parameters = extra_float_digits -{{- $max_client_conn := 10000 }} -{{- $phy_memory := getContainerMemory ( index $.podSpec.containers 0 ) }} -{{- if gt $phy_memory 0 }} -{{- $max_client_conn = min ( div $phy_memory 9531392 ) 5000 }} -{{- end }} -max_client_conn = {{ $max_client_conn }} -admin_users = postgres -;;; [database] \ No newline at end of file +max_client_conn = 500 +default_pool_size = 20 +min_pool_size = 5 +reserve_pool_size = 5 +reserve_pool_timeout = 5 +max_db_connections = 80 +max_user_connections = 80 +server_idle_timeout = 600 +server_lifetime = 3600 +query_wait_timeout = 120 +client_idle_timeout = 0 diff --git a/addons/postgresql/scripts-ut-spec/pgbouncer_setup_spec.sh b/addons/postgresql/scripts-ut-spec/pgbouncer_setup_spec.sh new file mode 100644 index 0000000000..59ad8a8468 --- /dev/null +++ b/addons/postgresql/scripts-ut-spec/pgbouncer_setup_spec.sh @@ -0,0 +1,125 @@ +# shellcheck shell=sh +# shellcheck disable=SC2034,SC2317,SC2329 + +Describe "PostgreSQL PgBouncer setup contract" + Include ../scripts/pgbouncer-setup.sh + + setup() { + test_dir=$(mktemp -d -t pgbouncer-setup-XXXXXX) + mkdir -p "$test_dir/conf" "$test_dir/tmp" + sed -n '/^\[pgbouncer\]/,$p' ../config/pgbouncer-ini.tpl > "$test_dir/pgbouncer.ini.tpl" + pgbouncer_template_conf_file="$test_dir/pgbouncer.ini.tpl" + pgbouncer_conf_dir="$test_dir/conf" + pgbouncer_conf_file="$test_dir/conf/pgbouncer.ini" + pgbouncer_user_list_file="$test_dir/conf/userlist.txt" + POSTGRESQL_USERNAME=pgbouncer + POSTGRESQL_PASSWORD='pa"ss' + POSTGRESQL_HOST=sample-postgresql + POSTGRESQL_PORT=5432 + } + + cleanup() { + rm -rf "$test_dir" + } + + customize_pool_limits() { + sed \ + -e 's/max_client_conn = 500/max_client_conn = 1200/' \ + -e 's/default_pool_size = 20/default_pool_size = 30/' \ + -e 's/max_db_connections = 80/max_db_connections = 60/' \ + -e 's/max_user_connections = 80/max_user_connections = 40/' \ + "$pgbouncer_template_conf_file" > "$test_dir/custom.ini" + mv "$test_dir/custom.ini" "$pgbouncer_template_conf_file" + build_pgbouncer_conf + } + + rebuild_for_new_backend() { + build_pgbouncer_conf || return 1 + POSTGRESQL_HOST=replaced-postgresql + build_pgbouncer_conf + } + + BeforeEach 'setup' + AfterEach 'cleanup' + + It "doubles quotes in credentials before writing the auth file" + When call build_pgbouncer_conf + The status should be success + The path "$pgbouncer_user_list_file" should be file + The contents of file "$pgbouncer_user_list_file" should equal '"pgbouncer" "pa""ss"' + End + + It "loads the documented pool defaults from the managed configuration" + When call build_pgbouncer_conf + The status should be success + The contents of file "$pgbouncer_conf_file" should include "%include $pgbouncer_template_conf_file" + The contents of file "$pgbouncer_template_conf_file" should include "pool_mode = session" + The contents of file "$pgbouncer_template_conf_file" should include "max_client_conn = 500" + The contents of file "$pgbouncer_template_conf_file" should include "default_pool_size = 20" + The contents of file "$pgbouncer_template_conf_file" should include "min_pool_size = 5" + The contents of file "$pgbouncer_template_conf_file" should include "reserve_pool_size = 5" + The contents of file "$pgbouncer_template_conf_file" should include "max_db_connections = 80" + The contents of file "$pgbouncer_template_conf_file" should include "max_user_connections = 80" + End + + It "preserves user-provided pool limits without automatic recalculation" + When call customize_pool_limits + The status should be success + The contents of file "$pgbouncer_template_conf_file" should include "max_client_conn = 1200" + The contents of file "$pgbouncer_template_conf_file" should include "default_pool_size = 30" + The contents of file "$pgbouncer_template_conf_file" should include "max_db_connections = 60" + The contents of file "$pgbouncer_template_conf_file" should include "max_user_connections = 40" + End + + It "routes the PgBouncer component through the PostgreSQL Service" + POSTGRESQL_PORT=5433 + When call build_pgbouncer_conf + The status should be success + The contents of file "$pgbouncer_conf_file" should include "postgres=host=sample-postgresql port=5433 dbname=postgres" + The contents of file "$pgbouncer_conf_file" should include "*=host=sample-postgresql port=5433" + End + + It "atomically replaces an existing generated configuration" + When call rebuild_for_new_backend + The status should be success + The contents of file "$pgbouncer_conf_file" should include "host=replaced-postgresql" + The contents of file "$pgbouncer_conf_file" should not include "host=sample-postgresql" + End + + It "references and extends a read-only managed configuration" + chmod 0444 "$pgbouncer_template_conf_file" + When call build_pgbouncer_conf + The status should be success + The contents of file "$pgbouncer_conf_file" should include "%include $pgbouncer_template_conf_file" + The contents of file "$pgbouncer_conf_file" should include "[databases]" + End + + It "rejects line breaks in credentials before writing files" + POSTGRESQL_PASSWORD=$(printf 'bad\nsecret') + When call build_pgbouncer_conf + The status should be failure + The stderr should include "credentials contain an unsupported line break" + The path "$pgbouncer_user_list_file" should not be exist + End + + It "fails when the required PostgreSQL Service host is missing" + unset POSTGRESQL_HOST + When call build_pgbouncer_conf + The status should be failure + The stderr should include "POSTGRESQL_HOST or POSTGRESQL_PORT is not set" + End + + It "rejects an invalid PostgreSQL Service port" + POSTGRESQL_PORT=70000 + When call build_pgbouncer_conf + The status should be failure + The stderr should include "outside 1..65535" + End + + It "fails when the managed configuration is unavailable" + rm -f "$pgbouncer_template_conf_file" + When call build_pgbouncer_conf + The status should be failure + The stderr should include "configuration is not readable" + End +End diff --git a/addons/postgresql/scripts-ut-spec/version_matrix_spec.sh b/addons/postgresql/scripts-ut-spec/version_matrix_spec.sh index b3497391b5..d6186d1528 100644 --- a/addons/postgresql/scripts-ut-spec/version_matrix_spec.sh +++ b/addons/postgresql/scripts-ut-spec/version_matrix_spec.sh @@ -1,4 +1,5 @@ # shellcheck shell=sh +# shellcheck disable=SC2016 # Ruby snippets are intentionally single-quoted. Describe "PostgreSQL version matrix contract" @@ -23,6 +24,215 @@ Describe "PostgreSQL version matrix contract" render_chart | grep -c "$pattern" || true } + pgbouncer_component_contract() { + render_chart | RUBYOPT=-W0 ruby -ryaml -e ' + documents = YAML.load_stream(ARGF.read).compact + cluster_definition = documents.find { |document| document["kind"] == "ClusterDefinition" } + topologies = cluster_definition.dig("spec", "topologies") + abort unless topologies.map { |topology| topology["name"] } == ["replication"] + replication = topologies.first + abort unless replication["components"] == [ + {"name" => "postgresql", "compDef" => "postgresql-"}, + {"name" => "pgbouncer", "compDef" => "pgbouncer-1.0.6"} + ] + + definitions = documents.select { |document| document["kind"] == "ComponentDefinition" } + postgres = definitions.find { |definition| definition.dig("metadata", "name") == "postgresql-14-1.0.6" } + pgbouncer = definitions.find { |definition| definition.dig("metadata", "name") == "pgbouncer-1.0.6" } + abort unless pgbouncer.dig("spec", "replicasLimit") == {"minReplicas" => 0, "maxReplicas" => 64} + abort unless pgbouncer.dig("spec", "serviceVersion") == "1.25.2" + abort unless pgbouncer.dig("spec", "services", 0, "spec", "ports", 0, "port") == 6432 + abort if pgbouncer.dig("spec", "services", 0).key?("serviceName") + resources = pgbouncer.dig("spec", "runtime", "containers", 0, "resources") + abort unless resources == { + "requests" => {"cpu" => "100m", "memory" => "128Mi"}, + "limits" => {"cpu" => "500m", "memory" => "512Mi"} + } + refs = pgbouncer.dig("spec", "vars").map do |var| + var.dig("valueFrom", "serviceVarRef", "compDef") || var.dig("valueFrom", "credentialVarRef", "compDef") + end.compact + abort unless refs == Array.new(4, "postgresql-") + abort if pgbouncer.dig("spec", "vars").any? { |var| var["name"].start_with?("PGBOUNCER_") } + configs = pgbouncer.dig("spec", "configs") + abort unless configs.map { |config| config["name"] } == ["pgbouncer-configuration"] + abort unless configs.all? { |config| config["externalManaged"] == true && config["defaultMode"] == 0444 } + abort if configs.any? { |config| config.key?("restartOnFileChange") } + abort unless configs.map { |config| config["volumeName"] } == ["pgbouncer-config"] + + postgres_ports = postgres.dig("spec", "services", 0, "spec", "ports").map { |port| port["port"] } + abort unless postgres_ports == [5432] + postgres_containers = postgres.dig("spec", "runtime", "containers").map { |container| container["name"] } + abort if postgres_containers.include?("pgbouncer") + postgres_pcr = documents.find do |document| + document["kind"] == "ParamConfigRenderer" && document.dig("metadata", "name") == "postgresql14-pcr-1.0.6" + end + abort unless postgres_pcr.dig("spec", "configs").map { |config| config["name"] } == ["postgresql.conf"] + pgbouncer_pcr = documents.find do |document| + document["kind"] == "ParamConfigRenderer" && + document.dig("metadata", "name") == "pgbouncer-pcr-1.0.6" + end + abort unless pgbouncer_pcr.dig("spec", "componentDef") == "pgbouncer-1.0.6" + abort unless pgbouncer_pcr.dig("spec", "serviceVersion") == "1.25.2" + abort unless pgbouncer_pcr.dig("spec", "parametersDefs") == ["pgbouncer-pd-1.0.6"] + pgbouncer_format = pgbouncer_pcr.dig("spec", "configs", 0) + abort unless pgbouncer_format["name"] == "pgbouncer.ini" + abort unless pgbouncer_format.dig("fileFormatConfig", "format") == "ini" + abort unless pgbouncer_format.dig("fileFormatConfig", "iniConfig", "sectionName") == "pgbouncer" + + pgbouncer_pd = documents.find do |document| + document["kind"] == "ParametersDefinition" && + document.dig("metadata", "name") == "pgbouncer-pd-1.0.6" + end + exposed = %w[ + pool_mode max_client_conn default_pool_size min_pool_size + reserve_pool_size max_db_connections max_user_connections + ] + abort unless pgbouncer_pd.dig("spec", "fileName") == "pgbouncer.ini" + abort unless pgbouncer_pd.dig("spec", "dynamicParameters") == exposed + abort unless pgbouncer_pd.dig("spec", "parametersSchema", "topLevelKey") == "PgBouncerParameter" + reload_action = pgbouncer_pd.dig("spec", "reloadAction", "unixSignalTrigger") + abort unless reload_action == {"signal" => "SIGHUP", "processName" => "pgbouncer"} + abort if pgbouncer_pd.dig("spec").key?("staticParameters") + pgbouncer_schema = pgbouncer_pd.dig("spec", "parametersSchema", "cue") + exposed.each { |parameter| abort unless pgbouncer_schema.include?(parameter) } + abort unless pgbouncer_schema.include?(%q{pool_mode?: "session" | "transaction" | "statement" | *"session"}) + abort unless pgbouncer_schema.include?("max_client_conn?: int & >=1 & <=999999 | *500") + abort unless pgbouncer_schema.include?("max_db_connections?: int & >=0 & <=999999 | *80") + abort unless pgbouncer_schema.include?("#PgBouncerConfiguration: {\n\tpgbouncer: #PgBouncerConfig\n}") + abort unless pgbouncer_schema.include?("configuration: #PgBouncerConfiguration") + abort unless pgbouncer_schema.include?(%q{auth_type: "md5"}) + abort if pgbouncer_schema.match?(/^\s*\.\.\.\s*$/) + account_refs = pgbouncer.dig("spec", "vars").map { |var| var.dig("valueFrom", "credentialVarRef", "name") }.compact + abort unless account_refs == ["postgres", "postgres"] + probe = pgbouncer.dig("spec", "runtime", "containers", 0, "readinessProbe", "exec", "command", -1) + abort unless probe.include?("--port=6432") && probe.include?("$POSTGRESQL_USERNAME") + abort unless probe.include?("$CURRENT_POD_IP") && probe.include?("NOT pg_is_in_recovery()") + abort unless probe.include?("current_setting(\u0027transaction_read_only\u0027) = \u0027off\u0027") + abort if probe.include?("budget-ready") || probe.include?("budget-sync") + readiness = pgbouncer.dig("spec", "runtime", "containers", 0, "readinessProbe") + abort unless readiness["failureThreshold"] == 1 && readiness["periodSeconds"] == 5 + pod_security = pgbouncer.dig("spec", "runtime", "securityContext") + abort unless pod_security == { + "runAsUser" => 70, "runAsGroup" => 70, + "fsGroup" => 70, "fsGroupChangePolicy" => "OnRootMismatch" + } + volumes = pgbouncer.dig("spec", "runtime", "volumes") + abort unless volumes == [{"name" => "pgbouncer-state", "emptyDir" => {}}] + container = pgbouncer.dig("spec", "runtime", "containers", 0) + container_security = container["securityContext"] + abort unless container_security["runAsNonRoot"] == true + abort unless container_security["runAsUser"] == 70 && container_security["runAsGroup"] == 70 + abort unless container_security["allowPrivilegeEscalation"] == false + abort unless container_security.dig("capabilities", "drop") == ["ALL"] + config_mount = container["volumeMounts"].find { |mount| mount["name"] == "pgbouncer-config" } + abort unless config_mount["mountPath"] == "/opt/pgbouncer-template" + state_mount = container["volumeMounts"].find { |mount| mount["name"] == "pgbouncer-state" } + abort unless state_mount["mountPath"] == "/etc/pgbouncer" + abort if container["volumeMounts"].any? { |mount| mount["mountPath"] == "/var/run/pgbouncer" } + env = pgbouncer.dig("spec", "runtime", "containers", 0, "env") + pod_ip = env.find { |entry| entry["name"] == "CURRENT_POD_IP" } + abort unless pod_ip.dig("valueFrom", "fieldRef", "fieldPath") == "status.podIP" + abort unless env.map { |entry| entry["name"] } == [ + "CURRENT_POD_IP", "POSTGRESQL_HOST", "POSTGRESQL_PORT", "POSTGRESQL_USERNAME", "POSTGRESQL_PASSWORD" + ] + pgbouncer_config = documents.find do |document| + document["kind"] == "ConfigMap" && + document.dig("metadata", "name") == "pgbouncer-configuration-1.0.6" + end.dig("data", "pgbouncer.ini") + abort unless pgbouncer_config.lines.any? { |line| line.match?(/^\s*listen_addr\s*=\s*\*\s*$/) } + abort unless pgbouncer_config.lines.any? { |line| line.match?(/^\s*client_tls_sslmode\s*=\s*disable\s*$/) } + abort unless pgbouncer_config.lines.any? { |line| line.match?(/^\s*server_tls_sslmode\s*=\s*disable\s*$/) } + abort unless pgbouncer_config.include?("auth_file = /etc/pgbouncer/userlist.txt") + abort unless pgbouncer_config.include?("FROM pg_catalog.pg_authid") + abort unless pgbouncer_config.include?("rolvaliduntil") && pgbouncer_config.include?("rolcanlogin") + abort if pgbouncer_config.match?(/^\s*pidfile\s*=/) + abort if pgbouncer_config.include?("{{") + abort unless pgbouncer_config.include?("pool_mode = session") + abort unless pgbouncer_config.include?("max_client_conn = 500") + abort unless pgbouncer_config.include?("default_pool_size = 20") + abort unless pgbouncer_config.include?("min_pool_size = 5") + abort unless pgbouncer_config.include?("reserve_pool_size = 5") + abort unless pgbouncer_config.include?("max_db_connections = 80") + abort unless pgbouncer_config.include?("max_user_connections = 80") + abort if pgbouncer_config.lines.any? { |line| line.match?(/^\s*logfile\s*=\s*\/dev\/stderr\s*$/) } + main_config = documents.find do |document| + document["kind"] == "ConfigMap" && document.dig("metadata", "name") == "pgbouncer-configuration-1.0.6" + end + abort unless main_config["data"].keys == ["pgbouncer.ini"] + abort if documents.any? do |document| + document["kind"] == "ConfigMap" && document.dig("metadata", "name").to_s.include?("pgbouncer-budget") + end + + version = documents.find do |document| + document["kind"] == "ComponentVersion" && document.dig("metadata", "name") == "pgbouncer" + end + abort unless version.dig("spec", "compatibilityRules", 0, "compDefs") == ["pgbouncer-"] + release = version.dig("spec", "releases", 0) + abort unless release["name"] == "1.25.2" && release["serviceVersion"] == "1.25.2" + abort unless release.dig("images", "pgbouncer") == "docker.io/apecloud/pgbouncer:1.25.2" + postgres_version = documents.find do |document| + document["kind"] == "ComponentVersion" && document.dig("metadata", "name") == "postgresql" + end + abort unless postgres_version["spec"]["releases"].all? do |postgres_release| + postgres_release.dig("images", "pgbouncer") == "docker.io/apecloud/pgbouncer:1.19.0" + end + puts "ok" + ' + } + + pgbouncer_runtime_script_contract() { + script="$(chart_dir)/scripts/pgbouncer-setup.sh" + + test "$(sed -n '1p' "$script")" = '#!/bin/sh' || return 1 + grep -Fq 'exec "$pgbouncer_bin" "$pgbouncer_conf_file"' "$script" || return 1 + grep -Fq 'pgbouncer_backend_host="${POSTGRESQL_HOST:-}"' "$script" || return 1 + grep -Fq 'pgbouncer_backend_port="${POSTGRESQL_PORT:-}"' "$script" || return 1 + grep -Fq 'validate_runtime_inputs()' "$script" || return 1 + grep -Fq 'mktemp "${pgbouncer_conf_dir}/.pgbouncer.ini.XXXXXX"' "$script" || return 1 + grep -Fq "printf '%%include %s\\n\\n[databases]\\n'" "$script" || return 1 + + for duplicated_schema in validate_component_inputs validate_pgbouncer_template validate_pool_integer; do + if grep -Fq "$duplicated_schema" "$script"; then + return 1 + fi + done + + for dynamic in pgbouncer-budget PGBOUNCER_MEMORY_BYTES PGBOUNCER_DESIRED_REPLICAS \ + current_setting sync_backend_budget 'SHOW CONFIG' 'RELOAD;'; do + if grep -Fq "$dynamic" "$script"; then + return 1 + fi + done + + for sidecar_artifact in /opt/bitnami /etc/passwd /etc/group useradd 'su pgbouncer' CURRENT_POD_IP; do + if grep -Fq "$sidecar_artifact" "$script"; then + return 1 + fi + done + } + + pgbouncer_image_and_pull_policy_contract() { + helm template kb-addon-postgresql "$(chart_dir)" --namespace kb-system --dependency-update \ + --set pgbouncer.componentImage.versions[0].version=1.25.2 \ + --set pgbouncer.componentImage.versions[0].tag=1.25.2 \ + --set pgbouncer.componentImage.versions[0].isDefault=false \ + --set pgbouncer.componentImage.versions[1].version=1.25.2-test \ + --set pgbouncer.componentImage.versions[1].tag=1.25.2-test \ + --set pgbouncer.componentImage.versions[1].isDefault=true \ + --set pgbouncer.componentImage.pullPolicy=Always | RUBYOPT=-W0 ruby -ryaml -e ' + documents = YAML.load_stream(ARGF.read).compact + version = documents.find { |document| document["kind"] == "ComponentVersion" && document.dig("metadata", "name") == "pgbouncer" } + abort unless version.dig("spec", "compatibilityRules", 0, "releases") == ["1.25.2", "1.25.2-test"] + abort unless version.dig("spec", "releases").map { |release| release["name"] } == ["1.25.2", "1.25.2-test"] + abort unless version.dig("spec", "releases", 0, "images", "pgbouncer") == "docker.io/apecloud/pgbouncer:1.25.2" + abort unless version.dig("spec", "releases", 1, "images", "pgbouncer") == "docker.io/apecloud/pgbouncer:1.25.2-test" + definition = documents.find { |document| document["kind"] == "ComponentDefinition" && document.dig("metadata", "name") == "pgbouncer-1.0.6" } + abort unless definition.dig("spec", "serviceVersion") == "1.25.2-test" + abort unless definition.dig("spec", "runtime", "containers", 0, "imagePullPolicy") == "Always" + puts "ok" + ' + } + pg13_config_contract() { config="$(chart_dir)/config/pg13-config.tpl" schema="$(chart_dir)/config/pg13-config-constraint.cue" @@ -124,16 +334,33 @@ EOF fi } - It "advances the definition chart version to 1.0.5" + It "advances the definition chart version to 1.0.6" When call chart_version "$(chart_dir)" The status should eq 0 - The output should eq "1.0.5" + The output should eq "1.0.6" End - It "keeps the cluster chart version aligned at 1.0.5" + It "keeps the cluster chart version aligned at 1.0.6" When call chart_version "$(cluster_chart_dir)" The status should eq 0 - The output should eq "1.0.5" + The output should eq "1.0.6" + End + + It "keeps one replication topology and adds a zero-capable PgBouncer component" + When call pgbouncer_component_contract + The status should eq 0 + The output should eq "ok" + End + + It "honors the PgBouncer component image tag and pull policy" + When call pgbouncer_image_and_pull_policy_contract + The status should eq 0 + The output should eq "ok" + End + + It "uses the PgBouncer component image's non-root POSIX runtime contract" + When call pgbouncer_runtime_script_contract + The status should eq 0 End It "publishes PostgreSQL 13.23 in the ComponentDefinition, ParametersDefinition, and ComponentVersion" @@ -149,7 +376,7 @@ EOF End It "publishes the PostgreSQL 13 immutable ComponentDefinition identity" - When call render_count '^ name: postgresql-13-1.0.5$' + When call render_count '^ name: postgresql-13-1.0.6$' The status should eq 0 The output should eq "1" End diff --git a/addons/postgresql/scripts/pgbouncer-setup.sh b/addons/postgresql/scripts/pgbouncer-setup.sh index 4f144d647c..bff8b422dc 100755 --- a/addons/postgresql/scripts/pgbouncer-setup.sh +++ b/addons/postgresql/scripts/pgbouncer-setup.sh @@ -1,102 +1,94 @@ -#!/bin/bash +#!/bin/sh -pgbouncer_template_conf_file="/home/pgbouncer/conf/pgbouncer.ini" -pgbouncer_conf_dir="/opt/bitnami/pgbouncer/conf/" -pgbouncer_log_dir="/opt/bitnami/pgbouncer/logs/" -pgbouncer_tmp_dir="/opt/bitnami/pgbouncer/tmp/" -pgbouncer_conf_file="/opt/bitnami/pgbouncer/conf/pgbouncer.ini" -pgbouncer_user_list_file="/opt/bitnami/pgbouncer/conf/userlist.txt" +pgbouncer_template_conf_file="/opt/pgbouncer-template/pgbouncer.ini" +pgbouncer_conf_dir="/etc/pgbouncer" +pgbouncer_conf_file="${pgbouncer_conf_dir}/pgbouncer.ini" +pgbouncer_user_list_file="${pgbouncer_conf_dir}/userlist.txt" +pgbouncer_bin="/usr/bin/pgbouncer" -load_common_library() { - # the common.sh scripts is mounted to the same path which is defined in the cmpd.spec.scripts - common_library_file="/kb-scripts/common.sh" - # shellcheck disable=SC1090 - source "${common_library_file}" +pgbouncer_log() { + printf '%s\n' "pgbouncer-setup: $*" >&2 } -build_pgbouncer_conf() { - if is_empty "$POSTGRESQL_USERNAME" || is_empty "$POSTGRESQL_PASSWORD" || is_empty "$CURRENT_POD_IP"; then - echo "POSTGRESQL_USERNAME, POSTGRESQL_PASSWORD or CURRENT_POD_IP is not set. Exiting..." - exit 1 - fi - - mkdir -p $pgbouncer_conf_dir $pgbouncer_log_dir $pgbouncer_tmp_dir - cp $pgbouncer_template_conf_file $pgbouncer_conf_dir - echo "\"$POSTGRESQL_USERNAME\" \"$POSTGRESQL_PASSWORD\"" > $pgbouncer_user_list_file - # shellcheck disable=SC2129 - echo -e "\\n[databases]" >> $pgbouncer_conf_file - echo "postgres=host=$CURRENT_POD_IP port=5432 dbname=postgres" >> $pgbouncer_conf_file - echo "*=host=$CURRENT_POD_IP port=5432" >> $pgbouncer_conf_file - chmod 777 $pgbouncer_conf_file - chmod 777 $pgbouncer_user_list_file - - # Try to add user - useradd pgbouncer 2>/dev/null || true +validate_runtime_inputs() { + pgbouncer_backend_host="${POSTGRESQL_HOST:-}" + pgbouncer_backend_port="${POSTGRESQL_PORT:-}" - # NOTE: - # On Oracle Linux Server (especially in OKE environment) or OpenShift, useradd command may fail with error: - # "useradd: failure while writing changes to /etc/group" - # In this case, the user might be created but the group is not properly added to /etc/group file. - # This causes subsequent chown operations to fail. We need to handle this by: - # 1. Checking if user exists after useradd attempt - # 2. Separately checking if group exists (even if user was created) - # 3. Manually adding missing entries to /etc/passwd and /etc/group files when needed - - # Check if user exists - if ! id "pgbouncer" >/dev/null 2>&1; then - echo "useradd failed, attempting manual user creation..." - - # Get next available UID/GID - next_uid=$(awk -F: '$3 >= 1000 && $3 < 65534 {print $3}' /etc/passwd | sort -n | tail -1) - next_uid=$((next_uid + 1)) + if [ -z "${POSTGRESQL_USERNAME:-}" ] || [ -z "${POSTGRESQL_PASSWORD:-}" ] || + [ -z "$pgbouncer_backend_host" ] || [ -z "$pgbouncer_backend_port" ]; then + pgbouncer_log "POSTGRESQL_USERNAME, POSTGRESQL_PASSWORD, POSTGRESQL_HOST or POSTGRESQL_PORT is not set" + return 1 + fi - # Add user to /etc/passwd - echo "pgbouncer:x:$next_uid:$next_uid:pgbouncer user:/nonexistent:/bin/false" >> /etc/passwd - echo "Added pgbouncer user to /etc/passwd" + pgbouncer_credentials="${POSTGRESQL_USERNAME}${POSTGRESQL_PASSWORD}" + pgbouncer_sanitized_credentials=$(printf '%s' "$pgbouncer_credentials" | tr -d '\r\n') + if [ "$pgbouncer_credentials" != "$pgbouncer_sanitized_credentials" ]; then + pgbouncer_log "PostgreSQL credentials contain an unsupported line break" + return 1 fi - # Check if group exists (even if user was created by useradd) - if ! getent group pgbouncer >/dev/null 2>&1; then - echo "pgbouncer group not found, creating manually..." + case "$pgbouncer_backend_host" in + *[!A-Za-z0-9.-]*) + pgbouncer_log "PostgreSQL backend host contains unsupported characters" + return 1 + ;; + esac + case "$pgbouncer_backend_port" in + ''|*[!0-9]*) + pgbouncer_log "PostgreSQL backend port is invalid" + return 1 + ;; + esac + if [ "$pgbouncer_backend_port" -lt 1 ] || [ "$pgbouncer_backend_port" -gt 65535 ]; then + pgbouncer_log "PostgreSQL backend port is outside 1..65535" + return 1 + fi - # Get the user's GID if user exists - if id "pgbouncer" >/dev/null 2>&1; then - user_gid=$(id -g pgbouncer) - echo "pgbouncer:x:$user_gid:" >> /etc/group - echo "Added pgbouncer group with GID $user_gid to /etc/group" - else - # Fallback: use next available GID - next_gid=$(awk -F: '$3 >= 1000 && $3 < 65534 {print $3}' /etc/group | sort -n | tail -1) - next_gid=$((next_gid + 1)) - echo "pgbouncer:x:$next_gid:" >> /etc/group - echo "Added pgbouncer group with GID $next_gid to /etc/group" - fi + if [ ! -r "$pgbouncer_template_conf_file" ]; then + pgbouncer_log "rendered PgBouncer configuration is not readable" + return 1 fi +} - # Verify both user and group exist - if id "pgbouncer" >/dev/null 2>&1 && getent group pgbouncer >/dev/null 2>&1; then - echo "pgbouncer user and group are ready" - else - echo "Failed to create pgbouncer user or group. Exiting..." - exit 1 +build_pgbouncer_conf() { + umask 077 + validate_runtime_inputs || return 1 + mkdir -p "$pgbouncer_conf_dir" || return 1 + + pgbouncer_escaped_username=$(printf '%s' "$POSTGRESQL_USERNAME" | sed 's/"/""/g') + pgbouncer_escaped_password=$(printf '%s' "$POSTGRESQL_PASSWORD" | sed 's/"/""/g') + pgbouncer_generated_file=$(mktemp "${pgbouncer_conf_dir}/.pgbouncer.ini.XXXXXX") || return 1 + pgbouncer_user_list_tmp=$(mktemp "${pgbouncer_conf_dir}/.userlist.txt.XXXXXX") || { + rm -f "$pgbouncer_generated_file" + return 1 + } + + if ! printf '%%include %s\n\n[databases]\n' "$pgbouncer_template_conf_file" > "$pgbouncer_generated_file" || + ! printf 'postgres=host=%s port=%s dbname=postgres\n' "$pgbouncer_backend_host" "$pgbouncer_backend_port" >> "$pgbouncer_generated_file" || + ! printf '*=host=%s port=%s\n' "$pgbouncer_backend_host" "$pgbouncer_backend_port" >> "$pgbouncer_generated_file" || + ! chmod 600 "$pgbouncer_generated_file" || + ! printf '"%s" "%s"\n' "$pgbouncer_escaped_username" "$pgbouncer_escaped_password" > "$pgbouncer_user_list_tmp" || + ! chmod 600 "$pgbouncer_user_list_tmp"; then + rm -f "$pgbouncer_generated_file" "$pgbouncer_user_list_tmp" + return 1 fi - chown -R pgbouncer:pgbouncer $pgbouncer_conf_dir $pgbouncer_log_dir $pgbouncer_tmp_dir + if ! mv -f "$pgbouncer_user_list_tmp" "$pgbouncer_user_list_file" || + ! mv -f "$pgbouncer_generated_file" "$pgbouncer_conf_file"; then + rm -f "$pgbouncer_generated_file" "$pgbouncer_user_list_tmp" + return 1 + fi } start_pgbouncer() { - # https://github.com/bitnami/containers/blob/main/bitnami/pgbouncer/1/debian-12/rootfs/opt/bitnami/scripts/pgbouncer/run.sh - su pgbouncer -c "/opt/bitnami/scripts/pgbouncer/run.sh" + exec "$pgbouncer_bin" "$pgbouncer_conf_file" +} + +main() { + build_pgbouncer_conf || return 1 + start_pgbouncer } -# This is magic for shellspec ut framework. -# Sometime, functions are defined in a single shell script. -# You will want to test it. but you do not want to run the script. -# When included from shellspec, __SOURCED__ variable defined and script -# end here. The script path is assigned to the __SOURCED__ variable. ${__SOURCED__:+false} : || return 0 -# main -load_common_library -build_pgbouncer_conf -start_pgbouncer +main diff --git a/addons/postgresql/templates/_helpers.tpl b/addons/postgresql/templates/_helpers.tpl index 1d24c7b4b8..ed1e222e25 100644 --- a/addons/postgresql/templates/_helpers.tpl +++ b/addons/postgresql/templates/_helpers.tpl @@ -103,6 +103,21 @@ Define postgresql component definition name prefix by major version {{ printf "postgresql-%s-" .major }} {{- end -}} +{{/* +Define the PgBouncer component resource names. +*/}} +{{- define "pgbouncer.componentDefName" -}} +{{ include "pgbouncer.componentDefNamePrefix" . }}{{ .Chart.Version }} +{{- end -}} + +{{- define "pgbouncer.componentDefNamePrefix" -}} +pgbouncer- +{{- end -}} + +{{- define "pgbouncer.componentVersionName" -}} +pgbouncer +{{- end -}} + {{/* Get PostgreSQL image address by major and minor version Parameters: major (string), minor (string), root context @@ -139,6 +154,32 @@ Usage: {{ include "postgresql.imagePullPolicy" . }} {{- default "IfNotPresent" .Values.image.pullPolicy -}} {{- end -}} +{{/* +Get the PgBouncer component image pull policy. +*/}} +{{- define "pgbouncer.componentImagePullPolicy" -}} +{{- default "IfNotPresent" .Values.pgbouncer.componentImage.pullPolicy -}} +{{- end -}} + +{{/* +Get the default PgBouncer component service version. +*/}} +{{- define "pgbouncer.defaultServiceVersion" -}} +{{- $defaultVersion := "" -}} +{{- range .Values.pgbouncer.componentImage.versions -}} + {{- if .isDefault -}} + {{- if $defaultVersion -}} + {{- fail "pgbouncer.componentImage.versions must contain exactly one default version" -}} + {{- end -}} + {{- $defaultVersion = .version -}} + {{- end -}} +{{- end -}} +{{- if not $defaultVersion -}} +{{- fail "pgbouncer.componentImage.versions must contain exactly one default version" -}} +{{- end -}} +{{- $defaultVersion -}} +{{- end -}} + {{/* Get PostgreSQL componentDef by major version Parameters: major (string), root context @@ -219,13 +260,18 @@ Generate reloader scripts configmap {{- end }} {{- end }} -{{/* -Define pgbouncer configuration template name -*/}} {{- define "pgbouncer.configurationTemplate" -}} pgbouncer-configuration-{{ .Chart.Version }} {{- end -}} +{{- define "pgbouncer.parametersDefinition" -}} +pgbouncer-pd-{{ .Chart.Version }} +{{- end -}} + +{{- define "pgbouncer.pcr" -}} +pgbouncer-pcr-{{ .Chart.Version }} +{{- end -}} + {{/* Define image */}} @@ -233,8 +279,26 @@ Define image {{ .Values.image.registry | default "docker.io" }}/{{ .Values.image.repository }} {{- end }} -{{- define "postgresql.pgbouncerImage" -}} -{{ .Values.pgbouncer.image.registry | default (.Values.image.registry | default "docker.io") }}/{{ .Values.pgbouncer.image.repository }}:{{ .Values.pgbouncer.image.tag }} +{{- define "pgbouncer.sidecarImage" -}} +{{- $repository := printf "%s/%s" (.Values.pgbouncer.image.registry | default (.Values.image.registry | default "docker.io")) .Values.pgbouncer.image.repository -}} +{{- printf "%s:%s" $repository .Values.pgbouncer.image.tag -}} +{{- end }} + +{{- define "pgbouncer.componentImageByVersion" -}} +{{- $version := .version -}} +{{- $root := .root -}} +{{- $tag := "" -}} +{{- range $root.Values.pgbouncer.componentImage.versions -}} + {{- if eq .version $version -}} + {{- $tag = .tag -}} + {{- break -}} + {{- end -}} +{{- end -}} +{{- if not $tag -}} +{{- fail (printf "PgBouncer component image tag not found for version: %s" $version) -}} +{{- end -}} +{{- $repository := printf "%s/%s" ($root.Values.pgbouncer.componentImage.registry | default ($root.Values.image.registry | default "docker.io")) $root.Values.pgbouncer.componentImage.repository -}} +{{- printf "%s:%s" $repository $tag -}} {{- end }} {{- define "postgresql.metricsImage" -}} @@ -251,4 +315,4 @@ Define image {{- define "postgresql.initImage" -}} {{ .Values.image.registry | default "docker.io" }}/{{ .Values.image.init.repository }}:{{ .Values.image.init.tag }} -{{- end }} \ No newline at end of file +{{- end }} diff --git a/addons/postgresql/templates/clusterdefinition.yaml b/addons/postgresql/templates/clusterdefinition.yaml index 7808c5d146..52e8a26ae1 100644 --- a/addons/postgresql/templates/clusterdefinition.yaml +++ b/addons/postgresql/templates/clusterdefinition.yaml @@ -12,4 +12,16 @@ spec: components: - name: postgresql compDef: {{ include "postgresql.componentDefNamePrefix" . }} + - name: pgbouncer + compDef: {{ include "pgbouncer.componentDefName" . }} + orders: + provision: + - postgresql + - pgbouncer + terminate: + - pgbouncer + - postgresql + update: + - postgresql + - pgbouncer default: true diff --git a/addons/postgresql/templates/cmpd-pgbouncer.yaml b/addons/postgresql/templates/cmpd-pgbouncer.yaml new file mode 100644 index 0000000000..ff138f8e75 --- /dev/null +++ b/addons/postgresql/templates/cmpd-pgbouncer.yaml @@ -0,0 +1,148 @@ +apiVersion: apps.kubeblocks.io/v1 +kind: ComponentDefinition +metadata: + name: {{ include "pgbouncer.componentDefName" . }} + labels: + {{- include "postgresql.labels" . | nindent 4 }} + annotations: + {{- include "postgresql.annotations" . | nindent 4 }} +spec: + provider: kubeblocks + description: A managed PgBouncer connection pool for PostgreSQL + serviceKind: pgbouncer + serviceVersion: {{ include "pgbouncer.defaultServiceVersion" . | quote }} + replicasLimit: + minReplicas: 0 + maxReplicas: 64 + updateStrategy: Serial + podManagementPolicy: Parallel + services: + - name: pgbouncer + spec: + ports: + - name: pgbouncer + port: 6432 + targetPort: pgbouncer + configs: + - name: pgbouncer-configuration + template: {{ include "pgbouncer.configurationTemplate" . }} + namespace: {{ .Release.Namespace }} + volumeName: pgbouncer-config + defaultMode: 0444 + externalManaged: true + scripts: + - name: postgresql-scripts + template: {{ include "postgresql.scriptsTemplate" . }} + namespace: {{ .Release.Namespace }} + volumeName: scripts + defaultMode: 0555 + vars: + - name: POSTGRESQL_HOST + valueFrom: + serviceVarRef: + compDef: {{ include "postgresql.componentDefNamePrefix" . }} + name: postgresql + host: Required + optional: false + - name: POSTGRESQL_PORT + valueFrom: + serviceVarRef: + compDef: {{ include "postgresql.componentDefNamePrefix" . }} + name: postgresql + port: + name: tcp-postgresql + option: Required + optional: false + - name: POSTGRESQL_USERNAME + valueFrom: + credentialVarRef: + compDef: {{ include "postgresql.componentDefNamePrefix" . }} + name: postgres + username: Required + optional: false + - name: POSTGRESQL_PASSWORD + valueFrom: + credentialVarRef: + compDef: {{ include "postgresql.componentDefNamePrefix" . }} + name: postgres + password: Required + optional: false + runtime: + securityContext: + runAsUser: 70 + runAsGroup: 70 + fsGroup: 70 + fsGroupChangePolicy: OnRootMismatch + volumes: + - name: pgbouncer-state + emptyDir: {} + containers: + - name: pgbouncer + imagePullPolicy: {{ include "pgbouncer.componentImagePullPolicy" . }} + resources: + requests: + cpu: 100m + memory: 128Mi + limits: + cpu: 500m + memory: 512Mi + securityContext: + runAsNonRoot: true + runAsUser: 70 + runAsGroup: 70 + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + command: + - /kb-scripts/pgbouncer-setup.sh + ports: + - name: pgbouncer + containerPort: 6432 + protocol: TCP + volumeMounts: + - name: pgbouncer-config + mountPath: /opt/pgbouncer-template + - name: scripts + mountPath: /kb-scripts + - name: pgbouncer-state + mountPath: /etc/pgbouncer + livenessProbe: + failureThreshold: 3 + initialDelaySeconds: 15 + periodSeconds: 30 + timeoutSeconds: 5 + tcpSocket: + port: pgbouncer + readinessProbe: + failureThreshold: 1 + initialDelaySeconds: 5 + periodSeconds: 5 + timeoutSeconds: 5 + exec: + command: + - /bin/sh + - -ec + - | + primary="$( + PGPASSWORD="$POSTGRESQL_PASSWORD" PGCONNECT_TIMEOUT=3 \ + psql --no-password --username="$POSTGRESQL_USERNAME" \ + --host="$CURRENT_POD_IP" --port=6432 --dbname=postgres \ + --tuples-only --no-align --set=ON_ERROR_STOP=1 \ + --command="SELECT NOT pg_is_in_recovery() AND current_setting('transaction_read_only') = 'off'" + )" + test "$primary" = "t" + env: + - name: CURRENT_POD_IP + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: status.podIP + - name: POSTGRESQL_HOST + value: $(POSTGRESQL_HOST) + - name: POSTGRESQL_PORT + value: $(POSTGRESQL_PORT) + - name: POSTGRESQL_USERNAME + value: $(POSTGRESQL_USERNAME) + - name: POSTGRESQL_PASSWORD + value: $(POSTGRESQL_PASSWORD) diff --git a/addons/postgresql/templates/cmpd.yaml b/addons/postgresql/templates/cmpd.yaml index 49e6188c2b..b0eb367625 100644 --- a/addons/postgresql/templates/cmpd.yaml +++ b/addons/postgresql/templates/cmpd.yaml @@ -33,9 +33,6 @@ spec: - name: tcp-postgresql port: 5432 targetPort: tcp-postgresql - - name: tcp-pgbouncer - port: 6432 - targetPort: tcp-pgbouncer roleSelector: primary updateStrategy: BestEffortParallel podManagementPolicy: Parallel @@ -65,12 +62,6 @@ spec: volumeName: postgresql-config defaultMode: 0777 externalManaged: true - - name: pgbouncer-configuration - template: {{ include "pgbouncer.configurationTemplate" $ }} - namespace: {{ $.Release.Namespace }} - volumeName: pgbouncer-config - defaultMode: 0777 - externalManaged: true - name: postgresql-custom-metrics template: {{ include "postgresql.metricsConfiguration" (dict "major" .major "root" $) }} namespace: {{ $.Release.Namespace }} @@ -414,66 +405,6 @@ spec: value: $(CURRENT_POD_IP) - name: POD_NAMESPACE value: $(CLUSTER_NAMESPACE) - - name: pgbouncer - imagePullPolicy: {{ include "postgresql.imagePullPolicy" $ }} - securityContext: - runAsUser: 0 - ports: - - name: tcp-pgbouncer - containerPort: 6432 - volumeMounts: - - name: pgbouncer-config - mountPath: /home/pgbouncer/conf - - name: scripts - mountPath: /kb-scripts - command: - - /kb-scripts/pgbouncer-setup.sh - livenessProbe: - failureThreshold: 3 - initialDelaySeconds: 15 - periodSeconds: 30 - successThreshold: 1 - timeoutSeconds: 5 - tcpSocket: - port: tcp-pgbouncer - readinessProbe: - failureThreshold: 3 - initialDelaySeconds: 15 - periodSeconds: 30 - successThreshold: 1 - timeoutSeconds: 5 - tcpSocket: - port: tcp-pgbouncer - env: - - name: PGBOUNCER_AUTH_TYPE - value: md5 - - name: POSTGRESQL_USERNAME - value: $(POSTGRES_USER) - - name: POSTGRESQL_PASSWORD - value: $(POSTGRES_PASSWORD) - - name: POSTGRESQL_PORT - value: "5432" - - name: PGBOUNCER_PORT - value: "6432" - - name: PGBOUNCER_BIND_ADDRESS - value: "0.0.0.0" - - name: CURRENT_POD_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.name - - name: CURRENT_POD_IP - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: status.podIP - - name: CURRENT_POD_HOST_IP - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: status.hostIP - - name: POSTGRESQL_HOST - value: $(CURRENT_POD_IP) - name: exporter ports: - name: http-metrics diff --git a/addons/postgresql/templates/cmpv-pgbouncer.yaml b/addons/postgresql/templates/cmpv-pgbouncer.yaml new file mode 100644 index 0000000000..e2d4488963 --- /dev/null +++ b/addons/postgresql/templates/cmpv-pgbouncer.yaml @@ -0,0 +1,23 @@ +apiVersion: apps.kubeblocks.io/v1 +kind: ComponentVersion +metadata: + name: {{ include "pgbouncer.componentVersionName" . }} + labels: + {{- include "postgresql.labels" . | nindent 4 }} + annotations: + {{- include "postgresql.apiVersion" . | nindent 4 }} +spec: + compatibilityRules: + - compDefs: + - {{ include "pgbouncer.componentDefNamePrefix" . }} + releases: + {{- range .Values.pgbouncer.componentImage.versions }} + - {{ .version | quote }} + {{- end }} + releases: + {{- range .Values.pgbouncer.componentImage.versions }} + - name: {{ .version | quote }} + serviceVersion: {{ .version | quote }} + images: + pgbouncer: {{ include "pgbouncer.componentImageByVersion" (dict "version" .version "root" $) }} + {{- end }} diff --git a/addons/postgresql/templates/cmpv.yaml b/addons/postgresql/templates/cmpv.yaml index 834217e7f4..c8f9b693dc 100644 --- a/addons/postgresql/templates/cmpv.yaml +++ b/addons/postgresql/templates/cmpv.yaml @@ -26,7 +26,7 @@ spec: pg-init-container: {{ include "postgresql.initImage" $ }} exporter: {{ include "postgresql.metricsImage" $ }} postgresql: {{ include "postgresql.imageByVersion" (dict "major" $major "minor" .version "root" $) }} - pgbouncer: {{ include "postgresql.pgbouncerImage" $ }} + pgbouncer: {{ include "pgbouncer.sidecarImage" $ }} switchover: {{ include "postgresql.imageByVersion" (dict "major" $major "minor" .version "root" $) }} accountProvision: {{ include "postgresql.imageByVersion" (dict "major" $major "minor" .version "root" $) }} dbctl: {{ include "postgresql.dbctlImage" $ }} diff --git a/addons/postgresql/templates/paramsdef-pgbouncer.yaml b/addons/postgresql/templates/paramsdef-pgbouncer.yaml new file mode 100644 index 0000000000..4e874289e6 --- /dev/null +++ b/addons/postgresql/templates/paramsdef-pgbouncer.yaml @@ -0,0 +1,26 @@ +apiVersion: parameters.kubeblocks.io/v1alpha1 +kind: ParametersDefinition +metadata: + name: {{ include "pgbouncer.parametersDefinition" . }} + labels: + {{- include "postgresql.labels" . | nindent 4 }} + annotations: + {{- include "postgresql.annotations" . | nindent 4 }} +spec: + fileName: pgbouncer.ini + reloadAction: + unixSignalTrigger: + signal: SIGHUP + processName: pgbouncer + parametersSchema: + topLevelKey: PgBouncerParameter + cue: |- + {{- .Files.Get "config/pgbouncer-config-constraint.cue" | nindent 6 }} + dynamicParameters: + - pool_mode + - max_client_conn + - default_pool_size + - min_pool_size + - reserve_pool_size + - max_db_connections + - max_user_connections diff --git a/addons/postgresql/templates/pcr-pgbouncer.yaml b/addons/postgresql/templates/pcr-pgbouncer.yaml new file mode 100644 index 0000000000..25b11cd887 --- /dev/null +++ b/addons/postgresql/templates/pcr-pgbouncer.yaml @@ -0,0 +1,19 @@ +apiVersion: parameters.kubeblocks.io/v1alpha1 +kind: ParamConfigRenderer +metadata: + name: {{ include "pgbouncer.pcr" . }} + labels: + {{- include "postgresql.labels" . | nindent 4 }} + annotations: + {{- include "postgresql.annotations" . | nindent 4 }} +spec: + componentDef: {{ include "pgbouncer.componentDefName" . }} + serviceVersion: {{ include "pgbouncer.defaultServiceVersion" . | quote }} + parametersDefs: + - {{ include "pgbouncer.parametersDefinition" . }} + configs: + - name: pgbouncer.ini + fileFormatConfig: + format: ini + iniConfig: + sectionName: pgbouncer diff --git a/addons/postgresql/templates/pcr.yaml b/addons/postgresql/templates/pcr.yaml index b32829e7c1..05add882de 100644 --- a/addons/postgresql/templates/pcr.yaml +++ b/addons/postgresql/templates/pcr.yaml @@ -21,9 +21,4 @@ spec: reRenderResourceTypes: - vscale - tls - - name: pgbouncer.ini - fileFormatConfig: - format: ini - iniConfig: - sectionName: pgbouncer -{{- end -}} \ No newline at end of file +{{- end -}} diff --git a/addons/postgresql/templates/pgbouncer-configmap.yaml b/addons/postgresql/templates/pgbouncer-configmap.yaml index a868c2f553..fef8721772 100644 --- a/addons/postgresql/templates/pgbouncer-configmap.yaml +++ b/addons/postgresql/templates/pgbouncer-configmap.yaml @@ -8,4 +8,4 @@ metadata: {{- include "postgresql.annotations" . | nindent 4 }} data: pgbouncer.ini: |- - {{- .Files.Get "config/pgbouncer-ini.tpl" | nindent 4 }} \ No newline at end of file + {{- .Files.Get "config/pgbouncer-ini.tpl" | nindent 4 }} diff --git a/addons/postgresql/values.yaml b/addons/postgresql/values.yaml index b43733255f..00ecc4c9c2 100644 --- a/addons/postgresql/values.yaml +++ b/addons/postgresql/values.yaml @@ -148,13 +148,22 @@ versions: ## @section pgbouncer Parameters pgbouncer: + # PgBouncer sidecar image. image: - # refer: https://hub.docker.com/r/bitnami/pgbouncer # if the value of pgbouncer.image.registry is not specified using `--set`, it will be set to the value of 'image.registry' by default registry: "" repository: apecloud/pgbouncer tag: 1.19.0 pullPolicy: IfNotPresent + # PgBouncer component image. + componentImage: + registry: "" + repository: apecloud/pgbouncer + pullPolicy: IfNotPresent + versions: + - version: 1.25.2 + tag: 1.25.2 + isDefault: true dataMountPath: /home/postgres/pgdata