diff --git a/documentation/docs/apps/index.md b/documentation/docs/apps/index.md new file mode 100644 index 00000000000..90591c01244 --- /dev/null +++ b/documentation/docs/apps/index.md @@ -0,0 +1,15 @@ +# Manage + +!!! warning "Tech Preview" + Management capabilities are not production-ready. Use for testing and feedback only. + +PMM is expanding beyond monitoring into database management. Through the Management framework integration, formerly known as SEP (Services Enablement Platform), you can now trigger and track database operations on your hosts directly from PMM, without SSH access or extra software on those hosts. + +This is the first step in a broader initiative to give you a single place to monitor, manage, and act on your entire database infrastructure. + +## Available capabilities + +- [MySQL Backups](mysql-backup.md) — run and schedule MySQL backups using XtraBackup, Mydumper, or Binlog, and restore from them. +- [Support Diagnostics](support-diagnostics.md) — collect diagnostic data from your hosts and send it directly to your Percona support case in ServiceNow. + +More operations and database types will be added in future releases. diff --git a/documentation/docs/apps/mysql-backup.md b/documentation/docs/apps/mysql-backup.md new file mode 100644 index 00000000000..942740fd2b9 --- /dev/null +++ b/documentation/docs/apps/mysql-backup.md @@ -0,0 +1,220 @@ +# MySQL Backups + +!!! warning "Tech Preview" + This feature is not production-ready. Use for testing and feedback only. + +MySQL Backup lets you run and schedule MySQL backups against MySQL services in your inventory, restore from those backups, and track job status and execution history. + +It is available under **Apps > MySQL Backups** in the sidebar, part of PMM's growing set of [database management apps](index.md). + +The legacy PMM MySQL backup feature under **Backups > All backups** continues to work alongside this app. Backups created in one are not visible in the other. This app is intended to replace the legacy feature when the [SEP integration](index.md) reaches GA. + +## Supported backup types + +| Type | Tool | Best for | +|---|---|---| +| **XtraBackup** | `xtrabackup`, `mariadb-backup`, or `innobackupex` | Physical hot backups of large datasets with minimal locking. Supports full and incremental backups. Must run directly on the database host. | +| **Mydumper** | `mydumper` | Logical SQL dumps with selective database and table restores. Can run from a remote executor host. | +| **Binlog** | `mysqlbinlog` | Continuous binary log capture. Use alongside a base XtraBackup or Mydumper backup for point-in-time recovery — not a standalone complete backup. | + +MariaDB is supported via the `mariadb-backup` binary. + +## Before you start + +### Enable Nomad on PMM Server + +The MySQL Backups app dispatches tasks via Nomad. To enable Nomad, start PMM Server with both `PMM_ENABLE_NOMAD=1` and `PMM_PUBLIC_ADDRESS` set. See [Configure Nomad](../reference/nomad.md). + +### Install PMM Client on the execution host + +The execution host must have PMM Client installed with the Nomad client enabled. PMM Client ships the Nomad client so no separate Nomad installation is required. + +### Install required tools on the execution host + +Install the tool for your backup type and make sure it is available on `$PATH`: + +| Backup type | Required tool | +|---|---| +| XtraBackup | `xtrabackup`, `mariadb-backup`, or `innobackupex` | +| Mydumper | `mydumper` | +| Binlog | `mysqlbinlog` | + +XtraBackup requires **root** on the execution host. Mydumper and Binlog do not. + +For XtraBackup, use a version that matches your MySQL version: + +| MySQL version | Percona XtraBackup version | +|---|---| +| 5.5, 5.6, 5.7 | PXB 2.4.x | +| 8.0.0–8.0.33 | PXB 8.0.x (same version or newer) | +| 8.0.34+ | PXB 8.0.34+ | +| 8.1.x, 8.2.x, 8.3.x | Matching PXB version | +| 8.4.x | Any PXB 8.4.x | + +### Configure database credentials on the host + +SEP reads MySQL credentials from `~/.my.cnf` or `~/.mylogin.cnf` on the executor host. The backup task does not prompt for a password — this file must exist and be readable before running a backup. + +For XtraBackup on MySQL 8.0+, the MySQL user must have the `BACKUP_ADMIN` privilege. + + + +### Prepare a backup directory + +Backups are written to a local directory on the execution host. Create the directory and confirm it is writable before creating a task. Set the path per task in the **Backup directory** field. + +### Sync your MySQL service + +The MySQL service must appear in the SEP inventory. SEP syncs from PMM — if a service is registered in PMM but a sync has not run, it will not appear in the backup form. Trigger a sync from **Inventory** if needed. + +## Storage + +Backups are written locally to the execution host by default. You can optionally upload to one or more remote destinations, configured per task: + +| Provider | Required field | +|---|---| +| S3-compatible storage | S3 bucket | +| Google Cloud Storage | GCS bucket | +| Rsync | Rsync destination path | + +You can select multiple upload providers simultaneously. + +### Retention + +Without retention configured, backups accumulate until deleted manually. Set retention per task: + +| Backup type | Retention options | +|---|---| +| Mydumper | Daily purge (days), Weekly purge (weeks) | +| XtraBackup | Number of copies to keep | +| Binlog | Purge after (days) | + +## Compression and encryption + +### Compression + +Enable **Compress backup data** and select an algorithm. Available algorithms vary by backup type: + +| Backup type | Supported algorithms | +|---|---| +| XtraBackup | zstd, lz4, quicklz | +| Mydumper | gzip, zstd | + +### Encryption + +Two independent GPG encryption modes are available: + +- **Encrypt backup** — encrypts the backup in place during the run. Combine with **Encrypt using tmpdir** to write to a temporary directory during encryption. +- **Encrypt after backup completes** — GPG-encrypts the finished backup as a post-run step. Mutually exclusive with **Encrypt using tmpdir**. + +Both modes require setting an **Encryption recipient** (GPG key or recipient ID). + +XtraBackup also supports **AES-256 encryption** via a keyfile, configured in the **AES-256 key file path** field. + +## About execution hosts + +The execution host is the Nomad agent that runs the backup task. The backup type determines where it must run: + +- **XtraBackup** — the executor must be the database host itself. The task always connects to `localhost`. +- **Mydumper** — the executor can be any host with network access to the database. +- **Binlog** — the executor can be any host with network access to the database. Use **Alternative binlog host** to stream logs from a specific source host. + +For remote or cloud-hosted databases, select an executor host that has network access to the target. + +## Run a MySQL backup + +To run a MySQL backup: +{.power-number} + +1. Go to **Apps > MySQL Backups** in the sidebar. +2. Click **+ New MySQL Backup**. +3. Enter a task name, select the **backup type**, and select the **execution host**. For XtraBackup, the execution host must be the host running the MySQL service. +4. Under **Upload**, select one or more **Upload providers** and fill in the destination fields if uploading off-host. +5. Optionally configure compression, encryption, or retention in the relevant sections of the form. +6. Optionally check **Alert on failure** to receive an alert if the backup task fails. +7. Click **Run** to start immediately, or set a schedule and click **Schedule**. + +Completed XtraBackup and Mydumper runs are recorded in the backup catalog with their location, upload destination, size, and timestamps. Binlog runs are not catalogued. + +### Incremental XtraBackup backups + +XtraBackup supports two incremental methods. Select one in **Incremental method**: + +- **less_space** — smaller incremental files. Set **Incremental cycle** to control when the full backup runs: `daily`, `weekly`, or a specific weekday (Monday–Sunday). +- **fast_restore** — optimized for faster restores. The cycle is not configurable. + +## Restore from a backup + +To restore from a backup: +{.power-number} + +1. Go to **Apps > MySQL Backups** and select the **Restore** tab. +2. Click **+ New MySQL Restore**. +3. Select the **backup type**. +4. Optionally select a **destination service**. Selecting a known service populates the **Backup source** list with that service's recorded backups. You can also enter a path directly: + + | Format | Example | + |---|---| + | Local path | `/backups/mydumper/20240101` | + | Remote path | `db01:/path/to/backup` | + | S3 | `s3://bucket/path` | + | GCS | `gs://bucket/path` | + + Append `/latest` to any path to use the most recent backup automatically. + +5. Configure restore options for your backup type (see below). +6. Optionally check **Alert on failure** to receive an alert if the restore task fails. +7. Click **Create MySQL Restore**. + +### Restore options by backup type + +**Mydumper** + +The destination must be a MySQL service in inventory. Optionally scope the restore with **Include databases**, **Skip databases**, or **Restore to Database** to target a single schema. + +**XtraBackup** + +The destination service is optional — you can restore to any reachable host, including hosts not in inventory. Key options: + +| Option | Description | +|---|---| +| Kill MySQL | Kills the MySQL process before restoring. MySQL does **not** restart automatically — start it manually after the restore completes. | +| Skip incrementals | Applies the full backup only, skipping incremental layers. | +| XtraBackup parallel | Number of threads for the restore (default: 4). | +| Data directory | Override the target datadir path. | +| Restore my.cnf | Restores the `my.cnf` configuration file as part of the restore. | + +**Binlog — point-in-time recovery** + +Set start and stop positions to control how far to replay logs: + +| Field | Description | +|---|---| +| Start file / Start position | Where to begin replaying. | +| Stop file / Stop position | Where to stop. Leave empty to replay all available logs. | + +### Restoring to a different host + +| Backup type | Cross-host restore | +|---|---| +| Mydumper | Destination must be a MySQL service in inventory. | +| XtraBackup | Any reachable host, including hosts not in inventory. Configure access via **SSH user**, **SSH port**, and **SSH key**. | +| Binlog | Any reachable host. Same SSH options as XtraBackup. | + + + +### Pre and post scripts + +All restore types support **Pre-script** and **Post-script** — shell scripts that run on the execution host before and after the restore. + +## Scheduling + +To manage scheduled backup or restore tasks, click **Schedules** on the **MySQL Backups** or **Restore** page. Scheduled tasks are listed under **Scheduled Tasks** and can be added with **+ Add new**, edited, or deleted without losing their execution history. + + + +## Monitoring + +Task status and execution history are visible in the **Apps > MySQL Backups** list. Use the **Status** filter to narrow results. + + diff --git a/documentation/docs/apps/support-diagnostics.md b/documentation/docs/apps/support-diagnostics.md new file mode 100644 index 00000000000..b258b5495b7 --- /dev/null +++ b/documentation/docs/apps/support-diagnostics.md @@ -0,0 +1,28 @@ +# Support Diagnostics + +!!! warning "Tech Preview" + This feature is not production-ready. Use for testing and feedback only. + +Support Diagnostics runs targeted diagnostic scripts directly on your database hosts and sends the results straight to your Percona support case in ServiceNow, without connecting to any server or uploading files yourself. + +This capability is part of the [Management framework](index.md) integration. + +## Support Diagnostics vs PMM Dump + +Support Diagnostics runs specific diagnostic scripts on your database hosts to investigate a particular issue, and ships the output to your support case. + +[PMM Dump](../../get-help.md) exports PMM's own monitoring data (metrics and dashboards) compressed for Percona to analyze. +If Percona Support asks you for monitoring data from PMM, use PMM Dump. If they ask you to run diagnostics on your databases, use Support Diagnostics. + +## Before you start + +- PMM Client 3.10.0 or later must be installed on the monitored host. +- You must have an open support case in Percona's ServiceNow. + +## Run a diagnostic collection + +1. Go to **Apps > Support Diagnostics** in the left navigation. +2. Select the target host and your ServiceNow case number. +3. Click **Run**. + +PMM collects the diagnostic data from the host and uploads the results directly to your support case. No files to download or upload manually. diff --git a/documentation/docs/backup/index.md b/documentation/docs/backup/index.md index 381591642cc..9fc2e9a5d76 100644 --- a/documentation/docs/backup/index.md +++ b/documentation/docs/backup/index.md @@ -4,31 +4,33 @@ Losing your data can destroy your business. This is why backing up data is criti Even more important than backing up data, is the ability to restore it in the event of data loss. PMM enables you to do all this with zero downtime and minimal performance impact. -- MongoDB (Generally Available) -- MySQL (in Technical Preview) +## MySQL backups -Enable the **Backup Management** option in PMM's Advanced Settings to activate the **Backup** page from where you can: +For MySQL, use [MySQL Backups](../manage/mysql-backup.md) under **Apps > MySQL Backups**. It supports XtraBackup, Mydumper, and Binlog, with local storage, S3, GCS, and rsync upload, and is the primary MySQL backup option going forward. -- Create and restore MongoDB and MySQL backups +!!! warning "Legacy MySQL backup feature deprecated" + The MySQL backup feature under **Backups > All backups** is deprecated and will be replaced by [MySQL Backups](../manage/mysql-backup.md) when the [SEP integration](../manage/index.md) reaches GA. Both options are currently in Tech Preview. Backups created in one are not visible in the other. + +## MongoDB backups + +Enable the **Backup Management** option in PMM's Advanced Settings to activate the **Backup** page from where you can: + +- Create and restore MongoDB backups - Automate backup scheduling - Set retention policies - Monitor your backup and restore activity -## Supported setups - -For MySQL databases, you can create and restore on-demand and scheduled physical backups. For MongoDB, you can create and restore physical, logical and Point-in-Time-Recovery (PITR) backups, both on-demand and scheduled. +For MongoDB, you can create and restore physical, logical, and Point-in-Time-Recovery (PITR) backups, both on-demand and scheduled. ### Sharded MongoDB cluster configurations PMM 3 supports creating backups of sharded MongoDB clusters. However, the restoring process is not handled end-to-end, and requires you to manually restore the artifacts using the CLI in Percona Backup for MongoDB. - ## Start here -To learn how to create and restore backups, check out subtopics below: - [Prepare a storage location](prepare_storage_location.md) -- ![!](../images/MongoDB_Logo.png) [MongoDB backups](mongodb-backup/create_mongo_on_demand.md) -- ![!](../images/mysql_logo.png) [MySQL backups](mysql-backup/create_mysql_backup.md) +- ![!](../images/MongoDB_Logo.png) [MongoDB backups](mongodb-backup/create_mongo_on_demand.md) +- ![!](../images/mysql_logo.png) [MySQL Backups](../manage/mysql-backup.md) ??? info "Additional resources" diff --git a/documentation/docs/backup/mysql-backup/backup_mysql.md b/documentation/docs/backup/mysql-backup/backup_mysql.md index 866109991fa..82b1354ec82 100644 --- a/documentation/docs/backup/mysql-backup/backup_mysql.md +++ b/documentation/docs/backup/mysql-backup/backup_mysql.md @@ -2,6 +2,9 @@ !!! caution alert alert-warning "Important" MySQL backup functionality is still in Technical Preview. + +!!! warning "Deprecated" + This feature is deprecated and will be replaced by [MySQL Backups](../../manage/mysql-backup.md) under **Apps > MySQL Backups** when the [SEP integration](../../manage/index.md) reaches GA. Both options are currently in Tech Preview and run alongside each other. Backups created in one are not visible in the other. PMM supports MySQL database server for: diff --git a/documentation/docs/backup/mysql-backup/create_mysql_backup.md b/documentation/docs/backup/mysql-backup/create_mysql_backup.md index df905cc3b95..24732e4b476 100644 --- a/documentation/docs/backup/mysql-backup/create_mysql_backup.md +++ b/documentation/docs/backup/mysql-backup/create_mysql_backup.md @@ -1,5 +1,8 @@ # Create a MySQL backup +!!! warning "Deprecated" + This feature is deprecated and will be replaced by [MySQL Backups](../../manage/mysql-backup.md) under **Apps > MySQL Backups** when the [SEP integration](../../manage/index.md) reaches GA. Both options are currently in Tech Preview and run alongside each other. Backups created in one are not visible in the other. + Before creating a backup, make sure to check the [MySQL backup prerequisites](mysql_prerequisites.md). To create a backup: diff --git a/documentation/docs/backup/mysql-backup/mysql_prerequisites.md b/documentation/docs/backup/mysql-backup/mysql_prerequisites.md index 11bb1c93f1c..d9f23919d9d 100644 --- a/documentation/docs/backup/mysql-backup/mysql_prerequisites.md +++ b/documentation/docs/backup/mysql-backup/mysql_prerequisites.md @@ -1,5 +1,8 @@ # MySQL backup prerequisites +!!! warning "Deprecated" + This feature is deprecated and will be replaced by [MySQL Backups](../../manage/mysql-backup.md) under **Apps > MySQL Backups** when the [SEP integration](../../manage/index.md) reaches GA. Both options are currently in Tech Preview and run alongside each other. Backups created in one are not visible in the other. + Before creating MySQL backups: {.power-number} diff --git a/documentation/docs/backup/mysql-backup/restore_mysql_backup.md b/documentation/docs/backup/mysql-backup/restore_mysql_backup.md index 5ad17d9c83c..4f4af5c8ed4 100644 --- a/documentation/docs/backup/mysql-backup/restore_mysql_backup.md +++ b/documentation/docs/backup/mysql-backup/restore_mysql_backup.md @@ -1,5 +1,8 @@ # Restore a MySQL backup +!!! warning "Deprecated" + This feature is deprecated and will be replaced by [MySQL Backups](../../manage/mysql-backup.md) under **Apps > MySQL Backups** when the [SEP integration](../../manage/index.md) reaches GA. Both options are currently in Tech Preview and run alongside each other. Backups created in one are not visible in the other. + ## Restore compatibility MySQL backups can be restored to the same service it was created from, or to a compatible one. diff --git a/documentation/docs/images/management-mysql-backup.png b/documentation/docs/images/management-mysql-backup.png new file mode 100644 index 00000000000..d2754bc3353 Binary files /dev/null and b/documentation/docs/images/management-mysql-backup.png differ diff --git a/documentation/docs/images/management-support-diagnostics.png b/documentation/docs/images/management-support-diagnostics.png new file mode 100644 index 00000000000..84f327ae9db Binary files /dev/null and b/documentation/docs/images/management-support-diagnostics.png differ diff --git a/documentation/docs/release-notes/3.10.0.md b/documentation/docs/release-notes/3.10.0.md index 86863e8e7d1..55e73a7f51d 100644 --- a/documentation/docs/release-notes/3.10.0.md +++ b/documentation/docs/release-notes/3.10.0.md @@ -12,6 +12,36 @@ Percona Monitoring and Management (PMM) is an open source database monitoring, m ## Release summary +### New Apps: MySQL Backups and Support Diagnostics (Tech Preview) + +!!! warning "Tech Preview" + This feature is not production-ready. Use for testing and feedback only. + +PMM has always been about monitoring your databases. Starting with 3.10.0, you can manage and act on them too. + +PMM 3.10.0 introduces its first management apps, letting you run database operations directly from PMM without SSH access or extra software. These apps connect PMM to Percona's Services Enablement Platform (SEP), the same tool Percona teams use to manage database operations on customer hosts. + +With this initial integration, you can run MySQL backups and send diagnostic results to your Percona Support case. More operations and database types will follow in future releases. + + +#### [MySQL Backups](../apps/mysql-backup.md) + +Available under **Apps > MySQL Backups** in the sidebar, part of PMM's growing set of database management apps. Schedule and run MySQL backups for services in your PMM inventory using XtraBackup, Mydumper, or Binlog, restore from backups, and track job status, backup type, and execution history. + +It runs alongside the existing PMM MySQL backup feature and is intended to replace it once the SEP integration reaches GA. + +![Management MySQL Backup screen](../images/management-mysql-backup.png) + +#### [Support Diagnostics](../manage/support-diagnostics.md) + +Run targeted diagnostic scripts on your monitored hosts and send the results directly to your Percona Support case in ServiceNow, without SSH access or manual file uploads. + +![Management Support Diagnostics screen](../images/management-support-diagnostics.png) + + +To get started, see [Apps](../apps/index.md). + + ### MongoDB Unused Indexes dashboard You can now spot unused MongoDB indexes directly in PMM, without running manual queries against each node. Go to **Dashboards > MongoDB > MongoDB Unused Indexes** to see indexes with zero accesses since the last `mongod` restart, along with access trends and a least-used index ranking to help you prioritize cleanup. @@ -28,6 +58,9 @@ Requires the `indexstats` collector on the MongoDB exporter. See [MongoDB Unused ## 📈 Improvements + +- [PMM-15205](https://perconadev.atlassian.net/browse/PMM-15205): Introduced the Apps section, adding [MySQL Backups](../manage/mysql-backup.md) and [Support Diagnostics](../manage/support-diagnostics.md) as the first management apps in PMM. + - [PMM-15071](https://perconadev.atlassian.net/browse/PMM-15071): Added the [MongoDB Unused Indexes](../reference/dashboards/dashboard-mongodb-unused-indexes.md) dashboard for identifying indexes with zero accesses since the last `mongod` restart. Requires the `indexstats` collector. - [PMM-15189](https://perconadev.atlassian.net/browse/PMM-15189): Added documentation for starting and stopping Real-Time Analytics (RTA) on MongoDB services from the command line. See [pmm-admin inventory add agent rta-mongodb-agent](../use/commands/pmm-admin/inventory.md#pmm-admin-inventory-add-agent-rta-mongodb-agent) and [pmm-admin inventory remove agent](../use/commands/pmm-admin/inventory.md#pmm-admin-inventory-remove-agent). diff --git a/documentation/mkdocs-base.yml b/documentation/mkdocs-base.yml index 7891b1530d8..77ce9c209b3 100644 --- a/documentation/mkdocs-base.yml +++ b/documentation/mkdocs-base.yml @@ -417,6 +417,10 @@ nav: - backup/mysql-backup/restore_mysql_backup.md - backup/edit_scheduled.md - backup/delete_a_backup.md + - Manage: + - manage/index.md + - manage/mysql-backup.md + - manage/support-diagnostics.md - Configure: - configure-pmm/configure.md - configure-pmm/metrics_res.md