diff --git a/docs/operator/setting_up_demo.md b/docs/operator/setting_up_demo.md index 8191a811b8d..5a36595292a 100644 --- a/docs/operator/setting_up_demo.md +++ b/docs/operator/setting_up_demo.md @@ -426,3 +426,206 @@ $ rucio list-file-replicas test:mynewdataset | test | file4 | 10.486 MB | 65786e49 | XRD3: root://xrd3:1096//rucio/... | +-------+-------+-----------+----------+-----------------------------------+ ``` + +# Configuring S3 Storage and Multi-Hop Transfers in Rucio + +This tutorial covers how to register S3-compatible storage (MinIO) as Rucio Storage Elements (RSEs), configure credentials for both Rucio and FTS, and set up RSE distances to enable multi-hop transfers between S3 and XRootD endpoints. + +The examples use a Docker Compose playground environment with two MinIO instances (MINIO1, MINIO2) and three XRootD servers (XRD1, XRD2, XRD3). The commands assume you are already operating within a Rucio admin environment with the `rucio` and `rucio-admin` CLI tools available. + +## Enabling HTTPS on XRD3 for Multi-Hop + +XRD3 acts as the intermediate hop between S3 and XRootD storage. To allow it to communicate with S3 backends, add an HTTPS protocol entry to the XRD3 RSE: + +```bash +rucio rse protocol add XRD3 \ + --host xrd3 \ + --scheme https \ + --prefix //rucio \ + --port 1096 \ + --impl rucio.rse.protocols.gfal.Default \ + --domain-json '{"wan": {"read": 2, "write": 2, "delete": 2, "third_party_copy_read": 2, "third_party_copy_write": 2}, "lan": {"read": 2, "write": 2, "delete": 2}}' +``` + +The priority values (`"read": 2` etc.) ensure that the existing XRootD protocol remains preferred for direct transfers, while HTTPS is available for multi-hop routing. + +## Creating Buckets on MinIO + +Before registering MinIO instances as RSEs, create the `rucio` bucket on each. This uses the MinIO Client (`mc`) from within each MinIO container: + +```bash +# On MINIO1 +export MC_INSECURE=true +mc alias set local https://localhost:9001 admin password +mc mb local/rucio + +# On MINIO2 +export MC_INSECURE=true +mc alias set local https://localhost:9002 admin password +mc mb local/rucio +``` + +## Registering MinIO RSEs + +Register both MinIO instances as RSEs with S3 protocol configuration. The `gfal.NoRename` implementation is used because S3 does not support server-side rename operations. + +```bash +rucio rse add MINIO1 +rucio rse protocol add MINIO1 \ + --host minio1 \ + --port 9001 \ + --scheme https \ + --prefix /rucio/ \ + --impl rucio.rse.protocols.gfal.NoRename \ + --domain-json '{"lan": {"read": 1, "write": 1, "delete": 1}, "wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}}' +rucio rse attribute add MINIO1 --key sign_url --value s3 +rucio rse attribute add MINIO1 --key s3_url_style --value path +rucio rse attribute add MINIO1 --key verify_checksum --value False +rucio rse attribute add MINIO1 --key skip_upload_stat --value True +rucio rse attribute add MINIO1 --key strict_copy --value True +rucio rse attribute add MINIO1 --key fts --value https://fts:8446 +rucio account limit add root --rse MINIO1 --bytes infinity + +rucio rse add MINIO2 +rucio rse protocol add MINIO2 \ + --host minio2 \ + --port 9002 \ + --scheme https \ + --prefix /rucio/ \ + --impl rucio.rse.protocols.gfal.NoRename \ + --domain-json '{"lan": {"read": 1, "write": 1, "delete": 1}, "wan": {"read": 1, "write": 1, "delete": 1, "third_party_copy_read": 1, "third_party_copy_write": 1}}' +rucio rse attribute add MINIO2 --key sign_url --value s3 +rucio rse attribute add MINIO2 --key s3_url_style --value path +rucio rse attribute add MINIO2 --key verify_checksum --value False +rucio rse attribute add MINIO2 --key skip_upload_stat --value True +rucio rse attribute add MINIO2 --key strict_copy --value True +rucio rse attribute add MINIO2 --key fts --value https://fts:8446 +rucio account limit add root --rse MINIO2 --bytes infinity +``` + +### Setting RSE Credentials + +Rucio needs S3 credentials to generate presigned URLs for transfers. These are stored in `rse-accounts.cfg`, keyed by RSE ID: + +```bash +ID1=$(rucio rse show MINIO1 | grep '^ id:' | awk '{print$2}') +ID2=$(rucio rse show MINIO2 | grep '^ id:' | awk '{print$2}') +cat >/opt/rucio/etc/rse-accounts.cfg <