feat: enable basic LDAP functionalities by stevapple · Pull Request #77 · juicedata/minio

stevapple · 2025-03-07T04:46:30Z

Description

LDAP support exists in Apache-2.0 MinIO, but it's lacking some key functionalities. In this PR we add:

support for listing valid users with LDAP;
support for STS policies with LDAP;
support for logging into the browser with LDAP STS;
support for listing groups with LDAP.

This PR is the core of unblocking juicedata/juicefs#5368

Motivation and Context

How to test this PR?

Suppose you have an LDAP server running on ldaps://ldap.example.com:636 with root DN dc=example,dc=com, and a test FS sqlite3://test.db.

# Build
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN="ou=groups,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER="(&(objectClass=groupOfUniqueNames)(uniqueMember=%d))"
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN="cn=juicefs,ou=services,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD="jfs-password"
export MINIO_IDENTITY_LDAP_SERVER_ADDR="ldap.example.com:636"
export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_BASE_DN="ou=users,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_FILTER="(&(objectClass=inetOrgPerson)(uid=%s))"
export MINIO_ROOT_USER="admin"
export MINIO_ROOT_PASSWORD="juicefs"
juicefs gateway sqlite3://test.db localhost:9000 --multi-buckets

You can use the browser to log into the gateway via http://localhost:9000/minio.

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Optimization (provides speedup with no functional changes)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

Fixes a regression (If yes, please add commit-id or PR # here)
Documentation updated
Unit tests added/updated

Xuyuchao-juice · 2025-06-25T04:23:11Z

I failed to test successfully using your method. Could you please help me see where the problem lies

Steps to Reproduce

Setup LDAP Server

docker run -d --name openldap \
  -p 5389:389 \
  -e LDAP_ORGANISATION="JuiceFS Test" \
  -e LDAP_DOMAIN="example.com" \
  -e LDAP_ADMIN_PASSWORD="123" \
  osixia/openldap:1.5.0
sleep 15  # Wait for service initialization

Configure LDAP Test Data
Executed the script below to populate users/groups:

#!/bin/bash

if ! ldapadd -x -H ldap://0.0.0.0:5389 -D "cn=admin,dc=example,dc=com" -w 123 <<EOF
   dn: ou=users,dc=example,dc=com
   objectClass: organizationalUnit
   ou: users

   dn: ou=groups,dc=example,dc=com
   objectClass: organizationalUnit
   ou: groups

   dn: cn=jfsusers,ou=groups,dc=example,dc=com
   objectClass: groupOfUniqueNames
   cn: jfsusers
   uniqueMember: cn=user1,ou=users,dc=example,dc=com
   uniqueMember: cn=user2,ou=users,dc=example,dc=com

   dn: cn=user1,ou=users,dc=example,dc=com
   objectClass: inetOrgPerson
   cn: user1
   sn: user1
   uid: user1
   userPassword: pass1

   dn: cn=user2,ou=users,dc=example,dc=com
   objectClass: inetOrgPerson
   cn: user2
   sn: user2
   uid: user2
   userPassword: pass2
   EOF
then
   echo "error" >&2
   exit 1
fi

echo "LDAP数据结构添加成功！"

Verification:
✅ Confirmed entries: user1 (pass1), user2 (pass2), group jfsusers.

Build & Configure JuiceFS Gateway

git checkout <PR_BRANCH>
make build

export MINIO_IDENTITY_LDAP_SERVER_ADDR="0.0.0.0:5389"
export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_BASE_DN="ou=users,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_USER_DN_SEARCH_FILTER="(&(objectClass=inetOrgPerson)(uid=%s))"
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_BASE_DN="ou=groups,dc=example,dc=com"
export MINIO_IDENTITY_LDAP_GROUP_SEARCH_FILTER="(&(objectClass=groupOfUniqueNames)(uniqueMember=%d))"
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN="cn=admin,dc=example,dc=com"  # Fixed missing account
export MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD="123"
export MINIO_ROOT_USER="admin"
export MINIO_ROOT_PASSWORD="juicefs123"

juicefs format sqlite3://test.db myjfs
./juicefs gateway sqlite3://test.db 0.0.0.0:9000 --multi-buckets -v  # Added verbose logging

Attempt Login
- Accessed WebUI: http://localhost:9000/minio
- Tried credentials:
  - Access Key = user1, Secret Key = pass1
  - Access Key = user2, Secret Key = pass2
    ❌ Result: Invalid credentials error.

stevapple · 2025-06-27T02:37:34Z

@Xuyuchao-juice If you're using plain-text LDAP (port 389), you'll need to specify MINIO_IDENTITY_LDAP_SERVER_INSECURE explicitly:

export MINIO_IDENTITY_LDAP_SERVER_INSECURE=true

The default protocol for MinIO is LDAPS. Have you noticed any connection failure in the gateway log?

Xuyuchao-juice · 2025-06-30T14:22:58Z

I added MINIO_IDENTITY_LDAP_SERVER_INSECURE=true to the environment variable, but it still didn't work.
gateway startup does have an error, the log is as follow :

root@iZ92wx62d5s1ruZ:~/code/juicefs# ./juicefs gateway sqlite3://test.db 0.0.0.0:9000 --multi-buckets
2025/06/30 22:14:22.852594 juicefs[960381] <INFO>: Meta address: sqlite3://test.db [NewClient@interface.go:578]
2025/06/30 22:14:22.857308 juicefs[960381] <INFO>: Data use file:///var/jfs/myjfs/ [initForSvc@gateway.go:245]
2025/06/30 22:14:22.857850 juicefs[960381] <INFO>: Disk cache (/var/jfsCache/9ce55670-b25a-4fd9-aafb-4d209edfe460/): used ratio - [space 79.6%, inode 15%] [newCacheStore@disk_cache.go:147]
2025/06/30 22:14:22.857928 juicefs[960381] <INFO>: Adjusted cache capacity based on freeratio: from 107374182400 to 37694648140 bytes [setlimitByFreeRatio@disk_cache.go:171]
2025/06/30 22:14:22.857954 juicefs[960381] <INFO>: Adjusted max items based on freeratio: from 0 to 2347329 items [setlimitByFreeRatio@disk_cache.go:176]
2025/06/30 22:14:22.861218 juicefs[960381] <INFO>: Create session 20 OK with version: 1.3.0-dev+2025-06-06.ccd547d7 [NewSession@base.go:535]
2025/06/30 22:14:22.862102 juicefs[960381] <INFO>: Prometheus metrics listening on 127.0.0.1:40327 [exposeMetrics@mount.go:134]

API: SYSTEM(bucket=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5, object=5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5)
Time: 22:14:22 CST 06/30/2025
RemoteHost: 5e76d207cf4ab20866fdc03c83e8a0f4e8f458e880777956ec0bae4e9f23f6c5
Error: *fmt.wrapError
      16: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config-current.go:373:cmd.lookupConfigs()
      15: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config-current.go:606:cmd.loadConfig()
      14: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config.go:242:cmd.initConfig()
      13: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/config.go:196:cmd.(*ConfigSys).Init()
      12: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/server-main.go:335:cmd.initAllSubsystems()
      11: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/server-main.go:297:cmd.initServer()
      10: /root/go/pkg/mod/github.com/stevapple/juicedata-minio@v0.0.0-20250307050705-9f083ad96a41/cmd/server-main.go:600:cmd.ServerMainForJFS()
       9: /root/code/juicefs/cmd/gateway.go:218:cmd.gateway2()
       8: /root/go/pkg/mod/github.com/minio/cli@v1.24.2/app.go:490:cli.HandleAction()
       7: /root/go/pkg/mod/github.com/minio/cli@v1.24.2/app.go:269:cli.(*App).Run()
       6: /root/code/juicefs/cmd/gateway.go:212:cmd.gateway()
       5: /root/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/command.go:177:v2.(*Command).Run()
       4: /root/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:389:v2.(*App).RunContext()
       3: /root/go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:254:v2.(*App).Run()
       2: /root/code/juicefs/cmd/main.go:101:cmd.Main()
       1: /root/code/juicefs/main.go:29:main.main()
Endpoint: http://0.0.0.0:9000 

Browser Access:
   http://0.0.0.0:9000

Object API (Amazon S3 compatible):
   Go:         https://docs.min.io/docs/golang-client-quickstart-guide
   Java:       https://docs.min.io/docs/java-client-quickstart-guide
   Python:     https://docs.min.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
IAM initialization complete

929496959 · 2025-08-13T02:48:47Z

Currently, when using Windows PowerShell and executing the command "minio.exe server ./data", an error is reported
`PS C:\Users\Administrator\Desktop\minio-new-base\miniobin> .\minio.exe server ./data

+---------------------------------------------------------------+
| You are running an older version of MinIO released 6 days ago |
| Update: Run mc admin update |
+---------------------------------------------------------------+

Waiting for all MinIO sub-systems to be initialized.. possible cause (Unable to list buckets to heal: disk not found)
Waiting for all MinIO sub-systems to be initialized.. trying to acquire lock
Waiting for all MinIO sub-systems to be initialized.. trying to acquire lock
Exiting on signal: INTERRUPT

API: SYSTEM()
Time: 10:46:41 CST 08/13/2025
DeploymentID: 6f16890a-c61e-4ca2-9bfe-ae8e15bbb42a
Error: Operation timed out (cmd.OperationTimedOut)
1: C:\Users\Administrator\Desktop\minio-new-base\cmd\data-scanner.go:144:cmd.runDataScanner()`

stevapple added 5 commits March 7, 2025 01:07

feat: listing LDAP users by policy mapping

657d4c6

fix: LDAP users are enabled

53d81c4

feat: web login (JWT) for LDAP users

c3c085a

fix: web login permission for LDAP users

3443c5e

fix: serializing LDAP user in web JWT

6b7d61c

stevapple force-pushed the ldap branch from b595ac2 to 6b7d61c Compare March 7, 2025 04:54

stevapple added 2 commits March 7, 2025 13:00

feat: listing LDAP groups by policy mapping

0d8689f

fix: don't list temp users

9f083ad

stevapple marked this pull request as ready for review March 7, 2025 05:10

zhijian-pro requested a review from eltonxiao124 May 30, 2025 03:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: enable basic LDAP functionalities#77

feat: enable basic LDAP functionalities#77
stevapple wants to merge 7 commits intojuicedata:new-basefrom
stevapple:ldap

stevapple commented Mar 7, 2025 •

edited

Loading

Uh oh!

Xuyuchao-juice commented Jun 25, 2025

Uh oh!

stevapple commented Jun 27, 2025

Uh oh!

Xuyuchao-juice commented Jun 30, 2025

Uh oh!

929496959 commented Aug 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

stevapple commented Mar 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How to test this PR?

Types of changes

Checklist:

Uh oh!

Xuyuchao-juice commented Jun 25, 2025

Steps to Reproduce

Uh oh!

stevapple commented Jun 27, 2025

Uh oh!

Xuyuchao-juice commented Jun 30, 2025

Uh oh!

929496959 commented Aug 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

stevapple commented Mar 7, 2025 •

edited

Loading