Of IAM and Ceph Accounts

By Paul Heinlein | Apr 24, 2026

Beginning with release 19.2.0 (codenamed “Squid”), the Ceph Object Gateway, Ceph’s S3-workalike feature, supports user accounts “as an optional feature to enable the self-service management of Users, Groups and Roles similar to those in AWS Identity and Access Management (IAM).”

At work, our requirements differ somewhat from general object-storage usage:

  • A project, group, or lab purchases an object-storage quota, in terabytes.
  • The Principal Investigator (PI) specifies the names of buckets to create.
  • The PI identifies users whom should be given access and what level of access (read/write, read-only) each should be given.
  • Our group does the Ceph work to create the project and its quota, the buckets, users, and access rules.

The Old Way

We formerly created all users using the radosgw-admin utility. One of the users would be the project user; that user would hold the quota and be used to create and own all project buckets.

radosgw-admin user create --uid BigProject --display-name 'BigProject'
radosgw-admin quota set --quota-scope user --uid BigProject --max-size 10T --max-objects 100000
radosgw-admin quota enable --quota-scope user --uid BigProject

Normal users would then be created with what amounted to zero permissions:

# max-buckets needs to be set to "1," because "0" means "unlimited"
radosgw-admin user create --uid 'jdoe' --display-name 'Jane Doe' --max-buckets 1
radosgw-admin quota set --quota-scope=user --uid 'jdoe' --max-size=0
radosgw-admin quota enable --quota-scope=user --uid 'jdoe'

Then we added bucket policies that assigned read/write or read-only permissions to users who were allowed to access the project bucket(s).

So we effectively created two classes of Ceph users: project owners and project members. We had naming conventions that helped us identify them by name, which helped, but some people were members of multiple projects and were named in multiple bucket policies, which could be confusing.

The IAM Way

The new-style user accounts available beginning in Squid help almost completely separate project owners from project members. Additionally, access permissions can be using standard AWS policy language, such as that used in the widely deployed AmazonS3FullAccess policy.

Here’s the high-level overview of the process for creating the new accounts and their users:

  • Use radosgw-admin Ceph utility to
    • create the account (which I would typically call the “project,” but “account” is Ceph’s the official language) and its quota,
    • create the account’s root or superuser account.
  • Use the aws command-line utility with the root user’s credentials (access key ID and secret access key) to
    • create the account’s bucket(s),
    • create the account’s normal user identities,
    • create user group(s),
    • add user identities to the appropriate group(s),
    • assign policies to the group(s).
  • Distribute credentials for the IAM user identities to the appropriate personnel.

Here’s a shell script that does a very, very basic version of that process. It would need to be run as root (or under sudo permissions) on a machine with access to the ceph cluster (version 19.2.0 or higher) and that has recent versions of the aws and jq utilities installed.

ACCOUNTNAME="MyLabAccount"
ACCOUNTROOT="MyLabRoot"
AWS="aws --profile $ACCOUNTROOT --region default --endpoint https://your.cephrgw.com"

# space-separated list of buckets to create
BUCKETS="rawdata artifacts"

# In our setup, 1T is the minimum quota allocation, but not all
# situations will require quotas.
QUOTA="2T"

# space-separated list of usernames
USERS="david maria alex marco anna"

# create the account that holds the quota
radosgw-admin account get --account-name $ACCOUNTNAME >/dev/null 2>&1
if test $? -ne 0; then
  radosgw-admin account create --account-name=$ACCOUNTNAME
  if test $? -eq 0; then
    ACCOUNTID=$(radosgw-admin account get --account-name $ACCOUNTNAME | jq -r '.id')
  else
    echo "FATAL: could not create ceph account $ACCOUNTNAME"
    exit 1
  fi
else
  echo "FATAL: the account $ACCOUNTNAME already exists"
  exit 1
fi

# set and enable account quota, if necessary
if test -n "$QUOTA"
then
  radosgw-admin quota set \
    --quota-scope account --account-id $ACCOUNTID \
    --max-objects 100000 --max-size $QUOTA
  radosgw-admin quota enable --quota-scope account --account-id $ACCOUNTID
fi

# create the quota-account's root user and add its keys to
# /root/.aws/credentials.
radosgw-admin user info --uid $ACCOUNTROOT >/dev/null 2>&1
if test $? -ne 0; then
  radosgw-admin user create \
    --uid $ACCOUNTROOT --display-name $ACCOUNTROOT \
    --account-id $ACCOUNTID --account-root
  AKEY=$(radosgw-admin user info --uid $ACCOUNTROOT | jq -r '.keys[].access_key')
  SKEY=$(radosgw-admin user info --uid $ACCOUNTROOT | jq -r '.keys[].secret_key')
  echo "[${ACCOUNTROOT}]" >> /root/.aws/credentials
  echo "aws_access_key_id = $AKEY" >> /root/.aws/credentials
  echo "aws_secret_access_key = $SKEY" >> /root/.aws/credentials
else
  echo "user account $ACCOUNTROOT already exists."
fi

# the new root user now creates the buckets
for BUCKET in $BUCKETS
do
  $AWS s3api head-bucket --bucket $BUCKET >/dev/null 2>&1
  if test $? -ne 0; then
    $AWS s3api create-bucket --bucket $BUCKET
  else
    echo "a bucket named '$BUCKET' already exists in this"
    echo "ceph cluster. that is probably bad."
  fi
done

# in this simple version, all users have full S3 permissions;
# a more complex setup might have groups, each with its
# own access policy
for USER in $USERS
do
  $AWS iam create-user --user-name $USER
  $AWS iam create-access-key --user-name $USER
  $AWS iam attach-user-policy --user-name $USER \
    --policy-arn arn:aws:iam::aws:policy/AmazonS3FullAccess
done

Notes and Considerations

A couple items mentioned above deserve some further explanation.

Full access

In our environment, we don’t let normal users create or delete buckets (that’s up the PI) or use ACLs, so we replace the standard AmazonS3FullAccess with a local inline policy:

{
  "Version" : "2012-10-17",
  "Statement" : [
    {
      "Sid" : "UserDeny",
      "Effect" : "Deny",
      "Action" : [
        "s3:CreateBucket", "s3:DeleteBucket",
        "s3:PutObjectAcl", "s3:PutObjectVersionAcl", "s3:PutBucketAcl"
      ],
      "Resource": "*"
    },
    {
      "Sid" : "UserAllow",
      "Effect" : "Allow",
      "Action" : [ "s3:*", "s3-object-lambda:*" ],
      "Resource" : "*"
    }
  ]
}

An inline policy like this, unlike a standard AWS policy, gets “put” rather than “attached”:

$AWS iam put-user-policy --user-name $USER \
  --policy-name LocalFullAccess --policy-document file://policy.json

Also, inline policies are removed using delete-user-policy rather than detach-user-policy.

IAM users and ceph

When a ceph user account is created using the aws utility within an account, that user account is still visible to the radosgw-admin utility. The difference is that the user’s UID will not be the username but a GUID. For example,

[root]# radosgw-admin user list
[
    "59f571e9-74a5-43c3-903f-7eb57b63e275",
    "MyLabAccount",
    "08fe54bf-3559-431e-b9d4-6572402718c8",
    "MyLabRoot",
    "d28ce0be-70e9-4ecd-aa6a-311bfc331b0e",
]

radosgw-admin can query and manage those accounts just as they would old-style user account.