V. cephx authentication mechanism
- https://docs.ceph.com/en/pacific/rados/configuration/auth-config-ref/ # Configuration reference
- https://docs.ceph.com/en/pacific/architecture/#high -Availability authentication # high availability authentication
5.1 cephx agreement
- ceph uses cephx protocol to authenticate the client.
- cephx is used to authenticate, access and authorize the data stored in ceph, and to authenticate and authorize requests to access ceph. Requests to communicate with Mon must pass ceph authentication. However, ceph authentication can also be turned off at mon node, but any access will be allowed after turning off authentication, so the security of data cannot be guaranteed.
5.2 authorization process
- Each mon node can authenticate the client and distribute the secret key, so there is no single point of failure and authentication performance bottleneck for multiple mon nodes.
- The mon node will return the data structure used for identity authentication, including the session key used to obtain CEPH service. The session key is encrypted through the client secret key, which is configured in advance at the client, / etc/ceph/ceph.client.admin.keying
- The client uses the session key to request the required services from Mon. Mon provides a ticket to the client. The user verifies the client's identity to the OSD and other services that actually process data. Mon and OSD share a secret. Therefore, OSD will inform all the tickets issued by mon. The ticket has an expiration date.
- cephx authentication function is only limited to the components of ceph and cannot be extended to other non ceph components. ceph is only responsible for authentication and authorization and cannot solve the encryption problem of data transmission.
- The client requests authentication from mon;
- mon generates a session key, encrypts it with the key and sends it to the client;
- After receiving it, the client decrypts the key, obtains the session key, and applies to mon for tikey;
- After receiving the request, mon verifies the session key and sends tikey to the client;
- After receiving the ticket, the client uses tikey to access OSD;
- OSD verifies the ticket and returns data.
5.3 access process
- Regardless of the type of ceph client, such as block device, object storage, file system, ceph will store all data as objects in the storage pool.
- ceph users need access to the storage pool to read and write data.
- ceph users must have execution permission to use ceph management commands.
5.4 ceph users
- Users refer to individuals (ceph managers) or system participants (mon/osd/mds).
- By creating a user, you can control whether the user or that participant can access the ceph storage cluster, the accessible storage pool and the data of the storage pool.
- ceph supports many types of users, but all manageable users belong to client type.
- The reason for distinguishing user types is that system components such as mon/osd/mds use cephx protocol, but they are not clients.
- The user type and user name are separated by a point number. The format is TYPE.ID, such as client.admin
ceph@ceph-deploy:~/ceph-cluster$ cat /etc/ceph/ceph.client.admin.keyring
[client.admin]
key = AQALKythrxA6IhAAVDlgRrH+xWLn2FYbXAFRaw==
caps mds = "allow *"
caps mgr = "allow *"
caps mon = "allow *"
caps osd = "allow *"5.5 ceph authorization and enabling
5.5.1 authorization basis
- ceph describes the authorization scope or level that users can use for mon/osd or mds based on Capabilities (caps).
- General syntax format: daemon type 'allow caps' [...]
5.5.2 list of capacity
Click to view the coder: Grant read permission to the user. access monitor (mon)To retrieve CRUSH This capability is required when running the diagram. w: Grant the user write permission to the object. x: Grant user to call class methods(Including read and write)And the ability to execute in the monitor auth Ability to operate. *: Grant users access to specific Daemons/Read, write, and execute permissions for the storage pool, and the ability to execute administrative commandsclass-read: Grant the user the ability to call a class to read a method. It belongs to yes x Subset of capabilities.
class-write: Grant the user the ability to call the class write method. It belongs to yes x Subset of capabilities.profile osd: Grant a user a OSD Connect to other as OSD Or monitor permissions. grant OSD Permissions, make OSD Ability to handle duplicate detection signal traffic and status reports (get OSD Status information for).
profile mds: Grant a user a MDS Connect to other as MDS Or monitor permissions.
profile bootstrap-osd: Grant user boot OSD Permissions for(initialization OSD And will OSD join ceph colony)，Authorize the deployment tool to boot OSD You have the right to add a secret key.
profile bootstrap-mds: Grant the user the permission to boot the metadata server, and authorize the deployment tool to add a secret key when booting the metadata server.
5.5.3 MON capability
Click to view the code#Including r/w/x and allow profile cap (running diagram of caph) For example: mon 'allow rwx' mon 'allow profile osd'
5.5.4 OSD capability
#It includes r/w/x, clas read, class write and profile osd. In addition, the OSD capability also allows storage pool and namespace settings.for example:
osd 'allow capability' [pool=poolname] [namespace=namespace-name]
5.5.5 MDS capability
#Only allow or empty are allowedFor example:
mds 'allow'
5.6 ceph user management
5.6.1 user management foundation
- The user management function enables the ceph Cluster Administrator to directly create, update and delete users in the ceph cluster.
- When creating or deleting a user in the CEPH cluster, it may be necessary to distribute the secret key to the client in order to add the secret key to the secret key file / etc/ceph/ceph.client.admin.keying. This file can contain one or more user authentication information. Any node that owns this file will have the right to access CEPH and can use the right of any account, This file is similar to the / etc/passwd file of linux system.
5.6.2 ceph auth help
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth -h
General usage:
==============
usage: ceph [-h] [-c CEPHCONF] [-i INPUT_FILE] [-o OUTPUT_FILE]
[--setuser SETUSER] [--setgroup SETGROUP] [--id CLIENT_ID]
[--name CLIENT_NAME] [--cluster CLUSTER]
[--admin-daemon ADMIN_SOCKET] [-s] [-w] [--watch-debug]
[--watch-info] [--watch-sec] [--watch-warn] [--watch-error]
[-W WATCH_CHANNEL] [--version] [--verbose] [--concise]
[-f {json,json-pretty,xml,xml-pretty,plain,yaml}]
[--connect-timeout CLUSTER_TIMEOUT] [--block] [--period PERIOD]
Ceph administration tool
optional arguments:
-h, --help request mon help
-c CEPHCONF, --conf CEPHCONF
ceph configuration file
-i INPUT_FILE, --in-file INPUT_FILE
input file, or "-" for stdin
-o OUTPUT_FILE, --out-file OUTPUT_FILE
output file, or "-" for stdout
--setuser SETUSER set user file permission
--setgroup SETGROUP set group file permission
--id CLIENT_ID, --user CLIENT_ID
client id for authentication
--name CLIENT_NAME, -n CLIENT_NAME
client name for authentication
--cluster CLUSTER cluster name
--admin-daemon ADMIN_SOCKET
submit admin-socket commands ("help" for help)
-s, --status show cluster status
-w, --watch watch live cluster changes
--watch-debug watch debug events
--watch-info watch info events
--watch-sec watch security events
--watch-warn watch warn events
--watch-error watch error events
-W WATCH_CHANNEL, --watch-channel WATCH_CHANNEL
watch live cluster changes on a specific channel
(e.g., cluster, audit, cephadm, or '*' for all)
--version, -v display version
--verbose make verbose
--concise make less verbose
-f {json,json-pretty,xml,xml-pretty,plain,yaml}, --format {json,json-pretty,xml,xml-pretty,plain,yaml}
--connect-timeout CLUSTER_TIMEOUT
set a timeout for connecting to the cluster
--block block until completion (scrub and deep-scrub only)
--period PERIOD, -p PERIOD
polling period, default 1.0 second (for polling
commands only)
Local commands
ping <mon.id> Send simple presence/life test to a mon
<mon.id> may be 'mon.*' for all mons
daemon {type.id|path} <cmd>
Same as --admin-daemon, but auto-find admin socket
daemonperf {type.id | path} [stat-pats] [priority] [<interval>] [<count>]
daemonperf {type.id | path} list|ls [stat-pats] [priority]
Get selected perf stats from daemon/admin socket
Optional shell-glob comma-delim match string stat-pats
Optional selection priority (can abbreviate name):
critical, interesting, useful, noninteresting, debug
List shows a table of all available stats
Run <count> times (default forever),
once per <interval> seconds (default 1)
Monitor commands:
auth add <entity> [<caps>...] add auth info for <entity> from input file,
or random key if no input is given, and/or
any caps specified in the command
auth caps <entity> <caps>... update caps for <name> from caps specified
in the command
auth export [<entity>] write keyring for requested entity, or
master keyring if none given
auth get <entity> write keyring file with requested key
auth get-key <entity> display requested key
auth get-or-create <entity> [<caps>...] add auth info for <entity> from input file,
or random key if no input given, and/or any
caps specified in the command
auth get-or-create-key <entity> [<caps>...] get, or add, key for <name> from system/caps
pairs specified in the command. If key
already exists, any given caps must match
the existing caps for that key.
auth import auth import: read keyring file from -i <file>
auth ls list authentication state
auth print-key <entity> display requested key
auth print_key <entity> display requested key
auth rm <entity> remove all caps for <name>
5.6.3 user management
5.6.3.1 list users
5.6.3.1.1 list users
Click to view the code# TYPE.ID representation is adopted for users. For example, osd.0 specifies a user (node) with OSD class and ID 0, and client.admin is a user of client type with ID admin.
ceph@ceph-deploy:~/ceph-cluster$ ceph auth ls
osd.0
key: AQCGsDVhhhXOAxAAD7GOy2/7BctI1wRwHK17OA==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.1
key: AQAVsTVhp80wCxAA5nJA080MT3BjWtEzgjjwTQ==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.2
key: AQCA5DVhV53BFRAA/HbGphI7pVsUGIlliF1GYw==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.3
key: AQDLtDVhbgzOGhAAuxYy1eISS2jAgWNQMKp+Cg==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.4
key: AQArtTVh67tKIxAA9fR3RaP9tHE2LQ0dHABKqw==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.5
key: AQCftTVhcVw0ChAAH5oEXBvQ8VEl51tTBJ7gMQ==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.6
key: AQBZtjVhSeWQGRAAKEt6qSw0JhaS1cXHsH7ZzQ==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.7
key: AQD4tzVhpmG3JRAAWBQYh+BFXrN82qsQxlrUHw==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
osd.8
key: AQA5uDVhkEoKARAAa6Z4eEcme3yxVUN1hzBplQ==
caps: [mgr] allow profile osd
caps: [mon] allow profile osd
caps: [osd] allow *
client.admin
key: AQALKythrxA6IhAAVDlgRrH+xWLn2FYbXAFRaw==
caps: [mds] allow *
caps: [mgr] allow *
caps: [mon] allow *
caps: [osd] allow *
client.bootstrap-mds
key: AQALKythdSI6IhAAP/QG4NkILRTELptXEMYspA==
caps: [mon] allow profile bootstrap-mds
client.bootstrap-mgr
key: AQALKythMS86IhAAYztFGK8yvtXvrhU1GJM2Uw==
caps: [mon] allow profile bootstrap-mgr
client.bootstrap-osd
key: AQALKythwDk6IhAA9kqvq3TMGiIkjsS1JWTtLg==
caps: [mon] allow profile bootstrap-osd
client.bootstrap-rbd
key: AQALKythB0U6IhAAS4daHonswhJtdbtuWvGk1w==
caps: [mon] allow profile bootstrap-rbd
client.bootstrap-rbd-mirror
key: AQALKythSlA6IhAAHuhETKA8j5l+U6kUJHpfZQ==
caps: [mon] allow profile bootstrap-rbd-mirror
client.bootstrap-rgw
key: AQALKythNFs6IhAA3fq6Q6KJQpxEZG0OEL2fUA==
caps: [mon] allow profile bootstrap-rgw
mgr.ceph-mgr-01
key: AQCCPithsDrjERAArrm5PowGLYrEUnjHlmaA/Q==
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
mgr.ceph-mgr-02
key: AQBbITdh+dHoCxAAAYdKmuFK09dJ9axa0w72cA==
caps: [mds] allow *
caps: [mon] allow profile mgr
caps: [osd] allow *
installed auth entries:5.6.3.1.2 Export user information
ceph@ceph-deploy:~/ceph-cluster$ ceph auth ls -o auth_all.key installed auth entries:
5.6.3.2 Get the specified user information
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.admin
[client.admin]
key = AQALKythrxA6IhAAVDlgRrH+xWLn2FYbXAFRaw==
caps mds = "allow *"
caps mgr = "allow *"
caps mon = "allow *"
caps osd = "allow *"
exported keyring for client.admin5.6.3.3 ceph auth add
- This command is a canonical method for adding users. It creates users, generates secret keys, and adds all specified capabilities.
5.6.3.3.1 add user
ceph@ceph-deploy:~/ceph-cluster$ ceph auth add client.wgs mon 'allow r' osd 'allow rwx pool=wgspool' added key for client.wgs
5.6.3.3.2 user authentication
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.wgs for <entity> from input file, or random key if no input is[client.wgs] any caps specified in the command
key = AQCzwkZhgvOIEhAAJxUykVvuDsqPBfCWDoBG9w== r <name> from caps specified in the command
caps mon = "allow r" for requested entity, or master keyring if none given
caps osd = "allow rwx pool=wgspool" file with requested key
exported keyring for client.wgs 5.6.3.4 ceph auth get-or-create
- This command is one of the most common ways to create a user. It will return the key file containing the user name and key. If the user already exists, this command will only return the user name and key in the key file format. You can also use the - o specify file name option to save the output to a file.
5.6.3.4.1 create user
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get-or-create client.wgs01 mon 'allow r' osd 'allow rwx pool=wgspool'
[client.wgs01]
key = AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==5.6.3.4.2 user authentication
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.wgs01[client.wgs01]
key = AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==
caps mon = "allow r"
caps osd = "allow rwx pool=wgspool"
exported keyring for client.wgs015.6.3.4.3 create an existing user
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get-or-create client.wgs01 mon 'allow r' osd 'allow rwx pool=wgspool'
[client.wgs01]
key = AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==5.6.3.4.4 exporting user information
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get-or-create client.wgs01 mon 'allow r' osd 'allow rwx pool=wgspool' -o wgs.key
ceph@ceph-deploy:~/ceph-cluster$ cat wgs.key
[client.wgs01]
key = AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==5.6.3.5 ceph auth get-or-cereate-key
- This command creates a user and returns the user's secret key. This command is very useful for clients that only need the secret key. If the user already exists, this command returns only the key
- When creating a client, technology creates users who do not have capabilities. Users without capabilities can authenticate, but cannot perform other operations. Such clients cannot retrieve the cluster map from the monitor. If you want to add capabilities later, you can use the ceph auth caps command to create a user without capabilities.
- Typical users have at least read functions for ceph monitor and read and write functions for ceph osd. In addition, users' osd permissions are usually limited to accessing specific storage pools.
#If the user has a key, it will be displayed and not created ceph@ceph-deploy:~/ceph-cluster$ ceph auth get-or-create-key client.wgs01 mon 'allow r' osd 'allow rwx pool=wgspool' AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==
5.6.3.6 ceph auth print-key
Click to view the code#Only the key information of a single specified user is obtained ceph@ceph-deploy:~/ceph-cluster$ ceph auth print-key client.wgs01 AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==
5.6.3.7 ceph auth caps
- Use the ceph auth caps command to specify the user and change the user's capabilities. Setting a new capability will completely cover the current capability. Therefore, you need to add the existing and new capabilities of the previous user. If you look at the current capability, you can run ceph auth get USERTYPE.USERID. If you want to add a capability, you need to specify the existing capability when using the following format
- ceph auth USERTYPE.ID daemon 'allow [r|w|x] [pool=pool-name] [namespace=namespace-name] '
5.6.3.7.1 view user's current permissions
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.wgs01
[client.wgs01]
key = AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==
caps mon = "allow r"
caps osd = "allow rwx pool=wgspool"
exported keyring for client.wgs015.6.3.7.2 modify user permissions
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth caps client.wgs01 mon 'allow r' osd 'allow rw pool=wgspool' updated caps for client.wgs01
5.6.3.7.3 verification authority
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.wgs01
[client.wgs01]
key = AQCixkZhajt0FBAAB9Tvd938Ntv3zAM7rNjLdw==
caps mon = "allow r"
caps osd = "allow rw pool=wgspool"
exported keyring for client.wgs015.6.3.8 ceph auth del
- To delete a user, use ceph auth del TYPE.ID, where TYPE is one of client, osd, mon or mds, and ID is the user name or daemon ID
ceph@ceph-deploy:~/ceph-cluster$ ceph auth del client.wgs01 updated
5.7 key management
5.7.1 key management basics
- ceph key ring is a keyring file (collection file) that stores secrets, keys and certificates and allows clients to access ceph through authentication. A keyring file can store one or more authentication information. Each key has an entity name and permission. The type is:
- ｛client,mon,mds,osd｝.name
- When the client accesses the ceph cluster, ceph will use the following four key ring files to preset the key ring settings
- /Etc / CEPH / < $cluster name >. < user $type >. < user. $ID >. keyring # saves the keyring of a single user
- /etc/ceph/cluster.keyring # Save Keyrings for multiple users
- /etc/ceph/keyring # is the keyring of multiple users who define the cluster name
- /etc/ceph/keyring.bin # compiled binary
5.7.2 create user key ring file
- Use the CEPH authtool command to create a user Keyring file for it.
5.7.2.1 CEPH authtool help
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool -h
usage: ceph-authtool keyringfile [OPTIONS]...
where the options are:
-l, --list will list all keys and capabilities present in
the keyring
-p, --print-key will print an encoded key for the specified
entityname. This is suitable for the
'mount -o secret=..' argument
-C, --create-keyring will create a new keyring, overwriting any
existing keyringfile
-g, --gen-key will generate a new secret key for the
specified entityname
--gen-print-key will generate a new secret key without set it
to the keyringfile, prints the secret to stdout
--import-keyring FILE will import the content of a given keyring
into the keyringfile
-n NAME, --name NAME specify entityname to operate on
-a BASE64, --add-key BASE64 will add an encoded key to the keyring
--cap SUBSYSTEM CAPABILITY will set the capability for given subsystem
--caps CAPSFILE will set all of capabilities associated with a
given key, for all subsystems
--mode MODE will set the desired file mode to the keyring
e.g: '0644', defaults to '0600'5.7.2.2 create key file
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool --create-keyring wgs.key creating wgs.key
5.7.2.3 verification documents
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool -l wgs.key -rw-rw-r-- 1 ceph ceph 0 Sep 19 15:42 wgs.key
5.7.3 export user authentication information to keyring file
5.7.3.1 export user authentication information
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.wgs -o ceph.client.wgs.keyring exported keyring for client.wgs
5.7.3.2 verification and certification information
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool -l ceph.client.wgs.keyring
[client.wgs]
key = AQCzwkZhgvOIEhAAJxUykVvuDsqPBfCWDoBG9w==
caps mon = "allow r"
caps osd = "allow rwx pool=wgspool"5.7.4 recovering user authentication information from keyring file
- You can use ceph auth import -i to specify the keyring file and import it into ceph. In fact, it serves the purpose of user backup and recovery.
5.7.4.1 verifying user authentication documents
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool -l ceph.client.wgs.keyring
[client.wgs]
key = AQCzwkZhgvOIEhAAJxUykVvuDsqPBfCWDoBG9w==
caps mon = "allow r"
caps osd = "allow rwx pool=wgspool"5.7.4.2 delete user
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth del client.wgs updated
5.7.4.3 verifying and deleting users
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.wgs Error ENOENT: failed to find client.wgs in keyring
5.7.4.4 importing users
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth import -i ceph.client.wgs.keyring imported keyring
5.7.4.5 verifying users
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth get client.wgs
[client.wgs]
key = AQCzwkZhgvOIEhAAJxUykVvuDsqPBfCWDoBG9w==
caps mon = "allow r"
caps osd = "allow rwx pool=wgspool"
exported keyring for client.wgs5.7.5 the key ring file contains multiple users
5.7.5.1 create keyring file
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool --create-keyring ceph.client.user.keyring creating ceph.client.user.keyring
5.7.5.2 verifying the creation of keyring files
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool -l ceph.client.user.keyring
5.7.5.3 create test user information
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph auth add client.wgs mon 'allow r' osd 'allow rwx pool=wgspool' added key for client.wgs ceph@ceph-deploy:~/ceph-cluster$ ceph auth add client.wgs01 mon 'allow r' osd 'allow rwx pool=wgspool' added key for client.wgs01 ceph@ceph-deploy:~/ceph-cluster$ ceph auth add client.wgs02 mon 'allow r' osd 'allow rwx pool=wgspool' added key for client.wgs02
5.7.5.4 exporting user information
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool ceph.client.user.keyring --import-keyring ceph.client.admin.keyring importing contents of ceph.client.admin.keyring into ceph.client.user.keyring ceph@ceph-deploy:~/ceph-cluster$ ceph-authtool ceph.client.user.keyring --import-keyring ceph.client.wgs.keyring importing contents of ceph.client.wgs.keyring into ceph.client.user.keyring
5.7.5.5 verify the keyring file again
Click to view the codeceph@ceph-deploy:~/ceph-cluster$ ceph-authtool -l ceph.client.user.keyring
[client.admin]
key = AQALKythrxA6IhAAVDlgRrH+xWLn2FYbXAFRaw==
caps mds = "allow *"
caps mgr = "allow *"
caps mon = "allow *"
caps osd = "allow *"
[client.wgs]
key = AQCzwkZhgvOIEhAAJxUykVvuDsqPBfCWDoBG9w==
caps mon = "allow r"
caps osd = "allow rwx pool=wgspool"