Corso Cloud Storage Es2
Jump to navigation
Jump to search
Esercitazione 2
Creazione degli NSD
Creare gli NSD per il file system. Si devono creare i seguenti NSD:
- disco da 10 GB per metadati
- disco da 10 GB per metadati
- disco da 50 GB per dati
I due dischi da 10 GB devono avere failure group diverso. Tutti e tre i dischi devono essere esportati dall'NSD server gpfs-srv-0.
Comandi: lsblk, mmcrnsd, mmlsnsd
Creazione dello stanza file
- Visualizzare i dischi disponibili:
[root@gpfs-srv-0 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom vda 252:0 0 32G 0 disk ├─vda1 252:1 0 500M 0 part /boot └─vda2 252:2 0 31.5G 0 part ├─centos_gpfs--srv--0-root 253:0 0 28.3G 0 lvm / └─centos_gpfs--srv--0-swap 253:1 0 3.2G 0 lvm [SWAP] vdb 252:16 0 10G 0 disk vdc 252:32 0 10G 0 disk vdd 252:48 0 50G 0 disk vde 252:64 0 50G 0 disk
- Creare lo stanza file modificando opportunamente i nomi dei device:
[root@gpfs-srv-0 ~]# cat nsd_stanza.txt %nsd: device=/dev/vdb nsd=metadata1 servers=gpfs-srv-0 usage=metadataOnly failureGroup=1 %nsd: device=/dev/vdc nsd=metadata2 servers=gpfs-srv-0 usage=metadataOnly failureGroup=2 %nsd: device=/dev/vdd nsd=data1 servers=gpfs-srv-0 usage=dataOnly failureGroup=1
- Creare gli NSD
[root@gpfs-srv-0 ~]# mmcrnsd -F nsd_stanza.txt mmcrnsd: Processing disk vdb mmcrnsd: Processing disk vdc mmcrnsd: Processing disk vdd mmcrnsd: Propagating the cluster configuration data to all affected nodes. This is an asynchronous process. [root@gpfs-srv-0 ~]# mmlsnsd File system Disk name NSD servers --------------------------------------------------------------------------- (free disk) data1 gpfs-srv-0.recas.ba.infn.it (free disk) metadata1 gpfs-srv-0.recas.ba.infn.it (free disk) metadata2 gpfs-srv-0.recas.ba.infn.it [root@gpfs-srv-0 ~]# mmlsnsd -m Disk name NSD volume ID Device Node name Remarks --------------------------------------------------------------------------------------- data1 AC14005F57F0FBEB /dev/vdd gpfs-srv-0.recas.ba.infn.it server node metadata1 AC14005F57F0FBE9 /dev/vdb gpfs-srv-0.recas.ba.infn.it server node metadata2 AC14005F57F0FBEA /dev/vdc gpfs-srv-0.recas.ba.infn.it server node
Creazione e mount del file system
Creare un file system GPFS con gli NSD appena creati.
Il file system deve avere le seguenti caratteristiche:
- Device name: /dev/gpfs_dev
- Mount point: /gpfs
- Block size: 1M
- Deny-write open lock abilitato
- ACL posix e nfs4
- Quota
- max replica 3 per dati e metadati, default replica 2 per metadati, 1 per dati
- numero di max inode: 10M
- Fileset df abilitato
- Quota abilitata per fileset
Quindi montare il file system su tutti i nodi del cluster.
Comandi: mmcrfs, mmchfs, mmlsfs, mmdf, mmmount, mmlsmount, mmlsdisk
Creazione del file system
[root@gpfs-srv-0 ~]# mmcrfs /dev/gpfs_dev -F nsd_stanza.txt -D nfs4 -B 1M -j scatter \ -k all -m 2 -M 3 -r 1 -R 3 -Q yes -T /gpfs --inode-limit 10M \ --filesetdf --perfileset-quota The following disks of gpfs_dev will be formatted on node gpfs-srv-0.recas.ba.infn.it: metadata1: size 10240 MB metadata2: size 10240 MB data1: size 51200 MB Formatting file system ... Disks up to size 856 GB can be added to storage pool system. Creating Inode File 39 % complete on Sun Oct 2 12:39:17 2016 79 % complete on Sun Oct 2 12:39:22 2016 100 % complete on Sun Oct 2 12:39:25 2016 Creating Allocation Maps Creating Log Files Clearing Inode Allocation Map Clearing Block Allocation Map Formatting Allocation Map for storage pool system Completed creation of file system /dev/gpfs_dev. mmcrfs: Propagating the cluster configuration data to all affected nodes. This is an asynchronous process.
Mount del file system
[root@gpfs-srv-0 ~]# mmmount /dev/gpfs_dev -a Sun Oct 2 12:40:30 UTC 2016: mmmount: Mounting file systems ... [root@gpfs-srv-0 ~]# mmlsmount gpfs_dev -L File system gpfs_dev is mounted on 2 nodes: 172.20.0.95 gpfs-srv-0 172.20.0.55 stack-group-00 [root@gpfs-srv-0 ~]# mmdf gpfs_dev disk disk size failure holds holds free KB free KB name in KB group metadata data in full blocks in fragments --------------- ------------- -------- -------- ----- -------------------- ------------------- Disks in storage pool: system (Maximum disk size allowed is 797 GB) metadata1 10485760 1 Yes No 9975808 ( 95%) 2624 ( 0%) data1 52428800 1 No Yes 52361216 (100%) 1888 ( 0%) metadata2 10485760 2 Yes No 9975808 ( 95%) 2624 ( 0%) ------------- -------------------- ------------------- (pool total) 73400320 72312832 ( 99%) 7136 ( 0%) ============= ==================== =================== (data) 52428800 52361216 (100%) 1888 ( 0%) (metadata) 20971520 19951616 ( 95%) 5248 ( 0%) ============= ==================== =================== (total) 73400320 72312832 ( 99%) 7136 ( 0%) Inode Information ----------------- Number of used inodes: 4038 Number of free inodes: 68154 Number of allocated inodes: 72192 Maximum number of inodes: 72192
Visualizzazione dello stato dei dischi
[root@gpfs-srv-0 ~]# mmlsdisk gpfs_dev disk driver sector failure holds holds storage name type size group metadata data status availability pool ------------ -------- ------ ----------- -------- ----- ------------- ------------ ------------ metadata1 nsd 512 1 Yes No ready up system metadata2 nsd 512 2 Yes No ready up system data1 nsd 512 1 No Yes ready up system [root@gpfs-srv-0 ~]# mmlsdisk gpfs_dev -L disk driver sector failure holds holds storage name type size group metadata data status availability disk id pool remarks ------------ -------- ------ ----------- -------- ----- ------------- ------------ ------- ------------ --------- metadata1 nsd 512 1 Yes No ready up 1 system desc metadata2 nsd 512 2 Yes No ready up 2 system desc data1 nsd 512 1 No Yes ready up 3 system desc Number of quorum disks: 3 Read quorum value: 2 Write quorum value: 2 [root@gpfs-srv-0 ~]# mmlsdisk gpfs_dev -m Disk name IO performed on node Device Availability ------------ ----------------------- ----------------- ------------ metadata1 localhost /dev/vdb up metadata2 localhost /dev/vdc up data1 localhost /dev/vdd up
L'ultimo comando eseguito sul nodo stack-group-00 mostra l'utilizzo dell'NSD server:
root@stack-group-00:~# mmlsdisk gpfs_dev -m Disk name IO performed on node Device Availability ------------ ----------------------- ----------------- ------------ metadata1 gpfs-srv-0.recas.ba.infn.it - up metadata2 gpfs-srv-0.recas.ba.infn.it - up data1 gpfs-srv-0.recas.ba.infn.it - up