Corso Cloud Storage Es2

From Centro Calcolo
Jump to navigation Jump to search

Esercitazione 2

Creazione degli NSD

Creare gli NSD per il file system. Si devono creare i seguenti NSD:

  • disco da 10 GB per metadati
  • disco da 10 GB per metadati
  • disco da 50 GB per dati

I due dischi da 10 GB devono avere failure group diverso. Tutti e tre i dischi devono essere esportati dall'NSD server gpfs-srv-0.

Comandi: lsblk, mmcrnsd, mmlsnsd

Creazione dello stanza file

  • Visualizzare i dischi disponibili:
[root@gpfs-srv-0 ~]# lsblk
NAME                         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0                           11:0    1 1024M  0 rom  
vda                          252:0    0   32G  0 disk 
├─vda1                       252:1    0  500M  0 part /boot
└─vda2                       252:2    0 31.5G  0 part 
  ├─centos_gpfs--srv--0-root 253:0    0 28.3G  0 lvm  /
  └─centos_gpfs--srv--0-swap 253:1    0  3.2G  0 lvm  [SWAP]
vdb                          252:16   0   10G  0 disk 
vdc                          252:32   0   10G  0 disk 
vdd                          252:48   0   50G  0 disk 
vde                          252:64   0   50G  0 disk 
  • Creare lo stanza file modificando opportunamente i nomi dei device:
[root@gpfs-srv-0 ~]# cat nsd_stanza.txt 
%nsd:
  device=/dev/vdb
  nsd=metadata1
  servers=gpfs-srv-0
  usage=metadataOnly
  failureGroup=1

%nsd:
  device=/dev/vdc
  nsd=metadata2
  servers=gpfs-srv-0
  usage=metadataOnly
  failureGroup=2

%nsd:
  device=/dev/vdd
  nsd=data1
  servers=gpfs-srv-0
  usage=dataOnly
  failureGroup=1
  • Creare gli NSD
[root@gpfs-srv-0 ~]# mmcrnsd -F nsd_stanza.txt 
mmcrnsd: Processing disk vdb
mmcrnsd: Processing disk vdc
mmcrnsd: Processing disk vdd
mmcrnsd: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.

[root@gpfs-srv-0 ~]# mmlsnsd

 File system   Disk name    NSD servers                                    
---------------------------------------------------------------------------
 (free disk)   data1        gpfs-srv-0.recas.ba.infn.it 
 (free disk)   metadata1    gpfs-srv-0.recas.ba.infn.it 
 (free disk)   metadata2    gpfs-srv-0.recas.ba.infn.it 

[root@gpfs-srv-0 ~]# mmlsnsd -m

 Disk name    NSD volume ID      Device         Node name                Remarks       
---------------------------------------------------------------------------------------
 data1        AC14005F57F0FBEB   /dev/vdd       gpfs-srv-0.recas.ba.infn.it server node
 metadata1    AC14005F57F0FBE9   /dev/vdb       gpfs-srv-0.recas.ba.infn.it server node
 metadata2    AC14005F57F0FBEA   /dev/vdc       gpfs-srv-0.recas.ba.infn.it server node

Creazione e mount del file system

Creare un file system GPFS con gli NSD appena creati.

Il file system deve avere le seguenti caratteristiche:

  • Device name: /dev/gpfs_dev
  • Mount point: /gpfs
  • Block size: 1M
  • Deny-write open lock abilitato
  • ACL posix e nfs4
  • Quota
  • max replica 3 per dati e metadati, default replica 2 per metadati, 1 per dati
  • numero di max inode: 10M
  • Fileset df abilitato
  • Quota abilitata per fileset

Quindi montare il file system su tutti i nodi del cluster.

Comandi: mmcrfs, mmchfs, mmlsfs, mmdf, mmmount, mmlsmount, mmlsdisk

Creazione del file system

[root@gpfs-srv-0 ~]# mmcrfs /dev/gpfs_dev -F nsd_stanza.txt -D nfs4 -B 1M -j scatter \
    -k all -m 2 -M 3 -r 1 -R 3 -Q yes -T /gpfs --inode-limit 10M \
    --filesetdf --perfileset-quota

The following disks of gpfs_dev will be formatted on node gpfs-srv-0.recas.ba.infn.it:
    metadata1: size 10240 MB
    metadata2: size 10240 MB
    data1: size 51200 MB
Formatting file system ...
Disks up to size 856 GB can be added to storage pool system.
Creating Inode File
  39 % complete on Sun Oct  2 12:39:17 2016
  79 % complete on Sun Oct  2 12:39:22 2016
 100 % complete on Sun Oct  2 12:39:25 2016
Creating Allocation Maps
Creating Log Files
Clearing Inode Allocation Map
Clearing Block Allocation Map
Formatting Allocation Map for storage pool system
Completed creation of file system /dev/gpfs_dev.
mmcrfs: Propagating the cluster configuration data to all
  affected nodes.  This is an asynchronous process.

Mount del file system

[root@gpfs-srv-0 ~]# mmmount /dev/gpfs_dev -a
Sun Oct  2 12:40:30 UTC 2016: mmmount: Mounting file systems ...
[root@gpfs-srv-0 ~]# mmlsmount gpfs_dev -L
                                            
File system gpfs_dev is mounted on 2 nodes:
  172.20.0.95     gpfs-srv-0                
  172.20.0.55     stack-group-00            

[root@gpfs-srv-0 ~]# mmdf gpfs_dev
disk                disk size  failure holds    holds              free KB             free KB
name                    in KB    group metadata data        in full blocks        in fragments
--------------- ------------- -------- -------- ----- -------------------- -------------------
Disks in storage pool: system (Maximum disk size allowed is 797 GB)
metadata1            10485760        1 Yes      No          9975808 ( 95%)          2624 ( 0%) 
data1                52428800        1 No       Yes        52361216 (100%)          1888 ( 0%) 
metadata2            10485760        2 Yes      No          9975808 ( 95%)          2624 ( 0%) 
                -------------                         -------------------- -------------------
(pool total)         73400320                              72312832 ( 99%)          7136 ( 0%)

                =============                         ==================== ===================
(data)               52428800                              52361216 (100%)          1888 ( 0%)
(metadata)           20971520                              19951616 ( 95%)          5248 ( 0%)
                =============                         ==================== ===================
(total)              73400320                              72312832 ( 99%)          7136 ( 0%)

Inode Information
-----------------
Number of used inodes:            4038
Number of free inodes:           68154
Number of allocated inodes:      72192
Maximum number of inodes:        72192

Visualizzazione dello stato dei dischi

[root@gpfs-srv-0 ~]# mmlsdisk gpfs_dev
disk         driver   sector     failure holds    holds                            storage
name         type       size       group metadata data  status        availability pool
------------ -------- ------ ----------- -------- ----- ------------- ------------ ------------
metadata1    nsd         512           1 Yes      No    ready         up           system       
metadata2    nsd         512           2 Yes      No    ready         up           system       
data1        nsd         512           1 No       Yes   ready         up           system       

[root@gpfs-srv-0 ~]# mmlsdisk gpfs_dev -L
disk         driver   sector     failure holds    holds                                    storage
name         type       size       group metadata data  status        availability disk id pool         remarks   
------------ -------- ------ ----------- -------- ----- ------------- ------------ ------- ------------ ---------
metadata1    nsd         512           1 Yes      No    ready         up                 1 system        desc
metadata2    nsd         512           2 Yes      No    ready         up                 2 system        desc
data1        nsd         512           1 No       Yes   ready         up                 3 system        desc
Number of quorum disks: 3 
Read quorum value:      2
Write quorum value:     2

[root@gpfs-srv-0 ~]# mmlsdisk gpfs_dev -m

Disk name     IO performed on node     Device             Availability
------------  -----------------------  -----------------  ------------
metadata1     localhost                /dev/vdb           up
metadata2     localhost                /dev/vdc           up
data1         localhost                /dev/vdd           up

L'ultimo comando eseguito sul nodo stack-group-00 mostra l'utilizzo dell'NSD server:

root@stack-group-00:~# mmlsdisk gpfs_dev -m

Disk name     IO performed on node     Device             Availability
------------  -----------------------  -----------------  ------------
metadata1     gpfs-srv-0.recas.ba.infn.it -                  up
metadata2     gpfs-srv-0.recas.ba.infn.it -                  up
data1         gpfs-srv-0.recas.ba.infn.it -                  up