How to add space to root partition in Amazon.(LVM –> The normal tricks won’t work)

You may end up here because you did what I did and still got this error:

[root@ip-172-31-3-242 ec2-user]# xfs_growfs -d /
...
data size unchanged, skipping
[root@ip-172-31-3-242 ec2-user]# resize2fs /dev/mapper/centos-root # doesn't work on xfs / lvm
resize2fs 1.42.9 (28-Dec-2013)
resize2fs: Bad magic number in super-block while trying to open /dev/mapper/centos-root
Couldn't find valid filesystem superblock.
[root@ip-172-31-3-242 ec2-user]# lvextend -L +100G /dev/mapper/centos-root 
Insufficient free space: 25600 extents needed, but only 205 available

I decided to use a community Centos for my build.  ( I needed Centos because of some testing I wanted to complete.)  Of course the OS drive was only set to 8 Gigs, and the project I was building required had a huge git repo.  I am sure I could have paired down things but I might want to test other things later so I thought I’ll just expand the drive.

If I follow the standard documentation to expand a drive and it wasn’t working.  I have to say I love Amazon’s documentation and this was a rare miss, where I did exactly what I was supposed to and it did not work.  (I even tried turning off SWAP and removing the swap partitions as I thought maybe it was blocking my ability to expand).

This is when it occurred to me that I was going about this all wrong.  The partition was setup as LVM, so I didn’t need to expand the root partition, I could just add a drive and then add a new physical drive to the LVM group.  (If it’s been a while and you don’t know what I”m talking about here’s a good LVM explanation as a refresher.) The whole beauty of LVM is you can add a drive later and map it in and the magic is handled for you, you don’t need to get as low level as you used to, the OS will handle it for you.  (Let’s remember that this is a reason it’s not used for performance, the OS overhead turns out to be too much for good performance under heavy load but for testing/non performant systems this is totally acceptable, and makes things easy.)

So instead of expanding the root partition, I added and attached a new drive in AWS Console.  I then booted up the VM to map in the space.  OK the new drive has been added:(xvdb)

[root@ip-172-31-11-148 ec2-user]# lsblk
NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
xvda            202:0    0   20G  0 disk 
├─xvda1         202:1    0    1G  0 part /boot
└─xvda2         202:2    0    7G  0 part 
  ├─centos-root 253:0    0  6.2G  0 lvm  /
  └─centos-swap 253:1    0  820M  0 lvm  [SWAP]
xvdb            202:16   0   30G  0 disk 
[root@ip-172-31-11-148 ec2-user]#

I check the space of my current volume group:

[root@ip-172-31-11-148 ec2-user]# sudo vgdisplay
  --- Volume group ---
  VG Name               centos
  System ID             
  ...
  VG Size               <7.00 GiB
  PE Size               4.00 MiB
  Total PE              1791
  Alloc PE / Size       1791 / <7.00 GiB
  Free  PE / Size       0 / 0   
  VG UUID               Y4uKAX-qGPZ-gO8e-qvIa-LzoD-6Jps-UCPJ53

I then use fdisk to find the path to the partition I want to extend: (/dev/mapper/centos-root)

[root@ip-172-31-11-148 ec2-user]# fdisk -l

Disk /dev/xvda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000a3bc8

    Device Boot      Start         End      Blocks   Id  System
/dev/xvda1   *        2048     2099199     1048576   83  Linux
/dev/xvda2         2099200    16777215     7339008   8e  Linux LVM

Disk /dev/xvdb: 32.2 GB, 32212254720 bytes, 62914560 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/centos-root: 6652 MB, 6652166144 bytes, 12992512 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/centos-swap: 859 MB, 859832320 bytes, 1679360 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Now here’s the magic to add the drive into the centos volume group.  (If you need a refresher on the commands for LVM check out this website.)

Create the new disk as a physical volume so you can add it to the volume group:

[root@ip-172-31-11-148 ec2-user]# pvcreate /dev/xvdb
Physical volume "/dev/xvdb" successfully created.

Extend the volume group:

[root@ip-172-31-11-148 ec2-user]# vgextend centos /dev/xvdb
Volume group "centos" successfully extended

Extend the partition that you wish to be larger:

[root@ip-172-31-11-148 ec2-user]# lvextend -L32G /dev/mapper/centos-root 
Size of logical volume centos/root changed from <6.20 GiB (1586 extents) to 32.00 GiB (8192 extents).
Logical volume centos/root successfully resized.

Validate your results:

[root@ip-172-31-11-148 ec2-user]# sudo vgdisplay
--- Volume group ---
VG Name centos
System ID 
...
VG Size 36.99 GiB
PE Size 4.00 MiB
Total PE 9470
Alloc PE / Size 8397 / 32.80 GiB
Free PE / Size 1073 / 4.19 GiB
VG UUID Y4uKAX-qGPZ-gO8e-qvIa-LzoD-6Jps-UCPJ53

Extend your parition:

[ec2-user@ip-172-31-11-148 bigtop]$ sudo xfs_growfs -d /
meta-data=/dev/mapper/centos-root isize=512 agcount=4, agsize=406016 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=0 spinodes=0
data = bsize=4096 blocks=1624064, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0 ftype=1
log =internal bsize=4096 blocks=2560, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
data blocks changed from 1624064 to 8388608

Check with df -h

[ec2-user@ip-172-31-11-148 bigtop]$ df -h
Filesystem               Size  Used Avail Use% Mounted on
devtmpfs                 1.9G     0  1.9G   0% /dev
tmpfs                    1.9G     0  1.9G   0% /dev/shm
tmpfs                    1.9G   17M  1.9G   1% /run
tmpfs                    1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/mapper/centos-root   32G  3.2G   29G  10% /
/dev/xvda1              1014M  349M  666M  35% /boot
tmpfs                    379M     0  379M   0% /run/user/1000
tmpfs                    379M     0  379M   0% /run/user/0

References:

https://www.howtogeek.com/howto/40702/how-to-manage-and-use-lvm-logical-volume-management-in-ubuntu/

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/recognize-expanded-volume-linux.html

https://www.slashroot.in/advanced-guide-lvm-logical-volume-management-linux-part-1