Shrinking a boot disk in GCP

TL;DR

Turn off instance
Detach boot disk
Create snapshot (as a backup)
Create target disk with new size
Spin up new instance, attaching the two disks
Execute a few partitioning commands
Execute a few dd commands for copying data over
Update GRUB and initramfs
Create similar instance to original, specifying the new disk as boot disk
Double check the instance starts, you can connect through SSH and it has all the data
Cleanup snapshots, old disks and old instances
Enjoy

Created a disk way too big

Recently, while working on a project, I had to increase the size of an existing boot disk attached to a GCP instance to be able to hold all the temporary files my pipeline was creating. Naively I resized to a huge disk, that obviously has a big impact on the cost at the end of the month. After running my pipeline, and potentially realizing I could just throw those files to a disposable external disk without affecting the main system, I found out you cannot just shrink a disk on the go.

I tried creating a snapshot and an image, but these were created with the new disk size in mind so I could never create a smaller disk.

Copy your data over, one might think. This was a boot disk, contains the whole linux distribution with its libraries, packages, manually built libraries from source and more. Copying data would just end up with a messed up new system. I needed an exact clone.

But these disks have several partitions too. In this case a root partition, where the main system is installed, a boot partition and a EFI partition. So we need specialized tools to make sure the new disk is an clone of the old one, just with free space.

Requirements

In order to clone the disk we need the following:

A new empty disk
A new instance to perform the copy (cheapest one)
Commands like lsblk, gdisk, dd and rsync

Playing with the disks

Assuming you know how to perform the first 5 steps, lets continue with step #6: Execute a few partitioning commands. First gather information about your disks using lsblk. Identify the original disk and the target disk. Then use df -h to get the sizes of the partitions. They would be something like sda, sdb or sdc. Based on that information, adjust the steps below to your situation. Add a few MB to each partition to avoid issues with space when copying data over.

Shrink root partition on original disk:

sudo resize2fs /dev/sdb1 98G

Partition new disk (make sure you are targeting the right disk):

# 0. Clean disk
sudo wipefs -a /dev/sda
sudo sgdisk --zap-all /dev/sda

# 1. Start gdisk on the target disk
sudo gdisk /dev/sda

# 2. Delete existing partitions (if any)
# Type 'o' (create new empty GPT), then 'y' to confirm.
o
y

# 3. Create sda14 (4M + 1M = 5M partition)
# Type 'n' (new partition)
n
# Partition number (14 to match original)
14
# First sector (default is fine)
<Enter>
# Last sector (+5M)
+5M
# Hex code or GUID (Default Linux filesystem '8300')
<Enter>

# 4. Create sda15 (EFI System Partition - 106M + 1M = 107M minimum, using 128M safe buffer)
n
# Partition number (15)
15
# First sector (default is fine)
<Enter>
# Last sector (+128M)
+128M
# Hex code or GUID (EFI System Partition code)
ef00

# 5. Create sda1 (The main root/data partition - 98G + 1M)
n
# Partition number (1)
1
# First sector (default is fine)
<Enter>
# Last sector (+98G+1M)
+98G+1M
# Hex code or GUID (Linux filesystem)
<Enter>

# 6. Create sda16 (Dedicated BOOT partition - 913M + 1M = 914M)
n
# Partition number (16)
16
# First sector (default is fine)
<Enter>
# Last sector (+914M)
+914M
# Hex code or GUID (Linux filesystem)
<Enter>

# 7. Review and Save
# Type 'p' (print partition table) to verify, then 'w' (write table to disk)
p
w
y

Format the partitions:

# Create mount points for root, efi, and the separate boot partition
mkdir -p /mnt/target_root /mnt/target_efi /mnt/target_boot
mkdir -p /mnt/source_root /mnt/source_efi /mnt/source_boot

# Mount sdc partitions (source)
sudo mount /dev/sdc1 /mnt/source_root
sudo mount /dev/sdc15 /mnt/source_efi
sudo mount /dev/sdc16 /mnt/source_boot

# Format sda partitions (target)
# Ensure these labels are exactly correct as per your fstab
sudo mkfs.ext4 /dev/sda1 -L cloudimg-rootfs -F  # Main Root (Label matched)
sudo mkfs.vfat -F 32 /dev/sda15 -n UEFI # EFI (Label matched)
sudo mkfs.ext4 /dev/sda16 -L BOOT -F # Separate Boot (Label matched)

# Mount sda partitions (target)
sudo mount /dev/sda1 /mnt/target_root
sudo mount /dev/sda15 /mnt/target_efi
sudo mount /dev/sda16 /mnt/target_boot

Copy data:

# 1. Copy the main root filesystem (sdc1 -> sda1)
# -a: archive mode (preserves permissions, ownership, timestamps, etc.)
# -x: don't cross filesystem boundaries
sudo rsync -axHAX --info=progress2 /mnt/source_root/ /mnt/target_root

# 2. Copy the EFI System Partition (sdc15 -> sda15)
sudo rsync -axHAX --info=progress2 /mnt/source_efi/ /mnt/target_efi

# 3. Copy the separate BOOT partition (sdc16 -> sda16)
sudo rsync -axHAX --info=progress2 /mnt/source_boot/ /mnt/target_boot

# Clone sdc14 to sda14 (5M)
# We use a larger block size for faster transfer
sudo dd if=/dev/sdc14 of=/dev/sda14 bs=4M status=progress

Finally, we have to manually update the grub configuration to update the UUIDs of the partitions. This wouldnt be necessary if GRUB was configured to use LABELs instead of UUIDs, and if the default Grub configuration in these GCP instances wouldn't hardcode these UUIDs.

# Get the UUIDs for your new sda partitions (sda1, sda15, sda16)
sudo blkid /dev/sda1 /dev/sda15 /dev/sda16

# Bind mount necessary filesystems for chroot
for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt/target_root$i; done

# 1. Mount the separate /boot partition
sudo mount /dev/sda16 /mnt/target_root/boot

# 2. FIX: Mount the EFI partition to the location GRUB expects (/boot/efi)
sudo mkdir -p /mnt/target_root/boot/efi
sudo mount /dev/sda15 /mnt/target_root/boot/efi

# Chroot into the new system (now rooted at sda1)
sudo chroot /mnt/target_root /bin/bash

# Reinstall the bootloader (GRUB) to the MBR of the new disk
# Make sure you are targeting the right disk
grub-install /dev/sda

# Update GRUB configuration
update-grub

# Manually update GRUB configuration
# Manually replace partial uuids with the partial uuid of your root partition
# Manually replace the uuid for the old boot partition with the uuid of the new boot partition
sudo vim /boot/grub/grub.cfg

# Rebuild initramfs to ensure the kernel is aware of the new UUID/PARTUUID
update-initramfs -u -k all

# Exit chroot
exit

# Unmount the EFI partition first, then the separate boot partition, then the rest.
sudo umount /mnt/target_root/boot/efi
sudo umount /mnt/target_root/boot

# Unmount all remaining partitions
sudo umount -R /mnt/target_root

Final test

Now go to your original instance and search for "Create similar". In the creation step, attach the disk we just created and cloned. Spin up the instance and check everything is fine.

It might be that the instance does start but you are unable to connect to it. Then you have to check the "Serial Console" logs. Select your instance in GCP and search for a link that says "Serial port 1 (console)". You will find the startup logs of the machine and might be able to tell what is wrong with it. Go back to attaching the disk to the helper instance, fix the issues using the commands above and all you have learned here and try again.