So we just add more space to the appliance and reboot it to expand the filesystem.
Here is how it is done -
Login to the VIC appliance and run df -h
You will see the filesystem /storage/data is almost full
Go to the vSphere Console and expand the drive to the desired value.
Note : You can hot add drive space.
The VIC appliance has 4 virtual disks attached to it. Refer to this table to find out which disk to expand.
This is a quick and dirty solution but in our case we kept adding space and went well over 1 TB. It was time to cleanup and put a permanent fix to this problem.
Harbor is the open source registry that runs in the VIC appliance. Behind the scenes it is nothing but an open source docker distribution with added functionalities such as security, identity and management.
As per the VIC documentation, you can simply enable garbage collection and after you reboot the appliance it should take effect. Unfortunately this did not work for us.
If you have not noticed yet, within the VIC appliance all the components run as individual containers (pretty cool)
SSH to the VIC appliance and run the simple command
docker ps -a
You will notice that all services are running as individual containers.
We are going to work on the highlighted Harbor registry container.
A garbage collector is the one that actually deleted the leftover blobs from different image tags.
If you are familiar with the docker exec command, we will use this simple command on the registry container to see what blobs can be marked for deletion.
docker exec registry bin/registry garbage-collect --dry-run /etc/registry/config.yml
To mark these blobs for deletion, run the above command without the --dry-run flag -
docker exec registry bin/registry garbage-collect /etc/registry/config.yml
Eligible blobs will be marked for deletion.
Next just restart the harbor service and the blogs will be actually deleted from the disk.
systemctl restart harbor
Happy days !