Upgrading Postgres database running in Podman or Docker containers

💡

This guide shows instructions specific to versions 13 to 14, but the instructions are also valid for versions 15 and 16.

💡

Quick steps to upgrade Postgres database server (may apply for any future versions)

Background

I have Podman containers running with the container images auto-updated on a recurring schedule. The containers were running fine with the latest version (v13) and patches until today, but the container failed to start with the error message below after the container image was auto-upgraded to v14.

Error Message

💡

The data directory was initialized by PostgreSQL version 13,which is not compatible with this version 14.0

Reason

The reason is pretty self-explanatory: the container image has been updated to version 14, whereas the data directory mapped to your host path uses version 13 data. We will look at solving this issue quickly and in simple steps.

💡

In my case, I am using podman, but the steps are precisely the same for docker; you have to replace the "podman" keyword with the "docker" keyword to perform the same action.

Steps to fix the issue

1. Temporarily change the podman service to use the old version.

💡

Applicable only if you are using a systemd podman service

For example, see below:

systemd unit file screenshot

After restarting the service, it should still be able to start the database service successfully.

2. Take a backup of old version data (v13) in a file.

a. Login to the podman/docker image bash shell by running the command:

podman exec -it postgres bash

b. Go to the data directory:

cd /var/lib/postgresql/data

c. Run the dump all command. Note: this command should export all the databases.

pg_dumpall -U {postgres_user} > dump_13.0.sql

d. Exit from the container command prompt. using (ctrl+D)

3. Rename the folder of the host path data folder and create a new empty directory.

a. For example,

cd ${HOST_PATH}
mv data data_old

b. Create a new empty directory in place of the old directory,

mkdir data

4. Change the Podman systems service back to the original and restart the container service.

💡

In the case of the docker service, you can restart the container.

For example:

a. change the version back to the original.

screenshot of PostgresSQL podman systemd service file

vi /usr/lib/systemd/system/postgres.service
systemctl daemon-reload
systemctl restart postgres

5. Verify the database is back up and running with an empty database.

a. Check the container logs and see if the error related to the incompatible data directory error went away.
b. Try to connect and see data using your credentials set for the Postgres or root user.

6. Copy the dump file from the old data directory to the new one.

Commands:

cd ${HOST_PATH}cp data-old/dump-13.0.sql data/

7. Open a bash prompt inside the container and execute the data load command.

Commands:

podman exec -it postgres bash
cd /var/lib/postgresql/data
psql -U ${PGUSER} < dump_13.0.sql

8. Restart the container and verify the data and Postgres database version.

Command:

systemctl restart postgres