Vryjdfkk

I am attempting to migrate from Postgres 9.6 to 10.3 and during the restore each index is recreated one by one - this is a problem.

So far I thought pg_dumpall is a good option.

pg_dumpall -U postgres -h localhost -p 5432 --clean --file=dumpall_clean.sql

Once this is done the file is around 1.2TB in size and I can load it to the new 10.3 instance with

psql -U postgres -h localhost -p 5433 < dumpall_clean.sql

simple.

Problem

As I learned the indicies are not backed up like tables are, they are simply recreated, and that is my problem.

The cluster has thousands of partitions each with several million rows and two indices (one BTREE and one GIST). This takes days since each index is created one by one.

As I have enough resources and I know which indices have to be created, I would perfer to do this step after the dump has been restored. Initially I made 8 FOR loops (to run in parallel) to go through the partitions, and created an index by moving a partition to a faster tablespace (SSD), create the index, then move the table and the index back to the default tablespace. So far this has worked for me.

Question
How can I have the same result* of a pg_dumpall dump without recreating the indices when loading the dumpall_clean.sql file?
A pg_dumpall --without-index would be nice.

"This currently includes information about database users and groups, tablespaces, and properties such as access permissions that apply to databases as a whole." - pg_dumpall manual

I can see one workaround for this, by using pg_dumpall in two steps:

pg_dumpall --schema-only ....

Then edit the file and extract the index definitions into a second file. You also need to extract the foreign keys, because you have to run them manually after the import (probably together with the index creation script)

Then run that script (without the indexes) to create the (empty) tables. You

pg_dumpall --data-only ....

Then run that script to import the data into the new database. After that run the FK and index creation scripts.

it should be possible to just filter them out using "grep" :

grep -v '^CREATE INDEX [^t]*;$'  dump.sql | psql

or

pg_dumpall "source db connection string"  | 

grep -v '^CREATE INDEX [^t]*;$'          | 

psql "destination db connection string"

should be safe unless you have matching lines inside stored code.

in your specific case:

grep -v '^CREATE INDEX [^t]*;$' dumpall_clean.sql | psql -U postgres -h localhost -p 5433