Transfer data between databases with PostgreSQL

Question

I need to transfer some data from another database. The old database is called paw1.moviesDB and the new database is paw1. The schema of each table are the following.

Awards (name of the table)(new DB)
Id [PK] Serial           Award

Nominations (name of the table) (old DB)
Id [PK] Serial           nominations

How do I copy the data from old database to the new database?

Solution 1

I just had to do this exact thing so I figured I'd post the recipe here. This assumes that both databases are on the same server.

First, copy the table from the old db to the new db. At the commandline:

pg_dump -U postgres -t <old_table> <old_database> | psql -U postgres -d <new_database>

Next, grant permissions of the copied table to the user of the new database. Log into psql:

psql -U postgres -d <new_database>

ALTER TABLE <old_table> OWNER TO <new_user>;

\q

At this point your copied table in your new database still has the name <old_table> from your old database. Assuming you want to move the data somewhere else, say to <new_table>, you can just use regular SQL queries:

INSERT INTO <new_table> (field1, field2, field3) 
SELECT field1, field2, field3 from <old_table>;

Done!

Solution 2

Databases are isolated in PostgreSQL; when you connect to a PostgreSQL server you connect to just one database, you can't copy data from one database to another using a SQL query.

If you come from MySQL: what MySQL calls (loosely) "databases" are "schemas" in PostgreSQL - sort of namespaces. A PostgreSQL database can have many schemas, each one with its tables and views, and you can copy from one schema to another with the schema.table syntax.

If you really have two distinct PostgreSQL databases, the common way of transferring data from one to another would be to export your tables (with pg_dump -t ) to a file, and import them into the other database (with psql).

If you really need to get data from a distinct PostgreSQL database, another option - mentioned in Grant Johnson's answer - is dblink, which is an additional module (in contrib/).

Update:

Postgres introduced "foreign data wrapper" in 9.1 (which was released after the question was asked). Foreign data wrappers allow the creation of foreign tables through the Postgres FDW which makes it possible to access a remote table (on a different server and database) as if it was a local table.

Solution 3

This worked for me to copy a table remotely from my localhost to Heroku's postgresql:

pg_dump -C -t source_table -h localhost source_db | psql -h destination_host -U destination_user -p destination_port destination_db

This creates the table for you.

For the other direction (from Heroku to local) pg_dump -C -t source_table -h source_host -U source_user -p source_port source_db | psql -h localhost destination_db

Solution 4

From: hxxp://dbaspot.c om/postgresql/348627-pg_dump-t-give-where-condition.html (NOTE: the link is now broken)

# create temp table with the data
psql mydb
CREATE TABLE temp1 (LIKE mytable);
INSERT INTO temp1 SELECT * FROM mytable WHERE myconditions;
\q

# export the data to a sql file
pg_dump --data-only --column-inserts -t temp1 mtdb > out.sql
psql mydb
DROP TABLE temp1;
\q

# import temp1 rows in another database
cat out.sql | psql -d [other_db]
psql other_db
INSERT INTO mytable (SELECT * FROM temp1);
DROP TABLE temp1;

Another method useful in remotes

  # export a table csv and import in another database
  psql-remote> COPY elements TO '/tmp/elements.csv' DELIMITER ',' CSV HEADER;
  $ scp host.com:/tmp/elements.csv /tmp/elements.csv
  psql-local> COPY elements FROM '/tmp/elements.csv' DELIMITER ',' CSV;

Solution 5

There are three options for copying it if this is a one off:

1. Use a db_link (I think it is still in contrib)
2. Have the application do the work.
3. Export/import

If this is an ongoing need, the answers are:

1. Change to schemas in the same DB
2. db_link

View more solutions

Comments

[b0fh]

If you have a decently recent version of Postgres (>=8.1) you can do ALTER TABLE Nominations SET SCHEMA target

[Lester Cheung]

Setting the search_path variable using "SET search_path TO blah" is a good way to work with different schemas without hurting your pinkies. You can make your changes permanent with "ALTER USER user SET search_path TO blah" - love it! ;-)

[Dev R]

how to do it for remote server ?

[Simon B.]

Doing the move using dblink, see stackoverflow.com/questions/14797327/…

[Fil]

In case you have different port and username in localhost the command is: pg_dump -C -t source_table -h localhost -p local_port -U local_user source_db | psql -h destination_host -U destination_user -p destination_port destination_db

[rdo]

@DevR, just add -h<remote hostname>

[sudo]

Thanks. This is the simplest solution that doesn't involve any extensions. Note that --column-inserts significantly slows it down, so you can remove that if the target database's table is known not to have any conflicts.

[sudo]

In case this is unclear to anyone, the second half is this: Create the table on the other DB: psql -d [other_db] -c "CREATE TABLE temp1 (LIKE mytable);", then insert into your other DB: cat out.sql | psql -d [other_db], then insert into the main table: psql -d [other_db] -c "INSERT INTO mytable (SELECT * FROM temp1);".

[sudo]

Schemas are nothing but namespaces, and they don't really provide isolation. A different database could be running on a different computer, or maybe on the same but with different performance/memory settings, and you might have two databases that have the same schemas (which we do have). A different database could even be a different version of Postgres, assuming they're compatible.

[medley56]

Actually Postgres does support this now, via foreign data wrappers.

[Vokail]

I want to point out how much easy is this approach, if you have simple data, without binary, blobs etc..

[dgan]

for future readers: canonical way of doing it now is to use postgres_fdw extension and foreign tables

[Stephen]

Note in the first command, if you need a password for both pg_dump and psql, they will both serve you the prompt at once. What worked for me is to enter the password, hit enter, then enter the password a second time, and hit enter again. The prompt itself looks messed up while you're doing this, but it works.