I'm trying to migrate a database over to a new app from an existing one using pgbackups, but I'm running into issues. I've read the documentation on heroku's dev site but I'm still getting errors.
I've got the plugin installed for both databases and I can successfully go to my source db and copy/capture it.
$ heroku pgbackups:capture -a costrecovery --expire
HEROKU_POSTGRESQL_ONYX_URL (DATABASE_URL) ----backup---> b007
←[0KCapturing... doneB -
←[0KStoring... done
And then I change to the directory of the app I want to copy the db to and followed the instructions listed by heroku. The problem is that I don't know if it's not working because their is a bug or if it's because I'm not interpreting the instructions properly, which is entirely possible. First I'll list the instructions from heroku's dev site and then the commands I've tried.
HEROKUS INSTRUCTIONS
$ heroku pgbackups:restore DATABASE -a target-app \
`heroku pgbackups:url -a source-app`
COMMANDS I'VE TRIED
$ heroku pgbackups:restore DATABASE_URL -a boiling-reef-2060 \
> heroku pgbackups:url -a costrecovery
! Backup not found
$ heroku pgbackups:restore HEROKU_POSTGRESQL_GREEN_URL -a boiling-reef-2060 \
> 'heroku pgbackups:url -a costrecovery'
! Backup not found
$ heroku pgbackups:restore DATABASE -a boiling-reef-2060 \
> 'heroku pgbackups:url -a costrecovery'
! Backup not found
$ heroku pgbackups:restore DATABASE_URL -a costrecovery-copy2 \
> heroku pgbackups:"https://s3.amazonaws.com/hkpgbackups/app405411#heroku.com/b
007.dump?AWSAccessKe
> yId=AKIAJFDIRYCGYNFXR4FQ&Expires=1365184330&Signature=po0wZ982Jbx%2Fkv0bKk0iv
P%2
> FRWac%3D"
! Resource not found
Can someone helpl me out with the proper syntax?
Thanks
pgbackups can restore from a database on your same app, or from any pgbackups URL, as in, the one you captured in your source application/database. The instructions use backticks (`) to shell out and grab the pgbackups URL from your source application. The pgbackups:url command will provide such an URL. Try running this, to understand what's happening:
heroku pgbackups:url -a costrecovery
(assuming costrecovery is where you captured your data).
Knowing this, you should be able to simply run:
heroku pgbackups:restore DATABASE_URL --app boiling-reef-2060 `heroku pgbackups:url --app costrecovery
Related
I'm currently working on a simple project which is accessing the Github API.
PostgreSQL is used only to get more familiar with it.
I encountered a problem when trying to run the migration scripts from a makefile. This issue does only occur when I'm using make, by applying the migration scripts manually works fine.
The idea is to do this when running "make install":
Start the PostgreSQL container
Run migration scripts
Stop & remove the container
include ./config/dbConfig.env
all: install
.PHONY: all
install :
#runns all migration script. To run a specific migration version build more targets.
#echo "Get latest https://github.com/golang-migrate/migrate image"
docker pull migrate/migrate
#echo "Starting postgreSQL database"
docker run -t -d \
--name dev \
--env-file $(shell pwd)/config/dbConfig.env \
-p 5432:5432 \
-v $(shell pwd)/postgres:/var/lib/postgresql/data postgres
#echo "Wait for database"
sleep 5
#echo "Sun migration scripts"
docker run \
-v $(shell pwd)/migrations:/migrations \
--network host migrate/migrate \
-path=/migrations/ \
-database "postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}#localhost:5432/?sslmode=disable" up
# for specific migration: ....432/?sslmode=disable" up {{add integer migration number}}
#echo "Stop and remove PostgreSQL container"
docker stop `docker ps -aqf "name=dev"`
docker rm `docker ps -aqf "name=dev"`
You can also find the latest version is on the branch "develop" or this specific commit here
Error:
error: read tcp [::1]:34022->[::1]:5432: read: connection reset by peer
Why am I unable to connect to the server from the makefile but if I run the same procedure manually it works? Do you maybe have any ideas about why the behavior is not consistent?
When running postgresql alpine image with podman :
podman run --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=test -e POSTGRES_USER=test -d postgres:11-alpine
the result is :
Error: /usr/bin/slirp4netns failed: "open(\"/dev/net/tun\"): No such device\nWARNING: Support for sandboxing is experimental\nchild failed(1)\nWARNING: Support for sandboxing is experimental\n"
The running system is archlinux. Is there a way to fix this error or a turn arround ?
Thanks
Is slirp4netns correctly installed? Check the project Site for information.
Sometimes the flag order matters. try -d first and -p last (directly infornt of the image) looking like:
podman run -d --name postgres -e POSTGRES_PASSWORD=test -e POSTGRES_USER=test -p 5432:5432 postgres:11-alpine
Try only creating the neccessary password, then log into your container and create manually (this always worked for me)
podman run -d --name postgres -e POSTGRES_PASSWORD=test -p 5432:5432 postgres:11-alpline
podman exec -it postgres bash
Create default user postgres
su - postgres
start postgres
psql
create databases and tables
CREATE USER testuser WITH PASSWORD 'testpassword' | Doku
CREATE DATABASE testdata WITH OWNER testuser
Check if it worked
\l+
Connect to your Database via IP and Port
I assume you upgraded Arch packages recently. Most likely your system needs a restart.
I'm trying to clone private git repository from github. I did a Dockerfile like this:
FROM ubuntu:12.04
RUN apt-get update
RUN apt-get install -y git
RUN mkdir -p /root/.ssh/
ADD ./id_rsa /root/.ssh/id_rsa
RUN git clone git#github.com:usr/repo.git
I use this repo with this key just fine locally, so it seems I'm missing something inside docker.
One more thing I may be missing is that both ~ and $HOME inside docker point to / instead of /root, but I'm not sure if that can be related.
What's the output of the build process?
Random guess: try to chmod 600 the private key.
If it still doesn't work, try to RUN ssh -v git#github.com (after adding the key); the output should explain what's happening.
RUN ssh-keyscan github.com >> ~/.ssh/known_hosts
The keyscan works great since it accepts the host. The following complete answer worked:
RUN mkdir -p /root/.ssh
RUN cp /var/my-app/id_rsa /root/.ssh/id_rsa
RUN chmod 600 /root/.ssh/id_rsa
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts
Also as mentioned:
RUN ssh -v git#github.com
^ Great way to debug the flow. That's how I realized I needed the keyscan >> known_hosts
(Will probably not fit your needs)
There is another approach: https://stackoverflow.com/a/29464430/990356
Go to Settings > Personal access tokens and generate a personal access token with repo scope enabled.
Now you can do git clone https://MY_TOKEN#github.com/user-or-org/repo
Pros:
very simple approach
token can be easily revoked
Cons:
if someone has access to the Dockerfile he has access to the token
To fix this, you can use an environment variable to store the token
Below approach is using https with Personal Access Token, and it works like charm.
ARG git_personal_token
RUN git config --global url."https://${git_personal_token}:#github.com/".insteadOf "https://github.com/"
RUN git clone https://github.com/your/project.git /project
Then, supply a docker argument as below.
docker build --build-arg git_personal_token={your_token} .
Basic idea is from https://medium.com/paperchain/fetching-private-github-repos-from-a-docker-container-273f25ec5a74
I want to deploy my application on remote test server using capistrano gem.
Both git and rails should run on same server.
I have 2 users 'git' for git repositories and 'rails' with installed rvm. After git push i want to execute hook post-receive which runs su rails and then cap deploy.
When i tried push i got message:
remote: su: must be run from a terminal
How can i work around this message. Can i enable tty some way over git ssh connection?
I can give up from capistrano for this case but still i want rvm and rails to be used only by user rails (so su probably have to be used in each case).
edit
Now i walk around problem. Probably this is very bad solution but works ;). From bellow script's i removed original paths and echo's.
post-receive hook before walkaround
#!/bin/bash
while read oldrev newrev ref
do
su rails #here script fails
cd /path/to/rails/app/current/ && cap deploy
done
post-recive now
#!/bin/bash
while read oldrev newrev ref
do
ssh rails#localhost '/path/to/scripts/deploy.sh'
done
deploy.sh script
#!/bin/bash
CAP_DIR="/path/to/capistrano/dir"
RUBY="1.9.3-p194"
GEMSET="gemset_name"
[[ -s "$HOME/.rvm/scripts/rvm" ]] && . "$HOME/.rvm/scripts/rvm"
rvm use $RUBY
rvm gemset use $GEMSET
cd $CAP_DIR
cap deploy
I have downloaded the database from an existing Heroku app onto my local machine. The data I downloaded is in a single file; let's call it herokuapp.db. I also have Postgres installed and I can successfully create a new Postgres database and have my Rails app reference that. What I want to do is move the data I downloaded from Heroku into this database or create a new Postgres database using the downloaded data.
Use pg_restore --verbose --clean --no-acl --no-owner -d [database_name] [herokuapp.db]
see https://devcenter.heroku.com/articles/export-from-heroku-postgres for more information
I just solved the same problem with a similar technique that Will suggested (thanks Will!).
$ heroku pgbackups:capture
$ curl -o latest.dump `heroku pgbackups:url`
$ pg_restore --verbose --clean --no-acl --no-owner -d [database_name] latest.dump
The database_name can be found in the development section of the database.yml file.
EDIT
I recently performed this task again with Postgres.app and the last command above did not work. I was getting this error message:
pg_restore: connecting to database for restore
pg_restore: [archiver (db)] connection to database "[database_name]" failed: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/pgsql_socket/.s.PGSQL.5432"?
pg_restore: *** aborted because of error
Here is the updated command that worked:
$ pg_restore --verbose --clean --no-acl --no-owner -h localhost -U $USER -d [database_name] latest.dump