Connect SQL IDE to docker db container? - sql

I am using docker with three services:
version: '2'
services:
nginx:
image: nginx:latest
container_name: nz01
ports:
- "8000:8000"
volumes:
- ./src:/src
- ./config/nginx:/etc/nginx/conf.d
depends_on:
- web
web:
build: .
container_name: dz01
depends_on:
- db
volumes:
- ./src:/src
expose:
- "8000"
db:
image: postgres:latest
container_name: pz01
volumes:
- ./postgres-data:/var/lib/postgresql/data
All seems to work pretty well, and I know that db service is running on 172.18.0.2 but when I use Datagrip to connect to the DB to work faster with table creation and inserts and things like that It does not work.
My configuration at django app is:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'pz01',
'PORT': 5432,
}
}
The configuration at Datagrip:
'DB_NAME': 'postgres',
'USER': 'postgres',
'HOST': 'pz01',
'PORT': 5432,
I also tried with host as the ip, and localhost and neither works.
The password as null, because the db service log shows that by default it does not have password.
How can I connect to my docker db from my macbook?
EDIT:
As #Dihgg says I fixed the port at DB service.
I set at
db:
image: postgres:latest
container_name: pz01
ports:
- "5433:5432"
volumes:
- ./postgres-data:/var/lib/postgresql/data
The port is not 5432:5432 because when this is the value I get:
ERROR: for db Cannot start service db: b'driver failed programming external connectivity on endpoint pz01 (146bb3cdada29f75766aa888143b4af17b267d13096db60fb0ccdaedb710e77e): Error starting userland proxy: Bind for 0.0.0.0:5432 failed: port is already allocated'
ERROR: Encountered errors while bringing up the project.
Despite that, I can't connect from Datagrip, when I use pz01 as host the error is(trying with 5432 or 5433 port and no password set):
Host is unknown
When I use as host the ip the error is: Connection failed

Try expose the port in the db container
db:
image: postgres:latest
container_name: pz01
ports:
- "5432:5432"
volumes:
- ./postgres-data:/var/lib/postgresql/data
And then, use the localhost:5432 to connect

Related

Openldap setup with server and UI fails

I'm trying to setup open ldap in linux VM and I am using openldap server from bitnami and
also the UI container. My docker-compose file is as follows:
version: '3'
networks:
openldap:
name: openldap
services:
openldap:
image: bitnami/openldap:latest
restart: unless-stopped
ports:
- '1389:1389'
- '1636:1636'
environment:
- LDAP_ORGANISATION=company
- LDAP_DOMAIN=company.network
- LDAP_ROOT=dc=company,dc=network
- LDAP_ADMIN_USERNAME=admin
- LDAP_ADMIN_PASSWORD=password
networks:
- openldap
volumes:
- 'openldap_data:/bitnami/openldap'
openldap-ui:
image: wheelybird/ldap-user-manager:latest
restart: unless-stopped
ports:
- 8082:80
environment:
- SERVER_HOSTNAME=localhost:8082
- LDAP_URI=ldap://openldap
- LDAP_BASE_DN=dc=company,dc=network
- LDAP_ADMINS_GROUP=admins
- LDAP_ADMIN_BIND_DN=cn=admin,dc=company,dc=network
- LDAP_ADMIN_BIND_PWD=password
- LDAP_IGNORE_CERT_ERRORS=true
- NO_HTTPS=true
networks:
- openldap
depends_on:
- openldap
volumes:
openldap_data:
driver: local
As per this documentation, when i try to do the initial setup via UI (http://ip-address:8082/setup), using the password 'password', I always get the following error.
Problem: Failed to bind as cn=admin,dc=company,dc=network
Wondering if anyone help identify what'm missing here?

Java Application on Docker (MySQL + Rabbit)

I am making an REST API which has a POST method which accepts a String as JSON and then sends it over RabbitMQ Topic and then a consumer is converting into entity and saving into a MySQL db.
Everything is running fine locally, but my task is to upload it on Docker and to be run online.
When I try to change "localhost" in properties with container names my Maven build fails....
How can i fix that?
I am putting my .properties:
spring.main.allow-bean-definition-overriding=true
spring.datasource.url=jdbc:mysql://localhost:3306/rabbit_messages?createDatabaseIfNotExist=true
spring.datasource.username=root
spring.datasource.password=1234
spring.rabbitmq.host = 127.0.0.1
spring.rabbitmq.port = 5672
spring.rabbitmq.username = guest
spring.rabbitmq.password = guest
and my docker-compose:
version: '3'
services:
my-app:
container_name: myapp
build:
context: .
dockerfile: Dockerfile
ports:
- "8080:8080"
links:
- rabbitmq
- mysql
environment:
- SPRING_DATASOURCE_URL=jdbc:mysql://mysql:3307/rabbit_messages?createDatabaseIfNotExist=true
- SPRING_DATASOURCE_USERNAME=root
- SPRING_DATASOURCE_PASSWORD=1234
- SPRING_RABBITMQ_HOST=rabbitmq
depends_on:
- rabbitmq
- mysql
java:
image: openjdk:latest
container_name: openjdk
rabbitmq:
image: rabbitmq:3-management
container_name: rabbitmq
ports:
- "5672:5672"
- "15672:15672"
erlang:
image: erlang:latest
container_name: erlang
mysql:
image: mysql:latest
container_name: mysql
ports:
- "3307:3307"
environment:
- MYSQL_DATABASE=rabbit_messages
- MYSQL_ROOT_PASSWORD=1234
- MYSQL_PASSWORD=1234
restart: on-failure

Running multiple docker-compose files with nginx reverse proxy

I asked a question here and got part of my problem solved, but I was advised to create another question because it started to get a bit lengthy in the comments.
I'm trying to use docker to run multiple PHP,MySQL & Apache based apps on my Mac, all of which would use different docker-compose.yml files (more details in the post I linked). I have quite a few repositories, some of which communicate with one another, and not all of them are the same PHP version. Because of this, I don't think it's wise for me to cram 20+ separate repositories into one single docker-compose.yml file. I'd like to have separate docker-compose.yml files for each repository and I want to be able to use an /etc/hosts entry for each app so that I don't have to specify the port. Ex: I would access 2 different repositories such as http://dockertest.com and http://dockertest2.com (using /etc/hosts entries), rather than having to specify the port like http://dockertest.com:8080 and http://dockertest.com:8081.
Using the accepted answer from my other post I was able to get one app running at a time (one docker-compose.yml file), but if I try to launch another with docker-compose up -d it results in an error because port 80 is already taken. How can I runn multiple docker apps at the same time, each with their own docker-compose.yml files and without having to specify the port in the url?
Here's a docker-compose.yml file for the app I made. In my /etc/hosts I have 127.0.0.1 dockertest.com
version: "3.3"
services:
php:
build: './php/'
networks:
- backend
volumes:
- ./public_html/:/var/www/html/
apache:
build: './apache/'
depends_on:
- php
- mysql
networks:
- frontend
- backend
volumes:
- ./public_html/:/var/www/html/
environment:
- VIRTUAL_HOST=dockertest.com
mysql:
image: mysql:5.6.40
networks:
- backend
environment:
- MYSQL_ROOT_PASSWORD=rootpassword
nginx-proxy:
image: jwilder/nginx-proxy
networks:
- backend
ports:
- 80:80
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
networks:
frontend:
backend:
I would suggest to extract the nginx-proxy to a separate docker-compose.yml and create a repository for the "reverse proxy" configuration with the following:
A file with extra contents to add to /etc/hosts
127.0.0.1 dockertest.com
127.0.0.1 anothertest.com
127.0.0.1 third-domain.net
And a docker-compose.yml which will have only the reverse proxy
version: "3.3"
services:
nginx-proxy:
image: jwilder/nginx-proxy
ports:
- 80:80
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
Next, as you already mentioned, create a docker-compose.yml for each of your repositories that act as web endpoints. You will need to add VIRTUAL_HOST env var to the services that serve your applications (eg. Apache).
The nginx-proxy container can run in "permanent mode", as it has a small footprint. This way whenever you start a new container with VIRTUAL_HOST env var, the configuration of nginx-proxy will be automatically updated to include the new local domain. (You will still have to update /etc/hosts with the new entry).
If you decide to use networks, your web endpoint containers will have to be in the same network as nginx-proxy, so your docker-compose files will have to be modified similar to this:
# nginx-proxy/docker-compose.yml
version: "3.3"
services:
nginx-proxy:
image: jwilder/nginx-proxy
ports:
- 80:80
networks:
- reverse-proxy
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
networks:
reverse-proxy:
# service1/docker-compose.yml
version: "3.3"
services:
php1:
...
networks:
- backend1
apache1:
...
networks:
- nginx-proxy_reverse-proxy
- backend1
environment:
- VIRTUAL_HOST=dockertest.com
mysql1:
...
networks:
- backend1
networks:
backend1:
nginx-proxy_reverse-proxy:
external: true
# service2/docker-compose.yml
version: "3.3"
services:
php2:
...
networks:
- backend2
apache2:
...
networks:
- nginx-proxy_reverse-proxy
- backend2
environment:
- VIRTUAL_HOST=anothertest.com
mysql2:
...
networks:
- backend2
networks:
backend2:
nginx-proxy_reverse-proxy:
external: true
The reverse-proxy network that is created in nginx-proxy/docker-compose.yml is referred as nginx-proxy_reverse-proxy in the other docker-compose files because whenever you define a network - its final name will be {{folder name}}_{{network name}}
If you want to have a look at a solution that relies on browser proxy extension instead of /etc/hosts, check out mitm-proxy-nginx-companion

reverse proxy docker container to another two docker containers, how to have multiple instances on a single computer

In this project I have an apache docker container (called loadbalancer) which points to either of two apache docker containers. If the path is "/support*" then it goes to the support container otherwise it goes to webapp. Currently to achieve this I have hard coded my docker compose networks subnet and each containers ipv4 address. Then an apache conf file just points to those hard coded ips. This works great for local development environments.
However, it doesn't work for staging servers which need to host multiple instances of the project. I can't spin up more than one instance of this docker-compose network due to the hardcoded subnet/ipv4 addresses. How can I achieve this load balancer setup without hard coding the subnet so I can have multiple instances. Or is there a better solution to achieve the desired effect of many copies being hosted on a single server such as many vhosts in apache container. What would you suggest? As I have no clue as to what would be best practice here.
loadbalancer.conf
<VirtualHost *:80>
TimeOut -1
ProxyPass "/support" "http://172.20.0.5/support"
ProxyPassReverse "/support" "http://172.20.0.5/support"
ProxyPass "/" "http://172.20.0.2/"
ProxyPassReverse "/" "http://172.20.0.2/"
ProxyPreserveHost On
TimeOut -1
</VirtualHost>
docker-compose.yml
version: '3.7'
networks:
pi-net:
ipam:
config:
- subnet: 172.20.0.0/24
services:
cli:
container_name: cli
build: ./docker/cli
networks:
pi-net:
ipv4_address: 172.20.0.3
volumes:
- type: bind
source: .
target: /srv/www
- type: bind
source: $HOME/.gitconfig
target: /home/developer/.gitconfig
extra_hosts:
- "pi.docker:172.20.0.2"
user: developer
stdin_open: true
tty: true
environment:
GIT_PAGER: cat
webapp:
container_name: webapp
build:
context: ./docker/web-server
args:
- vhostsFileName=webapp.conf
networks:
pi-net:
ipv4_address: 172.20.0.2
ports:
- 80
volumes:
- type: bind
source: .
target: /srv/www
# depends on cli because cli entrypoint.sh is creating var/ files needed by httpd
depends_on:
- "cli"
support:
container_name: support
build:
context: ./docker/web-server
args:
- vhostsFileName=support.conf
networks:
pi-net:
ipv4_address: 172.20.0.5
ports:
- 80
volumes:
- type: bind
source: .
target: /srv/www
# depends on cli because cli entrypoint.sh is creating var/ files needed by httpd
depends_on:
- "cli"
loadbalancer:
container_name: loadbalancer
build:
context: ./docker/web-server
args:
- vhostsFileName=loadbalancer.conf
networks:
pi-net:
ipv4_address: 172.20.0.6
ports:
- 80:80
db:
container_name: db
build: ./docker/mysql
networks:
pi-net:
ipv4_address: 172.20.0.4
ports:
- 3306:3306
volumes:
- db:/var/lib/mysql
environment:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: pi
MYSQL_USER: root
MYSQL_PASSWORD: root
restart: always
volumes:
db:
driver: local
Docker provides an internal DNS service to resolve container names as host names, and Docker Compose provides a network for you. You should make two changes:
In your Apache configuration, replace the explicit IP addresses with the name of the corresponding service block in the docker-compose.yml: http://support/support, for example.
Delete all of the networks: and container_name: settings in the docker-compose.yml, since they're redundant and limit reuse of the file. (Docker will assign IP addresses for you and Docker Compose will pick container names, but there's nothing wrong with these defaults.)
(Many questions of this form also use the outdated links: functionality; it's safe to delete all of the links: blocks too.)

502 Proxy Error ( docker + traefik + apache )

I'm trying to setup traefik for SSL termination on my local development instance. Following up this guide I have the following configuration.
docker-compose.yml
version: '2.1'
services:
mariadb:
image: wodby/mariadb:10.2-3.0.2
healthcheck:
test: "/usr/bin/mysql --user=dummyuser --password=dummypasswd --execute \"SHOW DATABASES;\" | grep database"
interval: 3s
timeout: 1s
retries: 5
restart: always
environment:
MYSQL_ROOT_PASSWORD: dummy
MYSQL_DATABASE: database
volumes:
- ./mariadb-init:/docker-entrypoint-initdb.d # Place init .sql file(s) here.
- mysql:/var/lib/mysql # I want to manage volumes manually.
php:
depends_on:
mariadb:
condition: service_healthy
ports:
- "25:25"
- "587:587"
environment:
PHP_FPM_CLEAR_ENV: "no"
DB_HOST: mariadb
#DB_USER: dummy
DB_PASSWORD: dummypasswd
DB_NAME: database
DB_DRIVER: mysql
PHP_POST_MAX_SIZE: "256M"
PHP_UPLOAD_MAX_FILESIZE: "256M"
PHP_MAX_EXECUTION_TIME: 300
volumes:
- codebase:/var/www/html/
- private:/var/www/html/private
solr:
image: mxr576/apachesolr-4.x-drupal-docker
ports:
- "8983:8983"
labels:
- 'traefik.backend=solr'
- 'traefik.port=8983'
# - 'traefik.frontend.rule=Host:192.168.33.10'
volumes:
- solr:/opt/solr/example/solr/collection1/data
restart: always
portainer:
image: portainer/portainer
command: --no-auth -H unix:///var/run/docker.sock
volumes:
- /var/run/docker.sock:/var/run/docker.sock
labels:
- 'traefik.backend=portainer'
- 'traefik.port=9000'
restart: always
apache:
image: wodby/php-apache:2.4-2.0.2
# ports:
# - "80:80"
depends_on:
- php
environment:
APACHE_LOG_LEVEL: warn
APACHE_BACKEND_HOST: php
APACHE_SERVER_ROOT: /var/www/html/drupal
volumes:
- codebase:/var/www/html/
- private:/var/www/html/private
labels:
- 'traefik.backend=apache'
- 'traefik.docker.network=proxy'
- "traefik.frontend.rule=Host:127.0.0.1"
- "traefik.enable=true"
- "traefik.port=80"
- "traefik.default.protocol=http"
restart: always
networks:
- proxy
traefik:
image: traefik
command: -c /traefik.toml --web --docker --logLevel=INFO
ports:
- '80:80'
- '443:443'
- '8888:8080' # Dashboard
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /codebase/traefik.toml:/traefik.toml
- /codebase/certs/cert.crt:/cert.crt
- /codebase/certs/cert.key:/cert.key
volumes:
solr:
external: true
mysql:
external: true
codebase:
external: true
private:
external: true
networks:
proxy:
external: true
traefik.toml
logLevel = "DEBUG" # <---
defaultEntryPoints = ["https", "http"] # <---
[accessLog]
[traefikLog]
[entryPoints]
[entryPoints.http]
address = ":80"
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
[entryPoints.https.tls]
[[entryPoints.https.tls.certificates]]
certFile = "/cert.crt"
keyFile = "/cert.key"
[retry]
[docker]
endpoint = "unix:///var/run/docker.sock"
watch = true
exposedbydefault = false
When trying to verify the instance, I get a 502 Bad Gateway
curl -i -k https://127.0.0.1
HTTP/1.1 502 Bad Gateway
Content-Length: 392
Content-Type: text/html; charset=iso-8859-1
Date: Fri, 14 Sep 2018 16:34:36 GMT
Server: Apache/2.4.29 (Unix) LibreSSL/2.5.5
X-Content-Type-Options: nosniff
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em>GET /index.php</em>.<p>
Reason: <strong>DNS lookup failure for: php</strong></p></p>
</body></html>
A reset for docker-compose and the docker network didn't help.
I've checked the issue on their repo and it seems like nobody got a definitive solution. Anybody has an idea on how to solve this?
Edit:Update for full docker-compose file.
You are trying to connect to php container from apache service using service discovery. But php container is not attached to the network proxy, Because you haven't declared network for it. The same case is with mariabd as well. So, When you connect to apache/traefik they look for host php which is not attached to the network proxy and throw error 502.
Unless and until you specify external network, Docker containers will not be connected to them.
Hence, You have to specify the network as follows for all the services in order to make docker service discovery work properly.
networks:
- proxy
Bonus:
Since you have done port mapping. You can also use public Ip of your host machine followed by the port to connect to services from docker container and from outside containers as well.
Example:
Let us assume your ip is 192.168.0.123 then you can connect to php from
any services in docker container and even from outside docker as 192.168.0.123:25 and 192.168.0.123:587. This is because you have exposed ports
25,587 by mapping them to host ports 25,587.
Some references:
Docker networking
Networking using the host network
Connect a container to a user-defined bridge
Networking with standalone containers
Service discovery
Networking in Compose (check "Specify custom networks" section)