I am doing quite expensive for the memory parsing of 50000 HTML pages. While parsing, I am writing them to the database which is defined in my docker-compose.yml file. After 25000 pages are parsed parsing stops and I get such an error:
AsyncExecutor.default-11" java.lang.OutOfMemoryError: Java heap space
How can I change Java heap size in my docker-compose file so all the data could fit into the database?
My docker-compose file looks like this:
version: '2'
services:
app:
image: "user/app:latest"
depends_on:
- "db"
api:
image: "user/=api:latest"
ports:
- "7070:7070"
depends_on:
- "db"
db:
image: postgres
environment:
JVM_OPTS: "-Xms8g -Xms8g"
POSTGRES_DB: postgresdb
POSTGRES_USER: user
POSTGRES_PASSWORD: password
I would be grateful for any advice!
I've faced the same problem, I did limit the service memory usage (ex. I used 800m) by adding the following attribute:
"mem_limit: 800m"
Related
I have my docker-compose.override.yml set up below in Visual Studio 2022
elasticsearch:
container_name: elasticsearch
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- discovery.type=single-node
restart: always
ports:
- "9200:9200"
- "9300:9300"
networks:
elastic:
kibana:
container_name: kibana
restart: always
environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
depends_on:
- elasticsearch
ports:
- "5601:5601"
networks:
elastic:
networks:
elastic:
driver: bridge
Every time I try to configure kibana, after supplying a token, I get the following error from the kibana container w/out kibana getting fully configured.
2023-02-06 21:40:38 [2023-02-07T04:40:38.210+00:00][WARN ][plugins.alerting] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
2023-02-06 21:40:38 [2023-02-07T04:40:38.366+00:00][INFO ][plugins.ruleRegistry] Installing common resources shared between all indices
2023-02-06 21:40:38 [2023-02-07T04:40:38.508+00:00][INFO ][plugins.cloudSecurityPosture] Registered task successfully [Task: cloud_security_posture-stats_task]
2023-02-06 21:40:39 [2023-02-07T04:40:39.605+00:00][INFO ][plugins.screenshotting.config] Chromium sandbox provides an additional layer of protection, and is supported for Linux Ubuntu 20.04 OS. Automatically enabling Chromium sandbox.
2023-02-06 21:40:39 [2023-02-07T04:40:39.741+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. socket hang up - Local: 192.168.128.3:60856, Remote: 192.168.128.2:9200
2023-02-06 21:40:40 [2023-02-07T04:40:40.773+00:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/screenshotting/chromium/headless_shell-linux_x64/headless_shell
2023-02-06 21:40:42 [2023-02-07T04:40:42.194+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. socket hang up - Local: 192.168.128.3:44378, Remote: 192.168.128.2:9200
I am a bit at a loss as to why I can't get it configured. Any help appreciated.
I have a ASP.Net core Razor app that calls an api called TokenAPI, as well as another API called DevOps that calls the Token API, using the Service Invocation block. All of these are hosted in DAPR.
The web app and 2 APIs are hosted locally with Docker Compose. The web app can sucessfully call the Token API using the service invocation block, but the DevOps API fails with
"Exception":"System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:3500)"
I am at a loss as to why this is not working and have been comparing the configuration in my docker-compose file, but cannot figure out why the web-to-api works, whilst the api-to-api fails. Both the Web App and DevOps api uses the same code and config used to call the Token API when creating calling CreateInvokeHttpClient during app startup. So, with code being te same, the only place I can think of, must be the docker-compose:
cas.tokens.api:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://+:80;https://+:443
ports:
- "5105:80"
- "5115:443"
- "50005:50001"
volumes:
- ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
- ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
networks:
- mynetwork
cas.devops.api:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://+:80;https://+:443
ports:
- "5106:80"
- "5116:443"
- "50006:50001"
volumes:
- ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
- ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
networks:
- mynetwork
cas.web:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=https://+:443;http://+:80
ports:
- "5111:443"
volumes:
- ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
- ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
networks:
- mynetwork
castokensapi-dapr:
command: ["./daprd",
"-app-id", "tokensapi",
"-app-port", "80", --enable-api-logging,
"-dapr-http-port", "3605",
"-placement-host-address", "dapr-placement:50000",
"-components-path", "/components",
"-config", "/configuration/config.yaml"
]
volumes:
- "./dapr/components/:/components"
- "./dapr/configuration/:/configuration"
casdevopsapi-dapr:
command: ["./daprd",
"-app-id", "devopsapi",
"-app-port", "80", --enable-api-logging,
"-dapr-http-port", "3606",
"-placement-host-address", "dapr-placement:50000",
"-components-path", "/components",
"-config", "/configuration/config.yaml"
]
volumes:
- "./dapr/components/:/components"
- "./dapr/configuration/:/configuration"
casweb-dapr:
command: [
"./daprd",
"-app-id", "casweb", --enable-api-logging,
"-placement-host-address", "dapr-placement:50000",
"-components-path", "/components",
"-config", "/configuration/config.yaml"
]
volumes:
- "./dapr/components/:/components"
- "./dapr/configuration/:/configuration"
I am confused as to why DAPR resolves the endpoint to be 127.0.0.1:3500 instead of 127.0.0.1:3605, as per the config?
Below is a screenshot of what the app-id resolves for both service invocations from web-to-api to api-to-api.
The issue seems to be that the service location is either not working or, there is something more fundamental that I misunderstand.
Any help is much appreciated
[UPDATE]
I managed to get this to work, but not entirely sure why yet lol. Removing the "-dapr-http-port" fixes this issue, but I would still like to know why.
Will post here when I know more.
-Mike
Error: I won't open a connection to 127.0.0.1 (only to 172.20.0.1) (code=500)
My docker file is as below, I have tried passive ports range too but not sure about --expose here, I am able to connect at first through my code but at a second attempt or reusing it is throwing the above error.
I understand it has something with the passive mode but I have the configuration but not sure what is missing, please help.
version: '3'
services:
ftpd_server:
image: stilliard/pure-ftpd
container_name: pure-ftpd
privileged: true
ports:
- "21:21"
- "30000-30099:30000-30099"
volumes: # remember to replace /folder_on_disk/ with the path to where you want to store the files on the host machine
#- "/tmp/test/data:/home/username/"
#- "/tmp/test/passwd:/etc/pure-ftpd/passwd"
- "/upload:/home/upload"
#- "./certs:/etc/ssl/private"
environment:
PUBLICHOST: "localhost"
FTP_USER_NAME: lktransfer
FTP_USER_PASS: lktransfer
FTP_USER_HOME: /home/upload/
#FTP_PASSIVE_PORTS: "30000-30009"
FTP_MAX_CLIENTS: 50
FTP_PASSIVE_PORTS: 30000:30099
FTP_MAX_CONNECTIONS: 50
ADDED_FLAGS: "--tls=1"
#ADDED_FLAGS: "--tls=2"
TLS_USE_DSAPRAM: "true"
TLS_CN: "localhost"
TLS_ORG: "Ellkay"
TLS_C: "IN"
restart: always
I have few containers, brought up with docker-compose and I want to perform authentication on of the containers.
Below the piece that I assume should do that, but it doesn't go to the authentication-backend-nginx-private, directly lands on the mds-backend-nginx-private. I'm out of idea, what could be wrong about the config...
it works if authforward configured globally: in toml file under entrypoint section, but I want it to be per particular container..
mds-backend-nginx-private:
<<: *nginx-common
ports:
- 8186:80
networks:
- cloud_private
- mds-backend
restart: on-failure
environment:
- NGINX_SERVER_NAME=mds-backend-nginx-private
- WEBSITE_PROXY_NAME=mds-backend-web-private
- WEBSITE_PROXY_PORT=8000
labels:
- "traefik.http.middlewares.authf.ForwardAuth.Address=http://authentication-backend-nginx-private/api/v1/gateway/account?with_credentials=true"
- "traefik.docker.network=cloud_private"
- "traefik.http.routers.mds-backend.middlewares=authf"
- "traefik.frontend.rule=PathPrefix: /api/v1/mds/"```
Maybe, you are trying to use "middleware feature" with old traefik version.
Works in the toml file because you are using the "forward feature" present in old versions.
Check traefik tag image is equal or greater than 2.0
https://hub.docker.com/_/traefik
I got two Containers defined in a docker-compose file:
tomcat_webserver_api:
image: tomcat:8
volumes:
- ./API/Docker/API.war:/usr/local/tomcat/webapps/API.war
ports:
- "8080:8080"
depends_on:
- mysql_database
tomcat_webserver_anwendung:
image: tomcat:8
ports:
- "8081:8080"
volumes:
- ./Anwendung/Docker/Anwendung.war:/usr/local/tomcat/webapps/Anwendung.war
depends_on:
- tomcat_webserver_api
environment:
API_HOST: tomcat_webserver_api
API_PORT: 8080
Now i want to access the URL http://tomcat_webserver_api:8080/API/restaurants/Wochentag from Inside the Java Web Application with an HttpURLConnection.
Issue: It returns an 400 Error
java.io.IOException: Server returned HTTP response code: 400 for URL: http://tomcat_webserver_api:8080/API/restaurants/Wochentag
The Code is like that (The Headers are nearly the same when i try to connect to the URL via curl - this works inside the container huh):
URL api = UriBuilder.fromUri("http://" + "tomcat_webserver_api" + ":" + "8080" +"/API/restaurants/RestaurantSpeisen").build().toURL();
System.setProperty("http.agent", "curl/7.52.1");
HttpURLConnection connection = (HttpURLConnection) api.openConnection();
connection.setRequestMethod("GET");
connection.setRequestProperty("Host", "localhost");
connection.setRequestProperty("User-Agent", "curl/7.52.1");
connection.connect();
BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream(), "UTF-8"));
If i try to connect to http://172.20.0.3:8080/API/restaurants/Wochentag i got an 200-ok HTTP Response Code and the JSON-Data.
If i exec the API Container and inspect the logs i can see the 400 GET-Request.
Why is this happen?
http://172.20.0.3:8080/API/restaurants/Wochentag - Works
http://tomcat_webserver_api:8080/API/restaurants/Wochentag - Won't Work but not with an 404 Error
I have had the same issue as you, apparently underscore are not allowed as virtualhost, try to remove it, for example, use just tomcatwebserverapi, that should fix your problem.
See Can (domain name) subdomains have an underscore "_" in it? for more information about valid letters in hostnames.
Please give explicit container names a try:
tomcat_webserver_api:
image: tomcat:8
container_name: tomcat_webserver_api
volumes:
- ./API/Docker/API.war:/usr/local/tomcat/webapps/API.war
ports:
- "8080:8080"
depends_on:
- mysql_database
tomcat_webserver_anwendung:
container_name: tomcat_webserver_app
image: tomcat:8
ports:
- "8081:8080"
volumes:
- ./Anwendung/Docker/Anwendung.war:/usr/local/tomcat/webapps/Anwendung.war
depends_on:
- tomcat_webserver_api
environment:
API_HOST: tomcat_webserver_api
API_PORT: 8080
The "local only" configuration needs explicit container names to activate Docker's name lookup mechanism. In Swarm mode you wouldn't need to set container names.