How to resolve CORS issues when querying Elasticsearch? - apache

I am working on a software project and I am using Elasticsearch to support my search functionality. I have a server that I use to test with and I am using docker compose to create my cluster. Whenever I use my apps search bar (hosted via Apache on the same server as ES), I can never get the responses back due to CORS restrictions (I am getting around this using a Chrome extension, but I don't want users to have to install this to search).
I have tried enabled the proper http settings in the elasticsearch.yml file, but that doesn't seem to have done anything. I am using the Elasticsearch javascript module to make requests.
docker-compose:
version: '3.6'
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.0
container_name: elasticsearch
environment:
- node.name=es01
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
nproc: 65535
memlock:
soft: -1
hard: -1
cap_add:
- ALL
privileged: true
deploy:
mode: global
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
volumes:
- esdata1:/usr/share/elasticsearch/data
- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
ports:
- 9200:9200
- 9300:9300
elasticsearch2:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.0
container_name: elasticsearch2
environment:
- node.name=es02
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.zen.ping.unicast.hosts=elasticsearch"
ulimits:
nproc: 65535
memlock:
soft: -1
hard: -1
cap_add:
- ALL
privileged: true
deploy:
mode: global
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
volumes:
- esdata2:/usr/share/elasticsearch/data
- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
elasticsearch3:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.0
container_name: elasticsearch3
environment:
- node.name=es03
- cluster.name=docker-cluster
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "discovery.zen.ping.unicast.hosts=elasticsearch"
ulimits:
nproc: 65535
memlock:
soft: -1
hard: -1
cap_add:
- ALL
privileged: true
deploy:
mode: global
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
volumes:
- esdata3:/usr/share/elasticsearch/data
- ./elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
kibana:
image: docker.elastic.co/kibana/kibana-oss:6.5.0
container_name: kibana
environment:
SERVER_NAME: localhost
ELASTICSEARCH_URL: http://elasticsearch:9200/
ports:
- 5601:5601
ulimits:
nproc: 65535
memlock:
soft: -1
hard: -1
cap_add:
- ALL
deploy:
mode: global
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
volumes:
esdata1:
esdata2:
esdata3:
elasticsearch.yml:
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: "X-Requested-With,X-Auth-Token,Content-Type, Content-Length, Authorization"
how I connect to ES (in javascript):
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
host: 'myServersIP:9200'
});
So when I go to my servers webpage (www.mydomain.com) I get my application, everything looks good. When I try to search, I get the COR issue and can't get any results back. I think this might be because I am connecting to my client via "myServersIP:9200" and not some proxied subdomain, but I am not sure. Any ideas on what I could be doing wrong/how to fix this issue?

Try the one under C:\ProgramData\Elastic\Elasticsearch\config (windows case)
Edit it and add the CORS cofiguration (responsibly) as the "*" is quite dangerous
Hope this will help you

Related

grpc_health_probe timeout with dotnet GRPC API

I'm running a dotnet GRPC API (recipients-api) and I want to use grpcui to test it. Both services are been declared on a docker-compose file. I'm declaring grpcui as a dependant service of recipients-api, and as I need it to be fully available before grpcui can run, I'm also using a health probe to ensure it's alive. The problem is that it seems like recipients-api entry point is not been run, it's like the process were stuck waiting for the health check to be sucessfull before hiting the container entrypoint, so it alsways throws a timeout. Any clues?
Here is my docker-compose file:
version: '3.4'
services:
recipientservice-api:
image: ${DOCKER_REGISTRY-}dataintegrationrecipientserviceapi
container_name: recipientservice-api
build:
context: ..
dockerfile: ../src/DataIntegration.RecipientService.Api/Dockerfile
environment:
ASPNETCORE_ENVIRONMENT: "Development"
ASPNETCORE_URLS: "http://+:5000"
MongoDb__ConnectionString: mongodb://wfuser:MyPassw0rd_#mongodb:27017/RecipientService?tls=false
MongoDb__"Database": "RecipientService"
expose:
- "5000"
ports:
- "5000:5000"
depends_on:
- mongodb
networks:
- recipients-network
healthcheck:
test: ["CMD", "bin/grpc_health_probe", "-connect-timeout 10s", "-rpc-timeout 4s", "-addr=localhost:5000"]
interval: 2s
retries: 5
start_period: 15s
timeout: 10s
grpcui:
image: fullstorydev/grpcui
container_name: grpcui
depends_on:
recipientservice-api:
condition: service_healthy
command:
- -plaintext
- -vvv
- recipientservice-api:5000
networks:
- recipients-network
ports:
- "8080:8080"
mongodb:
image: mongo:5.0
container_name: "mongodb"
hostname: mongodb
environment:
MONGO_INITDB_ROOT_USERNAME: admin
MONGO_INITDB_ROOT_PASSWORD: MyPassw0rd_123
MONGO_INITDB_DATABASE: RecipientService
volumes:
- mongo-recipients:/var/opt/mongodb
- ./init-mongo.js:/docker-entrypoint-initdb.d/init-mongo.js:ro
ports:
- "27017:27017"
expose:
- 27017
networks:
- recipients-network
networks:
recipients-network:
name: recipients-network
driver: bridge
volumes:
mongo-recipients:

How to setup Traefik as reverse proxy for ASP.NET Core app with kestrel?

So long ago I started studying the data of the solution, but I am so stupid that I have not found a solution to how to configure file docker-compose for a simple ASP.NET Core for traefik proxy.
I took an example of a simple ASP.NET Core application from the Microsoft site, which, after deployment, is available at localhost:8443 using https, because earlier I released a self-signed (aspnetapp.pfx), ok.
Then I unfolded the traefik and configured the dashboard I see that traefik gets information about the aspnet_demo container, but at web app addresses, or webapp.mydomen.com/ or localhost nothing is available - maximum I get the error ERR_TOO_MANY_REDIRECTS in browser.
In logs traefik when referring to webapp.mydomen.com I get "RequestURI ": "/ "
What did you forget to point out?
I understand that the content aspnet_demo get on 443 port, so I tell Traefik where to look, but nothing...
Help me please understanding this. Thank you
My docker compose ASP.NET Core app looks like this:
version: "3.8"
services:
aspnet_demo:
image: mcr.microsoft.com/dotnet/core/samples:aspnetapp
container_name: aspnet_sample
ports:
- 8080:80
- 8443:443
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=https://+:443;http://+:80
- ASPNETCORE_Kestrel__Certificates__Default__Password=password
- ASPNETCORE_Kestrel__Certificates__Default__Path=/https/aspnetapp.pfx
volumes:
- ~/.aspnet/https:/https:ro
networks:
- traefik-reverse-proxy
labels:
- traefik.enable=true
- traefik.http.routers.aspnet.entrypoints=web
- traefik.http.routers.aspnet.rule=Host(`webapp`)
- traefik.http.routers.aspnet_secure.entrypoints=web-secure
- traefik.http.routers.aspnet_secure.rule=Host(`webapp.mydomen.com`)
- traefik.http.routers.aspnet_secure.tls=true
- traefik.http.services.aspnet.loadbalancer.server.port=443
networks:
traefik-reverse-proxy:
external: true
My docker compose Traefik looks like this:
version: "3.8"
services:
traefik:
image: traefik:v2.9
ports:
- "80:80"
- "443:443"
- "8080:8080"
networks:
- traefik-reverse-proxy
volumes:
- "/var/run/docker.sock:/var/run/docker.sock:ro"
- "./configuration/dynamic.yaml:/traefik_conf/dynamic.yaml"
- "./configuration/traefik.yml:/traefik.yml:ro"
- "./cert/:/traefik_conf/cert/"
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.service=api#internal"
- "traefik.http.routers.traefik.entrypoints=web-secure"
- "traefik.http.routers.traefik.rule=Host(`traefiklocal.mydomen.com`)"
- "traefik.http.routers.traefik.tls=true"
- "traefik.http.routers.traefik.tls.certresolver=tls"
- "traefik.http.routers.traefik.middlewares=traefik-auth"
- "traefik.http.middlewares.traefik-auth.basicauth.users=unixhost:$$apr1$$vqyMX723$$6nZ1lC3/2JN6QJyeEhJB8/"
networks:
traefik-reverse-proxy:
external: true
My static config Traefik looks like this:
api:
dashboard: true
insecure: true
log:
level: DEBUG
entryPoints:
web:
address: ":80"
forwardedHeaders:
insecure: true
http:
redirections:
entryPoint:
to: web-secure
web-secure:
address: ":443"
providers:
docker:
watch: true
exposedbydefault: false
file:
directory: /traefik_conf/
watch: true
filename: dynamic.yaml
My dynamic config Traefik:
tls:
certificates:
# first certificate
- certFile: "/traefik_conf/cert/pem_com_2022.pem"
keyfile: "/traefik_conf/cert/star_com_2022.key"
# second certificate
- certFile: "/traefik_conf/cert/aspnetapp.pem"
keyfile: "/traefik_conf/cert/aspnetapp.key"
stores:
- default

Gitea + drone ruuner-ssh and runner-exec not working

I had deploy Gitea + drone + runner for my group. Normally runner-docker works fine, but runner-ssh and runner-exec don't work
deploy by docker compose
gitea
version: "3"
services:
gitea:
image: gitea/gitea:1.15.7
# container_name: gitea
environment:
- USER_UID=1000
- USER_GID=1000
- DB_TYPE=mysql
- DB_HOST=db:3306
- DB_NAME=gitea
- DB_USER=gitea
- DB_PASSWD=xxxxxxx
restart: always
volumes:
- ./gitea:/data
- /home/git/.ssh/:/data/git/.ssh
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
ports:
- "33333:3000"
- "22:22"
depends_on:
- db
db:
image: mariadb
restart: always
environment:
- MYSQL_ROOT_PASSWORD=xxxxxx
- MYSQL_USER=gitea
- MYSQL_PASSWORD=xxxxxx
- MYSQL_DATABASE=gitea
volumes:
- ./db:/var/lib/mysql
drone
version: "3"
services:
drone:
image: drone/drone
container_name: drone
ports:
- "8800:80"
# - "44443:443"
volumes:
- ./drone:/data
environment:
- DRONE_GITEA_SERVER=https://git.ioiox.com
- DRONE_GITEA_CLIENT_ID=xxxxxxxxx
- DRONE_GITEA_CLIENT_SECRET=xxxxxxxxx
- DRONE_RPC_SECRET=yyyyyyyyyyyyyy
- DRONE_SERVER_HOST=drone.ioiox.com
- DRONE_SERVER_PROTO=https
- DRONE_GIT_ALWAYS_AUTH=true
- DRONE_USER_CREATE=username:stilleshan,admin:true
restart: always
runner-docker & runner-ssh
version: "3"
services:
drone-runner-docker:
image: drone/drone-runner-docker:1
container_name: drone-runner-docker
# ports:
# - "3000:3000"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
environment:
- TZ=Asia/Shanghai
- DRONE_RPC_PROTO=https
- DRONE_RPC_HOST=drone.ioiox.com
- DRONE_RPC_SECRET=yyyyyyyyyyyyyy
- DRONE_RUNNER_CAPACITY=5
- DRONE_RUNNER_NAME=runner-docker
restart: always
drone-runner-ssh:
image: drone/drone-runner-ssh
container_name: drone-runner-ssh
# ports:
# - 3001:3000
environment:
- TZ=Asia/Shanghai
- DRONE_RPC_PROTO=https
- DRONE_RPC_HOST=drone.ioiox.com
- DRONE_RPC_SECRET=yyyyyyyyyyyyyy
- DRONE_RUNNER_CAPACITY=5
- DRONE_RUNNER_NAME=runner-ssh
restart: always
runner-exec
Install in centos7 by drone.io documents
issue
runner-exec
When I push to private repo activate runner, something wrong about git clone. But public repo works fine.
I tried setting DRONE_GIT_ALWAYS_AUTH=true or false also to no effect
enter image description here
runner-ssh
I use type: ssh in .drone.yml, Can trigger workflows. But got clone: skipped error, when i set disable clone step, also got error "greeting: skipped", I think the whole workflow can not working.
enter image description here

Prometheus sql doesn't seem to execute the query. Always display -1 as result

I am using https://github.com/chop-dbhi/prometheus-sql for prometheus to execute sql queries. The containers are all up and running.
The value of query_result_example is always -1 (query-value-on-error = -1 as configured in prometheus-sql.yml)provided the actual value is 3.
I have executed the same query on Azure Data studio with same connection and credentials which returns 3.
I am unable to figure out where i am going wrong. Any help is appreciated.
docker-compose file:
prometheus:
image: prom/prometheus:v2.19.2
container_name: prometheus
volumes:
- ./prometheus:/etc/prometheus
- prometheus_data:/prometheus
command:
- '--config.file=/etc/prometheus/prometheus.yml'
- '--storage.tsdb.path=/prometheus'
- '--web.console.libraries=/etc/prometheus/console_libraries'
- '--web.console.templates=/etc/prometheus/consoles'
- '--storage.tsdb.retention.time=200h'
- '--web.enable-lifecycle'
restart: unless-stopped
ports:
- "9090:9090"
expose:
- 9090
networks:
- monitor-net
labels:
org.label-schema.group: "monitoring"
sql-server-db:
container_name: sql-server-db
image: microsoft/mssql-server-linux:2017-latest
ports:
- "1433:1433"
expose:
- 1433
environment:
SA_PASSWORD: "Password"
ACCEPT_EULA: "Y"
sqlagent:
image: dbhi/sql-agent
container_name: sqlagent
ports:
- "5000:5000"
expose:
- 5000
prometheussql:
image: dbhi/prometheus-sql
container_name: prometheussql
volumes:
- ${PWD}/prometheus-sql.yml:/prometheus-sql.yml
- ${PWD}/test/queries.yml:/queries.yml
links:
- sqlagent:sqlagent
command: >
-service http://sqlagent:5000
-config /prometheus-sql.yml
ports:
- "8080:8080"
expose:
- 8080
networks:
- monitor-net
labels:
org.label-schema.group: "monitoring"
queries.yml: (the below query has to return 3)
- example:
sql: >
select count(*) from MSreplication_options
interval: 5m
prometheus-sql.yml
defaults:
data-source: my-ds
query-interval: 10s
query-timeout: 5s
query-value-on-error: -1
# Defined data sources
data-sources:
my-ds:
driver: mssql
properties:
host: localhost
port: 1433
user: sa
password: Password
database: master

Traefik SSL proxy returning 404

I want to use Traefik as a reverse proxy inside swarm and pass SSL traffic to a specific container (wordpress:latest). The SSL certificate seems to work ok, but I am getting 404 page not found on https://fakepage.com (I'm using placeholder domain to avoid exposing private information).
Traffic through http works normally.
My docker-compose configuration for the docker stack I'm using.
version: "3.6"
services:
traefik:
image: traefik
ports:
- 80:80
- 443:443
- 8080:8080
deploy:
replicas: 1
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
placement:
constraints: [node.role == worker]
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- traefik_data:/tmp
command:
- "--docker"
- "--docker.domain=fakepage.com"
- "--logLevel=DEBUG"
- "--api"
- "--api.statistics"
- "--entryPoints=Name:http Address::80"
- "--entryPoints=Name:https Address::443 TLS"
- "--docker.endpoint=unix:///var/run/docker.sock"
- "--acme=true"
- "--acme.entrypoint=https"
- "--acme.httpchallenge"
- "--acme.httpchallenge.entrypoint=http"
- "--acme.domains=fakepage.com"
- "--acme.email=admin#fakepage.com"
- "--acme.storage=/tmp/acme.json"
db:
image: "mysql:5.7"
environment:
MYSQL_ROOT_PASSWORD: example
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
MYSQL_DATABASE: wordpress
deploy:
replicas: 1
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
placement:
constraints: [node.role == worker]
volumes:
- db_data:/var/lib/mysql
wordpress:
depends_on:
- db
image: wordpress:latest
deploy:
replicas: 1
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
placement:
constraints: [node.role == worker]
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
volumes:
- wp_data:/var/www/html
labels:
traefik.enable: "true"
traefik.frontend.rule: "Host:fakepage.com"
volumes:
db_data:
wp_data:
traefik_data:
The problem was that I was missing a segment flag under container.
traefik.frontend.entryPoints: "http,https"