I followed this link to create a dockerized proxy server for my registry.
I ran these commands:
htpasswd -c ~/my-registry.htpasswd USERNAME
mkdir ~/certs
docker run --rm -e COMMON_NAME=hostName -e KEY_NAME=my-registry -v ~/certs:/certs centurylink/openssl
docker run -d --expose=5000 --name=registry registry:2
docker run -d -p 8080:8080 -e PUBLIC_IP_ADDR=hostName --link registry:registry -v ~/certs/my-registry.crt:/etc/ssl/certs/docker-registry -v ~/certs/my-registry.key:/etc/ssl/private/docker-registry -v ~/my-registry.htpasswd:/etc/nginx/docker-registry.htpasswd centurylinklabs/nginx-ssl-proxy
I then opened the https://hostName:8080/v2/ in my browser, exported the certificate, and installed the crt file on my windows client, and also saved a copy to C:\ProgramData\Docker\certs.d\hostName8080\ca.crt
The login to registry is completely successful, but when I run
docker push hostName:8080/base
The following response is returned:
90c0b5c927ee: Pushing [==================================================>] 316.4kB
48a81194835d: Pushing [==================================================>] 2.048kB
024e3f43ffc2: Pushing [==================================================>] 8.382MB/8.382MB
error parsing HTTP 400 response body: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>400 The plain HTTP request was sent to HTTPS port</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>The plain HTTP request was sent to HTTPS port</center>\r\n<hr><center>nginx/1.7.9</center>\r\n</body>\r\n</html>\r\n"
And the image isn't uploaded.
This is the docker logs of the push:
"GET /v2/ HTTP/1.1" 401 194 "-" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.9.60-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \x5C(windows\x5C))" "-"
"POST /v2/base/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.9.60-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \x5C(windows\x5C))" "-"
"POST /v2/base/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.9.60-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \x5C(windows\x5C))" "-"
"POST /v2/base/blobs/uploads/ HTTP/1.1" 202 0 "-" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.9.60-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \x5C(windows\x5C))" "-"
"PATCH /v2/base/blobs/uploads/65e201c4-66b9-4723-a074-7fdcc8535665?_state=H4ygF8XuJkl24AWQeN19_7fgpAiyDeNlWnwbmPdZ8Tl7Ik5hbWUiOiJiYXNlIiwiVVVJRCI6IjY1ZTIwMWM0LTY2YjktNDcyMy1hMDc0LTdmZGNjODUzNTY2NSIsIk9mZnNldCI6MCwiU3RhcnRlZEF0IjoiMjAxOC0wMS0yMVQwNTo1MToxNy4yMzg0OTIwMjZaIn0%3D HTTP/1.1" 400 270 "-" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.9.60-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \x5C(windows\x5C))" "-"
"PATCH /v2/base/blobs/uploads/a431532f-a2dc-4ac4-aea3-67b62594a57e?_state=D2FM5ALYWtMF1xotyibSEmHyAH6pdJJXWXZFpqWR2hl7Ik5hbWUiOiJiYXNlIiwiVVVJRCI6ImE0MzE1MzJmLWEyZGMtNGFjNC1hZWEzLTY3YjYyNTk0YTU3ZSIsIk9mZnNldCI6MCwiU3RhcnRlZEF0IjoiMjAxOC0wMS0yMVQwNTo1MToxNy4yMjk5ODQwNDNaIn0%3D HTTP/1.1" 400 270 "-" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.9.60-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \x5C(windows\x5C))" "-"
"PATCH /v2/base/blobs/uploads/7868c561-5524-4c32-8d2a-c2121e54fb00?_state=i4KyRqAo_LI3S6ED7j9sC7EQbLwAWpXLnWXlQuPf2yR7Ik5hbWUiOiJiYXNlIiwiVVVJRCI6Ijc4NjhjNTYxLTU1MjQtNGMzMi04ZDJhLWMyMTIxZTU0ZmIwMCIsIk9mZnNldCI6MCwiU3RhcnRlZEF0IjoiMjAxOC0wMS0yMVQwNTo1MToxNy4yMjk0ODIyMjZaIn0%3D HTTP/1.1" 400 270 "-" "docker/17.12.0-ce go/go1.9.2 git-commit/c97c6d6 kernel/4.9.60-linuxkit-aufs os/linux arch/amd64 UpstreamClient(Docker-Client/17.12.0-ce \x5C(windows\x5C))" "-"
Related
I would like to find IP addresses and user agents that have made more than 10 server requests per second. Help me, please. How can I do this? Maybe with awk, grep?
Here's a typical lines from my Access Log:
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-korpuse-pechki-honda-cr-v-200-pr-1019 HTTP/1.0" 200 387654 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-salone-honda-cr-v-2002-2006-pr-1018 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-naruzhnogo-vozduha-defekt-honda-pr-12550 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/klapan-otopitelya-benzin-honda-cr-v-2002-2006-797-pr-1006 HTTP/1.0" 200 387449 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/kronshtejn-radiatora-kondicionera-pravyj-honda-cr-pr-1008 HTTP/1.0" 200 387996 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/provodka-pechki-honda-cr-v-2-2002-2006-pr-12228 HTTP/1.0" 200 386676 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-korpuse-pechki-honda-cr-v-200-pr-1019 HTTP/1.0" 200 387654 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-salone-honda-cr-v-2002-2006-pr-1018 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-naruzhnogo-vozduha-defekt-honda-pr-12550 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/klapan-otopitelya-benzin-honda-cr-v-2002-2006-797-pr-1006 HTTP/1.0" 200 387449 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/kronshtejn-radiatora-kondicionera-pravyj-honda-cr-pr-1008 HTTP/1.0" 200 387996 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/provodka-pechki-honda-cr-v-2-2002-2006-pr-12228 HTTP/1.0" 200 386676 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:51 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/rozshiryuvalnij-klapan-kondicionera-honda-cr-v-2-2-pr-11867 HTTP/1.0" 200 387227 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:55 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/servoprivod-zaslonki-pechki-pravyj1-honda-cr-v-200-pr-1001 HTTP/1.0" 200 387750 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:24:41 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970s9ab812m1-honda-cr-v-2-2002-2006-pr-12578 HTTP/1.0" 200 386162 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:24:45 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970s9ac812m1-honda-cr-v-2-2002-200-pr-12577 HTTP/1.0" 200 386159 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:24:53 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970scag911m1-honda-cr-v-2-2002-2006-pr-12575 HTTP/1.0" 200 386141 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:25:00 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-levyj-77940s9an810-honda-cr-pr-12557 HTTP/1.0" 200 386548 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
95.216.137.119 - - [28/Jan/2023:03:25:06 +0200] "POST /index.php?route=api/login HTTP/1.0" 200 190 "-" "-"
95.216.137.119 - - [28/Jan/2023:03:25:07 +0200] "GET /index.php?route=api/oneboxsync/getOrderValue/&token=95ghttSKDUuykyhc4fY4tcDCuAmpZxrQ&api_token= HTTP/1.0" 200 216719 "-" "-"
176.9.50.244 - - [28/Jan/2023:03:25:09 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-levyj-honda-cr-v-2002-2006-pr-992 HTTP/1.0" 200 386813 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:25:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-pravyj-77930s9an810-honda-cr-pr-12574 HTTP/1.0" 200 386777 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
For finding info about UA's I'm running this: (Gives me the amount of hits each unique UA has)
awk -F\" '{print $6}' /www/logs/www.example.com-access.log | sort | uniq -c | sort -fr
What can I do differently to make these commands show me only IPs, from which was made , more than 10 requests per second?
If there are some IP from which was made 10 and more requests to the server, I want to find it in my log.
https://prnt.sc/1zKKp6tXx0UZ - this is 10 request per second from the same IP (176.9.50.244), so it mast be catched with the command and print to screen
In this case (corresponding to the new set of input data) the output must be like this - must be printed just one IP
176.9.50.244
When counting the number of reqs/sec, it must count all lines with the same timestamp and the same IP. Lets forget about UA, and try to focus on IP.
I use UA is for additional information only.
There are many ways to do this. One is below. Please mark as correct or add comment and I will address this further.
Mac_3.2.57$cat input.txt | awk '{print "IP=" $1 " time=" $4,$5}' | sort | uniq -c | awk '{if($1>10){print $0}}'
12 IP=176.9.50.244 time=[28/Jan/2023:03:26:14 +0200]
Mac_3.2.57$cat input.txt
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-korpuse-pechki-honda-cr-v-200-pr-1019 HTTP/1.0" 200 387654 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-salone-honda-cr-v-2002-2006-pr-1018 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-naruzhnogo-vozduha-defekt-honda-pr-12550 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/klapan-otopitelya-benzin-honda-cr-v-2002-2006-797-pr-1006 HTTP/1.0" 200 387449 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/kronshtejn-radiatora-kondicionera-pravyj-honda-cr-pr-1008 HTTP/1.0" 200 387996 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/provodka-pechki-honda-cr-v-2-2002-2006-pr-12228 HTTP/1.0" 200 386676 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-korpuse-pechki-honda-cr-v-200-pr-1019 HTTP/1.0" 200 387654 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-salone-honda-cr-v-2002-2006-pr-1018 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-naruzhnogo-vozduha-defekt-honda-pr-12550 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/klapan-otopitelya-benzin-honda-cr-v-2002-2006-797-pr-1006 HTTP/1.0" 200 387449 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/kronshtejn-radiatora-kondicionera-pravyj-honda-cr-pr-1008 HTTP/1.0" 200 387996 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/provodka-pechki-honda-cr-v-2-2002-2006-pr-12228 HTTP/1.0" 200 386676 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:51 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/rozshiryuvalnij-klapan-kondicionera-honda-cr-v-2-2-pr-11867 HTTP/1.0" 200 387227 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:55 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/servoprivod-zaslonki-pechki-pravyj1-honda-cr-v-200-pr-1001 HTTP/1.0" 200 387750 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:24:41 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970s9ab812m1-honda-cr-v-2-2002-2006-pr-12578 HTTP/1.0" 200 386162 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:24:45 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970s9ac812m1-honda-cr-v-2-2002-200-pr-12577 HTTP/1.0" 200 386159 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:24:53 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970scag911m1-honda-cr-v-2-2002-2006-pr-12575 HTTP/1.0" 200 386141 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:25:00 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-levyj-77940s9an810-honda-cr-pr-12557 HTTP/1.0" 200 386548 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
95.216.137.119 - - [28/Jan/2023:03:25:06 +0200] "POST /index.php?route=api/login HTTP/1.0" 200 190 "-" "-"
95.216.137.119 - - [28/Jan/2023:03:25:07 +0200] "GET /index.php?route=api/oneboxsync/getOrderValue/&token=95ghttSKDUuykyhc4fY4tcDCuAmpZxrQ&api_token= HTTP/1.0" 200 216719 "-" "-"
176.9.50.244 - - [28/Jan/2023:03:25:09 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-levyj-honda-cr-v-2002-2006-pr-992 HTTP/1.0" 200 386813 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:25:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-pravyj-77930s9an810-honda-cr-pr-12574 HTTP/1.0" 200 386777 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
Mac_3.2.57$
One awk idea:
awk -v limit=10 ' # set awk variable "limit" to number of matches per second
{ ip_ts[$1 FS $4]++ } # keep count of how many times we see this ip/timestamp combo
END { for (i in ip_ts) # loop through all ip/timestamp indices and ...
if (ip_ts[i] >= limit) { # if the count >= "limit" then ...
split(i,a) # split the ip/timestamp pair (on FS) and ...
print a[1] # print the ip
}
}
' access.log
Adding some variation to the input file:
$ cat access.log
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-korpuse-pechki-honda-cr-v-200-pr-1019 HTTP/1.0" 200 387654 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-salone-honda-cr-v-2002-2006-pr-1018 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-naruzhnogo-vozduha-defekt-honda-pr-12550 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/klapan-otopitelya-benzin-honda-cr-v-2002-2006-797-pr-1006 HTTP/1.0" 200 387449 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/kronshtejn-radiatora-kondicionera-pravyj-honda-cr-pr-1008 HTTP/1.0" 200 387996 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/provodka-pechki-honda-cr-v-2-2002-2006-pr-12228 HTTP/1.0" 200 386676 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-korpuse-pechki-honda-cr-v-200-pr-1019 HTTP/1.0" 200 387654 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-v-salone-honda-cr-v-2002-2006-pr-1018 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/datchik-temperatury-naruzhnogo-vozduha-defekt-honda-pr-12550 HTTP/1.0" 200 387484 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/klapan-otopitelya-benzin-honda-cr-v-2002-2006-797-pr-1006 HTTP/1.0" 200 387449 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/kronshtejn-radiatora-kondicionera-pravyj-honda-cr-pr-1008 HTTP/1.0" 200 387996 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:26:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/provodka-pechki-honda-cr-v-2-2002-2006-pr-12228 HTTP/1.0" 200 386676 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
223.1.37.179 - - [28/Jan/2023:03:26:51 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/rozshiryuvalnij-klapan-kondicionera-honda-cr-v-2-2-pr-11867 HTTP/1.0" 200 387227 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
223.1.37.179 - - [28/Jan/2023:03:26:51 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-obogreva-i-klimata-54/servoprivod-zaslonki-pechki-pravyj1-honda-cr-v-200-pr-1001 HTTP/1.0" 200 387750 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
223.1.37.179 - - [28/Jan/2023:03:26:51 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970s9ab812m1-honda-cr-v-2-2002-2006-pr-12578 HTTP/1.0" 200 386162 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
223.1.37.179 - - [28/Jan/2023:03:26:51 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970s9ac812m1-honda-cr-v-2-2002-200-pr-12577 HTTP/1.0" 200 386159 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:34:53 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-77970scag911m1-honda-cr-v-2-2002-2006-pr-12575 HTTP/1.0" 200 386141 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:34.53 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-levyj-77940s9an810-honda-cr-pr-12557 HTTP/1.0" 200 386548 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
95.216.137.119 - - [28/Jan/2023:03:25:06 +0200] "POST /index.php?route=api/login HTTP/1.0" 200 190 "-" "-"
95.216.137.119 - - [28/Jan/2023:03:25:07 +0200] "GET /index.php?route=api/oneboxsync/getOrderValue/&token=95ghttSKDUuykyhc4fY4tcDCuAmpZxrQ&api_token= HTTP/1.0" 200 216719 "-" "-"
176.9.50.244 - - [28/Jan/2023:03:25:09 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-levyj-honda-cr-v-2002-2006-pr-992 HTTP/1.0" 200 386813 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
176.9.50.244 - - [28/Jan/2023:03:25:14 +0200] "GET /honda-1/honda-cr-v-2-2002-2006-12/sistema-bezopasnosti-52/datchik-udara-perednij-pravyj-77930s9an810-honda-cr-pr-12574 HTTP/1.0" 200 386777 "-" "Mozilla/5.0 (compatible; MegaIndex.ru/2.0; +http://megaindex.com/crawler)"
For limit=10 this generates:
176.9.50.244
For limit=4 this generates:
176.9.50.244
223.1.37.179
For limit=2 this generates:
176.9.50.244
176.9.50.244
223.1.37.179
NOTES:
for the 3rd set of output 176.9.50.244 shows up twice; it's not clear from OP's description if we should only show an ip once in the final output, or once for each unique set of timestamps that match the criteria
order of the output lines is not guaranteed; if output order is important OP can pipe the results to sort or add code to have awk sort the data before printing to stdout
I have GitHub Runner and docker registry on my server and I'm trying to set up ci/cd for some applications.
Directly pushing to the container name with port works smoothly
docker pull alpine
docker tag alpine:latest docker-registry:5000/alpine:latest
docker login docker-registry:5000
Login Succeeded
docker push docker-registry:5000/alpine:latest
The push refers to repository [docker-registry:5000/alpine]
ded7a220bb05: Pushed
But I get an problem when I use the domain
docker pull alpine
docker tag alpine:latest registry.mydomain.com/alpine:latest
docker login registry.mydomain.com
Login Succeeded
docker push registry.mydomain.com/alpine:latest
The push refers to repository [registry.mydomain.com/alpine]
ded7a220bb05: Retrying in 1 second
received unexpected HTTP status: 200 OK
My docker containers
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
eab92ef5610b registry:2 "/entrypoint.sh /etc…" 22 seconds ago Up 21 seconds 5000/tcp docker-registry
bdd4f0886c19 nginx:1.20.2 "/docker-entrypoint.…" 3 months ago Up About an hour 0.0.0.0:80->80/tcp, :::80->80/tcp nginx-ingress
5e7e8e1b591e myoung34/github-runner:2.299.1-ubuntu-jammy "/entrypoint.sh ./bi…" 37 hours ago Up 27 minutes github-runner
My Nginx config
upstream registry {
server docker-registry:5000;
}
map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
'' 'registry/2.0';
}
server {
listen 443;
server_name registry.mydomain.com;
client_max_body_size 0;
chunked_transfer_encoding on;
location /v2/ {
if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
return 404;
}
auth_basic "Registry Realm";
auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;
add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;
proxy_pass http://registry;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; # default => $scheme | cloudflare => https
proxy_read_timeout 900;
}
}
Docker registry logs
time="2022-12-01T13:05:21.576274727Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.16.15 instance.id=9fae9ec3-fe4f-443e-b6b3-d472db92412a service=registry version="v2.8.1+unknown"
time="2022-12-01T13:05:21.576362865Z" level=info msg="redis not configured" go.version=go1.16.15 instance.id=9fae9ec3-fe4f-443e-b6b3-d472db92412a service=registry version="v2.8.1+unknown"
time="2022-12-01T13:05:21.576419048Z" level=info msg="Starting upload purge in 9m0s" go.version=go1.16.15 instance.id=9fae9ec3-fe4f-443e-b6b3-d472db92412a service=registry version="v2.8.1+unknown"
time="2022-12-01T13:05:21.588466586Z" level=info msg="using inmemory blob descriptor cache" go.version=go1.16.15 instance.id=9fae9ec3-fe4f-443e-b6b3-d472db92412a service=registry version="v2.8.1+unknown"
time="2022-12-01T13:05:21.588770043Z" level=info msg="listening on [::]:5000" go.version=go1.16.15 instance.id=9fae9ec3-fe4f-443e-b6b3-d472db92412a service=registry version="v2.8.1+unknown"
172.18.0.1 - - [01/Dec/2022:13:05:28 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:28.906573279Z" level=warning msg="error authorizing context: basic authentication challenge for realm "Registry Realm": invalid authorization credential" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=4d19514d-336c-4c1c-ad8a-91a8a4be1e0c http.request.method=GET http.request.remoteaddr="172.18.0.1:42346" http.request.uri="/v2/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))"
172.18.0.1 - - [01/Dec/2022:13:05:28 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:28.91635475Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=6f0cd72e-52dd-4408-812c-c770c1daeb4c http.request.method=GET http.request.remoteaddr="172.18.0.1:42348" http.request.uri="/v2/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))"
time="2022-12-01T13:05:28.916428075Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=6f0cd72e-52dd-4408-812c-c770c1daeb4c http.request.method=GET http.request.remoteaddr="172.18.0.1:42348" http.request.uri="/v2/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=8.872969ms http.response.status=200 http.response.written=2
172.18.0.1 - - [01/Dec/2022:13:05:47 +0000] "GET /v2/ HTTP/1.1" 401 87 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:47.955166053Z" level=warning msg="error authorizing context: basic authentication challenge for realm "Registry Realm": invalid authorization credential" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=060b8ad8-027d-4e1f-aae8-853e694aa071 http.request.method=GET http.request.remoteaddr="172.18.0.1:42352" http.request.uri="/v2/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))"
time="2022-12-01T13:05:47.961400261Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=23ab1f44-c687-41a4-854c-88d5223f3c19 http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42354" http.request.uri="/v2/alpine/blobs/sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.digest="sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" vars.name=alpine
time="2022-12-01T13:05:47.961616963Z" level=error msg="response completed with error" auth.user.name="docker_user" err.code="blob unknown" err.detail=sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 err.message="blob unknown to registry" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=23ab1f44-c687-41a4-854c-88d5223f3c19 http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42354" http.request.uri="/v2/alpine/blobs/sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=3.888659ms http.response.status=404 http.response.written=157 vars.digest="sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" vars.name=alpine
172.18.0.1 - - [01/Dec/2022:13:05:47 +0000] "HEAD /v2/alpine/blobs/sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 HTTP/1.1" 404 157 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:48.013676521Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=76e692cf-3fec-4739-ac68-dd7e785bc3c0 http.request.method=POST http.request.remoteaddr="172.18.0.1:42356" http.request.uri="/v2/alpine/blobs/uploads/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.name=alpine
time="2022-12-01T13:05:48.026046059Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=76e692cf-3fec-4739-ac68-dd7e785bc3c0 http.request.method=POST http.request.remoteaddr="172.18.0.1:42356" http.request.uri="/v2/alpine/blobs/uploads/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.duration=16.091457ms http.response.status=202 http.response.written=0
172.18.0.1 - - [01/Dec/2022:13:05:48 +0000] "POST /v2/alpine/blobs/uploads/ HTTP/1.1" 202 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:48.037049651Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=9d35eb65-a80e-43c7-bcad-42bcd0723ad1 http.request.method=PATCH http.request.remoteaddr="172.18.0.1:42358" http.request.uri="/v2/alpine/blobs/uploads/cb21fc83-6e19-4c78-b5c0-badb0ff8fe04?_state=TRS7R1oNENXqtiEnnSLbJD3l9O9Pq9tEtZudktNkQQF7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiY2IyMWZjODMtNmUxOS00Yzc4LWI1YzAtYmFkYjBmZjhmZTA0IiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ4LjAxMzc0MDcxN1oifQ%3D%3D" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.name=alpine vars.uuid=cb21fc83-6e19-4c78-b5c0-badb0ff8fe04
time="2022-12-01T13:05:48.956206497Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=9d35eb65-a80e-43c7-bcad-42bcd0723ad1 http.request.method=PATCH http.request.remoteaddr="172.18.0.1:42358" http.request.uri="/v2/alpine/blobs/uploads/cb21fc83-6e19-4c78-b5c0-badb0ff8fe04?_state=TRS7R1oNENXqtiEnnSLbJD3l9O9Pq9tEtZudktNkQQF7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiY2IyMWZjODMtNmUxOS00Yzc4LWI1YzAtYmFkYjBmZjhmZTA0IiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ4LjAxMzc0MDcxN1oifQ%3D%3D" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.duration=929.13763ms http.response.status=202 http.response.written=0
172.18.0.1 - - [01/Dec/2022:13:05:48 +0000] "PATCH /v2/alpine/blobs/uploads/cb21fc83-6e19-4c78-b5c0-badb0ff8fe04?_state=TRS7R1oNENXqtiEnnSLbJD3l9O9Pq9tEtZudktNkQQF7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiY2IyMWZjODMtNmUxOS00Yzc4LWI1YzAtYmFkYjBmZjhmZTA0IiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ4LjAxMzc0MDcxN1oifQ%3D%3D HTTP/1.1" 202 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:48.960962747Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=b4955223-ae69-4d0f-8643-b70148b764fd http.request.method=PUT http.request.remoteaddr="172.18.0.1:42360" http.request.uri="/v2/alpine/blobs/uploads/cb21fc83-6e19-4c78-b5c0-badb0ff8fe04?_state=NVqJ2JVvha-PdWnWeVonnGI1qxrWbyJiE3ECtgBK_v17Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiY2IyMWZjODMtNmUxOS00Yzc4LWI1YzAtYmFkYjBmZjhmZTA0IiwiT2Zmc2V0IjozMzcwNzA2LCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ4WiJ9&digest=sha256%3Ac158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.name=alpine vars.uuid=cb21fc83-6e19-4c78-b5c0-badb0ff8fe04
time="2022-12-01T13:05:48.975845667Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=b4955223-ae69-4d0f-8643-b70148b764fd http.request.method=PUT http.request.remoteaddr="172.18.0.1:42360" http.request.uri="/v2/alpine/blobs/uploads/cb21fc83-6e19-4c78-b5c0-badb0ff8fe04?_state=NVqJ2JVvha-PdWnWeVonnGI1qxrWbyJiE3ECtgBK_v17Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiY2IyMWZjODMtNmUxOS00Yzc4LWI1YzAtYmFkYjBmZjhmZTA0IiwiT2Zmc2V0IjozMzcwNzA2LCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ4WiJ9&digest=sha256%3Ac158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.duration=18.779158ms http.response.status=201 http.response.written=0
172.18.0.1 - - [01/Dec/2022:13:05:48 +0000] "PUT /v2/alpine/blobs/uploads/cb21fc83-6e19-4c78-b5c0-badb0ff8fe04?_state=NVqJ2JVvha-PdWnWeVonnGI1qxrWbyJiE3ECtgBK_v17Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiY2IyMWZjODMtNmUxOS00Yzc4LWI1YzAtYmFkYjBmZjhmZTA0IiwiT2Zmc2V0IjozMzcwNzA2LCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ4WiJ9&digest=sha256%3Ac158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 HTTP/1.1" 201 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:48.987290535Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=53c874f2-6f55-4cde-b5ca-596dda13583a http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42362" http.request.uri="/v2/alpine/blobs/sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.digest="sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" vars.name=alpine
time="2022-12-01T13:05:48.987671994Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=53c874f2-6f55-4cde-b5ca-596dda13583a http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42362" http.request.uri="/v2/alpine/blobs/sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.contenttype="application/octet-stream" http.response.duration=10.691772ms http.response.status=200 http.response.written=0
172.18.0.1 - - [01/Dec/2022:13:05:48 +0000] "HEAD /v2/alpine/blobs/sha256:c158987b05517b6f2c5913f3acef1f2182a32345a304fe357e3ace5fadcad715 HTTP/1.1" 200 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:48.99996587Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=867604cb-09cd-43f4-91b0-52ead32d664e http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42364" http.request.uri="/v2/alpine/blobs/sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.digest="sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" vars.name=alpine
time="2022-12-01T13:05:49.000171622Z" level=error msg="response completed with error" auth.user.name="docker_user" err.code="blob unknown" err.detail=sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da err.message="blob unknown to registry" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=867604cb-09cd-43f4-91b0-52ead32d664e http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42364" http.request.uri="/v2/alpine/blobs/sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.contenttype="application/json; charset=utf-8" http.response.duration=4.875849ms http.response.status=404 http.response.written=157 vars.digest="sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" vars.name=alpine
172.18.0.1 - - [01/Dec/2022:13:05:48 +0000] "HEAD /v2/alpine/blobs/sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da HTTP/1.1" 404 157 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:49.009848946Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=7e29e9ea-0864-44e2-bc75-732d246af8a9 http.request.method=POST http.request.remoteaddr="172.18.0.1:42366" http.request.uri="/v2/alpine/blobs/uploads/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.name=alpine
172.18.0.1 - - [01/Dec/2022:13:05:49 +0000] "POST /v2/alpine/blobs/uploads/ HTTP/1.1" 202 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:49.022631954Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=7e29e9ea-0864-44e2-bc75-732d246af8a9 http.request.method=POST http.request.remoteaddr="172.18.0.1:42366" http.request.uri="/v2/alpine/blobs/uploads/" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.duration=21.407075ms http.response.status=202 http.response.written=0
time="2022-12-01T13:05:49.032911791Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=abab2c41-40ee-4c76-8456-3b30b62061af http.request.method=PATCH http.request.remoteaddr="172.18.0.1:42370" http.request.uri="/v2/alpine/blobs/uploads/32b60b00-82d6-4003-a470-7502f715be61?_state=E8EC15n0xy4YiaxJAynDeAl0SaoKMX0tvtbqakyT13R7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiMzJiNjBiMDAtODJkNi00MDAzLWE0NzAtNzUwMmY3MTViZTYxIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ5LjAwOTk2MjU3WiJ9" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.name=alpine vars.uuid=32b60b00-82d6-4003-a470-7502f715be61
172.18.0.1 - - [01/Dec/2022:13:05:49 +0000] "PATCH /v2/alpine/blobs/uploads/32b60b00-82d6-4003-a470-7502f715be61?_state=E8EC15n0xy4YiaxJAynDeAl0SaoKMX0tvtbqakyT13R7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiMzJiNjBiMDAtODJkNi00MDAzLWE0NzAtNzUwMmY3MTViZTYxIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ5LjAwOTk2MjU3WiJ9 HTTP/1.1" 202 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:49.046509563Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=abab2c41-40ee-4c76-8456-3b30b62061af http.request.method=PATCH http.request.remoteaddr="172.18.0.1:42370" http.request.uri="/v2/alpine/blobs/uploads/32b60b00-82d6-4003-a470-7502f715be61?_state=E8EC15n0xy4YiaxJAynDeAl0SaoKMX0tvtbqakyT13R7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiMzJiNjBiMDAtODJkNi00MDAzLWE0NzAtNzUwMmY3MTViZTYxIiwiT2Zmc2V0IjowLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ5LjAwOTk2MjU3WiJ9" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.duration=22.857284ms http.response.status=202 http.response.written=0
time="2022-12-01T13:05:49.052387956Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=8d17d425-4b89-4597-9314-1f2c35e1afdb http.request.method=PUT http.request.remoteaddr="172.18.0.1:42372" http.request.uri="/v2/alpine/blobs/uploads/32b60b00-82d6-4003-a470-7502f715be61?_state=uN8teJhzBMwlrLnstzPV492DSwZgwtADfHJWpqUsGpx7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiMzJiNjBiMDAtODJkNi00MDAzLWE0NzAtNzUwMmY3MTViZTYxIiwiT2Zmc2V0IjoxNDcyLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ5WiJ9&digest=sha256%3A49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.name=alpine vars.uuid=32b60b00-82d6-4003-a470-7502f715be61
time="2022-12-01T13:05:49.067768248Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=8d17d425-4b89-4597-9314-1f2c35e1afdb http.request.method=PUT http.request.remoteaddr="172.18.0.1:42372" http.request.uri="/v2/alpine/blobs/uploads/32b60b00-82d6-4003-a470-7502f715be61?_state=uN8teJhzBMwlrLnstzPV492DSwZgwtADfHJWpqUsGpx7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiMzJiNjBiMDAtODJkNi00MDAzLWE0NzAtNzUwMmY3MTViZTYxIiwiT2Zmc2V0IjoxNDcyLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ5WiJ9&digest=sha256%3A49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.duration=20.176783ms http.response.status=201 http.response.written=0
172.18.0.1 - - [01/Dec/2022:13:05:49 +0000] "PUT /v2/alpine/blobs/uploads/32b60b00-82d6-4003-a470-7502f715be61?_state=uN8teJhzBMwlrLnstzPV492DSwZgwtADfHJWpqUsGpx7Ik5hbWUiOiJhbHBpbmUiLCJVVUlEIjoiMzJiNjBiMDAtODJkNi00MDAzLWE0NzAtNzUwMmY3MTViZTYxIiwiT2Zmc2V0IjoxNDcyLCJTdGFydGVkQXQiOiIyMDIyLTEyLTAxVDEzOjA1OjQ5WiJ9&digest=sha256%3A49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da HTTP/1.1" 201 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:49.073531198Z" level=info msg="authorized request" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=d77e1f84-f55d-4917-bc8c-aceb4606c23e http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42374" http.request.uri="/v2/alpine/blobs/sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.digest="sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" vars.name=alpine
172.18.0.1 - - [01/Dec/2022:13:05:49 +0000] "HEAD /v2/alpine/blobs/sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da HTTP/1.1" 200 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
time="2022-12-01T13:05:49.073748384Z" level=info msg="response completed" go.version=go1.16.15 http.request.host="docker-registry:5000" http.request.id=d77e1f84-f55d-4917-bc8c-aceb4606c23e http.request.method=HEAD http.request.remoteaddr="172.18.0.1:42374" http.request.uri="/v2/alpine/blobs/sha256:49176f190c7e9cdb51ac85ab6c6d5e4512352218190cd69b08e6fd803ffbf3da" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.contenttype="application/octet-stream" http.response.duration=4.891873ms http.response.status=200 http.response.written=0
time="2022-12-01T13:05:49.081141753Z" level=info msg="authorized request" go.version=go1.16.15 http.request.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.request.host="docker-registry:5000" http.request.id=b2329d48-38dd-471c-8691-6b08c1eb7ec5 http.request.method=PUT http.request.remoteaddr="172.18.0.1:42376" http.request.uri="/v2/alpine/manifests/latest" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" vars.name=alpine vars.reference=latest
time="2022-12-01T13:05:49.102368205Z" level=info msg="response completed" go.version=go1.16.15 http.request.contenttype="application/vnd.docker.distribution.manifest.v2+json" http.request.host="docker-registry:5000" http.request.id=b2329d48-38dd-471c-8691-6b08c1eb7ec5 http.request.method=PUT http.request.remoteaddr="172.18.0.1:42376" http.request.uri="/v2/alpine/manifests/latest" http.request.useragent="docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \(linux\))" http.response.duration=27.598063ms http.response.status=201 http.response.written=0
172.18.0.1 - - [01/Dec/2022:13:05:49 +0000] "PUT /v2/alpine/manifests/latest HTTP/1.1" 201 0 "" "docker/20.10.17 go/go1.17.11 git-commit/a89b842 kernel/5.15.0-46-generic os/linux arch/amd64 UpstreamClient(Docker-Client/20.10.17 \\(linux\\))"
Cloudflare is used for SSL. It automatically redirects to 443
From the comments:
Is your proxy configuration dropping the /v2 from the requests to the backend?
Yes, it does
That's an issue. The /v2 is part of the API. I believe the following will include the /v2 in the proxied requests:
proxy_pass http://registry/v2/;
I solved the problem
One of the main problems was that I was only listening to requests from port 443. This was wrong because my Cloudflare's encryption mode was "Flexible". In this mode, all connections between Cloudflare and your origin are made through HTTP
I changed 443 to 80, location, and the value of "proxy_set_header X-Forwarded-Proto" with "https", problem solved ))
server {
listen 80;
....
....
location / {
....
proxy_set_header X-Forwarded-Proto https;
....
}
}
I have the code below to initialise a setup of Selenium + Chrome driver + Browsermob Proxy to login to a certain website.
If I just use the setup above, the login fails with HTTP 403. However, if I had another Proxy (Selenium + Chrome driver + Browsermob Proxy + e.g. Fiddler Everywhere, see lines after "to pass requests from proxy server to Fiddler Everywhere") and force the requests from Browsermob Proxy to be proxied by Fiddler Everywhere, then the login works well.
Can anyone provide me a hint on how to debug and/or clear this problem?
I have investigated the headers between all legs:
"Via: 1.1 browsermobproxy" is being added by Browsermob but I replayed a login request after removing "Via" header from Browsermob, and did not have any effect.
BrowserMob is removing "Connection:keep-alive". Could this be the cause (or hint at the cause) ?
I also checked:
The body of the request is not changed across the full chain.
HTTP protocol version does not change across the full chain (HTTP/1.1).
Here is a detailed log of the request without Fiddler:
POST /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784 HTTP/1.1
Host: www.example.com
Connection: keep-alive
Content-Length: 147
AccountType: Real
X-STS-ClientTime: 2020-08-27T18:10:51
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36
X-STS-AppDomain: https://www.example.com
Content-Type: application/json;charset=UTF-8
Accept: application/json, text/plain, */*
X-STS-GatewayAppId: 90631448-9A01-4860-9FA5-B4EBCDE5EA1D
ApplicationIdentifier: Rexample
ApplicationVersion: 268.0.1
X-DEVICE-ID: 77f281a4-7914-4417-b144-4811998c3e5b
Origin: https://www.example.com
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://www.example.com/login
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: __cfduid=d722b1913f887e5df9b0c740ccef4744a1598548223; exampleLocale=en-gb; TS01047baf=01f1b32d7e1354b9f541c1bab1a81960be72277a638bce5a4865b5b1d5b2e0a9cfddddbcba0d21185b68a7d599391e972d5f065031; __cflb=02DiuEAg8LPSYevHEYiyx9rnANkoz3ETfpNySVtv8fQVr; TMIS2=9a74f2a90237486aab4dcb9d44dec14a0520aaf521b9e87e81f271264e25038c2919d29ee219162aa2ad6bdb68fbe78a86ce282ef510aa77456a3876e4e790d3fdbb0ca31af4d206c67f7b9b5fda93e11b80782ed9095c6a7d8766b4642a69785e7ba9d0a86cd98f3b42c2172a8821bbe6906b6fafc8178f0d742271591193; __cf_bm=50d03de4ce1301af67fa03c59387373fba4d8b3e-1598548241-1800-AZ39AzpjNXxDQ+qFhgfI0OYbMauZ3vXOA1MUejwqJsfE3kOEmEVmKXtsstKg5vZZ5ie/d3yHbgzM8RzzLqqSQEVnd5Vy+l06FUm5Z90kFf54ffUqgq+ywM4EOcGhdP22aetxTezQvFieWgfScPvH0SzqDmFpvrmZJmRuLvnywgcLX14Ys9uA2tMCLWSlDesiQA==; mp_dbbd7bd9566da85f012f7ca5d8c6c944_mixpanel=%7B%22distinct_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24device_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; _gcl_au=1.1.706805090.1598548242; G_ENABLED_IDPS=google; _uetsid=5c779e7394432399f739848297b9a350; _uetvid=d421e48c4acc1f88b81573014b4fa429; _fbp=fb.1.1598548244752.1489311425; __adal_ses=*; __adal_id=c4b052ee-96b4-4643-b30e-af4af0f7460d.1598548246.1.1598548246.1598548246.2a5a85bd-5415-4275-942c-e02a8048acc2; __adal_ca=so%3Ddirect%26me%3Dnone%26ca%3Ddirect%26co%3D%28not%2520set%29%26ke%3D%28not%2520set%29; __adal_cw=1598548245533; _ga=GA1.2.436171435.1598548249; _gid=GA1.2.1493657885.1598548249; _gat=1; _gat_UA-2056847-65=1
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:183) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Received raw request: DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784 HTTP/1.1
Host: www.example.com
Connection: keep-alive
Content-Length: 147
AccountType: Real
X-STS-ClientTime: 2020-08-27T18:10:51
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36
X-STS-AppDomain: https://www.example.com
Content-Type: application/json;charset=UTF-8
Accept: application/json, text/plain, */*
X-STS-GatewayAppId: 90631448-9A01-4860-9FA5-B4EBCDE5EA1D
ApplicationIdentifier: Rexample
ApplicationVersion: 268.0.1
X-DEVICE-ID: 77f281a4-7914-4417-b144-4811998c3e5b
Origin: https://www.example.com
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://www.example.com/login
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: __cfduid=d722b1913f887e5df9b0c740ccef4744a1598548223; exampleLocale=en-gb; TS01047baf=01f1b32d7e1354b9f541c1bab1a81960be72277a638bce5a4865b5b1d5b2e0a9cfddddbcba0d21185b68a7d599391e972d5f065031; __cflb=02DiuEAg8LPSYevHEYiyx9rnANkoz3ETfpNySVtv8fQVr; TMIS2=9a74f2a90237486aab4dcb9d44dec14a0520aaf521b9e87e81f271264e25038c2919d29ee219162aa2ad6bdb68fbe78a86ce282ef510aa77456a3876e4e790d3fdbb0ca31af4d206c67f7b9b5fda93e11b80782ed9095c6a7d8766b4642a69785e7ba9d0a86cd98f3b42c2172a8821bbe6906b6fafc8178f0d742271591193; __cf_bm=50d03de4ce1301af67fa03c59387373fba4d8b3e-1598548241-1800-AZ39AzpjNXxDQ+qFhgfI0OYbMauZ3vXOA1MUejwqJsfE3kOEmEVmKXtsstKg5vZZ5ie/d3yHbgzM8RzzLqqSQEVnd5Vy+l06FUm5Z90kFf54ffUqgq+ywM4EOcGhdP22aetxTezQvFieWgfScPvH0SzqDmFpvrmZJmRuLvnywgcLX14Ys9uA2tMCLWSlDesiQA==; mp_dbbd7bd9566da85f012f7ca5d8c6c944_mixpanel=%7B%22distinct_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24device_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; _gcl_au=1.1.706805090.1598548242; G_ENABLED_IDPS=google; _uetsid=5c779e7394432399f739848297b9a350; _uetvid=d421e48c4acc1f88b81573014b4fa429; _fbp=fb.1.1598548244752.1489311425; __adal_ses=*; __adal_id=c4b052ee-96b4-4643-b30e-af4af0f7460d.1598548246.1.1598548246.1598548246.2a5a85bd-5415-4275-942c-e02a8048acc2; __adal_ca=so%3Ddirect%26me%3Dnone%26ca%3Ddirect%26co%3D%28not%2520set%29%26ke%3D%28not%2520set%29; __adal_cw=1598548245533; _ga=GA1.2.436171435.1598548249; _gid=GA1.2.1493657885.1598548249; _gat=1; _gat_UA-2056847-65=1
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:269) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Ensuring that hostAndPort are available in /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:281) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Finding ProxyToServerConnection for: www.example.com
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:331) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Reusing existing server connection: org.littleshoot.proxy.impl.ProxyToServerConnection#7c392fd2
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:1084) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Modifying request for proxy chaining
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:1088) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Stripped host from uri: /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784 yielding: /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:1093) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Modifying request headers for proxying
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ClientToProxyConnection.java:350) - (AWAITING_INITIAL) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Writing request to ProxyToServerConnection
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyToServerConnection.java:316) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Requested write of DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784 HTTP/1.1
Host: www.example.com
Content-Length: 147
AccountType: Real
X-STS-ClientTime: 2020-08-27T18:10:51
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36
X-STS-AppDomain: https://www.example.com
Content-Type: application/json;charset=UTF-8
Accept: application/json, text/plain, */*
X-STS-GatewayAppId: 90631448-9A01-4860-9FA5-B4EBCDE5EA1D
ApplicationIdentifier: Rexample
ApplicationVersion: 268.0.1
X-DEVICE-ID: 77f281a4-7914-4417-b144-4811998c3e5b
Origin: https://www.example.com
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://www.example.com/login
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: __cfduid=d722b1913f887e5df9b0c740ccef4744a1598548223; exampleLocale=en-gb; TS01047baf=01f1b32d7e1354b9f541c1bab1a81960be72277a638bce5a4865b5b1d5b2e0a9cfddddbcba0d21185b68a7d599391e972d5f065031; __cflb=02DiuEAg8LPSYevHEYiyx9rnANkoz3ETfpNySVtv8fQVr; TMIS2=9a74f2a90237486aab4dcb9d44dec14a0520aaf521b9e87e81f271264e25038c2919d29ee219162aa2ad6bdb68fbe78a86ce282ef510aa77456a3876e4e790d3fdbb0ca31af4d206c67f7b9b5fda93e11b80782ed9095c6a7d8766b4642a69785e7ba9d0a86cd98f3b42c2172a8821bbe6906b6fafc8178f0d742271591193; __cf_bm=50d03de4ce1301af67fa03c59387373fba4d8b3e-1598548241-1800-AZ39AzpjNXxDQ+qFhgfI0OYbMauZ3vXOA1MUejwqJsfE3kOEmEVmKXtsstKg5vZZ5ie/d3yHbgzM8RzzLqqSQEVnd5Vy+l06FUm5Z90kFf54ffUqgq+ywM4EOcGhdP22aetxTezQvFieWgfScPvH0SzqDmFpvrmZJmRuLvnywgcLX14Ys9uA2tMCLWSlDesiQA==; mp_dbbd7bd9566da85f012f7ca5d8c6c944_mixpanel=%7B%22distinct_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24device_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; _gcl_au=1.1.706805090.1598548242; G_ENABLED_IDPS=google; _uetsid=5c779e7394432399f739848297b9a350; _uetvid=d421e48c4acc1f88b81573014b4fa429; _fbp=fb.1.1598548244752.1489311425; __adal_ses=*; __adal_id=c4b052ee-96b4-4643-b30e-af4af0f7460d.1598548246.1.1598548246.1598548246.2a5a85bd-5415-4275-942c-e02a8048acc2; __adal_ca=so%3Ddirect%26me%3Dnone%26ca%3Ddirect%26co%3D%28not%2520set%29%26ke%3D%28not%2520set%29; __adal_cw=1598548245533; _ga=GA1.2.436171435.1598548249; _gid=GA1.2.1493657885.1598548249; _gat=1; _gat_UA-2056847-65=1
Accept-Encoding: gzip, deflate, br
Via: 1.1 browsermobproxy
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyToServerConnection.java:348) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Using existing connection to: www.example.com/104.18.19.67:443
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyConnection.java:228) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Writing: DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784 HTTP/1.1
Host: www.example.com
Content-Length: 147
AccountType: Real
X-STS-ClientTime: 2020-08-27T18:10:51
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36
X-STS-AppDomain: https://www.example.com
Content-Type: application/json;charset=UTF-8
Accept: application/json, text/plain, */*
X-STS-GatewayAppId: 90631448-9A01-4860-9FA5-B4EBCDE5EA1D
ApplicationIdentifier: Rexample
ApplicationVersion: 268.0.1
X-DEVICE-ID: 77f281a4-7914-4417-b144-4811998c3e5b
Origin: https://www.example.com
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://www.example.com/login
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: __cfduid=d722b1913f887e5df9b0c740ccef4744a1598548223; exampleLocale=en-gb; TS01047baf=01f1b32d7e1354b9f541c1bab1a81960be72277a638bce5a4865b5b1d5b2e0a9cfddddbcba0d21185b68a7d599391e972d5f065031; __cflb=02DiuEAg8LPSYevHEYiyx9rnANkoz3ETfpNySVtv8fQVr; TMIS2=9a74f2a90237486aab4dcb9d44dec14a0520aaf521b9e87e81f271264e25038c2919d29ee219162aa2ad6bdb68fbe78a86ce282ef510aa77456a3876e4e790d3fdbb0ca31af4d206c67f7b9b5fda93e11b80782ed9095c6a7d8766b4642a69785e7ba9d0a86cd98f3b42c2172a8821bbe6906b6fafc8178f0d742271591193; __cf_bm=50d03de4ce1301af67fa03c59387373fba4d8b3e-1598548241-1800-AZ39AzpjNXxDQ+qFhgfI0OYbMauZ3vXOA1MUejwqJsfE3kOEmEVmKXtsstKg5vZZ5ie/d3yHbgzM8RzzLqqSQEVnd5Vy+l06FUm5Z90kFf54ffUqgq+ywM4EOcGhdP22aetxTezQvFieWgfScPvH0SzqDmFpvrmZJmRuLvnywgcLX14Ys9uA2tMCLWSlDesiQA==; mp_dbbd7bd9566da85f012f7ca5d8c6c944_mixpanel=%7B%22distinct_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24device_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; _gcl_au=1.1.706805090.1598548242; G_ENABLED_IDPS=google; _uetsid=5c779e7394432399f739848297b9a350; _uetvid=d421e48c4acc1f88b81573014b4fa429; _fbp=fb.1.1598548244752.1489311425; __adal_ses=*; __adal_id=c4b052ee-96b4-4643-b30e-af4af0f7460d.1598548246.1.1598548246.1598548246.2a5a85bd-5415-4275-942c-e02a8048acc2; __adal_ca=so%3Ddirect%26me%3Dnone%26ca%3Ddirect%26co%3D%28not%2520set%29%26ke%3D%28not%2520set%29; __adal_cw=1598548245533; _ga=GA1.2.436171435.1598548249; _gid=GA1.2.1493657885.1598548249; _gat=1; _gat_UA-2056847-65=1
Accept-Encoding: gzip, deflate, br
Via: 1.1 browsermobproxy
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyConnection.java:237) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Wrote: DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST /api/sts/v2/oauth/auth?client_request_id=34634696-b122-40de-b144-ab812a4b3784 HTTP/1.1
Host: www.example.com
Content-Length: 147
AccountType: Real
X-STS-ClientTime: 2020-08-27T18:10:51
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36
X-STS-AppDomain: https://www.example.com
Content-Type: application/json;charset=UTF-8
Accept: application/json, text/plain, */*
X-STS-GatewayAppId: 90631448-9A01-4860-9FA5-B4EBCDE5EA1D
ApplicationIdentifier: Rexample
ApplicationVersion: 268.0.1
X-DEVICE-ID: 77f281a4-7914-4417-b144-4811998c3e5b
Origin: https://www.example.com
Sec-Fetch-Site: same-origin
Sec-Fetch-Mode: cors
Sec-Fetch-Dest: empty
Referer: https://www.example.com/login
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: __cfduid=d722b1913f887e5df9b0c740ccef4744a1598548223; exampleLocale=en-gb; TS01047baf=01f1b32d7e1354b9f541c1bab1a81960be72277a638bce5a4865b5b1d5b2e0a9cfddddbcba0d21185b68a7d599391e972d5f065031; __cflb=02DiuEAg8LPSYevHEYiyx9rnANkoz3ETfpNySVtv8fQVr; TMIS2=9a74f2a90237486aab4dcb9d44dec14a0520aaf521b9e87e81f271264e25038c2919d29ee219162aa2ad6bdb68fbe78a86ce282ef510aa77456a3876e4e790d3fdbb0ca31af4d206c67f7b9b5fda93e11b80782ed9095c6a7d8766b4642a69785e7ba9d0a86cd98f3b42c2172a8821bbe6906b6fafc8178f0d742271591193; __cf_bm=50d03de4ce1301af67fa03c59387373fba4d8b3e-1598548241-1800-AZ39AzpjNXxDQ+qFhgfI0OYbMauZ3vXOA1MUejwqJsfE3kOEmEVmKXtsstKg5vZZ5ie/d3yHbgzM8RzzLqqSQEVnd5Vy+l06FUm5Z90kFf54ffUqgq+ywM4EOcGhdP22aetxTezQvFieWgfScPvH0SzqDmFpvrmZJmRuLvnywgcLX14Ys9uA2tMCLWSlDesiQA==; mp_dbbd7bd9566da85f012f7ca5d8c6c944_mixpanel=%7B%22distinct_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24device_id%22%3A%20%2217430e66d1679d-015c94d6c30eda-31647304-fa000-17430e66d17591%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; _gcl_au=1.1.706805090.1598548242; G_ENABLED_IDPS=google; _uetsid=5c779e7394432399f739848297b9a350; _uetvid=d421e48c4acc1f88b81573014b4fa429; _fbp=fb.1.1598548244752.1489311425; __adal_ses=*; __adal_id=c4b052ee-96b4-4643-b30e-af4af0f7460d.1598548246.1.1598548246.1598548246.2a5a85bd-5415-4275-942c-e02a8048acc2; __adal_ca=so%3Ddirect%26me%3Dnone%26ca%3Ddirect%26co%3D%28not%2520set%29%26ke%3D%28not%2520set%29; __adal_cw=1598548245533; _ga=GA1.2.436171435.1598548249; _gid=GA1.2.1493657885.1598548249; _gat=1; _gat_UA-2056847-65=1
Accept-Encoding: gzip, deflate, br
Via: 1.1 browsermobproxy
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyConnection.java:111) - (AWAITING_CHUNK) [id: 0x367d8cd0, L:/[0:0:0:0:0:0:0:1]:64108 - R:/[0:0:0:0:0:0:0:1]:64132]: Reading: DefaultLastHttpContent(data: SlicedAbstractByteBuf(ridx: 0, widx: 147, cap: 147/147, unwrapped: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf(ridx: 147, widx: 147, cap: 176)), decoderResult: success)
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyToServerConnection.java:316) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Requested write of DefaultLastHttpContent(data: SlicedAbstractByteBuf(ridx: 0, widx: 147, cap: 147/147, unwrapped: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf(ridx: 147, widx: 147, cap: 176)), decoderResult: success)
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyToServerConnection.java:319) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Retaining reference counted message
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyToServerConnection.java:348) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Using existing connection to: www.example.com/104.18.19.67:443
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyConnection.java:228) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Writing: DefaultLastHttpContent(data: SlicedAbstractByteBuf(ridx: 0, widx: 147, cap: 147/147, unwrapped: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf(ridx: 147, widx: 147, cap: 176)), decoderResult: success)
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyConnection.java:249) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Writing an empty buffer to signal the end of our chunked transfer
DEBUG [LittleProxy-0-ClientToProxyWorker-2] (ProxyConnection.java:237) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Wrote: DefaultLastHttpContent(data: SlicedAbstractByteBuf(ridx: 0, widx: 147, cap: 147/147, unwrapped: UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf(ridx: 147, widx: 147, cap: 176)), decoderResult: success)
DEBUG [LittleProxy-0-ProxyToServerWorker-0] (ProxyConnection.java:111) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Reading: DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
HTTP/1.1 403 Forbidden
Date: Thu, 27 Aug 2020 17:10:51 GMT
Content-Type: text/html; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Frame-Options: SAMEORIGIN
Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Expires: Thu, 01 Jan 1970 00:00:01 GMT
cf-request-id: 04d27fab5700001bfece21f200000001
Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
Vary: Accept-Encoding
Strict-Transport-Security: max-age=15552000; includeSubDomains; preload
X-Content-Type-Options: nosniff
Server: cloudflare
CF-RAY: 5c97688bbb9f1bfe-OSL
Content-Encoding: br
alt-svc: h3-27=":443"; ma=86400, h3-28=":443"; ma=86400, h3-29=":443"; ma=86400
DEBUG [LittleProxy-0-ProxyToServerWorker-0] (ProxyToServerConnection.java:220) - (AWAITING_INITIAL) [id: 0xba528f83, L:/172.20.10.2:64134 - R:www.example.com/104.18.19.67:443]: Received raw response: DefaultHttpResponse(decodeResult: success, version: HTTP/1.1)
Thank you!
System.setProperty("webdriver.chrome.driver", "/usr/local/bin/chromedriver");
System.setProperty("webdriver.chrome.silentOutput", "true");
System.setProperty("webdriver.chrome.verboseLogging", "false");
ChromeOptions options = new ChromeOptions();
// route Chrome requests to proxy server
proxy = new BrowserMobProxyServer();
// to pass requests from proxy server to Fiddler Everywhere
// InetSocketAddress inetsockaddr = new InetSocketAddress('localhost', 8866);
// proxy.setTrustAllServers(true);
// proxy.setChainedProxy(inetsockaddr);
// start proxy server
proxy.start(0);
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
seleniumProxy.setHttpProxy("localhost:"+ proxy.getPort());
seleniumProxy.setSslProxy("localhost:"+ proxy.getPort());
options.setCapability(CapabilityType.PROXY, seleniumProxy);
options.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
options.setCapability (CapabilityType.ACCEPT_INSECURE_CERTS, true);
//options.addArguments("headless");
//options.addArguments("window-size=1200x600");
options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
options.setExperimentalOption("useAutomationExtension", false);
options.addArguments("--disable-blink-features=AutomationControlled");
driver = new ChromeDriver(options);`
Solved this by replacing the JDK, from OpenJDK to Oracle
I have to configure an NGINX server in order to load balance 2 ore more https websites.
I wrote this code:
user nginx;
#worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA';
ssl_prefer_server_ciphers on;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
# sendfile on;
#tcp_nopush on;
# keepalive_timeout 65;
upstream servers_alpha {
least_conn;
#server 10.200.0.43:801 max_fails=3 fail_timeout=30s;
server 10.200.0.43:443 max_fails=3 fail_timeout=30s;
#server 10.200.0.70:801 max_fails=3 fail_timeout=30s;
server 10.200.0.141:443 max_fails=3 fail_timeout=30s;
}
server {
#listen 80;
listen 443 ssl;
#ssl on;
ssl_certificate /etc/nginx/ssl/proxy1/server.crt;
ssl_certificate_key /etc/nginx/ssl/proxy1/server.key;
server_name proxy1;
location / {
proxy_pass https://servers_alpha;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /nginx_status {
stub_status on;
access_log off;
allow 127.0.0.1;
# allow 10.200.0.111;
allow 10.5.1.101;
deny all;
}
}
#location / {
# root /usr/share/nginx/html;
# index index.html index.htm;
# }
}
The 2 servers I'm using in this example are:
10.200.0.43 --> Spacewalk
10.200.0.141 --> Ansible
If I browse those sites directly there are no issues, the sites are fine and browsable.
If I pass through NGINX the sites are incomplete and not working: useless, unusable, as you can see in the images:
Ansible 1
Ansible 2
Spacewalk
This is the access.log:
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET / HTTP/1.1" 302 216 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /rhn/Login.do HTTP/1.1" 404 3360 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/css/bootstrap.min.css HTTP/1.1" 404 5086 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/api/api.css?v=3.0.2 HTTP/1.1" 200 7442 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/css/default.css HTTP/1.1" 200 1132 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/js/csrf.js HTTP/1.1" 200 1832 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/css/prettify.css HTTP/1.1" 404 5086 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/js/bootstrap.min.js HTTP/1.1" 200 36816 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/js/prettify-min.js HTTP/1.1" 200 13632 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/js/default.js HTTP/1.1" 200 1760 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/api/api.js?v=3.0.2 HTTP/1.1" 200 3145 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/assets/main_menu_logo.png HTTP/1.1" 200 6359 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/js/ajax-form.js HTTP/1.1" 404 5092 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/js/jquery-1.11.3.min.js HTTP/1.1" 200 95957 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/rest_framework/js/ajax-form.js HTTP/1.1" 404 5092 "https://10.200.0.69/rhn/Login.do" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
10.5.1.101 - - [04/Sep/2018:17:38:36 +0200] "GET /static/assets/favicon.ico?v=3.0.2 HTTP/1.1" 200 15086 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0" "-"
While the error.log:
2018/09/04 17:38:36 [warn] 17259#17259: *243 an upstream response is buffered to a temporary file /var/cache/nginx/proxy_temp/2/00/0000000002 while reading upstream, client: 10.5.1.101, server: proxy1, request: "GET /static/rest_framework/js/jquery-1.11.3.min.js HTTP/1.1", upstream: "https://10.200.0.141:443/static/rest_framework/js/jquery-1.11.3.min.js", host: "10.200.0.69", referrer: "https://10.200.0.69/rhn/Login.do"`
Can you please tell me what I'm doing wrong?
ps: If I redirect on http it works fine
for one of my projects I needed to deploy the POS module on one of my client's computer, so I followed the following link for the Packaged installers on a windows system.
https://www.odoo.com/documentation/10.0/setup/install.html#setup-install-packaged
after the installation, when i tried to open odoo on localhost:8069 it would not give any error but would show the interface of the odoo imroperly, maybe the css or less files were not correctly loaded, i installed lessc but it did not solve my problem. my browser has javascript enable but still the interface is not correctly loaded
so if anyone help me regarding this, would appreciate it.
here are the last lines of the log file:
2017-01-18 16:57:41,072 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 16:57:41] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 16:58:31,105 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 16:58:31] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 16:59:19,237 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 16:59:19] "POST /calendar/notify HTTP/1.1" 200 -
2017-01-18 16:59:21,144 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 16:59:21] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:00:11,176 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:00:11] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:01:01,223 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:01:01] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:01:51,285 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:01:51] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:02:41,329 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:02:41] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:03:31,361 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:03:31] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:04:04,611 4768 INFO testdb openerp.addons.base.ir.ir_autovacuum: GC'd 0 user log entries
2017-01-18 17:04:19,220 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:04:19] "POST /calendar/notify HTTP/1.1" 200 -
2017-01-18 17:04:21,392 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:04:21] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:05:11,437 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:05:11] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:06:01,469 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:06:01] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:06:51,500 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:06:51] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:07:41,546 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:07:41] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:08:31,579 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:08:31] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:09:19,377 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:09:19] "POST /calendar/notify HTTP/1.1" 200 -
2017-01-18 17:09:21,627 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:09:21] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:10:11,657 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:10:11] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:11:01,704 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:11:01] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:11:51,790 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:11:51] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:12:41,979 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:12:41] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:13:32,026 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:13:32] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:14:19,224 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:14:19] "POST /calendar/notify HTTP/1.1" 200 -
2017-01-18 17:14:22,069 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:14:22] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:15:12,099 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:15:12] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:16:02,147 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:16:02] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:16:52,194 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:16:52] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:17:42,233 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:17:42] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:18:19,141 4768 INFO test_odoo openerp.addons.base.ir.ir_autovacuum: GC'd 0 user log entries
2017-01-18 17:18:32,266 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:18:32] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:19:19,234 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:19:19] "POST /calendar/notify HTTP/1.1" 200 -
2017-01-18 17:19:22,312 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:19:22] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:20:12,351 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:20:12] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:20:12,351 4768 ERROR test_odoo openerp.service.server: Exception happened during processing of request from ('127.0.0.1', 63941)
Traceback (most recent call last):
File "SocketServer.pyc", line 593, in process_request_thread
File "SocketServer.pyc", line 334, in finish_request
File "SocketServer.pyc", line 651, in init
File "SocketServer.pyc", line 710, in finish
File "socket.pyc", line 279, in close
File "socket.pyc", line 303, in flush
error: [Errno 10053] An established connection was aborted by the software in your host machine
2017-01-18 17:20:49,263 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:20:49] "POST /longpolling/poll HTTP/1.1" 200 -
2017-01-18 17:21:39,305 4768 INFO test_odoo werkzeug: 127.0.0.1 - - [18/Jan/2017 17:21:39] "POST /longpolling/poll HTTP/1.1" 200 -