autodesk-forge, 2-legged Authentication (OAuth): oauth - authentication

Not sure what I did wrong, for 2 legged authentication. I am using cygwin terminal for curl.
Cullback URL is
http://localhost:3000/api/forge/callback/oauth
Please see the code below and let me know what is wrong. Not sure what is wrong.
jq version is 1.6, curl 7.64.1
GNU bash, version 4.4.12(3)-release (x86_64-unknown-cygwin)
To me it looks like, HTTP version is the reason of the error.I am very new for forge. So give me some idea, about what is wrong here.
$ curl -v 'https://developer.api.autodesk.com/authentication/v1/authenticate' -H 'Content-Type: application/x-www-form-urlencoded' -d 'client_id=qWSjFdCYTfhWsLZCSsLcSZCrDY2GsVSq' -d 'client_secret=PChX1GbaPFG42hQi' -d 'grant_type=client_credentials' -d 'scope=data:all'
output error report as below:
* STATE: INIT => CONNECT handle 0x8000770d8; line 1356 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x8000770d8; line 1397 (connection #0)
* Trying 34.212.2.95:443...
* TCP_NODELAY set
* STATE: WAITRESOLVE => WAITCONNECT handle 0x8000770d8; line 1476 (connection #0)
* Connected to developer.api.autodesk.com (34.212.2.95) port 443 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x8000770d8; line 1532 (connection #0)
* Marked for [keep alive]: HTTP default
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x8000770d8; line 1547 (connection #0)
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: businessCategory=Private Organization; jurisdictionC=US; jurisdictionST=Delaware; serialNumber=2401504; C=US; ST=California; L=San Rafael; O=Autodesk, Inc.; OU=MCP-ASRD-CP; CN=developer.api.autodesk.com
* start date: Mar 22 00:00:00 2019 GMT
* expire date: Mar 22 12:00:00 2020 GMT
* subjectAltName: host "developer.api.autodesk.com" matched cert's "developer.api.autodesk.com"
* issuer: C=US; O=DigiCert Inc; OU=www.digicert.com; CN=DigiCert SHA2 Extended Validation Server CA
* SSL certificate verify ok.
* STATE: PROTOCONNECT => DO handle 0x8000770d8; line 1566 (connection #0)
> POST /authentication/v1/authenticate HTTP/1.1
> Host: developer.api.autodesk.com
> User-Agent: curl/7.64.1
> Accept: */*
> Content-Type: application/x-www-form-urlencoded
> Content-Length: 118
>
* upload completely sent off: 118 out of 118 bytes
* STATE: DO => DO_DONE handle 0x8000770d8; line 1621 (connection #0)
* STATE: DO_DONE => PERFORM handle 0x8000770d8; line 1743 (connection #0)
* Mark bundle as not supporting multiuse
* HTTP 1.1 or later with persistent connection
< HTTP/1.1 400 Bad Request
< Content-Type: application/json
< Date: Thu, 22 Aug 2019 05:58:08 GMT
< Content-Length: 210
< Connection: keep-alive
<
* STATE: PERFORM => DONE handle 0x8000770d8; line 1933 (connection #0)
* multi_done
* Connection #0 to host developer.api.autodesk.com left intact
{"developerMessage":"Requested scopes must be blank or a subset of the provided scopes.","userMessage":"","errorCode":"AUTH-004","more info":"http://developer.api.autodesk.com/documentation/v1/errors/AUTH-004"}

As the error message suggests data:all is not a valid scope...
See the Resource Information section of the endpoints you are accessing to see what scopes are required - you can specify multiple scopes so long as they are valid delimited by space:
scope=data:read data:write bucket:read code:all

Related

curl authentication works with `--netrc` but not `--user`

With curl 7.74.0, the command
$ curl -b ~/.cookies -c ~/.cookies -L --user 'USERNAME:PASSWORD' https://daac.ornl.gov/daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt
yields HTTP 401 (verbose output listed below). However, the following works:
$ echo 'machine urs.earthdata.nasa.gov login USERNAME password PASSWORD' >>~/.netrc
$ curl -b ~/.cookies -c ~/.cookies -L --netrc https://daac.ornl.gov/daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt
Why is that? Shouldn't these approaches be equivalent? Perhaps there's an extra option I need to convince curl to pass the credentials set with --user down to urs.earthdata.nasa.gov.
You can create an Earthdata account at https://urs.earthdata.nasa.gov/users/new to test these commands.
(N.B. The --user version works if you already have a valid login cookie from the --netrc version. The question is why login doesn't work.)
The -v output for the --user version is (authenticating cookies etc. censored with "XXX"):
* Trying 160.91.19.24:443...
* Connected to daac.ornl.gov (160.91.19.24) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=Tennessee; L=Oak Ridge; O=Oak Ridge National Laboratory; OU=DAAC; CN=*.ornl.gov
* start date: Jul 14 00:00:00 2020 GMT
* expire date: Oct 11 12:00:00 2022 GMT
* subjectAltName: host "daac.ornl.gov" matched cert's "*.ornl.gov"
* issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
* SSL certificate verify ok.
* Server auth using Basic with user 'Kodiologist'
> GET /daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt HTTP/1.1
> Host: daac.ornl.gov
> Authorization: Basic XXX
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< Date: Mon, 25 Apr 2022 13:02:02 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000
< Content-Length: 381
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>401 Unauthorized</title>
</head><body>
<h1>Unauthorized</h1>
<p>This server could not verify that you
are authorized to access the document
requested. Either you supplied the wrong
credentials (e.g., bad password), or your
browser doesn't understand how to supply
the credentials required.</p>
</body></html>
* Connection #0 to host daac.ornl.gov left intact
The -v output for the --netrc version is:
* Couldn't find host daac.ornl.gov in the .netrc file; using defaults
* Trying 160.91.19.24:443...
* Connected to daac.ornl.gov (160.91.19.24) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=Tennessee; L=Oak Ridge; O=Oak Ridge National Laboratory; OU=DAAC; CN=*.ornl.gov
* start date: Jul 14 00:00:00 2020 GMT
* expire date: Oct 11 12:00:00 2022 GMT
* subjectAltName: host "daac.ornl.gov" matched cert's "*.ornl.gov"
* issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
* SSL certificate verify ok.
> GET /daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt HTTP/1.1
> Host: daac.ornl.gov
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Date: Mon, 25 Apr 2022 13:03:08 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000
< Location: https://urs.earthdata.nasa.gov/oauth/authorize?app_type=401&client_id=XXXA&response_type=code&redirect_uri=https%3A%2F%2Fdaac.ornl.gov%2Fdaacdata%2Fdoesntmater&state=XXX
< Content-Length: 518
< Content-Type: text/html; charset=iso-8859-1
<
* Ignoring the response-body
* Connection #0 to host daac.ornl.gov left intact
* Issue another request to this URL: 'https://urs.earthdata.nasa.gov/oauth/authorize?app_type=401&client_id=XXX&response_type=code&redirect_uri=https%3A%2F%2Fdaac.ornl.gov%2Fdaacdata%2Fdoesntmater&state=XXX'
* Trying 198.118.243.33:443...
* Connected to urs.earthdata.nasa.gov (198.118.243.33) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=Maryland; L=Greenbelt; jurisdictionC=US; O=NASA (National Aeronautics and Space Administration); businessCategory=Government Entity; serialNumber=1958-07-29; CN=urs.earthdata.nasa.gov
* start date: Aug 27 16:09:18 2021 GMT
* expire date: Sep 12 16:09:18 2022 GMT
* subjectAltName: host "urs.earthdata.nasa.gov" matched cert's "urs.earthdata.nasa.gov"
* issuer: C=US; O=Entrust, Inc.; OU=See www.entrust.net/legal-terms; OU=(c) 2014 Entrust, Inc. - for authorized use only; CN=Entrust Certification Authority - L1M
* SSL certificate verify ok.
* Server auth using Basic with user 'Kodiologist'
> GET /oauth/authorize?app_type=401&client_id=XXX&response_type=code&redirect_uri=https%3A%2F%2Fdaac.ornl.gov%2Fdaacdata%2Fdoesntmater&state=XXX HTTP/1.1
> Host: urs.earthdata.nasa.gov
> Authorization: Basic XXX
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Server: nginx/1.20.1
< Date: Mon, 25 Apr 2022 13:03:08 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< X-Download-Options: noopen
< X-Permitted-Cross-Domain-Policies: none
< Referrer-Policy: strict-origin-when-cross-origin
< Cache-Control: no-store
< Pragma: no-cache
< Expires: Fri, 01 Jan 1990 00:00:00 GMT
< Access-Control-Allow-Origin: null
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Methods: GET, POST
< Access-Control-Expose-Headers: true
< Location: https://daac.ornl.gov/daacdata/doesntmater?code=XXX&state=XXX
* Added cookie urs_user_already_logged="yes" for domain earthdata.nasa.gov, path /, expire 1650978188
< Set-Cookie: urs_user_already_logged=yes; domain=earthdata.nasa.gov; path=/; expires=Tue, 26 Apr 2022 13:03:08 GMT; secure; HttpOnly
* Added cookie _urs-gui_session="XXX" for domain urs.earthdata.nasa.gov, path /, expire 1650978188
< Set-Cookie: _urs-gui_session=XXX; path=/; expires=Tue, 26 Apr 2022 13:03:08 GMT; HttpOnly
< X-Request-Id: XXX
< X-Runtime: 0.053834
< Strict-Transport-Security: max-age=31536000
<
* Ignoring the response-body
* Connection #1 to host urs.earthdata.nasa.gov left intact
* Issue another request to this URL: 'https://daac.ornl.gov/daacdata/doesntmater?code=XXX&state=XXX'
* Couldn't find host daac.ornl.gov in the .netrc file; using defaults
* Found bundle for host daac.ornl.gov: XXX [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host daac.ornl.gov
* Connected to daac.ornl.gov (160.91.19.24) port 443 (#0)
> GET /daacdata/doesntmater?code=XXX&state=XXX HTTP/1.1
> Host: daac.ornl.gov
> User-Agent: curl/7.74.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
< Date: Mon, 25 Apr 2022 13:03:08 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000
* Added cookie AppAuth="XXX" for domain daac.ornl.gov, path /, expire 0
< Set-Cookie: AppAuth=XXX; Path=/
< Location: https://daac.ornl.gov/daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt
< Content-Length: 284
< Content-Type: text/html; charset=iso-8859-1
<
* Ignoring the response-body
* Connection #0 to host daac.ornl.gov left intact
* Issue another request to this URL: 'https://daac.ornl.gov/daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt'
* Couldn't find host daac.ornl.gov in the .netrc file; using defaults
* Found bundle for host daac.ornl.gov: XXX [serially]
* Can not multiplex, even if we wanted to!
* Connection 0 seems to be dead!
* Closing connection 0
* TLSv1.2 (OUT), TLS alert, close notify (256):
* Hostname daac.ornl.gov was found in DNS cache
* Trying 160.91.19.24:443...
* Connected to daac.ornl.gov (160.91.19.24) port 443 (#2)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* SSL re-using session ID
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=US; ST=Tennessee; L=Oak Ridge; O=Oak Ridge National Laboratory; OU=DAAC; CN=*.ornl.gov
* start date: Jul 14 00:00:00 2020 GMT
* expire date: Oct 11 12:00:00 2022 GMT
* subjectAltName: host "daac.ornl.gov" matched cert's "*.ornl.gov"
* issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
* SSL certificate verify ok.
> GET /daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt HTTP/1.1
> Host: daac.ornl.gov
> User-Agent: curl/7.74.0
> Accept: */*
> Cookie: AppAuth=XXX
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 25 Apr 2022 13:03:09 GMT
< Server: Apache
< Strict-Transport-Security: max-age=31536000
< Last-Modified: Mon, 07 Dec 2020 22:32:53 GMT
< ETag: "5b5-5b5e76833e87e"
< Accept-Ranges: bytes
< Content-Length: 1461
< Vary: Accept-Encoding
< Content-Type: text/plain; charset=UTF-8
<
station_id,station_name,stn_lat,stn_lon,stnz
RQC00660152,AGUIRRE,17.9556,-66.2222,7.6
RQC00664193,GUAYAMA 2E,17.9783,-66.0875,21.9
RQC00665693,MAGUEYES IS,17.9722,-67.0461,3.7
VQC00671740,CHRISTIANSTED FT,17.7469,-64.7014,9.1
VQW00011624,CHRISTIANSTED AP,17.7028,-64.8056,18.6
RQC00660061,ADJUNTAS SUBSTN,18.1747,-66.7978,557.8
RQC00660158,AIBONITO 1 S,18.1281,-66.2642,710.2
RQC00660426,ARECIBO OBSY,18.3494,-66.7525,323.1
RQC00662801,COLOSO,18.3808,-67.1569,12.2
RQC00663431,DOS BOCAS,18.3361,-66.6667,61.0
RQC00664614,HUMACAO NATURAL RESERVE,18.1506,-66.7719,2.7
RQC00664702,ISABELA SUBSTN,18.4653,-67.0525,128.0
RQC00665097,LAJAS SUBSTN,18.0331,-67.0722,27.4
RQC00665807,MANATI 2 E,18.4308,-66.4661,76.2
RQC00665908,MARICAO 2 SSW,18.1511,-66.9889,863.2
RQC00666730,PALMAREJO VEGA BAJA,18.385,-66.43,170.7
RQC00667292,PONCE 4 E,18.0258,-66.5253,21.3
RQC00668306,RIO PIEDRAS EXP STN,18.3906,-66.0542,28.0
RQC00669415,TOA BAJA LEVITTOWN,18.4356,-66.1678,8.5
RQC00669432,TORO NEGRO FOREST,18.1731,-66.4928,868.1
RQC00669521,TRUJILLO ALTO 2 SSW,18.3283,-66.0164,35.1
RQW00011641,SAN JUAN L M MARIN AP,18.4325,-66.0108,2.7
RQC00665064,JUNCOS 1 SE,18.2264,-65.9114,64.9
RQC00666343,CULEBRA HILL,18.2972,-65.29,71.3
RQC00666725,PALMA SOLA,18.3169,-65.8664,466.0
RQC00668814,WFO SAN JUAN,18.4311,-65.9917,3.0
RQW00011630,ROOSEVELT ROADS,18.255,-65.6408,10.1
VQC00677600,REDHOOK BAY ST THOMAS,18.3236,-64.8542,0.6
VQW00011640,CHARLOTTE AMALIE AP,18.3331,-64.9667,6.1
* Connection #2 to host daac.ornl.gov left intact
The --user version works if -L is replaced with --location-trusted and --anyauth is added.
in your netrc you specifically say "this is the username:password for urs.earthdata.nasa.gov", and your command first connects to a different domain: daac.ornl.gov. so your netrc does not trigger on daac.ornl.gov, your "broken" curl command sends the username:password to daac.ornl.gov and and daac.ornl.gov does not like this invalid username:password and your curl request get rejected. meanwhile your working curl command does not send username:password to daac.ornl.gov, then daac.ornl.gov proceed to redirect you to to urs.earthdata.nasa.gov , here your netrc kicks in and your username:password gets sent to urs.earthdata.nasa.gov as it should..
unfortunately curl does not (currently?) support loading netrc from stdin, and does not support setting username:password for specific domains via arguments (not as far as i know anyway?), so your best bet is probably:
NETRCTMPFILE="$(mktemp)"
echo 'machine urs.earthdata.nasa.gov login USERNAME password PASSWORD' > "$NETRCTMPFILE";
curl -b ~/.cookies -c ~/.cookies -L --netrc-file "$NETRCTMPFILE" https://daac.ornl.gov/daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt
rm "$NETRCTMPFILE"
You're using quotes around the username and password. In my experience this causes issues.
try
curl -b ~/.cookies -c ~/.cookies -L --user USERNAME:PASSWORD https://daac.ornl.gov/daacdata/daymet/Daymet_V4_Stn_Level_CrossVal/data/stnxvalmeta_tmin_pr_2016.txt
This has issues with your user and password being present in bash history.
This link has a lot of good info on other ways to do this safely too
https://stackoverflow.com/a/56130884/496405

Debugging SSL Handshake problems of kubernetes pods with the internet

I recently installed k3s on debian 10 and I am having problems connecting to the internet from inside the pods. Ping, DNS, HTTP, works but TLS is having problems with the Handshake. I made a tcpdump and see ClientHello's and then immediatly a response from the server with internal error SSL error 80. On the VM where cluster runs on everything works fine, just not inside the pods.
Example in container:
pacman -Syu
:: Synchronizing package databases...
error: failed retrieving file 'core.db' from mirror.pkgbuild.com : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed retrieving file 'core.db' from mirror.rackspace.com : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed retrieving file 'core.db' from mirror.leaseweb.net : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed to update core (download library error)
error: failed retrieving file 'extra.db' from mirror.pkgbuild.com : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed retrieving file 'extra.db' from mirror.rackspace.com : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed retrieving file 'extra.db' from mirror.leaseweb.net : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed to update extra (download library error)
error: failed retrieving file 'community.db' from mirror.pkgbuild.com : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed retrieving file 'community.db' from mirror.rackspace.com : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed retrieving file 'community.db' from mirror.leaseweb.net : error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
error: failed to update community (download library error)
error: failed to synchronize all databases
curl -4 -v https://www.google.com
* Trying 157.230.127.168:443...
* Connected to www.google.com (157.230.127.168) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS alert, internal error (592):
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
On VM:
curl -4 -v https://www.google.com
* Expire in 0 ms for 6 (transfer 0x556e74843fb0)
<snip>A lot of these expire messages</snip>
* Expire in 10 ms for 1 (transfer 0x556e74843fb0)
* Trying 172.217.23.100...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x556e74843fb0)
* Connected to www.google.com (172.217.23.100) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
* start date: Mar 11 15:00:19 2021 GMT
* expire date: Jun 3 15:00:18 2021 GMT
* subjectAltName: host "www.google.com" matched cert's "www.google.com"
* issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x556e74843fb0)
> GET / HTTP/2
> Host: www.google.com
> User-Agent: curl/7.64.0
> Accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
< date: Fri, 26 Mar 2021 01:32:52 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: gws
< x-xss-protection: 0
< x-frame-options: SAMEORIGIN
< set-cookie: NID=212=0Nbn1MMUvMRUNa8iBWgekw-YrWof3yeeBb22v94ZrQ4KkMeVm5wouwkK9ElA353VmZCp_TOXah6KfC_KQX7S48W-IgXlQUz1z4ytqKnDzSXM7X40rLw8tBwMi4oH7eLyE4nGGAfsSlUne28lwMNLobMxYls54iUbgM4x9-kuMCA; expires=Sat, 25-Sep-2021 01:32:52 GMT; path=/; domain=.google.com; HttpOnly
< alt-svc: h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
< accept-ranges: none
< vary: Accept-Encoding
<hmtl>...,</html>
Does anybody have an idea how I can debug this or fix it?
UPDATE
Here a curl from inside the VM and a pod. I fixed the Webserver to a known working one on the Internet (Vercel Hosted Website) aswell so we debug the same server. But it is always the same error inside the pod. no matter which server I contact.
kubectl run -i --tty --rm debug --image=archlinux --restart=Never -- bash
Pod:
[root#debug /]# curl -v --tlsv1.2 --tls-max 1.2 https://wegmueller.it/ > /dev/null
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 157.230.127.168:443...
* Connected to wegmueller.it (157.230.127.168) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: none
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [229 bytes data]
* TLSv1.2 (IN), TLS alert, internal error (592):
{ [2 bytes data]
* error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
* Closing connection 0
curl: (35) error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
[root#debug /]#
VM:
curl -v --tlsv1.2 --tls-max 1.2 https://wegmueller.it/ > /dev/null
* Expire in 0 ms for 6 (transfer 0x559d896cafb0)
* Expire in 1 ms for 1 (transfer 0x559d896cafb0)
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Expire in 0 ms for 1 (transfer 0x559d896cafb0)
* Expire in 1 ms for 1 (transfer 0x559d896cafb0)
<snip a ton of expire messages>
* Expire in 50 ms for 1 (transfer 0x559d896cafb0)
* Trying 76.76.21.21...
* TCP_NODELAY set
* Expire in 200 ms for 4 (transfer 0x559d896cafb0)
* Connected to wegmueller.it (76.76.21.21) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
} [5 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [223 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [106 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2458 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=wegmueller.it
* start date: Mar 16 21:08:10 2021 GMT
* expire date: Jun 14 21:08:10 2021 GMT
* subjectAltName: host "wegmueller.it" matched cert's "wegmueller.it"
* issuer: C=US; O=Let's Encrypt; CN=R3
* SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
} [5 bytes data]
* Using Stream ID: 1 (easy handle 0x559d896cafb0)
} [5 bytes data]
> GET / HTTP/2
> Host: wegmueller.it
> User-Agent: curl/7.64.0
> Accept: */*
>
{ [5 bytes data]
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
} [5 bytes data]
< HTTP/2 200
< date: Sun, 28 Mar 2021 15:24:59 GMT
< content-type: text/html; charset=utf-8
< content-disposition: inline; filename="index"
< cache-control: public, max-age=0, must-revalidate
< content-length: 3154
< access-control-allow-origin: *
< etag: W/"637eb49039bc1f6a7aa9e4d88a56b0708b722983f403c3c2f717371b2b25a472"
< accept-ranges: bytes
< x-vercel-cache: MISS
< age: 0
< server: Vercel
< x-vercel-id: cdg1::z6h9l-1616945098788-71384c5744be
< strict-transport-security: max-age=63072000
<
{ [1112 bytes data]
100 3154 100 3154 0 0 3604 0 --:--:-- --:--:-- --:--:-- 3600
* Connection #0 to host wegmueller.it left intact
error:14094438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error
openssl error code contains 3 parts.
First 8 bits is module serial number. 14(HEX) means ssl module which has serial number 20.
The next 12 bits is function SN. 094(HEX) is SSL_SRP_CTX_init func with SN 148(DEC).
The next 12 bits 438(HEX) is error reason SSL_R_TLSV1_ALERT_INTERNAL_ERROR with 1080(DEC).
To sum up,code 14094438 means that server error occurred when it tried to initiate SSL context.
But what's the reason?
client side
client side send CLIENT HELLO with its tls version, ciphers suite and etc to server side.
what's the difference of 2 CLIENT HELLO packets on pods or VM? This need to be clear.
Here you use newest version tls 1.3. For more infomation, you may downgrade client version.
curl has options to control the TLS version used. if you want to specify that TLS 1.2 is used but not 1.1 or 1.3 etc, you need something like
curl --tlsv1.2 --tls-max 1.2 ...
server side
I also noticed that www.google.com (157.230.127.168) is different from www.google.com (172.217.23.100).
Does server side's difference lead to this situation? We can confirm with sending a curl to the same IP.
I assume either you and/or the server uses an unmatched openssl library.But before drawing conclusion, we need more debug info.

servers returns "Invalid Host header" inside alpine when connecting via ssl

First of all, this question is not about how to fix this, but rather about why it happens.
Exact same curl (exact same version curl/7.65.1) with exact same request produces different results.
Why the certificate chain is different?
How server response depends on ssl connection flow? (Alpine get response body "Invalid host header", host OS downloads file w/o any issues)?
Why http version is different? How do the server and client agree on protocol version? Running the same command with --http curl flag fixes the issue.
docker run -i -t alpine /bin/sh:
apk add curl
curl -kfSL https://nginx.org/download/nginx-1.15.3.tar.gz -o nginx.tar.gz -vvv
Output:
* Trying 95.211.80.227:443...
* TCP_NODELAY set
* Connected to nginx.org (95.211.80.227) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=localhost
* start date: Jul 8 19:13:11 2019 GMT
* expire date: Aug 7 19:13:11 2019 GMT
* issuer: CN=localhost
* SSL certificate verify result: self signed certificate (18), continuing anyway.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55e3e4a37540)
> GET /download/nginx-1.15.3.tar.gz HTTP/2
> Host: nginx.org
> User-Agent: curl/7.65.1
> Accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 4294967295)!
< HTTP/2 200
< x-powered-by: Express
< content-type: text/html; charset=utf-8
< content-length: 19
< etag: W/"13-OxsTL6IB85fkJxv9HO8uum0slCI"
<
* Connection #0 to host nginx.org left intact
Invalid Host header
Same curl command works completely fine from my host machine (Archlinux bleeding edge) even without insecure (-k)option.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 95.211.80.227:443...
* TCP_NODELAY set
* Connected to nginx.org (95.211.80.227) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/certs/ca-certificates.crt
CApath: none
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2621 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [333 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [70 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
* subject: CN=nginx.org
* start date: May 14 19:45:30 2019 GMT
* expire date: Aug 12 19:45:30 2019 GMT
* subjectAltName: host "nginx.org" matched cert's "nginx.org"
* issuer: C=US; O=Let's Encrypt; CN=Let's Encrypt Authority X3
* SSL certificate verify ok.
} [5 bytes data]
> GET /download/nginx-1.15.3.tar.gz HTTP/1.1
> Host: nginx.org
> User-Agent: curl/7.65.1
> Accept: */*
>
{ [5 bytes data]
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: nginx/1.15.7
< Date: Wed, 10 Jul 2019 13:03:16 GMT
< Content-Type: application/octet-stream
< Content-Length: 1022881
< Last-Modified: Tue, 28 Aug 2018 15:40:55 GMT
< Connection: keep-alive
< Keep-Alive: timeout=15
< ETag: "5b856d07-f9ba1"
< Accept-Ranges: bytes
P.S. I'm aware of this response

Xively REST API with cURL - 403 Forbidden

I'm trying to follow the xively cURL tutorial. I have created new device in develop mode, copied the default auto-generated API key (with READ, WRITE, CREATE, DELETE permissions) to the example in tutorial and get response:
{"title":"Forbidden","errors":"You do not have the necessary permissions to access this resource"}
I must be missing some obvious step. Do I need to activate the API key somehow before using it in scripts?
The cURL command:
curl --request POST \
--data '{"title":"My feed", "version":"1.0.0"}' \
--header "X-ApiKey: cPHLfGw1WJdMAbU8FzbfsdFyJ8suayHEH3OChRrkpYwQCmrb" \
--verbose \
https://api.xively.com/v2/feeds
full verbose output:
* About to connect() to api.xively.com port 443 (#0)
* Trying 216.52.233.120...
* Connected to api.xively.com (216.52.233.120) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: none
CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-RC4-SHA
* Server certificate:
* subject: C=US; postalCode=01801; ST=MA; L=Woburn; street=First Floor; street=500 Unicorn Park Drive; O=LogMeIn Inc.; OU=Secure Link SSL Wildcard; CN=*.xively.com
* start date: 2013-05-07 00:00:00 GMT
* expire date: 2014-04-27 23:59:59 GMT
* subjectAltName: api.xively.com matched
* issuer: C=US; O=Network Solutions L.L.C.; CN=Network Solutions Certificate Authority
* SSL certificate verify ok.
> POST /v2/feeds HTTP/1.1
> User-Agent: curl/7.29.0
> Host: api.xively.com
> Accept: */*
> X-ApiKey: cPHLfGw1WJdMAbU8FzbfsdFyJ8suayHEH3OChRrkpYwQCmrb
> Content-Length: 38
> Content-Type: application/x-www-form-urlencoded
>
* upload completely sent off: 38 out of 38 bytes
< HTTP/1.1 403 Forbidden
< Date: Sat, 30 Nov 2013 11:03:15 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 98
< Connection: keep-alive
< X-Request-Id: 6cbb9676b448a4967187271dd246b423f7da2e39
<
* Connection #0 to host api.xively.com left intact
{"title":"Forbidden","errors":"You do not have the necessary permissions to access this resource"}
It depends what you are trying to do. POST requests to api.xively.com/v2/feeds are no longer supported with any API key. This is because, since the implementation of devices, programatic creation of feeds directly is no longer supported.
This is an oversight in the Xively tutorial and I will inform the appropriate people to make sure that it gets changed.
In the mean time, since you have already created a device, you are basically ready to start the cURL tutorial at step 3 "Update a Feed". Use the API key and Feed ID from the development device you have already created on the website. Make sure to change your URL, body, and from a POST request to a PUT request.
I think you need to add your feed id onto the end of the URL.

Curl error - Empty reply from server (ssl)

I have problems accessing certain URL using curl on OSX. For other urls I receive response, and for this specific UWL I can receive response from other, ubuntu machine.
I am stucked and do not know what component is broken and do not have any idea, other than upgrading whole system and rould really appreciate any advice that would prevent me do so.
Output of curl:
$ curl -v https://subdomain.example.com/pushdispatcher/dispatcher.asp
* About to connect() to subdomain.example.com port 443 (#0)
* Trying 213.229.251.147... connected
* Connected to subdomain.example.com (213.229.251.147) port 443 (#0)
* successfully set certificate verify locations:
* CAfile: /Users/bmihelac/.cert/cacert.pem
CApath: none
* SSLv2, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using RC4-MD5
* Server certificate:
* subject: /C=SI/ST=Ljubljana/L=Ljubljana/O=Example d.d./OU=Example/CN=subdomain.example.com
* start date: 2011-07-01 00:00:00 GMT
* expire date: 2012-07-26 23:59:59 GMT
* common name: subdomain.example.com (matched)
* issuer: /C=US/O=Thawte, Inc./CN=Thawte SSL CA
* SSL certificate verify ok.
> GET /pushdispatcher/dispatcher.asp HTTP/1.1
> User-Agent: curl/7.16.4 (i386-apple-darwin9.0) libcurl/7.16.4 OpenSSL/0.9.7l zlib/1.2.3
> Host: subdomain.example.com
> Accept: */*
>
* SSLv3, TLS handshake, Hello request (0):
* SSL read: error:00000000:lib(0):func(0):reason(0), errno 54
* Empty reply from server
* Connection #0 to host subdomain.example.com left intact
curl: (52) SSL read: error:00000000:lib(0):func(0):reason(0), errno 54
* Closing connection #0
It seems that the server is dropping your connection for some reason.
Try to add User-Agent headers to the request ( bot detection? )
Try to connect from other ip ( your ip is in black list? )