Fixing git HTTPS Error: "bad key length" on macOS 12 - ssl

Question

I am using a company-hosted (Bitbucket) git repository that is accessible via HTTPS. Accessing it (e.g. git fetch) worked using macOS 11 (Big Sur), but broke after an update to macOS 12 Monterey.
*
After the update of macOS to 12 Monterey my previous git setup broke. Now I am getting the following error message:
$ git fetch
fatal: unable to access 'https://.../':
error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length
For what it's worth, using curl does not work either:
$ curl --insecure -L -v https://...
* Trying ...
* Connected to ... (...) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
* CAfile: /etc/ssl/cert.pem
* CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length
* Closing connection 0
curl: (35) error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length
Accessing the same HTTPS-source via Safari or Firefox works.
As far as I understand, the underlying error "bad key length" error is coming from OpenSSL/LibreSSL, this would be consistent with both git and curl failing after an OS upgrade.
This is the output from openssl:
$ openssl s_client -servername ... -connect ...:443
CONNECTED(00000005)
depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root G2
verify return:1
depth=1 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = Thawte TLS RSA CA G1
verify return:1
depth=0 ...
4593010348:error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length:
/System/Volumes/Data/SWE/macOS/BuildRoots/b8ff8433dc/Library/Caches/com.apple.xbs
/Sources/libressl/libressl-75/libressl-2.8/crypto/apple/hmac/hmac.c:188:
---
Certificate chain
...
---
No client certificate CA names sent
Server Temp Key: DH, 2048 bits
---
SSL handshake has read 4105 bytes and written 318 bytes
---
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : DHE-RSA-AES256-GCM-SHA384
Session-ID: 1FA062DC9EEC9A310FF8231F1EB11A3BD6E0778F7AB6E98EAD1020A44CF1A407
Session-ID-ctx:
Master-Key:
Start Time: 1635319904
Timeout : 7200 (sec)
Verify return code: 0 (ok)
---
I did try to add the server's certificates into a custom pem file and setting http.sslCAInfo, but that didn't work. As a workaround, I am currently using a proxy that decrypts/re-encrypts HTTPS traffic.
How do I configure git (or all LibreSSL users) to accept the server's certificate?

Answer 1

Setting this ENV var (in e.g. ~/.zshrc) worked for me
export CURL_SSL_BACKEND="secure-transport"

Answer 2

Unfortunately I can't provide you with a fix, but I've found a workaround for that exact same problem (company-hosted bitbucket resulting in exact same error).
I also don't know exactly why the problem occurs, but my best guess would be that the libressl library shipped with Monterey has some sort of problem with specific (?TLSv1.3) certs. This guess is because the brew-installed openssl v1.1 and v3 don't throw that error when executed with /opt/homebrew/opt/openssl/bin/openssl s_client -connect ...:443
To get around that error, I've built git from source built against different openssl and curl implementations:
install autoconf, openssl and curl with brew (I think you can select the openssl lib you like, i.e. v1.1 or v3, I chose v3)
clone git version you like, i.e. git clone --branch v2.33.1 https://github.com/git/git.git
cd git
make configure (that is why autoconf is needed)
execute LDFLAGS="-L/opt/homebrew/opt/openssl#3/lib -L/opt/homebrew/opt/curl/lib" CPPFLAGS="-I/opt/homebrew/opt/openssl#3/include -I/opt/homebrew/opt/curl/include" ./configure --prefix=$HOME/git (here LDFLAGS and CPPFLAGS include the libs git will be built against, the right flags are emitted by brew on install success of curl and openssl; --prefix is the install directory of git, defaults to /usr/local but can be changed)
make install
ensure to add the install directory's subfolder /bin to the front of your $PATH to "override" the default git shipped by Monterey
restart terminal
check that git version shows the new version
This should help for now, but as I already said, this is only a workaround, hopefully Apple fixes their libressl fork ASAP.

Answer 3

Edit: macOS 12.3 was officially released on march 13th, 2022 and the issue reported on this question has been fixed. Personally I have removed the workaround I had deployed and I suggest others do the same to keep up with the new versions.
Previous answer:
Apparently Apple has updated the LibreSSL version on macOS Monterey 12.3 developer beta 2 and later so this issue no longer happens. The stable 12.3 version is yet to be released but I have tested the developer beta and confirmed it fixed the issue for me, so there should soon be no need to deploy the workarounds mentioned on this question.
I guess macOS 12.3 will probably be released some time next month.

Answer 4

Spent almost 3 weeks to solve this on MACOS 12.1 Monterey. I was getting
fatal: unable to access 'https://.../':
error:06FFF089:digital envelope routines:CRYPTO_internal:bad key length
Here's what worked:
Execute in a terminal with zsh (Z Shell):
echo 'export CURL_SSL_BACKEND="secure-transport"
' >> ~/.zshenv
Reload Changes. Type below line in terminal
source ~/.zshenv
Check if your new variable is set properly. Type below line in terminal
echo CURL_SSL_BACKEND

Answer 5

Accepted answer worked, but you might have to check if the lib and include paths are correct with
brew info openssl
and
brew info curl