we've launched a private MWAA environment. We are able to access the UI, but we're having some trouble installing our python requirements.
MWAA picks up the requirements file from S3, but runs into a timeout when trying to install the python packages.
This is expected, because we're behind a proxy, so my question would be: how do we tell MWAA to use our proxy while installing our python dependencies?
This is what our CloudWatch logstream (requirements_install_ip*) tells us:
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None))
after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection
object at 0x7fda26b394d0>, 'Connection to pypi.org timed out. (connect timeout=15)')'
We have contacted AWS support on this, and apparently there is no such option as to pass the proxy variable. So we placed a feature request.
Even though I'm not sure if this is going to be implemented at all, anybody interested in this may feel free to subscribe to the MWAA document history feed.
You can set this in your pip.ini
[global]
index = https://eg.nexus.repo.url
index-url = https://eg.nexus.repo.url
To get where your pip.ini is, you can do:
pip config -v list
Related
I have a Jenkins job which I would like to invoke from my Codefresh pipeline.
Using the following example from the Codefresh docs, I have my Codefresh pipeline configured and ready:
https://codefresh.io/docs/docs/integrations/jenkins-integration/#calling-jenkins-jobs-from-codefresh-pipelines
The resulting build runs with the following output:
Pulling image codefresh/cf-run-jenkins-job:latest
Pulled layer '1160f4abea84'
Pulled layer '6df1582e0e0e'
Digest: sha256:a95b23c24b51d5fc1705731f7d18c5134590b4bc61b91dcf5a878faf2aec60b3
Status: Downloaded newer image for codefresh/cf-run-jenkins-job:latest
INFO[0000] Going to trigger <jenkins_job_name> job on https://<jenkins_host>:8443
ERRO[0000] Post https://<jenkins_host>:8443/job/<jenkins_job_name>/build: x509: failed to load system roots and no roots provided
Successfully ran freestyle step: Triggering Jenkins Job
Reading environment variable exporting file contents.
Reading environment variable exporting file contents.
As you can see, the build fails to successfully trigger the Jenkins job.
After some research in the Internet I came to conclusion that this is an SSL certificate issue.
But I have no idea how to proceed from here on. What exactly is missing and where it should be configured. I would really appreciate any help here.
Do you know that kind of SSL configuration your Jenkins server has? Is it mutual authentication or just a server-side certificate? Is it self-signed or not?
Have you tried to call the Jenkins API on your own (outside of Codefresh) and SSL works fine?
Also I would suggest you open a support ticket (from the top right menu in the Codefresh UI) and make sure to mention the URL of the build that has this issue.
I am trying to achieve the multicloud architecture. My network has 2 peers, 1 orderer and a webclient. This network is in Azure. I am trying to add a peer from Google Cloud Platform to the channel of Azure. For this, I created a crypto-config for 3rd peer from Azure webclient. But in the crypto-config, I made the changes like peers in Azure have their own certificates while for the 3rd peer, I placed the newly created certificates. Now I can install, instantiate, invoke and do queries in the peers(1 and 2). And I can install the chaincodes in 3rd peer. But I am unable to instantiate the chaincodes.
Getting the following error: Error: could not assemble transaction, err proposal response was not successful, error code 500, msg error starting container: error starting container: Post http://unix.sock/containers/create?name=dev-(CORE_PEER_ID)-documentCC-1: dial unix /var/run/docker.sock: connect: permission denied
Can anyone guide me on this.
Note: All the peers, orderer, webclient are running in different vm(s)
#soundarya
It doesn’t matter how many places your solution is deployed
The problem is you are running docker by using sudo command try to add docker to sudo group
Below block will help you out
https://www.digitalocean.com/community/questions/how-to-fix-docker-got-permission-denied-while-trying-to-connect-to-the-docker-daemon-socket
To learn more concept about docker.sock
You can refer to my answer in another Can anyone explain docker.sock
I have setup Spark SQL on Jypterhub using Apache Toree SQL kernel. I wrote a Python function to update Spark configuration options in the kernel.json file for my team to change configuration based on their queries and cluster configuration. But I have to shutdown the running notebook and re-open or restart the kernel after running Python function. In this way, I'm forcing the Toree kernel to read the JSON file to pick up the new configuration.
I thought of implementing this shutdown and restart of kernel in a programmatic way. I got to know about the Jupyterhub REST API documentation and am able implement it by invoking related API's. But the problem is, the single user server API port is set randomly by the Spawner object of Jupyterhub and it keeps changing every time I spin up a cluster. I want this to be fixed before launching the Jupyterhub service.
Here is a solution I tried based on Jupyterhub docs:
sudo echo "c.Spawner.port = 35289
c.Spawner.ip = '127.0.0.1'" >> /etc/jupyterhub/jupyterhub_config.py
But this did not work as the port was again set by the Spawner randomly. I think there is a way to fix this. Any help on this would be greatly appreciated. Thanks
I have anaconda install on my corporate laptop. I want to install 2 python packages( Potply & Fuzzywuzzy) but each time I try I have the same error message
Solving environment: failed
CondaHTTPError: HTTP 000 CONNECTION FAILED for url
<https://repo.anaconda.com/pk
gs/r/win-64/repodata.json.bz2>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on
your way.
If your current network has https://www.anaconda.com blocked, please file
a support request with your network engineering team.
ConnectTimeout(MaxRetryError("HTTPSConnectionPool
(host='repo.anaconda.com',pot
=443): Max retries exceeded with url: /pkgs/r/win-64/repodata.json.bz2
(Caused by ConnectTimeoutError(<urllib3.connection.VerifiedHTTPSConnection
object at 0x00000000054D45F8>, 'Connection to repo.anaconda.com timed out.
(connect timeout=9.
15)'))"))
I have tried to use use the command :
conda config --set ssl_verify no
or
conda config --set ssl_verify false
but none of them are working for me. Also because it's my company laptop I am not admin so I am not able to change the firewall and connection properties and I am not able to contact the service desk to help me on that.
So I 'll be more than happy to hear your solution(s).
I am trying to fetch a file over HTTPS in Io language:
url := URL with("https://api.example.com")
url fetch println
And I get this:
Error_0x7f97e1509a80:
location = "/opt/local/lib/io/addons/Socket/io/URL.io:232"
message = "Protocol 'https' unsupported"
I was trying to find something on the net, but, as everybody knows, it's not easy because of the name. I only found this thread http://tech.groups.yahoo.com/group/iolanguage/message/10898 but that's quite old.
How can I get the HTTPS support in Io?
EDIT
I've found that there is a SecureSocket addon, a wrapper over OpenSSL, in Io's source. It wasn't installed when I did sudo port io install on my MacBook with Mountain Lion, though. I tried building it from source, but no luck. It didn't build for me on a Linux machine, either.
EDIT2
I just tried to build Io from source (git clone https://github.com/stevedekorte/io.git) again (using the included script build.sh) and it turned out that cmake did detect OpenSSL:
-- Found OpenSSL: /usr/lib/libssl.dylib;/usr/lib/libcrypto.dylib
But then the SecureSocket addon is not built. Its readme file: https://github.com/stevedekorte/io/tree/master/addons/SecureSocket says:
The DTLS1 bindings are not usable unless the patches in this file are
applied to OpenSSL 0.9.8e. However, this patch includes a
deactivation of the handshake retransmission code in d1_both.c,
making it unsuitable for production environments. I take no
responsibility, etc, etc. If you want to use it anyway, apply the
patches(gathered from various newsgroups and my own experimentation)
and uncomment the commented-out block of build.io. For what it's
worth, DTLS support in OpenSSL is new as of 0.9.8 and is pretty buggy
to begin with. It's a nice idea, but it doesn't seem to be
production ready at all yet. These bindings are no exception.
If you can't get io to do it your best option would be calling an external tool like wget or curl which can and then loading the file/result locally or returning it via a pipe.
For anybody else interested in another workaround, it should be possible to put stud in front of an Io program which will do the SSL stuff. I have not tested that myself yet.
stud - The Scalable TLS Unwrapping Daemon stud is a network proxy that
terminates TLS/SSL connections and forwards the unencrypted traffic to
some backend. It's designed to handle 10s of thousands of connections
efficiently on multicore machines.