Why does Azure CosmosDB emulator intermittently fail to respond? - pymongo

I am developing a web server to be deployed on Azure and in the process I am using the CosmosDB emulator, locally installed on my Windows 10 machine. The problem is that the database seems to refuse to connect intermittently - let me stress this, it sometimes responds and most of the times it does not. Why is this so and what can I do to get around the problem?
Docker is not used, for server nor client.
OS is Windows 10.
I am using CosmosDB's MongoDB API.
I am using pymongo to connect synchronously (in scripts), and motor to connect asynchronously (for the web server).
Both with pymongo and motor I experience the same problem, namely the error output
pymongo.errors.ServerSelectionTimeoutError: localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129), Timeout: 100.0s, Topology Description: <TopologyDescription id: 63aaec350ac3d3aa70c8bcf7, topology_type: Unknown, servers: [<ServerDescription ('localhost', 10255) server_type: Unknown, rtt: None, error=AutoReconnect('localhost:10255: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129)')>]>
which occurs intermittently, ie not all the time, but most of the times.
As far as I can see, the pattern is that the failure occurs for a stretch of time, and then for a stretch of time it works without error.
I am thinking that this cannot be a certificate error, because the connection sometimes does work (and because everything is on the same machine). AFAIU it looks like a timeout error, but I have tried timeouts up to 200 seconds.
Code:
An example snippet of a data-import script
with open(data_filename, 'r') as infile:
new_items = json.load(infile)
CONNECTION_STRING = os.environ.get('MONGODB_URL')
client = pymongo.MongoClient(CONNECTION_STRING)
db = client[constants.DB_NAME]
collection = db[constants.COLLECTION_CONTACTS]
with pymongo.timeout(100):
for item in new_items:
existing_items = collection.find(filter={'name': item['name']})
if existing_items:
print(f'Item exists for {existing_items[0]["name"]} with id: {[i["_id"] for i in existing_items]}')
# continue
Other actions, such as retrieving data with motor, or listing databases with client.list_database_names(), will fail similarly.

Client issues such as these are nearly impossible to troubleshoot on a forum because of all the variables within users' PCs. The documentation for the emulator recommends that for connectivity issues, users collect trace files and open a support ticket.
You can learn more at Troubleshoot issues when using the Azure Cosmos DB Emulator

Related

Enabling TLS in Elasticsearch

I'm having problems enabling TLS in Elasticsearch 7.1.1 running on Windows 7.
I have a single node with certificates created as
elasticsearch-certutil ca
elasticsearch-certutil cert --ca elastic-stack-ca.p12
The elasticsearch.yml file has the following settings
node.name: node1
discovery.type: single-node
xpack.security.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
xpack.security.transport.ssl.truststore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
This works fine but when I add the below
xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
xpack.security.http.ssl.truststore.path: 'C:\elasticsearch-7.1.1\config\certs\elastic-certificates.p12'
and start up elasticsearch I see the following error
[2019-06-25T07:34:19,659][WARN ][o.e.h.AbstractHttpServerTransport]
[node1] caught exception while handling client http traffic, closing
connection Netty4HttpChannel{localAddress=0.0.0.0/0.0.0.0:9200,
remoteAddress=/127.0.0.1:6757}
io.netty.handler.codec.DecoderException:
io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record:
This is repeated every 10-15 seconds.
https is enabled though and I can access the node using https://localhost:9200
I don't know why I receive the above error though as nothing else is running and accessing elasticsearch.
Any help would be much appreciated.
Thanks heaps
It was pointed out to me, on the elastic forum, that the above is a warning and not an error. I still couldn't understand what was causing it as I wasn't running any service or anything else that could be causing it but eventually found something called heartbeat that was running. This was obviously setup in an earlier version/previous installation of Kibana and this was still running, making a call using http and thus causing the above error (this is used for creating dummy data to use for/by to demo Kibana).
I came across this problem too. And IF you have previously installed Elastic Search then high chance you got some residual indices with "red" status, which makes the process of enabling TLS unsuccessful.
Try this command to verify your indices and their statuses
curl -XGET https://localhost:9200/_cat/indices
then delete those with red status.

tensorflow serving: failed to connect to 'ipv4:127.0.0.1:9000'

I have installed and configured tensorflow serving on an "AWS t2.large Ubuntu 14.04" server.
When I attempt to test the server with the mnist_client utility by executing the command, bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000, I receive the following error message:
E0605 05:03:54.617558520 1232 tcp_client_posix.c:191] failed to connect to 'ipv4:127.0.0.1:9000': timeout occurred
Any idea how to fix this?
I haven't heard of anything like this, but did note that (at least with other test clients) when the server was not ready/up yet, requests would timeout. So my guess is that the server is not up yet or perhaps in some other bad state.
I met the same problem before. The root cause is mnist_client was run in my local machine instead of the server, because the command connects to localhost bazel-bin/tensorflow_serving/example/mnist_client --num_tests=1000 --server=localhost:9000
It succeeds when I run mnist_client utility in the server.

The request failed with HTTP status 417: Expectation failed

Without getting into much detailed code
I have an 'kiosk' application that is running in about 500-800 different 'kiosk' at about 50 locations. Very simple application that connects to internet via a Verizon MIFI (2-3 MIFI per location). We believe that Verizon has made some changes to the network and now randomly I get
The request failed with HTTP status 417: Expectation failed
I have viewed The request failed with HTTP status 417: Expectation Failed - Using Web Services
and FB Connect: (417) Expectation failed
But you see I already had used
System.Net.ServicePointManager.Expect100Continue = false
in my code.
So one of the issues I have is the application isn't easy to test, and it will fail for 20-30 minutes or several days, then clears itself up.
Changing the config to include
<system.net>
<settings>
<servicePointManager expect100Continue="false" />
</settings>
Would be a large task, I don't know it that would even fix it. Since it is random I'm having troubles because I typically can't get it to fail in my office at my desk more than 1 time.
I happen to use VB and .Net for the application and services that run with the 'kiosk'.
The issue seems to be with the config on the mifi and not the Verizon network itself. We recently switched APNs and when a mifi connects to the Verizon network it is supposed to update automatically. Sometimes the mifi will fail to update the APN setting and that is when we get this error message. There are two ways I have found to fix this issue. The first and easier is to log into the mifi and manually update the setting. If you are dealing with a user who is not tech savvy and walking them through logging into the mifi will not work you can call the Verizon wireless enterprise help desk and have them remove the feature set from the mifi, add the features back, and then pull the battery from the mifi and power cycle it, this will make the mifi request the configuration settings again.

RUN#Cloud consistently throws me out during a heavy operation

I'm using a large app instance to run a basic java web application (GWT + Spring). There's an expensive operation within my application (report) which takes a long time to execute.
I've tried running it with the cloudbees SDK on my local machine with similar settings as it would be on the cloud and it seems to function just fine. It runs in about 3-4 minutes.
On the cloud, it seems to be taking longer. The problem isn't the fact that it takes long. What happens in that cloudbees terminates the session after 5 minutes and gives me an error in my browser saying 'Unable to connect to server. Please contact your administrator'. A report which doesn't take as long runs just fine. My application has a session timeout of 30 minutes, so that isn't a problem either.
What could possibly be going wrong? Is it something to do with cloudbees?
This may be due to proxy buffering of your request through the routing layer (revproxy) - so it most likely isn't a session timeout - but the http connection getting cut.
You can either set proxyBuffering=false via the bees CLI command (eg when you deploy the app) - this will ensure longer running connections can work.
Ideally, however, you could change the app slightly to return to the browser with some token which you can poll with to get completion status, as even with a connection that lasts that long, over the internet it may provide a bad experience vs locally.

Apache upload failed when file size is over 100k

Below it is some information about my problem.
Our Apache2.2 is on windows 2008 server.
Basically the problem is user fails to upload file which is bigger than 100k to our server.
The error in Apache log is: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. : Error reading request entity data, referer: ......
There were a few times (not always) I could upload larger files(100k-800k, failed for 20m) in Chrome. In FF4 it always fails for uploading file over 100k. In IE8 it is similar to FF4.
It seems that it fails to get request from client, then I reset TimeOut in Apache setting to default value(300) which did not help at all.
I do not have the RequestLimitBody option set up and I am not using PHP. Anyone saw the similar error before? Now I am not sure what I can try next. Any advise would be appreciated!
Edit:
I just tried to use remote desk to upload files on the server and it worked fine. First thought was about the firewall which however is off all the time, Http Proxy is applied though.