I have an existing Cassandra DSE cluster managed by Opscenter with a backup schedule of all keyspaces to a S3 bucket. I spinned up a fresh cluster (with a separate instance of Opscenter) to attempt a full restore of the backup and I'm facing several issues;
Firstly, it's impossible to trigger a restore from the opscenter UI, after selecting "restore from a different location", entering the location, selecting all keyspaces and confirming the restore operation, after some time I'm getting a dialog with what I assume is a JS error: cannot read property "message" of undefined. Looking at the request on the network tab, I see the following nginx response:
<html><head>
<meta http-equiv="content-type" content="text/html;charset=utf-8">
<title>502 Server Error</title>
</head>
<body text=#000000 bgcolor=#ffffff>
<h1>Error: Server Error</h1>
<h2>The server encountered a temporary error and could not complete your request.<p>Please try again in 30 seconds.</h2>
<h2></h2>
</body></html>
I looked carefully at the Opscenter documentation and couldn't find any timeout configuration that might correlate to this. As a plan B I attempted to trigger the restore operation from the command line (basically copying the request as curl and running it on the opscenter node), here I faced different issues. The first attempts failed after some time with errors similar to:
{"message": "Failure applying schema change <REDACTED_CREATE_TABLE_QUERY>, failed to completed because: com.datastax.driver.core.exceptions.InvalidQueryException: Unknown type <REDACTED_KEYSPACE>.<REDACTED_TABLE>. ", "type": "AlterSchemaFailed"}
I find this very curious, because again, this is a fresh cluster with no data whatsoever. To solve this I had to cqlsh into the cluster and manually create the keyspaces and tables it was complaining about (this happened 3 times).
After that I got a different error which required me to add "force_schema": true to the request. After doing that, I finally managed to get a normal response and I could see the restore process running on the UI. The restore itself failed after some time with different errors, such as:
Failed query to http://10.32.4.6:61621/v1/ops/restore/<BACKUP_NAME>/<KEYSPACE_NAME>?req-id=6925a343-9900-4765-b239-128175f4dd6d&parent_req=5fab0458-b377-4107-b899-b35a18762379 : org.httpkit.ProtocolException: No status
Related
I'm using Hyperledger Fabric and now I'm trying to make a backup of the current situation and restore it on a different computer.
I'm following the procedure found in hyperledger-fabric-backup-and-restore.
The main steps being:
Copy the crypto-config and the channel-artifacts directory
Copy the content of all peers and orderer containers
Modify the docker-compose.yaml to link containers volumes to the local directory where I have the backup copy.
Yet it's not working properly in my case: when I restart the network with ./byfn.hs up I first have all the containers correctly up and running then, whatever operation I try and execute on the channel (peer channel create, peer channel join, peer channel update) fails with error:
Error: got unexpected status: BAD_REQUEST -- error applying config update to existing channel 'mychannel': error authorizing update: error validating ReadSet: proposed update requires that key [Group] /Channel/Application be at version 0, but it is currently at version 1
Is there anything I should do which is not mentioned on hyperledger-fabric-backup-and-restore ?
I got the same error while trying to create a channel. Turning the "network down" and then "network up" solved my problem.
New to Amazon S3 usage.I get the following error when trying to access the file from Amazon S3 using a simple java method.
2016-08-23 09:46:48 INFO request:450 - Received successful response:200, AWS Request ID: F5EA01DB74D0D0F5
Caught an AmazonClientException, which means the client encountered an
internal error while trying to communicate with S3, such as not being
able to access the network.
Error Message: Unable to store object contents to disk: Read timed out
The exact lines of code worked yesterday.I was able to download 100% of 5GB file in 12 min. Today I'm in a better connected environment but only 2% or 3% of the file is downloaded and then the program fails.
Code that I'm using to download.
s3Client.getObject(new GetObjectRequest("mybucket", file.getKey()), localFile);
You need to set the connection timeout and the socket timeout in your client configuration.
Click here for a reference article
Here is an excerpt from the article:
Several HTTP transport options can be configured through the com.amazonaws.ClientConfiguration object. Default values will suffice for the majority of users, but users who want more control can configure:
Socket timeout
Connection timeout
Maximum retry attempts for retry-able errors
Maximum open HTTP connections
Here is an example on how to do it:
Downloading files >3Gb from S3 fails with "SocketTimeoutException: Read timed out"
I have Workflow Manager installation with an "OAuthS2SSecurityConfiguration" applied to the child workflows that I want to delete.
When I run any of the commands below, I get an "internal server error". Since I see nothing in the EventLogs or local logs, I can only assume that this is an authentication issues (I have deleted workflows in the past with no issue.)
var dcli = new WorkflowManagementClient("http://host:port/PubSubActivitiesSample")
dcli.Activities.Delete()
dcli.Workflows.Delete()
dcli.CurrentScope.Delete()
I've also tried overwriting the workflow with a null security configuration. I get an internal HTTP error on that too.
Finally I ran the following powershell command:
remove-wfscope -ScopeURI http://myhost:12291/PubSubActivitiesSample
I get an internal server error for this as well:
Remove-WFScope: An internal error occurred. For more details please see the server logs. HTTP headers received from the server: ...
I am an iphone/ipad developer, using objective c language, and I am
using couchDB for my application.
My issue is: if I delete my local couchDB (local database) or run for the first time,
I am getting the error:
OTHER: {'EXIT',{error,timeout,#Ref<0.0.0.415>}}
This is my workflow:
my application replicates to remote iriscouch
using xyz:a...#mmm.iriscouch.com/
databasename.
credentials are checked.
if the replication is success, everything works as expected.
if I reset my local couch database contents, and if I iterate the
above step.
'sometimes' I will get an error(mentioned below) and there will be no
more synchronization with the remote. and its hard to re-sync the application.
This is that error from log:
[info] [<0.140.0>] 127.0.0.1 - - GET /_replicator/_changes? feed=continuous&heartbeat=300000&since=1 200
1> OTHER: {'EXIT',{error,timeout,#Ref<0.0.0.506>}}
1> OTHER: {'EXIT',{error,timeout,#Ref<0.0.0.507>}}
1>
waiting for your response
Krishna.
Seems like some validation functions running at destination end but in this case, this is the message returning from an erlang process tree timing out. But it needs to restart by itself after some (probably 5) seconds.
Below it is some information about my problem.
Our Apache2.2 is on windows 2008 server.
Basically the problem is user fails to upload file which is bigger than 100k to our server.
The error in Apache log is: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. : Error reading request entity data, referer: ......
There were a few times (not always) I could upload larger files(100k-800k, failed for 20m) in Chrome. In FF4 it always fails for uploading file over 100k. In IE8 it is similar to FF4.
It seems that it fails to get request from client, then I reset TimeOut in Apache setting to default value(300) which did not help at all.
I do not have the RequestLimitBody option set up and I am not using PHP. Anyone saw the similar error before? Now I am not sure what I can try next. Any advise would be appreciated!
Edit:
I just tried to use remote desk to upload files on the server and it worked fine. First thought was about the firewall which however is off all the time, Http Proxy is applied though.