Unable to upload document with special character in AWS CloudSearch through Java SDK - amazon-cloudsearch

I am trying to upload a document which has special character in it. The JSON string is
[{
"type": "add",
"id": 1234,
"fields": {
"copyrightline": "© 2005 Some company. All Rights Reserved."
}
}]
When i remove '©' from the json, i am able to upload the document. When i have the character '©' the below is the error
AmazonCloudSearchDomainException: The request signature we calculated
does not match the signature you provided. Check your AWS Secret
Access Key and signing method. Consult the service documentation for
details. (Service: AmazonCloudSearchDomain; Status Code: 403; Error
Code: SignatureDoesNotMatch; Request ID:
d11a2497-aeac-11e9-b6fb-db6602f3004a)
Tried changing the encoding (UTF-8, UTF-16 and UTF-32) but with no success.
Here is the code which push the above string to CloudSearch
UploadDocumentsRequest uploadDocumentsRequest = new UploadDocumentsRequest();
InputStream inputStream = org.apache.commons.io.IOUtils.toInputStream(testDataString, "UTF-8");
uploadDocumentsRequest.setDocuments(inputStream);
uploadDocumentsRequest.setContentType(ContentType.Applicationjson);
uploadDocumentsRequest.setContentLength((long) testData.length());
UploadDocumentsResult uploadDocumentsResult = client.uploadDocuments(uploadDocumentsRequest);

Found out that the issue is with setContentLength(). If the length is invalid the error is thrown. So the following code change made sure it works.
uploadDocumentsRequest.setContentLength((long) cloudSearchAddRequest.getBytes("UTF-8").length);

Related

How to send image to Telegram Bot?

Trying to send image (or any type of file) to telegram bot.
It possible to take blob, convert it to Base64 and POST it?
I've tried to test it via SOAPUI but:
https://api.telegram.org/botXXX/sendDocument
{
"chat_id":XXX,
"document":"R0lGODlhKAAoAMIAAAAAAL6+vgCYAOfn5wAA/wAAAAAAAAAAACH5BAEAAAEALAAAAAAoACgAAAPYGLrc/jDKSau9OOvNu19AKIafIxJoioplqL4vyQEwIdw1AWw0fAu52w5D+xlxNeOw0rMdjb6fztJ0PpEp6FJSDQKjqS2kNxigygQ0Do1Om6fc89tNX5vbdPhYnmKb7Xl5Yg00eHSGfipogwyFb3eQh5GPZowgkpiJmgMClgqOmZOibp1xbmWop6h+q5ylEWStkJSzq0ITsX1pWTd3fa6elzkqR1HBwsNWWFnHIMtRUi+vTNFJzysZANe8Nc0PXdLdHQDgOeQt5ObnJY3k7uvs8fLz9PX29/EJADs="
}
In that case response:
{
"ok": false,
"error_code": 400,
"description": "Bad Request: wrong remote file identifier specified: Wrong character in the string"
}
You just had to send request with form-data)

Using Unnamed Parameter In Mikrotik Rest API

I'am making a script that can create and sign Certificate on Mikrotik using their REST API.
The script already worked in creating the Certificate, the problem is, signing it.
In Mikrotik Command Line, signing a certificate requires Unnamed Parameter, which i dont know how to specify it in the REST API Request.
Here is the detail about the Commands that translates to the REST API Request
Creating A Certificate (Worked)
CMD:
> /certificate add name=client-template-to-issue copy-from="client-template" common-name="$USERNAME#$CN"
REST:
URL: https://vpn.mydomain.com/rest/certificate/add
Body: {
"common-name": "$USERNAME#$CN",
"name":"client-template-to-issue",
"copy-from":"client-template"
}
Response: {
"ret": "*14"
}
Signing A Certificate (Diddnt Work)
CMD:
/certificate sign client-template-to-issue ca="$CN" name="$USERNAME#$CN"
REST:
URL: https://vpn.mydomain.com/rest/certificate/sign
Body: {
"ca": "$CN",
"name":"$USERNAME#$CN"
}
Response: {
"detail": "failure: At least one field specifying certificate name must be set!",
"error": 400,
"message": "Bad Request"
}
So how do i fix this? the command line suggest that there is a Unnamed Parameter to specify for it to work. i already tried with the URL like this
https://vpn.mydomain.com/rest/certificate/sign/*14
and it still doesnt work
The following worked for me:
URL: https://vpn.mydomain.com/rest/certificate/sign
Body: {
"ca": "$CN",
"number":"client-template-to-issue" # the "name" entry from your example of creating a certificate
}
I'm just not sure if this is a bug. Tested on ROS 7.7

SignatureDoesNotMatch on GET request to pre-signed URL (hellosign)

i am attempting to grab a pdf file for e-signature using pre-signed URLs. i use Amplify Storage to generate the URL. then i provide the URL to the HelloSign function to bring the file into the embedding window. i am able to open the document using window.open but this other method brings the SignatureDoesNotMatchThe request signature we calculated does not match the signature you provided. error.
The request signature we calculated does not match the signature you provided. Check your key and signing method.
key: string = 'bucketname/path/to/file.pdf'
expires: number = 60
const url = await Storage.get(key, {
level: 'public',
cacheControl: 'no-cache',
signatureVersion: 'v4',
ContentType: 'application/pdf',
expires: 60,
customPrefix: {
public: '',
protected: '',
private: '',
},
})
hellosign.open(url, {
clientId,
skipDomainVerification: true,
})
one fix i found on Github was to include the signatureVersion when initializing the S3 class instance using the SDK. i am not using the SDK however, so i tried providing it when configuring Amplify like so:
AWSS3: {
bucket,
region,
signatureVersion: 'v4',
},
needless to say it did not fix the issue. i could not find any references to this in the docs, as the Amplify.configure function takes any in their docs.
i tried clicking the pre-signed URL and was able to download the doc without issue. i inspected the request and saw the correct headers that match the outgoing request originating from hellosign.open. any ideas what i can try?
hellosign.open() is only expected to load embedded URLs. These would be URLs returned from requests to the following endpoints:
/embedded/sign_url/ (sign URL - embedded signing)
/template/create_embedded_draft (edit URL - embedded template creation)
/embedded/edit_url/ (edit URL - embedded template editing)
/unclaimed_draft/create_embedded (claim URL embedded requesting)
/unclaimed_draft/create_embedded_with_template (claim URL embedded requesting)
If you'd like to collect signatures on a document in an iframe on your site, embedded signing will likely be what you're looking for. In that case, you'd want to create your signature request with a request to:
/signature_request/create_embedded
and then use the URL you're creating via Amplify Storage as the value of the file_url[] request parameter.
Once you've created your request, locate the signature ID for your signer in the response object, and use that in a request to /embedded/sign_url/
This will return a sign URL that you can load using hellosign.open().

node S3 Object Storage Linode

Im trying to use the aws-sdk to acces my linode S3 compatible bucket, but everything I try doesn't work. Not sure what the correct endpoint should be? For testing purposes is my bucket set to public read/write.
const s3 = new S3({
endpoint: "https://linodeobjects.com",
region: eu-central-1,
accesKeyId: <accesKey>,
secretAccessKey: <secretKey>,
});
const params = {
Bucket: bucketName,
Key: "someKey",
Expires: 60,
};
const uploadURL = await s3.getSignedUrlPromise("putObject", params);
The error im getting
code: 'CredentialsError',
time: 2021-07-15T08:29:50.000Z,
retryable: true,
originalError: {
message: 'Could not load credentials from any providers',
code: 'CredentialsError',
time: 2021-07-15T08:29:50.000Z,
retryable: true,
originalError: {
message: 'EC2 Metadata roleName request returned error',
code: 'TimeoutError',
time: 2021-07-15T08:29:49.999Z,
retryable: true,
originalError: [Object]
}
}
}
It seems like a problem with the credentials of the environment that this code is executed in and not with the bucket permissions themselves.
The pre-signing of the URL is an operation that is done entirely locally. It uses local credentials (i.e., access key ID and secret access key) to create a sigv4 signature for the URL. This also means that whether or not the credentials used for signing the URL are valid is only checked at the moment the URL is used, and not at the moment of signing the URL itself.
The error simply indicates that from all the ways the SDK is trying to find credentials (more info here) it cannot find credentials it can use to sign the URL.
This might be unrelated, but according to the documentation, the endpoint should be the following: The endpoint URI to send requests to. The default endpoint is built from the configured region. The endpoint should be a string like 'https://{service}.{region}.amazonaws.com' or an Endpoint object. Which, in the code example above, is not the case.
You should set the endpoint to be eu-central-1.linodeobjects.com. When using Linode object storage the region is not determined by the endpoint that you use.

Hashicorp Vault API - Create User - unsupported path

Can we create a user in vault API using special characters. For example below POST url is used to create users.
POST : http://localhost:8200/v1/auth/userpass/users/myuser-1#beta_1.0$
Paylod :
{
"password": "myPassword",
"policies": "myuser-1#beta_1.0$",
"ttl": "120",
"max_ttl": "120"
}
Result :
{
"errors": [
"1 error occurred:\n\t* unsupported path\n\n"
]
}
the user myuser-1#beta_1.0$ contains special characters # _ $. I think the # $ chars should be encoded before passing to vault. However there is no information about url encoding in below documentation.
https://www.vaultproject.io/api/auth/userpass/index.html
Is encoding supported here or should it be replaced with other char before sending it to vault??
Note: Removing # $ chars, api works fine.