Multipart form upload of binary file using casperjs outside of state machine (can't use fill) - phantomjs

UPDATE 1: I've created a GIST with actual running code in a test jig to show exactly what I'm running up against. I've included working bot tokens (to a throw-away bot) and access to a telegram chat that the bot is already in, in case anyone wants to take a quick peek. It's
https://gist.github.com/pleasantone/59efe5f9d7f0bf1259afa0c1ae5a05fe
UPDATE 2: I've looked at the following articles for answers already (and a ton more):
https://github.com/francois2metz/html5-formdata/blob/master/formdata.js
PhantomJS - Upload a file without submitting a form
https://groups.google.com/forum/#!topic/casperjs/CHq3ZndjV0k
How to instantiate a File object in JavaScript?
How to create a File object from binary data in JavaScript
I've got a program written in casperjs (phantomjs) that successfully sends messages to Telegram via the BOT API, but I'm pulling my hair out trying to figure out how to send up a photo.
I can access my photo either as a file, off the local filesystem, or I've already got it as a base64 encoded string (it's a casper screen capture).
I know my photo is good, because I can post it via CURL using:
curl -X POST "https://api.telegram.org/bot<token>/sendPhoto" -F chat_id=<id> -F photo=#/tmp/photo.png
I know my code for connecting to the bot api from within capserjs is working, as I can do a sendMessage, just not a sendPhoto.
function sendMultipartResponse(url, params) {
var boundary = '-------------------' + Math.floor(Math.random() * Math.pow(10, 8));
var content = [];
for (var index in params) {
content.push('--' + boundary + '\r\n');
var mimeHeader = 'Content-Disposition: form-data; name="' + index + '";';
if (params[index].filename)
mimeHeader += ' filename="' + params[index].filename + '";';
content.push(mimeHeader + '\r\n');
if (params[index].type)
content.push('Content-Type: ' + params[index].type + '\r\n');
var data = params[index].content || params[index];
// if (data.length !== undefined)
// content.push('Content-Length: ' + data.length + '\r\n');
content.push('' + '\r\n');
content.push(data + '\r\n');
};
content.push('--' + boundary + '--' + '\r\n');
utils.dump(content);
var xhr = new XMLHttpRequest();
xhr.open("POST", url, false);
if (true) {
/*
* Heck, try making the whole thing a Blob to avoid string conversions
*/
body = new Blob(content, {type: "multipart/form-data; boundary=" + boundary});
utils.dump(body);
} else {
/*
* this didn't work either, but both work perfectly for sendMessage
*/
body = content.join('');
xhr.setRequestHeader("Content-Type", "multipart/form-data; boundary=" + boundary);
// xhr.setRequestHeader("Content-Length", body.length);
}
xhr.send(body);
casper.log(xhr.responseText, 'error');
};
Again, this is in a CASPERJS environment, not a nodejs environment, so I don't have things like fs.createReadableStream or the File() constructor.

Related

Convert PDF from Google Drive to Image and send to Telegram via Bot

Currently the step by step I use is as follows:
First step -> Create a PDF of the Page Jogos na TV from my spreadsheet:
function CreatePDF() {
var ss = SpreadsheetApp.getActive();
SpreadsheetApp.flush();
var theurl = 'https://docs.google.com/a/mydomain.org/spreadsheets/d/' +
'ID CODE TO SPREADSHEET' +
'/export?format=pdf' +
'&size=0' +
'&portrait=true' +
'&fitw=true' +
'&top_margin=0' +
'&bottom_margin=0' +
'&left_margin=0' +
'&right_margin=0' +
'&sheetnames=false&printtitle=false' +
'&pagenum=false' +
'&gridlines=false' +
'&fzr=FALSE' +
'&gid=' +
'ID CODE TO SPREADSHEET PAGE';
var token = ScriptApp.getOAuthToken();
var docurl = UrlFetchApp.fetch(theurl, { headers: { 'Authorization': 'Bearer ' + token } });
var pdfBlob = docurl.getBlob();
//...get token and Blob (do not create the file);
var fileName = ss.getSheetByName("Jogos na TV").getRange("A1").getValue();
//Access or create the 'PDF' folder;
var folder;
var folders = DriveApp.getFoldersByName("PDF");
if(folders.hasNext()) {
folder = folders.next();
}else {
folder = DriveApp.createFolder("PDF");
}
//Remove duplicate file with the same name;
var existing = folder.getFilesByName(fileName);
if(existing.hasNext()) {
var duplicate = existing.next();
if (duplicate.getOwner().getEmail() == Session.getActiveUser().getEmail()) {
var durl = 'https://www.googleapis.com/drive/v3/files/'+duplicate.getId();
var dres = UrlFetchApp.fetch(durl,{
method: 'delete',
muteHttpExceptions: true,
headers: {'Authorization': 'Bearer '+token}
});
var status = dres.getResponseCode();
if (status >=400) {
} else if (status == 204) {
folder.createFile(pdfBlob.setName(fileName));
}
}
} else {
folder.createFile(pdfBlob.setName(fileName));
}
}
Second Step -> Manually copy the PDF link created in Google Drive
Step Three -> I send the text with the PDF minature to my group on Telegram:
function EnviarTelegram(botSecret, chatId, photoUrl, caption) {
var response = UrlFetchApp.fetch("https://api.telegram.org/bot" + botSecret + "/sendPhoto?caption=" + encodeURIComponent(caption) + "&photo=" + encodeURIComponent(photoUrl) + "&chat_id=" + chatId + "&parse_mode=HTML");
}
The current formula for sending to Telegram via spreadsheet:
=EnviarTelegram("botSecret","chatId","Url to PDF","Programação de jogos na TV
"&TEXT('Jogos Hoje'!B1,"DD/MM/YYYY")&" e "&TEXT('Jogos Amanhã'!B1,"DD/MM/YYYY"))
The thumbnail created for the PDF is cut so the image sent to Telegram is also cut and the spreadsheets cannot be saved as an image, just document or PDF.
Is there any way to automatically convert PDF to image and be able to send to Telegram?
Issue and workarounds:
Unfortunately, there are no methods for directly converting the PDF format to the image data in the methods of Google Apps Script. So, in this case, I thought that it is required to use the workarounds for achieving your goal.
Workaround 1:
In this workaround, the external API is used. When you want to directly convert the PDF data to an image data, how about using an external API? Ref
You can see the sample script for this at this thread.
Workaround 2:
In this workaround, the range of sheet is exported as an image. When I saw your Spreadsheet, it seems that the data range of a sheet in Google Spreadsheet is exported as a PDF data. From this situation, as the other workaround, how about converting the range of Spreadsheet to an image?
You can see the sample script for this at this thread.
Workaround 3:
In this workaround, the PDF data is sent with sendDocument. Ref In this case, it seems that the data is required to be sent as multipart/form-data. The sample script is as follows.
Sample script:
var url = "https://api.telegram.org/bot" + botSecret + "/sendDocument?chat_id=" + chatId;
var blob = DriveApp.getFileById("### file ID of PDF file ###").getBlob();
var res = UrlFetchApp.fetch(url, {method: "post", payload: {document: blob}});
console.log(res.getContentText())
References:
PDF to PNG API
sendDocument
Related threads.
Convert a gdoc into image
How to copy a range from a spreadsheet as an image to Google Slides?

Incoming Express parameters are not the same as whats passed in

I have a strange issue where I have passed in parameters from a URL, into my Express server,
When I get the req.params.code & req.params.mode variables, they are different than what is passed in through the URL.
Allow me to show you...
Here is the Express code:
router.get('/verify/:user/:mode/:code', function(req,res){
console.log("STARTING VERIFICATION");
var code = req.params.code;
console.log('code: ' + code);
var user = req.params.user;
console.log('user: ' + user);
var mode = req.params.mode;
console.log('mode: ' + mode);
console.log('req.params: ' + JSON.stringify(req.params));
var regex = new RegExp(["^", req.params.user, "$"].join(""), "i");
console.log('REGEX: ' + regex);
var verified = false;
console.log('req.params: ' + req.params);
console.log('req.body: ' + req.body);
console.log("rx: "+ regex);
console.log('req.params.code: ' + req.params.code);
console.log('req.params.user: ' + req.params.user);
etc... etc... etc...
Here is the output in the console:
STARTING VERIFICATION
code: background-cycler.js
user: admin
mode: js
req.params: {"user":"admin","mode":"js","code":"background-cycler.js"}
REGEX: /^admin$/i
req.params: [object Object]
req.body: [object Object]
rx: /^admin$/i
req.params.code: background-cycler.js
req.params.user: admin
Here is the URL that is passed into the browser:
https://examplesite.com/verify/admin/sms/9484
I want to say that this code worked prior to dusting it off and moving an instance to google's cloud compute...
As you can see, the parameters passed in to the verify, code should be 9484 and mode should be sms. Instead i'm getting an unintended js filename, and a js mode instead.
UPDATE: As requested I added this within the Express route function:
console.log(req.originalUrl);
and I get this result:
/verify/admin/js/background-cycler.js
I can verify the URL that sent this was:
https://examplesite.com/verify/admin/sms/9484

Download from Azure Storage adding single quotes around filename

I'm working on a site hosted in Azure that has a download functionality. To reduce the load on our servers, the download is done use Shared Access Signatures. However, in Safari when downloading the file, the filename is wrapped in single quotes, as in myFile.txt downloads as 'myFile.txt'. This has made it so zips being downloaded have to be renamed by the client so the contents can be extracted.
Code for generated the Shared Access Signature is as follows:
CloudBlockBlob blob = container.GetBlockBlobReference(Helpers.StringHelper.TrimIfNotNull(blobName));
if (!blob.Exists())
{
return string.Empty;
}
var sasConstraints = new SharedAccessBlobPolicy();
sasConstraints.SharedAccessStartTime = DateTime.UtcNow.AddSeconds(-5);
sasConstraints.SharedAccessExpiryTime = DateTime.UtcNow.Add(duration);
sasConstraints.Permissions = SharedAccessBlobPermissions.Read | SharedAccessBlobPermissions.Write;
var headers = new SharedAccessBlobHeaders();
string filename = blobName;
if (filename.Contains("/"))
{
filename = blobName.Substring(blobName.LastIndexOf("/") + 1, blobName.Length - blobName.LastIndexOf("/") - 1);
}
headers.ContentDisposition = "attachment; filename='" + filename + "'";
//Generate the shared access signature on the blob, setting the constraints directly on the signature.
string sasBlobToken = blob.GetSharedAccessSignature(sasConstraints, headers);
//Return the URI string for the container, including the SAS token.
return blob.Uri + sasBlobToken;
This code has worked fine in Chrome, Firefox, and IE. Is there something I'm missing with the headers? The only one I'm modifying is content-disposition.
You should use double quotes for quoted strings in HTTP headers, as outlined in RFC2616.
So replace
headers.ContentDisposition = "attachment; filename='" + filename + "'";
with
headers.ContentDisposition = "attachment; filename=\"" + filename + "\"";

SharePoint 2010 Wiki Template Script Issue

I'm looking for a way to give my SharePoint users a way to create new wiki pages from an existing template. In the process of researching I found a great walkthrough that seems to fit the need (http://www.mssharepointtips.com/tip.asp?id=1072&page=2), but I'm having trouble getting it to work. The problem seems to lie in the assignment of a path to PATHTOWIKI-- if I use "/Weekly Update Wiki", the script returns an error of "There is no Web named '/Weekly Update Wiki'." If I use "Weekly Update Wiki" without the forward slash, I instead get an error of "There is no Web named '/sites/[parentSite]/[childSite]/Weekly Update Wiki/Weekly Update Wiki'."
Any ideas about what I'm not understanding here?
function myCreateProject() {
// Configure these for your environment
// include no slashes in paths
var PATHTOWIKI = "Weekly Update Wiki";
var PATHTOPAGES = "Pages";
// file name only for template page, no extension
var TEMPLATEFILENAME = "Template";
var myPathToWiki = encodeURIComponent(PATHTOWIKI);
var myPathToPages = PATHTOPAGES + "%2f";
var myTemplateFileName = encodeURIComponent(TEMPLATEFILENAME) + "%2easpx";
var EnteredProject = document.getElementById("NewProjName");
var myNewName = EnteredProject.value;
if(myNewName == "") {
alert('Please enter a name for the new project page');
} else {
myNewName = encodeURIComponent(myNewName) + "%2easpx"
$.ajax({
url: PATHTOWIKI + "/_vti_bin/_vti_aut/author.dll",
data: ( "method=move+document%3a14%2e0%2e0%2e4730&service%5fname="
+ myPathToWiki +
"&oldUrl=" + myPathToPages + myTemplateFileName +
"&newUrl=" + myPathToPages + myNewName +
"&url%5flist=%5b%5d&rename%5foption=nochangeall&put%5foption=edit&docopy=true"
),
success: function(data) {
var rpcmsg1 = getMessage(data, "message=", "<p>");
$("#myInfo").append("<br />" + rpcmsg1);
if(rpcmsg1.indexOf("successfully") < 0) {
// get error info
var rpcmsg2 = getMessage(data, "msg=", "<li>");
$("#myInfo").append("<br />" + rpcmsg2 + "<br />");
} else {
$("#myInfo").append("<br />Go to new page<br />");
}
},
type: "POST",
beforeSend: function(XMLHttpRequest) {
XMLHttpRequest.setRequestHeader("X-Vermeer-Content-Type",
"application/x-www-form-urlencoded");
}
});
}
}
Update: I figured out what needed to happen in my case. Since I couldn't get a grasp on the relative approach, I just went with the absolute path for PATHTOWIKI and slightly modified the append in the ajax call.
PATHTOWIKI:
var PATHTOWIKI = "https://[domain]/sites/[parentSite]/[childSite]";
append:
$("#myInfo").append("<br />Go to new page<br />");
The change in the latter line of code is subtle; since I used an absolute path in PATHTOWIKI, I just removed the leading forward slash in the anchor tag, so that <a href=\"/" became <a href=\"". This renders the script slightly less portable, but since it's a one-off effort I'll stick with this unless anything comes along to expand the scope.

IE crossdomain filter on flex application

I have an application that uses a flex form to capture user input. When the user has entered the form data (which includes a drawing area) the application creates a jpg image of the form and sends back to the server. Since the data is sensitive, it has to use https. Also, the client requires both jpg and pdf versions of the form to be stored on the server.
The application sends data back in three steps
1 - send the jpg snapshot with ordernumber
2 - send the form data fields as post data so it is not visible in the address bar
3 - send the pdf data
I am sending the jpg data first using urlloader and waiting for the server to respond before performing opperation 2 and 3 to ensure that the server has created the record associated with the new orderNumber.
This code works fine in IE over http. But If I try to use the application over https, IE blocks the page response from store jpg step and the complete event of the urlloader never fires. The application works fine in FireFox over http or https.
Here is the crossdomain.xml (I have replaced the domain with ""):
<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd">
<cross-domain-policy>
<allow-access-from domain="*.<mydomain>.com" to-ports="*" secure="false"/>
<allow-http-request-headers-from domain="*.<mydomain>.com" headers="*">
</cross-domain-policy>
Here is the code that is executed when the user presses the submit button:
private function loaderCompleteHandler(event:Event):void {
sendPDF();
sendPatientData();
}
private function submitOrder(pEvt:MouseEvent):void
{
//disable submit form so the order can't be submitted twice
formIsValid = false;
waitVisible = true;
//submit the jpg image first with the order number, userID, provID
//and order type. The receiveing asp will create the new order record
//and save the jpg file. jpg MUST be sent first.
orderNum = userID + "." + provID + "." + Date().toString() + "." + orderType;
var jpgURL:String = "https://orders.mydomain.com/orderSubmit.asp?sub=jpg&userID=" + userID + "&provID=" + provID + "&oNum=" + orderNum + "&oType=" + orderType;
var jpgSource:BitmapData = new BitmapData (vbxPrint.width, vbxPrint.height);
jpgSource.draw(vbxPrint);
var jpgEncoder:JPEGEncoder = new JPEGEncoder(100);
var jpgStream:ByteArray = jpgEncoder.encode(jpgSource);
var header:URLRequestHeader = new URLRequestHeader ("content-type", "application/octet-stream");
//Make sure to use the correct path to jpg_encoder_download.php
var jpgURLRequest:URLRequest = new URLRequest (jpgURL);
jpgURLRequest.requestHeaders.push(header);
jpgURLRequest.method = URLRequestMethod.POST;
jpgURLRequest.data = jpgStream;
//navigateToURL(jpgURLRequest, "_blank");
var jpgURLLoader:URLLoader = new URLLoader();
try
{
jpgURLLoader.load(jpgURLRequest);
}
catch (error:ArgumentError)
{
trace("An ArgumentError has occurred.");
}
catch (error:SecurityError)
{
trace("A SecurityError has occurred.");
}
jpgURLLoader.addEventListener(Event.COMPLETE, loaderCompleteHandler);
}
private function sendPatientData ():void
{
var dataURL:String = "https://orders.mydomain.com/orderSubmit.asp?sub=data&oNum=" + orderNum + "&oType=" + orderType;
//Make sure to use the correct path to jpg_encoder_download.php
var dataURLRequest:URLRequest = new URLRequest (dataURL);
dataURLRequest.method = URLRequestMethod.POST;
var dataUrlVariables:URLVariables = new URLVariables();
dataUrlVariables.userID = userID
dataUrlVariables.provID = provID
dataUrlVariables.name = txtPatientName.text
dataUrlVariables.dob = txtDOB.text
dataUrlVariables.contact = txtPatientContact.text
dataUrlVariables.sex=txtSex.text
dataUrlVariables.ind=txtIndications.text
dataURLRequest.data = dataUrlVariables
navigateToURL(dataURLRequest, "_self");
}
private function sendPDF():void
{
var url:String = "https://orders.mydomain.com/pdfOrderForm.asp"
var fileName:String = "orderPDF.pdf&sub=pdf&oNum=" + orderNum + "&oType=" + orderType + "&f=2&t=1" + "&mid=" + ModuleID.toString()
var jpgSource:BitmapData = new BitmapData (vbxPrint.width, vbxPrint.height);
jpgSource.draw(vbxPrint);
var jpgEncoder:JPEGEncoder = new JPEGEncoder(100);
var jpgStream:ByteArray = jpgEncoder.encode(jpgSource);
myPDF = new PDF( Orientation.LANDSCAPE,Unit.INCHES,Size.LETTER);
myPDF.addPage();
myPDF.addImageStream(jpgStream,0,0, 0, 0, 1,ResizeMode.FIT_TO_PAGE );
myPDF.save(Method.REMOTE,url,Download.ATTACHMENT,fileName);
}
The target asp page is not sending back any data, except the basic site page template.
Can anyone help me figure out how to get around this IE crossdomain issue? I have turned off the XSS filter in IE tools security settings, but that still didn't solve the problem.
THANKS
Do everything over https. Load the swf from an https url. Send the initial form post via https. Send the images via https.