I am trying to integrate 2 webapps to let one (new one) send a url to another (old one) that it will load into a text area. The URL is actually to a file in S3 (with the multiple redirects that come with that) and I am trying to do this in the receiver using an XMLHTTPRequest like so;
var client = new XMLHttpRequest();
client.withCredentials = true;
client.open('GET', geneIdFileUrl, true );
client.setRequestHeader("Content-type", "text/plain");
client.onreadystatechange = function() {
document.getElementById('geneList').value = client.responseText;
}
client.send();
This is fine in firefox but in webkit browsers (chrome, safari) the request is being sent out with the request header
Origin null
instead of the real origin of the page making the request. The get from S3 is coming as a ContentDisposition attachment so I can't just dump it into an iframe and read the contents from there.
Why do the webkit browsers not set the origin properly and is it possible to coecre them into doing so?
Related
I have a strange issue relating to S3 signed URL
I want to download the file from S3 on my browser. Every file type worked as expected, except the image files. I do not know why
Here is my javascript
<html>
<script>
fetch('<s3 signed url>', {
method: 'GET',
// For the image file, I always got the CORS error but for other file types, it works as expected
// mode: 'no-cors',
})
.then((res) => {
return res.blob();
})
.then((blob) => {
var url = window.URL.createObjectURL(new Blob([blob]));
var a = document.createElement('a');
a.href = url;
a.download = 'file.png';
document.body.appendChild(a);
a.click();
});
</script>
If I generated a signed URL for pdf or doc ... then download it with the above code, it works
But if I generated a signed URL for an image file and then download it with the above code, it does not work.
I always got this error in the console
Access to fetch at 'https://.......' from origin 'null' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Then I added the
{mode: 'no-cors'}
Then it works but the context of the file is always empty (ZERO bytes)
Why? how can I download an image from S3?
Did you try config CORS on S3 bucket policy?
https://docs.aws.amazon.com/AmazonS3/latest/userguide/ManageCorsUsing.html
I make GET - http request and in most TVs, which I test, everything is fine, but I have some TVs that they display "NETWORK ERROR". Of course the TVs are connected to the network and the server is working (meaning the URL is correct. Is exactly the same code in all TV sets in any case). The strangest thing is that these TVs (with the error) have access to the Internet, e.g. YouTube works well.
function httpGet(theUrl)
{
var xmlHttp = new XMLHttpRequest();
xmlHttp.open( "GET", theUrl, false ); // false for synchronous request
xmlHttp.send( null );
}
I don't care for the xmlHttp.responseText
I tried hard reset with no luck.
The approach using:
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange = function() {
// callback function implementation
};
xmlhttp.open("GET", url, true);
xmlhttp.send();
always works for us.
Perhaps some browser security feature is blocking you, such as CORS or XSS or HTTPS or ...?
You can try downloading a static image from the same origin to confirm.
I have Cloudfront in front of S3 serving images (png and jpg).
I have all png and jpg images in webp format in the same directory with .webp extension. For example:
png: /path/to/file.png
webp: /path/to/file.png.webp
I'd like to serve the webp file dynamically without changing the markup.
Since browsers flag webp support via Accept header, what i need to do is: if the user has support for webp (via Accept header) Cloudfront would pull the webp version (filename.png.webp), if not it should serve the original file (filename.png)
Is this possible to achieve?
Making Cloudfront serve different resources is easy (when you have done it a couple of times), but my concern is whether the entity making the request (i.e. browser) and possible caching elements between (proxies etc) expects to have different media types on the same request URI. But that is a bit beyond your question. I believe the usual way to handle this problem is with a element where the browser is free to choose an image from different media types like this:
<picture>
<source type="image/svg+xml" srcset="pyramid.svg" />
<source type="image/webp" srcset="pyramid.webp" />
<img
src="pyramid.png"
alt="regular pyramid built from four equilateral triangles" />
</picture>
But if you still want to serve different content from Cloufront for the same URL this is how you do it:
Cloudfront has 4 different points where you can inject a lamdba function for request manipulation (Lambda#Edge).
For your use case we need to create a Lambda#Edge function at the Origin Request location then associate this function with your Cloudfront Distribution.
Below is an example from AWS docs that looks on device type and does URL manipulation. For your use case, something similar can be done by looking at the "Accept" header.
'use strict';
/* This is an origin request function */
exports.handler = (event, context, callback) => {
const request = event.Records[0].cf.request;
const headers = request.headers;
/*
* Serve different versions of an object based on the device type.
* NOTE: 1. You must configure your distribution to cache based on the
* CloudFront-Is-*-Viewer headers. For more information, see
* the following documentation:
* https://docs.aws.amazon.com/console/cloudfront/cache-on-selected-headers
* https://docs.aws.amazon.com/console/cloudfront/cache-on-device-type
* 2. CloudFront adds the CloudFront-Is-*-Viewer headers after the viewer
* request event. To use this example, you must create a trigger for the
* origin request event.
*/
const desktopPath = '/desktop';
const mobilePath = '/mobile';
const tabletPath = '/tablet';
const smarttvPath = '/smarttv';
if (headers['cloudfront-is-desktop-viewer']
&& headers['cloudfront-is-desktop-viewer'][0].value === 'true') {
request.uri = desktopPath + request.uri;
} else if (headers['cloudfront-is-mobile-viewer']
&& headers['cloudfront-is-mobile-viewer'][0].value === 'true') {
request.uri = mobilePath + request.uri;
} else if (headers['cloudfront-is-tablet-viewer']
&& headers['cloudfront-is-tablet-viewer'][0].value === 'true') {
request.uri = tabletPath + request.uri;
} else if (headers['cloudfront-is-smarttv-viewer']
&& headers['cloudfront-is-smarttv-viewer'][0].value === 'true') {
request.uri = smarttvPath + request.uri;
}
console.log(`Request uri set to "${request.uri}"`);
callback(null, request);
};
Next you need to tell Cloudfront that you want to use the Accept header as a part of your cache key (otherwise Cloudfront would only execute your Origin Request lambda once and also not expose this header to your function).
You do this nowadays with cache and origin request policies. Or with legacy settings (Edit Behaviour under your Cloudfront distribution settings) such as:
Worth to note here is that, if you get low cache hit ratio due to different variants of the Accept header you need to pre-process / clean it. The way I would do it is with a Viewer Request Lamdba that gets executed for each request. This new Lambda would then check if the Accept header supports Webp and then add a single NEW header to the request that it passes on to the Origin Request above. That way the Origin Request can cache on this new header (which only has two different possible values)
There's more config/setup needed such as IAM policies to get Lamdba to run etc, but there's lots of great material out there that walks you through the steps. Maybe start here?
I'm developping an angular2 application (single page application). My page is never "reloaded", but it's content changes according to user interactions.
I'm having some cache problems especially with images.
Context :
My page contains an editable image list :
<ul>
<li><img src="myImageController/1">Edit</li>
<li><img src="myImageController/2">Edit</li>
<li><img src="myImageController/3">Edit</li>
</ul>
When i want to edit an image (Edit link), my dom content is completly changed to show another angular component with a fileupload component.
The myImageController returns the LastModified header, and cache-control : no-cache and must-revalidate.
After a refresh (hit F5), my page does a request to get all img src, which is correct : if image has been modified, it is downloaded, if not, i just get a 304 which is fine.
Note : my images are stored in database as blob fields.
Problem :
When my page content is dynamically reloaded with my single page app, containing img tags, the browser do not call a GET http request, but immediatly take image from cache. I assume this a browser optimization to avoid getting the same resource on the same page multiple times.
Wrong solutions :
The first solution is to add something like ?time=(new Date()).getTime() to generate unique urls and avoid browser cache. This won't send the If-Modified-Since header in the request, and i will download my image every time completly.
Do a "real" refresh : the first page load in angular apps is quite slow, and i don't to refresh all.
Tests
To simplify the problem, i trying to create a static html page containing 3 images with the exact same link to my controller : /myImageController/1. With the chrome developper tool, i can see that only one get request is called. If i manage to get mulitple server calls in this case, it would probably solve my problem.
Thank you for your help.
5th version of HTML specification describes this behavior. Browser may reuse images regardless of cache related HTTP headers. Check this answer for more information. You probably need to use XMLHttpRequest and blobs. In this case you also need to consider Same-origin policy.
You can use following function to make sure user agent performs every request:
var downloadImage = function ( imgNode, url ) {
var xhr = new XMLHttpRequest();
xhr.open("GET", url, true);
xhr.responseType = "blob";
xhr.onreadystatechange = function () {
if (xhr.readyState == 4) {
if (xhr.status == 200 || xhr.status == 304) {
var blobUrl = URL.createObjectURL(xhr.response);
imgNode.src = blobUrl;
// You can also use imgNode.onload callback to release blob resources.
setTimeout(function () {
URL.revokeObjectURL(blobUrl);
}, 1000);
}
}
};
xhr.send();
};
For more information check New Tricks in XMLHttpRequest2 article by Eric Bidelman, Working with files in JavaScript, Part 4: Object URLs article by Nicholas C. Zakas and URL.createObjectURL() MDN page and Same-origin policy MDN page.
You can use the random ID trick. This changes the URL so that the browser reloads the image. Not that this can be done in the query parameters to force a full cache break or in the hash to allow the browser to re-validate the image from the cache (and avoid re-downloading it if unchanged).
function reloadWithCache(img: HTMLImageElement, url: string) {
img.src = url.replace(/#.*/, "") + "#" + Math.random();
}
function reloadBypassCache(img: HTMLImageElement, url: string) {
let sep = img.indexOf("?") == -1? "?" : "&";
img.src = url + sep + "nocache=" + Math.random()
}
Note that if you are using reloadBypassCache regularly you are better off fixing your cache headers. This function will always hit your origin server leading to higher running costs and making CDNs ineffective.
My app uses OneDrive API feature to let unauthorized users get thumbnails for shared OneDrive files using old API request:
https:// apis.live.net/v5.0/skydrive/get_item_preview?type=normal&url=[shared_link_to_OneDrive_file]
This feature is broken now (any such links return XMLHttpRequest connection error 0x2eff).
And my Windows Store app can not longer provide this feature.
Anyone can try to check it, link to shared OneDrive file:
https://onedrive.live.com/redir?resid=AABF0E8064900F8D!27202&authkey=!AJTeSCuaHMc45eY&v=3&ithint=photo%2cjpg
links to a preview image for shared OneDrive file (according to old OneDrive API
"Displaying a preview of a OneDrive item" - https:// msdn.microsoft.com/en-us/library/jj680723.aspx):
https://apis.live.net/v5.0/skydrive/get_item_preview?type=normal&url=https%3A%2F%2Fonedrive.live.com%2Fredir%3Fresid%3DAABF0E8064900F8D!27202%26authkey%3D!AJTeSCuaHMc45eY%26v%3D3%26ithint%3Dphoto%252cjpg
generates error: SCRIPT7002: XMLHttpRequest: Network error 0x2eff
Сurrent OneDrive API thumbnail feature:
GET /drive/items/{item-id}/thumbnails/{thumb-id}/{size}
is just for authorized users and can not provide access to thumbnails for shared OneDrive files to unauthorized users
How can a Windows Store app let unauthorized users get thumbnails for shared OneDrive files (videos etc.) using the current OneDrive API?
Any ideas?
You need to make a call to the following API:
GET /drive/items/{item-id}/thumbnails/{thumb-id}/{size}/content
This call needs to use authorization and returns a redirect to a cache-safe thumbnail location. You can then use this new url to serve thumbnails to unauthenticated users.
e.g.
Request:
GET https://api.onedrive.com/v1.0/drive/items/D094522DE0B10F6D!152/thumbnails/0/small/content
Authorization: bearer <access token>
Response:
HTTP/1.1 302 Found
Location: https://qg3u2w.bn1302.livefilestore.com/y3m1LKnRaQvGEEhv_GU3mVsewg_-aizIXDaVczGwGFIqtNcVSCihLo7s2mNdUrKicuBnB2sGlSwMQTzQw7v34cHLkchKHL_5YC3IMx1SMcpndtdb9bmQ6y2iG4id0HHgCUlgctvYsDrE24XALwXv2KWRUwCCvDJC4hlkqYgnwGBUSQ
You can now use the link in the Location header to access the thumbnail without signing in. This url will change only if the contents of the file change.
You can read more in the documentation here.
I just figured it out. It is based on the information in this article from Microsoft...
https://learn.microsoft.com/en-ca/onedrive/developer/rest-api/api/driveitem_list_thumbnails?view=odsp-graph-online
... look at the section, "Getting thumbnails while listing DriveItems." It shows you the relevant JSON return structure from a call such as:
GET /me/drive/items/{item-id}/children?$expand=thumbnails
Basically, the JSON return structure gives you string URL's for each of the thumbnail formats. You then create URLSession's to upload these URL's (once you've converted them from String to URL)
Here is an excerpt of code using Swift (Apple):
////////////////////////////////////////////////////////////////////////////////
//
// Download a thumbnail with a URL and label the URLSession with an ID.
//
func downloadThumbnail(url: URL, id: String) {
// Create a URLSession. This is an object that controls the operation or flow
// control with respect to asynchronous operations. It sets the callback delegate
// when the operation is complete.
let urlSession: URLSession = {
//let config = URLSessionConfiguration.default
let config = URLSessionConfiguration.background(withIdentifier: id)
config.isDiscretionary = true
config.sessionSendsLaunchEvents = true
//config.identifier = "file download"
return URLSession(configuration: config, delegate: self as URLSessionDelegate, delegateQueue: OperationQueue.main)
}()
// Create the URLRequest. This is needed so that "Authorization" can be made, as well
// as the actual HTTP command. The url, on initialization, is the command... along
// with the "GET" setting of the httpMethod property.
var request = URLRequest(url: url)
// Set the Authorization header for the request. We use Bearer tokens, so we specify Bearer + the token we got from the result
request.setValue("Bearer \(self.accessToken)", forHTTPHeaderField: "Authorization")
request.httpMethod = "GET"
request.setValue("application/json", forHTTPHeaderField: "Content-Type")
// This initiates the asynchronous data task
let backgroundTask = urlSession.downloadTask(with: request)
//backgroundTask.earliestBeginDate = Date().addingTimeInterval(60 * 60)
backgroundTask.countOfBytesClientExpectsToSend = 60
backgroundTask.countOfBytesClientExpectsToReceive = 15 * 1024
backgroundTask.resume()
}
... of course you need to have the correct "accessToken," (shown above) but you also have to have written the generic callback function for URLSession, which is:
func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask,
didFinishDownloadingTo location: URL) {
Swift.print("DEBUG: urlSession callback reached")
// This was the identifier that you setup URLSession with
let id = session.configuration.identifier
// "location" is the temporary URL that the thumbnail was downloaded to
let temp = location
// You can convert this URL into any kind of image object. Just Google it!
}
Cheers, Andreas