Cloud Flare, R2, how to upload images? - cloudflare

Cloud Flare, R2, how to upload images??
I`m new to Cloud Flare world,
and I can upload the pictures by dragging but
how to upload image using coding? from application??
do I have to use "WORKERS" <-- things?

I have uploaded files to r2 successfully with rclone.
Configure rclone
First, install rclone on our PC.
And then create a rclone.conf file under the path ~/.config/rclone/.
[r2]
type = s3
provider = Cloudflare
access_key_id = <ACCESS_KEY>
secret_access_key = <SECRET_ACCESS_KEY>
region = auto
endpoint = https://<ACCOUNT_ID>.r2.cloudflarestorage.com
acl = private
[r2]: A custom name(an alias) for storage service. We need to use it to operate files.
type = s3: The type of file operation API. R2 supports the S3 standard protocol.
provider = Cloudflare: The storage provider ID. You could use man rclone to get the supported providers.
access_key_id: You need to create a token with edit permission on the R2 console.
secret_access_key: Same as above.
endpoint: The url that rclone uses to operate files. To get the account id on the top-right of R2 homepage.
Usage
run rclone lsf r2: to see your buckets and rclone lsf r2:my-bucket to show the file list within a bucket.
Especially notice the last symbol :
upload a file:
rclone copy /path/to/file r2:my-bucket

Hey I got stuck with this for two days and was not able to figure it out. So I wanted to share my solution.
My Goal
I was developing a software for collecting data from our end users to get information from them. But they needed an image to be submitted for us to verify them, that is why I needed to have a form to enable users to upload their file and an API endpoint to store their file.
Solution
I set up an Cloudflare worker as an API since it connects well with R2, you just have to define it in your worker settings.
My Cloudflare worker script & Example form for allowing users to upload files
// CLOUDFLARE WORKER SCRIPT
// ------------------------
export default {
async fetch(request, env) {
const corsHeaders = {
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Methods': 'GET, HEAD, POST, OPTIONS',
'Access-Control-Max-Age': '86400',
};
// Check for preflight request from the browser.
if (request.method === 'OPTIONS') {
return new Response(null, {
headers: {
...corsHeaders,
'Access-Control-Allow-Headers': request.headers.get(
'Access-Control-Request-Headers'
),
}
});
} else {
// Handle actual request and store image to bucket.
const { headers } = request;
// Key is date now since we want keys to be unique.
const key = Date.now();
await env.MY_BUCKET.put(key, request.body, {
httpMetadata: {
contentType: headers.get('content-type')
}
});
return new Response('success!', {
headers: {
...corsHeaders,
'Access-Control-Allow-Headers': request.headers.get(
'Access-Control-Request-Headers'
),
}
});
}
}
}
<!DOCTYPE html>
<html>
<head>
<title>Upload Images with Cloudflare R2</title>
</head>
<body>
<form method="POST" enctype="multipart/form-data">
<label for="image">Select image to upload:</label>
<input type="file" name="image" id="image" /><br /><br />
<input type="submit" value="Upload" />
</form>
<script>
async function uploadImage(file) {
fetch('https://<YOUR-OWN-WORKER>.workers.dev', {
method: 'POST',
headers: {
'Access-Control-Allow-Origin': '*',
'Content-Type': file.type
},
body: file
})
.then((response) => response.text())
.then((data) => console.log(data))
.catch((error) => console.error(error));
}
const image = document.getElementById('image');
const onSelectFile = () => uploadImage(image.files[0]);
image.addEventListener('change', onSelectFile, false);
</script>
</body>
</html>

Related

When trying to download certain video it redirects to a new URL and the video starts to play. "disposable content type" is not received from server

I want to download certain videos with a click. For that, I created a Button and attached a Function that should trigger the associated video download.
But I am only able to download the link of the video, not the video. I am able to download videos with an external downloader or simply drag the URL to the download section of the browser. But unable to trigger that activity via JavaScript. Please help Me.
I tried multiple ways to tackle this problem:
Using a Simple Blob Technique without Axios:
const blob = new Blob([this.src_url], { type: 'video/mp4' })
const link = document.createElement('a')
link.href = URL.createObjectURL(blob)
link.download = this.src_url.replace(
>! // 'https://redis-test.com/videos/',
link.click()
URL.revokeObjectURL(link.href)
endpoint: video URL get downloaded as a file of 122 bytes
Then using File Saver Package:
var FileSaver = require('file-saver')
console.log(this.src_url)
var blob = new Blob([this.src_url], { type: 'video/mp4' })
FileSaver.saveAs(blob, 'hello world.mp4')
Then using the form method:
<form method="get" action="file.doc">
<button type="submit">Download!</button>
</form>
endpoint: video starts to play in the same window
Using href download attribute:
function download(url) {
const a = document.createElement('a')
a.href = url
a.download = url.split('/').pop()
document.body.appendChild(a)
a.click()
document.body.removeChild(a)
}
endpoint: video starts to play in the same window
Using your method:
const link = document.createElement('a')
link.href = url
link.click()
endpoint: video starts to play in the same windows
With Axios defaults now:
axios.defaults.withCredentials = true
window.open(
'https://cdn.pixaandom_urlrbay.com/vieo/487508532/Woman%20-%2058142.mp4?rendition=source&expiry=1666842719&hash=7dd6d178d9dbbd8adaf68dafd80c9167e91eca21&download'
)
endpoint: video starts to play in the new window
With attaching disposable content type in headers with AXIOS:
axios
.get(
String(nuxtConfig.axios.mediaURL) +
this.src_url.replace(
'https://redisrandom_url.com/videos/',
''
),
{
headers: {
mode: 'no-cors',
referrerPolicy: 'no-referrer',
'Content-Disposition': 'attachment; filename=Woman - 58142.mp4',
Host: 'redis-nfs',
'User-Agent': 'PostmanRuntime/7.29.2',
Accept: '*/*',
'Accept-Language': 'en-US,en;q=0.5',
'Accept-Encoding': 'gzip, deflate, br',
Connection: 'keep-alive',
Cookie:
'tk_or=%22https%3A%2F%2Fwww.google.com%2F%22; tk_lr=%22https%3A%2F%2Fwww.google.com%2F%22; _gcl_au=1.1.954672920.1660108804; _ga=GA1.2.1392122600.1660108808; _fbp=fb.1.1660108809200.1970395787',
'Upgrade-Insecure-Requests': '1',
'Sec-Fetch-Dest': 'document',
'Sec-Fetch-Mode': 'navigate',
'Sec-Fetch-Site': 'none',
'Sec-Fetch-User': '?1',
Pragma: 'no-cache',
'Cache-Control': 'no-cache',
},
}
)
.then((response) => {
console.log(response)
const url = window.URL.createObjectURL(new Blob([response.data]))
const link = document.createElement('a')
link.href = url
link.setAttribute('download', 'title')
document.body.appendChild(link)
link.click()
})
.catch((error) => {
console.log('rex')
})
endpoint: Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at redis-random_url/videos/be319-72e1-2e79-8dc3-bcef1/…. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing). Status code: 200
"...But I am only able to download the link of the video, not the video."
I don't use VueJS but I suspect this.src_url is just text of the path to video URL.
In HTML5 you can only download those files that exist on your server. If the file is external then you need a PHP script (on same server as your HTML file) to read those external bytes back into your JS buffer array.
const blob = new Blob([this.src_url], { type: 'video/mp4' })
Should be:
let myBytes = //# update variable with data result of reading files bytes
let myBlob = new Blob( [ Uint8Array.from( myBytes ) ] , {type: "application/octet-stream"} );
Where the bytes reading can be done with FileReader API or Fetch API.
When you can read a file's bytes into an Array using VueJS then your problem is solved.

Auth info required to subscribe to private-App.User.undefined using Laravel, Vue and Pusher

I am trying to send a notification that broadcast on a private channel using pusher with laravel api. Everything is ok while sending notification and pusher server is receiving the event. However, when I try to listen to those events nothing display in the console. The error logs in pusher shows the following:
Here is my code of main.js to listen to broadcast
let userId=document.head.querySelector('meta[name="user-id"').content;
console.log(userId)
window.Echo.private('App.User.' + userId)
.notification((notification) => {
console.log(notification.type);
});
This is my code of bootstrap.js
import Echo from 'laravel-echo';
window.Pusher = require('pusher-js');
window.Echo = new Echo({
broadcaster: 'pusher',
key: process.env.MIX_PUSHER_APP_KEY,
cluster: process.env.MIX_PUSHER_APP_CLUSTER,
forceTLS: true,
});
This is my meta tags from welcome.blade.php
<meta name="csrf-token" content="{{ csrf_token() }}">
<meta name="user-id" content="{{Auth::check() ? Auth::user()->id:''}}">
This is my channels.php file code
Broadcast::channel('App.User.{id}', function ($user, $id) {
return (int) $user->id === (int) $id;
},['guards'=>['api']]);
Finally the boot function from BroadcastServiceProvider.php
public function boot()
{
Broadcast::routes(['middleware' => ['auth:api']]);
require base_path('routes/channels.php');
}
Any help would highly appreciated.Thank in advance.
I had the same problem but using Djago and Javascript. My suggestion is:
make sure the URL that Pusher uses for private channels authentication is working
"Enable client events" from "App settings" is ON
I'm grappling with the same issue. From what I've read, if you're using api, you need to add authentication headers to the echo instance in your bootstrap.js file like so:
window.Echo = new Echo({
broadcaster: 'pusher',
key: process.env.MIX_PUSHER_APP_KEY,
cluster: process.env.MIX_PUSHER_APP_CLUSTER,
forceTLS: true,
auth:{
headers: {
Accept: 'application/json',
Authorization: 'Bearer ',
},
},
});
Try this out.

Display PDF from azure blob in browsers using Microsoft Azure Storage SDK for Node.js and JavaScript for Browsers

I am trying to use Microsoft Azure Storage SDK for Node.js and JavaScript for Browsers (https://github.com/Azure/azure-storage-node) to display PDF contents stored in Azure blob in browsers. So far I couldn't find any examples on how to do it.
I tried to follow the suggestion from https://github.com/Azure/azure-storage-node/issues/440, But couldn't make it work. I am using Azure function.
module.exports = async function (context, req) {
let accessToken = await getAccessToken();
let container = req.params.container;
let filename = req.params.filename;
let tokenCredential = new azure.TokenCredential(accessToken);
let storageAccountName = process.env.StorageAccountName;
let blobService = azure.createBlobServiceWithTokenCredential(`https://${storageAccountName}.blob.core.windows.net/`, tokenCredential);
return new Promise((resolve, reject) => {
let readStream = blobService.createReadStream(container, filename, function (error, result, response) {
if (error) {
context.log(error);
context.log(response);
context.res = {
status: 400,
body: response
};
resolve(context.res);
}
});
let body = '';
readStream.on('data', (chunk) => {
body += chunk;
});
readStream.on('end', () => {
context.res = {
headers: {
'Content-Type': "application/pdf"
},
body: body
};
resolve(context.res);
});
});
};
But I got "Couldn't open PDF" error message in the browser or timeout error.
For downloading blob in browser environment, using URL with SAS is recommended, and in the framework you are using, would an accessible URL pointing to PDF be enough?
Please follow example:
Download Blob
BlobService provides interfaces for downloading a blob into browser memory. Because of browser's sandbox limitation, we cannot save the downloaded data trunks into disk until we get all the data trunks of a blob into browser memory. The browser's memory size is also limited especially for downloading huge blobs, so it's recommended to download a blob in browser with SAS Token authorized link directly.
Shared access signatures (SAS) are a secure way to provide granular access to blobs and containers without providing your storage account name or keys. Shared access signatures are often used to provide limited access to your data, such as allowing a mobile app to access blobs.

How to send multiple images in a Expressjs api get request with sendFIle()

I'm looking for away to send multiple images in one GET request from an Expressjs server through an api.
I want to create an image gallery of each users uploaded images in a MEAN stack. When images are uploaded using multer, the image information is saved to mongodb, including the userid of whoever uploaded it.
When on angularjs, I want user to have access to any of the images they have previously uploaded. Currently I'm sending one file on a GET request based on user id. Is there anyway of sending multiple files in one json. I'm currently using Expressjs's res.sendFile, but haven't found any info about sending multiple back yet.
https://expressjs.com/en/api.html#res.sendFile
Here is my current get request:
exports.getUpload = function(req, res) {
Upload.find({createdby: req.params.Id}).exec(function(err, upload) {
errorhandle.errorconsole(err, 'file found');
console.log(upload[0]);
var options = {
root: '/usr/src/app/server/public/uploads/images'
};
var name = "" + upload[0].storedname +"";
console.log(name);
res.sendFile(name, options,function(err) {
errorhandle.errorconsole(err, 'file sent');
});
});
};
You can't with res.sendFile. In fact I don't think you can at all. Maybe with HTTP/2 Server Push
, but I'm not sure.
What you can do is send a JSON response with a link to all the images:
exports.getUpload = async (req, res) => {
const uploads = await Upload.find({ createdby: req.params.Id }).exec()
const response = uploads.map(image => {name: `https://example.com/uploads/images/${image.storedname}`})
res.json(response)
}
Note error handling omitted.

Rails - Uploading large files directly to S3 with Jquery File Upload (hosted on Heroku )

I'm using Heroku, which means I have to upload multiple large files to S3 directly.. I'm using Rails 3.2.11, and Ruby 1.9.3. I do not wish to use carrierwave or paperclip gems, or really change much at this point - I just need to get this what I have working.
Before trying to move to S3, if I ran my app locally, I could upload multiple large files to the local file system. When I ran it on Heroku, small files upload but large ones failed. Hence the switch to S3..
I tried several tweaks, and also this link below, but it's just too much of a change to what I have that already working with the local server's file system (and Heroku as well, but Heroku just can't handle large files ..)
Tried: https://devcenter.heroku.com/articles/direct-to-s3-image-uploads-in-rails
I've tried some of the other examples here on Stack Overflow but they are too much of a change for what works locally, and well, I don't grasp everything they are doing.
Now, what happens when I do try to upload images?
It's as if the file upload works - the preview images are successfully created, but nothing is ever uploaded to Amazon s3, and I don't receive any kind of error messages (like s3 authentication failure or anything.. nothing)
What do I need to change in order to get the files over to my s3 storage, and what can I write out to console to detect problems, if any, connecting to my s3?
My form:
<%= form_for #status do |f| %>
{A FEW HTML FIELDS USED FOR A DESCRIPTION OF THE FILES - NOT IMPORTANT FOR THE QUESTION}
File:<input id="fileupload" multiple="multiple" name="image"
type="file" data-form-data = <%= #s3_direct_post.fields%>
data-url= <%= #s3_direct_post.url %>
data-host =<%=URI.parse(#s3_direct_post.url).host%> >
<%= link_to 'submit', "#", :id=>'submit' , :remote=>true%>
<% end %>
My jquery is:
....
$('#fileupload').fileupload({
formData: {
batch: createUUID(),
authenticity_token:$('meta[name="csrf-token"]').attr('content')
},
dataType: 'json',
acceptFileTypes: /(\.|\/)(gif|jpe?g|png)$/i,
maxFileSize: 5000000, // 5 MB
previewMaxWidth: 400,
previewMaxHeight: 400,
previewCrop: true,
add: function (e, data) {
tmpImg.src = URL.createObjectURL(data.files[0]) ; // create image preview
$('#'+ fn + '_inner' ).append(tmpImg);
...
My controller:
def index
#it's in the index just to simplify getting it working
#s3_direct_post = S3_BUCKET.presigned_post(key: "uploads/#{SecureRandom.uuid}/${filename}", success_action_status: '201', acl: 'public-read')
end
The element that is generated for the form is (via Inspect Element):
<input id="fileupload" multiple="multiple" name="image"
data-form-data="{"key"=>"uploads/34a64607-8d1b-4704-806b-159ecc47745e/${filename}"," "success_action_status"="
>"201"," "acl"=">"public-read"," "policy"=">"[encryped stuff - no need to post]","
"x-amz-credential"=">"
[AWS access key]/[some number]/us-east-1/s3/aws4_request"
," "x-amz-algorithm"=">"AWS4-HMAC-SHA256"
," "x-amz-date"=">"20150924T234656Z"
," "x-amz-signature"=">"[some encrypted stuff]"}"
data-url="https://nunyabizness.s3.amazonaws.com" data-host="nunyabizness.s3.amazonaws.com" type="file">
Help!
With S3 there actually is no easy out of the box solutions to upload files, because Amazon is a rather complex instrument.
I had a similar issue back in the day and spend two weeks trying to figure out how S3 works, and now use a working solution for uploading files onto S3. I can tell you a solution that works for me, I never tried the one proposed by Heroku.
A plugin of choice I use is Plupload, since it is the only component I actually managed to get working, apart from simple direct S3 uploads via XHR, and offers the use of percentage indicators and in-browser image resizing, which I find completely mandatory for production applications, where some users have 20mb images that they want to upload as their avatar.
Some basics in S3:
Step 1
Amazon bucket needs correct configuration in its CORS file to allow external uploads in the first place. The Heroku totorial already told you how to put the configuration in the right place.
http://docs.aws.amazon.com/AmazonS3/latest/dev/cors.html
Step 2
Policy data is needed, otherwise your client will not be able to access the corresponding bucket file. I find generating policies to be better done via Ajax calls, so that, for example, admin gets the ability to upload files into the folders of different users.
In my example, cancan is used to manage security for the given user and figaro is used to manage ENV variables.
def aws_policy_image
user = User.find_by_id(params[:user_id])
authorize! :upload_image, current_user
options = {}
bucket = Rails.configuration.bucket
access_key_id = ENV["AWS_ACCESS_KEY_ID"]
secret_access_key = ENV["AWS_SECRET_ACCESS_KEY"]
options[:key] ||= "users/" + params[:user_id] # folder on AWS to store file in
options[:acl] ||= 'private'
options[:expiration_date] ||= 10.hours.from_now.utc.iso8601
options[:max_filesize] ||= 10.megabytes
options[:content_type] ||= 'image/' # Videos would be binary/octet-stream
options[:filter_title] ||= 'Images'
options[:filter_extentions] ||= 'jpg,jpeg,gif,png,bmp'
policy = Base64.encode64(
"{'expiration': '#{options[:expiration_date]}',
'conditions': [
{'x-amz-server-side-encryption': 'AES256'},
{'bucket': '#{bucket}'},
{'acl': '#{options[:acl]}'},
{'success_action_status': '201'},
['content-length-range', 0, #{options[:max_filesize]}],
['starts-with', '$key', '#{options[:key]}'],
['starts-with', '$Content-Type', ''],
['starts-with', '$name', ''],
['starts-with', '$Filename', '']
]
}").gsub(/\n|\r/, '')
signature = Base64.encode64(
OpenSSL::HMAC.digest(
OpenSSL::Digest::Digest.new('sha1'),
secret_access_key, policy)).gsub("\n", "")
render :json => {:access_key_id => access_key_id, :policy => policy, :signature => signature, :bucket => bucket}
end
I went as far as put this method into the application controller, although you could find a better place for it.
Path to this function should be put into the route, of course.
Step 3
Frontend, get plupload: http://www.plupload.com/ make some link to act as the upload button:
<a id="upload_button" href="#">Upload</a>
Make a script that configures the plupload initialization.
function Plupload(config_x, access_key_id, policy, signature, bucket) {
var $this = this;
$this.config = $.extend({
key: 'error',
acl: 'private',
content_type: '',
filter_title: 'Images',
filter_extentions: 'jpg,jpeg,gif,png,bmp',
select_button: "upload_button",
multi_selection: true,
callback: function (params) {
},
add_files_callback: function (up, files) {
},
complete_callback: function (params) {
}
}, config_x);
$this.params = {
runtimes: 'html5',
browse_button: $this.config.select_button,
max_file_size: $this.config.max_file_size,
url: 'https://' + bucket + '.s3.amazonaws.com/',
flash_swf_url: '/assets/plupload/js/Moxie.swf',
silverlight_xap_url: '/assets/plupload/js/Moxie.xap',
init: {
FilesRemoved: function (up, files) {
/*if (up.files.length < 1) {
$('#' + config.select_button).fadeIn('slow');
}*/
}
},
multi_selection: $this.config.multi_selection,
multipart: true,
// resize: {width: 1000, height: 1000}, // currently causes "blob" problem
multipart_params: {
'acl': $this.config.acl,
'Content-Type': $this.config.content_type,
'success_action_status': '201',
'AWSAccessKeyId': access_key_id,
'x-amz-server-side-encryption': "AES256",
'policy': policy,
'signature': signature
},
// Resize images on clientside if we can
resize: {
preserve_headers: false, // (!)
width: 1200,
height: 1200,
quality: 70
},
filters: [
{
title: $this.config.filter_title,
extensions: $this.config.filter_extentions
}
],
file_data_name: 'file'
};
$this.uploader = new plupload.Uploader($this.params);
$this.uploader.init();
$this.uploader.bind('UploadProgress', function (up, file) {
$('#' + file.id + ' .percent').text(file.percent + '%');
});
// before upload
$this.uploader.bind('BeforeUpload', function (up, file) {
// optional: regen the filename, otherwise the user will upload image.jpg that will overwrite each other
var extension = file.name.split('.').pop();
var file_name = extension + "_" + (+new Date);
up.settings.multipart_params.key = $this.config.key + '/' + file_name + '.' + extension;
up.settings.multipart_params.Filename = $this.config.key + '/' + file_name + '.' + extension;
file.name = file_name + '.' + extension;
});
// shows error object in the browser console (for now)
$this.uploader.bind('Error', function (up, error) {
console.log('Expand the error object below to see the error. Use WireShark to debug.');
alert_x(".validation-error", error.message);
});
// files added
$this.uploader.bind('FilesAdded', function (up, files) {
$this.config.add_files_callback(up, files, $this.uploader);
// p(uploader);
// uploader.start();
});
// when file gets uploaded
$this.uploader.bind('FileUploaded', function (up, file) {
$this.config.callback(file);
up.refresh();
});
// when all files are uploaded
$this.uploader.bind('UploadComplete', function (up, file) {
$this.config.complete_callback(file);
up.refresh();
});
}
Plupload.prototype.init = function () {
//
}
Step 4
The implemetation of the general multi-purpose file uploader function:
ImageUploader = {
init: function (user_id, config, callback) {
$.ajax({
type: "get",
url: "/aws_policy_image",
data: {user_id: user_id},
error: function (request, status, error) {
alert(request.responseText);
},
success: function (msg) {
// set aws credentials
callback(config, msg);
}
});
},
},
// local functions
photo_uploader: function (user_id) {
var container = "#photos .unverified_images" // for example;
var can_render = false;
this.init(user_id,
{
select_button: "upload_photos",
callback: function (file) {
file.aws_id = file.id;
file.id = "0";
file.album_title = "userpics"; // I use this param to manage photo directory
file.user_id = user_id;
//console.log(file);
[** your ajax code here that saves the image object in the database via file variable you get here **]
});
},
add_files_callback: function (up, files, uploader) {
$.each(files, function (index, value) {
// do something like adding a progress bar html
});
uploader.start();
},
complete_callback: function (files) {
can_render = true;
}
}, function (config, msg) {
config.key = "users/" + user_id;
// Most important part:
window.photo_uploader = new Plupload(config, msg.access_key_id, msg.policy, msg.signature, msg.bucket);
});
}
can_render variable is useful so that you can make the application only then re-render the page, when the uploader is actually done.
And to make the button work from somewhere else call:
ImageUploader.photo_uploader(user_id);
And the button will act as a Plupload uploader button.
What is important is that Policy is made in a way so that noone can upload the photo into someone else's directory.
It would be great to have a version that does the same not via ajax callbacks, but with web hooks, this is something I want to do in the future.
Again, this is not a perfect solution, but something that works good enough from my experience for the purpose of uploading images and videos onto amazon.
Note in case someone asks why I have this complex object-oriented structure of uploader objects, the reason is that my application has all different kinds of uploaders that behave differently and they need to have an initializer with common behavior. The way I did it I can write an initializer for, say, videos, with minimum amount of code, that will do similar things to the existing image uploader.