Scrapy image download pipeline - image expiry - scrapy

A simple question. Scrap allows for an image expiry to prevent downloading the same images multiple times. How does scrapy know which images it has already downloaded, if these images were downloaded during a previous run?

Related

How to delay until thumbnail created?

I have a vue page that has a file upload component. This allows the user to upload a file to server storage then display a thumbnail of the file uploaded for later retrieval.
The upload works fine and the server creates a thumbnail using imagemagick which is displayed on the page, this all works fine.
My problem is that the creation of the thumbnail takes a second or two which means when the file upload is finished the thumbnail isn't available immediately and I get an "image cannot be displayed" placeholder where the thumbnail should be. Refreshing the page shows the thumbnail fine.
I've jerry rigged things by adding a delay which gives the server time to create the thumbnail but it's a bit hit and miss as some files take longer to generate than others and it seems a bit poor for the user to have to wait 500ms or more before their action is complete.
How do I go about improving the user experience?

Upload same image in different sizes - Dropzonejs

There is any possibility to upload a file with different sizes in DropzoneJs?
I'm using vue-dropzone which is made with dropzonejs and i have to upload the same file with different sizing for srcset.
Example:
I want to upload the file test.png which is 1000x500 px. There is any possibility to upload it at the same time in original resolution and also in 500x250px?
Image resizing in the browser has been a seat-of-the pants experience for a long time. Web assemblies are the way of the future for processing-intensive tasks in web apps. I came across this project the other day. It looks fantastic and I really can't wait to strip out our home-baked image resizing with canvas and replace it with this.
The usual reason for doing this is to avoid large uploads. It's a little bit weird to want to resize in the browser then upload the original. You might be better resizing on the server. You'll save bandwidth and the server libraries will be more mature than what's available on the client.
Along with the original image object you can add one more your custom resized image to the array of images by using resize config of dropzone. You can do the above on drop event or adddedFile event of dropzone.

How to save all images from website using webscraping or macro

In my job of e-commerce i have to save images from MFG. sites to upload them to my client webpage, sometimes when product came like apple i6 there are hundreds of images in product description
so i want to save all images from mfg url with one click
For example:
URL- http://www.gsmarena.com/samsung_galaxy_a9_%282016%29-pictures-7641.php
it have 7 images, so i wanna get all images downloaded in single click
image links:
http://cdn.gsmarena.com/imgroot/reviews/15/apple-iphone-6s/-347x151/thumb.jpg
http://cdn2.gsmarena.com/vv/bigpic/samsung-galaxy-a9-2016-.jpg
http://cdn2.gsmarena.com/vv/bigpic/samsung-galaxy-a9-2016-.jpg
http://cdn2.gsmarena.com/vv/pics/samsung/samsung-galaxy-a9-2016-6.jpg
http://cdn2.gsmarena.com/vv/pics/samsung/samsung-galaxy-a9-2016-7.jpg
http://cdn2.gsmarena.com/vv/pics/samsung/samsung-galaxy-a9-2016-5.jpg
http://cdn2.gsmarena.com/vv/bigpic/samsung-galaxy-a8-.jpg
Maybe this way?
Image Picker Plugin for Firefox
Seems it does the same job without any installation (but plugin installation).
Not sure if it is, what you are looking for. But I tryed it and found an easy option for you.
Use the tool: jdownloader2
Copy the link of your desired URL and paste it into the link collector. It gathers automatically all the pictures on that page and downloads it for you. All images with a single click...

Download images from an external source iOS

Is it possible to download images from a web server and save them in such a way that
[UIImage imageNamed:#"filename"]
would still work for filename.png and filename#2x.png just as if they had been sent along in the application package all along?
Adding my comment as an answer. You can try to send the scale based on retina/non-retina and then download a single image based on that. You dont need to worry about combining two images and downloading since a device can be either retina or non-retina only. You dont need both images at the same time.
You can download the images based on retina/non-retina of the device itself. No need to keep both in the same device.
As you are considering downloading both images, so bandwidth per se is not of concern, why not combine the two images into a multi-resolution TIFF and download that from the web server? If you do that and load the image from the file the correct resolution will be picked automatically.

Loading pictures to gallery

I need some advice. In my application people can upload images. I will store the images on a server. I want to make a photo gallery where you can see all uploaded images. Every user has a account. And it will download all uploaded images.
The point is that when you uploaded 1 image it has to download 1 image. But when you upload 10 or more the app will have to download all images and cache the images to your device.
What is the best way to download all images and keep the app running without getting stuck. Also what is the best way to cache all images.
If you want to create some photo gallery apps, check out this open source sample.
Created by Michael Waterfall, which includes photo gallery from images stored on server or locally.
You can modify the code as per your project requirements. And it is easy to handle the images.