I am using scrapy to scrape a website and I can download the file from the page, however everything that is being download is a plain text file. How do I download it with it's extension type? I am downloading scripts and as such, having the proper extension type on my download is necessary.
For example, if I am downloading exploits from exploit-db, the link that I go to to download them would be for example: https://www.exploit-db.com/exploits/19832/
and the link i would extract from there to download from is https://www.exploit-db.com/download/19832 which will, if I click on it normally, download a ruby file. But through scrapy it gets saved as a text file. Is there a way to download it as a .rb through scrapy?
Just save it as filename.rb. All files are text/binary files. Extension is there just to tell your operating system what to use to understand that file.
(In some operating systems extension isn't even required since files have headers at the beginning of the file telling what they are)
You can do try this:
scrapy shell https://www.exploit-db.com/download/19832
Then in the shell or your spider just do:
with open('ruby_file.rb', 'wb') as ruby_file:
ruby_file.write(response.body)
Related
I have used Spyder (Anaconda) to generate a Python GUI App. The app can browse & load any time series csv file on the user's pc, perform few statistical tests and print the results on to a txt file and save it to the user's desktop screen.
Is it possible to upload the executable file on to any repository so that others could try it out. For example, Google Earth Engine based apps can be easily shared via a link and anyone with that link can access the app. Similarly, is there anything for my case ?
This may not be the answer your looking for,
But you can upload .exe to Google drive and share it. So anyone could download it from the link generated.
File types: Users can upload any type of file, including executables
(for example, .exe or .vbs) and compressed files.
source
Can I use links in different site in setValue in file upload using nightwatch? I look up a link in stackoverflow and see this: File Upload Testing in Nightwatch.js but the answer said that it will not work if the file is coming from 'http://localhost:3000/testfile.txt'. Is this not possible?
The simple answer is no you can not specify the web server path.
You can read more about here.
What you can do is First download file and then use its absolutes or relative paths to upload.
I am trying to use WinSCP to access a remote HTML file, and I do have root permission. However, if I open that HTML, it seems that all the sources and scripts are dysfunctional, e.g no images, buttons etc are loaded, and no js functions work. So how can I open a HTML using WinSCP so that all the sources are loaded and work properly? Thanks a lot.
I assume you "Open" the HTML file from a remote panel of WinSCP.
This makes WinSCP download the file to a temporary local folder and open the file from there.
As all the references to images and java script files are typically relative paths, they won't work from the temporary folder as the referenced files are not there. So it cannot work.
Either you need to download all files (including images and .js) to a local folder and open the HTML there.
Or you actually want to open the file from the server, via HTTP.
For that use the WinSCP extension "Generate HTTP URL".
Right now, i am working on a go ahead embedded web server. i have an old 2.1 version of this server, which was open source. i want to upload .json file which i create from the firmware, to the web server and then want the page to process that file using flot tool,and display a graph.but that version does not support file uploading capability. on internet i have found that the new version of this web server support the file upload capabilities, but i have not found a proper example which explains the syntax that i would use to upload the file. can any one tell me which functions of this new version i would have to use to get things working.
can any one give a proper full example.
You ask how to upload using goahead.
When you build the source, it should build a test executable called goahead-test. This uses test/test.c as a main program. Test.c defines an upload action handler that is invoked when you do a file upload to the url /action/uploadTest. This handler will echo back to the browser the various file upload details. You can cut/paste from test.c into your own main program.
I’m making a simple web browser for work eeh, what I’d like to know is if its possible to save a file of a particular extension to a particular file.
I currently use google chrome when downloading a file it places this (regardless of extension) in a downloads folder without asking where I ant to download this too.
I want to achieve the same except that downloads with the extension .dwg are placed automatically in a folder named DWG DOWNLOADS…
How to achieve this in vb.net?
In any browser you have a config section.
In Firefox you have browser.download.useDownloadDir;true and browser.download.folderList;1
you can add your own config to allow different saved folder paths and dynamically modify them depending on the extension of the file you uploaded.
See a complete list of the web browser config with about:config in address bar.
Not real sure what you are asking, but if you are actually making a web browser just check the extension of the file you are requesting to download. If the extension is ".dwg" then save the file to the folder you want.
If you are wanting to automatically move Google Chrome downloads to a different directory, you can use a FileSystemWatcher to monitor for new files in Chrome's download directory and move them to another folder based on each file's extension.