I've got some data files that are stored compressed on our company's server with a .Z extension (UNIX compress utility used to zip them down).
Can SQL Server's BULK IMPORT operation read these files in that format? Or must I uncompress them before getting at the data?
The BULK IMPORT would not natively be able to do this however if you are on SQL2005 or greater you can use SSIS. The first step would be to perform an Exectute Process Task and use a zip utility to unzip the file. The second step is to use the SSIS Bulk Insert task to push the data into SQL Server.
EDIT: use the compress from unixutils rather than cygwin to uncompress the files as it understands native windows filenames. This means that you don't have to maintain /cygdrive paths as well as native paths.
Related
I'm writing a Python script to check the file hash of a malware sample against the VirusTotal database.
At the moment, I'm using a build in open and read functions provided in a standard python library:
with open(file_path, 'rb') as file:
file_sha1 = hashlib.sha1(file.read())
I am testing it in a VM to ensure my host system is not compromised. But I am wondering would Python script that reads the file in binary form execute the file or is it just reading the binary content and there is no chance of executing the file, thus not compromising my machine.
When dealing with malware it is better to be safe than sorry I was wondering are there any precautions to take in my project.
How using the program wkhtmltopdf I convert the entire folder with html files?
Let's say there are 10,000 html files in the folder, and I need to make 10,000 pdf out of them.
You cannot do that directly with wkhtmltopdf, you will need some external logic to handle going through your files.
This could be a shell script, or it could be a portion of the program you are writing. If you simply have a task that states "convert these files to pdf" and you are running on a standard Linux server, I would advise you to write a shell script that loops over the files and executes wkhtmltopdf for each file separately.
Technologies used:
Frontend: Angular JS
Backend: SQL
We need to upload all formats of files into SQL DB by using Angular JS and need to download the uploaded files. Here by using some methods we are converting the uploaded file into "Blob" Format and stored in SQL.
After insertion into SQL we are trying to download the file in same format. But here we are able to download only text format files by using "ng-filesaver" concept. But remaining file format are not able to download. So which concept we need to use to download any format of files by using AngularJS. Please help us we are working from last 15 days on this.
My goal is to locally save a BigQuery table to be able to perform some analyses. To save it locally, i tried to export it to Google Cloud Storage as a csv file. Alas the dataset is too big to move it as one file, thus it is splitted into many different files, looking like this:
exampledata.csv000000000000
exampledata.csv000000000001
...
Is there a way to put them back together again in the Google Cloud Storage? Maybe even change the format to csv?
My approach was to download it and try to change it manually. Clicking on it does not work, as it will save it as a BIN.file and is also very time consuming. Furthermore I do not know how to assemble them back together.
I also tried to get it via the gsutil command, and I was able to save them on my machine, but as zipped files. When unzipping with WinRar, it gives me exampledata.out files, which I do not know what to do with. Additionally I am clueless how to put them back together in one file..
How can I get the table to my computer, as one file, and as a csv?
The computer I am working with runs on Ubuntu, but I need to have the data on a Google Virtual Machine, using Windows Server 2012.
try using the following to merge all files into one from the windows command prompt
copy *.cs* merged.csv
Suggest you to save the file as .gzip file, then you can download it from Google Cloud easily as BIN file. If you get these splited files in bigquery as following:
Export Table -> csv format, compression as GZIP, URI: file_name*
Then you can combine them back by doing steps as below:
In windows:
add .zip at the end all these files.
use 7-zip to unzip the first .zip file, with name "...000000000000", then it will automatically detect all the rest .zip files. This is just like the normal way to unzip a splitted .zip file.
In Ubuntu:
I failed to unzip the file following the method I can find in internet. Will update the answer if I figure it out.
I am trying to figure out how to create a job/transformation to uncompress and load a .tar.gz file. Does anyone have any advice for getting this to work?
you want to read a text file that is compressed?
Just specify the file in the text file input step in the transformation - and specify the compression (GZip). Kettle can read directly from compressed files.
If you do need the file uncompressed then use a job step - not sure if there is a native uncompress, but if not just use a shell script step.
There is not such component in kettle to uncompress the tar.gz file i found.
But if we have the csv file text compressed in gizip format we can use gzip input component.