Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have a bunch of (largish, 10GB each) files in bz2 format. I would like to upload them and then perform some queries on them. Does big query "understand" bzip as it does gzip? Should I convert them? What would be the best way to upload them?
I assume the files are in CSV or JSON format. Per BigQuery documentation (https://cloud.google.com/bigquery/preparing-data-for-loading), only gzip compression is supported. Bit even if bz2 was supported, it wouldn't be a good idea to work with 10GB sized compressed files. The problem is that unlike with uncompressed file - BigQuery won't be able to split them into pieces, and will have to work with entire 10GB file, which will be very slow.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 8 days ago.
Improve this question
What kind of performance (latencies) can I expect with Amazon Athena queries on an S3 bucket containing about 200 million files?
More details: Each file is only 2KB in size. Let's assume that the file format is either CSV or Parquet (not both) and we have about 50 searchable fields.
I could possibly use s3DistCp to merge files to improve performance, but I don't want to go that route unless it is really needed.
Thanks for the help!
Esther
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I want to insert jpeg file in SQL. How to do this?
You probably want to use the binary, varbinary or varbinary(max) data type.
Of course, you have to convert the (JPEG) image to a byte array first in order to store it in the database. Depending on the programming language of your choice this can be easy or hard.
Not sure if you can read files from a T-SQL command and store/retrieve them from a database. Then again, I don't think you would want such a scenario in real-life.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I'm looking to create a program that will sift and sort through tabular data created in excel format. I would like to be able to iterate through several .csv files within a folder, and create dictionaries of information that can be used to generate metrics and reports. I've been developing for a couple of years now, and I've never had to use xcode for anything like this.
I am curious if this is an easy solution to manipulating data or is there a preferred way of dealing with data in this form?
You can use CHCSVParser to read and parse the .csv files.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
There are string,property list, rich text file to store values which among the three is better and having too much of plist in a project will it slow down the speed of app.
Property list is better for key-value pair storage ,In general which is most recommended?
If you storing large amount of interrelated data you would go for core data or sqlite.
If you have flat data there is nothing wrong in having normal text files.
All you have to remember is ,
Disk I/O is slow. Too many read and write leads to bad performance.
Primary memory is scarce. Use it judiciously.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I want to create some diagrams for some papers.
Diagrams will contain some text, e.g. some console output. I need images for using also in html files.
There is TikZ so can create images like this:
http://www.texample.net/tikz/examples/boxes-with-text-and-math/
http://www.texample.net/tikz/examples/rule-based-diagram/
http://www.texample.net/tikz/examples/scenario-tree/
but as a result I get some ps/pdf files, not images.
What's more I want to generate the pictures from text files as I want to track changes in some VCS, any binary files are not suitable for that.
The program convert from the ImageMagick suite can convert PDF files to other formats, like PNG. In its simplest form:
convert diagram.pdf diagram.png
See the manual for additional options.