I need to use VBA to import a large CSV excel file into an Access table. The delimiter is "" (double quotes) except for some reason the first value is followed by " (only one quote) instead of two like every other value. The first row contains the column headers and are delimited the same way. At the bottom I have attached an example.
The CSV files are generated automatically by an accounting system daily so I cannot change the format. They are also quite large (150,000+ lines, many columns). I'm fairly new to VBA, so as much detail as is possible would be much appreciated.
Thanks in advance!
Example of format
That doesn't sound like a CSV file. Can you open it in Excel, convert it to a true CSV, and then import that into Access? You will find many VBA-driven import options at the URL below.
http://www.accessmvp.com/KDSnell/EXCEL_Import.htm
Also, take a look at these URLs.
http://www.erlandsendata.no/english/index.php?d=envbadacimportado
http://www.erlandsendata.no/english/index.php?d=envbadacimportdao
Related
While running an experiment I made an error. If PsychoPy has correct participant codes, then the two conditions are combined into one .csv per session. (If not two .psydat files are created.) I know that .psydat is a format which contains more information than the csv format I need, but I'm not sure how to access the information which is there so it can be added to my dataframe and analysed in R. For example, I need to combine the data in MA22BI1.psydat and MAMA22BI1.psydat into MA22BI1.csv. I couldn't find anything remotely related in the PsychoPy forum. Thanks for any ideas!
You can regenerate a .csv file from a .psydat file using the code supplied here: psychopy.org/general/dataOutputs.html.
This is perhaps one of those many times discussed questions with solutions more specific to actual system that outputs the data into a CSV file.
Is there a simple way to export data like 3332401187555, 9992401187000 into a CSV file in a way that later when opened in Excel, the columns won't show them in "scientific" format? Should this be important, the data is retrieved directly by an SQL SELECT statement from any DBMS.
This also means that I've tried solutions like surrounding the values with apostrophes '3332401187555' and the Excel cell recognizes those as text and doesn't do any conversions/masking. Was wondering if there was a more elegant way without actually it being a pre-set Excel template with text data fields.
1. Try exporting the numbers prefixed with single quote. Example: '3332401187555.
2. In excel, select the column containing number values
and then select Number in Format Cells.
Just have to save your file with Excel the option CSV file. And you have the in file in requested format.
I am working on a project that basically entails on importing a CSV file into a SQL Server 2008 R2 database. The CSV file is generated from an Excel file that is populated by a "manager" with PR hours for his employees. This also includes some additional information such as which job and phase the employees were working on and also includes the number of hours for an equipment (if used).
Once you generate a CSV file for that, it's not exactly the usual straighforward "column" based CSV file. It's more like a "row" based CSV file with each row being kind of unique. Due to this caveat involved, I cannot do a straight dump (using BULK insert or OPENROWSET) to SQL, which would essential create a (temp) table with the appropriate column filled data.
I am looking to use the fields within the CSV file based on the "location" of that field in the row.
So, basically the positions of the data will remain the same, since every CSV is based on a TEMPLATE file - so all I have to do is navigate through the CSV file using SQL code to find the right field based on it's position in the ROW. I hope that gives you guys a better understanding of what I am trying to achieve here. Sorry for the long wall of text.
I researched a bit and here's what I have come up with so far:
Reads CSV files into a temp table through a custom SQL function (Reading lines from a file)
https://www.simple-talk.com/sql/t-sql-programming/reading-and-writing-files-in-sql-server-using-t-sql/
This one is interesting. Dumps the whole file as a BLOB and then you can sift through the data.
http://www.mssqltips.com/sqlservertip/1643/using-openrowset-to-read-large-files-into-sql-server/
Finally, this one essential splits out the rows and creates separates records per row. Interesting..
http://ask.sqlservercentral.com/questions/17408/how-to-read-a-text-file.html
If anyone has any suggestions or steps that I could follow to get through this, I would greatly appreciate it.
To the Mods: If I have posted something (especially the links) that shouldn't be here, please feel free to remove it. I apologize if I did.
Thanks much.. Hope to hear some positive responses! :)
Warm Regards,
Pranav
If the file is not too large, another option is to post-process the file in Excel using a VBA macro. Of course, you'd need to come up to speed using the Excel object model and VBA, but the recording function makes it fairly simple. One advantage of the VBA approach is that it seems you really do want to do row by row processing, and VBA is better for that, whereas SQL is better for set-based operations.
I am stuck with a CSV file with over 100,000 rows that contains product images from a provider. Here are the details of the issue, I would really appreciate some tips to help resolve this. Thanks.
The File has 1 Row per product and the following 4 columns.
ID,URL,HEIGHT,WIDTH
example: 1,http://i.img.com,100,200
Problem starts when a product has multiple images.
Instead of having 1 row per image the file has more columns in same row.
example:
1,http://i.img.com,100,200,//i.img.com,20,100,//i.img.com,30,50
Note that only first image has "http://" remaining images start with "//"
There is no telling how many images per product hence no way to tell how many total columns per row or max columns.
How can I import this using SSIS or sql import wizard.
Also I need to do this on regular intervals.
Thank you for your help.
I don't think that you can use any standard SSIS task or wizard to do this. You're going to have to write some custom code which parses each line. You can do this in SSIS using VB code or you can import the file into a staging table that's just a single column to hold each row and do the parsing in SQL. SSIS will probably be faster for this kind of operation.
Another possibility is to preprocess the file using regex or a search-and-replace command. Try to get double-quotes around the image list then you should be able to import the whole file fine, with the quoted part going into a single column. Catching the start of the string should be easy enough given the "http:\" for which you can search. Determining where the end quote goes might be more of a problem.
A third potential solution would be to get the source to fix the data. Even if you can't get the images in separate rows (or another file with separate rows, which would be ideal), maybe you can get the double-quotes added from the source as part of the export. This would likely be less error-prone than using the search-and-replace method.
Good luck!
I need help creating a VBA macro that downloads closing price data directly from Yahoo Finance's Historic Quotes website and imports the data into an Excel spreadsheet column.
Background information:
This is the link to Yahoo Finance's Historical Quote database -
To download as a TXT file:
http://ichart.finance.yahoo.com/table.txt?s="StockTicker"&d="EndingMonth"&e="EndingDay"&f="EndingYear"&g=d&a="EndingMonth"&b="EndingDay"&c="EndingYear"&ignore=.txt
Formatting Issue:
By default, i.e. using Excel's Web Data Import Wizard, Excel imports the entire table which includes more columns than needed. I am trying to isolate the "close" column. I created a macro that formats the table to isolate the "close" column, but this macro requires me to manually download the data from Yahoo Finance as a txt file:
Sub Test_DownloadTextFile()
Dim text As String
'The variables for this URL (in light blue) should be retrieved from the spreadsheet, i.e. "StockTicker" will reference a cell with the ticker symbol "AAPL" in it and that will appear in the URL'
text = DownloadTextFile("http://ichart.finance.yahoo.com/table.txt?s="StockTicker"&d="EndingMonth"&e="EndingDay"&f="EndingYear"&g=d&a="EndingMonth"&b="EndingDay"&c="EndingYear"&ignore=.txt")
'At this point I should have the historical quotes table stored in the variable text. How do I select the 4th column and import it into a specific spreadsheet column?'
Debug.Print text
End Sub
How can I create a macro that:
1. Refers to the spreadsheet for key variables, e.g. "StockTicker", "EndingMonth", etc..
2. Downloads the corresponding historic data from Yahoo Finance
3. Imports the data closing price data as a single column into the spreadsheet
I would very much appreciate a practical solution to this problem. Let me know if I need clarify my question or the task at hand. Thank you!
Suggestion: this seems to be the perfect case for a Web Query.
Do you have any reason not to use that ? You can copy just the columns you need afterwards.
You did not specify you Excel version, but on 2003 it's on Data/Import External data.
Check out this SO discussion. Several suggestions that seem worthwhile.