So the following code, reads in a tab delimited file in my datagridview, however the formatting is all over the place. One field in particular, is a sequence number, padded with zeros. After the import, all leading zeros go away leaving me with 1,2,3,etc. There is also a date field, after import it tacks on the time.
In theory, I need to treat every field as text on import. However, the number of fields on import could change. So one file might have 10 fields, the next file might have 200 fields.
Here's my connection string:
Dim ConStr = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source='C:\Temp';Extended Properties='text';"
My schema includes the following:
Format=TabDelimited
TextDelimiter="
I've read where you can identify each field with a specific format. Very tedious to do this, especially if the fields can grow and shrink in size.
Suggestions?
Related
I've a csv file which contains data with new lines for a single row i.e. one row data comes in two lines and I want to insert the new lines data into respective columns. I've loaded the data into sql but now I want to replace the second row data into 1st row with respective column values.
output details:
I wouldn't recommend fixing this in SQL because this is an issue with the CSV file. The issue is that file contains new lines, which causes rows split.
I strongly encourage fixing CSV files, if possible. It's going to be difficult to fix that in SQL given there are going to be more cases like that.
If you're doing the import with SSIS (or if you have the option of doing it with SSIS if you are not currently), the package can be configured to manage embedded carriage returns.
Define your file import connection manager with the columns you're expecting.
In the connection manager's Properties window, set the AlwaysCheckForRowDelimiters property to False. The default value is True.
By setting the property to False, SSIS will ignore mid-row carriage return/line feeds and will parse your data into the required number of columns.
Credit to Martin Smith for helping me out when I had a very similar problem some time ago.
I loaded an Excel file into an SQL table. The Excel file, one field consists of VARCHAR data (of data type general). When loaded into an SQL table, some of these values are prefixed with zero.
Example: in the Excel file it is 1081999 the same value become 01081999 in the SQL table.
What might be the reason for this ?
Excel will hide leading 0's as it identifies the fields content as a number and displays it as such. I would assume that the excel worksheet does indeed contain these leading 0's and they are simply not shown by Excel. If you change the type of the column from General to Text do they show up??
As a side note, if these are indeed numbers you should be storing them in a numeric datatype in the database...
I have a SQL Server view I'm pulling into an excel macro workbook. However, one of the date fields is being treated as a number by excel and it's leading 0 is subsequently being stripped off. Is there a datatype I can convert my date field to in SQL so that excel does not strip off it's leading zero when the view is imported into the workbook?
How are you pulling the data in? AFAIK there's no Excel-wide setting to keep the leading zeros, however there are ways to keep them, or to add them back:
From Microsoft
If you're using the import wizard,
Convert the number to text when you import text data
In Step 3 of the Text Import Wizard (On the Data tab, in the Get
External Data group, click From Text), you can select the column of
data that contains the credit card number, and then explicitly choose
a Text column data format.
Alternatively, if you already imported the data and there's a pattern/structure to it, you can create a custom format:
Use a custom format to keep the leading zeros
If you want to resolve the issue just within the workbook because it's
not used by other programs as a data source, you can use a custom or a
special format to keep the leading zeros. This works for number codes
that contain fewer than 16 digits,
In addition, you can separate some of the digits in your number codes
with dashes by adding these dashes to the custom format. For example,
to make a phone number more readable, you can add a dash between the
international code, the country/region code, the area code, the
prefix, and the last few numbers.
Edit: I'm not to keen on SQL, so can only offer more Excel focused ideas :/
I have a number of .csv files in the following format with example data listed:
ID,Lat,Long
1,-43.76120167,158.0299917
2,-43.76119833,158.03
3,-43.7612,158.0299983
4,-43.76120167,158.0299967
The values change from file to file, but they're always the same format and similar amounts. What you see above is exactly how it shows up in the .csv file (when opened with a texpad/notepad not in Excel - so we can eliminate any Excel problems now).
However, when I run the following INSERT statement as an Access SQL query:
INSERT INTO Table1 SELECT * FROM [Text;Database=C:\;Hdr=Yes].[ImportFile.csv];
This is what shows up in my Access database table:
ID,Lat,Long
1,-43,158
2,-43,158
3,-43,158
4,-43,158
Now, I know what you're thinking. Let me just say that my table design in Access is set up such that ID is a Number/Long Integer, and both of my Lat and Long fields are set up as Number/Double with 4 Decimal Places. I've double checked this a million times and it can be confirmed because not all input files share this problem.
This is what is troubling me... where are all the digits after my decimal point going? I need to have them.
What confuses me even more is that some files read just fine and the decimal points stay in there just fine... same table, same insert query. Every file is generated from the same source and formatted the same for what it's worth.
However, if I fire up Access itself and run the import from text file wizard, the values end up just fine. Access automatically makes the field a double with auto decimals (I have also tried using auto decimals in my desired table, to no avail).
Anyone have any idea what is going on?
Thanks!
This is well known problem.
The Jet database engine determines the data types from data source. One of solutions is to use/create Schema.ini file.
Also, keep in mind that, in order to determine data type for columns, only first few rows are scanned.
For more info, please see here
I am exporting a file from a system as .csv. My aim is to link to this file as a table (which matches the output field for field) and then run the queries and export.
The problem I am having is that, upon import, all the fields are 255 bytes wide rather than what they need to be.
Here's what I've tried so far:
I've looked at ALTER TABLE but I cannot run multiple ALTER TABLE statements in one macro.
I've also tried appending the table into another table with the correct structure but it seems to overwrite the structure.
I've also tried using the Left function with the appropriate field length, but when I try to export, I pretty much just see 5 bytes per column.
What I would like is a suggestion as to what is the best path to take given my situation. I am not able to amend the initial .csv export, and I would like to avoid VBA if possible, as I am not at all familiar with it.
You don't really need to worry about the size of Text fields in an Access linked table that is connected to a CSV file. Access simply assigns each Text field the largest possible maximum size: 255. It does not mean that every value is actually 255 characters long, it just means that any values in those fields can be at most 255 characters long.
Even if you could change the structure of the linked table (which you can't), it wouldn't make any real difference except to possibly truncate longer Text values, and you could easily do that with a String function. For example, if a particular field had to be restricted to 15 characters then you could simply use Left([fieldName], 15) as a query column or as the control source in a report.
In the end, as the data set is not that large, I have set this up to append from my source data into a table with the correct structure. I can now run my processes against this table as per normal.