This question already has an answer here:
Failing to read String value from an excel column
(1 answer)
Closed 3 years ago.
I have a SSIS package which sources from an Excel file, performs a lookup in SQL, and then writes the fields from the lookup to a flat file. For some reason, any of the fields in the SQL table that are of data type varchar 256 are not getting written. They are coming in as nulls. My other fields, including varchar 255, are coming across fine. I have tried flat file and Excel as destination with no luck.
I've tried converting the varchar with a data conversion to both 256 and to a Unicode string and no luck.
Even when I preview a simple query in the source component (ex: select lastname from xyz), the preview shows the lastname as null. It doesnt show other fields that have different data types as nulls.
This is usually a case when the excel driver only reads the first 8 rows of data and misinterprets the correct data type because of the lack of data it's checking. Here are some of the known issues from the Microsoft site: Reference
Issues with importing
Empty rows
When you specify a worksheet or a named range as the source, the driver reads the contiguous block of cells starting with the first non-empty cell in the upper-left corner of the worksheet or range. As a result, your data doesn't have to start in row 1, but you can't have empty rows in the source data. For example, you can't have an empty row between the column headers and the data rows, or a title followed by empty rows at the top of the worksheet.
If there are empty rows above your data, you can't query the data as a worksheet. In Excel, you have to select your range of data and assign a name to the range, and then query the named range instead of the worksheet.
Missing values
The Excel driver reads a certain number of rows (by default, eight rows) in the specified source to guess at the data type of each column. When a column appears to contain mixed data types, especially numeric data mixed with text data, the driver decides in favor of the majority data type, and returns null values for cells that contain data of the other type. (In a tie, the numeric type wins.) Most cell formatting options in the Excel worksheet do not seem to affect this data type determination.
You can modify this behavior of the Excel driver by specifying Import Mode to import all values as text. To specify Import Mode, add IMEX=1 to the value of Extended Properties in the connection string of the Excel connection manager in the Properties window.
Truncated text
When the driver determines that an Excel column contains text data, the driver selects the data type (string or memo) based on the longest value that it samples. If the driver does not discover any values longer than 255 characters in the rows that it samples, it treats the column as a 255-character string column instead of a memo column. Therefore, values longer than 255 characters may be truncated.
To import data from a memo column without truncation, you have two options:
Make sure that the memo column in at least one of the sampled rows contains a value longer than 255 characters
Increase the number of rows sampled by the driver to include such a row. You can increase the number of rows sampled by increasing the value of TypeGuessRows under the following registry key:
Redistributable components version - Registry key
Excel 2016 - HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Office\16.0\Access Connectivity Engine\Engines\Excel
Excel 2010 - HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Office\14.0\Access Connectivity Engine\Engines\Excel
Related
I have a SQL table with ERP data containing over 1 million records for seven different companies. This data needs to be exported to Excel files breaking the records by company, and for each company by a date range. Based on the companies and date ranges it will produce > 100 separate files.
The results are being imported into a second ERP system which requires the import file to be Excel format and must include the column names as the first row. The records need to be sorted by a [PostDate] column which must NOT be exported to the final record set.
The client's SQL server does not have the MS Jet Engine components installed in order for me to export directly to Excel. Client will not allow install of the MS Jet Engine on this server.
I already had code from a similar project that exported to CSV files so have been attempting that. My T-SQL script loops through the table, parses out the records by company and by date range for each iteration, and uses BCP to export to CSV files. The BCP command includes the column names as the first row by using an initial SELECT 'Col_1_Name', 'Col_2_Name', etc. with a UNION SELECT [Col_1], [Col_2],...
This UNION works AS LONG AS I INCLUDE THE [PostDate] column needed for the ORDER BY in the SELECT. This exports the records I need in proper order, with column names, but also includes the [PostDate] column which the import routine will not accept. If I remove [PostDate] from the SELECT, then the UNION with ORDER BY [PostDate] fails.
We don't want to have the consultant spend the time to delete the unwanted column from each file for 100+ files.
Furthermore, one of the VarChar columns being exported ([Department]) contains rows that have a leading zero, "0999999" for example.
The user opens the CSV files by double-clicking on the file in Windows file explorer to review the data, notes the [Department] column values are ok with leading zero displayed, and then saves as Excel and closes the file to launch the import into the second ERP system. This process causes the leading zeros to be dropped from [Department] resulting in import failure.
How can I (1) export directly to Excel, (2) including column names in row 1, (3) sorting the rows by [PostDate] column, (4) excluding [PostDate] column from the exported results, and (5) preserving the leading zeros in [Department] column?
You could expand my answer to this question SSMS: Automatically save multiple result sets from same SQL script into separate tabs in Excel? by adding the sort functionality you require.
Alternatively, a better approach would be to use SSIS.
I have an ETL process set up to take data from an Excel spreadsheet and store it in a database using SSIS. However, one of the columns in the the Excel file is formatted as a percent, and it will sometimes erroneously be stored as a NULL value in the database, as if there was some sort of translation error.
Pictured is the exact format being used for the column in Excel.
Interestingly, these percent values do load properly on some days, but for some reason one particular Excel sheet I was given as an example of this issue will not load any of them at all when put through the SSIS processor.
In Excel, these values will show up like "50.00%", and when the SSIS processor is able to translate them properly it will display as the decimal equivalent in the database, "0.5", which is what I want instead of the NULL values. The data type I am using in SSIS for this is Unicode string [DT_WSTR], and it is saved as an NVARCHAR in the database.
Any insight as to why these values will sometimes not display/translate as intended? I have tried messing around with the data types in SSIS/SQL Server, but it has either resulted in no change or error. When I put test values in the Excel sheet, such as "test" to see if it is importing anything at all from this column, it does seem to work (just not for the percent numbers that I need).
The issue was caused by the "mixed data types" that were present in the first few rows of my data (the "mixed" part being blank fields), which would explain why some sheets would work and others wouldn't.
https://stackoverflow.com/a/542573/11815822
Setting the connection string to accommodate for this fixed the issue.
I made a simple query function as this: =QUERY(range,"select *",1), in my Google Spreadsheet, but the results dose not show any letters, it shows only fields that contains numbers.
Study this link
Syntax
QUERY(data, query, [headers])
data - The range of cells to perform the query on.
Each column of data can only hold boolean, numeric (including
date/time types) or string values.
In case of mixed data types in a single column, the majority data type
determines the data type of the column for query purposes. Minority
data types are considered null values.
Just format your range as a plain text using Format > Number > Plain text option, the below images show how to do it:
I loaded an Excel file into an SQL table. The Excel file, one field consists of VARCHAR data (of data type general). When loaded into an SQL table, some of these values are prefixed with zero.
Example: in the Excel file it is 1081999 the same value become 01081999 in the SQL table.
What might be the reason for this ?
Excel will hide leading 0's as it identifies the fields content as a number and displays it as such. I would assume that the excel worksheet does indeed contain these leading 0's and they are simply not shown by Excel. If you change the type of the column from General to Text do they show up??
As a side note, if these are indeed numbers you should be storing them in a numeric datatype in the database...
Why is the length for my longcomment field (defined as varchar(600)) only up to 255 coming from an Excel spreadsheet? The length of the value in the spreadsheet is over 300 characters and the field in the table is defined at varchar(600). However, when I select on that field in SQL, it is truncating at 255.
Thanks!
When an Excel files is parsed for import, only a certain number of rows are scanned to determine column size of the input. How you are importing the data makes a difference on what you need to change, basically you either need to override the detected column size or increase the number of rows scanned. Leave a comment with what import method you are using if you need additional help.
In Microsoft SQL Server Management Studio, you can change the size by going to the Tools/Option menu and opening the Query Results branch on the tree control. Then under the Results to Text leaf is the the “Maximum number of characters in a column” value.