I am using oledb to get data from .txt file and i have encountered error.
Dim oleDB = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\\CompName\C$\Path;ExtendedProperties="Text;HDR=Yes;FMT=Fixed""
where CompName and Path are real values
I get Unspecified error in adapter fill
Using connection As New OleDbConnection(oleDb)
Using command As New OleDbCommand(sql, connection)
Using adapter As New OleDbDataAdapter(command)
adapter.Fill(s)
End Using
End Using
End Using
Return s
End Function
anyone tried geting data across intranet from different computer using oledb?
To use the OleDb Text Driver with a text file formatted with fixed length columns you need to have a SCHEMA.INI file in the same folder where the text files are located.
The SCHEMA.INI allows to define various property for the text file like Format, Field Names, Widths and Types, character sets and some conversion rules.
From MSDN
When the Text driver is used, the format of the text file is
determined by using a schema information file. The schema information
file is always named Schema.ini and always kept in the same directory
as the text data source. The schema information file provides the
IISAM with information about the general format of the file, the
column name and data type information, and several other data
characteristics. A Schema.ini file is always required for accessing
fixed-length data. You should use a Schema.ini file when your text
table contains DateTime, Currency, or Decimal data, or any time that
you want more control over the handling of the data in the table.
More details about the SCHEMA.INI file could be found on this MSDN page
Related
I am trying to create a project that will produce a separate excel file for every account in a specific table. The first steps to pull the data into a temp file and load an array variable with the accounts for the loop, this is working fine.
I am getting the 2 errors below. And when I execute the package it fails on the export to excel data flow task.
Error 1 Error loading RAW_DATA_EXPORT.dtsx: The connection string
format is not valid. It must consist of one or more components of the
form X=Y, separated by semicolons. This error occurs when a connection
string with zero components is set on database connection manager.
Error 2 Error loading RAW_DATA_EXPORT.dtsx: The result of the
expression ""\\server\DATA\Status\Testing\Filename" +
#[User::Accts] + ".xls"" on property "ConnectionString" cannot be
written to the property. The expression was evaluated, but cannot be
set on the property.
This is what my dynamic connection string looks like evaluated:
Provider=Microsoft.Jet.OLEDB.4.0;Data Source=\\server\DATA\Status\Testing\Filename03500.xls;Extended Properties="EXCEL 8.0;HDR=YES";
I think the issue is since the excel file are non-existent, the validation fails when it gets to the step. From what I found online I need to either create the sheets through a a script task (not fond of) or use a variable value to create the sheet in the file. I have not been successful on either.
(...) since the excel file are non-existent, the validation fails when it gets to the step.
Have you tried delaying validation in the Data Flow task that exports to Excel? Check Data Flow's properties window.
The following code I have reads a tab delimited file into a DataGridView. It works fine, but there are a couple of issues I'm not exactly sure how to address.
Dim query = From line In IO.File.ReadAllLines("C:\Temp\Temp.txt")
Let Data = line.Split(vbTab)
Let field1 = Data(0)
Let field2 = Data(1)
Let field3 = Data(2)
Let field4 = Data(3)
DataGridView1.DataSource = query.ToList
DataGridView1.Columns(0).Visible = False
How do I go about adding fields (columns) based on the number of fields in the header row? The header row currently contains 110 fields, which I'd hate to define in a similar manner to Let field1 = Data(0)
I'd also need to skip the header row and only display the lines after this.
Is there a better way to handle this then what I'm currently doing?
There are several tools to parse this type of file. One is OleDB.
I cant quite figure out how the (deleted) answer works because, HDR=No; tells the Text Driver the first row does not contain column names. But this is sometimes ignored after it reads the first 8 lines without IMEX.
However, FMT=Delimited\""" looks like it was copied from a C# answer because VB doesnt use \ to escape chars. It also looks like it is confusing the column delimiter (comma or tab in this case) and text delimiter (usually ")
If the file is tab delimited, the correct value would be FMT=TabDelimited. I am guessing that the fields are text delimited with quotes (e.g. "France" "Paris" "2.25") and OleDB is chopping the data by quotes rather than tabs to accidentally get the same result.
The correct ACE string would be:
Dim connstr = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source='C:\Temp';Extended Properties='TEXT;HDR=Yes;FMT=TabDelimited';"
Using just the connection string will import each filed as string. You can also have OleDB convert the data read to whatever datatype it is meant to be so that you do not have to litter your code with lots of Convert.ToXXXX to convert the String data to whatever.
This requires using a Schema.INI to define the file. This replaces most of the Extended Properties in the connection string leaving only Extended Properties='TEXT';" (which means use the TEXT Driver). Create a file name Schema.INI in the same folder as the data:
[Capitals.txt]
ColNameHeader=True
CharacterSet=437
Format=TabDelimited
TextDelimiter="
DecimalSymbol=.
CurrencySymbol=$
Col1="Country" Text Width 254
Col2="Capital City" Text Width 254
Col3="Population" Single
Col4="Fake" Integer
One Schema.INI can contain the layout for many files. Each file has its own section titled with the name of the file (e.g. [FooBar.CSV], [Capitals.txt]etc)
Most of the entries should be self-explanatory, but FORMAT defines the column delimiter (TabDelimited, CSVDelimited or custom Delimited(;)); TextDelimiter is the character is used to enclose column data when it might contain spaces or other special characters. Things like CurrencySymbol lets you allow for a foreign symbol and can be omitted.
The ColN= listings are where you can rename columns and specify the datatype. This might be tedious to enter for 100+ columns, however it would probably be mostly copy and paste. Once it is done you'd always have it and be able to easily use typed data.
You do not need to specify the column names/size/type to use a Schema.INI If the file includes column names as the first row (ColNameHeader=True), you can use the Schema simply to specify the various parameters in a clear and readable fashion rather than squeezing them into the connection string.
OleDB looks for a Schema.INI in the same folder as the import file, and then looks for a section bearing the exact name of the "table" used in the SQL:
' form level DT var
Private capDT As DataTable
' procedure code to load the file:
Dim connstr = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source='C:\Temp';Extended Properties='TEXT';"
Dim SQL = "SELECT * FROM Capitals.txt"
capDT = New DataTable
' USING will close and dispose of resources
Using cn As New OleDbConnection(connstr),
cmd As New OleDbCommand(SQL, cn)
cn.Open()
Using da As New OleDbDataAdapter(cmd)
da.Fill(capDT)
End Using
End Using ' close and dispose
The DataTable is now ready to use. If we iterate the columns, you can see they match the Type specified in the schema:
' display data types
For n As Int32 = 0 To capDT.Columns.Count - 1
Console.WriteLine("name: {0}, datatype: {1}",
capDT.Columns(n).ColumnName,
capDT.Columns(n).DataType.ToString)
Next
Output:
name: Country, datatype: System.String
name: Capital City, datatype: System.String
name: Population, datatype: System.Single
name: Fake, datatype: System.Int32
See also:
Schema.INI for most legal settings
Code Page Identifiers for the values to use for CharacterSet
I have this txt file with the following information:
National_Insurence_Number;Name;Surname;Hours_Worked;Price_Per_Hour so:
eg.: aa-12-34-56-a;Peter;Smith;36;12
This data has been inputed to the txt file through a VB form which works totally fine, the problem comes when, on another form. This is what I expect it to do:
The user will input into a text box the employees NI Number.
The program will then search through the file that NI Number and, if found;
It will fill in the appropriate text boxes with its data.
(Then the program calculates tax and national insurance which i got working fine)
So basically the problem comes telling the program to search that NI number and introduce each ";" delimited field into its corresponding text box.
Thanks for all.
You just need to parse the file like a csv, you can use Microsoft.VisualBasic.FileIO.TextFieldParser to do this or you can use CSVHelper - https://github.com/JoshClose/CsvHelper
I've used csv helper in the past and it works great, it allows you to create a class with the structure of the records in your data file then imports the data into a list of these for searching.
You can look here for more info on TextFieldParser if you want to go that way -
Parse Delimited CSV in .NET
Dim afile As FileIO.TextFieldParser = New FileIO.TextFieldParser(FileName)
Dim CurrentRecord As String() ' this array will hold each line of data
afile.TextFieldType = FileIO.FieldType.Delimited
afile.Delimiters = New String() {";"}
afile.HasFieldsEnclosedInQuotes = True
' parse the actual file
Do While Not afile.EndOfData
Try
CurrentRecord = afile.ReadFields
Catch ex As FileIO.MalformedLineException
Stop
End Try
Loop
I'd recommend using CsvHelper though, the documentation is pretty good and working with objects is much easier opposed to the raw string data.
Once you have found the record you can then manually set the text of each text box on your form or use a bindingsource.
I am using asp.Net MVC application to upload the excel data from its CSV form to database. While reading the csv file using the Stream Reader, if line contains lower case letter followed by Upper case, it splits in two line . EX.
Line :"1,This is nothing but the Example to explanationIt results wrong, testing example"
This line splits to :
Line 1: 1,This is nothing but the Example to explanation"
Line 2:""
Line 3:It results wrong, testing example
where as CSV file generates right as ""1,This is nothing but the Example to explanationIt results wrong, testing example"
code :
Dim csvFileReader As New StreamReader("my csv file Path")
While Not csvFileReader.EndOfStream()
Dim _line = csvFileReader.ReadLine()
End While
Why should this is happening ? how to resolve this.
When a cell in an excel spreadsheet contains multiple lines, and it is saved to a CSV file, excel separates the lines in the cell with a line-feed character (ASCII value 0x0A). Each row in the spreadsheet is separated with the typical carriage-return/line-feed pair (0x0D 0x0A). When you open the CSV file in notepad, it does not show the lone LF character at all, so it looks like it all runs together on one line. So, in the CSV file, even though notepad doesn't show it, it actually looks like this:
' 1,"This is nothing but the Example to explanation{LF}It results wrong",testing example{CR}{LF}
According to the MSDN documentation on the StreamReader.Readline method:
A line is defined as a sequence of characters followed by a line feed ("\n"), a carriage return ("\r"), or a carriage return immediately followed by a line feed ("\r\n").
Therefore, when you call ReadLine, it will stop reading at the end of the first line in a multi-line cell. To avoid this, you would need to use a different "read" method and then split on CR/LF pairs rather than on either individually.
However, this isn't the only issue you will run into with reading CSV files. For instance, you also need to properly handle the way quotation characters in a cell are escaped in CSV. In such cases, unless it's really necessary to implement it in your own way, it's better to use an existing library to read the file. In this case, Microsoft provides a class in the .NET framework that properly handles reading CSV files (including ones with multi-line cells). The name of the class is TextFieldParser and it's in the Microsoft.VisualBasic.FileIO namespace. Here's the link to a page in the MSDN that explains how to use it to read a CSV file:
http://msdn.microsoft.com/en-us/library/cakac7e6
Here's an example:
Using reader As New TextFieldParser("my csv file Path")
reader.TextFieldType = FieldType.Delimited
reader.SetDelimiters(",")
While Not reader.EndOfData
Try
Dim fields() as String = reader.ReadFields()
' Process fields in this row ...
Catch ex As MalformedLineException
' Handle exception ...
End Try
End While
End Using
How can I configure a dataflow task that takes data from a MS SQL Server 2008 datasource and puts it in an Excel file where the filename looks like 'date filename'.xls?
Excel is the biggest pain to deal with in SSIS. Usually I store a template file that just has the column headers and nothing else. I start with a task to copy the template file to the processing directory. You can use variables to create the file name in an expression at this point. Alternatively, you can create the file in the dataflow and then rename the file in a step after the data flow. With text files, I have dynamically created the connection in an expression, but Excel seems to be funny about that.
Provided that your column definition don't change.... you can go to
Right Click on Excel Connection Manager
Expression
Select connectionstring
bulid expression (for Example : (DT_WSTR, 50) GETDATE() + #[user::FileName] +".xlsx")
Select the properties for Excel Connection Manager instance, Click on the ellipsis for 'Expressions 'property and set an expression for 'ExcelFilePath' to a variable with a valid path to an excel file, this takes cares of the connection string.
You do need a variable valid excel file at the design time, otherwise connection manager does not work, you can overwrite it at run time using a script task to point to the excel file that does not exist at design time.
HLGEM's answer certainly helps. As #sql-rookie requested, I'll provide a sample about how to copy the excel file to a new file named with current date. Hope this will help others with the same question in the future.
Basically you just need to add an additional File System Task after the Data Flow Task. And use a variable for the Destination File Path: "\\\\file\\"+SUBSTRING((DT_STR,30, 1252) GETDATE(), 1, 10) +".xlsx"