Pentaho Data Integration (CE) 5.0.1-stable is trying to generate SQL with a column that has a data type as UNKNOWN
example:
, ad_unit_type VARCHAR(255)
, creation_time UNKNOWN
, title VARCHAR(255)
Original Table Input column is DATETIME
There are no empty/null field values
There are no transformations on field
Is there a way to force Pentaho to recognize the field as DATETIME in the transformation stream?
Best
Insert Select values step
Click to open/edit step
Tab over to Meta-data
Create new field with stream field name (ex: creation_time)
Set type as Date
Press OK
Can also directly define the data type in the table input step (using SQL's CAST/CONVERT).
Now, if you go to Action > View SQL you can see that the UNKNOWN data type has been replaced with DATETIME.
Related
I am receiving an error
Conversion failed when converting the varchar value to data type int
while trying to insert data from one table into another. Both have the same table structure (table being inserted is an exact copy of the one used in the Select) and data types on the columns are the same.
INSERT INTO PS_PSOPRDEFN_BA
SELECT *
FROM PSOPRDEFN
Error:
Conversion failed when converting the varchar value '11000_600' to data type int.
The column this is inserting with this value is a varchar(30) in both tables, so I don't know why SQL is trying to convert it to int. Any ideas are appreciated.
When doing inserts, always include the columns:
INSERT INTO PS_PSOPRDEFN_BA ( . . . ) -- column list here
SELECT . . . -- column list here
FROM PSOPRDEFN;
You have a value which is a string which is being assigned to an integer column, and the value cannot be converted.
When doing an insert, the columns are aligned by order in the column list, not by name. So, merely having the same name doesn't mean that the code will work. The tables have to have exactly the same columns defined in the same order with compatible types for your code to work.
I am running below query in Jasper (using Oracle DB) but returning empty because the data type of the name is varchar but it is returning if I select the column which has number or date as a data type.
select name from student_data;
attached is the screenshot of the property. and below is the query
select name, date_of_birth from student_data. i will have data for date_of_birth but name column is empty.
enter image description here
Jaspersoft-Studio and also the older version iReport have the Problem, that they are implementet with the wrong datatype. for example
DB column: ID(int)
iReport_FIELD: ID(BigDecimal)
But you should be able to simply change the Datatype of the Field.
Go to the "Fields" part of your Navigation-Window and select the Field with the wrong Datatype
On the Property-Window you can see one of The Properties of the Fiels is "Field Class", There you can change the Class of your name Field.
I have a table input and gets some data from a SQL Server table. One field has values of type time, e.g. 02:22:57.0000000, the destination table (table output ) is a PostgreSQL table and has data type of time for that field. But PDI seems think the time from the source table is of type string and generates an error.
ERROR: column "contact_time" is of type time without time zone but expression is of type character varying
I tried using select value step, but there is no time type, only date and timestamp. How should I do?
You can use Select Values step and in the meta-data tab, select Type as Timestamp and Format as HH:mm:ss
This will format your string input to timestamp.
Hope this helps :)
I have a program where the user will have the option to map a variety of data source with an unpredictable column schema. So they might have a SQL Server database with 10 fields or an excel file with 20 - the names can be anything and data types of the fields can be a mixture of text and numeric, dates, etc.
The user then has to provide a mapping of what each field means. So column 4 is a "LocName", column 2 is a "LocX", column 1 is a "LocDate", etc. The names and data types that the user is presented as options to map to is well defined by a DataSet DataTable (XSD xchema file).
For example, if the source contains data formatted like this:
User Column 1: "LocationDate" of type string
User Column 2: "XCoord" of type string
User Column 3: "YCoord" of type string
User Column 4: "LocationName" of type int
and the user provides a mapping that would require that translates to this for the Application required DataTable:
Application Column "LocName" of type string = Column **4** of user table
Application Column "LocX" of type double = Column **2** of user table
Application Column "LocY" of type double = Column **3** of user table
Application Column "LocDate" of type datetime = Column **1** of user table
I have routines that connect to the source and pull out the data for a user query in "raw" format as a DataTable - so it takes the schema of the source.
My question is, what is the best way to then "transform" the data from the raw DataTable to the required application DataTable bearing in mine that this projection has to account for type conversions?
A foreach would obviously work but that seems like brute force since it will have to account for the data types with every loop on each row. Is the a "slick" way to do it with LINQ or ADO.NET?
I would normally do in select that "looks like" the destination table, but with data from the source table. You would apply the data conversions also as required.
Select
Cast (LocationNameLocationName As varChar(...) As LocName
, LocX As XCoord
, ...
From SourceTable
Hard to describe in a simple answer. what I've done in the past is issue an "empty" query like "Select * From sourcetable Where 1=0" which returns no rows but makes all the columns and their types available in the result set. You can cycle through the column ADO objects to get the type of each. You can then use that info to dynamically build a real SQL statement with the conversions.
You still have a lot of logic to decide the conversion, but they all happen as you're building the statement, not as the table is being read. You still have to say in code "if the source column is Integer and the destination column is character, then I want to generate into the select ', Cast ( as Varchar) '"
When you finish building the text of the select, you have a select you can run in ADO to get the rows and the actual move becomes as simple as read/write with the field coming in just as you want them. You can also use that select for an "Insert into Select ...".
Hope this makes sense. The concept is easier to do than to describe.
I'm making a report with Visual Studio 2008, pulling the info from a table in a database. Let's say I just want to show the Type of employee (which is represented by an int, say 1-10) and their Name. Then my dataset would be this query:
SELECT Type, Name
FROM Employees
WHERE Type = #Type
This is SSRS, so I do not need to declare or set the variable (correct me if I'm wrong). When I run the report, it will give me an option to type in an int for the Type and it will create the corresponding report. My question is how can I set it so that I can type 1,2,3 in for the Type so that I get a report with those types of employees? So having the int variable be able to accept a list of ints or just one int. Essentially the "resulting query" from that example would look like this:
SELECT Type, Name
FROM Employees
WHERE Type = 1 AND Type = 2 AND Type = 3
On the left side in the Report Data window under the Parameters folder, right click the variable, hit Parameter Properties, then check 'Allow Multiple Values' and select the correct datatype in the dropdown. I'm not sure why it decides to make a dropdown when you run the report, and you have to enter the values each on their own line instead of separated by commas, but it seems to work fine. Then just change the WHERE clause in your dataset from WHERE Type = #Type to WHERE Type IN (#Type). I don't know if the parentheses are necessary.
Also if you create a separate dataset that will present certain values you can have those show up instead of having to type them out. For example, create a dataset that contains this query
SELECT DISTINCT Type
FROM Employees
ORDER BY Type
This will create a list of distinct values for type that you can check/uncheck. You can make it more complex obviously as well.