my input excel sheet has the field with two different types of values column in the format YYYY/MM/DD
Now, when I have added the excel sheet into Pentaho the columns along with datatype I got which shows string datatype in the date formats column. which you can see below
After this, I tried to integrate with postgres but I am unable to find the result the error which I got attached below
Updated
I tried with the given timestamp format yyyy/MM/dd HH:mm:ss this works fine for me but this format yyyy/MM/dd hh.00.00 is not present in the format column.
You have a column named Date in the field definition tab. Choose Date [data type] from the Dropdown box in the row date [column name] and timestamp [column name].
Try to get the data in the Excel Input step. If it does not work, try to write yyyy/MM/dd in the Format column for the date field and yyyy/MM/dd hh.00.00 for timestamp field. Note that the format is most probably unnecessary for the Excel Input.
Once you can get all the row in the Excel Input with a preview limit 0, and not before, try to put the data in the Postgres database.
Normally it should work. If not, use a Select Step which has a tab Meta-data, to change among other the format of the dates. Chose a format accepted by Postgres. Again, this change of format is most probably unnecessary.
To explain what happens under the wood, remember each field in the PDI has a type. You define time and timestamp as string. It is not an issue by itself, until you try to put those string in the database, which no not accept such date formats. The best way is to use the date type (which DO NOT have a format until you want to read or write them).
Select or put the corresponding format in [Format] column on [Fields] Tab.
Related
I have an excel file which has a date column with different datatypes.
For eg: few values have Date data-type while others have string format.
I tried to import the data and change the column metadata type to string but it changes the date values completely.
I have attached a few screenshots of the data, and I very new to Pentaho, so can anybody help me understand how to tackle this problem.
I tried changing the metadata type or using str2date function in js step but still, no use as data imported is different from the data from the file
When importing from an Excel sheet with some invalid dates, you can import as string format, then use a Select Values step.
Specify the date field on the Meta-data tab with the correct format (dd/MM/yyyy) and Date format Lenient? set to Y. This should change the 29/02/2017 to 01/03/2017, which is a decent option.
Also, don't use Excel to inspect the results, because it might be screwing up the conversion on re-import. Look at the preview data in Spoon or export to csv and look with a text editor to see if the format is correct first.
After changing the data type and importing the date values as string. Using SQL to store the date in string format and then formatting it while retrieving solved the issue.
I have infragistics2 v10.3.
I have ultra grid with two columns
|String|DateTime|
For DateTime I have custom formatting like "dd.MM.yyyy HH:mm" System Regional settings have short Date format like "dd-MMM-yy".
So grid shows 07.02.2013 14:00 and user See this.
The sytem have 07-Feb-13
When i want to filter all columns with text "fe" it is also shows me all rows which contains 07.02.2013.
During filtering it uses own formating and ignores my custom In InitializeLayout event we have done the following code to set date format for the StartDate column
e.Layout.Bands(0).Columns("StartDate").Style = Win.UltraWinGrid.ColumnStyle.Date
e.Layout.Bands(0).Columns("StartDate").Format = "dd/MM/yyyy"
e.Layout.Bands(0).Columns("StartDate").ButtonDisplayStyle = UltraWinGrid.ButtonDisplayStyle.Always
This date field column filter displaying based on Local system date format in opened calendar.
Please suggest me where I am doing wrong
As your are using string column type to store dates, I suspect how the dates are converted: 07/08/2015 is treated as August 7 or July 8?
You are setting the format for your grid but the data loaded in the DropDown may be in its original form as in the Database.
If you do not want to change the data type of your date column, I would advise that you set DateFormat within the select query as demonstrated in change-data-type-of-data-retrieved-by-select-query (but beware of the conversion errors or the date change scenario as pointed above)
If you are using SQL Server (I'm not sure about other databases) you can use a UDF (User Defined Function) in the select query as in this example of UDF on TechNet or this post on SO: using-udf-in-select-statement
Along with that don't forget to set the Format of your DropDown, if defined explicitly on initialize or on entering edit mode.
You need to set MaskInput property of the column like this:
e.Layout.Bands(0).Columns("StartDate").MaskInput = "dd/mm/yyyy";
Keep in mind when cell is in edit mode it uses an editor to display the edited value. So when you need to fix anything in edit mode you need to fix it in the editor.
I have noticed some dates in one table are stored like 21/08/2014 or 21/08/14 so when I run my parameter dates i dd/mm/yyyy it will miss some records out. How best ca I format this table colum to ensure this doesnt happen?
To correct erroneous data in that column you could do a find and replace on "/14" changing to "/2014".
To prevent future erroneous data entry you could go into the table's properties and use an input mask to only accept dd/mm/yyyy dates.
I have an input spreadsheet that needs to get sorted by date. The current format of the date is in the UK format (dd/mm/yyyy) but I need it in yyyy-mm-dd (actually I don't, I just need to sort it and that format is the most foolproof way of sorting). This all needs to be done in VBA as it's part of a bigger project that allows a bunch of data collation at once. The other problem is that the input sheet can be quite large (150,000+ rows). So, while I could parse through each row of data and change it around to the way I need, this would be horrifically slow and is NOT an option.
Currently I'm using this bit of code to format the date to yyyy-mm-dd:
inputGADRSheet.Columns(7).NumberFormat = "yyyy-mm-dd"
But, Excel outsmarts me and assumes that the date format of the column is originally in the US format (mm/dd/yyyy) which messes everything up and half of the values in the column don't meet that requirement (days above the 12th) so they don't get formatted at all. Is there any way to tell Excel what format the current data is in? That way it won't just assume that it's in the US date format...
Is the solution to change my Excel region to the UK. I assume this could be done using VBA, but it seems risky...
If your data is already in an Excel column, you can't reinterpret the values: Excel date values are (internally) number, 1 representing 1900-01-01. After the data has been (mis-)interpreted by Excel there's no way back.
The question is: Where do you get the input data sheet from? If the dates are entered correctly, reformatting is possible without any problem and does not affect sorting (which depends only on the numeric value of the date). If your data comes from a text file (probably .csv-kind), be sure to read ii as text and use Excel worksheet functions or VBA to interpret the values.
I am importing an excel file into Access 2010 and the date field(CALLDATE) comes in as text(YYYYMMDD). I would like to use an update query to update a new field "dateofcall" but using a date/time format. I tried to use:
UPDATE tbl_Import SET tbl_Import.dateofcall = CDate([tbl_Import].[CALLDATE]);
I thought it was going to be that simple but it shows up blank with a date format. I also tried to use DateSerriel() but still was getting errors. Any suggestions?
You can use left, right and mid string functions to construct a date from the various parts of the string.
For example:
DateSerial(Left(MyTextDate,4),Mid(MyTextDate,5,2),Right(MytextDate,2))
You can use the above in an Update query to update a date type coulmn 9field) to a the date from the text column.
My date came in as text looking like this:"2013-03-23 00:00:00.0"
I take the left 10 characters only, "2013-03-23", this makes it so Access can recognize it is a date field and then I just switch around the format.
Format(Left([WEEKEND],10),"m/d/yyyy")