Related
I want to create a calculation that adds the dimensions current month and previous month to a Cognos Data Module. The Month format is 2022/11. This is what I tried. I do not get an error message, but the calculation does not return a result.
Case
when (Month_Adj = #timestampMask(_add_months($current_timestamp,0),'yyyy')+'/'+timestampMask(_add_months($current_timestamp,0),'mm')#) then 'Last Month'
when (Month_Adj = #timestampMask(_add_months($current_timestamp,-1),'yyyy')+'/'+timestampMask(_add_months($current_timestamp,-1),'mm')#) then 'Previous Month'
else null
end
Please find a screenshot for reference.
You are comparing a decimal result from your macro to a character value in your data. You don't have syntax errors because SQL implicitly casts the decimal value for the comparison. But the values are unlikely to match.
Using today's date, your macro code...
#timestampMask(_add_months($current_timestamp,0),'yyyy')+'/'+timestampMask(_add_months($current_timestamp,0),'mm')#
...should produce...
CAST(2022 AS DOUBLE PRECISION) / 11
So the resulting SQL is effectively...
Case
when (Month_Adj = 183.818181812) then 'Last Month'
when (Month_Adj = 202.2) then 'Previous Month'
else null
end
Use the sq() function to put quotes around the values returned by the macro functions before concatenating the three parts of the expression.
#sq(timestampMask(_add_months($current_timestamp, 0),'yyyy'))# + '/' + #sq(timestampMask(_add_months($current_timestamp, 0),'mm'))#
You can see the SQL that Cognos is producing by...
At the report level, More (elipsis) | Show generated SQL/MDX
In the query editor, Properties | DATA | Generated SQL
The Cognos relative time filters allow you to create a set of predefined relative time measures without too much mucking about.
If you want to have calculations referencing two or more relative time measures, either from the same fact or from different fact tables, you would need to create stand alone calculations and make sure that the calculate after aggregation flag is on.
I think the trickiest part is getting the lookup reference in the fact table set up.
Here are two expressions which could be used as patterns for your lookup references.
The first one is suitable for converting key values in the 202211 format into Year-month date data types.
cast (
substring(
cast( MONTH_KEY as varchar(8)) ,1,4 ) || '-'
+
substring(
cast( MONTH_KEY as varchar(8)) ,5,2 )||'-01'
, date)
The second one is suitable for converting key values in the 20221128 format into year-month-day date data types.
cast (
substring(
cast( SHIP_DAY_KEY as varchar(10)) ,1,4 ) || '-'
+
substring(
cast( SHIP_DAY_KEY as varchar(10)) ,5,2 )|| '-'
||
substring(
cast( SHIP_DAY_KEY as varchar(10)) ,7,2 )
, date)
If the key values are more abstract you would need to find a way to generate values which would map to those of the relative time filters.
Here is a link to an overview of relative time.
https://www.ibm.com/docs/en/cognos-analytics/11.2.0?topic=analysis-sample-calendars
Here's how to customize them:
https://www.ibm.com/docs/en/cognos-analytics/11.2.0?topic=analysis-creating-relative-date-filters
https://www.ibm.com/docs/en/cognos-analytics/11.2.0?topic=calendars-creating-custom-retail-calendar
https://pmsquare.com/analytics-blog/2020/1/10/creating-custom-calendars-in-cognos-analytics-111
https://senturus.com/blog/how-to-customize-relative-time-in-cognos-data-modules/
In 11.2.3 They have implemented where clause support for relative time, which makes queries with them a bit more efficient as you are not doing a bunch of local processing schlepping through the query results looking for the values which fit into the filters as the query will now be filtered to only have those results you want.
https://www.ibm.com/docs/en/cognos-analytics/11.2.0?topic=analysis-query-optimization-relative-date-measures
I would like to store dates in the format CCYYMMDD in Teradata, but I fail to do so. Find below what I tried so far:
query 1:
SEL CAST(CAST(CURRENT_DATE AS DATE FORMAT 'YYYYMMDD') AS VARCHAR(8))
-- Output: 20191230 ==> this works!
query 2:
SEL CAST(CAST(CURRENT_DATE AS DATE FORMAT 'CCYYMMDD') AS VARCHAR(8))
-- output: SELECT Failed. [3530] Invalid FORMAT string 'CCYYMMDD'.
It seems that the CCYYMMDD is not available in Teradata right away. Is there a workaround?
Tool used: Teradata SQL assistant
Internally, dates are stored as integers in Teradata. So when you say you want to store them in a different format, I don't think you can do that. But you can choose how to display / return the values.
I'm sure there's a cleaner way to get the format you want, but here's one way:
WITH cte (mydate) AS (
SELECT CAST(CAST(CURRENT_DATE AS DATE FORMAT 'YYYYMMDD') AS CHAR(8)) AS mydate
)
SELECT
CAST(
(CAST(SUBSTRING(mydate FROM 1 FOR 2) AS INTEGER) + 1) -- generate "century" value
AS CHAR(2) -- cast value as string
) || SUBSTRING(mydate FROM 3) AS new_date -- add remaining portion of date string
FROM cte
SQL Fiddle - Postgres
You'd have to add some extra logic to handle years before 1000 and after 9999. I don't have a TD system to test, but give it a try and let me know.
I got a column called DateOfBirth in my csv file with Excel Date Serial Number Date
Example:
36464
37104
35412
When i formatted cells in excel these are converted as
36464 => 1/11/1999
37104 => 1/08/2001
35412 => 13/12/1996
I need to do this transformation in SSIS or in SQL. How can this be achieved?
In SQL:
select dateadd(d,36464,'1899-12-30')
-- or thanks to rcdmk
select CAST(36464 - 2 as SmallDateTime)
In SSIS, see here
http://msdn.microsoft.com/en-us/library/ms141719.aspx
The marked answer is not working fine, please change the date to "1899-12-30" instead of "1899-12-31".
select dateadd(d,36464,'1899-12-30')
You can cast it to a SQL SMALLDATETIME:
CAST(36464 - 2 as SMALLDATETIME)
MS SQL Server counts its dates from 01/01/1900 and Excel from 12/30/1899 = 2 days less.
tldr:
select cast(#Input - 2e as datetime)
Explanation:
Excel stores datetimes as a floating point number that represents elapsed time since the beginning of the 20th century, and SQL Server can readily cast between floats and datetimes in the same manner. The difference between Excel and SQL server's conversion of this number to datetimes is 2 days (as of 1900-03-01, that is). Using a literal of 2e for this difference informs SQL Server to implicitly convert other datatypes to floats for very input-friendly and simple queries:
select
cast('43861.875433912' - 2e as datetime) as ExcelToSql, -- even varchar works!
cast(cast('2020-01-31 21:00:37.490' as datetime) + 2e as float) as SqlToExcel
-- Results:
-- ExcelToSql SqlToExcel
-- 2020-01-31 21:00:37.490 43861.875433912
this actually worked for me
dateadd(mi,CONVERT(numeric(17,5),41869.166666666664)*1440,'1899-12-30')
(minus 1 more day in the date)
referring to the negative commented post
SSIS Solution
The DT_DATE data type is implemented using an 8-byte floating-point number. Days are represented by whole number increments, starting with 30 December 1899, and midnight as time zero. Hour values are expressed as the absolute value of the fractional part of the number. However, a floating point value cannot represent all real values; therefore, there are limits on the range of dates that can be presented in DT_DATE. Read more
From the description above you can see that you can convert these values implicitly when mapping them to a DT_DATE Column after converting it to a 8-byte floating-point number DT_R8.
Use a derived column transformation to convert this column to 8-byte floating-point number:
(DT_R8)[dateColumn]
Then map it to a DT_DATE column
Or cast it twice:
(DT_DATE)(DT_R8)[dateColumn]
You can check my full answer here:
Is there a better way to parse [Integer].[Integer] style dates in SSIS?
Found this topic helpful so much so created a quick SQL UDF for it.
CREATE FUNCTION dbo.ConvertExcelSerialDateToSQL
(
#serial INT
)
RETURNS DATETIME
AS
BEGIN
DECLARE #dt AS DATETIME
SELECT #dt =
CASE
WHEN #serial is not null THEN CAST(#serial - 2 AS DATETIME)
ELSE NULL
END
RETURN #dt
END
GO
I had to take this to the next level because my Excel dates also had times, so I had values like this:
42039.46406 --> 02/04/2015 11:08 AM
42002.37709 --> 12/29/2014 09:03 AM
42032.61869 --> 01/28/2015 02:50 PM
(also, to complicate it a little more, my numeric value with decimal was saved as an NVARCHAR)
The SQL I used to make this conversion is:
SELECT DATEADD(SECOND, (
CONVERT(FLOAT, t.ColumnName) -
FLOOR(CONVERT(FLOAT, t.ColumnName))
) * 86400,
DATEADD(DAY, CONVERT(FLOAT, t.ColumnName), '1899-12-30')
)
In postgresql, you can use the following syntax:
SELECT ((DATE('1899-12-30') + INTERVAL '1 day' * FLOOR(38242.7711805556)) + (INTERVAL '1 sec' * (38242.7711805556 - FLOOR(38242.7711805556)) * 3600 * 24)) as date
In this case, 38242.7711805556 represents 2004-09-12 18:30:30 in excel format
In addition of #Nick.McDermaid answer I would like to post this solution, which convert not only the day but also the hours, minutes and seconds:
SELECT DATEADD(s, (42948.123 - FLOOR(42948.123))*3600*24, dateadd(d, FLOOR(42948.123),'1899-12-30'))
For example
42948.123 to 2017-08-01 02:57:07.000
42818.7166666667 to 2017-03-24 17:12:00.000
You can do this if you just need to display the date in a view:
CAST will be faster than CONVERT if you have a large amount of data, also remember to subtract (2) from the excel date:
CAST(CAST(CAST([Column_With_Date]-2 AS INT)AS smalldatetime) AS DATE)
If you need to update the column to show a date you can either update through a join (self join if necessary) or simply try the following:
You may not need to cast the excel date as INT but since the table I was working with was a varchar I had to do that manipulation first. I also did not want the "time" element so I needed to remove that element with the final cast as "date."
UPDATE [Table_with_Date]
SET [Column_With_Excel_Date] = CAST(CAST(CAST([Column_With_Excel_Date]-2 AS INT)AS smalldatetime) AS DATE)
If you are unsure of what you would like to do with this test and re-test! Make a copy of your table if you need. You can always create a view!
Google BigQuery solution
Standard SQL
Select Date, DATETIME_ADD(DATETIME(xy, xm, xd, 0, 0, 0), INTERVAL xonlyseconds SECOND) xaxsa
from (
Select Date, EXTRACT(YEAR FROM xonlydate) xy, EXTRACT(MONTH FROM xonlydate) xm, EXTRACT(DAY FROM xonlydate) xd, xonlyseconds
From (
Select Date
, DATE_ADD(DATE '1899-12-30', INTERVAL cast(FLOOR(cast(Date as FLOAT64)) as INT64) DAY ) xonlydate
, cast(FLOOR( ( cast(Date as FLOAT64) - cast(FLOOR( cast(Date as FLOAT64)) as INT64) ) * 86400 ) as INT64) xonlyseconds
FROM (Select '43168.682974537034' Date) -- 09.03.2018 16:23:28
) xx1
)
For those looking how to do this in excel (outside of formatting to a date field) you can do this by using the Text function https://exceljet.net/excel-functions/excel-text-function
i.e.
A1 = 132134
=Text(A1,"MM-DD-YYYY") will result in a date
This worked for me because sometimes the field was a numeric to get the time portion.
Command:
dateadd(mi,CONVERT(numeric(17,5),41869.166666666664)*1440,'1899-12-31')
I want to convert dateformat from mm/dd/yyyy to yyyy/mm/dd. I want the output in datetime format.
I tried this
convert(datetime,convert(varchar,getdate(),111),123)
but doesn't work. The error is "explicit conversion to datetime not available"
What is the best way to solve this problem? I'm using Sybase.
Try this
select convert(varchar,CAST('12/11/2010' as DateTime),111)
That won't work. The DATETIME data type has its own format that is really the amount of time that has passed since a fixed reference date; if you ask for a DATETIME it will always be returned according to that format.
How it is displayed to an end user is a function of the client. You can use CONVERT to convert it to a string and specify a format for how it is displayed in the string, but then you're returning a string, not a DATETIME. You can return it as a DATETIME (which has no inherent display format), and then it is up to the client application or OS to define how it is formatted for display. In client applications you also typically have formatting functions that display a date/time according to a format you specify. And if you haven't specified it explicitly in an application, then the display of the date/time will typically be defined by the localization settings in the OS.
Basically, there is a difference between the data type - DATETIME - and its representation to end users.
Formatting is something that should be done in the presentation tier not the data tier. However, most vendors, like Sybase, provide the ability to do rudimentary formatting:
Select Cast( Year(GetDate()) As char(4) )
+ '/' + Right( '00' + Cast( Month(GetDate()) As varchar(2) ), 2 )
+ '/' + Right( '00' + Cast( Day(GetDate()) As varchar(2) ), 2 )
Try this query
select (CONVERT(varchar(10), GETDATE(), 120))
I have to write an SQL view that returns the time part of a datetime column as a string in the format hhmmss (apparently SAP BW doesn't understand hh:mm:ss).
This code is the SAP recommended way to do this, but I think there must be a better, more elegant way to accomplish this
TIME = case len(convert(varchar(2), datepart(hh, timecolumn)))
when 1 then /* Hour Part of TIMES */
case convert(varchar(2), datepart(hh, timecolumn))
when '0' then '24' /* Map 00 to 24 ( TIMES ) */
else '0' + convert(varchar(1), datepart(hh, timecolumn))
end
else convert(varchar(2), datepart(hh, timecolumn))
end
+ case len(convert(varchar(2), datepart(mi, timecolumn)))
when 1 then '0' + convert(varchar(1), datepart(mi, timecolumn))
else convert(varchar(2), datepart(mi, timecolumn))
end
+ case len(convert(varchar(2), datepart(ss, timecolumn)))
when 1 then '0' + convert(varchar(1), datepart(ss, timecolumn))
else convert(varchar(2), datepart(ss, timecolumn))
end
This accomplishes the desired result, 21:10:45 is displayed as 211045.
I'd love for something more compact and easily readable but so far I've come up with nothing that works.
NOTE:
The question says that the column is of datatype DATETIME, not the newer (SQL Server 2008) TIME datatype.
ANSWER:
REPLACE(CONVERT(VARCHAR(8),timecolumn,8),':','')
Let's unpack that.
First, CONVERT formats the time portion of the datetime into a varchar, in format 'hh:mi:ss' (24-hour clock), as specified by the format style value of 8.
Next, the REPLACE function removes the colons, to get varchar in format 'hhmiss'.
That should be sufficient to get a usable string in the format you'd need.
FOLLOW-UP QUESTION
(asked by the OP question)
Is an inline expression faster/less server intensive than a user defined function?
The quick answer is yes. The longer answer is: it depends on several factors, and you really need to measure the performance to determine if that's actually true or not.
I created and executed a rudimentary test case:
-- sample table
create table tmp.dummy_datetimes (c1 datetime)
-- populate with a row for every minute between two dates
insert into tmp.dummy_datetimes
select * from udfDateTimes('2007-01-01','2009-01-01',1,'minute')
(1052641 row(s) affected)
-- verify table contents
select min(c1) as _max
, max(c1) as _min
, count(1) as _cnt
from tmp.dummy_datetimes
_cnt _min _max
------- ----------------------- -----------------------
1052641 2007-01-01 00:00:00.000 2009-01-01 00:00:00.000
(Note, the udfDateTimes function returns the set of all datetime values between two datetime values at the specified interval. In this case, I populated the dummy table with rows for each minute for two entire years. That's on the order of a million ( 2x365x24x60 ) rows.
Now, user defined function that performs the same conversion as the inline expression, using identical syntax:
CREATE FUNCTION [tmp].[udfStrippedTime] (#ad DATETIME)
RETURNS VARCHAR(6)
BEGIN
-- Purpose: format time portion of datetime argument to 'hhmiss'
-- (for performance comparison to equivalent inline expression)
-- Modified:
-- 28-MAY-2009 spencer7593
RETURN replace(convert(varchar(8),#ad,8),':','')
END
NOTE: I know the function is not defined to be DETERMINISTIC. (I think that requires the function be declared with schema binding and some other declaration, like the PRAGMA required Oracle.) But since every datetime value is unique in the table, that shouldn't matter. The function is going to have to executed for each distinct value, even if it were properly declared to be DETERMINISTIC.
I'm not a SQL Server 'user defined function' guru here, so there may be something else I missed that will inadvertently and unnecessarily slow down the function.
Okay.
So for the test, I ran each of these queries alternately, first one, then the other, over and over in succession. The elapsed time of the first run was right in line with the subsequent runs. (Often that's not the case, and we want to throw out the time for first run.) SQL Server Management Studio reports query elapsed times to the nearest second, in format hh:mi:ss, so that's what I've reported here.
-- elapsed times for inline expression
select replace(convert(varchar(8),c1,8),':','') from tmp.dummy_datetimes
00:00:10
00:00:11
00:00:10
-- elapsed times for equivalent user defined function
select tmp.udfStrippedTime(c1) from tmp.dummy_datetimes
00:00:15
00:00:15
00:00:15
For this test case, we observe that the user defined function is on the order of 45% slower than an equivalent inline expression.
HTH
you could use a user-defined function like:
create FUNCTION [dbo].[udfStrippedTime]
(
#dt datetime
)
RETURNS varchar(32)
AS
BEGIN
declare #t varchar(32)
set #t = convert( varchar(32), #dt, 108 )
set #t = left(#t,2) + substring(#t,4,2)
RETURN #t
END
then
select dbo.udfStrippedTime(GETDATE())
the logic for the seconds is left as an exercise for the reader
Here's a question. Does the formatting need to happen on the Db Server? The server itself really only care about, and is optimized for storing the data. Viewing the data is usually the responsibility of hte layer above the Db (in a strictly academic sense, the real world is a bit more messy)
For instance, if you are outputting the result into an ASP.NET page bound to a GridControl you would just specify your DataFormattingString when you bind to the column. If you were using c# to write it to a text file, when you pull the data you would just pass the format string to the .ToString() function.
If you need it to be on the DbServer specifically, then yeah pretty much every solution is going to be messy because the time format you need, while compact and logical, is not a time format the server will recognize so you will need to manipulate it as a string.
This handles the 00 - > 24 conversion
SELECT CASE WHEN DATEPART(hh,timecolumn) = 0
THEN '24' + SUBSTRING(REPLACE(CONVERT(varchar(8),timecolumn, 108),':',''),3,4)
ELSE REPLACE(CONVERT(varchar(8),timecolumn, 108),':','') END
Edit 2: updated to handle 0 --> 24 conversion, and a shorter version:
select replace(left(replace(convert(char,getdate(),8),':',''),2),'00','24') + right(replace(convert(char,getdate(),8),':',''),4)
Back to the slightly longer version :)
SELECT replace(convert(varchar(15),datetimefield, 108), ':','')
from Table
SELECT REPLACE('2009-05-27 12:49:19', ':', '')
2009-05-27 124919