I wonder if someone is able to help me out here.
My company is a Google Workspace based company, we pipe our data through to BigQuery using the settings within the Admin Panel. That all works fantastically and there is no issue there.
Within BigQuery, one of the data points is time_usec - which is the Epoch time.
In one table (activity), this is done in Micros which is fine as I can transform the data in BigQuery using TIMESTAMP_MICROS(time_usec).
However, in Looker/Data Studio we are bringing the data in its raw format.
I have tried the calculated field TODATE(time_usec, "MICROS", "%x") which certainly transforms the data for display purposes but it doesn't appear to work with Date Ranges.
To explain this better:
Here is an example of the data in BigQuery. This is part of the Activity table with:
time_usec being epoch time (in microseconds)
email being the user account (every activity that is logged has a time_usec entry)
device_type being the type of device (everything from Windows machines, to iOS etc)
device_model being the model of the device
os_version being the version of the OS running on the device
To be fair, the data for the most part is irrelevant here (other than the time_usec) but I'm adding it to give a better idea of the type of data.
time_usec
email
device_type
device_model
os_version
1659952732837000
new.user.3#somemailservice.com
DESKTOP_CHROME
ChromeOs 14816.131.0
1659952299942000
some.email#somemailservice.com
WINDOWS
HP EliteBook 850 G5
Windows 10.0.19044
1659952366245000
new.user.3#somemailservice.com
DESKTOP_CHROME
1659952736142000
new.user.3#somemailservice.com
DESKTOP_CHROME
ChromeOs 14816.131.0
1659945047719000
some.email#somemailservice.com
WINDOWS
HP EliteBook 850 G5
Windows 10.0.19044
1659959338167000
another.email#somemailservice.com
DESKTOP_CHROME
HP Elite Dragonfly Chromebook
ChromeOs 14909.100.0
1659959340697000
another.email#somemailservice.com
DESKTOP_CHROME
HP Elite Dragonfly Chromebook
ChromeOs 14909.100.0
1659961092792000
some.email#somemailservice.com
WINDOWS
HP EliteBook 850 G5
Windows 10.0.19044
1659958186331000
another.email#somemailservice.com
WINDOWS
HP COMPAQ PRO 6305 SFF
Windows 10.0.19044
1659957469855000
some.email#somemailservice.com
WINDOWS
HP EliteBook 850 G5
Windows 10.0.19044
Here is how the connection is set up in Looker/Data Studio:
Here is how the time_usec field in configured in Looker/Data Studio:
Here is the page in Looker/Data Studio:
The column titled Date with the red underline is using TODATE(time_usec, "MICROS", "%x").
This is configured as a Date Range Dimension using the above method, then added as a normal dimension after. For the purpose of displaying a date, this works fine.
The yellow box gives us the ability to select a date range. In the image it is set to 'Auto date range' and it works fine. However, if I change this to, let's say the last 14 days it returns No data on the table.
As far as the Data itself is concerned, the time_usec field is set as a Number with no default aggregation. I have seen other questions about this answered in the past, with people saying that you can configure the field to be a Date rather than a number. However, attempting to do so produces the following message:
Looker Studio can't convert time_usec (Number) to a Date or Date &
Time because it doesn't recognize the date format.
Possible solutions:
Change your data to use a supported format
Create a calculated field to convert time_usec to a valid date. For example,
to convert "202011" to a date consisting of year and month: Example:
PARSE_DATE("%Y%m", time_usec)
I have also tried PARSE_DATETIME("%x", time_usec) and PARSE_DATE("%x", time_usec) as a calculated field. Again, for the purpose of displaying a date these seem to work fine. However, when then appying the date range it breaks with a message saying:
Am I doing something wrong here? I would rather not mess with the data at the BigQuery level. And I know I can use custom SQL to do the TIMESTAMP_MICROS(time_usec) then bring it in. But surely there is a better way to do this within Looker/Data Studio?
EDITED:
Added an example table with data and some config screenshot.
here's a formula that's working good on my end with date range control filter as well. please try it
DATE_FROM_UNIX_DATE(CAST(Time Only/86400000000 as INT64))
I searched for a long time for a solution to my similar problem.
I had the time in Unix microseconds. This is how I converted it to a date:
PARSE_DATETIME("%d-%m-%Y %H:%M:%S", CONCAT(DAY(time_usec, "MICROS"),"-",MONTH(time_usec, "MICROS"),"-",YEAR(time_usec, "MICROS")," ",HOUR(time_usec, "MICROS"),":",MINUTE(time_usec, "MICROS"),":",SECOND(time_usec, "MICROS")))
moment version is 2.24.0
My date value is 2019-04-23 03:16:00 +0000 UTC
I use moment to let it become to Asia time just like:
const moment = require('moment');
const localTime = moment(date).format('YYYY/MM/DD HH:mm');
<Text>{localTime}</Text>
localTime will show 2019/04/23 11:16
It works when I test it on debug mode.
But when I close the debug mode localTime will be
invalid date
The issue happen both of Android and IOS.
Any ideas ?
For anyone else experiencing date/time issues with Moment on Android especially if you're moving from React Native 0.59 (or older) to 0.60+, it appears that Hermes changes the way the Android works with Moment/dates. However, it would work when the debugger was enabled. Turns out, when you run the debugger, it switches back to the Chromium engine (or V8?) from Hermes. Resulted in us having to use console logs to track down Moment parsing issues. Oddly, the issues also occurred when trying the same manipulations in Safari.
If you're parsing dates passed in via different vars for day, month, year and the day or month does not have preceding zeroes (ex: 01 vs 1) then I recommend doing this:
const momentFormat = { y: birthYear, m: birthMonth, d: birthDay };
return Moment(momentFormat).format('MMMM Do, YYYY');
This manually sets the values rather than relying on each value being correctly formatted or having to write custom code to ensure the day/month values have the 0 when needed.
I am working on following mail data in a file.. (data source:infochimps)
Message-ID: <33025919.1075857594206.JavaMail.evans#thyme>
Date: Wed, 13 Dec 2000 13:09:00 -0800 (PST)
From: john.arnold#enron.com
To: slafontaine#globalp.com
Subject: re:spreads
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
X-From: John Arnold
X-To: slafontaine#globalp.com # ENRON
X-cc:
X-bcc:
X-Folder: \John_Arnold_Dec2000\Notes Folders\'sent mail
X-Origin: Arnold-J
X-FileName: Jarnold.nsf
saw a lot of the bulls sell summer against length in front to mitigate
margins/absolute position limits/var. as these guys are taking off the
front, they are also buying back summer. el paso large buyer of next winter
today taking off spreads. certainly a reason why the spreads were so strong
on the way up and such a piece now. really the only one left with any risk
premium built in is h/j now. it was trading equivalent of 180 on access,
down 40+ from this morning. certainly if we are entering a period of bearish
................]
I am loading above data as:-
A = load '/root/test/enron_mail/maildir/*/*/*' using PigStorage(':') as (f1:chararray,f2:chararray);
but for the message body I am getting separate tuples as message body includes new lines..
how to consolidate last lines into one ?
I want below part in single tuple as:
saw a lot of the bulls sell summer against length in front to mitigate
margins/absolute position limits/var. as these guys are taking off the
front, they are also buying back summer. el paso large buyer of next winter
today taking off spreads. certainly a reason why the spreads were so strong
on the way up and such a piece now. really the only one left with any risk
premium built in is h/j now. it was trading equivalent of 180 on access,
down 40+ from this morning. certainly if we are entering a period of bearish
I have a Classic ASP application that I am working with date cut offs. My server resides in Central Time, but I am in Eastern time. What happens is my app thinks it is an hour earlier and my cut offs are an hour late. I am sure they would be 2 hours early if a user was in Pacific time.
What I am trying to figure out is if there is a way to either
tell the server to show me local time when you do a GetDate() on SQL or Now() in ASP
figure out some way to do an offset that I can run when the page first loads and use as needed.
I tried server side javascript, it returns Central Time too. Any help would be greatly appreciated!! Thanks in advance!
Dennis
UPDATE - 4/11/12 # 1:12pm:
I think that I found a work around for my application, but it would not work generically. I have geographic data for the location I am working with - zip code. I can grab the timezone from that - it would not fully work right for users in other timezones looking at the location, but it does not matter for my app since I just need to be focused on the end time for that location.
This is the other other way(s) I found were provided by JohnB below (specifically #4). thanks everyone. http://www.webmasterworld.com/forum47/600.htm (bottom)
EDIT
I tried server side javascript, it returns Central Time too.
Did you mean to say client side JavaScript? You definitely need to use client side script to get the user's device time (not server side script).
You should read this:
Daylight saving time and Timezone best practices
Primer on dealing with multiple time zones:
1) Make sure your database server is set to the correct Date/Time in its time zone. Properly account for Daylight Savings Time in its location. Set the server to do this automatically.
Configure automatic date and time synchronization on Windows Server 2008 R2
2) Create a table in your database with time zones and their offset from UTC (GMT).
Time zone
3) Always store Now() Date/Time in UTC. Every database vendor should have a UTC Date/Time Now() function (i.e. SYSUTCDATETIME() for SQL Server). This way all times are stored in a universal format agnostic to where the user happens to be sitting. Call Now() from your database, not the client, because mobile devices could be anywhere, but your database server stays in one spot.
Date and Time Data Types and Functions (Transact-SQL)
4) Have user input their local time zone and store it in your database.
5) When displaying Date/Time stored in UTC back to the user, convert the UTC Date/Time back to the user's time zone using the user's time zone offset. SQL Server makes this a little easier with datetimeoffset.
SQL Server How to persist and use a time across different time zones
The Death of DateTime?
6) If the user is setting an alarm, have them enter the trigger Date/Time in their local time zone. This way the user can change their local time zone if they move. Also, if time zone rules change you can just fix your time zone table (#2) and then the alarm will still trigger correctly. In your code, to test for alarm trigger, convert trigger time to UTC, and then compare against server time in UTC (i.e. SYSUTCDATETIME()).
7) Daylight Savings Time is tricky! (see 1st link)
In general, Time zone manipulation can't be done directly in classic ASP.
However, if you have full control of the server where the code is running, you can install a COM component written in a language that does have time zone support, then use that component from your classic ASP environment.
For example, you might write the following component in .NET with C#:
using System;
using System.Runtime.InteropServices;
namespace TimeZoneInfoCom
{
[ComVisible(true)]
[ClassInterface(ClassInterfaceType.AutoDispatch)]
[Guid("E0C70A94-352D-4C0B-8C2E-8066C88565C5")]
public class TimeZoneConverter
{
public DateTime NowInZone(string timeZoneId)
{
return TimeZoneInfo.ConvertTimeBySystemTimeZoneId(DateTime.UtcNow, timeZoneId);
}
public DateTime Convert(DateTime dateTime, string sourceZoneId, string targetZoneId)
{
TimeZoneInfo sourceTimeZone = TimeZoneInfo.FindSystemTimeZoneById(sourceZoneId);
TimeZoneInfo targetTimeZone = TimeZoneInfo.FindSystemTimeZoneById(targetZoneId);
return TimeZoneInfo.ConvertTime(dateTime, sourceTimeZone, targetTimeZone);
}
}
}
You would then compile this, copy the DLL to your server, and register it as a COM component (using RegAsm.exe).
Then you could call it in your Classic ASP page, like so:
<html>
<body>
Server Time: <%= Now() %><br>
<br>
<%
Dim tzconverter
Set tzconverter = Server.CreateObject("TimeZoneInfoCom.TimeZoneConverter")
%>
US Pacific Time: <%= tzconverter.NowInZone("Pacific Standard Time") %><br>
US Mountain Time: <%= tzconverter.NowInZone("Mountain Standard Time") %><br>
US Central Time: <%= tzconverter.NowInZone("Central Standard Time") %><br>
US Eastern Time: <%= tzconverter.NowInZone("Eastern Standard Time") %><br>
UTC: <%= tzconverter.NowInZone("UTC") %><br>
<br>
Conversion Example:
<%
Dim originalTime, convertedTime
originalTime = #12/31/2014 00:00:00#
convertedTime = tzconverter.Convert(originalTime, "UTC", "Tokyo Standard Time")
Response.Write(convertedTime)
%>
<%
' Don't forget to destroy the com object!
Set tzconverter = Nothing
%>
</body>
</html>
If you get an "ActiveX component can't create object" error, be sure that you have set "Enable 32-Bit Applications" to True in IIS, under the advanced settings for your application pool.
With regard to SQL Server - If you search, you may find a handful of posts showing ways you can manipulate time in SQL Server, through elaborate stored procedures that either have fixed offsets, fixed time zone rules, or rely on tables of time zone data. I usually advise against any of these approaches because they are too brittle.
Fixed offsets are bad because they don't account for daylight saving time.
Fixed rules are bad because time zone rules can (and do) change. Editing stored procs to keep up with these changes is too fragile (IMHO).
Maintaining tables of time zone data is a little better, but usually I find these tables to not be maintained well. If you go down this route, be sure to put a procedure in place for updating the tables periodically.
Getting the users time zone
http://www.pageloom.com/automatic-timezone-detection-with-javascript
From what I understand about this JavaScript code it is very accurate and will be able to return the offset from UST, the corresponding Olson Database timezone name, and handle the daylight savings time issue (ex. -5:00, America/New_york, true).
The only hurdle you will face after getting this code working on your html page will likely be getting these values to asp and then to sql if that is what you need. I achieved this by sending these values as a $.post using JQuery. I think this is the easiest way to do it. I believe the other alternatives are using an AJAX command or cookies.
After I got the values from the JavaScript code to the server I stored them as session variables so they would change if the user on my site logged in from a different timezone then usual. However they could easily be saved to the database as well.
//replace this comment with the most updated timezonedetect code from that first link
var timezone = jstz.determine_timezone();
var tzoffset = timezone.offset();
var tzname = timezone.name();
var tzdst = timezone.dst();
$.post("tzdetect.asp", { tzoffset: tzoffset, tzname: tzname, tzdst: tzdst } );
Then you need to set up the receiving file tzdetect.asp on the server to store the time zone once it's sent.
Working with the time zone once you have it
This article has a good solution to your problem: http://www.codeproject.com/Articles/31146/SQL-2005-Time-Zone-Conversion-Functions
Approach is to set up a scalar function called
NEW_TIME that takes three parameters:
date to convert
original time zone value
conversion time zone
and the function returns the
converted value.
New solution to an old question...
We just had this issue when moving a server from "eastern standard time" to "utc", with all our asp classic apps.
But, we're using SQL server 2017.
And, if you're using SQL server 2016+, you can use the new "at time zone" keyword.
Get your current "offset" from that query:
select datediff(hour, GETUTCDATE() at time zone 'eastern standard time', GETUTCDATE()) as offset
Then, use that offset to handle all your dates in your ASP code, with functions like:
function fromUTC(dt)
fromUTC = dateadd("h", tzOFFSET, dt)
end function
function toUTC(dt)
toUTC = dateadd("h", -1 * tzOFFSET, dt)
end function
function getNow
getNow = fromUTC(now)
end function
You can replace all "now" with "getNow" in your code.
Warning: that this offset is fixed according to the current date. If you need to handle dates from different daylight period, you can use the "at time zone" syntax directly in your SQL query, like:
DECLARE #timezone NVARCHAR(100) = N'eastern standard time'
SELECT
u.name,
u.last_visit at time zone #timezone as last_visit_tz,
u.date_created at time zone #timezone as date_created_tz
FROM users u
You can check e.g. the website http://worldclockapi.com. You can take the current time as timezone.
Example: At http://worldclockapi.com/api/json/est/now.
you can see EST current date, time and other data..
Example: At http://worldclockapi.com/api/json/pst/now you can see PST current date, time and other data...
And you can use XMLHTTP for getting data from external site.
private Function GETHTTP(adres)
Set StrHTTP = Server.CreateObject("Microsoft.XMLHTTP" )
StrHTTP.Open "GET" , adres, false
StrHTTP.sEnd
GETHTTP = StrHTTP.Responsetext
Set StrHTTP = Nothing
End Function
full_data= GETHTTP("http://worldclockapi.com/api/json/est/now")
Afterwards, you use split to to separate by comma:
parts=split(full_data,",")
response.write parts(1)
I think if you choose from below the one you need and then put at the top of your page it will default to that location instead of the server - http://msdn.microsoft.com/en-us/library/ms524330(v=vs.90).aspx
' This file does not need #LCID or #CODEPAGE and
' it does not need to be saved in UTF-8 format because
' there are no literal strings that need formatting or encoding.
Response.Codepage = 65001
Response.Charset = "utf-8"
' See what happens when you uncomment the lines below.
'Response.Codepage = 1252
'Response.Charset = "windows-1252"
ShowDateTimeCurrency 1033, "North America"
ShowDateTimeCurrency 1041, "Japan"
ShowDateTimeCurrency 1049, "Russia"
ShowDateTimeCurrency 1031, "Germany"
ShowDateTimeCurrency 1025, "Saudi Arabia"
ShowDateTimeCurrency 1081, "India"
ShowDateTimeCurrency 2052, "China"
ShowDateTimeCurrency 1042, "Korea"
Sub ShowDateTimeCurrency(iLCID, sLocale)
Response.LCID = iLCID
Response.Write "<B>" & sLocale & "</B><BR>"
Response.Write FormatDateTime(Date, 1) & "<BR>"
Response.Write FormatDateTime(Time, 3) & "<BR>"
Response.Write FormatCurrency(1000) & "<BR>"
Response.Write FormatNumber(50, 3, 0, 0, -1) & " & " & FormatNumber(.02, 3, 0, 0, -1) & "<BR><BR>"
End Sub
Have a look here - http://msdn.microsoft.com/en-us/library/ms524330(v=vs.90).aspx
or another way would be to use date addadd function for time
Please Mark as answer if it helps
thanks
I send a vcal through mail in web application with convert datetime to universal datetime.
If i run web application locally (local sevser in India) i get correct time in my vcal.
But run appication live (server in US) then not get corret time with a difference of 1 and half hour.Please suggest me.
code :
Dim result As StringBuilder = New StringBuilder()
result.AppendFormat("BEGIN:VCALENDAR{0}", System.Environment.NewLine)
result.AppendFormat("BEGIN:VEVENT{0}", System.Environment.NewLine)
result.AppendFormat("SUMMARY:{0}{1}", subject, System.Environment.NewLine)
result.AppendFormat("LOCATION:{0}{1}", location, System.Environment.NewLine)
result.AppendFormat("DTSTART:{0}{1}", startDate.ToUniversalTime().ToString("yyyyMMdd\THHmmss\Z"), System.Environment.NewLine)
result.AppendFormat("DTEND:{0}{1}", endDate.ToUniversalTime().ToString("yyyyMMdd\THHmmss\Z"), System.Environment.NewLine)
result.AppendFormat("DTSTAMP:{0}{1}", DateTime.Now.ToUniversalTime().ToString("yyyyMMdd\THHmmss\Z"), System.Environment.NewLine)
result.AppendFormat("DESCRIPTION:{0}{1}", description, System.Environment.NewLine)
result.AppendFormat("END:VEVENT{0}", System.Environment.NewLine)
result.AppendFormat("END:VCALENDAR{0}", System.Environment.NewLine)
Return result.ToString()
Since IST (India Standard Time) is at UTC+5:30 and the US time zones are at UTC-4 (Eastern) to UTC-7 (Pacific), you could appear to be one and a half hours off (actually, ten and a half hours off but ignoring am/pm) if the server is running in Central Daylight-Saving (CDT) time.
I found vcalendar at www.vcalendar.org which swiftly redirects you. It appears to be a PHP application. And looking at the PHP manual, it appears that the H format is 24-hour time; but it requires a single H, not a double H. The PDF for the documentation at vcalendar.org is AWOL (404 - except it redirects you to a more useful page than simply 404, but it still isn't there!).
Suggestion: establish which time zone your server is running in in the USA (there are 4 main time zones, and sundry auxilliary ones). Check whether the '1.5 hours = 10.5 hours modulo 12' hypothesis is plausible. And look hard at the date/time formats and the meaning of 'HH'.