I'm using Firebase to register some events from an iOS/Android app and log them into BigQuery. As I understood from the documentation, BigQuery creates a different table each day in order to store the events of the single day.
Each day, Firebase Analytics creates a new table in the BigQuery dataset corresponding to the app. The tables are named using the pattern app_events_YYYYMMDD and contain the events recorded for the specified day.
However I'm getting some events in a certain day registered in the table of the following day. For example the table app_events_20160727 contains some events from July 26th, the table app_events_20160728 contains some events from July 27th.
Am I missing something?
Thanks for your support
Sep, 14 Update
I'll try to better explain the issue through an example: the events recorded in the first part of the day (let's say until 3PM/4PM but I don't see any pattern) are collected in the table of that day, the events of the last part of the day are collected in the table of the following day.
So, let's take the events of Sep, 12: here below the screenshot of the first and last entries of the tables related to Sep 12 and Sep 13
First entries of Sep, 13
Last entries of Sep, 13
First entries of Sep, 12
Last entries of Sep, 12
As you can see, the events from Sep, 12 are split into two tables.
Thanks for your support.
Firebase register the timestamp of when the event was track client side.
This is likely to happen in that scenario:
You trigger an event while offline, day N
Your user reconnect to internet only the following day, day N+1, (or the day after)
Thus Firebase base receive the event of day N, on day N+1.
During day N, firebase will export all the event he received (erver side) on day N. on day N+1 he'll export all the event he received on day N+1, even the one actually track client side on day N, but not sent to server on day N.
I'm unsure the explanation is clear, can you tell if it was clear ?
Related
I’m a data analyst in the insurance industry and we currently have a program in SAS EG that tracks catastrophe development week by week since the start of the event for all of the catastrophic events that are reported.
(I.E week 1 is catastrophe start date + 7 days, week 2 would be end of week 1 + 7 days and so on) then all transaction amounts (dollars) for the specific catastrophes would be grouped into the respective weeks based on the date each transaction was made.
Problem that we’re faced with is we are moving away from SAS EG to GCP big query and the current process of calculating those weeks is a manually read in list which isn’t very efficient and not easily translated to BigQuery.
Curious if anybody has an idea that would allow me to calculate each week number in periods of 7 days since the start of an event in SQL or has an idea specific for BigQuery? There would be different start dates for each event.
It is complex, I know and I’m willing to give more explanation as needed. Open to any ideas for this as I haven’t been able to find anything.
I have a report that, according to users, started miscalculating dates in one field in November 2015. After some digging around, I found that one of the tables the field referenced seemed to have an end date on 2015-10-31.
The "D" field seems to represent the day of the week, with Sunday being day 1 and Saturday being 7.
Is there a way to extend the calendar so that it ends further into the future, for example 2049-12-31?
Our calendar table, for a variety of reasons, goes the the end of the current year. We have written a query that adds a new year to this table. This query takes care of most of the fields in that table. It does not touch the holiday field. That is updated manually through a web page.
We send ourselves reminders. Starting in March, we send monthly reminders that we should think about adding another year. After ensuring that the database segment has space, and that none of the definitions, such as fiscal periods, have changed, we run the query that adds a year.
Later in the year we start mailing ourselves reminders about the holidays. Then we check to see if HR has declared them, and if so, update the records accordingly.
This meets our business requirements. Yours will be different of course.
I'm trying to create a search with Splunk that will allow me to have only the results during non working ours. I mean, Splunk to filter out from the logs all the events that occur from 8am to 5am.
Currently, the query I'm using is: earliest=-1mon so I get all the events from last month, but I only need those events that occurred outside working hours.
Is it possible?
There are probably multiple ways of doing this in Splunk. Below is one. I am extracting the hour field (24 hour format) into c_time and then limiting my results to ones that are between 8p and 5a. You can specify other filters like earliest and latest to be more specific. Hope this helps.
... | convert timeformat="%H" ctime(_time) AS c_time | search c_time>= 20 c_time<= 05
-Neeraj.
I've started thinking about an employee shift management application to handle the shifts (who works when, trading, etc) at my current workplace (that uses pen and paper and hasn't got anyway for us employees to communicate about changes without going through the boss and be on site).
Currently the shifts are modeled loosely as:
There is a recurring 4 week period (from Monday week 1 to Sunday week 4)
There is a template for placing employees in this 4 week period
Every 4 months (ie 3 times a year) the 4 week template is projected over the next 4 month period
The shifts have been the same for a long time and it seems many employees would prefer to have them changed (I can say this by the requests for change that come in every time a new 4 month is set).
What I'm aiming at are the models:
Shift_group_tpl (the 4 week period above)
Shift_tpl (a single shift in the 4 week period, including info on who defaults to work this shift)
Shift_group (a set period of time whit actual shifts)
Shift (a set shift whit a real time period and an employee - and the possibility to be changed both in start_time, end_time and employee)
I've thought of a way to do this with recurring iCalendar events: Creating RRULE's (without an endtime) and then calculate (using temporary start and end times) if that specific Shift_group_tpl could be used within a real Shift_group. (The problem with this approach is that I can't figure out how to trim the Shift_group_tpl's to fit into the start or end of a Shift_group.)
What I'm looking for are some other perspectives or ways of doing it or even just a pat on the shoulder letting me know that I'm on the right track (and then giving advice on the trimming problem).
/iole1
What I'm aiming at are the models:
Shift_group_tpl (the 4 week period above)
Shift_tpl (a single shift in the 4 week period, including info on who defaults to work this shift)
Shift_group (a set period of time whit actual shifts)
Shift (a set shift whit a real time period and an employee - and the possibility to be changed both in start_time, end_time and employee)
You have "sql" as a tag for this post? So im guessing you want these as SQL tables?
By the sounds, the problem is that your considering the data you have, rather than the abstract concepts you need to store that data. Which is what you'd need to do to create an application. (Most likely a "Shifts" table, rather than the four tables above).
There is little information here to help, Consider refining your thoughts and ask another question.
I noticed that when I search future events via event_search method, sorted by date, that first events in result-set have start date from past. Problem with these events is that they are repeatable events (e.g. weekly), but API does not return events' recurrence type (daily, weekly, monthly, other). Method should at least return next event start date and recurrance type.
Hidden feature:
Add "display=repeat_schedule" to your API requests to reveal a series of start_date, end_date pairs for each repeating instance.
Unfortunately, Eventbrite's "repeating events" feature (available on their website) is not totally compatible with their API services:
Most storage models have an associated identifier or 'handle' to reference each asset or resource. In REST-speak these identifiers are known as 'resource ids'.
Eventbrite's repeating events share a single event id, and they include a string that describes the schedule when they repeat.
The head of the chain of repeating events is returned, with a "repeats" attribute that is set to "yes". And, the "repeat_schedule" attribute should now be present on all repeating events.
The string that is revealed will have a different format depending how the event is configured to repeat:
Daily repeating events:
"daily-4-07/26/2012" - every fourth day, from the date/time of the
start_date until 2012-07-26.
"daily-mf-07/26/2012" - every Monday
through Friday, from the start_date day/time until "2012-07-26"
Weekly repeating events:
"weekly-3-Y,N,N,N,N,N,N-09/16/2012" - every 3rd week, on each day marked with a "Y" until "2012-09-16"
Monthly repeating events:
"monthly-2-10-06/30/2012" - every second month, on the 10th day of the month until "2012-06-30"
"monthly-2-second/sat-06/30/2012" - every second month, on the second Saturday of that month, until "2012-06-30".
Custom repeat schedule (not fully supported):
"custom-3199915" - This format is really rare. Unfortunately, these events can not be easily summarized using a single string.
This bug needs to be fixed by EventBrite
However Would this work has a temp workaround? I would work in some examples I have seen.
I make request for events on March 17, 2012
I get back a past date
<repeats>yes</repeats>
<start_date>2011-12-04 09:30:00</start_date>
<end_date>2011-12-04 10:30:00</end_date>
Can I assume this is a repeating event that occcurs on March 17, 2012
at the same times?
There's a new endpoint that can be used to fetch all the events that are part of a series, the only values required are: OAuth Token and the event_series_id which is the parent event ID or the ID of the event where the series was created.
This is the endpoint:
GET - https://www.eventbriteapi.com/v3/series/{event_series_id}/events/
Replace {event_series_id} with your event series ID.
More details at: https://www.eventbrite.com/platform/api#/reference/event/list/list-events-by-series