How can i create a temporal table in Bigquery to store historical data from another table? i didn't find a useful syntax,
for example i have table-1 which contain data of users, in this table i have coulmn that will show 1 if they were active for the last 30 days, but after 30 days i lose this information and i can't say in March we have 30 users who were active, so i need to store this as historical table, will appreciate your help
Related
I was recieving the daily table with below example dates format. Data inside the table loading from last 14 days starts from yesterday's (20221121_20221204) and table creation date was today. Example : dailytable_20221121_20221204_20221205_041703 How can I write query with these date formats for daily newly added table? I want to conctanate these data into my main_table.
Contacting daily a new table to the main table. Schema and the dataset are same.
Probably somebody has created this before, but I need to create a dummy table to be able to union to my other transaction table.
The use case is, I have daily transactional data with Fields like
"customer", "date", "sales"
No I want a dummy data with the same fields populated by zero values at day level,
customer information should read from customer master but the date field should be pre-generated at day level at least for current year and previous 3 years.
The idea is I want to union this dummy table to my original transaction table to be able to produced no null transactions at day level, and those days with no sales should be populated by zero.
This requirement is for BI Analytics projects.
The make it simple I don't want my table to have a missing transactions at day level.
Here is the sample screenshot
I have database with 10 tables. Each table has 70 monthly databases data since 2011 to current and which will increase every month going forward. Each table has Identity ROW_ID with primary key Clustered index. Also there are Non Unique Non Clustered indexes on ID and Date in each table. There are more than 90 million rows in tables.
All I need to create Fact table and then Dimension tables from those 10 tables to create data warehouse cube. I did search so many blogs in Google and could not able to figure out the solution. Can anyone suggest what is the best approach to create Fact and Dimension tables.
Final solution is if once cube build with 70 monthly dtabases data cube needs to refresh every month only to load monthly data. I do not want to make cube reload everything from 2011.
Any suggestions will be helpful.
Thanks.
How can I see the number of new rows added to each of my database's tables in the past day?
Example result:
table_name new_rows
---------- -----------
users 32
questions 150
answers 98
...
I'm not seeing any table that stores this information in PostGRES statistics collector: http://www.postgresql.org/docs/9.1/static/monitoring-stats.html
The only solution I can think of, is create a database table that stores the row_count of each table at midnight each day.
Edit: I need this to work with any table, regardless of whether it has a "created_at" or other timestamp column. Many of the tables I would like to see the growth rate in, do not have timestamps columns & can't have one added.
The easiest way is to add a column in your table that keep a track of the insert/updated date.
Then to retrieve the rows, you can do a simple select for the last day.
From my knowledge, and I've also done a couple research to make sure, there is no intern functionality that allow you to do that without creating a field.
My question is about table partitioning in SQL Server 2008.
I have a program that loads data into a table every 10 mins or so. Approx 40 million rows per day.
The data is bcp'ed into the table and needs to be able to be loaded very quickly.
I would like to partition this table based on the date the data is inserted into the table. Each partition would contain the data loaded in one particular day.
The table should hold the last 50 days of data, so every night I need to drop any partitions older than 50 days.
I would like to have a process that aggregates data loaded into the current partition every hour into some aggregation tables. The summary will only ever run on the latest partition (since all other partitions will already be summarised) so it is important it is partitioned on insert_date.
Generally when querying the data, the insert date is specified (or multiple insert dates). The detailed data is queried by drilling down from the summarised data and as this is summarised based on insert date, the insert date is always specified when querying the detailed data in the partitioned table.
Can I create a default column in the table "Insert_date" that gets a value of Getdate() and then partition on this somehow?
OR
I can create a column in the table "insert_date" and put a hard coded value of today's date.
What would the partition function look like?
Would seperate tables and a partitioned view be better suited?
I have tried both, and even though I think partition tables are cooler. But after trying to teach how to maintain the code afterwards it just wasten't justified. In that scenario we used a hard coded field date field that was in the insert statement.
Now I use different tables ( 31 days / 31 tables ) + aggrigation table and there is an ugly union all query that joins togeather the monthly data.
Advantage. Super timple sql, and simple c# code for bcp and nobody has complained about complexity.
But if you have the infrastructure and a gaggle of .net / sql gurus I would choose the partitioning strategy.