I know there are a lot of solutions to this but I am looking for a simple query to get all the dates between two dates.
I cannot declare variables.
As per the comment above, it's just guesswork without your table structures and further detail. Also, are you using a 3NF database or star schema structures, etc. Is this a transaction system or a data warehouse?
As a general answer, I would recommend creating a Calendar table, that way you can create multiple columns for Working Day, Weekend Day, Business Day, etc. and add a date key value, starting at 1 and incrementing each day.
Your query then is a very simple sub-select or join to the table to do something like
SELECT date FROM Calendar WHERE date BETWEEN <x> AND <y>
How to create a Calender table for 100 years in Sql
There are other options like creating the calendar table using iterations (eg, as a CTE table) and linking to that.
SQL - Create a temp table or CTE of first day of the month and month names
Related
We have a large database with monthly partitioned tables. I need to aggregate a selection of these tables every month but I don't want to update the union all every month to add the new monthly table.
CREATE VIEW dynamic_view AS
SELECT timestamp,
traffic
FROM traffic_table_m_2017_01
UNION ALL
SELECT timestamp,
traffic
FROM traffic_table_m_2017_02
Is this where I would use a stored procedure? I am not really familiar with them.
I think it would also work as:
SELECT timestamp,
traffic
FROM REPLACE(REPLACE('traffic_table_m_yyyy_mm',
yyyy, FORMAT(GETDATE(),'yyyy', 'en-us')),
mm, FORMAT(GETDATE(),'mm', 'en-us'));
This might work for the current month but I would need to save the data from the past months which would also be an issue.
you should append each table as it arrives to 1 larger table then run your queries against that. there are many ways to do this but probable the fastest and most elegant is to use.
ALTER TABLE APPEND
Instructions here https://docs.aws.amazon.com/redshift/latest/dg/r_ALTER_TABLE_APPEND.html
My question is about table partitioning in SQL Server 2008.
I have a program that loads data into a table every 10 mins or so. Approx 40 million rows per day.
The data is bcp'ed into the table and needs to be able to be loaded very quickly.
I would like to partition this table based on the date the data is inserted into the table. Each partition would contain the data loaded in one particular day.
The table should hold the last 50 days of data, so every night I need to drop any partitions older than 50 days.
I would like to have a process that aggregates data loaded into the current partition every hour into some aggregation tables. The summary will only ever run on the latest partition (since all other partitions will already be summarised) so it is important it is partitioned on insert_date.
Generally when querying the data, the insert date is specified (or multiple insert dates). The detailed data is queried by drilling down from the summarised data and as this is summarised based on insert date, the insert date is always specified when querying the detailed data in the partitioned table.
Can I create a default column in the table "Insert_date" that gets a value of Getdate() and then partition on this somehow?
OR
I can create a column in the table "insert_date" and put a hard coded value of today's date.
What would the partition function look like?
Would seperate tables and a partitioned view be better suited?
I have tried both, and even though I think partition tables are cooler. But after trying to teach how to maintain the code afterwards it just wasten't justified. In that scenario we used a hard coded field date field that was in the insert statement.
Now I use different tables ( 31 days / 31 tables ) + aggrigation table and there is an ugly union all query that joins togeather the monthly data.
Advantage. Super timple sql, and simple c# code for bcp and nobody has complained about complexity.
But if you have the infrastructure and a gaggle of .net / sql gurus I would choose the partitioning strategy.
I have these 2 tables and I need to create a relationship between them so that I can import them into SSAS Tabular and run some analysis.
The first table has RollingQuarter(Moving Quarter) data. The second is a basic Date table with Date as PK.
Can anyone suggest ways to create a relationship with these?
Ill be using SQL Server 2012.
I could re-create a new date table also.
I think you may have a rough time finding a relationship with these tables.
Your top data table is derived data. It's an average over three months, reported monthly. The Quantity column applies to that window, not to a particular date like all of the stuff in the second table. So what would any relationship really mean?
If you have the primary data that were used to calculate your moving average, then use those instead. Then you can relate dates between the two tables.
But if your analysis is such that you don't need the primary data for the top table, then just pick the middle of each quarter (March 15th 2001 for the first record) and use that as your independent variable for your time series on the top. Then you can relate them by that.
I'm trying to generate monthly records in one table based on instructions in another table. Software - MS Access 2007, though I'm looking for an SQL solution here. To greatly simplify the matter, let's say the following describes the tables:
TaskManager:
- DayDue
- TaskName
Task:
- DateDue
- TaskName
So what happens is that there may be an entry in TaskManager {15, "Accounts due"}, so this should lead to an "Account due" record in the Task table with the due date being the 15th of each month. I'd want it to create records for the last few months and the next year.
What I'm thinking that I need to do is first create a SELECT query that results in x records for each record in the TaskManager table, with a date for each month. After that, I do an INSERT query which inserts records into the Task table if they do not EXIST in the aforementioned SELECT query.
I think I can manage the INSERT query, though I'm having trouble figuring out how to do the SELECT query. Could someone give me a pointer?
You could use a calendar table.
INSERT INTO Task ( DateDue, TaskName )
SELECT calendar.CalDate, TaskManager.TaskName
FROM calendar, TaskManager
WHERE (((Day([CalDate]))=TaskManager.DayDue)
AND ((calendar.CalDate)<#7/1/2013#));
The calendar table would simply contain all dates and other such relevant fields as work day (yesno). Calendar tables are generally quite useful.
Here is the solution I developed using Remou's Calendar table idea.
First create a Calendar table, which simply contains all dates for a desired range. It's easy to just make the dates in Excel and paste them into the table. This is also a very reliable way of doing it, as Excel handles leap years correctly for the modern range of dates.
After building this table, there are three queries to run. The first is a SELECT, which selects every possible task generated by the TaskManager based on the date and frequency. This query is called TaskManagerQryAllOptions, and has the following code:
SELECT TaskManager.ID, Calendar.CalendarDate
FROM TaskManager INNER JOIN Calendar ON
TaskManager.DateDay = Day(Calendar.CalendarDate)
WHERE (TaskManager.Frequency = "Monthly")
OR (TaskManager.Frequency = "Yearly" AND
TaskManager.DateMonth = Month(Calendar.CalendarDate))
OR (TaskManager.Frequency = "Quarterly" AND
(((Month(Calendar.CalendarDate)- TaskManager.DateMonth) Mod 3) = 0));
The bulk of the above is to cover the different options a quarterly Day and Month pair could cover. The next step is another SELECT query, which selects records from the TaskManagerQryAllOptions in which the date is within the required range. This query is called TaskManagerQrySelect.
SELECT TaskManagerQryAllOptions.ID, TaskManager.TaskName,
TaskManagerQryAllOptions.CalendarDate
FROM TaskManagerQryAllOptions INNER JOIN TaskManager
ON TaskManagerQryAllOptions.ID = TaskManager.ID
WHERE (TaskManagerQryAllOptions.CalendarDate > Date()-60)
AND (TaskManagerQryAllOptions.CalendarDate < Date()+370)
AND (TaskManagerQryAllOptions.CalendarDate >= TaskManager.Start)
AND ((TaskManagerQryAllOptions.CalendarDate <= TaskManager.Finish)
OR (TaskManager.Finish Is Null))
ORDER BY TaskManagerQryAllOptions.CalendarDate;
The final query is an INSERT. As we will be using this query frequently, we don't want it to generate duplicates, so we need to filter out already created records.
INSERT INTO Task ( TaskName, TaskDate )
SELECT TaskManagerQrySelect.TaskName, TaskManagerQrySelect.CalendarDate
FROM TaskManagerQrySelect
WHERE Not Exists(
SELECT *
FROM Task
WHERE Task.TaskName = TaskManagerQrySelect.TaskName
AND Task.TaskDate = TaskManagerQrySelect.CalendarDate);
One limitation of this method is that if the date of repetition (e.g. the 15th of each month) is changed, the future records with the wrong day will remain. A solution to this would be to update all the future records with the adjusted date, then run the insert.
One possibility could be to create a table of Months, and a table of Years (prior year, current, and next one). I could run a SELECT query which takes the Day from the TaskManager table, the Month from the Month table, and the Year from the Year table - I imagine that this could somehow create my desired multiple records for a single TaskManager record. Though I'm not sure what the exact SQL would be.
I am in a mission to devise an application database in MS ACCESS. In my database there are 5 tables:
Master
Cash
Cheque
Detail
Month (displays month in a year)
Here I have made Master as parent record and 3 others Cash, Cheque and Detail are children to Master table.
Here are the fields in master table
Lt no Name Regfee AssessmentYear April May June .......... March
The last 12 fields are months in a financial which takes amount as value.
These values should be populated from cheque/cash table through a query.
cheque
LTno **month** chqueno date bank **amount** are fields.
In this cheque table amount for a particular month is to be populated on master table for the corresponding month. What query do I make.
Expecting your valuable suggestions.
As the database is not normalised, you will have to make a very complicated query to update the table. You have to either make twelve updates that look almost the same, or a huge query that does almost the same thing twelve times.
It would be better to move the month values out of the master table and into a separate table where the month is a field in the table instead of a field name. Then it would be easy to add the records to it:
insert into MasterMonths (LTno, month, value)
select LTno, month, sum(amount)
from cheque
group by LTno, month
I do something similar, except that I include credit card slips and cash purchases as well as checks. It's basic bookkeeping.
For a variety of reasons, I chose to have a table called "almanac" with one row (record) for every date. There are columns (fields) for such things as day of the week, month of the year, and so on. I populate this table with a little code fragment written in VB. Even with ten years worth of dates, that's only about 3,653 rows in the table. I then use simple joins with this table to reduce transaction data to data that's summarized by month. I can summarize in other ways, too.
The query is so easy that I just did it with the graphical query interface. However, my summary would have twelve times as many rows as yours, with only one month in each row.
When I want data laid out in the format of your master table, I use one of two tools: the crosstab query tool in MS Access, or the Pivot Table tool in MS Excel. They are both very powerful, but the pivot tool is more flexible. I had to install an add in called MS query in Excel in order to query database data from Excel. That may be a function of the version I'm using.
This is very different from your framework, and it's your choice whether to use it. It's worked well for me.