I'm developing an application for managing delivery in a company using netbeans and php myadmin.
I have to save in the database daily hundreds of deliveries with specific data for each but all with the date of that day, for query later like
select * from table_1 where 'date'='02/10/2016' for example.
I can create a field in the table with type "date" but this date will be redundancy hundreds of times in the table just to specify one day, and the next day also and so on...
What's the best way to stop the redundancy ??
You could use datetime as its type which will reduce your redundancy.
And when you wish to retrive all entries of a particular date, you could try using select * from table_1 where 'date' LIKE '2016-10-02%' in your query
Related
Guys I have a table in database like this.
open to see
I need to fetch data such that i have only variety and number of grafts each date summed up without repetition.
I get this instead: result
I think, this is the query you are looking for
select sum(variety), sum(number_grafts), date
from <table_name>
group by date
I'm working on BigQuery and have created a view using multiple tables. Each day data needs to be synced with multiple platforms. I need to insert a date or some other field via SQL through which I can identify which rows were added into the view each day or which rows got updated so only that data I can take forward each day instead of syncing all every day. Best way I can think is to somehow add the the current date wherever an update to a row happens but that date needs to be constant until a further update happens for that record.
Ex:
Sample data
Say we get the view T1 on 1st September and T2 on 2nd. I need to to only spot ID:2 for 1st September and ID:3,4,5 on 2nd September. Note: no such date column is there.I need help in creating such column or any other approach to verify which rows are getting updated/added daily
You can create a BigQuery schedule queries with frequency as daily (24 hours) using below INSERT statement:
INSERT INTO dataset.T1
SELECT
*
FROM
dataset.T2
WHERE
date > (SELECT MAX(date) FROM dataset.T1);
Your table where the data is getting streamed to (in your case: sample data) needs to be configured as a partitioned table. Therefor you use "Partition by ingestion time" so that you don't need to handle the date yourself.
Configuration in BQ
After you recreated that table append your existing data to that new table with the help of the format options in BQ (append) and RUN.
Then you create a view based on that table with:
SELECT * EXCEPT (rank)
FROM (
SELECT
*,
ROW_NUMBER() OVER (GROUP BY invoice_id ORDER BY _PARTITIONTIME desc) AS rank
FROM `your_dataset.your_sample_data_table`
)
WHERE rank = 1
Always use the view from that on.
I have a table which lists advertising actions by brands. Each action is recorded by one entry as shown in the excel file which you can see in the drive folder below. There are about 7mil records in the table.
I am running several queries which run fine but i am facing an issue with one specific report. This report lists and ranks the clients (Column L in Excel file) by total amounts spent on advertising. This report takes about a minute to generate if the date range set is anywhere from two months to two years.
Below is a copy of the query
select customer_id, sum(value) as value
from `data`
where ((`date` >= '2019-01-01' and `date` <= '2019-12-31'))
group by `customer_id`
order by `value` desc, `customer_id` asc;
This report takes about a minute to generate if the date range set is anywhere from 2 months to two years. If my date range selection is for one month or less, it takes less than 3 seconds.
I need to get this query to process in less than 10-15 seconds max. We tried to come up with ideas such as creating a new table in the DB especially for this query but it hit a wall when we saw that we still had to keep all date records, and we could therefore not group the results in the table.
We really are open to any kind of idea that would make this query faster, including DB changes.
Below is the link to the folder which contains a copy of the DB structure and a sample data set exported from the data table which contains all the data.
Drive Folder
You should make an index on fields that are essential to the criteria of your query. In this case something like:
CREATE INDEX `idx_data_date` ON `data`(`date`);
CREATE INDEX `idx_data_customer_id` ON `data`(`customer_id`);
You should also avoid storing dates as text. Use DATETIME if you can.
We have a large database with monthly partitioned tables. I need to aggregate a selection of these tables every month but I don't want to update the union all every month to add the new monthly table.
CREATE VIEW dynamic_view AS
SELECT timestamp,
traffic
FROM traffic_table_m_2017_01
UNION ALL
SELECT timestamp,
traffic
FROM traffic_table_m_2017_02
Is this where I would use a stored procedure? I am not really familiar with them.
I think it would also work as:
SELECT timestamp,
traffic
FROM REPLACE(REPLACE('traffic_table_m_yyyy_mm',
yyyy, FORMAT(GETDATE(),'yyyy', 'en-us')),
mm, FORMAT(GETDATE(),'mm', 'en-us'));
This might work for the current month but I would need to save the data from the past months which would also be an issue.
you should append each table as it arrives to 1 larger table then run your queries against that. there are many ways to do this but probable the fastest and most elegant is to use.
ALTER TABLE APPEND
Instructions here https://docs.aws.amazon.com/redshift/latest/dg/r_ALTER_TABLE_APPEND.html
I know there are a lot of solutions to this but I am looking for a simple query to get all the dates between two dates.
I cannot declare variables.
As per the comment above, it's just guesswork without your table structures and further detail. Also, are you using a 3NF database or star schema structures, etc. Is this a transaction system or a data warehouse?
As a general answer, I would recommend creating a Calendar table, that way you can create multiple columns for Working Day, Weekend Day, Business Day, etc. and add a date key value, starting at 1 and incrementing each day.
Your query then is a very simple sub-select or join to the table to do something like
SELECT date FROM Calendar WHERE date BETWEEN <x> AND <y>
How to create a Calender table for 100 years in Sql
There are other options like creating the calendar table using iterations (eg, as a CTE table) and linking to that.
SQL - Create a temp table or CTE of first day of the month and month names