I have table DB_TBL where historical data are inserted from tool on 5th of every month. As its historical data the number of rows inserted is very large. For example i checked the count for this month and its 1626521.
This table contains 50 columns. But i am interested to check only 5 columns. There is gui from the front end which is using this 5 columns and where i am providing the filter to get the one month data everytime for example for previous month as closure_date between 01.01.2020 and 31.01.2020.
Select Client_key, Portfolia_Name, Closure_date,
Currency_Type, Balance from DB_TBL
where CLOSURE_DATE between
to_date ('01.01.2020','dd.mm.yyyy') and to_date ('31.01.2020','dd.mm.yyyy');
This query is very slow and takes more than 30 minute to run. This problem is very critical as the same issue i faced from the gui. I have provided index to Closure_date column. But still it does not improve performance.
Is there any way i can improve query performance ? May be i can create Virtual based index or create View ?
First, I would write the query using date literals:
Select Client_key, Portfolia_Name, Closure_date, Currency_Type, Balance
from DB_TBL
where CLOSURE_DATE between date '2020-01-01' and date '2020-01-31'
For this query, you want an index on db_tbl(closure_date). If you already have this index, then presumably you have such a large volume of data that it takes a lot of time to return the data. You can change the index to a covering index by including the rest of the columns in the index (after CLOSURE_DATE, which should be first).
That said, if you want all data from January 2020, I would recommend:
where CLOSURE_DATE >= date '2020-01-01' and
CLOSURE_DATE < date '2020-02-01'
This will return all rows, even those that have a time component. This is particularly important in Oracle, because the date data type can have a time component -- even though user interfaces often show only the date.
Related
In my program, I have a data grid view. I make some amounts due for payment today. I made a display of the amounts that are due and have not been paid (late) I want a code that displays the dates less than the current date of the day I tried that following code but it only fetches the lower days and does not look For the month or year if it is greater or not than the current day's date
tbl = db.readData("SELECT * from Payments where date_batch < CONVERT(varchar(50),GetDate(), 103)", "");
DgvSearch.DataSource = tbl;
The problem with the previous code is that it doesn't fetch the date lower by day, month and year.
Fetches the date less than the current date in terms of day only I want in terms of day, month and year
Ok, so I'm going to assume date_batch is a VARCHAR(10) or similar and contains data like:
28/12/2021
29/11/2021
30/08/2021
31/12/2021
As you can see these "strings that look like dates to a human" are in order. They are not in date order, they are in alphabetical order. Big difference - SQLServer sorts strings alphabetically. When you ask for strings "less than x" it uses alphabetical sorting rules to determine "less than"-ness
Don't stores dates in a string. SQLServer has several date specific datatypes. Use them.
The following process will dig you out of the hole you've dug yourself into:
ALTER TABLE Payments ADD COLUMN BatchDate DATE;
UPDATE Payments SET BatchDate = TRY_CONVERT(Date, date_batch, 103);
Now go look at your table and sanity check it:
SELECT * FROM payments WHERE batchdate is null and date_batch is not null
This shows any dates that didn't convert. Correct their wonky bad data and run the update again.
Do another select, of all the data, and eyeball it; does it look sensible? Do you have any dates that have been put in as 02/03/2021 when they should have been 03/02/2021 etc
Now your table is full of nice dates, get rid of the strings;
ALTER TABLE Payments DROP COLUMN date_batch;
Maybe rename the column, but in SQLServer and c# WeCallThingsNamesLikeThis, we_dont_call_them_names_like_this
sp_rename 'Payments.BatchDate', 'date-batch', 'COLUMN';
Now you can do:
SELECT * FROM payments WHERE batchDate < GetDate()
And never again store dates in a string
I'm looking to only select one data point from each date in my report. I want to ensure each day is accounted for and has at least one row of information, as we had to do a few different things to move a large data file into our data warehouse (import one large Google Sheet for some data, use Python for daily pulls of some of the other data - want to make sure no date was left out), and this data goes from now through last summer. I could do a COUNT DISTINCT clause to just make sure the number of days between the first data point and yesterday (the latest data point), but I want to verify each day is accounted for. Should mention I am in BigQuery. Also, an example of the created_at style is: 2021-02-09 17:05:44.583 UTC
This is what I have so far:
SELECT FIRST(created_at)
FROM 'large_table'
ORDER BY created_at
**I know FIRST is probably not the best clause for this case, and it's currently acting to grab the very first data point in created_at, but just as a jumping-off point.
You can use aggregation:
select any_value(lt).*
from large_table lt
group by created_at
order by min(created_at);
Note: This assumes that created_at is a date -- or at least only has one value per date. You might need to convert it to a date:
select any_value(lt).*
from large_table lt
group by date(created_at)
order by min(created_at);
BigQuery equivalent of the query in your question
SELECT created_at
FROM 'large_table'
ORDER BY created_at
LIMIT 1
I have a Calendar table pulled from our mainframe DBs and saved as a local Access table. The table has history back to the 1930s (and I know we use back to the 50s in at least one place), resulting in 31k records. This Calendar table has 3 fields of interest:
Bus_Dt - every day, not just business days. Primary Key
Bus_Day_Ind - indicates if the day was a valid business day for the stock market.
Prir_Bus_Dt - the prior business day. Contains some errors (about 50), all old.
I have written a query to retrieve the first business day on or after the current calendar day, but it runs supremely slowly. (5+ minutes) I have examined the showplan output and see it is being run via an x-join, which between 30k+ record tables gives a solution space (and date comparisons) in the order of nearly 10 million. However, the actual task is not hard, and could be preformed comfortably by excel in minimal time using a simple sort.
My question is thus, is there any way to fix the poor performance of the query, or is this an inherent failing of SQL? (DB2 run on the mainframe also is slow, though not crushingly so. Throwing cycles at the problem and all that.) Secondarily, if I were to trust prir_bus_dt, can I get there better? Or restrict the date range (aka, "cheat"), or any other tricks I didn't think of yet?
SQL:
SELECT TE2Clndr.BUS_DT AS Cal_Dt
, Min(TE2Clndr_1.BUS_DT) AS Next_Bus_Dt
FROM TE2Clndr
, TE2Clndr AS TE2Clndr_1
WHERE TE2Clndr_1.BUS_DAY_IND="Y" AND
TE2Clndr.BUS_DT<=[te2clndr_1].[bus_dt]
GROUP BY TE2Clndr.BUS_DT;
Showplan:
Inputs to Query
Table 'TE2Clndr'
Table 'TE2Clndr'
End inputs to Query
01) Restrict rows of table TE2Clndr
by scanning
testing expression "TE2Clndr_1.BUS_DAY_IND="Y""
store result in temporary table
02) Inner Join table 'TE2Clndr' to result of '01)'
using X-Prod join
then test expression "TE2Clndr.BUS_DT<=[te2clndr_1].[bus_dt]"
03) Group result of '02)'
Again, the question is, can this be made better (faster), or is this already as good as it gets?
I have a new query that is much faster for the same job, but it depends on the prir_bus_dt field (which has some errors). It also isn't great theory since prior business day is not necessarily available on everyone's calendar. So I don't consider this "the" answer, merely an answer.
New query:
SELECT TE2Clndr.BUS_DT as Cal_Dt
, Max(TE2Clndr_1.BUS_DT) AS Next_Bus_Dt
FROM TE2Clndr
INNER JOIN TE2Clndr AS TE2Clndr_1
ON TE2Clndr.PRIR_BUS_DT = TE2Clndr_1.PRIR_BUS_DT
GROUP BY TE2Clndr.BUS_DT;
What about this approach
select min(bus_dt)
from te2Clndr
where bus_dt >= date()
and bus_day_ind = 'Y'
This is my reference for date() representing the current date
I have a date of birth DATE column in a customer table with ~13 million rows. I would like to query this table to find all customers who were born on a certain month and day of that month, but any year.
Can I do this by casting the date into a char and doing a subscript query on the cast, or should I create an aditional char column, update it to hold just the month and day, or create three new integer columns to hold month, day and year, respectively?
This will be a very frequently used query criteria...
EDIT:... and the table has ~13 million rows.
Can you please provide an example of your best solution?
If it will be frequently used, consider a 'functional index'. Searching on that term at the Informix 11.70 InfoCentre produces a number of relevant hits.
You can use:
WHERE MONTH(date_col) = 12 AND DAY(date_col) = 25;
You can also play games such as:
WHERE MONTH(date_col) * 100 + DAY(date_col) = 1225;
This might be more suitable for a functional index, but isn't as clear for everyday use. You could easily write a stored procedure too:
Note that in the absence of a functional index, invoking functions on a column in the criterion means that an index is unlikely to be used.
CREATE FUNCTION mmdd(date_val DATE DEFAULT TODAY) RETURNING SMALLINT AS mmdd;
RETURN MONTH(date_val) * 100 + DAY(date_val);
END FUNCTION;
And use it as:
WHERE mmdd(date_col) = 1225;
Depending on how frequently you do this and how fast it needs to run you might think about splitting the date column into day, month and year columns. This would make search faster but cause all sorts of other problems when you want to retrieve a whole date (and also problems in validating that it is a date) - not a great idea.
Assuming speed isn't a probem I would do something like:
select *
FROM Table
WHERE Month(*DateOfBirthColumn*) = *SomeMonth* AND DAY(*DateOfBirthColumn*) = *SomeDay*
I don't have informix in front of me at the moment but I think the syntax is right.
Could somebody recommend the query to retrieve records up to today or certain dates?
I'm required to produce an Oracle report where user needs to enter a date and records up to that date will be shown.
I tried
select * from the_table where the_date <= sysdate
However it seems to produce an inaccurate result. What is the better query for this. For now I'm just playing around with sysdate. Later I will need to use a certain date keyed in by the user and all the records up to that date needs to be shown.
Any suggestions?
Sometimes you get inaccurate records because of little differences like minutes and seconds when two dates have the same day/month/year. Try the following
select * from the_table where TRUNC(the_date) <= sysdate
The TRUNC removes the minute and the seconds. Sometimes you get inaccurate records without using that