I have one small question for partitioning table.
I have created a partition function on Date datatype, most of the table covers under it but some big table have to DataTime data type.
So the question is how can i use both data types in singe partition function and scheme? Or how can I handle this situation?
Appreciated your Help and Support.
Related
We have a data warehouse with tables and views from our ERP-System. The views contain the same information as the tables but the date has a different type.
The views have a date like '01.01.2020'
The tables have a date like a number '19001'.
Now, from a performance/ refresh point of view. What is more performant? Should I use the original table with the date as a number?
We are talking about 10mio rows.
Conversion on the fly of your numeric is easy enough to do
SQL> select to_date('19041','YYDDD') from dual;
TO_DATE('
---------
10-FEB-19
but rest assured that storing dates as anything but dates is generally going to get you into trouble down the track. If you can convert it on load, or even use a virtual column and then gather optimizer stats on that column, you'll be better off then using the number column directly.
Here's some video demonstrations of where things generally go bad once you start drifting away from the "natural" data type for data.
https://youtu.be/cqnuwNc4-1M
I need help to improve a table creation in teradata.
this table has a huge data and we retrieve the data from it heavily
To improve data retrieval you may use the below:
1. partition the table(range,case,date).
2. use compress in columns, so that values occurring again and again will be stored as part of table headers.
3. choose indexes(UPI/PI) wisely.
Lets say I have a table called Test.
Test has a Column called CreatedDate. This is the DateTimeOffset at the time of creation of the row.
Test also has a Column called ExternalDate. This is an externally provided time through an API.
What I require to do is to calculate the difference between CreatedDate and ExternalDate. CreatedDate is DateTimeOffset, but ExternalDate is always provided as DateTime2. The external system providing this time does not provide offset or timezone data.
So we can see here, that the calculation can be off if we are in DST or not by an hour.
We are using SQL2008 unfortunately.
I am thinking of creating a table of DST dates, and performing a join when migrating the dating to figure out this issue as mentioned in another thread here. So historical migration is ok. (Other Thread: Migrating SQL stored DateTime values to DateTimeOffset best practice?)
Question is, I will continue to have the external system send in DateTime2 with no offset value. I am worried if I do the calculation on the fly with a join of this same DST table there may be performance implications? I am not too familiar with SQL performance. What does a SQL expert out there think about this? or is there some other more efficient way to do this on the fly?
Thanks! Much appreciated!
I was searching for best practices to create partitions by date, using amazon-redshift-spectrum, but the examples shows the problem being solved by partitioning the table by one date only. What to do if I have more than one date field?
Eg: Mobile events with user_install_date and event_date
How performative is to partition your s3 like:
installdate=2015-01-01/eventdate=2017-01-01
installdate=2015-01-01/eventdate=2017-01-02
installdate=2015-01-01/eventdate=2017-01-03
Will It kill my select performance ? What is the best strategy in this case?
If you data was partitioned in the above manner, then a query that merely had eventdate in the WHERE clause (without installdate) would be less efficient.
It would still need to look through every installdate directory, but it could skip over eventdate directories that do not match the predicate.
Put the less-used parameter second.
I'm trying to partition my tables in BQ, I've read the documentation and it always points to timePartition. I understand that this may be the default partition, but is it possible to define your table's column/s as the partition?
Any inputs would help. Thanks!
Not as of today. The only available partition type is "DAY"