I'm using MariaDB and suffering index problem.
here is information about table.
column
type
col1
varchar(10)
col2
timestamp(6)
Query
select col1 from table where col1 is not null and col2>='2022-04-10 00:00:00' and col2<'2022-04-11 00:00:00' group by col1;
i set index option in both of columns.
but, when i see execution plan about my query, 'Using index' is never seen in extra field.
give me a help.
Related
I have a query
Select Column1, column2.... column30
From Table
Where (#var IS NULL OR Col1 = #var)
This can get resolved to a clustered index scan where every row in a table getting checked against this condition #var IS NULL and the index set up on the column Col1 is disregarded. Is there a way to re-write this query to enable the optimizer to use the index on col1? Thank you!
At the job we have an update script for some Oracle 11g database that takes around 20 hours, and some of the most demanding queries are updates where we change some values, something like:
UPDATE table1 SET
column1 = DECODE(table1.column1,null,null,'no info','no info','default value'),
column2 = DECODE(table1.column2,null,null,'no info','no info','another default value'),
column3 = 'default value';
And like this, we have many updates. The problem is that the tables have around 10 millions of rows. We also have some updates where some columns are going to have a default value but they are nullable (I know if they have the not null and a default constrains then the add of such columns is almost immediate because the values are in a catalog), and then the update or add of such columns is costing a lot of time.
My approach is to recreate the table (as TOM said in https://asktom.oracle.com/pls/asktom/f?p=100:11:0::NO::P11_QUESTION_ID:6407993912330 ). But I have no idea on how to retrive some columns from the original table, that are going to remain the same and also other that are going to change to a default value (and before the update such column had a sensible info), this because we need to keep some info private.
So, my approach is something like this:
CREATE TABLE table1_tmp PARALLEL NOLOGGING
AS (select col1,col2,col3,col4 from table1);
ALTER TABLE table1_tmp ADD ( col5 VARCHAR(10) default('some info') NOT NULL;
ALTER TABLE table1_tmp ADD ( col6 VARCHAR(10) default('some info') NOT NULL;
ALTER TABLE table1_tmp ADD ( col7 VARCHAR(10);
ALTER TABLE table1_tmp ADD ( col8 VARCHAR(10);
MERGE INTO table1_tmp tt
USING table1 t
ON (t.col1 = tt.col1)
WHEN MATCHED THEN
UPDATE SET
tt.col7 = 'some defaul value that may be null',
tt.col7 = 'some value that may be null';
I also have tried to create the nullable values as not null to do it fast, and worked, the problem is when I return the columns to null, then that operation takes too much time. The last code ended up consuming also a great amount of time (more tha one hour in the merge).
Hope have an idea on how to improve performance in stuff like this.
Thanks in advance!
Maybe you can try using NVL while joining in merge:
MERGE INTO table1_tmp tt
USING table1 t
ON (nlv(t.col1,'-3') = nvl(tt.col1,'-3'))
WHEN MATCHED THEN ....
If you don't want update null values you can also do like this:
MERGE INTO table1_tmp tt
USING table1 t
ON (nlv(t.col1,'-3') = nvl(tt.col1,'-2'))
WHEN MATCHED THEN .....
At the end, I finished creating a temp table with data from the original table, and while doing the create, inserting the default values and decodes and any other stuff, like if I wanted to set something to NULL, I did the cast. Something like:
CREATE TABLE table1_tmp AS (
column1 default "default message",
column2, --This column with no change at all
column3, --This will take the value from the decode below
) AS SELECT
"default message" column1,
column2 --This column with no change at all,
decode(column3, "Something", NULL, "A", "B") column3,
FROM table1;
That is how I solved the problem. The time for coping a 23 million row's table was about 3 to 5 minutes, while updating used to take hours. Now just need to set privileges, constraints, indexes, comment, and that's it, but that stuff just takes seconds.
Thanks for the answer #thehazal could not check your approach, but sounds interesting.
I have an identity field in one of my tables, specified as
User_Key BIGINT PRIMARY KEY IDENTITY(-1,1) NOT NULL
The seed is set to -1 to account for an unknown value (universal across all my tables) and the table in question is appended to rather than being dropped and recreated each time, but the process is set up to recreate the table if it is accidentally dropped.
Since RedShift doesn't support if statements (ideally, I'd only insert the unknown value if the table didn't already exist), my workaround (using EXCEPT) is shown below.
CREATE TABLE IF NOT EXISTS TBL
(User_Key BIGINT PRIMARY KEY IDENTITY(-1,1) NOT NULL
,Col1 VARCHAR(255) NOT NULL
,Col2 INT
,COL3 VARCHAR(255) NOT NULL
INSERT INTO TBL
(Col1
,Col2
,Col3)
SELECT
'Unknown'
NULL
'Etc'
EXCEPT
SELECT
Col1
,Col2
,Col3
FROM TBL
With the EXCEPT clause, the value in the User_Key field varies (but never is -1 as expected). Without the EXCEPT clause, the User_Key field works exactly as expected without fail. Testing this process has involved dropping and recreating this table with each iteration.
I've done the usual sweep of process documentation/forums/etc., but can't see this being reported elsewhere. Is this a known issue that can be fixed by specifying additional parameters? I have a workaround for this (enclosing the create statement and unknown value prior to the rest of the procedure in my application, but I'd rather keep the script in as few parts as possible.
Are you sure the remaining columns have the correct values? I suspect the error might be along those lines.
You can do this using not exists:
INSERT INTO TBL (Col1, Col2, Col3)
SELECT x.*
FROM (SELECT 'Unknown' as col1, NULL as col2, 'Etc' as col3) x
WHERE NOT EXISTS (SELECT 1 FROM tbl WHERE tbl.user_key = -1)
This question already has answers here:
Is there a quick way to check if ANY column is NULL?
(4 answers)
Closed 7 years ago.
Working with MSSSQL I have a table with more than 20 columns, and I want to retrieve rows which have at least one of the column values in NULL. Which would be the most efficient way, or do you have to write something like this?:
Select * from tableName where col1 is null or col2 is null or col3 is null....
Thank you!
Consider that SQL Server will need to check each column in each row to make the determination (at least until it finds a NULL value in a given column). That means that it's going to be doing a full table scan no matter how you happen to organize your query. So, while there may be "prettier" ways to do your query, when it comes to performance I doubt that you'll find anything that is going to be quicker.
The calculated column that #Denis suggested is an exception, but that's because SQL Server is effectively doing the same work ahead of time (or even a little more work, because now it has to write out to the persisted index). At that point it's a matter of where the performance hit is most important to you - finding these columns, or every time that you write to the table.
The most efficient way is to pre-calculate these values (persisted calculated column) (if this appropriate for your case), then index them and query. Here is the idea behind:
CREATE TABLE #temp
(
col1 int,
col2 int,
col3 int,
col_calculated AS CASE WHEN col1 IS NULL OR col2 IS NULL OR col3 IS NULL THEN 0 ELSE 1 END PERSISTED
);
CREATE INDEX IX_TEMP_COLCALCULATED ON #temp(col_calculated);
INSERT INTO #temp VALUES(1, 2, 3), (5, 6, NULL);
SELECT * FROM #temp WHERE col_calculated = 0;
I have the following table in postgres:
create table1 (col1 character varying, col2 character varying);
My table has the following data:
col1 col2
Questions Tags Users
Value1 Value2 Val
I want find the length of col1 and col2 and when the length of values of column 1 and column2 exceeds 6, I want to truncate it and discard the remaining values. i.e. I want my final table to look like the following:
col1 col2
Questi Tags U
Value1 Value2
Actually the reason why I want to do this is, when I create index on table1 then I am getting the following error:
ERROR: index row size 2744 exceeds maximum 2712 for index "allstrings_string_key"
HINT: Values larger than 1/3 of a buffer page cannot be indexed.
Consider a function index of an MD5 hash of the value, or use full text indexing.
I know I can do this by importing the values to some programming language and then truncating the value. Is there some way by which I may achieve the same using an sql query in postgres.
Couldn't you just update them to contain only strings of length 6 at max?
I am no postrgres pro, so this is probably not the best method, but should do the job anyways:
UPDATE table1 SET col1 = SUBSTRING(col1, 1, 6) WHERE LEN(col1) > 6
UPDATE table1 SET col2 = SUBSTRING(col2, 1, 6) WHERE LEN(col2) > 6
I'd suggest that you actually follow the advice from Postgres, rather than changing your data. Clearly, that column with a 2k character long string shouldn't be indexed -- or not with a btree index anyway.
If the idea behind the index is searching, use full text search instead:
http://www.postgresql.org/docs/current/static/textsearch.html
If the idea behind the need is for sorting, use a functional index instead. For instance:
create index tbl_sort on (substring(col from 1 for 20));
Then, instead of ordering by col, order by substring(col from 1 for 20).
Have you tried changing the type of the column to CHAR instead of VARCHAR?
ALTER TABLE table1
ALTER COLUMN col1 SET DATA TYPE CHAR(6),
ALTER COLUMN col2 SET DATA TYPE CHAR(6)
If you need the column to be variable length, you can specify a limit (note that this is a PostgreSQL extension):
ALTER TABLE table1
ALTER COLUMN col1 SET DATA TYPE CHARACTER VARYING(6),
ALTER COLUMN col2 SET DATA TYPE CHARACTER VARYING(6)