I need to create a 4th column in either table as a sum of columns 1,2 and 3. How can I do it?.
LOAD column1,
KeyColumn
FROM
[C:\Qlik\file1.xlsx]
(ooxml, embedded labels, table is sheet1);
LOAD column2,
column3,
KeyColumn
FROM
[C:\Qlik\file1.xlsx]
(ooxml, embedded labels, table is sheet2);
I changed your LOAD script into a INLINE version, so it is not depending on
external files and other users can copy and paste it easy.
My LOAD script:
Data:
LOAD * INLINE [
column1, KeyColumn
1, Product_A
2, Product_B
3, Product_C
];
OUTER JOIN
LOAD * INLINE [
column2, column3, KeyColumn
10, 100, Product_A
20, 200, Product_B
30, 300, Product_C
];
I use the outer join to be sure that all values for KeyColumn are in the resulting table.
In the dashboard I load all available columns and get:
Next step is to iterate over the existing values and add the three columns and store them in column4.
LOAD
Num#(column1 ,'##.###') + Num#(column2 ,'##.###') + Num#(column3 ,'##.###') as column4
Resident Data;
Attention. This
column1 + column2 + column3 as column4
will not work!
After adding the new column to the dashboard you get:
I hope thats what you were asking for.
Update after comments.
In your case
Data:
LOAD column1,
KeyColumn
FROM
[C:\Qlik\file1.xlsx]
(ooxml, embedded labels, table is sheet1);
OUTER JOIN
LOAD column2,
column3,
KeyColumn
FROM
[C:\Qlik\file1.xlsx]
(ooxml, embedded labels, table is sheet2);
LOAD
Num#(column1 ,'##.###') + Num#(column2 ,'##.###') + Num#(column3 ,'##.###') as column4
Resident Data;
should work.
Related
I have to run a query with say around 30 columns in where clause. Each column have 1000+ value to compare. I know IN clause is not the best way to do so. Can any one suggest how to run this query without processing error. E.g below
select *
from table
where column1 not in (1,2,3,4......1000+ )
and column2 not in (1,2,3,4......1000+ ) and column3 not in (1,2,3,4......1000+) and so on upto column30.
I am getting error:
SQL Server query processor ran out of internal resources.
I explored other link but did not find solution or suggestion to implement it in a best way.
Create a temp table holding all the possible values:
CREATE TABLE #temp(column1 int, column2 int ....)
INSERT INTO #temp values
(1,1,1...),
(1,2,2...),
.
.
Now, apply SELECT query filter accordingly
Select * from table where column1 not in (SELECT column1 FROM #temp)
and column2 not in (SELECT column2 from #temp)
there is one more approach of using left outer joins
Select * from table as t
LEFT OUTER JOIN #temp as t1
on t.column1 = t1.column1
LEFT OUTER JOIN #temp as t2
on t.column2 = t1.column2
.
.
WHERE t.column1 IS NULL AND t.column2 IS NULL
There are a couple routes you can take based on assumptions.
Assumption 1: 1,2,3,4......1000+ values are same across all columns' NOT IN comparisons
With this assumption, I believe the problem is that SQL Server's resources are exhausted in a query like this:
Note that I am calling your table as MYTABLE.
select *
from MYTABLE -- <---- Remember the name I gave your table
where column1 not in (2, 5, 100, 22, 44, ... thousand other values)
and column2 not in (2, 5, 100, 22, 44, ... thousand other values)
and column3 not in (2, 5, 100, 22, 44, ... thousand other values)
Notice the values we are comparing. All columns are compared to the same set of values.
Solution
Create a table called values_to_compare like so:
create table values_to_compare (comparison_value int primary key);
Just remember that it might be much, much easier to populate values_to_compare table if you just import a text file with values into it rather than typing out a giant insert statement. But, if you choose to write an insert statement, here's how you'd write it. You might have to break your insert statement up in batches of a few hundred entries if SQL Server complains about a large insert statement.
insert into comparison values (2), (5), (100), (22), (44), ... thousand other values;
Then, create an index on MYTABLE like so:
create index on MYTABLE (column1);
Then, write your query in pieces. First, do this:
select *
from MYTABLE
where not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column1);
Hopefully, this will run fast. If that runs fast enough for you, it's time to add 4 more indexes:
create index on MYTABLE (column2);
create index on MYTABLE (column3);
create index on MYTABLE (column4);
create index on MYTABLE (column5);
Then, add 4 more lines so that the query looks like this:
select *
from MYTABLE
where not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column1)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column2)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column3)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column4)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column5);
If this works well, add 25 more indexes on each of the columns in MYTABLE. Then, expand the query above to compare all 30 columns. I believe SQL Server will perform well.
Assumption 2: Values you compare with column1..30 are all different
With this assumption, I believe the problem is that SQL Server's resources are exhausted in a query like this:
select *
from MYTABLE
where column1 not in (2, 5, 100, 22, 44, ... thousand other values)
and column2 not in (1, 225, 5619, 8, 9000, ... thousand other values)
and column3 not in (2024, 5223, 0, 552, 4564, ... thousand other values)
Notice the values we are comparing. All columns are compared to different set of values.
Solution
Create a table called values_to_compare like so:
create table values_to_compare (compare_column as varchar(50), comparison_value int, primary key (compare_column, compare_value));
Just remember that it might be much, much easier to populate values_to_compare table if you just import a text file with values into it rather than typing out a giant insert statement. But, if you choose to write an insert statement, here's how you'd write it. You might have to break your insert statement up in batches of a few hundred entries if SQL Server complains about a large insert statement.
insert into comparison values
('column1', 2), ('column1', 5), ('column1', 100), ('column1', 22), ('column1', 44), ... thousand other values ...
, ('column2', 1), ('column2', 225), ('column2', 5619), ('column2', 8), ('column2', 9000), ... thousand other values ...
, ('column3', 2024), ('column3', 5223), ('column3', 0), ('column3', 552), ('column3', 4564) ... thousand other values ...
;
Then, create an index on MYTABLE like so:
create index on MYTABLE (column1);
Then, write your query in pieces. First, do this:
select *
from MYTABLE
where not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column1 and compare_column = 'column1');
Hopefully, this will run fast. If that runs fast enough for you, it's time to add 4 more indexes:
create index on MYTABLE (column2);
create index on MYTABLE (column3);
create index on MYTABLE (column4);
create index on MYTABLE (column5);
Then, add 4 more lines so that the query looks like this:
select *
from MYTABLE
where not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column1 and compare_column = 'column1')
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column2 and compare_column = 'column2')
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column3 and compare_column = 'column3')
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column4 and compare_column = 'column4')
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column5 and compare_column = 'column5');
If this works well, add 25 more indexes on each of the columns in MYTABLE. Then, expand the query above to compare all 30 columns. I believe SQL Server will perform well.
Give this a shot.
EDIT
Based on the new information that SQL Server is able to do comparisons up to ~20 columns, we can do split operation.
select * into MYTABLE_TEMP from MYTABLE where 1=2;
We now have a temporary table to store data. Then, execute query comparing only 15 columns. Take the output and dump it into MYTABLE_TEMP.
insert into MYTABLE_TEMP
select *
from MYTABLE
where not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column1)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column2)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column3)
...
...
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column15);
Create 15 indexes on MYTABLE_TEMP.
create index on MYTABLE_TEMP (column16);
create index on MYTABLE_TEMP (column17);
...
...
create index on MYTABLE_TEMP (column30);
Then, run a query on MYTABLE_TEMP.
select *
from MYTABLE_TEMP
where not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column16)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column17)
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column18)
...
...
and not exists (select 1 from values_to_compare where comparison_value = MYTABLE.column30);
See if that helps.
WITH group1 AS
(
SELECT
[column1],
[column2]
FROM
table1
),
Group2 AS
(
SELECT
(column3),
COUNT(column3)
FROM
table 2 AS Count
WHERE
(year (date_value) = 2018 and month(Date_vaLue) = 2)
GROUP BY
column2
)
SELECT *
FROM group1
JOIN group2 ON group1. table1 = group2.table2;
I get an error:
No column name was specified for column 2 of 'group2'
As this isn't a column and is just an identifier I am confused why it thinks the code (Group2 AS (Select (column3 ),) is a column.
I am new at sql so this might just be a silly error
Column 1 is a name and column two is a unique key for that name
Column 2 and column 3 contain the same exact data and I am simply trying to show the number of times it occurs in the DB on the column 3 table, including 0, and relate it back to column 1.
Each datapoint in column 3 contains only data from column2.
Thanks in advance!
There are so many errors in that query, I don't know where to start
In a cte each column must have a name. select columnname makes the resulting column named columnname. An aggregation function like count does not set a column name, so your second column in your second cte does not have a name, as the error states. Use
SELECT column, count(othercolumn) AS ctcol ...
You can't add columns you don't use in the grouping to your select list without an aggregation function. Furthermore you can't add a column aggregated and unggregated to the select list. But I suppose, that's only a typo
SELECT column2, COUNT(column3) AS ctcol
FROM tablexy
...
GROUP BY column2
Your cte don't have any columns named table1 or table2, so your join won't work. Use column named from the cte
SELECT * FROM group1 JOIN group2 ON group1.column2 = group2.column2
I think you need to name the column COUNT(column3) , so...
Group2 AS (Select (column3 ),
COUNT (column3) as cntr
From table 2 as Count
Where (year (date_value) = 2018 and month(Date_vaLue) = 2)
Group by column2
)
I have the following case, i have implemented a small query that merge the adjacent polygons of the same "type". But of course I lose all the information, except the geometry and the "type".
The following image sumarize the first step:
But I want to concatenate the information of the two old blue polygon in the new one.
I try to implement a query where differents fields are based on differents group by.
Something that look like:
SELECT ST_DUMP(ST_Union(geom)).geom as geom,string_agg(param1,',') as param1, string_agg(param2,',') as param2, type
FROM t1
GROUP BY type (for the st_union function)
GROUP BY geom (for the string_agg function)
But i'm unable to understand how to manage this part !
A bit late to the party but what you need to do is to first merge the polygons and then find which of the old polygons intersect with the merged ones. You basically need 2 new tables. One table with the new, grouped polygons and one join table to connect the new merged polygons to your original polygons (containing only the grouped_polygon_ids and the original_polygon_ids).
This can be done with something like this:
CREATE TABLE grouped_polygons AS
SELECT uuid_generate_v4() as id,
ST_DUMP(ST_Union(geom)).geom as geom,
string_agg(param1,',') as param1,
string_agg(param2,',') as param2,
type
FROM t1
GROUP BY type (for the st_union function)
GROUP BY geom (for the string_agg function);
CREATE TABLE join_table AS
SELECT t1.id as original_polygon_id,
grouped_polygons.id as grouped_polygon_id
FROM t1
JOIN grouped_polygons
ON st_intersects(t1.geom, grouped_polygons.geom);
Then you can query the results like this:
SELECT gb.id, gp.geom, t1.*
FROM grouped_polygons gp
JOIN join_table jt
ON gp.id = jt.grouped_polygon_id
JOIN t1
ON t1.id = jt.original_polygon_id
ORDER BY gp.id;
This way you can have all the attributes of t1 connected to the new polygons. You can choose which aggregation function to use to group them together if you want.
I tested this script in a simple environment:
select geom, my_type ,
case when the_path is not null then values1[the_path] else values1[1] end as value1,
case when the_path is not null then values2[the_path] else values2[1] end as value2
from (
select
st_asewkt( (st_dump(st_union(geom))).geom ) as geom,
(st_dump(st_union(geom))).path[1] as the_path ,
my_type,
array_agg(value1) as values1,
array_agg(value2) as values2
from t1
group by my_type
) tx
I hope it could help you to figure out this problem.
This is the script for the simple environment:
drop table t1;
create table t1(
value1 text,
value2 text,
my_type text,
geom geometry(Polygon)
);
insert into t1 (value1,value2,my_type,geom) values ('1-one','2-one','red',ST_GeomFromText('POLYGON((0 0, 0 1, 1 1, 1 0, 0 0))'));
insert into t1 (value1,value2,my_type,geom) values ('1-two','2-two','red',ST_GeomFromText('POLYGON((1 0, 1 1, 2 1, 2 0, 1 0))'));
insert into t1 (value1,value2,my_type,geom) values ('1-three','2-three','blue',ST_GeomFromText('POLYGON((4 0, 4 1, 5 1, 5 0, 4 0))'));
insert into t1 (value1,value2,my_type,geom) values ('1-four','2-four','blue',ST_GeomFromText('POLYGON((7 0, 7 1, 8 1, 8 0, 7 0))'));
and the result
This might sound a bit confusing at first. But I have a table with data and I want to create duplicates of all the rows in this table. So, if i do:
SELECT *
FROM the_table
it lists all the data in this table. Now, i want to create a copy of all returned results, except for that I want to change data for one column (the date). This will make the new rows unique.
The reason I want to do this is because I want more data in this table since im building statistics out of it for testing purposes. So, if I have this:
**Column1 Column2 Column3**
abc aaa bbb
abcd aaaa bbbb
abcde aaaaa bbbbb
The table will now contain:
**Column1 Column2 Column3**
abc aaa bbb
abcd aaaa bbbb
abcde aaaaa bbbbb
abc aaa bbb_new
abcd aaaa bbbb_new
abcde aaaaa bbbbb_new
insert into your_table
select col1, col2, concat(col3, '_new') from your_table
Consider making a Cartesian Join on your table. This will give you way more data quickly :)
INSERT INTO TABLEDUPLICATES
SELECT * FROM the_table
SELECT * FROM TABLEDUPLICATES UNION
SELECT * FROM the_table
Assuming there is an identity column (ID) you might generate dates (A_Date) like this:
insert into the_table (Column1, Column2, A_Date)
select Column1, Column2, A_Date + (rand(ID) - 0.5) * 100
from the_table
To duplicate rows (all columns) you simply could use
insert into tblname
select * from tblname
to change one column that can be modified to
insert into tblname
select column1, column2, 'fixedvalueforcolumn3' from tblname
But you need a unique value for column 3, so you have to change 'fixedvalueforcolumn3' to a function that will generate some random (unique) value (date in your case) for column 3
insert into tblname
select column1, column2, generateRandomValue() from tblname
Hope that will help you
Assuming I have a table SomeTable with the following data:
Primary Key Column1 Column2 Column3 Column4 Column5 Num
1 dat1 abc1 dat3 dat4 por7 1
2 dat1 gcd4 dat3 dat4 yrt8 6
3 dat1 iut7 dat3 dat4 asd6 2
4 other1 other2 other3 other4 other5 4
Another table SomeTableGrouped with a "Group Byed" version created using a query like this:
INSERT INTO SomeTableGrouped
SELECT Column1, Column3, Column4, SUM(Num)
FROM SomeTable
GROUP BY Column1, Column3, Column4
Primary Key Column1 Column3 Column4 Num
100 dat1 dat3 dat4 9
200 other1 other3 other4 4
What I'd like to be able to do is, if I have a primary key of SomeTableGrouped, I need to be able to tell which specific rows from SomeTable it came from.
Example:
In a separate table RandomTable, I have data like this:
Primary Key Column1 SomeTableGroupedId
1 dat1 100
2 dat2 100
If I look at the first row, I need to be able to list out row 1 - 3 from SomeTable
How can I do this? I can't change the schema very much (ie. I can only add new columns, remove columns, add a new table) so a dirty solution is perfectly fine with me.
I think this is what you want.
SELECT id
FROM SomeTable
INNER JOIN SomeTableGrouped ON
(SomeTable.Column1 = SomeTableGrouped.Column1) AND
(SomeTable.Column2 = SomeTableGrouped.Column2) AND
(SomeTable.Column3 = SomeTableGrouped.Column3)
WHERE SomeTableGrouped.id = ...
You don't even need to create all those tables, you only need SomeTable. But here we go...
If you want to find the IDs of the records that summed up, just relate them as they were created:
select st.PrimaryKey as STPK, stg.PrimaryKey as STGPK
from SomeTable st
inner join SomeTableGrouped stg
on (st.Column1 = stg.Column1 and
st.Column3 = stg.Column3 and
st.Column5 = stg.Column5)
However, you should not even have created SomeTableGroupedas a table. It could be a view (look here to see how create views in DB2).
That way, you make sure data is always up-to-date and you don't have to worry about back tracking ("what if Num gets updated?").