Below is the query i am using :
SELECT T.abcd,
String_agg(T.yyy, ',') AS yyys,
T.bbb
FROM (SELECT s.abcd,
up.yyy,
s.bbb,
s.secondary_id
FROM A s
join B su
ON su.search_term_id = s.id
join lll_yyy up
ON up.lll = su.lll
ORDER BY s.abcd
su.page_no,
su.position) T GROUP BY T.abcd, T.bbb
Basically the order of data produced after my aggregated function is not as expected .
The output should be sorted by abcd and page_no and position .
Expected output:
A | 1,2,3,4 | XX
Actual Output
A |2,4,1,3 | XX
the second column in not sorted based on page_no,position as given in the query .
The abcd column has a wide variety of data with numbers,special chars
etc. example: 0900 dr jne pink, 0900 dr jne pink,098 lakhani shoe, iphone, c??mpu men shoe sport are some sample terms in the abcd column
I tried using collate "C" option
Is there a way to figure out which word is screwing up the sort order
Use the ORDER BY clause in the aggregate expression:
SELECT T.abcd,
String_agg(T.yyy, ',' ORDER BY s.abcd, su.page_no, su.position) AS yyys,
T.bbb
FROM (SELECT s.abcd,
up.yyy,
s.bbb,
s.secondary_id
FROM A s
join B su
ON su.search_term_id = s.id
join lll_yyy up
ON up.lll = su.lll) T GROUP BY T.abcd, T.bbb
The ORDER BY clause in a derived table is ignored.
You are showing an incomplete query. However, it seems to be like this:
SELECT ... FROM (SELECT ... ORDER BY ...)
Your main select doesn't have an ORDER BY clause here. You can hence get the result rows in any order. The ORDER BY clause in the subquery can be ignored by the DBMS, because data in a table (which includes derived tables, i.e. subqueries) is considered an unordered set.
Related
Details:
I've got an order line table(oeel) where I'm left joining a serial number table(icets). There is only one oeel line but there may be 6 or seven icets(serial#) lines returned for the same oeel.line. Instead of the query returning multiple rows of data for the serial number, we'd like a single row where one of the columns has all the serial numbers in a single cell separated by commas.
It would look something like this:
Order number | line number | Product | prod qty | serial1,serial2,serial 3,serial 4 |
You are looking for string aggregation. The standard SQL is something like:
select o.Ordernumber, o.linenumber, o.Product, o.prodqty,
listagg(i.serial, ',') within group (order by serial)
from oeel o left join
icets i
on o.? = i.?
group by o.Ordernumber, o.linenumber, o.Product, o.prodqty ;
Some databases use group_concat() or string_agg() instead of listagg().
Two tables:
Parts Table:
Part_Number Load_Date TQTY
m-123 19940102 32
1234Cf 20010809 3
wf9-2 20160421 14
Locations Table:
PartNo Condition Location QTY
m-123 U A02 2
1234Cf S A02 3
m-123 U B01 1
wf9-2 S A06 7
m-123 S A18 29
wf9-2 U F16 7
Result:
Part_Number Load_Date TQTY U_LOC UQTY S_LOC SQTY
m-123 19940102 32 A02,B01 3 A18 29
1234Cf 20010809 3 A02 3
wf9-2 20160421 14 F16 7 A06 7
I am having trouble finding a solution to this with my current DB2 version. I am not completely sure how to find the version, but it is running on an AS400 system, and it seems the version of DB2, is tied to the OS version. Which the box is using: Operating system: i5/OS Version: V5R4M0
(I tried some commands to get the DB2 version using these suggestions Here but none of them worked, like most stated).
In regards to concatenating multiple rows of column data into one row I have come across many articles stating to use XMLAGG or xmlserialize, Here and, Here but I get an error stating the command is not recognized.
Not sure where to go from here, as there seem to be solutions, but I can't get those already suggested functions to work.
EDIT:
Using the accepted answer and explanation, as well as the example
HERE to get a basic idea of recursion with a simple example, and it was
HERE using the "SELECT rownumber() over(partition by category)" statements that really helped pull it all together. Once I understood that statement of course.
I also learned to make sure the data used in the recursion is as narrowed down as possible and then joined up with extra data later. This makes for exponentially faster results. <-- This seems pretty obvious, but when trying to figure all of this out, it wasn't obvious and my query was pretty slow. Once I understood what was actually happening better it was easier to make adjustments for really fast results.
This is rather complicated, so I will show all my work:
Table definitions
create table parts
(part_number Varchar(64),
load_date Date,
total_qty Dec(5,0));
create table locations
(part_number Varchar(64),
condition Char(1),
location Char(3),
qty Dec(5,0));
insert into parts
values ('m-123', '1994-01-02', 32),
('1234Cf', '2001-08-09', 3),
('wf9-2', '2016-04-21', 14);
insert into locations
values ('m-123', 'U', 'A02', 2),
('1234Cf', 'S', 'A02', 3),
('m-123', 'U', 'B01', 1),
('wf9-2', 'S', 'A06', 7),
('m-123', 'S', 'A18', 29),
('wf9-2', 'U', 'F16', 7);
The query:
with -- CTE's
-- This collects locations into a comma seperated list
tmp (part_number, condition, location, csv, level) as (
select part_number, condition, min(location),
cast(min(location) as varchar(128)), 1
from locations
group by part_number, condition
union all
select a.part_number, a.condition, b.location,
a.csv || ',' || b.location, a.level + 1
from tmp a
join locations b using (part_number, condition)
where a.csv not like '%' || b.location || '%'
and b.location > a.location),
-- This chooses the correct csv list, and adds quantity for the condition
tmp2 (part_number, condition, csv, qty) as (
select t.part_number, t.condition, t.csv,
(select sum(qty) qty
from locations
where part_number = t.part_number
and condition = t.condition)
from tmp t
where level = (select max(level)
from tmp
where part_number = t.part_number
and condition = t.condition))
-- This is the final select that combines the parts file with
-- the second stage CTE and arranges things horizontally by condition
select p.part_number, p.load_date,
(select sum(qty)
from locations
where part_number = p.part_number) as total_qty,
coalesce(u.csv, '') as u_loc,
coalesce(u.qty, 0) as uqty,
coalesce(s.csv, '') as s_loc,
coalesce(s.qty, 0) as sqty
from parts p
left outer join tmp2 u
on u.part_number = p.part_number and u.condition = 'U'
left outer join tmp2 s
on s.part_number = p.part_number and s.condition = 'S'
order by p.load_date;
EDIT I have had to add some extra bits in here to support more than two locations for a part/condition, and I have made the column naming in the CTEs more consistent. Ok, so let me explain this a bit, there are 3 parts to this quety, 2 CTEs and the query, you can see the three parts are separated by comments. The first CTE is a recursive CTE. It's purpose is to produce the comma separated location list. You should be able to run the select by itself to see just what it does. tmp is the table name, part_number, condition, csv, and level are the column names. A recursive CTE needs a SELECT to prime the CTE and a UNION ALL with a SELECT that fills in the next details. In this case the priming SELECT retrieves a part number, a condition, and the first location (alphabetically) for that combination. level is set to 1. If you run just the priming select, you will get:
part_number condition location csv level
----------- --------- -------- --- -----
1234Cf S A01 A02 1
m-123 S A18 A18 1
m-123 U A02 A02 1
wf9-2 U F16 F16 1
wf9-2 S A06 A06 1
Note one line per part/condition. The remainder of the recursive CTE will fill in the remaining locations in csv, but it will actually add additional records so we need to filter the results here and later. So records are processed as they are added. The first rows listed above are joined with the location file
on part_number and condition. Note in the priming select I have a cast of the second min(location) to a varchar(128). This leaves room for the CSV column to expand. Without this, it will still expand, but not enough to hold more than 2 locations.
The second select in the recursive CTE concatenates a comma and the next location to the end of CSV. The specific bit that does this is a.csv || ',' || b.location. It also increments the level column. This helps us keep track of where we are in the query. Eventually, the row with the highest level is the one we want to use. We also have a way to end the recursion, and some filters to reduce the number of rows added to the temporary result set. If we have 2 locations, A02 and B02, left unchecked, we will get the following rows: A02, A02,A02, A02,B02, A02,A02,A02, A02,B02,A02, A02,A02,B02, A02,B02,B02, ... ad infinitum. The anti-duplication filter where a.csv not like '%' || b.location || '%' is sufficient for two locations to end the recursion, and minimize rows, like above, for locations A02 and B02, with the anti-duplication filter, we will get rows A02, and A02,B02. Note that none of the other results from the first example with duplicate locations are returned. Adding a third location C02 will yield, with anti-duplication filter, the following rows: A02, A02,B02, A02,C02, A02,B02,C02, A02,C02,B02. No duplicates here, but we do have redundant rows, and as you add locations, it gets worse. This is where we need a way to detect these redundant rows. Since we are starting with the lowest location number, we can always make sure that locations added to CSV are greater than the previously added location. To do that all we need to do is include a column in the result set that indicates which column was added (we could interrogate CSV, but that is harder). This is why we need the location column in tmp. Then we can write filter b.location > a.location. In the above 3 location example, this filter prevents row A02,C02,B02 leaving just a single row with all three locations. Adding more than three locations to the locations file will cause the number of rows to expand even more in TMP, but for each part and condition, there will only be one row with all locations, and it will contain all locations in ascending order.
The second CTE does two things. First, it filters TMP to drop all but the rows containing all locations for a given part/condition. Second, it accumulates the total quantity for each part/condition.
The bit that performs the filtering is in the where clause:
where level = (select max(level)
from tmp
where part_number = t.part_number
and condition = t.condition))
Pretty straight forward. The bit that accumulates the total quantity for a part/condition is also an easy to understand sub-query:
(select sum(qty) qty
from locations
where part_number = t.part_number
and condition = t.condition)
The final piece of this monster query is the main select. It joins the parts file with the results of the second CTE to form the ultimate result set:
select p.part_number, p.load_date,
(select sum(qty) from locations where part_number = p.part_number) as total_qty,
coalesce(u.csv, '') as u_loc, coalesce(u.qty, 0) as uqty,
coalesce(s.csv, '') as s_loc, coalesce(s.qty, 0) as sqty
from parts p
left outer join tmp2 u
on u.part_number = p.part_number and u.condition = 'U'
left outer join tmp2 s
on s.part_number = p.part_number and s.condition = 'S'
order by p.load_date
Bits of note are the subquery to retrieve the total quantity from the locations table. You could use the tqty field in parts, but that can get out of sync with the actual quantities in the locations table. In addition there are two left outer joins with tmp2, one for condition U, and another for condition S. These construct the horizontal array of Location/Quantity in the result row. The last thing is the coalesce functions. These give null values (when a result from an outer join is missing) a default value.
End of EDIT
The final result is:
part_number load_date tqty u_loc uqty s_loc sqty
----------- ---------- ---- ------- ---- ----- ----
m-123 1994-01-02 32 A02,B01 3 A18 29
1234Cf 2001-08-09 3 0 A02 3
wf9-2 2016-04-21 14 F16 7 A06 7
Note XMLAGG and XMLSERIALIZE became available at DB2 for i v7.1 and LISTAGG became available at DB2 for i v7.2. Most recent version as of 8/9/2017 is v7.3. As you are on v5r4, it is likely you will need not only a software, but also a hardware upgrade to get current.
No idea what the rules are for UQTY, S_LOC, SQTY but here is the column you asked about ---
SELECT
P.Part_Number,
P.Load_Date,
P.TQTY,
LISTAGG(L.Location, ', ') WITHIN GROUP (ORDER BY L.Location) AS U_LOC
FROM "Parts Table" AS P
LEFT JOIN "Locations Table" AS L ON P.Part_Number = L.Part_Number
GROUP BY P.Part_Number, P.Load_Date, P.TQTY
The statement produces the following error.
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression.
I presume I somehow need to concatenate the field names in the subquery?
SELECT (
SELECT COALESCE(Table_Field, Field) AS Fields
FROM API_Objects_Fields
WHERE Field IN (
'fullname'
,'confirmed'
,'primary_email'
,'location_short'
)
)
FROM user_basics U
INNER JOIN Pod_Membership PM ON U.UserID = PM.UserID
WHERE PM.PodID = 164
ORDER BY U.Ctime DESC
The sub query specifies the fields to be returned from the table.
DECLARE #Name VARCHAR(1000)
Select #Name =
COALESCE(#Name,'') +Table_Field + ';'
FROM API_Objects_Fields
WHERE Field IN
( 'fullname' ,'confirmed' ,'primary_email' ,'location_short' )
Select #Name As FieldName
#akfkmupiwu need to do like this for above comment
WITH CTE AS
(SELECT (
SELECT DISTINCT TOP 1 COALESCE(Table_Field, Field)
FROM API_Objects_Fields F
WHERE F.UserID = PM.UserID AND F.Field IN (
'fullname'
,'confirmed'
,'primary_email'
,'location_short'
)
)AS Fields,
ROW_NUMBER()OVER (PARTITION BY Table_Field ORDER BY FIELD)AS RN
FROM user_basics U
INNER JOIN Pod_Membership PM ON U.UserID = PM.UserID
WHERE PM.PodID = 164
ORDER BY U.Ctime DESC
)
Select * from CTE WHERE RN = 1
It is an assumption query basing on your question
What the error is telling you
The problem with your query is exactly what the error says, it brings back more than one result. Since your subquery is in the select portion of the outer query (as opposed to the from or the where), sql is looking for the one value to populate the specific column. Think of it more in terms of filling in an excel spreadsheet. You cannot add two separate values to one cell. Instead, you need the data to go into two separate rows.
On another note, coalesce checks if the first value is null, if it is then it returns the second value. If the first value is not null, that value is returned. It sounds to me that this is not the behavior that you are looking for.
How to fix this
You need to either change your query to pull back different rows for each of the possible values that Fields can be or you need to find a way to specify only one value to return for Fields. Since I am unsure what you are looking for, I am going to demonstrate the first way of solving this.
Data
Your question does not provide any data for API_Objects_Fields, so I am going to make some up. Let's assume the columns in this table are Field_ID, Table_Field, and Field and let's say that your table looks like this:
Field_ID | Table_Field | Field
1 | Alan Turing | fullname
2 | Catherine Zeta Jones | fullname
3 | True | confirmed
4 | MN | location_short
5 | 123-456-7890 | phone_number
As I mentioned before, right now your query would try to pull back the rows where the field is fullname, confirmed, or location_short all. Instead of trying to stuff one column of one row. full of 4 results, let's change your query to bring back 4 rows
The Query
SELECT f.Table_Field, Field
FROM user_basics U
INNER JOIN Pod_Membership PM ON U.UserID = PM.UserID
INNER JOIN (
SELECT Table_Field, Field
FROM API_Objects_Fields
WHERE Field IN (
'fullname'
,'confirmed'
,'primary_email'
,'location_short'
)
) f
WHERE PM.PodID = 164
ORDER BY U.Ctime DESC
What will happen
This query will now pull back data that looks more like this:
Table_Field | Fields
Alan Turing | fullname
Catherine Zeta Jones | fullname
True | confirmed
MN | location_short
However, I think you will be surprised with the results you actually end up with. Since the query does not connect the data in API_Objects_Fields with any other tables, you would get the values from the results table above over and over again. In fact, you would get the values above for every single row returned by
Select *
From user_basics u
INNER JOIN Pod_Membership PM ON U.UserID = PM.UserID
WHERE PM.PodID = 164
If this query returns 12 results, you would end up with 12 Alan Turings, 12 Catherine Zeta Jones, 12 Trues, and 12 MNs. If this is not the result you are looking for, you will need to add an ON portion to the inner join so the results from f are connected with the other tables.
I have a table that looks like this:
Column A | Column B | Counter
---------------------------------------------
A | B | 53
B | C | 23
A | D | 11
C | B | 22
I need to remove the last row because it's cyclic to the second row. Can't seem to figure out how to do it.
EDIT
There is an indexed date field. This is for Sankey diagram. The data in the sample table is actually the result of a query. The underlying table has:
date | source node | target node | path count
The query to build the table is:
SELECT source_node, target_node, COUNT(1)
FROM sankey_table
WHERE TO_CHAR(data_date, 'yyyy-mm-dd')='2013-08-19'
GROUP BY source_node, target_node
In the sample, the last row C to B is going backwards and I need to ignore it or the Sankey won't display. I need to only show forward path.
Removing all edges from your graph where the tuple (source_node, target_node) is not ordered alphabetically and the symmetric row exists should give you what you want:
DELETE
FROM sankey_table t1
WHERE source_node > target_node
AND EXISTS (
SELECT NULL from sankey_table t2
WHERE t2.source_node = t1.target_node
AND t2.target_node = t1.source_node)
If you don't want to DELETE them, just use this WHERE clause in your query for generating the input for the diagram.
If you can adjust how your table is populated, you can change the query you're using to only retrieve the values for the first direction (for that date) in the first place, with a little bit an analytic manipulation:
SELECT source_node, target_node, counter FROM (
SELECT source_node,
target_node,
COUNT(*) OVER (PARTITION BY source_node, target_node) AS counter,
RANK () OVER (PARTITION BY GREATEST(source_node, target_node),
LEAST(source_node, target_node), TRUNC(data_date)
ORDER BY data_date) AS rnk
FROM sankey_table
WHERE TO_CHAR(data_date, 'yyyy-mm-dd')='2013-08-19'
)
WHERE rnk = 1;
The inner query gets the same data you collect now but adds a ranking column, which will be 1 for the first row for any source/target pair in any order for a given day. The outer query then just ignores everything else.
This might be a candidate for a materialised view if you're truncating and repopulating it daily.
If you can't change your intermediate table but can still see the underlying table you could join back to it using the same kind of idea; assuming the table you're querying from is called sankey_agg_table:
SELECT sat.source_node, sat.target_node, sat.counter
FROM sankey_agg_table sat
JOIN (SELECT source_node, target_node,
RANK () OVER (PARTITION BY GREATEST(source_node, target_node),
LEAST(source_node, target_node), TRUNC(data_date)
ORDER BY data_date) AS rnk
FROM sankey_table) st
ON st.source_node = sat.source_node
AND st.target_node = sat.target_node
AND st.rnk = 1;
SQL Fiddle demos.
DELETE FROM yourTable
where [Column A]='C'
given that these are all your rows
EDIT
I would recommend that you clean up your source data if you can, i.e. delete the rows that you call backwards, if those rows are incorrect as you state in your comments.
Hi I know how to use the group by clause for sql. I am not sure how to explain this so Ill draw some charts. Here is my original data:
Name Location
----------------------
user1 1
user1 9
user1 3
user2 1
user2 10
user3 97
Here is the output I need
Name Location
----------------------
user1 1
9
3
user2 1
10
user3 97
Is this even possible?
The normal method for this is to handle it in the presentation layer, not the database layer.
Reasons:
The Name field is a property of that data row
If you leave the Name out, how do you know what Location goes with which name?
You are implicitly relying on the order of the data, which in SQL is a very bad practice (since there is no inherent ordering to the returned data)
Any solution will need to involve a cursor or a loop, which is not what SQL is optimized for - it likes working in SETS not on individual rows
Hope this helps
SELECT A.FINAL_NAME, A.LOCATION
FROM (SELECT DISTINCT DECODE((LAG(YT.NAME, 1) OVER(ORDER BY YT.NAME)),
YT.NAME,
NULL,
YT.NAME) AS FINAL_NAME,
YT.NAME,
YT.LOCATION
FROM YOUR_TABLE_7 YT) A
As Jirka correctly pointed out, I was using the Outer select, distinct and raw Name unnecessarily. My mistake was that as I used DISTINCT , I got the resulted sorted like
1 1
2 user2 1
3 user3 97
4 user1 1
5 3
6 9
7 10
I wanted to avoid output like this.
Hence I added the raw id and outer select
However , removing the DISTINCT solves the problem.
Hence only this much is enough
SELECT DECODE((LAG(YT.NAME, 1) OVER(ORDER BY YT.NAME)),
YT.NAME,
NULL,
YT.NAME) AS FINAL_NAME,
YT.LOCATION
FROM SO_BUFFER_TABLE_7 YT
Thanks Jirka
If you're using straight SQL*Plus to make your report (don't laugh, you can do some pretty cool stuff with it), you can do this with the BREAK command:
SQL> break on name
SQL> WITH q AS (
SELECT 'user1' NAME, 1 LOCATION FROM dual
UNION ALL
SELECT 'user1', 9 FROM dual
UNION ALL
SELECT 'user1', 3 FROM dual
UNION ALL
SELECT 'user2', 1 FROM dual
UNION ALL
SELECT 'user2', 10 FROM dual
UNION ALL
SELECT 'user3', 97 FROM dual
)
SELECT NAME,LOCATION
FROM q
ORDER BY name;
NAME LOCATION
----- ----------
user1 1
9
3
user2 1
10
user3 97
6 rows selected.
SQL>
I cannot but agree with the other commenters that this kind of problem does not look like it should ever be solved using SQL, but let us face it anyway.
SELECT
CASE main.name WHERE preceding_id IS NULL THEN main.name ELSE null END,
main.location
FROM mytable main LEFT JOIN mytable preceding
ON main.name = preceding.name AND MIN(preceding.id) < main.id
GROUP BY main.id, main.name, main.location, preceding.name
ORDER BY main.id
The GROUP BY clause is not responsible for the grouping job, at least not directly. In the first approximation, an outer join to the same table (LEFT JOIN below) can be used to determine on which row a particular value occurs for the first time. This is what we are after. This assumes that there are some unique id values that make it possible to arbitrarily order all the records. (The ORDER BY clause does NOT do this; it orders the output, not the input of the whole computation, but it is still necessary to make sure that the output is presented correctly, because the remaining SQL does not imply any particular order of processing.)
As you can see, there is still a GROUP BY clause in the SQL, but with a perhaps unexpected purpose. Its job is to "undo" a side effect of the LEFT JOIN, which is duplication of all main records that have many "preceding" ( = successfully joined) records.
This is quite normal with GROUP BY. The typical effect of a GROUP BY clause is a reduction of the number of records; and impossibility to query or test columns NOT listed in the GROUP BY clause, except through aggregate functions like COUNT, MIN, MAX, or SUM. This is because these columns really represent "groups of values" due to the GROUP BY, not just specific values.
If you are using SQL*Plus, use the BREAK function. In this case, break on NAME.
If you are using another reporting tool, you may be able to compare the "name" field to the previous record and suppress printing when they are equal.
If you use GROUP BY, output rows are sorted according to the GROUP BY columns as if you had an ORDER BY for the same columns. To avoid the overhead of sorting that GROUP BY produces, add ORDER BY NULL:
SELECT a, COUNT(b) FROM test_table GROUP BY a ORDER BY NULL;
Relying on implicit GROUP BY sorting in MySQL 5.6 is deprecated. To achieve a specific sort order of grouped results, it is preferable to use an explicit ORDER BY clause. GROUP BY sorting is a MySQL extension that may change in a future release; for example, to make it possible for the optimizer to order groupings in whatever manner it deems most efficient and to avoid the sorting overhead.
For full information - http://academy.comingweek.com/sql-groupby-clause/
SQL GROUP BY STATEMENT
SQL GROUP BY clause is used in collaboration with the SELECT statement to arrange identical data into groups.
Syntax:
1. SELECT column_nm, aggregate_function(column_nm) FROM table_nm WHERE column_nm operator value GROUP BY column_nm;
Example :
To understand the GROUP BY clauserefer the sample database.Below table showing fields from “order” table:
1. |EMPORD_ID|employee1ID|customerID|shippers_ID|
Below table showing fields from “shipper” table:
1. | shippers_ID| shippers_Name |
Below table showing fields from “table_emp1” table:
1. | employee1ID| first1_nm | last1_nm |
Example :
To find the number of orders sent by each shipper.
1. SELECT shipper.shippers_Name, COUNT (orders.EMPORD_ID) AS No_of_orders FROM orders LEFT JOIN shipper ON orders.shippers_ID = shipper.shippers_ID GROUP BY shippers_Name;
1. | shippers_Name | No_of_orders |
Example :
To use GROUP BY statement on more than one column.
1. SELECT shipper.shippers_Name, table_emp1.last1_nm, COUNT (orders.EMPORD_ID) AS No_of_orders FROM ((orders INNER JOIN shipper ON orders.shippers_ID=shipper.shippers_ID) INNER JOIN table_emp1 ON orders.employee1ID = table_emp1.employee1ID)
2. GROUP BY shippers_Name,last1_nm;
| shippers_Name | last1_nm |No_of_orders |
for more clarification refer my link
http://academy.comingweek.com/sql-groupby-clause/