How to Order By within an Order By in SQL? - sql

I'm trying to create a SQL statement which will recreate the hierarchical container/folder/test structure in SilkCentral Test Manager.
Test containers have no ParentID
Test folders contain a ParentID and IsLeaf = 0
Tests contain a ParentID and IsLeaf = 1
This Query results in all of the test containers, folders, and tests:
SELECT "NodeID", "ParentID", "Name", "IsLeaf", "OrderNumber"
FROM "Silk"."TM_TestPlanNodes" AS TPN
WHERE PROJECTID = 36
ORDER BY "ParentID", "OrderNumber", "IsLeaf"
Here are some of the Results:
NodeID ParentID Name IsLeaf OrderNumber
65408 Installation and Upgrades 0 0
65445 Connectivity 0 1
65448 Focus 0 2
65409 GINA / PLAP 0 3
65446 Graphical User Interface 0 4
71038 Login Properties 0 5
65449 Miscellaneous 0 6
70636 Net Firewall 0 7
70998 Software Updates 0 8
65447 Third-party Services 0 9
70805 SilkTest Automated Tests 0 10
68812 65408 0. Setup 0 0
65454 65408 1. Installations & Upgrades 0 1
65450 65408 Typical/Custom Installation 0 2
I would like this ordering instead:
The ParentID is sorted, but if there exists a Node with the ParentID=thePreviousNode'sID, then that is chosen next. If there are multiple of those nodes, they should be ordered by IsLeaf and then, OrderNumber.
How to accomplish this? I'm very limited in what I can do, because I think very complicated syntax will end up throwing errors in Silk. I was going to try a nested SELECT statement:
SELECT "NodeID", "ParentID", "Name","IsLeaf"
FROM "Silk"."TM_TestPlanNodes"
WHERE PROJECTID = '36'AND ParentID LIKE (
SELECT ParentID
FROM "Silk"."TM_TestPlanNodes"
WHERE NAME = 'Installation and Upgrades')
But this is getting this error: "Could not execute report query: Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression."
This is why I'm fiddling with Order By.

You can use a recursive cte to create a hidden column and orderby that column. The hidden column should be Something like:
WITH cte (NodeID, ParentID, Name, IsLeaf, [Order])
AS
(
SELECT NodeID, ParentID, Name, IsLeaf, cast(NodeID as nvarchar(10))
FROM "Silk"."TM_TestPlanNodes"
WHERE PROJECTID = '36'
UNION ALL
SELECT "NodeID", "ParentID", "Name","IsLeaf", cast(leftNode.ParentID as nvarchar(10)) + cast(leftNode.NodeID as nvarchar(10))
FROM "Silk"."TM_TestPlanNodes" as leftNode
INNER JOIN cte on cte.NodeID = leftNode.ParentID
WHERE leftNode.ParentID = cte.NodeID
)
select "NodeID", "ParentID", "Name","IsLeaf" from cte
order by cast([Order] as nvarchar(50))
This was written in notepad so is possible to have some errors, but the idea is to make an [order] column that for example for 65530 would be 654086554569530 (the parent_parent, the parent and the node)
EDIT:
this only works if the ids are all 5 characters long, but from here you can make the proper tweaks.

Although it might not be a PERFECT fit, it is very close with a nested hierarchical representation of parent-child records in a self-joined list and incorporated proper ordering concerns. You may have to tweak it a bit for your table, but here's a link to a prior solution
To clarify that problem with the menu and the corresponding data.
id | parentid | name
1 | 0 | CatOne
2 | 0 | CatTwo
3 | 0 | CatThree
4 | 1 | SubCatOne
5 | 1 | SubCatOne2
6 | 3 | SubCatThree
Desired output
CatOne 1
--SubCatOne 4
--SubCatOne2 5
CatTwo 2
CatThree 3
--SubCatThree 6
The FIRST case is pre-grouping all the like ID's based on the parent... So, when the parent ID is 0, it IS the top-most level, so we keep it's ID. Then, any children under it, we want their respective PARENT IDs so all of the same are correctly pre-grouped.
The purpose of the SECOND group by is to force the entry that represents the actual TOP LEVEL menu item to the top of the list regardless of the child entries.
Say you have a table where IDs are already established, and you now add a new item into position ID = 7 for "New Top Level" and want to move ID #s 2 and 3 into the new "top-level section. If you just to the query with the first CASE, your records would be simulated returned as
ID Parent Name (natural order from the table)
2 7 CatTwo
3 7 CatThree
7 0 New Category. (we want THIS one in FIRST POSITION of the group)
As you can see, this would be a bad representation of the sub-grouping order. The top-level item actually is in the 3rd position... To bring it to the front, we are now sub-grouping and saying... if the Parent ID of the record = 0, then sort it as if it were a '1' priority. Anything else is considered a '2' priority and would simulate the result like
ID Parent Name SubPrioritySort
7 0 New Category. 1
2 7 CatTwo 2
3 7 CatThree 2
Since you are not actually returning these "CASE" values in your result query, you wouldn't otherwise visually see it... but for grins, add them as columns to your query to see the impact. Hopefully this clarified the answer for you.
In your question, you would obviously be able to add your sort order column to the basis of this query.

Related

Getting parenthood of tree structure

I am trying to retrieve a tree structure i want specific levels of 'parenthood'. my table has level of depth, pathIndex and mapping. my first approach was to make some kinds of substrings to be able to look for the value via the mapping, but I am getting multiple errors on conversion of strings. one thing that might be possible is that if i try and query an item that is not at the lowest level it should return null for the levels it is missing.
In the table if i where to query for the line while asterisks
Id depth pathindex ItemNumber
4CF91F7F-832E-468D-B44A-E14DC66E710A 0 0 0.0
D34784A3-2134-4D09-828E-0EDA0C275C43 1 1 1
38158804-3EBC-4841-B1AF-1B86AD153010 2 1 1.1
8E25D494-322F-45F9-8A91-2A385F561C71 3 1 1.1.1
**64EB6C43-FF9C-0FF9-133F-01F4F21DA14F** 4 1 1.1.1.1
13AFA35C-80F8-405A-8980-33C3F7733EE2 2 2 1.2
3F1332E9-4D42-4BD8-9423-598430E94CB5 3 1 1.2.1
B3CC1306-A122-46F6-8F67-30FBABA3B590 4 1 1.2.1.1
C3F27C8E-F96B-4498-A85F-E4FC8EA90ED7 4 2 1.2.1.2
This is how it should be looking for the information, the static string are the ones i don't know how to generate in order to get nulls when asking for a level that is not that deep.
Select top 1 VehicleGroupId as Region
from GroupHierarchy where GroupHierarchy.numericalmapping = '1'
Select top 1 VehicleGroupId as gz
from GroupHierarchy where GroupHierarchy.numericalmapping = '1.1'
Select top 1 VehicleGroupId as cedis
from GroupHierarchy where GroupHierarchy.numericalmapping = '1.1.1'
Instead of putting decimals between your hierarchy members rather use forwardslashes (1/2/3) and then you can use Microsoft SQL hiearchy data type and functions to easily join and retain the structure:
https://www.sqlshack.com/use-hierarchyid-sql-server/

How to label a big set of “transitive groups” with a constraint?

EDIT after #NealB solution: the #NealB's solution is very very fast comparated with any another one, and dispenses this new question about "add a constraint to improve performance". The #NealB's not need any improve, have O(n) time and is very simple.
The problem of "label transitive groups with SQL" have an elegant solution using recursion and CTE... But this solution consumes an exponential time (!). I need to work with 10000 itens: with 1000 itens need 1 second, with 2000 need 1 day...
Constraint: in my case is possible to break the problem into pieces of ~100 itens or less, but only to select one group of ~10 itens, and discard all the other ~90 labeled itens...
There are a generic algotithm to add and use this kind of "pre-selection", to reduce the quadratic, O(N^2), time? Perhaps, as showed by comments and #wildplasser, a O(N log(N)) time; but I expect, with "pre-selection" to reduce to O(N) time.
(EDIT)
I try to use alternative algorithm, but it need some improvement to use as solution here; or, to really increase performance (to O(N) time), need to use "pre-selection".
The "pre-selection" (constraint) is based on a "super-set grouping"... Stating by the original "How to label 'transitive groups' with SQL?" question t1 table,
table T1
(original T1 augmented by "super-set grouping label" ssg, and more one row)
ID1 | ID2 | ssg
1 | 2 | 1
1 | 5 | 1
4 | 7 | 1
7 | 8 | 1
9 | 1 | 1
10 | 11 | 2
So there are three groups,
g1: {1,2,5,9} because "1 t 2", "1 t 5" and "9 t 1"
g2: {4,7,8} because "4 t 7" and "7 t 8"
g3: {10,11} because "10 t 11"
The super-group is only a auxiliary grouping,
ssg1: {g1,g2}
ssg2: {g3}
If we have M super-group-items and N total T1 items, the average group length will be less tham N/M. We can suppose (for my typical problem) also that ssg maximum length is ~N/M.
So, the "label algorithm" need to run only M times with ~N/M items if it use the ssg constraint.
An SQL only soulution appears to be a bit of a problem here. With the help of some procedural
programming on top of SQL the solution appears to be failry simple and efficient. Here is a brief outline
of a solution as could be implemented using any procedural language invoking SQL.
Declare table R with primary key ID where ID corresponds the same domain as ID1 and ID2 of table T1.
Table R contains one other non-key column, a Label number
Populate table R with the range of values found in T1. Set Label to zero (no label).
Using your example data, the initial setup for R would look like:
Table R
ID Label
== =====
1 0
2 0
4 0
5 0
7 0
8 0
9 0
Using a host language cursor plus an auxiliary counter, read each row from T1. Lookup ID1 and ID2 in R. You will find one of
four cases:
Case 1: ID1.Label == 0 and ID2.Label == 0
In this case neither one of these IDs have been "seen" before: Add 1 to the counter and then update both
rows of R to the value of the counter: update R set R.Label = :counter where R.ID in (:ID1, :ID2)
Case 2: ID1.Label == 0 and ID2.Label <> 0
In this case, ID1 is new but ID2 has already been assigned a label. ID1 needs to be assigned to the
same label as ID2: update R set R.Lablel = :ID2.Label where R.ID = :ID1
Case 3: ID1.Label <> 0 and ID2.Label == 0
In this case, ID2 is new but ID1 has already been assigned a label. ID2 needs to be assigned to the
same label as ID1: update R set R.Lablel = :ID1.Label where R.ID = :ID2
Case 4: ID1.Label <> 0 and ID2.Label <> 0
In this case, the row contains redundant information. Both rows of R should contain the same Label value. If not,
there is some sort of data integrity problem. Ahhhh... not quite see edit...
EDIT I just realized that there are situations where both Label values here could be non-zero and different. If both are non-zero and different then two Label groups need to be merged at this point. All you need to do is choose one Label and update the others to match with something like: update R set R.Label to ID1.Label where R.Label = ID2.Label. Now both groups have been merged with the same Label value.
Upon completion of the cursor, table R will contain Label values needed to update T2.
Table R
ID Label
== =====
1 1
2 1
4 2
5 1
7 2
8 2
9 1
Process table T2
using something along the lines of: set T2.Label to R.Label where T2.ID1 = R.ID. The end result should be:
table T2
ID1 | ID2 | LABEL
1 | 2 | 1
1 | 5 | 1
4 | 7 | 2
7 | 8 | 2
9 | 1 | 1
This process is puerly iterative and should scale to fairly large tables without difficulty.
I suggest you check this and use some
general-purpose language for solving it.
http://en.wikipedia.org/wiki/Disjoint-set_data_structure
Traverse the graph, maybe run DFS or BFS from each node,
then use this disjoint set hint. I think this should work.
The #NealB solution is the faster(!) See an example of PostgreSQL implementation here.
Below an example of another "brute force algorithm", only for curiosity!
As #peter.petrov and #RBarryYoung suggested, some performance problems can be avoided abandoning the CTE recursion... I do some issues at the basic labeler, and, abover I add the constraint for grouping by a super-set label. This new transgroup1_loop() function is working!
PS: this solution still have performance limitations, please post your answer with better, or with some adaptation of this one.
-- DROP table transgroup1;
CREATE TABLE transgroup1 (
id serial NOT NULL PRIMARY KEY,
items integer[], -- two or more items in the transitive relationship
ssg_label varchar(12), -- the super-set gropuping label
dels integer[] DEFAULT array[]::integer[]
);
INSERT INTO transgroup1(items,ssg_label) values
(array[1, 2],'1'),
(array[1, 5],'1'),
(array[4, 7],'1'),
(array[7, 8],'1'),
(array[9, 1],'1'),
(array[10, 11],'2');
-- or SELECT array[id1, id2],ssg_label FROM t1, with 10000 items
them, with these two functions we can solve the problem,
CREATE FUNCTION transgroup1_loop(p_ssg varchar, p_max_i integer DEFAULT 100)
RETURNS integer AS $funcBody$
DECLARE
cp_dels integer[];
i integer;
BEGIN
i:=1;
LOOP
UPDATE transgroup1
SET items = array_uunion(transgroup1.items,t2.items),
dels = transgroup1.dels || t2.id
FROM transgroup1 AS t1, transgroup1 AS t2
WHERE transgroup1.id=t1.id AND t1.ssg_label=$1 AND
t1.id>t2.id AND t1.items && t2.items;
cp_dels := array(
SELECT DISTINCT unnest(dels) FROM transgroup1
); -- ensures all itens to del
RAISE NOTICE '-- bug, repeting dels, item-%; % dels! %', i, array_length(cp_dels,1), array_to_string(cp_dels,';','*');
EXIT WHEN i>p_max_i OR array_length(cp_dels,1)=0;
DELETE FROM transgroup1
WHERE ssg_label=$1 AND id IN (SELECT unnest(cp_dels));
UPDATE transgroup1 SET dels=array[]::integer[];
i:=i+1;
END LOOP;
UPDATE transgroup1 -- only to beautify
SET items = ARRAY(SELECT unnest(items) ORDER BY 1 desc);
RETURN i;
END;
$funcBody$ LANGUAGE plpgsql VOLATILE;
to run and see results, you can use
SELECT transgroup1_loop('1'); -- run with ssg-1 items only
SELECT transgroup1_loop('2'); -- run with ssg-2 items only
-- show all with a sequential group label:
SELECT *, dense_rank() over (ORDER BY id) AS group_label from transgroup1;
results:
id | items | ssg_label | dels | group_label
----+-----------+-----------+------+-------------
4 | {8,7,4} | 1 | {} | 1
5 | {9,5,2,1} | 1 | {} | 2
6 | {11,10} | 2 | {} | 3
PS: the function array_uunion() is the same as original,
CREATE FUNCTION array_uunion(anyarray,anyarray) RETURNS anyarray AS $$
-- ensures distinct items of a concatemation
SELECT ARRAY(SELECT unnest($1) UNION SELECT unnest($2))
$$ LANGUAGE sql immutable;

List category/subcategory tree and display its sub-categories in the same row

I have a hierarchical table of Regions and sub-regions, and I need to list a tree of regions and sub-regions (which is easy), but also, I need a column that displays, for each region, all the ids of it's sub regions.
For example:
id name superiorId
-------------------------------
1 RJ NULL
2 Tijuca 1
3 Leblon 1
4 Gavea 2
5 Humaita 2
6 Barra 4
I need the result to be something like:
id name superiorId sub-regions
-----------------------------------------
1 RJ NULL 2,3,4,5,6
2 Tijuca 1 4,5,6
3 Leblon 1 null
4 Gavea 2 4
5 Humaita 2 null
6 Barra 4 null
I have done that by creating a function that retrieves a STUFF() of a region row,
but when I'm selecting all regions from a country, for example, the query becomes really, really slow, since I execute the function to get the region sons for each region.
Does anybody know how to get that in an optimized way?
The function that "retrieves all the ids as a row" is:
I meant that the function returns all the sub-region's ids as a string, separated by a comma.
The function is:
CREATE FUNCTION getSubRegions (#RegionId int)
RETURNS TABLE
AS
RETURN(
select stuff((SELECT CAST( wine_reg.wine_reg_id as varchar)+','
from (select wine_reg_id
, wine_reg_name
, wine_region_superior
from wine_region as t1
where wine_region_superior = #RegionId
or exists
( select *
from wine_region as t2
where wine_reg_id = t1.wine_region_superior
and (
wine_region_superior = #RegionId
)
) ) wine_reg
ORDER BY wine_reg.wine_reg_name ASC for XML path('')),1,0,'')as Sons)
GO
When we used to make these concatenated lists in the database we took a similar approach to what you are doing at first
then when we looked for speed
we made them into CLR functions
http://msdn.microsoft.com/en-US/library/a8s4s5dz(v=VS.90).aspx
and now our database is only responsible for storing and retrieving data
this sort of thing will be in our data layer in the application

How to fetch categories and sub-categories in a single query in sql? (mysql)

I would like to know if it's possible to extract the categories and sub-categories in a single DB fetch.
My DB table is something similar to that shown below
table
cat_id parent_id
1 0
2 1
3 2
4 3
5 3
6 1
i.e. when the input is 3, then all the rows with parent_id as 3 AND the row 3 itself AND all the parents of row 3 should be fetched.
output
cat_id parent_id
3 2 -> The row 3 itself
4 3 -> Row with parent as 3
5 3 -> Row with parent as 3
2 1 -> 2 is the parent of row 3
1 0 -> 1 is the parent of row 2
Can this be done using stored procedures and loops? If so, will it be a single DB fetch or multiple? Or are there any other better methods?
Thanks!!!
If you asking about "Is there in mysql recursive queries?" answer "NO".
But there is very good approach to handle it.
Create helper table (saying CatHierarchy)
CatHierarchy:
SuperId, ChildId, Distance
------------------------------
1 1 0
1 2 1
2 2 0
This redundant data allows easily in 1 query to select any hierarchy, and in 2 insert support any hierarchy (deletion also performed in 1 query with help of delete cascade integrity).
So what does this mean. You track all path in hierarchy. Each node of Cat must add reference to itself (distance 0), then support duplication by adding redundant data about nodes are linked.
To select category with sub just write:
SELECT c.* from Category c inner join CatHierarchy ch ON ch.ChildId=c.cat_id
WHERE ch.SuperId = :someSpecifiedRootOfCat
someSpecifiedRootOfCat - is parameter to specify root of category
THATS ALL!
Theres a really good article about this on Sitepoint - look especially at Modified Preorder Tree Traversal
It's tricky. I assume you want to display categories, kind of like a folder view? Three fields: MainID, ParentID, Name... Apply to your table, and it should work like a charm. I think it's called a recursive query?
WITH CATEGORYVIEW (catid, parentid, categoryname) AS
(
SELECT catid, ParentID, cast(categoryname as varchar(255))
FROM [CATEGORIES]
WHERE isnull(ParentID,0) = 0
UNION ALL
SELECT C.catid, C.ParentID, cast(CATEGORYVIEW.categoryname+'/'+C.categoryname as varchar(255))
FROM [CATEGORIES] C
JOIN CATEGORYVIEW ON CATEGORYVIEW.catID = C.ParentID
)
SELECT * FROM CATEGORYVIEW ORDER BY CATEGORYNAME

Finding breadcrumbs for nested sets

I'm using nested sets (aka modified preorder tree traversal) to store a list of groups, and I'm trying to find a quick way to generate breadcrumbs (as a string, not a table) for ALL of the groups at once. My data is also stored using the adjacency list model (there are triggers to keep the two in sync).
So for example:
ID Name ParentId Left Right
0 Node A 0 1 12
1 Node B 0 2 5
2 Node C 1 3 4
3 Node D 0 6 11
4 Node E 3 7 8
5 Node F 4 9 9
Which represents the tree:
Node A
Node B
Node C
Node D
Node E
Node F
I would like to be able to have a user-defined function that returns a table:
ID Breadcrumb
0 Node A
1 Node A > Node B
2 Node A > Node B > Node C
3 Node A > Node D
4 Node A > Node D > Node E
5 Node A > Node D > Node F
To make this slightly more complicated (though it's sort of out of the scope of the question), I also have user restrictions that need to be respected. So for example, if I only have access to id=3, when I run the query I should get:
ID Breadcrumb
3 Node D
4 Node D > Node E
5 Node D > Node F
I do have a user-defined function that takes a userid as a parameter, and returns a table with the ids of all groups that are valid, so as long as somewhere in the query
WHERE group.id IN (SELECT id FROM dbo.getUserGroups(#userid))
it will work.
I have an existing scalar function that can do this, but it just does not work on any reasonable number of groups (takes >10 seconds on 2000 groups). It takes a groupid and userid as a parameter, and returns a nvarchar. It finds the given groups parents (1 query to grab the left/right values, another to find the parents), restricts the list to the groups the user has access to (using the same WHERE clause as above, so yet another query), and then uses a cursor to go through each group and append it to a string, before finally returning that value.
I need a method to do this that will run quickly (eg. <= 1s), on the fly.
This is on SQL Server 2005.
here's the SQL that worked for me to get the "breadcrumb" path from any point in the tree. Hope it helps.
SELECT ancestor.id, ancestor.title, ancestor.alias
FROM `categories` child, `categories` ancestor
WHERE child.lft >= ancestor.lft AND child.lft <= ancestor.rgt
AND child.id = MY_CURRENT_ID
ORDER BY ancestor.lft
Kath
Ok. This is for MySQL, not SQL Server 2005. It uses a GROUP_CONCAT with a subquery.
This should return the full breadcrumb as single column.
SELECT
(SELECT GROUP_CONCAT(parent.name SEPARATOR ' > ')
FROM category parent
WHERE node.Left >= parent.Left
AND node.Right <= parent.Right
ORDER BY Left
) as breadcrumb
FROM category node
ORDER BY Left
If you can, use a path (or I think I've heard it referred as a lineage) field like:
ID Name ParentId Left Right Path
0 Node A 0 1 12 0,
1 Node B 0 2 5 0,1,
2 Node C 1 3 4 0,1,2,
3 Node D 0 6 11 0,3,
4 Node E 3 7 8 0,3,4,
5 Node F 4 9 9 0,3,4,
To get just node D and onward (psuedocode):
path = SELECT Path FROM Nodes WHERE ID = 3
SELECT * FROM Nodes WHERE Path LIKE = path + '%'
I modified the Statement of Kathy to get breadcrumbs for every element
SELECT
GROUP_CONCAT(
ancestor.name
ORDER BY ancestor.lft ASC
SEPARATOR ' > '
),
child.*
FROM `categories` child
JOIN `categories` ancestor
ON child.lft >= ancestor.lft
AND child.lft <= ancestor.rgt
GROUP BY child.lft
ORDER BY child.lft
Feel free to add a WHERE condition e.g.
WHERE ancestor.lft BETWEEN 6 AND 11
What I ended up doing is making a large join that simply ties this table to itself, over and over for every level.
First I populate a table #topLevelGroups with just the 1st level groups (if you only have one root you can skip this step), and then #userGroups with the groups that user can see.
SELECT groupid,
(level1
+ CASE WHEN level2 IS NOT NULL THEN ' > ' + level2 ELSE '' END
+ CASE WHEN level3 IS NOT NULL THEN ' > ' + level3 ELSE '' END
)as [breadcrumb]
FROM (
SELECT g3.*
,g1.name as level1
,g2.name as level2
,g3.name as level3
FROM #topLevelGroups g1
INNER JOIN #userGroups g2 ON g2.parentid = g1.groupid and g2.groupid <> g1.groupid
INNER JOIN #userGroups g3 ON g3.parentid = g2.groupid
UNION
SELECT g2.*
,g1.name as level1
,g2.name as level2
,NULL as level3
FROM #topLevelGroups g1
INNER JOIN #userGroups g2 ON g2.parentid = g1.groupid and g2.groupid <> g1.groupid
UNION
SELECT g1.*
,g1.name as level1
,NULL as level2
,NULL as level3
FROM #topLevelGroups g1
) a
ORDER BY [breadcrumb]
This is a pretty big hack, and is obviously limited to a certain number of levels (for my app, there is a reasonable limit I can pick), with the problem that the more levels are supported, it increases the number of joins exponentially, and thus is much slower.
Doing it in code is most certainly easier, but for me that is simply not always an option - there are times when I need this available directly from a SQL query.
I'm accepting this as the answer, since it's what I ended up doing and it may work for other people -- however, if someone can come up with a more efficient method I'll change it to them.
no sql server specific code, but are you simply looking for :
SELECT * FROM table WHERE left < (currentid.left) AND right > (currentid.right)