Ok so I am writing a report against a third party database which is in sql server 2005. For the most part its normalized except for one field in one table. They have a table of users (which includes groups.) This table has a UserID field (PK), a IsGroup field (bit) , a members field (text) this members field has a comma separated list of all the members of this group or (if not a group) a comma separated list of the groups this member belongs to.
The question is what is the best way to write a stored procedure that displays what users are in what groups? I have a function that parses out the ids into a table. So the best way I could come up with was to create a cursor that cycles through each group and parse out the userid, write them to a temp table (with the group id) and then select out from the temp table?
UserTable
Example:
ID|IsGroup|Name|Members
1|True|Admin|3
2|True|Power|3,4
3|False|Bob|1,3
4|False|Susan|2
5|True|Normal|6
6|False|Bill|5
I want my query to show:
GroupID|UserID
1|3
2|3
2|4
5|6
Hope that makes sense...
If you have (or could create) a separate table containing the groups you could join it with the users table and match them with the charindex function with comma padding of your data on both sides. I would test the performance of this method with some fairly extreme workloads before deploying. However, it does have the advantage of being self-contained and simple. Note that changing the example to use a cross-join with a where clause produces the exact same execution plan as this one.
Example with data:
SELECT *
FROM (SELECT 1 AS ID,
'1,2,3' AS MEMBERS
UNION
SELECT 2,
'2'
UNION
SELECT 3,
'3,1'
UNION
SELECT 4,
'2,1') USERS
LEFT JOIN (SELECT '1' AS MEMBER
UNION
SELECT '2'
UNION
SELECT '3'
UNION
SELECT '4') GROUPS
ON CHARINDEX(',' + GROUPS.MEMBER + ',',',' + USERS.MEMBERS + ',') > 0
Results:
id members group
1 1,2,3 1
1 1,2,3 2
1 1,2,3 3
2 2 2
3 3,1 1
3 3,1 3
4 2,1 1
4 2,1 2
Your technique will probably be the best method.
Related
I use Oracle 11g and have a massive number of tables representing inheritance, where a base parent table has a primary key NUMBER ID. The subsequent tables inherit from it, representing through the shared primary key NUMBER ID. Let's assume there is a multiple layers of such inheritance.
To have a clear picture, let's work with the following simplified structure and assume the hierarchy is quite complex:
- TABLE FOOD
- TABLE FRUIT
- TABLE CYTRUS
- TABLE ORANGE
- TABLE GREPFRUIT
- TABLE VEGETABLE
- TABLE MEAT
- TABLE BEEF
- TABLE SIRLOIN
- TABLE RIB EYE
- TABLE CHICKEN
This is not taxative, regardless of how dumb the example is, assume such a multi-layered hierarchy using Class Table Inheritance (aka Table Per Type Inheritance).
If you want to insert a record to a table ORANGE having a certain generated ID, there must be inserted records to the parent tables (CYTRUS, FRUIT and FOOD) as well. Assume an ORM engine takes care after this as keeping such consistency would be very complex.
Let's also assume each of the tables in the hierarchy ends with a certain word (let's say FOOD: FRUIT_FOOD, CYTRUS_FOOD etc.) - I didn't include it to the chart above for sake of clarity.
Question: I have found a record in FOOD table with ID = 123 based on certain criteria. Thanks to the hierarchical structure, how do I find what tables contain the record with the very same ID using SQL only? I.e. my goal is to find out what * the lowest type in the hierarchy* the certain ID is related to.
Note: If you have also an answer for a newer version of Oracle, don't hesitate to include it as long as others might find it useful.
Assuming all these tables have a column ID but you may adjust based on the example.
Q1. what tables contain the record with the very same ID using SQL only
You could use a series unions to determine this eg.
SELECT
id,
table_type,
heirarchy_level
FROM (
SELECT ID, 'FOOD', 1 FROM FOOD
UNION ALL
SELECT ID,'FRUIT',2 FROM FRUIT
UNION ALL
SELECT ID,'CYTRUS',3 FROM CYTRUS
UNION ALL
SELECT ID,'ORANGE',4 FROM ORANGE
UNION ALL
SELECT ID,'GREPFRUIT',4 FROM GREPFRUIT
UNION ALL
SELECT ID,'VEGETABLE',2 FROM VEGETABLE
UNION ALL
SELECT ID,'MEAT',2 FROM MEAT
UNION ALL
SELECT ID,'BEEF',3 FROM BEEF
UNION ALL
SELECT ID,'SIRLOIN',4 FROM SIRLOIN
UNION ALL
SELECT ID,'RIBEYE',4 FROM RIBEYE
UNION ALL
SELECT ID,'CHICKEN',3 FROM CHICKEN
) t
WHERE
id = 123
This would return a table with the id=123 but more importantly a table listing all tables where the record was present along with the depth/level in the hierarchy. You could then use MAX or order by to determine the deepest level
Q2. what is the lowest type in the hierarchy the certain ID is related to
This would return only one record with the lowest type
SELECT
id,
table_type,
heirarchy_level
FROM (
SELECT ID, 'FOOD', 1 FROM FOOD
UNION ALL
SELECT ID,'FRUIT',2 FROM FRUIT
UNION ALL
SELECT ID,'CYTRUS',3 FROM CYTRUS
UNION ALL
SELECT ID,'ORANGE',4 FROM ORANGE
UNION ALL
SELECT ID,'GREPFRUIT',4 FROM GREPFRUIT
UNION ALL
SELECT ID,'VEGETABLE',2 FROM VEGETABLE
UNION ALL
SELECT ID,'MEAT',2 FROM MEAT
UNION ALL
SELECT ID,'BEEF',3 FROM BEEF
UNION ALL
SELECT ID,'SIRLOIN',4 FROM SIRLOIN
UNION ALL
SELECT ID,'RIBEYE',4 FROM RIBEYE
UNION ALL
SELECT ID,'CHICKEN',3 FROM CHICKEN
) t
WHERE
id = 123
ORDER BY
heirarchy_level desc
LIMIT 1
I have various SQL queries, which return me unique / distinct value from DB, (or count them),
like:
SELECT buyer as counterparty
FROM public.order
UNION
SELECT seller as counterparty
FROM public.order
or
SELECT COUNT(*)
FROM (
SELECT DISTINCT p
FROM public.order
CROSS JOIN LATERAL (VALUES(seller),(buyer)) AS C(p)
) AS internalQuery
Example structure of my table:
id buyer seller
0 A B
1 B A
2 B D
3 D A
4 A D
Desired result:
3 or A,B,D
I'd like to rewrite them with TypORM query builder, but I can't figure out, how to replace CROSS JOIN LATERAL (VALUES(seller),(buyer)) AS C(p) or UNION in my case. TypeORM is pretty poor with examples and doc coverage in this case.
Does there any option with that?
I have seen various methods like .getCount and .distinct(true) which could help me and easily find the solution for one column.
So I understood, that if I want to find the exact number, instead of doc results, I should use .getCount instead of .getMany
But I can't understand, how to select (and unite) values from multiple columns via typeORM to receive distinct values from multiple columns.
I am working with PostgrSQL, so when I am trying:
const query = repository.createQueryBuilder('order')
.distinctOn(['buyer', 'seller'])
.limit(100)
.getMany()
I receive docs with each distinct value in each field, so instead of 3 I get 6 values (3 distinct by column1, and 3 by column2)
I have an Oracle table that represents parent-child relationships, and I want to improve the performance of a query that searches the hierarchy for an ancestor record. I'm testing with the small data set here, though the real table is much larger:
id name parent_id tagged
== ==== ========= ======
1 One null null
2 Two 1 1
3 Three 2 null
4 Four 3 null
5 Five null null
6 Six 5 1
7 Seven 6 null
8 Eight null null
9 Nine 8 null
parent_id refers back to id in this same table in a foreign key relationship.
I want to write a query that returns each leaf record (those records that have no descendants... id 4 and id 7 in this example) which has an ancestor record that has tagged = 1 (walking back through the parent_id relationship).
So, for the above source data, I want my query to return:
id name tagged_ancestor_id
== ==== ==================
4 Four 2
7 Seven 6
My current query to retrieve these records is:
select * from (
select id,
name,
connect_by_root id tagged_ancestor_id
from mytree
connect by prior id = parent_id
start with tagged is not null
) m1
where not exists (
select * from mytree m2 where m2.parent_id = m1.id
)
This query works fine on this simple little example table, but its performance is terrible on my real table which has about 11,000,000 records. The query takes over a minute to run.
There are indexes on both fields in the connect by clause.
The "tagged" field in the start with clause also has an index on it, and there are about 1,500,000 records in my table with non-null values in this field.
The where clause doesn't seem to be the problem, because when modify it to return a specific name (also indexed) with where name = 'somename' instead of where not exists ..., the query still takes about the same amount of time.
So, what are some strategies I can use to try to make these types queries on this hierarchy run faster?
Here is what I would check first:
Make sure your table has a primary key.
Make sure the statistics on the table are current. Use DBMS_STATS.GATHER_TABLE_STATS to collect the statistics. See this URL: (for ORACLE version 11.1):
http://docs.oracle.com/cd/B28359_01/appdev.111/b28419/d_stats.htm
Even if you have indexes on both fields individually, you still need
an index on the 2 fields combined; Create an index on the ID and PARENT_ID:
CREATE INDEX on TABLE_NAME(ID, PARENT_ID);
See this URL:
Optimizing Oracle CONNECT BY when used with WHERE clause
Make sure the underlying table does not have row chaining or other problems (E.G. corruption).
Make sure the table and all indexes are in the same tablespace.
I'm not sure if this is any faster without the volume of data to test with... but something to consider. I guess I'm hoping by starting with only those that are tagged, and only those that are leafs we are dealing with a smaller volume to process which may result in a performance gain. but the overhead for the string manipulation seems hackish.
with cte(id, name, parent_id, tagged) as (
SELECT 1, 'ONE', null, null from dual union all
SELECT 2, 'TWO', 1, 1 from dual union all
SELECT 3, 'THREE', 2, null from dual union all
SELECT 4, 'FOUR', 3, null from dual union all
select 5, 'FIVE', null, null from dual union all
select 6, 'SIX', 5, 1 from dual union all
select 7, 'SEVEN', 6, null from dual union all
select 8, 'EIGHT', null, null from dual union all
select 9, 'NINE', 8, null from dual),
Leafs(id, name) as (select id, Name
from cte
where connect_by_isleaf = 1
Start with parent_Id is null
connect by nocycle prior id =parent_id),
Tagged as (SELECT id, name, SYS_CONNECT_BY_PATH(ID, '/') Path, substr(SYS_CONNECT_BY_PATH(ID, '/'),2,instr(SYS_CONNECT_BY_PATH(ID, '/'),'/',2)-2) as Leaf
from cte
where tagged=1
start with id in (select id from leafs)
connect by nocycle prior parent_id = id)
select l.*, T.ID as Tagged_ancestor from leafs L
inner join tagged t
on l.id = t.leaf
In essence I created 3 cte's one for the data (Cte) one for the leafs(leafs) and one for the tagged records (tagged)
We traverse the hierarchy twice. Once to get all the leafs, once to get all the tagged. We then parse out the first leaf value from the tagged hierarchy and join it back to leafs to get the leafs related to tagged records.
As to if this is faster than what you're doing... Shrug I didn't want to spend the time testing since I don't have your indexes nor do I have your data volume
I have an organization table with information about the organization and a table which stores the contacts within that organization. I want to create a query which will select an organization and all the phone numbers associated with their contacts in columns.
EX Org | Phone1 | Phone 2 | Phone 3
I have been looking around at pivot table selects and things like that but I have only seen information on how to select it in the same table not multiple tables.
I want to select it all in one row so I can have clean data for a custom excel sheet I'm working on.
I think the most performing way to do an unpivot on a large table (if you database does not support unpivot) is:
select org,
(case when n.n = 1 then phone1
when n.n = 2 then phone2
when n.n = 3 then phone3
end) as phone
from orgs o cross join
(select 1 as n union all select 2 union all select 3
) n;
(Note: in some databases you might need from dual for the n subqueries.)
You can also do this as a union all:
select org, phone1 from orgs union all
select org, phone2 from orgs union all
select org, phone3 from orgs;
The union all approach requires scanning the table three times, whereas the first approach will typically scan the data table only once.
The only way I can think to have it layout in the way you want seems like overkill but here goes some pseudo-code:
You're going to have to write some VBA.
First, write a query that gets every unique combination of phone number and org.
Second, set the active cell where you want to start.
Then iterate through that recordset and for each org and set the cell value to the org name.
Next iterate through and for each phone number with that org move the active cell to the right and make the cell value equal to the phone number.
Repeat until you reach the count of org instances then move back to the first cell and down a row.
Do this for every org not in your first column.
I've got a few queries (20+) which all return the following three columns:
Building | Room | Other
all of which are text fields. I'd like to take all of those queries and combine them; so I'd like to see what the queries return as a whole.
For example, if I had a query SELECT Building, Room, Other FROM tblOne WHERE Room=10 along with SELECT Building, Room, Other FROM tblOne WHERE Building=20, how might I combine those two into one? Obviously this is a very simple example and my real queries are much more complicated, so writing them as 1 query is not feasible.
I'd like the above example to output:
Building | Room | Other
```````````````````````
20 | 1 | Some Stuff
20 | 10 | Some More
5 | 10 | Some Other
15 | 10 | Some Extra
20 | 5 | Some Text
All the ways I've tried have come up with the error that "Building, Room and Other could refer to more than one table" (aka it doesn't want to combine them under one heading). What is the SQL syntax to fix this?
SELECT Building, Room, Other FROM tblOne WHERE Room=10
UNION ALL
SELECT Building, Room, Other FROM tblOne WHERE Building=20
Combine these two Query with the help of UNION ALL && UNION like this
Query 1
SELECT Building, Room, Other FROM tblOne WHERE Room=10
UNION ALL
SELECT Building, Room, Other FROM tblOne WHERE Building=20
Query 2
SELECT Building, Room, Other FROM tblOne WHERE Room=10
UNION
SELECT Building, Room, Other FROM tblOne WHERE Building=20
Notice
The UNION operator is used to combine the result-set of two or more SELECT statements.
Each SELECT statement within the UNION must have the same number of columns. The columns must also have similar data types. Also, the columns in each SELECT statement must be in the same order.
The UNION operator selects only distinct values by default. To allow duplicate values, use UNION ALL.