I am using Gephi to create a network graph, here is a small subset of the data that I have:
ID Label
1 Sleep quality
2 Stress
3 Healthy Eating
4 Tremor
5 Balance
6 Drooling
7 Exercise
8 Mood
9 Speech
10 Parkinson's On-Off
So I want my graph to have these 10 nodes.
Then for the edges, I have:
Source Target User
1 5 5346
5 4 5346
4 7 5346
7 6 5346
6 9 5346
9 3 5346
3 2 5346
2 8 5346
8 10 5346
The "User" column is something I have added to explain the problem I am having. I am using a big database (in SQL) to obtain this data. On a mobile phone app, users select 10 of the different choices available (as listed in the nodes). In SQL I can query the data easily so that I can obtain the 10 choices of each of the users.
It is easy to create a graph with the edges with the information in the edges table but I would also like to connect each edge to all other edges, this is important for me. So for example, 1 connects to all those in "target". Then 5 connects to all those in "target" and so until all nodes are connected to each other for each user.
I can do this manually but the original data set has 2000+ users and this will take a long time. I know that there is a way of using cross join, possibly in Excel or in SQL... but I'm unsure how to do this..
Thanks!
You can drop this cross join into your SQL: (It'll list all Source's with all possible Target's.)
(SELECT e.Source as Source, n.ID as Target
FROM
(SELECT DISTINCT Source FROM tblEdges) as e
cross join (SELECT DISTINCT ID FROM tblNodes) as n
) as xCross
Related
I am fairly new to writing queries in Snowflake and have run into a hiccup. I am trying to count how many times an item appears in a list all in the same column.
I was able to use the flatten function and then tried to add in the count function with no luck.
Here is a dummy version of my data:
Ticket# Tasks
1 ["cut apple","peel orange","slice cheese"]
2 ["slice cheese","peel orange"]
3 ["cut apple"]
4 ["cut apple","slice cheese"]
5 ["cut apple", "chop kiwi"]
Here is what I want the output to look like:
(hopefully auto populating the distinct list of tasks in desc order)
Tasks Quantity
Cut Apple 4
Slice Cheese 3
Peel Orange 2
Chop Kiwi 1
Step 1: Define a normalized data schema and put the schema into a database.
Your normalized data schema might look something like this:
Step 2: Add your data
Step 3: Then you will be able to use SQL COUNT with DISTINCT to find the unique rows in your data table(s)
SQL COUNT with DISTINCT
Too long for a comment, but guidance for you to look into, then try to write a sample query. While you have the opportunity to do so while learning, I would look into Data Normalization and adjust your "Tasks" column.
You should have a secondary lookup table that has a primary key ID and a description of each unique task (you'll see in the data normalization). So you can follow along from your data context to the document, I will provide the layout examples and see how that helps you.
Starting with your lookup task table...
Tasks Table
TaskID TaskDescription
1 cut apple
2 peel orange
3 slice cheese
4 chop kiwi
Then, you would have another table that has TicketID, and a third table shows multiple records for each TicketID.
Ticket Table
TicketID ExPurchaseDate
1 someDate
2 sameDate
3 etc...
Now, a detail table per ticket.
TicketTasks Table
TicketTaskID TicketID TaskID
1 1 1
2 1 2
3 1 3
4 2 3
5 2 2
6 3 1
7 4 1
7 4 3
8 5 1
9 5 4
Try to digest this some with the normalization and then look into writing a sql query with COUNT(*) and GROUP BY. More than happy to help you more after, but hope this HELPs guide you some.
I've been trying for a few hours without success, to recursively join a table to itself. It's probably simplest to see the problem in this SQL Fiddle, rather than me trying to explain the linking relationship:
http://sqlfiddle.com/#!4/367c3/14
In the above example, the actual data might nest up to 5 layers deep, perhaps more. I'd like to write a query that handles joins to any number of layers.
From doing some searches, it seems that it's possible to recursively join the data with CONNECT BY PRIOR, and other methods. I just haven't been able to get it to work :(. It'd be awesome if one of you guru's could show me how it's done.
Copying your tables from SQL Fiddle for others to see in case the external link becomes invalid in the future:
* DEVICE TABLE
DEVICEID DEVICENAME
-------------------
1 Device1
2 Device2
3 Device3
4 Device4
5 Device5
6 Device6
7 Device7
8 Device8
9 Device9
10 Device10
* CONNECTION TABLE
IDPARENT IDCHILD
----------------
1 2
3 4
4 5
6 7
7 8
4 8
4 8
5 9
I don't know what data exactly you want to query from the tables, but if you are just concerned with the self-"join" using a connect by prior, here you go:
select distinct d.deviceid, d.devicename, connect_by_root(deviceid) rootdeviceid
from device d
left join connection c on (d.deviceid = c.idchild)
connect by prior d.deviceid = c.idparent
start with c.idparent is null
order by d.deviceid
Note that there are 11 rows returned by this query given the table you provided -- this is because device 8 actually has two root nodes in the hierarchy you provided, 3 and 6.
**Schema & Dataset**
id version payment name
1 1 10 Rich
2 1 0 David
3 1 10 Marc
4 1 10 Jess
5 1 0 Steff
1 2 10 Rich
2 2 0 David
3 2 10 Marc
4 2 10 Jess
5 2 0 Steff
2 3 0 David
3 3 10 Marc
4 3 10 Jess
http://sqlfiddle.com/#!3/1c457/18 - Contains my schema and the dataset I'm working with.
Background
The data above is the final set after a stored proc has done it's processing so everything above is in one table and unfortunately I can't change it.
I need to identify in the dataset where a person has been deleted with a payment total greater than 0 in previous versions and insert a dummy row with a payment of 0. So in the above example, Rich has been deleted in version 3 with a payment total of 10 on previous versions. I need to first identify where this has happened in all instances and insert a dummy row with a 0 payment for that version. Steff has also been deleted on version 3 but she hasn't had a payment over 0 on previous versions so a dummy row is not needed for her.
Tried so far -
So I looked at pinal dave's example here and I can look back to the previous row which is great so it's a step in the right direction. I'm not sure however of how to go about achieving the above requirement. I've been toying with the idea of a case statement but I'm not certain that would be the best way to go about it. I'm really struggling with this one and would appreciate some advice on how to tackle it.
You can do this by generating all possible combinations of names and versions. Then filter out the ones you don't want according to the pay conditions:
select n.name, v.version, coalesce(de.payment, 0) as payment
from (select name, max(payment) as maxpay from dataextract group by name) n cross join
(select distinct version from dataextract) v left outer join
dataextract de
on de.name = n.name and de.version = v.version
where de.name is not null or n.maxpay > 0;
Here is a SQL Fiddle.
select p.intprojectid, p.vcprojectname, md.intmoduleid,
md.vcmodulename, md.intscreensfunc, md.vcname
from projects as p
left join (select m.intprojectid, m.intmoduleid, m.vcmodulename,
s.intscreensfunc, s.vcname
from modules as m
left join screens_func as s on m.intmoduleid = s.intmoduleid) md
on p.intprojectid = md.intprojectid
This query will return:
no |project-name|mod-id|mod-name | screen-id | screen-name
----------------------------------------------------------------
2 Project-1 4 mod-1 11 scr1
2 Project-1 4 mod-1 12 scr2
2 Project-1 4 mod-1 13 scr3
2 Project-1 4 mod-1 14 scr4
2 Project-1 8 Module-2 NULL NULL
Now I want to count no.of mod-name and no.of.screen-name in project-1. i.e. I want the query to return
project-name no.of.mod no.of.screen
------------------------------------------------
Project-1 2 4
It's definitely possible to return multiple counts.
In other words, your query could be modified as follows:
select p.vcprojectname, COUNT(DISTINCT md.intmoduleid) as no.of.mod, COUNT(md.intscreensfunc) as no.of.screen
from projects as p
left join (select m.intprojectid, m.intmoduleid, m.vcmodulename, s.intscreensfunc, s.vcname
from modules as m
left join screens_func as s
on m.intmoduleid=s.intmoduleid)md
on p.intprojectid=md.intprojectid
GROUP BY p.vcprojectname
Based on your example data, I inferred that there would be a one-many relationship between modules and screens and thus you would want a distinct count for modules but that the same requirement would not be needed for screens (since it appears that one screen would not appear multiple times in a given module) If that is not the case, you can also add distinct to the count of screens.
I have the following tables, the groups table which contains hierarchically ordered groups and group_member which stores which groups a user belongs to.
groups
---------
id
parent_id
name
group_member
---------
id
group_id
user_id
ID PARENT_ID NAME
---------------------------
1 NULL Cerebra
2 1 CATS
3 2 CATS 2.0
4 1 Cerepedia
5 4 Cerepedia 2.0
6 1 CMS
ID GROUP_ID USER_ID
---------------------------
1 1 3
2 1 4
3 1 5
4 2 7
5 2 6
6 4 6
7 5 12
8 4 9
9 1 10
I want to retrieve the visible groups for a given user. That it is to say groups a user belongs to and children of these groups. For example, with the above data:
USER VISIBLE_GROUPS
9 4, 5
3 1,2,4,5,6
12 5
I am getting these values using recursion and several database queries. But I would like to know if it is possible to do this with a single SQL query to improve my app performance. I am using MySQL.
Two things come to mind:
1 - You can repeatedly outer-join the table to itself to recursively walk up your tree, as in:
SELECT *
FROM
MY_GROUPS MG1
,MY_GROUPS MG2
,MY_GROUPS MG3
,MY_GROUPS MG4
,MY_GROUPS MG5
,MY_GROUP_MEMBERS MGM
WHERE MG1.PARENT_ID = MG2.UNIQID (+)
AND MG1.UNIQID = MGM.GROUP_ID (+)
AND MG2.PARENT_ID = MG3.UNIQID (+)
AND MG3.PARENT_ID = MG4.UNIQID (+)
AND MG4.PARENT_ID = MG5.UNIQID (+)
AND MGM.USER_ID = 9
That's gonna give you results like this:
UNIQID PARENT_ID NAME UNIQID_1 PARENT_ID_1 NAME_1 UNIQID_2 PARENT_ID_2 NAME_2 UNIQID_3 PARENT_ID_3 NAME_3 UNIQID_4 PARENT_ID_4 NAME_4 UNIQID_5 GROUP_ID USER_ID
4 2 Cerepedia 2 1 CATS 1 null Cerebra null null null null null null 8 4 9
The limit here is that you must add a new join for each "level" you want to walk up the tree. If your tree has less than, say, 20 levels, then you could probably get away with it by creating a view that showed 20 levels from every user.
2 - The only other approach that I know of is to create a recursive database function, and call that from code. You'll still have some lookup overhead that way (i.e., your # of queries will still be equal to the # of levels you are walking on the tree), but overall it should be faster since it's all taking place within the database.
I'm not sure about MySql, but in Oracle, such a function would be similar to this one (you'll have to change the table and field names; I'm just copying something I did in the past):
CREATE OR REPLACE FUNCTION GoUpLevel(WO_ID INTEGER, UPLEVEL INTEGER) RETURN INTEGER
IS
BEGIN
DECLARE
iResult INTEGER;
iParent INTEGER;
BEGIN
IF UPLEVEL <= 0 THEN
iResult := WO_ID;
ELSE
SELECT PARENT_ID
INTO iParent
FROM WOTREE
WHERE ID = WO_ID;
iResult := GoUpLevel(iParent,UPLEVEL-1); --recursive
END;
RETURN iResult;
EXCEPTION WHEN NO_DATA_FOUND THEN
RETURN NULL;
END;
END GoUpLevel;
/
Joe Cleko's books "SQL for Smarties" and "Trees and Hierarchies in SQL for Smarties" describe methods that avoid recursion entirely, by using nested sets. That complicates the updating, but makes other queries (that would normally need recursion) comparatively straightforward. There are some examples in this article written by Joe back in 1996.
I don't think that this can be accomplished without using recursion. You can accomplish it with with a single stored procedure using mySQL, but recursion is not allowed in stored procedures by default. This article has information about how to enable recursion. I'm not certain about how much impact this would have on performance verses the multiple query approach. mySQL may do some optimization of stored procedures, but otherwise I would expect the performance to be similar.
Didn't know if you had a Users table, so I get the list via the User_ID's stored in the Group_Member table...
SELECT GroupUsers.User_ID,
(
SELECT
STUFF((SELECT ',' +
Cast(Group_ID As Varchar(10))
FROM Group_Member Member (nolock)
WHERE Member.User_ID=GroupUsers.User_ID
FOR XML PATH('')),1,1,'')
) As Groups
FROM (SELECT User_ID FROM Group_Member GROUP BY User_ID) GroupUsers
That returns:
User_ID Groups
3 1
4 1
5 1
6 2,4
7 2
9 4
10 1
12 5
Which seems right according to the data in your table. But doesn't match up with your expected value list (e.g. User 9 is only in one group in your table data but you show it in the results as belonging to two)
EDIT: Dang. Just noticed that you're using MySQL. My solution was for SQL Server. Sorry.
-- Kevin Fairchild
There was already similar question raised.
Here is my answer (a bit edited):
I am not sure I understand correctly your question, but this could work My take on trees in SQL.
Linked post described method of storing tree in database -- PostgreSQL in that case -- but the method is clear enough, so it can be adopted easily for any database.
With this method you can easy update all the nodes depend on modified node K with about N simple SELECTs queries where N is distance of K from root node.
Good Luck!
I don't remember which SO question I found the link under, but this article on sitepoint.com (second page) shows another way of storing hierarchical trees in a table that makes it easy to find all child nodes, or the path to the top, things like that. Good explanation with example code.
PS. Newish to StackOverflow, is the above ok as an answer, or should it really have been a comment on the question since it's just a pointer to a different solution (not exactly answering the question itself)?
There's no way to do this in the SQL standard, but you can usually find vendor-specific extensions, e.g., CONNECT BY in Oracle.
UPDATE: As the comments point out, this was added in SQL 99.