SQL 2008 | .NET 4.0 | NHibernate 3.1 | NHibernate.Castle 3.1 | Castle.Core 2.5.2
So I have a linking table with metadata, like the author of this question NHibernate Mapping a Many to Many with Data on Join Table
Initially, I mapped just like the answer to this question as it seemed the most parsimonious way to handle it. However, after turning on show_sql and observing what was going on, the ID lookups ended up yielding N+1 queries where N is the number of associations.
Observe this example database that is analogous to my actual data, defined in sql-like syntax
CREATE TABLE [User]
(
Id int PRIMARY KEY
)
CREATE TABLE UserPref
(
Id int PRIMARY KEY,
Name varchar(32)
)
CREATE TABLE UserPrefAssociation
(
UserId int,
PrefId int,
Value varchar(32)
)
I hacked the following code together with this User one-to-many object mapping IList<UserPrefAssociation> Preferences { get; set; }
public IDictionary<string, string> GeneratePrefDict()
{
return Preferences
.ToDictionary(i => i.UserPref.Name, i => i.Value);
}
Sure, this works great, but as mentioned before, each i.UserPref.Name, is an additional query to SQL.
After playing in SQL, I have found the query that accomplishes what I want. My question then becomes how can I do this with NHibernate?
SELECT UserPref.Name, UserPrefAssociation.Value
FROM [User]
INNER JOIN UserPrefAssociation ON [User].Id = UserPrefAssociation.UserId
INNER JOIN UserPref ON UserPrefAssociation.UserPrefId = UserPref.Id
WHERE [User].Id = 1
~~~~SOLVED~~~~~
using NHibernate.Linq;
...
public IDictionary<string, string> GeneratePrefDict(ISession s)
{
return
(from entry in s.Query<User_UserPref>()
where entry.User == this
select new
{
key = entry.UserPref.Name,
value = entry.Value
})
.ToDictionary(i => i.key, i => i.value);
}
Generates this SQL
NHibernate: select userpref1_.Name as col_0_0_, user_userp0_.Value as col_1_0_ f
rom User_UserPref user_userp0_ left outer join UserPref userpref1_ on user_userp
0_.UserPrefId=userpref1_.Id where user_userp0_.UserId=#p0;#p0 = 1 [Type: Int32 (
0)]
Which is better than N+1 queries, and solves my issue.
I think you can achieve what you are wanting with Futures and QueryOver. Take a look at the following article:
Fighting cartesian product (x-join) when using NHibernate 3.0.0
If you can't visualize how to accomplish what you need from the above I can tailor that example more to your needs.
Related
Question
When dealing with a one-to-many or many-to-many SQL relationship in Golang, what is the best (efficient, recommended, "Go-like") way of mapping the rows to a struct?
Taking the example setup below I have tried to detail some approaches with Pros and Cons of each but was wondering what the community recommends.
Requirements
Works with PostgreSQL (can be generic but not include MySQL/Oracle specific features)
Efficiency - No brute forcing every combination
No ORM - Ideally using only database/sql and jmoiron/sqlx
Example
For sake of clarity I have removed error handling
Models
type Tag struct {
ID int
Name string
}
type Item struct {
ID int
Tags []Tag
}
Database
CREATE TABLE item (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY
);
CREATE TABLE tag (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
name VARCHAR(160),
item_id INT REFERENCES item(id)
);
Approach 1 - Select all Items, then select tags per item
var items []Item
sqlxdb.Select(&items, "SELECT * FROM item")
for i, item := range items {
var tags []Tag
sqlxdb.Select(&tags, "SELECT * FROM tag WHERE item_id = $1", item.ID)
items[i].Tags = tags
}
Pros
Simple
Easy to understand
Cons
Inefficient with the number of database queries increasing proportional with number of items
Approach 2 - Construct SQL join and loop through rows manually
var itemTags = make(map[int][]Tag)
var items = []Item{}
rows, _ := sqlxdb.Queryx("SELECT i.id, t.id, t.name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
for rows.Next() {
var (
itemID int
tagID int
tagName string
)
rows.Scan(&itemID, &tagID, &tagName)
if tags, ok := itemTags[itemID]; ok {
itemTags[itemID] = append(tags, Tag{ID: tagID, Name: tagName,})
} else {
itemTags[itemID] = []Tag{Tag{ID: tagID, Name: tagName,}}
}
}
for itemID, tags := range itemTags {
items = append(Item{
ID: itemID,
Tags: tags,
})
}
Pros
A single database call and cursor that can be looped through without eating too much memory
Cons
Complicated and harder to develop with multiple joins and many attributes on the struct
Not too performant; more memory usage and processing time vs. more network calls
Failed approach 3 - sqlx struct scanning
Despite failing I want to include this approach as I find it to be my current aim of efficiency paired with development simplicity. My hope was by explicitly setting the db tag on each struct field sqlx could do some advanced struct scanning
var items []Item
sqlxdb.Select(&items, "SELECT i.id AS item_id, t.id AS tag_id, t.name AS tag_name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
Unfortunately this errors out as missing destination name tag_id in *[]Item leading me to believe the StructScan is not advanced enough to recursively loop through rows (no criticism - it is a complicated scenario)
Possible approach 4 - PostgreSQL array aggregators and GROUP BY
While I am sure this will not work I have included this untested option to see if it could be improved upon so it may work.
var items = []Item{}
sqlxdb.Select(&items, "SELECT i.id as item_id, array_agg(t.*) as tags FROM item AS i JOIN tag AS t ON t.item_id = i.id GROUP BY i.id")
When I have some time I will try and run some experiments here.
the sql in postgres :
create schema temp;
set search_path = temp;
create table item
(
id INT generated by default as identity primary key
);
create table tag
(
id INT generated by default as identity primary key,
name VARCHAR(160),
item_id INT references item (id)
);
create view item_tags as
select id,
(
select
array_to_json(array_agg(row_to_json(taglist.*))) as array_to_json
from (
select tag.name, tag.id
from tag
where item_id = item.id
) taglist ) as tags
from item ;
-- golang query this maybe
select row_to_json(row)
from (
select * from item_tags
) row;
then golang query this sql:
select row_to_json(row)
from (
select * from item_tags
) row;
and unmarshall to go struct:
pro:
postgres manage the relation of data. add / update data with sql functions.
golang manage business model and logic.
it's easy way.
.
I can suggest another approach which I have used before.
You make a json of the tags in this case in the query and return it.
Pros: You have 1 call to the db, which aggregates the data, and all you have to do is parse the json into an array.
Cons: It's a bit ugly. Feel free to bash me for it.
type jointItem struct {
Item
ParsedTags string
Tags []Tag `gorm:"-"`
}
var jointItems []*jointItem
db.Raw(`SELECT
items.*,
(SELECT CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT('id', id,
'name', name
)
),
']'
)) as parsed_tags
FROM items`).Scan(&jointItems)
for _, o := range jointItems {
var tempTags []Tag
if err := json.Unmarshall(o.ParsedTags, &tempTags) ; err != nil {
// do something
}
o.Tags = tempTags
}
Edit: code might behave weirdly so I find it better to use a temporary tags array when moving instead of using the same struct.
You can use carta.Map() from https://github.com/jackskj/carta
It tracks has-many relationships automatically.
I have the following structure
task -> hasMany links -> hasOne user
what I would like to see returned is
task.id | [
{ linkType: 'foo', userid:1, taskid:1,
user : { name: 'jon', lastname: 'smith'},
... ]
being a sql noob, I have managed to get so far as this
select task.id, json_agg(links.*) as links from task
left outer join links on task.id = links.taskid
join "user" on "user".id = links.userid
group by task.id ;
which gives me
task.id | [{ linkType: 'foo', userid:1, taskid:1}, ... ]
but obviously missing the user
I'm kinda stuck now on how to add the user property to each of the link array items
I've read several documents but they always seem to stop at the first join
the schema design is
create table task (id int GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY)
create table "user" (id int GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY, username text)
create table links (taskid integer references task (id), userid integer references "user" (id) );
Your example does not reference the table structure you provided, because "user" has a name and a lastname column in the former and a single username column in the latter, but you did provide enough information to give an answer. I am using the two-column user, because it allows to show how to construct nested JSON objects.
SELECT task.id,
json_agg(
json_build_object(
'linkType', 'foo',
'taskid', links.taskid,
'userid', links.userid,
'user', json_build_object('name', u.name, 'lastname', u.lastname)
)
) AS links
FROM task
LEFT OUTER JOIN links ON task.id = links.taskid
JOIN "user" u ON u.id = links.userid
GROUP BY task.id;
The general problem of building json objects with the appropriate key values is discussed in this excellent DBA.SE answer. I have adopted what is listed there as solution number 3 because I think it is the most flexible and the most readable. Your tastes might differ.
Using ObjectContext. I'm wanting to do this by passing an SQL query via the ExecuteStoreCommand since I don't fancy retrieving all relevant entities just for the sake of deleting them after.
The Category table is as so:
CatID | CatName | ParentID
Where CatID is the primary key to the ParentID FK
I am wishing to delete a category and also all those that
are under it. Can be 2+ levels deep of sub cats, so different ParentID's
Thought I could do it as below and just set "cascade" on delete option
for the foreign key in the database, but it won't let me and it does not appear to want to
cascade delete down by using the CatID - ParentID relationship and the query gets
stopped by this very FK constraint.
public RedirectToRouteResult DelCat(int CatID)
{
if (CatID != 0)
{
_db.ExecuteStoreCommand("DELETE FROM Categories WHERE CatID={0}", CatID);
_db.SaveChanges();
}
return RedirectToAction("CatManage");
}
Recursive CTE allCategories produces list of all categories in hierarchy. Delete part, obviously, deletes them all.
; with allCategories as (
select CatID
from Categories
where CatID = #CatID_to_delete
union all
select Categories.CatID
from allCategories
inner join Categories
on allCategories.CatID = Categories.ParentID
)
delete Categories
from Categories
inner join allCategories
on Categories.CatID = allCategories.CatID
Try it with select * from allCategories, though, to check first.
There is TEST # Sql Fiddle.
Why not just send two statements in your batch?
DELETE Categories WHERE ParentID = {0};
DELETE Categories WHERE CatID = {0};
If the framework you're using "won't let you" do that, then do this right: put your logic in a stored procedure, and call the stored procedure.
I currently have a legacy system that uses SPs exclusively for access to the DB. My domain object looks something like this:
public class User : EntityBase
{
public virtual string Name {get;set;}
public virtual string CreatedBy {get;set;}
public virtual DateTime CreatedDate {get;set;}
}
The SP I have that mapped this looked like this:
CREATE PROCEDURE getUser
{
#ID int
}
select
Name
,(SELECT TOP 1 UserName FROM AuditTrail WHERE EntityID = [User].[ID] AND EntityName = 'User' AND AuditActionID = 1 ORDER BY DateStamp) AS CreatedBy
,(SELECT TOP 1 DateStamp FROM AuditTrail WHERE EntityID = [User].[ID] AND EntityName = 'User' AND AuditActionID = 1 ORDER BY DateStamp) AS CreatedDate
FROM [User]
WHERE [User].ID = #ID
So, as you can see, the audit information is separated from the entity itself on the database and the CreatedBy/CreatedOn (and ModifiedBy/ModifiedOn) are stored in a separate table called AuditTrail. the AuditActionID field on the table specifies if it was a create/update.
How can I setup this mapping with NHibernate? I looked into JOIN but it doesn't give me the option to restrict by the additional values (and a join isn't what I want).
Also, if this is possible in Fluent NHibernate, that's a bonus but I'm fine with trying just standard NHibernate mapping config if it gets me there.
UPDATE:
I have found one way to do this, but I'm not a fan. I have setup a SQLQuery that reads the data and maps it back to an object. It works, but I'd love to do this via mapping. Is it even possible since the "values" from the database I'm mapping to is a subselect and not editable?
Solution:
Thanks to the tip from Diego, this was the final solution I found (using Fluent NHibernate, in my ClassMap file):
Map(x => x.CreatedBy).Formula("(SELECT TOP 1 AuditTrail.UserName FROM AuditTrail WHERE AuditTrail.EntityID = [ID] AND AuditTrail.EntityName = 'User' AND AuditTrail.AuditActionID = 1 ORDER BY AuditTrail.DateStamp)");
Map(x => x.CreatedDate).Formula("(SELECT TOP 1 AuditTrail.DateStamp FROM AuditTrail WHERE AuditTrail.EntityID = [ID] AND AuditTrail.EntityName = 'User' AND AuditTrail.AuditActionID = 1 ORDER BY AuditTrail.DateStamp)");
Thanks,
M
You can specify the select clause as the formula for your property.
My brain seems to be mush right now! I am using LINQ to Entity, and I need to get some data from one table that does NOT exist in another table.
For example: I need the groupID, groupname and groupnumber from TABLE A where they do not exist in TABLE B. The groupID will exist in TABLE B, along with other relevant information. The tables do not have any relationship. In SQL it would be quite simply (there is a more elegant and efficient solution, but I want to paint a picture of what I need)
SELECT
GroupID,
GroupName,
GroupNumber,
FROM
TableA
WHERE
GroupID NOT IN (SELECT GroupID FROM TableB)
Is there an easy/elegant way to do this using the Entity Framework/LINQ to Entity?
Right now I have a bunch of queries hitting the db, then comparing, etc. It's pretty messy.
You could use any
var temp =context.TableA
.Where(x=>!context.TableB.Any(y=>y.GroupID!=x.GroupID))
.Select(x=>new { GroupID = x.GroupID, GroupName=x.GroupName, GroupNumber = x.GroupNumber}).ToList();
It depends upon how you've met them, which you don't show, but, generally:
var q = from a in Context.TableA
where !a.Group.TableBs.Any()
select new
{
GroupID = a.GroupID,
GroupName = a.GroupName,
GroupNumber = a.GroupNumber
};
#Nix - Your result set should have been either:
var temp =context.TableA
.Where(x=>context.TableB.Any(y=>y.GroupID != x.GroupID))
.Select(x=>new { GroupID = x.GroupID, GroupName=x.GroupName, GroupNumber = x.GroupNumber}).ToList();
Or
var temp =context.TableA
.Where(x=> ! context.TableB.Any(y=>y.GroupID == x.GroupID))
.Select(x=>new { GroupID = x.GroupID, GroupName=x.GroupName, GroupNumber = x.GroupNumber}).ToList();
But NOT both, like you wrote it.