linq to sql sub-select - sql

I have two tables that represent say a Mixture and the components of the Mixture. The table layouts are like:
Mixture table:
MixtureID uniqueidentifier
MixtureDescription varchar(50)
Components table:
ComponentID uniqueidentifier
MixtureID uniqueidentifier (FK to previous table)
ComponentName varchar(50)
ComponentRatioPercentage int
Now, what I want to do is take a list of component names inputted from the user and find the ID's of any mixes that contain all of those components.
In SQL I could so something like:
select distinct MixtureID
from Mixture Mixture
where exists (select ComponentName
from Components Components1
where Components1.MixtureID = Mixture.MixtureID and
Components1.ComponentDescription = 'SomeComponentName')
and exists (select ComponentName
from Components2
where Components2.MixtureID = Mixture.MixtureID and
Components2.ComponentDescription = 'SomeOtherComponentName')
and exists....
etc, adding a sub-select for each component.
How would I do something like this in linq to sql? The number of components to look for would not be known in advance, until the user is done with input, though there would be a max of 10. Thanks in advance!

var components = new string[] {"SomeComponentName", "SomeOtherComponentName"};
var query = Mixtures.AsQueryable();
foreach (var component in components)
{
var tmpComponent = component;
query = query.Where(m => m.Components
.Any(c => t.ComponentDescription == tmpComponent)
);
}
var mixturesIds = query.Select(m=>m.MixtureId).Distinct();

Related

Efficiently mapping one-to-many many-to-many database to struct in Golang

Question
When dealing with a one-to-many or many-to-many SQL relationship in Golang, what is the best (efficient, recommended, "Go-like") way of mapping the rows to a struct?
Taking the example setup below I have tried to detail some approaches with Pros and Cons of each but was wondering what the community recommends.
Requirements
Works with PostgreSQL (can be generic but not include MySQL/Oracle specific features)
Efficiency - No brute forcing every combination
No ORM - Ideally using only database/sql and jmoiron/sqlx
Example
For sake of clarity I have removed error handling
Models
type Tag struct {
ID int
Name string
}
type Item struct {
ID int
Tags []Tag
}
Database
CREATE TABLE item (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY
);
CREATE TABLE tag (
id INT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
name VARCHAR(160),
item_id INT REFERENCES item(id)
);
Approach 1 - Select all Items, then select tags per item
var items []Item
sqlxdb.Select(&items, "SELECT * FROM item")
for i, item := range items {
var tags []Tag
sqlxdb.Select(&tags, "SELECT * FROM tag WHERE item_id = $1", item.ID)
items[i].Tags = tags
}
Pros
Simple
Easy to understand
Cons
Inefficient with the number of database queries increasing proportional with number of items
Approach 2 - Construct SQL join and loop through rows manually
var itemTags = make(map[int][]Tag)
var items = []Item{}
rows, _ := sqlxdb.Queryx("SELECT i.id, t.id, t.name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
for rows.Next() {
var (
itemID int
tagID int
tagName string
)
rows.Scan(&itemID, &tagID, &tagName)
if tags, ok := itemTags[itemID]; ok {
itemTags[itemID] = append(tags, Tag{ID: tagID, Name: tagName,})
} else {
itemTags[itemID] = []Tag{Tag{ID: tagID, Name: tagName,}}
}
}
for itemID, tags := range itemTags {
items = append(Item{
ID: itemID,
Tags: tags,
})
}
Pros
A single database call and cursor that can be looped through without eating too much memory
Cons
Complicated and harder to develop with multiple joins and many attributes on the struct
Not too performant; more memory usage and processing time vs. more network calls
Failed approach 3 - sqlx struct scanning
Despite failing I want to include this approach as I find it to be my current aim of efficiency paired with development simplicity. My hope was by explicitly setting the db tag on each struct field sqlx could do some advanced struct scanning
var items []Item
sqlxdb.Select(&items, "SELECT i.id AS item_id, t.id AS tag_id, t.name AS tag_name FROM item AS i JOIN tag AS t ON t.item_id = i.id")
Unfortunately this errors out as missing destination name tag_id in *[]Item leading me to believe the StructScan is not advanced enough to recursively loop through rows (no criticism - it is a complicated scenario)
Possible approach 4 - PostgreSQL array aggregators and GROUP BY
While I am sure this will not work I have included this untested option to see if it could be improved upon so it may work.
var items = []Item{}
sqlxdb.Select(&items, "SELECT i.id as item_id, array_agg(t.*) as tags FROM item AS i JOIN tag AS t ON t.item_id = i.id GROUP BY i.id")
When I have some time I will try and run some experiments here.
the sql in postgres :
create schema temp;
set search_path = temp;
create table item
(
id INT generated by default as identity primary key
);
create table tag
(
id INT generated by default as identity primary key,
name VARCHAR(160),
item_id INT references item (id)
);
create view item_tags as
select id,
(
select
array_to_json(array_agg(row_to_json(taglist.*))) as array_to_json
from (
select tag.name, tag.id
from tag
where item_id = item.id
) taglist ) as tags
from item ;
-- golang query this maybe
select row_to_json(row)
from (
select * from item_tags
) row;
then golang query this sql:
select row_to_json(row)
from (
select * from item_tags
) row;
and unmarshall to go struct:
pro:
postgres manage the relation of data. add / update data with sql functions.
golang manage business model and logic.
it's easy way.
.
I can suggest another approach which I have used before.
You make a json of the tags in this case in the query and return it.
Pros: You have 1 call to the db, which aggregates the data, and all you have to do is parse the json into an array.
Cons: It's a bit ugly. Feel free to bash me for it.
type jointItem struct {
Item
ParsedTags string
Tags []Tag `gorm:"-"`
}
var jointItems []*jointItem
db.Raw(`SELECT
items.*,
(SELECT CONCAT(
'[',
GROUP_CONCAT(
JSON_OBJECT('id', id,
'name', name
)
),
']'
)) as parsed_tags
FROM items`).Scan(&jointItems)
for _, o := range jointItems {
var tempTags []Tag
if err := json.Unmarshall(o.ParsedTags, &tempTags) ; err != nil {
// do something
}
o.Tags = tempTags
}
Edit: code might behave weirdly so I find it better to use a temporary tags array when moving instead of using the same struct.
You can use carta.Map() from https://github.com/jackskj/carta
It tracks has-many relationships automatically.

How do I map entity framework to a child-parent look up table based on a key

I have a database that has a parent-child look up for my drop down values, there is a domain table which represents what type of drop down it is ie "TITLE", "MARITAL_STATUS", "COUNTRY" and there is look up for the values associated with each of these drop downs that we call domain values
EG
TITLE (Domain table)
MR (Domain Value table)
MRS (Domain Value table)
MISS (Domain Value table)
DOCTOR (Domain Value table)
MARITAL_STATUS (Domain table)
SINGLE (Domain Value table)
MARRIED (Domain Value table)
DIVORCED (Domain Value table)
I find myself writing ugly code which exposes the under lying implementation similar to below
using (var db = new OrderContext())
{
...
var maritalStatusId = (
from sp in db.DomainValues
where sp.ShortCode == "MARRIED"
where sp.Domain.ShortCode == "MARITAL_STATUS"
select sp.Id
).FirstOrDefault();
if maritalStatusId != 0)
{
orderStore.maritalStatus = maritalStatusId;
}
...
db.OrderStores.Add(orderStore);
db.SaveChanges();
}
Is it possible to write code that looks similar to this
using (var db = new OrderContext())
{
...
orderStore.maritalStatus = new MaritalStatus { ShortCode = "MARRIED" }
...
db.OrderStores.Add(orderStore);
db.SaveChanges();
}
, where a navigation property handles the insert?
If so how do I set this up?
Any links to articles on this would be good too.
You can keep a cache with the different values of your dropdowns, that seems reasonnable, or you can use enumeration if the values are not dynamic.
Another solution would be to have the shortcode field to be the primary key of the table, all your problems would be resolved.

Error using entity framework to add rows to tables using foreign key relation

I've written the following C# code:
abcEntities entities = new abcEntities();
abc_Lids_New newUserEntry = new abc_Lids_New()
{
Lid = lid,
FName = fname,
LName = lname,
Phone = phone,
};
newUserEntry.abc_class_Assignments_New = new System.Data.Objects.DataClasses.EntityCollection<abc_class_Assignments_New>();
foreach (string classId in classIds)
{
newUserEntry.abc_class_Assignments_New.Add(new abc_class_Assignments_New()
{
Lid = lid,
classID = classId
});
}
entities.abc_Lids_New.AddObject(newUserEntry);
entities.SaveChanges();
The code is supposed to add a new row in abc_Lids_New representing a new user. The code is also supposed to add rows in abc_class_assignments_new corresponding to the user's lid and a number of class ids.
However, I am getting an error: Unable to update the EntitySet 'UCV_TF_Assignments_New' because it has a DefiningQuery and no <InsertFunction> element exists in the <ModificationFunctionMapping> element to support the current operation.
Up until recently, I had only used the Entity Framework with stored procedures, so this is something new to me.
Your EntitySet "'UCV_TF_Assignments_New" is either based on a View (in which case, you have to create the InsertFunction (stored procedure)) or it lacks a Primary Key (in which case, add one).

NHibernate Linking Table with Data

SQL 2008 | .NET 4.0 | NHibernate 3.1 | NHibernate.Castle 3.1 | Castle.Core 2.5.2
So I have a linking table with metadata, like the author of this question NHibernate Mapping a Many to Many with Data on Join Table
Initially, I mapped just like the answer to this question as it seemed the most parsimonious way to handle it. However, after turning on show_sql and observing what was going on, the ID lookups ended up yielding N+1 queries where N is the number of associations.
Observe this example database that is analogous to my actual data, defined in sql-like syntax
CREATE TABLE [User]
(
Id int PRIMARY KEY
)
CREATE TABLE UserPref
(
Id int PRIMARY KEY,
Name varchar(32)
)
CREATE TABLE UserPrefAssociation
(
UserId int,
PrefId int,
Value varchar(32)
)
I hacked the following code together with this User one-to-many object mapping IList<UserPrefAssociation> Preferences { get; set; }
public IDictionary<string, string> GeneratePrefDict()
{
return Preferences
.ToDictionary(i => i.UserPref.Name, i => i.Value);
}
Sure, this works great, but as mentioned before, each i.UserPref.Name, is an additional query to SQL.
After playing in SQL, I have found the query that accomplishes what I want. My question then becomes how can I do this with NHibernate?
SELECT UserPref.Name, UserPrefAssociation.Value
FROM [User]
INNER JOIN UserPrefAssociation ON [User].Id = UserPrefAssociation.UserId
INNER JOIN UserPref ON UserPrefAssociation.UserPrefId = UserPref.Id
WHERE [User].Id = 1
~~~~SOLVED~~~~~
using NHibernate.Linq;
...
public IDictionary<string, string> GeneratePrefDict(ISession s)
{
return
(from entry in s.Query<User_UserPref>()
where entry.User == this
select new
{
key = entry.UserPref.Name,
value = entry.Value
})
.ToDictionary(i => i.key, i => i.value);
}
Generates this SQL
NHibernate: select userpref1_.Name as col_0_0_, user_userp0_.Value as col_1_0_ f
rom User_UserPref user_userp0_ left outer join UserPref userpref1_ on user_userp
0_.UserPrefId=userpref1_.Id where user_userp0_.UserId=#p0;#p0 = 1 [Type: Int32 (
0)]
Which is better than N+1 queries, and solves my issue.
I think you can achieve what you are wanting with Futures and QueryOver. Take a look at the following article:
Fighting cartesian product (x-join) when using NHibernate 3.0.0
If you can't visualize how to accomplish what you need from the above I can tailor that example more to your needs.

LINQ to Entity, joining on NOT IN tables

My brain seems to be mush right now! I am using LINQ to Entity, and I need to get some data from one table that does NOT exist in another table.
For example: I need the groupID, groupname and groupnumber from TABLE A where they do not exist in TABLE B. The groupID will exist in TABLE B, along with other relevant information. The tables do not have any relationship. In SQL it would be quite simply (there is a more elegant and efficient solution, but I want to paint a picture of what I need)
SELECT
GroupID,
GroupName,
GroupNumber,
FROM
TableA
WHERE
GroupID NOT IN (SELECT GroupID FROM TableB)
Is there an easy/elegant way to do this using the Entity Framework/LINQ to Entity?
Right now I have a bunch of queries hitting the db, then comparing, etc. It's pretty messy.
You could use any
var temp =context.TableA
.Where(x=>!context.TableB.Any(y=>y.GroupID!=x.GroupID))
.Select(x=>new { GroupID = x.GroupID, GroupName=x.GroupName, GroupNumber = x.GroupNumber}).ToList();
It depends upon how you've met them, which you don't show, but, generally:
var q = from a in Context.TableA
where !a.Group.TableBs.Any()
select new
{
GroupID = a.GroupID,
GroupName = a.GroupName,
GroupNumber = a.GroupNumber
};
#Nix - Your result set should have been either:
var temp =context.TableA
.Where(x=>context.TableB.Any(y=>y.GroupID != x.GroupID))
.Select(x=>new { GroupID = x.GroupID, GroupName=x.GroupName, GroupNumber = x.GroupNumber}).ToList();
Or
var temp =context.TableA
.Where(x=> ! context.TableB.Any(y=>y.GroupID == x.GroupID))
.Select(x=>new { GroupID = x.GroupID, GroupName=x.GroupName, GroupNumber = x.GroupNumber}).ToList();
But NOT both, like you wrote it.