GORM many2many preload error - sql

Currently using GORM to connect to two databases: POSTGRES AND sqlite (using a code switch to choose which one to use). I have a 2 database tables defined in my schema that look like this:
type TableClient struct {
Model
Synchronised bool
FacilityID string `gorm:"primary_key"`
Age int
ClientSexID int
MaritalStatusID int
SpecificNeeds []TableOptionList`gorm:"many2many:options_specific_needs"`
}
type TableOptionList struct {
ID int `gorm:"primary_key"`
Name string
Value string
Text string
SortKey int
}
Previously, I would preload related table with code like this:
var dbClient TableClient
Db.Where("facility_id = ? AND client_id = ? AND id = ?;", URLFacilityID, URLClientID, URLIncidentID).
Preload("ClientSex").
Preload("MaritalStatus").
Preload("CareTakerRelationShip").
Preload("HighestLevelOfEducation").
Preload("Occupation").
Preload("SpecificNeeds").
First(&dbClient)
Now that lookup fails with a syntax error and when I look at the SQL generated, it shows the following SQL is generated:
SELECT * FROM "table_option_lists" INNER JOIN "options_specific_needs" ON "options_specific_needs"."table_option_list_id" = "table_option_lists"."id" WHERE (("options_specific_needs"."table_client_id","options_specific_needs"."table_client_facility_id") IN (('one','LS034')))
Pasting that into the sqlite console also fails with the same error: (near ",": syntax error)

The crux of your issue is in your WHERE clause, you need to use "OR", not ",".
This is likely an issue with GORM and you could open a GitHub issue with them for this. You can get around the issue using db.Raw() from GORM. In general, I prefer to avoid ORMs. Part of the reason is it is often easier to just build an SQL query by hand than try to figure out the strange way to do it in the ORM (for things beyond basic CRUD).

Related

Slick plain sql query with pagination

I have something like this, using Akka, Alpakka + Slick
Slick
.source(
sql"""select #${onlyTheseColumns.mkString(",")} from #${dbSource.table}"""
.as[Map[String, String]]
.withStatementParameters(rsType = ResultSetType.ForwardOnly, rsConcurrency = ResultSetConcurrency.ReadOnly, fetchSize = batchSize)
.transactionally
).map( doSomething )...
I want to update this plain sql query with skipping the first N-th element.
But that is very DB specific.
Is is possible to get the pagination bit generated by Slick? [like for type-safe queries one just do a drop, filter, take?]
ps: I don't have the Schema, so I cannot go the type-safe way, just want all tables as Map, filter, drop etc on them.
ps2: at akka level, the flow.drop works, but it's not optimal/slow, coz it still consumes the rows.
Cheers
Since you are using the plain SQL, you have to provide a workable SQL in code snippet. Plain SQL may not type-safe, but agile.
BTW, the most optimal way is to skip N-th element by Database, such as limit in mysql.
depending on your database engine, you could use something like
val page = 1
val pageSize = 10
val query = sql"""
select #${onlyTheseColumns.mkString(",")}
from #${dbSource.table}
limit #${pageSize + 1}
offset #${pageSize * (page - 1)}
"""
the pageSize+1 part tells you whether the next page exists
I want to update this plain sql query with skipping the first N-th element. But that is very DB specific.
As you're concerned about changing the SQL for different databases, I suggest you abstract away that part of the SQL and decide what to do based on the Slick profile being used.
If you are working with multiple database product, you've probably already abstracted away from any specific profile, perhaps using JdbcProfile. In that case you could place your "skip N elements" helper in a class and use the active slickProfile to decide on the SQL to use. (As an alternative you could of course check via some other means, such as an environment value you set).
In practice that could be something like this:
case class Paginate(profile: slick.jdbc.JdbcProfile) {
// Return the correct LIMIT/OFFSET SQL for the current Slick profile
def page(size: Int, firstRow: Int): String =
if (profile.isInstanceOf[slick.jdbc.H2Profile]) {
s"LIMIT $size OFFSET $firstRow"
} else if (profile.isInstanceOf[slick.jdbc.MySQLProfile]) {
s"LIMIT $firstRow, $size"
} else {
// And so on... or a default
// Danger: I've no idea if the above SQL is correct - it's just placeholder
???
}
}
Which you could use as:
// Import your profile
import slick.jdbc.H2Profile.api._
val paginate = Paginate(slickProfile)
val action: DBIO[Seq[Int]] =
sql""" SELECT cols FROM table #${paginate.page(100, 10)}""".as[Int]
In this way, you get to isolate (and control) RDBMS-specific SQL in one place.
To make the helper more usable, and as slickProfile is implicit, you could instead write:
def page(size: Int, firstRow: Int)(implicit profile: slick.jdbc.JdbcProfile) =
// Logic for deciding on SQL goes here
I feel obliged to comment that using a splice (#$) in plain SQL opens you to SQL injection attacks if any of the values are provided by a user.

How can I have slick generates query without double quotes?

I'm trying to using Slick to interact with Oracle DB. The mapping is:
trait EntityTable extends DataBaseConfig{
import driver.api._
class Log(tag: Tag) extends Table[LogEntity](tag,"LOG_SCHEMA.A_TABLE_WITH_VERY_LONG_NAME") {
def id = column[Option[Long]]("LOG_ID",O.PrimaryKey,O.AutoInc)
def log = column[String]("LOG_TEXT")
def * = (LOG_ID, LOG_TEXT) <> ((LogEntity.apply _).tupled, LogEntity.unapply)
}
protected val getLogs = TableQuery[Log]
}
From the debugger I see the SQL generated is:
select "LOG_ID", "LOG_TEXT" from "LOG_SCHEMA.A_TABLE_WITH_VERY_LONG_NAME"
This gives me an
ORA-00972: identifier is too long
How can I have slick generates query without double quotes? Or is there a better way to deal with long table name in different schemes which I do not have control over? Thanks!
In Oracle you can not have names (table, index, view) longer than 30 characters. In your example I guess you have a mistake, instead of
"LOG_SCHEMA.A_TABLE_WITH_VERY_LONG_NAME"
try this:
"""LOG_SCHEMA"."A_TABLE_WITH_VERY_LONG_NAME"""
or this:
"\"LOG_SCHEMA\".\"A_TABLE_WITH_VERY_LONG_NAME\""
It does not matter how the table has been created A_TABLE_WITH_VERY_LONG_NAME can not have more than 30 characters.

How to use LINQ to Entities to make a left join using a static value

I've got a few tables, Deployment, Deployment_Report and Workflow. In the event that the deployment is being reviewed they join together so you can see all details in the report. If a revision is going out, the new workflow doesn't exist yet new workflow is going into place so I'd like the values to return null as the revision doesn't exist yet.
Complications aside, this is a sample of the SQL that I'd like to have run:
DECLARE #WorkflowID int
SET #WorkflowID = 399 -- Set to -1 if new
SELECT *
FROM Deployment d
LEFT JOIN Deployment_Report r
ON d.FSJ_Deployment_ID = r.FSJ_Deployment_ID
AND r.Workflow_ID = #WorkflowID
WHERE d.FSJ_Deployment_ID = 339
The above in SQL works great and returns the full record if viewing an active workflow, or the left side of the record with empty fields for revision details which haven't been supplied in the event that a new report is being generated.
Using various samples around S.O. I've produced some Entity to SQL based on a few multiple on statements but I feel like I'm missing something fundamental to make this work:
int Workflow_ID = 399 // or -1 if new, just like the above example
from d in context.Deployments
join r in context.Deployment_Reports.DefaultIfEmpty()
on
new { d.Deployment_ID, Workflow_ID }
equals
new { r.Deployment_ID, r.Workflow_ID }
where d.FSJ_Deployment_ID == fsj_deployment_id
select new
{
...
}
Is the SQL query above possible to create using LINQ to Entities without employing Entity SQL? This is the first time I've needed to create such a join since it's very confusing to look at but in the report it's the only way to do it right since it should only return one record at all times.
The workflow ID is a value passed in to the call to retrieve the data source so in the outgoing query it would be considered a static value (for lack of better terminology on my part)
First of all don't kill yourself on learning the intricacies of EF as there are a LOT of things to learn about it. Unfortunately our deadlines don't like the learning curve!
Here's examples to learn over time:
http://msdn.microsoft.com/en-us/library/bb397895.aspx
In the mean time I've found this very nice workaround using EF for this kind of thing:
var query = "SELECT * Deployment d JOIN Deployment_Report r d.FSJ_Deployment_ID = r.Workflow_ID = #WorkflowID d.FSJ_Deployment_ID = 339"
var parm = new SqlParameter(parameterName="WorkFlowID" value = myvalue);
using (var db = new MyEntities()){
db.Database.SqlQuery<MyReturnType>(query, parm.ToArray());
}
All you have to do is create a model for what you want SQL to return and it will fill in all the values you want. The values you are after are all the fields that are returned by the "Select *"...
There's even a really cool way to get EF to help you. First find the table with the most fields, and get EF to generated the model for you. Then you can write another class that inherits from that class adding in the other fields you want. SQL is able to find all fields added regardless of class hierarchy. It makes your job simple.
Warning, make sure your filed names in the class are exactly the same (case sensitive) as those in the database. The goal is to make a super class model that contains all the fields of all the join activity. SQL just knows how to put them into that resultant class giving you strong typing ability and even more important use-ability with LINQ
You can even use dataannotations in the Super Class Model for displaying other names you prefer to the User, this is a super nice way to keep the table field names but show the user something more user friendly.

How to use a dynamic parameter in a IN clause of a JPA named query?

my problem is about this kind of query :
select * from SOMETABLE where SOMEFIELD in ('STRING1','STRING2');
the previous code works fine within Sql Developer.
The same static query also works fine and returns me a few results;
Query nativeQuery = em.createNativeQuery(thePreviousQuery,new someResultSet());
return nativeQuery.getResultList();
But when I try to parameterize this, I encounter a problem.
final String parameterizedQuery = "select * from SOMETABLE where SOMEFIELD in (?selectedValues)";
Query nativeQuery = em.createNativeQuery(parameterizedQuery ,new someResultSet());
nativeQuery.setParameter("selectedValues","'STRING1','STRING2'");
return nativeQuery.getResultList();
I got no result (but no error in console).
And when I look at the log, I see such a thing :
select * from SOMETABLE where SOMEFIELD in (?)
bind => [STRING1,STRING2]
I also tried to use no quotes (with similar result), or non ordered parameter (:selectedValues), which leads to such an error :
SQL Error: Missing IN or OUT parameter at index:: 1
I enventually tried to had the parentheses set directly in the parameter, instead of the query, but this didn't work either...
I could build my query at runtime, to match the first (working) case, but I'd rather do it the proper way; thus, if anyone has an idea, I'll read them with great interest!
FYI :
JPA version 1.0
Oracle 11G
JPA support the use of a collection as a list literal parameter only in JPQL queries, not in native queries. Some JPA providers support it as a proprietary feature, but it's not part of the JPA specification (see https://stackoverflow.com/a/3145275/1285097).
Named parameters in native queries also aren't part of the JPA specification. Their behavior depends on the persistence provider and/or the JDBC driver.
Hibernate with the JDBC driver for Oracle support both of these features.
List<String> selectedValues = Arrays.asList("STRING1", "STRING2");
final String parameterizedQuery = "select * from SOMETABLE where SOMEFIELD in (:selectedValues)";
return em.createNativeQuery(parameterizedQuery)
.setParameter("selectedValues", selectedValues)
.getResultList();
Instead of:
nativeQuery.setParameter("selectedValues", params);
I had to use:
nativeQuery.setParameterList("selectedValues", params);
This worked for me in derby. parameter without "()".
List<String> selectedValues = Arrays.asList("STRING1", "STRING2");
final String parameterizedQuery = "select * from SOMETABLE where SOMEFIELD in
:selectedValues";
return em.createNativeQuery(parameterizedQuery)
.setParameter("selectedValues", selectedValues)
.getResultList();
Replace this:
nativeQuery.setParameter("selectedValues","'STRING1','STRING2'");
with
List<String> params;
nativeQuery.setParameter("selectedValues",params);
I also faced the same issue.
This is what I did:
List<String> sample = new ArrayList<String>();
sample.add("sample1");
sample.add("sample2");
And now you, can set the sample in params.

How to change sql generated by linq-to-entities?

I am querying a MS SQL database using Linq and Entity Framework Code First. The requirement is to be able to run a WHERE SomeColumn LIKE '%sometext'clause against the table.
This, on the surface, is a simple requirement that could be accomplished using a simple Linq query like this:
var results = new List<MyTable>();
using(var context = new MyContext())
{
results = context.MyTableQueryable
.Where(x => x.SomeColumn.EndsWith("sometext"))
.ToList();
}
// use results
However, this was not effective in practice. The problem seems to be that the column SomeColumn is not varchar, rather it's a char(31). This means that if a string is saved in the column that is less than 31 characters then there will be spaces added on the end of the string to ensure a length of 31 characters, and that fouls up the .EndsWith() query.
I used SQL Profiler to lookup the exact sql that was generated from the .EndsWith() method. Here is what I found:
--previous query code removed for brevity
WHERE [Extent1].[SomeColumn] LIKE N'%sometext'
So that is interesting. I'm not sure what the N means before '%sometext'. (I'll Google it later.) But I do know that if I take the same query and run it in SSMS without the N like this:
--previous query code removed for brevity
WHERE [Extent1].[SomeColumn] LIKE '%sometext'
Then the query works fine. Is there a way to get Linq and Entity Framework to drop that N from the query?
Please try this...
.Where(x => x.SomeColumn.Trim().EndsWith("sometext"))
Just spoke to my colleague who had a similar issue, see if the following works for you:
[Column(TypeName = "varchar")]
public string SomeColumn
{
get;
set;
}
Apparently setting the type on the column mapping will force the query to recognise it as a VARCHAR, where a string is normally interpreted as an NVARCHAR.