Slick: Pass in column to update - sql

Let's say we have a FoodTable with the following columns: Name, Calories, Carbs, Protein. I have an entry for Name = Chocolate, Calories = 100, Carbs = "10g", and Protein = "2g".
I'm wondering if there's a way to pass in a column name and a new value to update with. For example, I want a method that's like
def updateFood(food, columnName, value):
table.filter(_.name === food).map(x => x.columnName).update(value)
It seems like dynamic columns are not possible with Slick? I want to avoid writing a SQL query because that could lead to security flaws or bugs in the code. Is there really no way to do this?
I also don't want to have to pass in the entire object to update, since ideally, it should be:
I want to update column X to value Y. I should only need to pass in the id of the object, the column, and the value to update to.

I'm wondering if there's a way to pass in a column name and a new value to update with
This depends a little bit on what you want the "column name" to be. To maintain safety, what I'd suggest is having the "column name" be a function that can select a column in your table.
At a high level that would look like this:
// Won't compile, but we'll fix that in a moment
def updateFood[V](food: Food, column: FoodTable => Rep[V], value: V): DBIO[Int] =
foods.filter(_.name === food.name).map(column).update(value)
...which we'd call like this:
updateFood(choc, _.calories, 99)
Notice how the "column name" is a function from FoodTable to a column of some value V. Then you provide a value for the V and we do a normal update.
The problem is that Slick knows how to map certain types of values (String, Int, etc) into SQL, but not any kind of value. And the code above won't compile because V is unconstrained.
We can sort of fix that my adding a constraint on V, and it mostly will work:
// Will compile, will work for basic types
def updateFood[V : slick.ast.BaseTypedType](food: Food, column: FoodTable => Rep[V], value: V): DBIO[Int] =
foods.filter(_.name === food.name).map(column).update(value)
However, if you have custom column mappings, they won't match the constraint. We need to go another step on and have an implicit shape in scope:
def updateFood[V](food: Food, column: FoodTable => Rep[V], value: V)(implicit shape: Shape[_ <: FlatShapeLevel, Rep[V], V, _]): DBIO[Int] =
foods.filter(_.name === food.name).map(column).update(value)
I think of Shape as an extra level of abstraction in Slick, above Rep[V]. The mechanisms of the "shape levels" and other details are not something I can explain because I don't understand them yet! (There is a talk that goes into the design of Slick called "Polymorphic Record Types in a Lifted Embedding" which you can find at http://slick.lightbend.com/docs/)
A final note: if you really want the column name to be a String or something like that, I'd suggest pattern matching the string (or validate in some way) to a FoodTable => Rep function and use that in your SQL. That's going to be tricky because your value V is going to have to match the type of the column you want to update.
Off the top of my head, that could look something like this:
def tryUpdateFood(food: Food, columnName: String, value: String): DBIO[Int] =
columnName match {
case "calories" => updateFood(food, _.calories, value.toInt)
case "carbs" => updateFood(food, _.carbs, value)
// etc...
case unknown => DBIO.failed(new Exception(s"Don't know how to update $unknown columns"))
}
I can imagine better error handling, safer or smarter parsing of the value, but in outline the above could work.
For hints at other ways to approach dynamic problems, take a look at the talk "Patterns for Slick database applications" (also listed at: http://slick.lightbend.com/docs/), and towards the end of the presentation there's a section on "Dynamic sorting".

Related

SQL Array aggregation in Haskell + Squeal

I'm using an SQL library called Squeal in Haskell.
What's the correct way to aggregate multiple text rows into one array in Squeal library?
Say I got a very simple Schema with just one table containing a column 'keyword' (of PG text type) + associated types:
import Squeal.PostgreSQL
import qualified GHC.Generics as GHC
import qualified Generics.SOP as SOP
type Constraints = '["pk_keywords" ::: 'PrimaryKey '["id"]]
type Columns
= '["id" ::: 'Def :=> 'NotNull 'PGint8, "keyword" ::: 'NoDef :=> 'NotNull 'PGtext]
type Table = 'Table (Constraints :=> KColumns)
type Schema = '["keywords" ::: Table]
type Schemas = '["public" ::: Schema]
newtype Keywords = Keywords {unKeywords :: [Text]} deriving (GHC.Generic)
instance SOP.Generic Keywords
instance SOP.HasDatatypeInfo Keywords
type instance PG Keywords = 'PGvararray ( 'NotNull 'PGtext)
This is the part I need help with:
I'm trying an aggregation query like this:
keywords :: Query_ Schemas () Keywords
keywords =
select_ ((arrayAgg (All #keyword)) `as` #fromOnly) (from (table #keywords))
However, I keep getting an error:
* Couldn't match type 'NotNull (PG [Text])
with 'Null ('PGvararray ty0)
arising from a use of `as'
From what I understand, arrayAgg can produce NULL so I need to provide a default of empty array [] somehow with fromNull from here:
https://hackage.haskell.org/package/squeal-postgresql-0.5.1.0/docs/Squeal-PostgreSQL-Expression-Null.html#v:fromNull
But I don't quite know how to provide that.
What about the value type mismatch (PG [Text] vs 'PGvararray ty0)? How to solve that?
For the record, the library's author provided a solution as follows:
keywords :: Query_ Schemas () (Only (VarArray [Text]))
keywords = select_
(fromNull (array [] & inferredtype) (arrayAgg (All #keyword)) `as` #fromOnly)
(from (table #keywords) & groupBy Nil)
The key factors here are:
Provide a default empty array with fromNull (array [] & inferredtype) .... This way we can avoid using Maybe in the return type
Provide grouping with groupBy Nil
Choose either Distinct or All rows in arrayAgg
Finally, the return type should be VarArray x

SQL Query to JSONiq Query

I want to convert an SQL query into a JSONiq Query, is there already an implementation for this, if not, what do I need to know to be able to create a program that can do this ?
I am not aware of an implementation, however, it is technically feasible and straightforward. JSONiq has 90% of its DNA coming from XQuery, which itself was partly designed by people involved in SQL as well.
From a data model perspective, a table is mapped to a collection and each row of the table is mapped to a flat JSON object, i.e., all fields are atomic values, like so:
{
"Name" : "Turing",
"First" : "Alan",
"Job" : "Inventor"
}
Then, the mapping is done by converting SELECT-FROM-WHERE queries to FLWOR expressions, which provide a superset of SQL's functionality.
For example:
SELECT Name, First
FROM people
WHERE Job = "Inventor"
Can be mapped to:
for $person in collection("people")
where $person.job eq "Inventor"
return project($person, ("Name", "First"))
More complicated queries can also be mapped quite straight-forwardly:
SELECT Name, COUNT(*)
FROM people
WHERE Job = "Inventor"
GROUP BY Name
HAVING COUNT(*) >= 2
to:
for $person in collection("people")
where $person.job eq "Inventor"
group by $name := $person.name
where count($person) ge 2
return {
name: $name,
count: count($person)
}
Actually, if for had been called from and return had been called select, and if these keywords were written uppercase, the syntax of JSONiq would be very similar to that of SQL: it's only cosmetics.

doctrine native sql not accepting parameter list

I'm trying to do native SQL in Doctrine. Basically I have 2 parameters:
CANDIDATE_ID - user for who we delete entries,
list of FILE_ID to keep
So I make
$this->getEntityManager()->getConnection()->
executeUpdate( "DELETE FROM FILE WHERE CANDIDATE_ID = :ID AND NOT ID IN :KEEPID",
array(
"ID" => $candidate->id,
"KEEPID" => array(2) )
);
But Doctrine fails:
Notice: Array to string conversion in D:\xampp\htdocs\azk\vendor\doctrine\dbal\lib\Doctrine\DBAL\Connection.php on line 786
Is this bug in Doctrine? I'm making somewhere else select with IN but with QueryBuilder and it's working. Maybe someone could suggest better way of deleting entries, with QueryBuilder for example?
$stmt = $conn->executeQuery('SELECT * FROM articles WHERE id IN (?)',
array(array(1, 2, 3, 4, 5, 6)),
array(\Doctrine\DBAL\Connection::PARAM_INT_ARRAY)
);
From Doctrine's documentation.
You can't pass an array of IDs to a parameter. You can do this for scalar values, but even if this had a 'toString', it wouldn't be what you want.
String concatenation is one method,
"DELETE FROM FILE WHERE CANDIDATE_ID = :ID AND NOT ID IN (". implode(",", $list_of_ids) .")"
But this method goes straight around parameters, and therefore suffers in terms of readability, and is limited to a certain maximum line length, which can vary between databases.
Another approach is to write a function returning a table result, which takes a string of IDs as a parameter.
You could also solve this with a join to a table containing the IDs to keep.
It's a problem I've seen many times with few good answers, but it's usually caused by a misunderstanding in the way the database is modelled. This is a 'code smell' for database access.

NHibernate Like with integer

I have a NHibernate search function where I receive integers and want to return results where at least the beginning coincides with the integers, e.g.
received integer: 729
returns: 729445, 7291 etc.
The database column is of type int, as is the property "Id" of Foo.
But
int id = 729;
var criteria = session.CreateCriteria(typeof(Foo))
criteria.Add(NHibernate.Criterion.Expression.InsensitiveLike("Id", id.ToString() + "%"));
return criteria.List<Foo>();
does result in an error (Could not convert parameter string to int32). Is there something wrong in the code, a work around, or other solution?
How about this:
int id = 729;
var criteria = session.CreateCriteria(typeof(Foo))
criteria.Add(Expression.Like(Projections.Cast(NHibernateUtil.String, Projections.Property("Id")), id.ToString(), MatchMode.Anywhere));
return criteria.List<Foo>();
Have you tried something like this:
int id = 729;
var criteria = session.CreateCriteria(typeof(Foo))
criteria.Add(NHibernate.Criterion.Expression.Like(Projections.SqlFunction("to_char", NHibernate.NHibernateUtil.String, Projections.Property("Id")), id.ToString() + "%"));
return criteria.List<Foo>();
The idea is convert the column before using a to_char function. Some databases do this automatically.
AFAIK, you'll need to store your integer as a string in the database if you want to use the built in NHibernate functionality for this (I would recommend this approach even without NHibernate - the minute you start doing 'like' searches you are dealing with a string, not a number - think US Zip Codes, etc...).
You could also do it mathematically in a database-specific function (or convert to a string as described in Thiago Azevedo's answer), but I imagine these options would be significantly slower, and also have potential to tie you to a specific database.

How to specify multiple values in where with AR query interface in rails3

Per section 2.2 of rails guide on Active Record query interface here:
which seems to indicate that I can pass a string specifying the condition(s), then an array of values that should be substituted at some point while the arel is being built. So I've got a statement that generates my conditions string, which can be a varying number of attributes chained together with either AND or OR between them, and I pass in an array as the second arg to the where method, and I get:
ActiveRecord::PreparedStatementInvalid: wrong number of bind variables (1 for 5)
which leads me to believe I'm doing this incorrectly. However, I'm not finding anything on how to do it correctly. To restate the problem another way, I need to pass in a string to the where method such as "table.attribute = ? AND table.attribute1 = ? OR table.attribute1 = ?" with an unknown number of these conditions anded or ored together, and then pass something, what I thought would be an array as the second argument that would be used to substitute the values in the first argument conditions string. Is this the correct approach, or, I'm just missing some other huge concept somewhere and I'm coming at this all wrong? I'd think that somehow, this has to be possible, short of just generating a raw sql string.
This is actually pretty simple:
Model.where(attribute: [value1,value2])
Sounds like you're doing something like this:
Model.where("attribute = ? OR attribute2 = ?", [value, value])
Whereas you need to do this:
# notice the lack of an array as the last argument
Model.where("attribute = ? OR attribute2 = ?", value, value)
Have a look at http://guides.rubyonrails.org/active_record_querying.html#array-conditions for more details on how this works.
Instead of passing the same parameter multiple times to where() like this
User.where(
"first_name like ? or last_name like ? or city like ?",
"%#{search}%", "%#{search}%", "%#{search}%"
)
you can easily provide a hash
User.where(
"first_name like :search or last_name like :search or city like :search",
{search: "%#{search}%"}
)
that makes your query much more readable for long argument lists.
Sounds like you're doing something like this:
Model.where("attribute = ? OR attribute2 = ?", [value, value])
Whereas you need to do this:
#notice the lack of an array as the last argument
Model.where("attribute = ? OR attribute2 = ?", value, value) Have a
look at
http://guides.rubyonrails.org/active_record_querying.html#array-conditions
for more details on how this works.
Was really close. You can turn an array into a list of arguments with *my_list.
Model.where("id = ? OR id = ?", *["1", "2"])
OR
params = ["1", "2"]
Model.where("id = ? OR id = ?", *params)
Should work
If you want to chain together an open-ended list of conditions (attribute names and values), I would suggest using an arel table.
It's a bit hard to give specifics since your question is so vague, so I'll just explain how to do this for a simple case of a Post model and a few attributes, say title, summary, and user_id (i.e. a user has_many posts).
First, get the arel table for the model:
table = Post.arel_table
Then, start building your predicate (which you will eventually use to create an SQL query):
relation = table[:title].eq("Foo")
relation = relation.or(table[:summary].eq("A post about foo"))
relation = relation.and(table[:user_id].eq(5))
Here, table[:title], table[:summary] and table[:user_id] are representations of columns in the posts table. When you call table[:title].eq("Foo"), you are creating a predicate, roughly equivalent to a find condition (get all rows whose title column equals "Foo"). These predicates can be chained together with and and or.
When your aggregate predicate is ready, you can get the result with:
Post.where(relation)
which will generate the SQL:
SELECT "posts".* FROM "posts"
WHERE (("posts"."title" = "Foo" OR "posts"."summary" = "A post about foo")
AND "posts"."user_id" = 5)
This will get you all posts that have either the title "Foo" or the summary "A post about foo", and which belong to a user with id 5.
Notice the way arel predicates can be endlessly chained together to create more and more complex queries. This means that if you have (say) a hash of attribute/value pairs, and some way of knowing whether to use AND or OR on each of them, you can loop through them one by one and build up your condition:
relation = table[:title].eq("Foo")
hash.each do |attr, value|
relation = relation.and(table[attr].eq(value))
# or relation = relation.or(table[attr].eq(value)) for an OR predicate
end
Post.where(relation)
Aside from the ease of chaining conditions, another advantage of arel tables is that they are independent of database, so you don't have to worry whether your MySQL query will work in PostgreSQL, etc.
Here's a Railscast with more on arel: http://railscasts.com/episodes/215-advanced-queries-in-rails-3?view=asciicast
Hope that helps.
You can use a hash rather than a string. Build up a hash with however many conditions and corresponding values you are going to have and put it into the first argument of the where method.
WRONG
This is what I used to do for some reason.
keys = params[:search].split(',').map!(&:downcase)
# keys are now ['brooklyn', 'queens']
query = 'lower(city) LIKE ?'
if keys.size > 1
# I need something like this depending on number of keys
# 'lower(city) LIKE ? OR lower(city) LIKE ? OR lower(city) LIKE ?'
query_array = []
keys.size.times { query_array << query }
#['lower(city) LIKE ?','lower(city) LIKE ?']
query = query_array.join(' OR ')
# which gives me 'lower(city) LIKE ? OR lower(city) LIKE ?'
end
# now I can query my model
# if keys size is one then keys are just 'brooklyn',
# in this case it is 'brooklyn', 'queens'
# #posts = Post.where('lower(city) LIKE ? OR lower(city) LIKE ?','brooklyn', 'queens' )
#posts = Post.where(query, *keys )
now however - yes - it's very simple. as nfriend21 mentioned
Model.where(attribute: [value1,value2])
does the same thing