My goal in writing a function is to allow callers to pass in the same condition arguments they would to a where call in ActiveRecord, and I want the corresponding Rails-generated SQL.
Example
If my function receives a hash like this as an argument
role: 'Admin', id: [4, 8, 15]
I would expect to generate this string
"users"."role" = 'Admin' AND "users"."id" IN (4, 8, 15)
Possible Solutions
I get the closest with to_sql.
pry(main)> User.where(role: 'Admin', id: [4, 8, 15])
=> "SELECT \"users\".* FROM \"users\" WHERE \"users\".\"role\" = 'Admin' AND \"users\".\"id\" IN (4, 8, 15)"
It returns almost exactly what I want; however, I would be more comfortable not stripping away the SELECT ... WHERE myself in case something changes in the way the SQL is generated. I realize the WHERE should always be there to split on, but I'd prefer an even less brittle approach.
My next approach was using Arel's where_sql function.
pry(main)> User.where(role: 'Admin', id: [4, 8, 15]).arel.where_sql
=> "WHERE \"users\".\"role\" = $1 AND \"users\".\"id\" IN (4, 8, 15)"
It gets rid of the SELECT but leaves the WHERE. I would prefer it to the above if it had already injected the sanitized role, but that renders it quite a bit less desirable.
I've also considered generating the SQL myself, but I would prefer to avoid that.
Do any of you know if there's some method right under my nose I just haven't found yet? Or is there a better way of doing this altogether?
Ruby 2.3.7
Rails 5.1.4
I too would like to know how to get the conditions without the leading WHERE. I see in https://coderwall.com/p/lsdnsw/chain-rails-scopes-with-or that they used string manipulation to get rid of the WHERE, which seems messy but maybe the only solution currently. :/
scope.arel.where_sql.gsub(/\AWHERE /i, "")
Related
I'm facing a strange problem maybe related with some cache that I cannot find.
I have the following Models:
class Incubadores(models.Model):
incubador = models.CharField(max_length=10, primary_key=True)
posicion = models.CharField(max_length=10)
class Tareas(TimeStampedModel):
priority = models.CharField(max_length=20, choices=PRIORITIES, default='normal')
incubador = models.ForeignKey(Incubadores, on_delete=models.CASCADE, null=True, db_column='incubador')
info = JSONField(null=True)
datos = JSONField(null=True)
class Meta:
ordering = ('priority','modified','created')
I previously didn't have the argument db_column, so the Postgres column for that field was incubador_id
I used the argument db_column to change the name of the column, and then I run python manage.py makemgrations and python manage.py migrate, but I'm still getting the column as incubadores_id whenever I perform a query such as:
>>> tareas = Tareas.objects.all().values()
>>> print(tareas)
<QuerySet [{'info': None, 'modified': datetime.datetime(2019, 11, 1, 15, 24, 58, 743803, tzinfo=<UTC>), 'created': datetime.datetime(2019, 11, 1, 15, 24, 58, 743803, tzinfo=<UTC>), 'datos': None, 'priority': 'normal', 'incubador_id': 'I1.1', 'id': 24}, {'info': None, 'modified': datetime.datetime(2019, 11, 1, 15, 25, 25, 49950, tzinfo=<UTC>), 'created': datetime.datetime(2019, 11, 1, 15, 25, 25, 49950, tzinfo=<UTC>), 'datos': None, 'priority': 'normal', 'incubador_id': 'I1.1', 'id': 25}]>
I need to modify this column name because I'm having other issues with Serializers. So the change is necessary.
If I perform the same query in other Models where I've also changed the name of the default field. The problem is exactly the same.
It happens both on the shell and on the code.
I've tried with different queries, to make sure it's not related to Django lazy query system, but the problem is the same. I've also tried executing django.db.connection.close().
If I do a direct SQL query to PostgreSQL, it cannot find incubador_id, but only incubador, which is correct.
Anyone has any idea of what can be happening? I've already been 2 days with this problem and I cannot find a reason :( It's a very basic operation.
Thanks!
This answer will explain why this is happening.
Django's built-in serializers don't have this issue, but probably won't yield exactly what you're looking for:
>>> from django.core import serializers
>>> serializers.serialize("json", Tareas.objects.all())
'[{"model": "inc.tareas", "pk": 1, "fields": {"priority": "normal", "incubador": "test-i"}}]'
You could use the fields attribute here, which seems like it would give you what you're looking for.
You don't specify what your "other issues with Serializers" are, but my suggestion would be to write custom serialization code. Relying on something like .values() or even serializers.serialize() is a bit too implicit for me; writing explicit serialization code makes it less likely you'll accidentally break a contract with a consumer of your serialized data if this model changes.
Note: Please try to make the example you provide minimal and reproducible. I removed some fields to make this work with stock Django, which is why the serialized value is missing fields; the _id issue was still present without the third-party apps you're using, and was resolved with serializers. This also isn't specific to PG; it happens in sqlite as well.
I'm using #Query from the spring data package and I want to query on the last element of an array in a document.
For example the data structure could be like this:
{
name : 'John',
scores: [10, 12, 14, 16]
},
{
name : 'Mary',
scores: [78, 20, 14]
},
So I've built a query, however it is complaining that "error message 'unknown operator: $slice' on server"
The $slice part of the query, when run separately, is fine:
db.getCollection('users').find({}, {scores: { $slice: -1 })
However as soon as I combine it with a more complex check, it gives the error as mentioned.
db.getCollection('users').find{{"$and":[{ } , {"scores" : { "$slice" : -1}} ,{"scores": "16"}]})
This query would return the list of users who had a last score of 16, in my example John would be returned but not Mary.
I've put it into a standard mongo query (to debug things), however ideally I need it to go into a spring-data #query construct - they should be fairly similar.
Is there anyway of doing this, without resorting to hand-cranked java calls? I don't see much documentation for #Query, other than it takes standard queries.
As commented with the link post, that refers to aggregate, how does that work with #Query, plus one of the main answers uses $where, this inefficient.
The general way forward with the problem is unfortunately the data, although #Veeram's response is correct, it will mean that you do not hit indexes. This is an issue where you've got very large data sets of course and you will see ever decreasing return times. It's something $where, $arrayElemAt cannot help you with. They have to pre-process the data and that means a full collection scan. We analysed several queries with these constructs and they involved a "COLSCAN".
The solution is ideally to create a field that contains the last item, for instance:
{
name : 'John',
scores: [10, 12, 14, 16],
lastScore: 16
},
{
name : 'Mary',
scores: [78, 20, 14],
lastScore: 14
}
You could create a listener to maintain this as follows:
#Component
public class ScoreListener extends AbstractMongoEventListener<Scores>
You then get the ability to sniff the data and make any updates:
#Override
public void onBeforeConvert(BeforeConvertEvent<Scores> event) {
// process any score and set lastScore
}
Don't forget to update your indexes (!):
#CompoundIndex(name = "lastScore", def = "{"
+ "'lastScore': 1"
+ " }")
Although this does contain a disadvantage of a slight duplication of data, in current Mongo (3.4) this really is the only way of doing this AND to include indexes in the search mechanism. The speed differences were dramatic, from nearly a minute response time down to milliseconds.
In Mongo 3.6 there may be better ways for doing that, however we are fixed on this version, so this has to be our solution.
I need to have an ActiveRecord Postgres query that returns results which match all the parameters passed in through an array.
Some background: I have a User model, which has many Topics (through Specialties). I'm passing in the Topic ids as a string (Parameters: {"topics"=>"1,8,3"}) and then turning them into an array with .split(',') so I end up with topic_params = ["1","8","3"].
Now I'm trying to return all Users who have Topics that match/include all of those. After following the answer in this question, I managed to return Users who match ANY of the Topics with this:
#users = User.includes(:topics, :organization).where(:topics => {:id => topic_params})
But I need it to return results that match ALL. I'd also be open to better ways to accomplish this sort of task overall.
One way would be something like this
User.joins(:topics).where(topics: { id: [1, 2, 3] }).group('users.id').having('count(distinct topics.id) = 3')
Obviously I haven't your exact schema so you might have to tweak it a bit, but this is the basic setup.
Important is that the having clause counter must match the number of items you're matching with.
I am new to rails. What I see that there are a lot of ways to find a record:
find_by_<columnname>(<columnvalue>)
find(:first, :conditions => { <columnname> => <columnvalue> }
where(<columnname> => <columnvalue>).first
And it looks like all of them end up generating exactly the same SQL. Also, I believe the same is true for finding multiple records:
find_all_by_<columnname>(<columnvalue>)
find(:all, :conditions => { <columnname> => <columnvalue> }
where(<columnname> => <columnvalue>)
Is there a rule of thumb or recommendation on which one to use?
where returns ActiveRecord::Relation
Now take a look at find_by implementation:
def find_by
where(*args).take
end
As you can see find_by is the same as where but it returns only one record. This method should be used for getting 1 record and where should be used for getting all records with some conditions.
Edit:
This answer is very old and other, better answers have come up since this post was made. I'd advise looking at the one posted below by #Hossam Khamis for more details.
Use whichever one you feel suits your needs best.
The find method is usually used to retrieve a row by ID:
Model.find(1)
It's worth noting that find will throw an exception if the item is not found by the attribute that you supply. Use where (as described below, which will return an empty array if the attribute is not found) to avoid an exception being thrown.
Other uses of find are usually replaced with things like this:
Model.all
Model.first
find_by is used as a helper when you're searching for information within a column, and it maps to such with naming conventions. For instance, if you have a column named name in your database, you'd use the following syntax:
Model.find_by(name: "Bob")
.where is more of a catch all that lets you use a bit more complex logic for when the conventional helpers won't do, and it returns an array of items that match your conditions (or an empty array otherwise).
Model.find
1- Parameter: ID of the object to find.
2- If found: It returns the object (One object only).
3- If not found: raises an ActiveRecord::RecordNotFound exception.
Model.find_by
1- Parameter: key/value
Example:
User.find_by name: 'John', email: 'john#doe.com'
2- If found: It returns the object.
3- If not found: returns nil.
Note: If you want it to raise ActiveRecord::RecordNotFound use find_by!
Model.where
1- Parameter: same as find_by
2- If found: It returns ActiveRecord::Relation containing one or more records matching the parameters.
3- If not found: It return an Empty ActiveRecord::Relation.
There is a difference between find and find_by in that find will return an error if not found, whereas find_by will return null.
Sometimes it is easier to read if you have a method like find_by email: "haha", as opposed to .where(email: some_params).first.
Since Rails 4 you can do:
User.find_by(name: 'Bob')
which is the equivalent find_by_name in Rails 3.
Use #where when #find and #find_by are not enough.
The accepted answer generally covers it all, but I'd like to add something,
just incase you are planning to work with the model in a way like updating, and you are retrieving a single record(whose id you do not know), Then find_by is the way to go, because it retrieves the record and does not put it in an array
irb(main):037:0> #kit = Kit.find_by(number: "3456")
Kit Load (0.9ms) SELECT "kits".* FROM "kits" WHERE "kits"."number" =
'3456' LIMIT 1
=> #<Kit id: 1, number: "3456", created_at: "2015-05-12 06:10:56",
updated_at: "2015-05-12 06:10:56", job_id: nil>
irb(main):038:0> #kit.update(job_id: 2)
(0.2ms) BEGIN Kit Exists (0.4ms) SELECT 1 AS one FROM "kits" WHERE
("kits"."number" = '3456' AND "kits"."id" != 1) LIMIT 1 SQL (0.5ms)
UPDATE "kits" SET "job_id" = $1, "updated_at" = $2 WHERE "kits"."id" =
1 [["job_id", 2], ["updated_at", Tue, 12 May 2015 07:16:58 UTC +00:00]]
(0.6ms) COMMIT => true
but if you use where then you can not update it directly
irb(main):039:0> #kit = Kit.where(number: "3456")
Kit Load (1.2ms) SELECT "kits".* FROM "kits" WHERE "kits"."number" =
'3456' => #<ActiveRecord::Relation [#<Kit id: 1, number: "3456",
created_at: "2015-05-12 06:10:56", updated_at: "2015-05-12 07:16:58",
job_id: 2>]>
irb(main):040:0> #kit.update(job_id: 3)
ArgumentError: wrong number of arguments (1 for 2)
in such a case you would have to specify it like this
irb(main):043:0> #kit[0].update(job_id: 3)
(0.2ms) BEGIN Kit Exists (0.6ms) SELECT 1 AS one FROM "kits" WHERE
("kits"."number" = '3456' AND "kits"."id" != 1) LIMIT 1 SQL (0.6ms)
UPDATE "kits" SET "job_id" = $1, "updated_at" = $2 WHERE "kits"."id" = 1
[["job_id", 3], ["updated_at", Tue, 12 May 2015 07:28:04 UTC +00:00]]
(0.5ms) COMMIT => true
Apart from accepted answer, following is also valid
Model.find() can accept array of ids, and will return all records which matches.
Model.find_by_id(123) also accept array but will only process first id value present in array
Model.find([1,2,3])
Model.find_by_id([1,2,3])
The answers given so far are all OK.
However, one interesting difference is that Model.find searches by id; if found, it returns a Model object (just a single record) but throws an ActiveRecord::RecordNotFound otherwise.
Model.find_by is very similar to Model.find and lets you search any column or group of columns in your database but it returns nil if no record matches the search.
Model.where on the other hand returns a Model::ActiveRecord_Relation object which is just like an array containing all the records that match the search. If no record was found, it returns an empty Model::ActiveRecord_Relation object.
I hope these would help you in deciding which to use at any point in time.
Suppose I have a model User
User.find(id)
Returns a row where primary key = id. The return type will be User object.
User.find_by(email:"abc#xyz.com")
Returns first row with matching attribute or email in this case. Return type will be User object again.
Note :- User.find_by(email: "abc#xyz.com") is similar to User.find_by_email("abc#xyz.com")
User.where(project_id:1)
Returns all users in users table where attribute matches.
Here return type will be ActiveRecord::Relation object. ActiveRecord::Relation class includes Ruby's Enumerable module so you can use it's object like an array and traverse on it.
Both #2s in your lists are being deprecated. You can still use find(params[:id]) though.
Generally, where() works in most situations.
Here's a great post: https://web.archive.org/web/20150206131559/http://m.onkey.org/active-record-query-interface
The best part of working with any open source technology is that you can inspect length and breadth of it.
Checkout this link
find_by ~> Finds the first record matching the specified conditions. There is no implied ordering so if order matters, you should specify it yourself. If no record is found, returns nil.
find ~> Finds the first record matching the specified conditions , but if no record is found, it raises an exception but that is done deliberately.
Do checkout the above link, it has all the explanation and use cases for the following two functions.
I will personally recommend using
where(< columnname> => < columnvalue>)
I have this scope in Rails:
scope :by_default_checks, {:conditions => ['cars.sold IS ? AND cars.deactivated IS ?', nil, false]}
#cars = Car.by_title(params[:search][:title_like]).by_greater(params[:search][:amount_gte]).by_smaller(params[:search][:amount_lte]).by_default_checks
and on Heroku I am getting this error:
Completed 500 Internal Server Error in 6ms
... AND cars.sold IS NULL AND cars.deactivated IS 'f')
SELECT "cars".* FROM "cars" WHERE (cars.title LIKE '%iphoe%') AND (cars.sold IS NULL AND cars.deactivated IS 'f')
PG::SyntaxError: ERROR: syntax error at or near "'f'"
This code is working on SQLite, but doesn't on PostgreSQL. How to replace it?
Thanks
You should use = to check for equality with non-null values:
['cars.sold IS ? AND cars.deactivated = ?', nil, false]
You usually use is in is null, is not null, is distinct from, and is not distinct from when you're faced with NULLs and a simple = comparison will not work the way you want it to. You can use is for booleans if you're using the true or false values but not the 't' and 'f' strings that ActiveRecord uses to represent PostgreSQL booleans.
See Comparison Operators in the manual for details.
Alternatively, you could let ActiveRecord build the whole thing instead of using the old-school :conditions stuff:
scope :by_default_checks, where(:sold => nil).where(:deactivated => false)
That way ActiveRecord is responsible for all the native-to-PostgreSQL type conversions and it will choose the correct comparison operators for you.
Also, developing on top of one database (SQLite) and deploying on another (PostgreSQL) is a really bad idea that will just lead to pain, suffering, and hair loss. There are all sorts of differences between databases that no ORM can insulate you from. Please fix this bug and then immediately switch your development environment to PostgreSQL.