Trying to get records that were created this year, I stumbled upon this great question. The second answer says you get all records from a model that were created today by saying:
Model.where("created_at >= ?", Time.now.beginning_of_day)
So, naturally, I tried the same thing with Time.now.beginning_of_year, and it works just fine.
However, what struck me as interesting is that the outputted query (I tried it in the console) is
SELECT COUNT(*) FROM `invoices` WHERE (created_at >= '2012-12-31 23:00:00')
I wasn't aware that 2013 already began at 2012-12-31 23:00:00? How's that?
If you haven't set it yet, you should set your timezone in the config/application.rb file. Look for the line that begins with config.time_zone. (If you aren't sure what value to give, you can run rake time:zones:all to get a list of all available timezones.)
Once you've set your timezone, you should use Time.zone.now, as opposed to Time.now. This will properly "scope" your times to your timezone.
Check the API for more details on TimeWithZone.
Related
I can mock Time.now with a great timecop gem.
Time.now
=> 2018-05-13 18:04:46 +0300
Timecop.travel(Time.parse('2018.03.12, 12:00'))
Time.now
=> 2018-03-12 12:00:04 +0300
TeacherVacation.first.ends_at
Thu, 15 Mar 2018 12:00:00 MSK +03:00
TeacherVacation.where('ends_at > ?', Time.now).count
1
But (obviously) this wouldn't work while using NOW() in a query:
TeacherVacation.where('ends_at > NOW()').count
0
Can I mock NOW() so that it would return the results for a certain time?
Timecop is a great gem! I would recommend using Timecop.freeze instead of traveling for your instance; you want to keep your tests deterministic.
As far as I could find, there doesn't seem to be a way to mock SQL's functions. Some languages like Postgres allow overloading functions, but you would still need a way to interject, and there doesn't seem to be a way to use environment variables in SQL.
A co-worker seemed to be certain you could actually drop system/language functions and make your own, but I was concerned about how to recover them after you do that. Trying to go that route sounds like a pain.
Solutions?
Here are a couple of "solutions" that I've come up with today while fighting this problem. Note: I don't really care for them to be honest, but if it gets tests in place ¯\_(ツ)_/¯ They at least offer a way to get things "working".
Unfortunately there's no snazzy gem to control the time in SQL. I imagine you would need something crazy like a plugin to the DB, a hack, a hook, a man in the middle, a container that you could trick SQL into thinking the system time was something else. None of those hack ideas would surely be portable/platform agnostic unfortunately either.
Apparently there are some ways to set time in a docker container, but that sounds like a painful overhead for local testing, and doesn't fit the granularity of a per-test time to be set.
Another thing to note, for me we're running large complex raw SQL queries, so that's why it's important that when I run the SQL file for a test I can have proper dates, otherwise I would just be doing it through activerecord like you mentioned.
String Interpolation
I ran across this in some large queries that were being ran.
This definitely helps if you need to push some environment variables through, and you can inject your own "current_date" if you want. This would help too if you needed to utilize a certain time across multiple queries.
my_query.rb
<<~HEREDOC
SELECT *
FROM #{#prefix}.my_table
WHERE date < #{#current_date} - INTERVAL '5 DAYS'
HEREDOC
sql_runner.rb
class SqlRunner
def initialize(file_path)
#file_path = file_path
#prefix = ENV['table_prefix']
#current_date = Date.today
end
def run
execute(eval(File.read #file_path))
end
private
def execute(sql)
ActiveRecord::Base.connection.execute(sql)
end
end
The Dirty Update
The idea is to update the value from ruby land pushing your "time-copped" time into the database to overwrite the value generated by the SQL DB. You may need to get creative with your update for times, like querying for a time greater than a given time that doesn't target your timecop time that you'll be updating rows to.
The reason I don't care for this method is because it ends up feeling like you're just testing activerecord's functionality since you're not relying on the DB to set values it should be setting. You may have computations in your SQL that you're then recreating in the test to set some value to the right date, and then you're no longer doing the computation in the SQL so then you're not even actually testing it.
large_insert.sql
INSERT INTO some_table (
name,
created_on
)
SELECT
name,
current_date
FROM projects
JOIN people ON projects.id = people.project_id
insert_spec.rb
describe 'insert_test.sql' do
ACTUAL_DATE = Date.today
LARGE_INSERT_SQL = File.read('sql/large_insert.sql')
before do
Timecop.freeze Date.new(2018, 10, 28)
end
after do
Timecop.return
end
context 'populated same_table' do
before do
execute(LARGE_INSERT_SQL)
mock_current_dates(ACTUAL_DATE)
end
it 'has the right date' do
expect(SomeTable.last.created_on).to eq(Date.parse('2018.10.28')
end
end
def execute(sql_command)
ActiveRecord::Base.connection.execute(sql_command)
end
def mock_current_dates(actual_date)
rows = SomeTable.where(created_on: actual_date)
# Use our timecop datetime
rows.update_all(created_on: Date.today)
end
Fun Caveat: specs wrap in their own transactions (you can turn that off, but it's a nice feature) so if your SQL has a transaction in it, you'll need to write code to remove it for the specs, or have your runner wrap your code in transactions if you need them. They'll run but then your SQL will kill off the spec transaction and you'll have a bad time. You can create a spec/support to help out with this if you go the route of cleaning up during tests, if I were in a newer project I would go with writing a runner that wraps the queries in transactions if you need them -- even though this isn't evident in the SQL files #abstraction.
Maybe there's something out there that lets you set your system time, but that sounds terrifying modifying your system's actual time.
I think the solution for this is DI (dependency injection)
def NOW(time = Time.now)
time
end
In test
current_test = Time.new(2018, 5, 13)
p NOW(current_test)
In production
p NOW
I've spent several hours going through StackOverflow and playing around with this query, but still can't get it to work! Hopefully an expert here on SO can make the pain go away...
I have two models, Event and EventInstance. An Event has_many EventInstances.
What I want to do is easily get a list of Events (not EventInstances), where:
Events are distinct and not repeated
Events are sorted by the start_date of the nearest EventInstance
Event instances have the attribute :active => true
Only event instances that have a start date in the future are returned
I currently have the query
Event.joins(:event_instances).select('distinct events.*').where('event_instances.start_date >= ?', Time.now).where('event_instances.active = true')
This returns a list of events, but not sorted by date. Excellent - so I am almost there!
If I change the query to add this on the end:
.order('event_instances.start_date')
I get the error:
PG::InvalidColumnReference: ERROR: for SELECT DISTINCT, ORDER BY expressions must appear in select list
So I moved it to the select statement:
select('distinct event_instances.start_date, events.*')
Now I get
PG::UndefinedFunction: ERROR: function count(date, events) does not exist
I've tried moving methods around, using includes, everything but I still can't get it to work. Any help would be really appreciated! Thank you.
try changing
.order('event_instances.start_date')
to
.order(:event_instances.start_date)
or if you need descending order add the .reverse_order method to the end of the query
This is the exact query which worked for my models Post and PostComments on both MySQL and PostgreSQL:
Post.joins(:post_comments).select('distinct post_comments.body, post_comments.created_at').order('post_comments.created_at desc')
So for you, it's equivalent should work too. If it still doesn't then please update your post with the fields of your model.
I am trying to save a view in BigQuery, and keep getting the same error:
Failed to create view. Unexpected. Please try again.
The query is as follows:
SELECT
interaction.id AS Interaction.ID,
interaction.author.name AS Interaction.Author.Name,
interaction.author.username AS Interaction.Author.Username,
interaction.content AS Interaction.Content,
interaction.created_at_timestamp AS Interaction.Created_At_Timestamp,
klout.score AS Klout.Score,
twitter.geo.latitude AS Twitter.Geo.Latitude,
twitter.geo.longitude AS Twitter.Geo.Longitude,
twitter.media.expanded_url AS Twitter.Media.ExpandedUrl,
twitter.media.type AS Twitter.Media.Type,
twitter.place.country AS Twitter.Place.Country,
twitter.user.followers_count AS Twitter.User.Followers,
twitter.user.friends_count AS Twitter.User.Friends,
twitter.user.listed_count AS Twitter.User.Listed,
twitter.retweet.count AS Twitter.Retweet.Count
FROM
[**DATASET_NAME_OMITTED**.main_table]
WHERE
(interaction.id IS NOT NULL)
AND (interaction.created_at_timestamp IS NOT NULL)
AND (interaction.created_at_timestamp >= DATE_ADD(USEC_TO_TIMESTAMP(UTC_USEC_TO_HOUR(NOW())), -1, "DAY"))
AND (interaction.created_at_timestamp < USEC_TO_TIMESTAMP(UTC_USEC_TO_HOUR(NOW())))
The query validates, and runs without any problems:
Valid: This query will process 203 MB when run.
I did notice that the twitter.media is of type REPEATED RECORD. That said, removing twitter.media.* fields does not fix the issue.
I have been able to successfully save other views with the same timestamp restrictions and naming conventions. Attempting to save this one consistently fails.
For context: This table is populated by DataSift via their BigQuery connector (default, catch-all schema).
This is really weird.
I ran an experiment and pulled out each of the alias operations, and it worked.
I then slowly added some of them back in, and again; it continued working. However it seems that certain aliases do not want to work (I have no idea why).
I ended up with the following, which contains most of your aliases, and seems to work as expected:
SELECT
interaction.id AS Interaction.ID,
interaction.author.name AS Interaction.Author.Name,
interaction.author.username AS Interaction.Author.Username,
interaction.content AS Interaction.Content,
interaction.created_at_timestamp AS Interaction.Created_At_Timestamp,
klout.score AS Klout.Score,
twitter.geo.latitude AS Twitter.Geo.Latitude,
twitter.geo.longitude AS Twitter.Geo.Longitude,
twitter.media.expanded_url,
twitter.media.type AS Twitter.Media.Type,
twitter.place.country AS Twitter.Place.Country,
twitter.user.followers_count,
twitter.user.friends_count,
twitter.user.listed_count,
twitter.retweet.count AS Twitter.Retweet.Count
FROM [**DATASET_NAME_OMITTED**.main_table]
WHERE
(interaction.id IS NOT NULL)
AND (interaction.created_at_timestamp IS NOT NULL)
AND (interaction.created_at_timestamp >= DATE_ADD(USEC_TO_TIMESTAMP(UTC_USEC_TO_HOUR(NOW())), -1, "DAY"))
AND (interaction.created_at_timestamp < USEC_TO_TIMESTAMP(UTC_USEC_TO_HOUR(NOW())))
What seems really weird is that there is no pattern between what will, and will not work. The twitter.user.* fields are integers, but will not accept aliases, however the integer field klout.score field does accept an integer.
rake aborted!
unknown attribute: date
I am finally able to migrate my database content but encounter this unfortunate error.
Is it because the date attributes are older than the actual new database or something? Date is not specifically defined in my model but obviously the data was in the old database before and def not manually entered. I think t.timestamp takes care of that initialization, so i guess the question is why I cannot seed into my database? Any ideas
My seed file looks something like this:
Indication.create([
{ :name => "general", :date => "2012-11-09 17:36:25" },
Looks like your Indication model does not have a date field. Do you need to create a migration to add one?
So just in case someone encounters the same problem, here is my solution:
My seed:dump had extracted the data in this case "2012-11-09 17:36:25" and assigned it the variable "date". While I am still not sure why this happened it has to do with the t.timestamp command.
t.timestamp creates 2 variables:
created_at
updated_at
The side file had the 2012-11-09 17:36:25 as "date" variable, which is a reasonable guess based on the format, but nevertheless it is wrong for the Rails application standard. My tables had only two variables that were datetime format which are the ones I named above.
I opened the seeds file and replaced all the ":date" for ":created_at" and it solved the problem. updated_at then acquired the value automatically (updated at was not transferred from my old database to this new one).
The created_as data did transfer and was correct. updated_at acquired the same value in all the data (I noticed it was the date and time and performed the rake db:seed:load command.
I'm having a strange problem in Rails with a Postgres query.
The query looks something like this:
WeeklyPlanner.find_or_create_by_user_id(current_user.id).recipes.find(:all,
:conditions => ["
weekly_planner_events.time_start =
date_part('epoch', to_timestamp(?)::timestamptz at time zone 'CDT')",
Time.local(Time.now.year, Time.now.month, Time.now.day).to_i
])
This generates (as I can view in the console), the following SQL statement:
SELECT "recipes".* FROM "recipes"
INNER JOIN "weekly_planner_events" ON "recipes"."id" = "weekly_planner_events"."recipe_id"
WHERE "weekly_planner_events"."weekly_planner_id" = 2
AND (weekly_planner_events.time_start = date_part('epoch', to_timestamp(1347426000)::timestamptz at time zone 'CDT'))
My problem is that the generated SQL statement works well on psql or pgAdmin, but on rails it returns an empty array. That is, if I copy and paste it as is on a postgres console, it works perfectly fine, but when I run it on the Rails console, it returns nothing, and I have no idea why its happening.
I've tried the following:
Parametrizing 'epoch' and 'CDT'/timezone (in order to remove 's
Switching to a where statement, with the same condition
Passing the variables with #{}s
Doing the search without the date_part('epoch', [float]) function works fine in Rails, but its obviously not the result I need.
I'm finding this issue quite confusing, if there is any other data you need please let me know and I will edit the post.
Thank you.
Maybe when you are using the find_or_create method it is using the create action, so you create a new WeeklyPlanner for a user, and this brand new WeeklyPlanner doesn't have recipes attached to it because it has just been created.
When you go to psql, you probably use a existent WeeklyPlanner.
But this is just a guess.