RSpec: How to mock SQL NOW() - sql

I can mock Time.now with a great timecop gem.
Time.now
=> 2018-05-13 18:04:46 +0300
Timecop.travel(Time.parse('2018.03.12, 12:00'))
Time.now
=> 2018-03-12 12:00:04 +0300
TeacherVacation.first.ends_at
Thu, 15 Mar 2018 12:00:00 MSK +03:00
TeacherVacation.where('ends_at > ?', Time.now).count
1
But (obviously) this wouldn't work while using NOW() in a query:
TeacherVacation.where('ends_at > NOW()').count
0
Can I mock NOW() so that it would return the results for a certain time?

Timecop is a great gem! I would recommend using Timecop.freeze instead of traveling for your instance; you want to keep your tests deterministic.
As far as I could find, there doesn't seem to be a way to mock SQL's functions. Some languages like Postgres allow overloading functions, but you would still need a way to interject, and there doesn't seem to be a way to use environment variables in SQL.
A co-worker seemed to be certain you could actually drop system/language functions and make your own, but I was concerned about how to recover them after you do that. Trying to go that route sounds like a pain.
Solutions?
Here are a couple of "solutions" that I've come up with today while fighting this problem. Note: I don't really care for them to be honest, but if it gets tests in place ¯\_(ツ)_/¯ They at least offer a way to get things "working".
Unfortunately there's no snazzy gem to control the time in SQL. I imagine you would need something crazy like a plugin to the DB, a hack, a hook, a man in the middle, a container that you could trick SQL into thinking the system time was something else. None of those hack ideas would surely be portable/platform agnostic unfortunately either.
Apparently there are some ways to set time in a docker container, but that sounds like a painful overhead for local testing, and doesn't fit the granularity of a per-test time to be set.
Another thing to note, for me we're running large complex raw SQL queries, so that's why it's important that when I run the SQL file for a test I can have proper dates, otherwise I would just be doing it through activerecord like you mentioned.
String Interpolation
I ran across this in some large queries that were being ran.
This definitely helps if you need to push some environment variables through, and you can inject your own "current_date" if you want. This would help too if you needed to utilize a certain time across multiple queries.
my_query.rb
<<~HEREDOC
SELECT *
FROM #{#prefix}.my_table
WHERE date < #{#current_date} - INTERVAL '5 DAYS'
HEREDOC
sql_runner.rb
class SqlRunner
def initialize(file_path)
#file_path = file_path
#prefix = ENV['table_prefix']
#current_date = Date.today
end
def run
execute(eval(File.read #file_path))
end
private
def execute(sql)
ActiveRecord::Base.connection.execute(sql)
end
end
The Dirty Update
The idea is to update the value from ruby land pushing your "time-copped" time into the database to overwrite the value generated by the SQL DB. You may need to get creative with your update for times, like querying for a time greater than a given time that doesn't target your timecop time that you'll be updating rows to.
The reason I don't care for this method is because it ends up feeling like you're just testing activerecord's functionality since you're not relying on the DB to set values it should be setting. You may have computations in your SQL that you're then recreating in the test to set some value to the right date, and then you're no longer doing the computation in the SQL so then you're not even actually testing it.
large_insert.sql
INSERT INTO some_table (
name,
created_on
)
SELECT
name,
current_date
FROM projects
JOIN people ON projects.id = people.project_id
insert_spec.rb
describe 'insert_test.sql' do
ACTUAL_DATE = Date.today
LARGE_INSERT_SQL = File.read('sql/large_insert.sql')
before do
Timecop.freeze Date.new(2018, 10, 28)
end
after do
Timecop.return
end
context 'populated same_table' do
before do
execute(LARGE_INSERT_SQL)
mock_current_dates(ACTUAL_DATE)
end
it 'has the right date' do
expect(SomeTable.last.created_on).to eq(Date.parse('2018.10.28')
end
end
def execute(sql_command)
ActiveRecord::Base.connection.execute(sql_command)
end
def mock_current_dates(actual_date)
rows = SomeTable.where(created_on: actual_date)
# Use our timecop datetime
rows.update_all(created_on: Date.today)
end
Fun Caveat: specs wrap in their own transactions (you can turn that off, but it's a nice feature) so if your SQL has a transaction in it, you'll need to write code to remove it for the specs, or have your runner wrap your code in transactions if you need them. They'll run but then your SQL will kill off the spec transaction and you'll have a bad time. You can create a spec/support to help out with this if you go the route of cleaning up during tests, if I were in a newer project I would go with writing a runner that wraps the queries in transactions if you need them -- even though this isn't evident in the SQL files #abstraction.
Maybe there's something out there that lets you set your system time, but that sounds terrifying modifying your system's actual time.

I think the solution for this is DI (dependency injection)
def NOW(time = Time.now)
time
end
In test
current_test = Time.new(2018, 5, 13)
p NOW(current_test)
In production
p NOW

Related

How can I reduce the database call time and number (Rails)?

So I'm working on a rails app for a building that keeps track of water usage/collection and electricity use/solar generation, etc. These are stored as measurement rows, attached to sensors, which are attached to programs (location in the building, essentially) and subtypes (attached to types - water, electricity).
I'm doing some graphing with chartkick, and the database calls related to this are way too slow. They'll be much faster on the production servers, but there will also be far more data.
Here's the helper method that has the chart generation and database call in it:
def stackedSubtypeChart(grouping)
rsubs = #resource.subtypes
.order(:usage?) #add usage types after gen types
.map{|stype| [
stype.name,stype.measurements #this takes too long!
.where("date >= ?", params[:start]) #(4 calls!!)
.where("date <= ?", params[:stop])
.group_by_period(grouping, :date).maximum(:amount)]}
rsubs = rsubs.map {|stype|
{name: stype[0],
data: stype[1]}}
ret = column_chart rsubs,
stacked: true,
library: { :series => {0 => { type: "line"}}}
end
#resource is defined in the controller as:
#resource = Type.includes(:subtypes => :sensors).find_by_resource('electricity')
I've commented the line that's responsible for there being multiple calls, which is definitely part of the problem. This takes two seconds to load on my (admittedly very very old) computer with a month of data.
I could really use help with both changing the map so that this is one call instead of however-many-subtypes calls, and with reducing what I'm pulling in so each call isn't taking half a second. I don't have a ton of experience optimizing this sort of thing and I'm not really sure how to start doing more than I have here already.
Might be helpful to look into ActiveRecord Explain to dig into the SQL. There's a good screencast that explains (pun totally intended) pretty well.
After a lot of bashing my head against a wall, I stumbled across this, which is a much faster single query that grabs all the data + data connections I need. It's a little hard to format but it works.
rsubs = Measurement
.where("measurements.date >= ? AND measurements.date <= ?",
offset(params[:start], -1, grouping),
offset(params[:stop], 1, grouping))
.joins(sensor: {subtype: :type})
.where("types.resource = ?", #rname)
.order('subtypes."usage?"')
.group_by_period(grouping, :date).group("subtypes.id, subtypes.name").maximum(:amount)

How to use update_all but each? Is there a better method?

Im current working on a small project and I want to seed my database faster. I migrated a new column called "grand_total_points" to my table of users. So originally I was using this code.
user = User.all
user.each do |x|
x.grand_total_points = x.total_points
x.save!
end
This takes me ages, because I have to update a million record.
Total_points have already been defined in my user model where it calculates all the users points that have been submitted. Forgive me for my explanation. Is there a way to use update_all method but with each included in it?
Yep, possible:
User.update_all('grand_total_points = total_points')
It will generate the following SQL query:
UPDATE "users" SET "grand_total_points" = 'total_points'
If total_points is not a column but an instance method, move the logic into update_all query.
User.update_all("grand_total_points = #{total_points calculation translated into SQL terms}")
I found something that could work. So basically i combine a ruby code with an execute SQL statement, and I put it in a migration file. Here's how the code works. I hope this helps. Make sure you follow the query according to your data.
class ChangeStuff < ActiveRecord::Migration
def change
points = Point.select('user_id, SUM(value) AS value').group(:user_id)
points.each do |point|
execute "UPDATE users SET grand_total_points = #{point.value} WHERE users.id = #{point.user_id}"
end
end
end
You should run bundle exec rake db:migrate after that. The normal way takes me 2-3hours. This only took me 2minutes.

Rails show different object every day

I want to match my user to a different user in his/her community every day. Currently, I use code like this:
#matched_user = User.near(#user).order("RANDOM()").first
But I want to have a different #matched_user on a daily basis. I haven't been able to find anything in Stack or in the APIs that has given me insight on how to do it. I feel it should be simpler than having to resort to a rake task with cron. (I'm on postgres.)
Whenever I find myself hankering for shared 'memory' or transient state, I think to myself "this is what (distributed) caches were invented for".
#matched_user = Rails.cache.fetch(#user.cache_key + '/daily_match', expires_in: 1.day) {
User.near(#user).order("RANDOM()").first
}
NOTE: While specifying a TTL for cache entry tells Rails/the cache system to try and keep that value for the given timeframe, there's NO guarantee that it will. In particular, a cache that aggressively tries to reclaim memory may expire an entry well before its desired expires_in time.
For this particular use case, it shouldn't be a big deal but in cases where the business/domain logic demands periodically generated values that are durable then you really have to factor that into your database.
How about using PostgreSQL's SETSEED function? I used the date to seed so that every day the seed will change, but within a day, the seed will be consistent.:
User.connection.execute "SELECT SETSEED(#{Date.today.strftime("%y%d%m").to_i/1000000.0})"
#matched_user = User.near(#user).order("RANDOM()").first
You may want to seed a random value after using this so that any future calls to random aren't biased:
random = User.connection.execute("SELECT RANDOM()").to_a.first["random"]
# Same code as above:
User.connection.execute "SELECT SETSEED(#{Date.today.strftime("%y%d%m").to_i/1000000.0})"
#matched_user = User.near(#user).order("RANDOM()").first
# Use random value before seed to make new seed:
User.connection.execute "SELECT SETSEED(#{random})"
I have split these steps in different sections just for readability. you can optimise query later.
1) Find all user records till today morning. so that the count will freeze.
usrs_till_today_morning = User.where("created_at <?", DateTime.now.in_time_zone(Time.zone).beginning_of_day)
2) Pluck all ID's
user_ids = usr_till_today_morning.pluck(:id)
3) Today date it will be a range (1..30) but will remain constant throughout the day.
day_today = Time.now.day
4) Select the same ID for the day
todays_user_id = user_ids[day_today % user_ids.count]
#matched_user = User.find(todays_user_id)
So it will give you random user records by maintaining same record throughout the day!!

Math with datetime in SQL query in rails

So I have an table of phone_numbers in Rails 3.2, and I would like to be able to make a query such as the following:
PhoneNumber.where("last_called_at - #{Time.now} > call_spacing * 60")
where last_called_at is a datetime and call_spacing is an integer, representing the number of minutes between calls.
I have tried the above, I have also tried using
PhoneNumber.where("datediff(mi, last_called_at, #{Time.now}) > call_spacing")
If anyone could help me make this work, or even recommend a current, up-to-date rails SQL gem, I would be very grateful.
Edit: So the best way to make this work for me was to instead add a ready_at column to my database, and update that whenever either call_spacing or last_called_at was updated, using the before_save method. This is probably the most semantic way to approach this problem in rails.
You have to use quotes around #{Time.now}. To make your first query work you may use TIME_TO_SEC() function:
PhoneNumber.where("(TIME_TO_SEC('#{Time.now}') - TIME_TO_SEC(last_called_at)) > (call_spacing * 60)")
Here is my way to do this:
PhoneNumber.where("last_called_at > '#{Time.zone.now.utc - (call_spacing * 60)}'")
also take a look at this article to be aware of how to play with timezones:
http://danilenko.org/2012/7/6/rails_timezones/

Time.now.beginning_of_year does not start at the beginning of the year

Trying to get records that were created this year, I stumbled upon this great question. The second answer says you get all records from a model that were created today by saying:
Model.where("created_at >= ?", Time.now.beginning_of_day)
So, naturally, I tried the same thing with Time.now.beginning_of_year, and it works just fine.
However, what struck me as interesting is that the outputted query (I tried it in the console) is
SELECT COUNT(*) FROM `invoices` WHERE (created_at >= '2012-12-31 23:00:00')
I wasn't aware that 2013 already began at 2012-12-31 23:00:00? How's that?
If you haven't set it yet, you should set your timezone in the config/application.rb file. Look for the line that begins with config.time_zone. (If you aren't sure what value to give, you can run rake time:zones:all to get a list of all available timezones.)
Once you've set your timezone, you should use Time.zone.now, as opposed to Time.now. This will properly "scope" your times to your timezone.
Check the API for more details on TimeWithZone.