I'm seeing an error in some integration specs, using rspec, capybara, capybara-webkit and timecop.
Capybara::FrozenInTime:
time appears to be frozen, Capybara does not work with libraries which freeze time, consider using time travelling instead
The only gem that I know that freezes time is Timecop, but I'm not using it in the test case that fails.
Since the error occurs only sometimes I can't even know if its gone or not after changing something.
The end of the error message holds the solution:
consider using time travelling instead
Just change Timecop.freeze to Timecop.travel. Timecop.freeze breaks Capybara's auto-wait feature.
In addition, I would call Timecop.return in an after block, as it will be associated with the most recent travel block:
after :each do
Timecop.return
end
The solution I found was to add
before :each do
Timecop.return
end
in spec_helper.rb.
This way we garantee that the time is not frozen before each test, although the only ones that have this problem are the ones executed in a webdriver different from rack-test. In my case capybara-webkit.
Related
I am currently running capybara specs that test different functionality of an app. When running my specs, it appears that the page is not being reached fast enough to test. I have three specs currently, and it takes 9 minutes to end up having them all fail with this same error. Here is the result of running the specs
Randomized with seed 38457
Capybara starting Puma...
* Version 3.11.4 , codename: Love Song
* Min threads: 0, max threads: 4
* Listening on tcp://127.0.0.1:50109
FFF
Failures:
1) Successful source is created
Got 0 failures and 2 other errors:
1.1) Failure/Error: visit ('/clients/new')
Net::ReadTimeout:
Net::ReadTimeout
# ./spec/qa/variables.rb:12:in `login_user'
# ./spec/qa/successful_source_spec.rb:7:in `block in (root)'
1.2) Failure/Error: #io.to_io.wait_readable(#read_timeout) or raise Net::ReadTimeout
Net::ReadTimeout:
Net::ReadTimeout
Also note that I am using selenium with chrome headless.
Is there anything I can do to have the page load faster so that I can test? This is also the first time these specs are being run.
I decided to go with a quick fix for this problem. It turns out that the rails server was not connecting to the page fast enough before the test would time out. I decided to change the default wait time in the spec_helper.rb to 120 seconds. It will take a lot longer the first time you run the specs, but they eventually will connect and run smoothly from then on. I do not think this is best practice, but it does provide a quick fix to begin testing your specs.
I recently started using capybara-webkit in order to speed up my acceptance tests. 90% of my tests run using the standard capybara DSL but some are slightly different.
One of the main ones that I am having trouble with is deleting cookies. Previously I used the following:
page.driver.browser.manage.delete_all_cookies
but this does not work with capybara-webkit. Receive this error:
undefined method `delete_cookie' for #<Selenium::WebDriver::Driver:0x007f86cb068b88> (NoMethodError)
Does anyone know how I can delete the cookies using capybara-webkit?
Thanks!
You can use clear_cookies method:
page.driver.browser.clear_cookies
I'm having a really tough time investigating the cause of a test failure. I'm a very experienced programmer and am well versed in general debugging techniques, but I'm new to Capybara and RSpec so I'm hoping there's some kind of facility I'm ignorant of that can help me.
In short, I have a test something like this:
expect { click('.fake_button'); sleep 1 }.to change { clicks.count }.by(1)
When the fake button is clicked, it triggers an AJAX call to the Rails app which, among other things, adds a click record to the database. I can think of dozens of things that could be causing this test to fail and have had only limited success getting information out of logs. The tests do not fail in development and it only fails sporadically in test. One of the differences of the test environment is that the tests are run on a server in our office against a server in the cloud, so there are network delays along with other possible issues.
This is very hard to diagnose because there's so little information coming out of the failed test and of course all the database information is thrown away by the time I read about the failure. I know clicks.count didn't change in the test and I can infer that click('.fake_button') succeeded, but due to server time sync issues I can't even be sure that the click happened on the right button or that the AJAX call fired.
What I'd like are some tools to help me follow this test case in the web server logs (maybe using automatic URL parameters, for example), detailed logging about what Capybara did, and a record of the web page as it was when the failure occurred, including cookie values. Can I get any of that? Anything like that?
Capybara simulates human actions. The test code does exactly what needed. It's something a real user should expect. I don't think you should complain the code.
I think it's okay to increase the wait time, say 1 to 2, due to your network latency, but it should not exceed a reasonable value otherwise the app does not work as real user expected.
To debug Capybara codes, there are three methods as I summarized:
Add "save_and_open_page" to the place you want to see result. Then a saved html page will appear during the test. (I forget if "launchy" gem should be added)
Temporarily set this test as JS to see how this test going.
scenario "a fake test", js: true do
# code here
end
By doing this a real browser will pop up and Capybara will show you step by step how it play the code.
Just run $ tail log/test.log to show what happened recently.
Building off what #Billy suggested, log/test.log was not giving me any useful information and I was already using js: true so I tried this:
begin
expect { click('.fake_button'); sleep 1 }.to change { clicks.count }.by(1)
rescue Exception => e
begin
timestamp = Time::now.strftime('%Y%m%d%H%M%S%L')
begin
screenshot_name = "tmp/capybara/capybara-screenshot-#{timestamp}.png"
$stderr.puts "Trying to save screenshot #{screenshot_name} due to test failure"
page.save_screenshot(screenshot_name)
rescue Exception => inner
$stderr.puts "Ignoring exception #{inner} while trying to save screenshot of test page"
end
begin
# Page saved by Capybara under tmp/capybara/ by default
save_page "capybara-html-#{timestamp}.html"
rescue Exception => inner
$stderr.puts "Ignoring exception #{inner} while trying to save HTML of failed test page"
end
ensure
raise e
end
end
Later I changed the test itself to take advantage of Capybara's AJAX synchronization features by doing something like this:
starting_count = clicks.count
click('.fake_button')
page.should have_css('.submitted') # Capybara is smart enough to wait for this to happen
clicks.count.should == starting_count + 1
Note that the CSS I'm looking for is something added to the page in JavaScript by the AJAX callback, so it showing up is a signal that the AJAX call completed.
The rescue blocks are important because the screenshot has a high failure rate from not having enough memory to render the full page and convert it to an image.
EDIT
Though I haven't tried it, a promising solution is Capybara::Screenshot which automatically saves the screenshot and HTML on any failure. Just reading the code it looks like it will have problems when the screenshot fails and I can't tell what state the page will be in by the time the screenshot is triggered, but it certainly looks like it's worth a try.
A nice way to debug tests is to use irb to watch what's actually happening in the browser. RSpec fails usually give decent information for simple cases, but for more complicated things I either split the case up until it is simple, or chuck it in irb for a live session to make sure its doing what it should do.
Make sure to use :selenium as your driver, and you should see firefox come up and be able to be driven by your irb session.
Here is a integration test case with capybara in rails 3.2.12. click_link 'New Log' is an ajax call. However the page opened starts with $() and has a bunch of js escape like \n and \log-log.
it "should work with link on show customer_comm_record page" do
visit customer_customer_comm_records_path(#cust)
#visit customer_customer_comm_record_path(#cust, #crecord)
click_link #crecord.id.to_s
click_link 'New Log'
save_and_open_page
end
We also tried to wrap the case with describe "", :js => true do, how there is an error saying
`An error occurred in an after hook ActiveRecord::StatementInvalid: SQLite3::BusyException: database is locked:`
There is no error out of code execution. What's wrong with the rspec case? Thanks for help.
It looks like your server is locking the database so that the test cannot clean up after it has run.
When you use Capybara without JavaScript the tests and server are all running in a single process and thread. This means they can share the same database connection and transaction. This means that RSpec can use a simple database transaction to roll back changes at the end of the test and why you don't see any lock contention between test and server.
When you run with :js => true things are a little different, the server starts up in a separate thread or process to your tests so they will each be using a separate database connection and transaction. This means that the database transaction strategy that RSpec uses by default to clean up won't work. It is also causing lock errors in your case.
The Capybara readme talks about this problem and recommends database_cleaner gem as an alternative in this situation. You will need to configure database cleaner to use the truncation strategy rather than transactions
Having a bit of an oddity with some capybara webkit-driver (:js => true) tests.
Tests run fine when they run on their own, but somehow in sequence, they fail.
For example, I have a request test that looks something like
describe "A", :js => true do
# tests here run fine
end
describe "B", :js => true do
# tests here fail
end
When I split describe B section into its own file, and run it using bundle exec rspec spec/requests/b_spec.rb however - tests run fine and pass.
Debugging this, it looks like when both sections are in the same file, somehow the webkit driver loads a 'dirty' browser session. Trying things like page.driver.reset! or Capybara.reset_sessions! or Capybara.reset! doesn't seem to have any effect...
When using spectator/spork this isn't a problem, since I can split the tests into different files and run them independently, but when running the full suite of tests using bundle exec rspec - these tests fail...
How can I reset the webkit driver / session properly between tests? Or I am chasing the wrong problem?
p.s. These tests are not hitting the database or altering state in any particular way, so I'm pretty sure it's some driver related problem.
Sometime it helps just writing the question for the solution to pop up.
The key for me was:
These tests are not hitting the database or altering state in any particular way, so I'm pretty sure it's some driver related problem.
Turns out there was a state-change. In my particular case, setting OmniAuth into test_mode, which required setting it back to false after the previous test was running...