Reproducible screenshots with Selenium and Firefox - selenium

Question

I use Selenium to automate some GUI tests using the firefox web driver.
It occurred to me that it would be also make sense to create screenshots during the run of the tests to use those in the manual.
The screens of my application are relatively static - no timestamps visible and so on. So my expectation would be, if I create a screenshot from lets say the start page, navigate later to the start page again, the screenshots should be identical. Also if I run the test twice, the screenshot of the start page should be identical between both runs.
I save the screenshots as PNG, I even process the screenshots (save without date) before saving them, so that the files should be really identical.
Nevertheless, if I compare the pictures with each other (e.g. subtract them from each other), there are minor differences between them (not visible to the naked eye), some faint lines at the border of tables, or around fonts.
My question:
1) Why are there differences at all?
2) What could be the easiest way to ensure that the screenshots are identical? (what kind of post processing could I do)
PS: I also tried to change the renderer from skia to windows to cairo, but although the differences are slightly different, it still doesn't solve the problem.

Answer 1

How do you save the images ?
In ruby I was using something like this ?
def take_screenshot(image_id)
scenario = Zpg::HooksHelper::FeatureHelper.scenario_name
Dir.mkdir('output', 0o777) unless File.directory?('output')
Dir.mkdir('output/screenshots', 0o777) unless File.directory?('output/screenshots')
screenshot = "./output/screenshots/#{image_id}#{scenario.tr(' ', '_').gsub(/[^0-9A-Za-z_]/, '')}.png"
if page.driver.browser.respond_to?(:save_screenshot)
page.driver.browser.save_screenshot(screenshot)
else
save_screenshot(screenshot)
end
FileUtils.chmod(0o777, screenshot)
end
And I was checking the image diff
# return[hash]
def image_diff(imageid_1 = nil, _imageid_2 = nil)
scenario = Zpg::HooksHelper::FeatureHelper.scenario_name
if imageid_1.class != Integer
image_1 = "./features/support/data/images/#{Zpg.brand}/#{imageid_1}.png"
image_2 = "./output/screenshots/#{_imageid_2}#{scenario.tr(' ', '_').gsub(/[^0-9A-Za-z_]/, '')}.png"
else
image_1 = "./output/screenshots/#{imageid_1}#{scenario.tr(' ', '_').gsub(/[^0-9A-Za-z_]/, '')}.png"
image_2 = "./output/screenshots/#{_imageid_2}#{scenario.tr(' ', '_').gsub(/[^0-9A-Za-z_]/, '')}.png"
end
images = [
ChunkyPNG::Image.from_file(image_1),
ChunkyPNG::Image.from_file(image_2)
]
diff = []
images.first.height.times do |y|
images.first.row(y).each_with_index do |pixel, x|
diff << [x, y] unless pixel == images.last[x, y]
end
end
puts "pixels (total): #{images.first.pixels.length}"
puts "pixels changed: #{diff.length}"
puts "pixels changed (%): #{(diff.length.to_f / images.first.pixels.length) * 100}%"
# init empty hash
diff_hash = {}
# return pixels changed number
diff_hash[:pixels_changed] = diff.length
# return pixels changed percentage
diff_hash[:pixels_changed_percentage] = (diff.length.to_f / images.first.pixels.length) * 100
# return diff hash
diff_hash
end
you can get different results if the browser DOM is not 100% loaded . I would try to have a threshold in which I would expect my image to be .
There is a very good project here in .Net https://www.codeproject.com/Articles/374386/Simple-image-comparison-in-NET which you can convert in any language you want.