please tell me why there is such a big difference between the included "Total Blocking Time" on the site https://developers.google.com/ and the local litehouse for a mobile device?
I am not able to give you the address and show you the site, there is just a text and a picture. The site is made on nuxt js and assembled completely statically.
My computer - mackbook air m1, google uses less powerful emulation and results in fewer parrots?
This official page explains it well: https://web.dev/performance-scoring/
Common problems:
A/B tests or changes in ads being served
Internet traffic routing changes
Testing on different devices, such as a high-performance desktop and a low-performance laptop
Browser extensions that inject JavaScript and add/modify network requests
Antivirus software
Otherwise, this one is another answer: https://www.debugbear.com/blog/why-is-my-lighthouse-score-different-from-pagespeed-insights
Google PageSpeed uses a “combo” of lab and real-world data[historical data of your website], whereas Lighthouse uses lab data only[locally tested on your machine] to build its report.
Note: You should trust PSI metrics over just lab data.
We have an issue where we have a website for test and an equivalent website for live. What we are finding is that due to carelessness our testers are using the wrong site (e.g. testing on the live site!).
We have total control over both sites, but since the test site is for acceptance testing by the users we don't want to make them different in any way. The sites must look the same and there is also a layer of management that will kick up a storm if the test and live sites are in any way different.
How have other people solved this problem? I am thinking of a browser plugin to make the browser look different somehow (e.g. changing the colour of the location bar when on the test website). Does anyone know of a plugin or a technique that would work? (We primarily use Firefox and Chrome)
Thanks,
Phil
UPDATE
We eventually settled on a program of: different credentials for the test and live site (this was not popular!) and making a series of plugins available for those who wanted them (colourize tabs for Chrome and Firefox users - we never did find a good plugin for IE).
Thanks to those who answered.
In our company we use different site's names:
www.dev.site.com - for developers
www.qa.site.com - for QA's
www.site.com - production site
Another good practic is to use different users credentials for dev\qa and prod sites.
Have done a lot of searching and still not found a site that offers real-user testing across the range of browsers including Ipad 1,2,3 in a non-virtual or automated way.
We recently had a site that crashed Ipad 1 (the client's Ipad of course) but could not be replicated in any of the online services so we now want to send our completed websites out and get back a report that it works across IE 7-10, Chrome and all Ipad generations etc.
Does anyone know of such a service. As said, it must be on actual devices not virtualised one.
Thanks in advance.
I have a system to test, which is a video ads distribution technology. I need to load every video like 1-2 mins to serve the ads. The videos are played in a Flash client and streamed as FLV streams like in YouTube.
The reason why I need to test it only via browsers -- and every other method won't work -- is to stress test both the video streaming servers and the ads servers simultaneously and displaying ads in real-time.
I have used Selenium, WatiN, Automation Anywhere and many other automation tools. However, when I am trying to start like 10000 browsers on my machine (32GB RAM, 16-core CPU), none of them are able to do the job.
With Selenium, I am able to start the maximum FireFox instances so far, but that's still too low: half of the instances don't run the test.
Any suggestions to do with Selenium?
You aren't going to run 10,000 browsers on your machine. That would give 3.2MB of physical memory per browser instance and I'm pretty sure FireFox just won't like that.
You could create a JMeter script that hits your server with many threads. It won't interact with the UI but would simulate the load of many clients hitting whatever URLs you tell it. I believe it also includes the ability to record a session and play it back for easy setup of your sessions.
Selenium isn't really optimized for load/stress testing, especially if you're running your browsers locally. Running 1000+ browsers is going to choke even the beefiest server. Though RAM is an obvious bottleneck, you also have limited CPU resources and bandwidth. The latter being a primary concern if you are loading videos.
Not to mention you'd be testing from a single IP with 10k browsers, so load balancing may not kick in properly, as well as the actual distribution of video ads to specific virtual users.
If you want to stick with existing Selenium tests, I've had good experiences with BrowserMob. They basically have a huge grid to do real browser load-testing, distributed across AWS.
Another recommendation would be an actual performance testing tool. I'd recommend Soasta CloudTest. They have a free version that runs 100 users so you can see if it will be a good fit for you. I have found that scripting for CloudTest is relatively simple.
Disclaimer: My experiences with both companies have been as a paying customer and I have never worked for either.
If you are using Windows machine then as per my experience there is a limit on number of browser window instances to be opened. As per my test last time, it does restrict between 100-150 browser windows.
I would recommend you using headless robot, which doesn't require opening browser window. I think latest version of Selenium has that capability. But it seems to be more like a load test as you are trying to simulate 10,000+ user instances, I would recommend you using load testing tool like JMeter or LoadRunner.
It looks to me that you are trying to verify what the client will see based on high traffic, no?
In that case, Joel is quite correct. If you absolutely have to see what the client sees, you could use threaded hits and just dump the results in a database. That'll show you anything the client would see anyway, and it's a lot easier to sort through than thousands of browser instances.
Either way, your client will not see errors if there are no errors present on the server side. If you're testing functionality in bandwidth restricted environments, CPU-intensive environments, or memory-intensive environments, those are much easier achieved than running thousands of browser instances.
Your post smells of some form of ad-based fraud to me, but either way: have you considered using different web browsers besides Firefox? PhantomJS is a headless webkit-based browser that is compatible with Selenium. It supports all the core browser features like DOM handling, CSS selectors, Javascript and Canvas. I do not know if it supports Flash.
This post has a decent list of other headless and automatable web-browsers that you might consider.
Also, if each browser instance is instantiating a Flash plugin, don't neglect the possibility that the issue could be with Flash and not Firefox. Alternatively, why instantiate several different Firefox processes? Can you accomplish what you want through the use of tabs instead?
The in-house way to this wiht selenium is using browsermob proxy and multiple broswser agents to recreate the experience of different users, changing the ip is more difficult because it requires changing your home network.
Here is a good example
Are there any testing platforms out there for testing WAP/WML pages besides that provided by Nokia? I have tried to get ahold of the Nokia Mobile Internet Toolkit but it's too tied down with authentication and certificates etc.
Nokia software is like Adobe and......sucks.
It's not really wap-oriented, but Aptana does have a plugin for symbian widgets, which I think is mostly html.
Openwave used to do an SDK for this purpose, which was very useful for testing compatibility with the large number of mobile phones shipped with Openwave Mobile Browsers. Since Openwave sold off their browser business it is no longer available on their websites or those of their successors - they appear to have deleted all the developer support information.
But a Google search for "Openwave SDK" may help.