Related
I have been thinking about this and know what a smoke test is by definition but I am still not able to wrap my head around this.
I would really appreciate it if someone can help me with this and maybe give a real use case or example.
I am confused about the part when ppl say to smoke test is to check major functionalities, what are major functionalities?
So if I have about 100 test cases (e2e) and have 5 tests related to component x how do I decide how many tests to run(smoke test) against the component x (major functionalities) it seems everything is major functionalities for me or I can run all the 5 tests.
Then isn't every individual test a smoke test?
This question might be better asked on sqa or even softwareengineering. but....
"You're sniffing for smoke to detect if there is a fire"
The short answer is: smoke testing is whatever tests you feel you need to run to initially move forward with further testing. In my experience it's going to be different for everyone.
Here is an example: in my case, I know I always need to check this one specific aspect of our app because it breaks a lot and it affects 75% of the application so I always run a few tests against that feature first before running the whole regression.
To directly answer your question about 'major functionalities'; a good example of that would be login. My app requires all users login with username and password. I would consider that "major functionality" since everything else a user does is dependent on logging in to work properly. There is very little point to me testing some other feature if login testing were broken.
I've been asked by an interviewer the following questions about the diagram below:
Why should less effort be required for UI testing?
Is the test pyramid meant for programmers or testers?
Thanks for your help on these questions.
My answer for 1. is that it does not mean "less effort" should be put into UI testing, but that "less code should be covered" by UI testing than the larger portions.
As for 2., it should by considered by anybody who is testing the code. In some organizations that is (and should be) most often a specialized tester and in others, a developer.
My impression is that these are not very good questions, but I do not mention this during the interview.
As per Test Pyramid: the key to good automated test strategy the following diagram represents the Test Pyramid which was originally reflected in one of Martin Fowler's article which was based on a concept developed by Mike Coen.
Test Pyramid
The test pyramid is a tool to fix the problem of over-reliance on long-running Automated / Manual UI tests.
Take aways from the TestPyramid
The pyramid implies that:
Tests on the lower levels are cheaper to write and maintain, and quicker to run.
Tests on the upper levels are more expensive to write and maintain, and slower to run.
Ideally, you should have lots of unit tests, some service tests, and very few UI tests.
This concept is exactly the reverse case of the Testing Ice Cream Cone which is the most often found Test Strategy followed across the software industry where we have very few unit tests, some UI test, lots of QA tests and lots of Manual Tests.
Testing Ice Cream Cone
The Testing Ice Cream Cone approach can be easily identified where the QA department has created an automated test suite, but the development team has not. As a result, tests will be very long running and flakey because the development team has not helped build the suite or architect the application in a way that makes it easy to test. The new check-ins by the developers breaks the product functionality regularly and they are relying on the QA department to report the defects to fix it up.
What is the difference between smoke testing and sanity testing? When do will perform smoke testing and when do will perform sanity testing?
Sanity testing
Sanity testing is the subset of regression testing and it is performed when we do not have enough time for doing testing.
Sanity testing is the surface level testing where QA engineer verifies that all the menus, functions, commands available in the product and project are working fine.
Example
For example, in a project there are 5 modules: Login Page, Home Page, User's Details Page, New User Creation and Task Creation.
Suppose we have a bug in the login page: the login page's username field accepts usernames which are shorter than 6 alphanumeric characters, and this is against the requirements, as in the requirements it is specified that the username should be at least 6 alphanumeric characters.
Now the bug is reported by the testing team to the developer team to fix it. After the developing team fixes the bug and passes the app to the testing team, the testing team also checks the other modules of the application in order to verify that the bug fix does not affect the functionality of the other modules.
But keep one point always in mind: the testing team only checks the extreme functionality of the modules, it does not go deep to test the details because of the short time.
Sanity testing is performed after the build has cleared the smoke tests and has been accepted by QA team for further testing. Sanity testing checks the major functionality with finer details.
Sanity testing is performed when the development team needs to know quickly the state of the product after they have done changes in the code, or there is some controlled code changed in a feature to fix any critical issue, and stringent release time-frame does not allow complete regression testing.
Smoke testing
Smoke Testing is performed after a software build to ascertain that the critical functionalities of the program are working fine. It is executed "before" any detailed functional or regression tests are executed on the software build.
The purpose is to reject a badly broken application, so that the QA team does not waste time installing and testing the software application.
In smoke testing, the test cases chosen cover the most important functionalities or components of the system. The objective is not to perform exhaustive testing, but to verify that the critical functionalities of the system are working fine.
For example, typical smoke tests would be:
verify that the application launches successfully,
Check that the GUI is responsive
Smoke testing
Smoke testing came from the hardware environment where testing should be done to check whether the development of a new piece of hardware causes no fire and smoke for the first time.
In the software environment, smoke testing is done to verify whether we can consider for further testing the functionality which is newly built.
Sanity testing
A subset of regression test cases are executed after receiving a functionality or code with small or minor changes in the functionality or code, to check whether it resolved the issues or software bugs and no other software bug is introduced by the new changes.
Difference between smoke testing and sanity testing
Smoke testing
Smoke testing is used to test all areas of the application without going into too deep.
A smoke test always use an automated test or a written set of tests. It is always scripted.
Smoke testing is designed to include every part of the application in a not thorough or detailed way.
Smoke testing always ensures whether the most crucial functions of a program are working, but not bothering with finer details.
Sanity testing
Sanity testing is a narrow test that focuses on one or a few areas of functionality, but not thoroughly or in-depth.
A sanity test is usually unscripted.
Sanity testing is used to ensure that after a minor change a small part of the application is still working.
Sanity testing is a cursory testing, which is performed to prove that the application is functioning according to the specifications. This level of testing is a subset of regression testing.
Hope these points help you to understand the difference between smoke testing and sanity testing.
References
http://www.softwaretestinghelp.com/smoke-testing-and-sanity-testing-difference/
https://www.guru99.com/smoke-sanity-testing.html
Smoke and sanity testing
In general, smoke and sanity testing seems very similar to many tester who have just started, because in both we talk about build, we talk about functionality and we talk about the rejection of builds, if build's health is not good for the feasible testing.
After going through several projects, from start ups to product base company, I figured out the basic difference between smoke and sanity testing.
I am writing difference between smoke testing and sanity testing here to help you in answering at least one question that normally all testers get asked in interview.
Smoke testing
Smoke testing is done to test the health of builds.
It is also known as the shallow and wide testing, in that we normally include those test cases which can cover all the functionality of the product.
We can say that it's the first step of testing and, after this, we normally do other kind of functional and system testing, including regression testing.
It's normally done by a developer with the help of certain scripts or certain tools, but in some cases it can be performed by a tester too.
It's valid for initial stage of a build confirmation. For example, suppose we have started the development of a certain product, and we are producing a build for the first time, then smoke testing becomes a necessity for the product.
Sanity testing
It is sub-regression
Sanity is done for those builds which have gone through many regression tests and a minor change in code has happened. In this case, we normally do the intensive testing of functionalities where this change has occurred or may have influenced.
Due to this, it is also known as "narrow" and "deep" testing
It's performed by a tester
It's done for mature builds, like those that are just going to hit production, and have gone through multiple regression processes.
It can be removed from the testing process, if regression is already being performed.
If any build doesn't pass the sanity tests, then it is thrown to developer back for the correction of the build.
Try to understand both by this example.
Suppose if you're buying a car from showroom.
The first thing you will check the car contains are for example if it's four tires, a staring, headlight, or all other basic things. This is called smoke testing.
If you're checking how much mileage the car is giving or what is max speed, then this is known as sanity testing.
Smoke Testing
Smoke testing is a wide approach where all areas of the software application are tested without getting into too deep
The test cases for smoke testing of the software can be either manual or automated
Smoke testing is done to ensure whether the main functions of the software application are working or not. During smoke testing of the software, we do not go into finer details.
Smoke testing of the software application is done to check whether the build can be accepted for through software testing
This testing is performed by the developers or testers
Smoke testing exercises the entire system from end to end
Smoke testing is like General Health Check Up
Smoke testing is usually documented or scripted
Santy Testing
Sanity software testing is a narrow regression testing with a focus on one or a small set of areas of functionality of the software application.
Sanity test is generally without test scripts or test cases.
Sanity testing is a cursory software testing type. It is done whenever a quick round of software testing can prove that the software application is functioning according to business / functional requirements.
Sanity testing of the software is to ensure whether the requirements are met or not.
Sanity testing is usually performed by testers
Sanity testing exercises only the particular component of the entire system
Sanity Testing is like specialized health check up
Sanity testing is usually not documented and is unscripted
For more visit Link
Smoke testing is about checking if the requirements are satisfied or not.
Smoke testing is a general health check up.
Sanity testing is about checking if a particular module is completely working or not. Sanity testing is specialized in particular health check up.
Smoke tests are tests which aim is to check if everything was build correctly. I mean here integration, connections. So you check from technically point of view if you can make wider tests. You have to execute some test cases and check if the results are positive.
Sanity tests in general have the same aim - check if we can make further test. But in sanity test you focus on business value so you execute some test cases but you check the logic.
In general people say smoke tests for both above because they are executed in the same time (sanity after smoke tests) and their aim is similar.
Smoke testing
Suppose a new build of an app is ready from the development phase.
We check if we are able to open the app without a crash. We login to the app. We check if the user is redirected to the proper URL and that the environment is stable. If the main aim of the app is to provide a "purchase" functionality to the user, check if the user's ID is redirected to the buying page.
After the smoke testing we confirm the build is in a testable form and is ready to go through sanity testing.
Sanity testing
In this phase, we check the basic functionalities, like
login with valid credentials,
login with invalid credentials,
user's info are properly displayed after logging in,
making a purchase order with a certain user's id,
the "thank you" page is displayed after the purchase
THERE IS NO DIFFERENCE BETWEEN smoke and sanity as per ISTQB.
sanity is synonym of smoke.
Check it here : https://glossary.istqb.org/en/search/sanity
Smoke Testing:-
Smoke test is scripted, i.e you have either manual test cases or automated scripts for it.
Sanity Testing:-
Sanity tests are mostly non scripted.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I don't have a clear idea about smoke testing and sanity testing, some books say that both are same but some tester in some project called as a smoke testing and some tester in some project called as sanity testing, So please give me clear cut idea about my question.
Sorry but there is no clear-cut. Like you explain in your question there is no consensus on the definition, or at least on the difference between sanity and smoke.
Now about smoke tests (or sanity tests!), those are the tests that you can run quickly to get a general idea of how your System Under Test (SUT) behaves. For software testing, this will obviously contain some kind of installation, setup, playing around with the feature and shutdown. If nothing goes wrong, then you now you can go on with your testing. This provides a quick feedback to the team and avoid starting a longer test campaign only to realise that some major features are broken and the SUT is not really usable.
This definition stands for both manual and automated tests. For example, if you use Jenkins (for CI) and Robot Framework (for test automation), you could create 2 jobs in Jenkins: smoke tests and full tests (using tags, this is straightforward). Smoke test job could last a couple of minutes (or max 15 minutes let's say) and the full tests job could as long as needed. Thus the smoke test job gives you a quick feedback on the SUT build (if your smoke tets is a child project of the SUT build of course)
Smoke testing also known as Build version Testing.
Smoke testing is the initial testing process exercised to check whether the software under test is ready/stable for further testing.
sanity testing is a type of testing to check the capability of a new software version is able to perform well enough to accept it for a major testing effort.
Think of the analogy of testing a new electronic device. The first thing you do is turn it on to see if it starts smoking. If it does, there's something fundamentally wrong, so no additional testing either can be done or is worth doing.
For a website, the simplest smoke test is to go to the website and see if the http response is 200. If not, there's no point in testing further. A more useful smoke test might be to hit every page once.
Smoke tests should run as fast as possible. The goal is quick feedback so that you can make a decision.
As for the difference between smoke tests and sanity tests... There is no significant difference. What you call them is irrelevant, as long as everyone in your organization has the same basic understanding. What's important is a quick verification that the system under test is running and has no blatantly obvious flaws.
The smoke test is designed to see if the device seems to work at all. - This is to determine if we can go on with more extensive testing or if something fundamental is broken.
The sanity tests are designed to test the most frequent usecases.
Example:
You are testing a cellphone.
Smoketest - Does it start up without crashing/starting to smoke etc. does it seem to work good enough to perform more extensive testing?
Sanity test - Can you place/recieve calls/messages - the most basic and most used features.
These are both done often and should be quick to run through, they are NOT extensive tests.
Smoke Testing is testing the basic and critical features of an application, before going ahead and doing thorough testing of that application.
Note: Only if the Smoke testing passes, we can carry ahead with other stages of testing, else the product is not fit to be tested and should be sent to Development team.
Sanity Testing: There is no clear definition as such, but this one I picked up from the Internet
Check the entire application at the basic level, focuses on Breadth rather than length.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 4 years ago.
Improve this question
What is exactly PAT, When we will do the pre acceptance testing?
I don't think it's a widely-used term or part of a standard. Therefore, what exactly it means is organization-specific and should be defined in a glossary somewhere. More likely though you'll just have to ask people what it means.
Any testing done before acceptance testing.
This would include:
Unit tests
Stress tests
Integration tests
Performance tests
There's no standardised meaning for the term - often it depends on your process- be it Agile or Extreme Programming etc.
Generally however, there are a number of tests done by developers or testing in a developer test environment. This can be unit tests, developer tests, sanity regression tests, performance tests - ie tests that the QA team wants done before they'll even look at it. At a bare minimum, it might be just testing that the software builds (although it's frightening how often I've had a developer fail to even check this).
Well I would like to share something which everyone may not agree to but this is what I feel Pre-Acceptance testing would be:
The testing done to perform that the system under test functions as per the designed requirements to cover the customer's business areas before entering the User Acceptance Test phase where users from the customer's side are invited to perform the testing at the vendor's location where development team assistance is available when any flaw occurs in the expected business flow. This will be called as Alpha Test. Please feel free to correct me if I have said something wrong.
Acceptance testing is a testing technique performed to determine whether or not the software system has met the requirement specifications. In SDLC, acceptance testing fit as follows:
Requirement Analysis - Acceptance Testing
High Level Design - System Testing
Low Level Design - Integration Testing
Coding - Unit Testing
Simply put, PAT is any test done before acceptance testing. There are various forms of acceptance testing such as User acceptance testing, business acceptance testing, alpha testing and beta testing.