Home

 › 

Articles

 › 

Understanding Sanity Testing, with Examples

programming languages for AI

Understanding Sanity Testing, with Examples

Key Points

  • Sanity testing is a basic type of regression testing that identifies crucial bugs in software builds
  • It is performed as the first level of software development testing and focuses on core functionalities
  • Sanity testing is quick and simple, providing rapid feedback and identifying missing objects
  • It has some disadvantages, such as not covering all scenarios and being dependent on tester expertise
  • Sanity testing differs from smoke testing and regression testing in scope and stage of development

Sanity testing is a crucial step in the development of any software build. It represents the first, often the most basic, type of software testing that occurs during the building process. It’s a subset of regression testing and is often performed alongside smoke testing. However, you might ask, “What exactly is this type of testing, and how do you perform one?” Let’s take a look.

What Is Sanity Testing in Software Development?

A sanity test is a low-level, basic type of regression testing. Its primary purpose is to find and report crucial, debilitating bugs in the software build. It is not designed to find and report all bugs. Sanity testing will only be able to find glaring bugs in the build that impact the core functionality.

It is typically performed as the first level of software development testing and will be performed on almost every build in its early stages of development. In mid-to-later stages of development, it will typically be performed alongside smoke testing and before full regression testing.

Examples of Sanity Testing

Sanity testing is designed to be quick and simple. It covers only the most basic and critical functions of the build. It will not uncover all bugs within the build. The tests that are performed depend entirely on the intended functionality of the build. Here are some examples of sanity tests that may be performed on different types of builds. 

Example 1: Login Page

The company is adding a login page to its new portal. A small group of sanity testers is given login credentials. Testers attempt to log in so developers can ensure that the page is allowing people with active credentials to log in while rejecting inactive credentials. 

Etsy Sign in
Testers are given credentials to test the login page to ensure it allows people to log in.

Example 2: Sign-Up Page

The company is also adding a sign-up page to its new portal. A small group of sanity testers is told to attempt to sign up for the portal. The goal is to ensure that the sign-up page is both allowing users to create new accounts, and also that the page is rejecting entries that do not fall within the entry requirements the company has set forth.

An example sign up form
Sanity testers attempt to sign up for the portal to ensure new users can create accounts.

Example 3: Game Hotfix

A company has identified a bug in their game that causes players to clip through the floor and fall to their deaths. Once they have identified the problem in the code, they develop a fix for it. They then pass the hotfix down to a small group of sanity testers. The testers walk across known hotspots for falling through the floor to ensure they can now walk through these zones without falling.

gaming monitor
Testers examine known hotspots to ensure players can navigate through these zones without issue.

Features of Sanity Testing: Why We Do It

Sanity testing has many crucial features that make it useful in software development. Aside from the obvious of identifying bugs in a build, it provides many irreplaceable benefits to the software development team as they move forward with introducing new features and hotfixes to their builds. Here are the most important features of it that make it a crucial step of software development.

  • Unscripted: Most sanity tests are done unscripted. While materials may be provided if necessary (such as login credentials for a login page), the tests are designed to allow the testers to explore new features to ensure their core functionality freely.
  • Undocumented: Sanity tests don’t typically require documentation, meaning they can be performed more frequently and by more people.
  • Narrow and Deep: Sanity tests are what is referred to as “narrow and deep,” meaning they cover a narrow set of functions but test them very deeply.

Advantages

Sanity testing has a lot of advantages in the programming world. For starters, this type of testing allows for quick and easy identification of major bugs that might be created when introducing a new feature to a build. It can also be carried out quickly. Feedback is received and processed more rapidly because it does not require extensive documentation to produce results. Once the testing is completed, builds with defects will be rejected. These builds will be sent back to development before ever reaching a full regression test.

Sanity testing also quickly and easily identifies missing objects while verifying whether a build is still functioning after major or minor code-level changes. It’s an amazing testing suite for products that have a limited testing time.

Disadvantages

Sanity testing does have a few disadvantages, especially when compared to a full-force regression test. For starters, it’s impossible to cover all scenarios with a sanity testing suite. This is due to the short testing period and narrow scope. Since the testing is also unscripted, it can’t be used to provide future references for further testing.

The process of sanity testing does not cover structure-level problems either. So it can be more difficult for the development team to identify and fix the problems found during the testing process. Additionally, the test results are dependent on the tester’s individual expertise with the program. If the testing pool isn’t particularly good, the test may yield incomplete results.

Sanity Testing vs. Smoke Testing: What’s the Difference?

Another test performed during the building of a program is smoke testing. Smoke testing is designed to test the stability of critical functionality. While these two tests might sound very similar initially, they’re typically performed at different points in the development process. Here are the major differences between sanity testing and smoke testing.

  • Sanity tests are designed to test the new functionalities introduced to a build, whereas smoke testing is designed to check the stability of critical functionality.
  • The purpose of sanity tests is to determine the “rationality” of the build, yet the purpose of smoke testing is to determine the “stability” of the build.
  • Sanity testing is a subset of regression testing, while smoke testing is a subset of acceptance testing.
  • Sanity testing is only performed by testers, but either developers or testers can perform smoke testing.
  • Sanity testing is typically undocumented and unscripted, but smoke testing is typically documented or scripted.
  • Sanity testing exercises only particular portions of the program, but smoke testing exercises the entire program from end to end.

Sanity Testing vs. Regression Testing: What’s the Difference?

While sanity tests are a subset of regression testing, they are not a replacement for actual regression testing. Regression testing is a crucial part of end-stage development that helps ensure the final product is suitable for users. Here are the differences between these two tests.

  • Sanity testing can be performed on a build of any stability, but regression testing is only performed on builds that have been declared to be in a stable development stage.
  • Sanity testing exercises the newly added functions to ensure their independent functionality, however regression testing exercises the functionality of all affected program areas that have been updated.
  • Sanity testing covers only certain modules and functions for which changes have been made, whereas regression testing covers all functionality of modules affected by the recent changes.
  • Human testers typically perform sanity testing, while regression testing is typically performed by automation with oversight by human testers.
  • Sanity testing is often performed when there is insufficient time to perform a full regression test, but regression testing will be performed when testers have ample time to investigate the build’s functionality.

Final Thoughts

When building a new software product, testing is crucial to ensure that the final product delivers the experience you want your users to have. To ensure the best results once you launch, sanity testing is just one of many types of tests you’ll want to employ when building your program. If you learned something new about the process of building a software product, drop us a comment here or on social media! We love to hear from our readers!

Summary Table

Testing TypeDescription
Sanity TestingBasic regression testing to find and report crucial bugs in the software build
Smoke TestingTests the stability of critical functionality in the software build
Regression TestingComprehensive testing to ensure the final product is suitable for users

Frequently Asked Questions

What is sanity testing?

Sanity testing is a type of test performed on software programs that have introduced new functions.

How do I sanity test my program?

Sanity testing is different based on what type of program you’re building and what functions you’re adding to the program. For instance, sanity testing a sign-up page will typically involve attempting several different case scenarios that might occur when a user attempts to sign up for your website.

What is the difference between sanity testing and smoke testing?

Smoke testing is used to determine the stability of all core functionality. Sanity testing determines the functionality of new features.

What is the difference between sanity testing and regression testing?

Sanity tests exercise newly added functionality. Regression tests are more exhaustive about testing use cases for all functions affected by the update.

What is the difference between sanity testing and acceptance testing?

Acceptance testing determines whether the product meets the user’s requirements. It is performed by prospective users or representatives. Sanity testing tests the functionality of new features.

To top