The importance of quality assurance management: why you should consider it

Despite the fact that the role of software testing may seem insignificant and few of us ever even think about the importance of software testing, this is especially true for people who are new to the software development lifecycle; the testing process, which is an equally integral part of development; and planning. Sooner or later, many organizations that have developed software come to the realization that they need to organize a quality assurance management system.

Any manager that faces the task of organizing the testing process has to ask the following questions:

• What is software testing?
• What is quality?
• What are the objectives and the purpose of testing?
• What is the importance of quality assurance, and is it really needed?
• When should we test?
• How and what should be tested?

What is software testing?

Software testing is a process of investigation and experiencing the software in order to obtain information about its quality. Testing is only the first step in determining the quality of software. It’s just one component of the more substantial concept of Quality Control. QC is responsible for measuring the quality of the product, analyzing the test results and quality of releases or product builds, and collecting and analyzing quality metrics. In other words, QC processes and organizes information obtained during development and testing to improve the quality of the product. Additionally, QC is a part of Quality Assurance. QA solves increasingly global problems, and chief among them is management of the process quality itself. QA strives to anticipate and prevent potential problems. The responsibilities of QA include the selection of practices, methods, approaches, tools, and so on. To get information about the product quality, it is first necessary to understand what exactly “quality” is.

Quality of a software product

Many people ask what is meant by the “quality” of a software product. This question has more than one answer because the quality is of value to the individual and is therefore subjective. Each user has their own requirements for the product, and for one user, this product may be of high quality and will fully satisfy their needs, but for another user, this product will be useless, and the user will simply remove the software and will no longer want to use it in the future. Look at the image below to try to determine which car is the better quality for you.

Remember that requirements are different for different people. Some people may choose the blue car because their only requirement is that the car can be driven. Other people will pick the white car because it is attractive, comfortable, fashionable, fast, safe, etc.

In order to determine the quality of the product, you first need to understand what this product is needed for and whether it meets the customer’s/user’s requirements. To summarize and answer the question, “What is the quality of the software?” quality is the degree of compliance of inherent characteristics in combination with the requirements. In other words, does this product do all the things that were stated in the requirements?

Objectives of a quality assurance management system and the purpose of testing

The objectives of a quality assurance management system are the following:

1. Detect defects and discover any situations where the software behaves incorrectly or in an undesirable manner.
2. Verify the product’s compliance with the stated requirements.
3. Provide feedback about the defects to all stakeholders.
4. Reduce the cost of development.

The purpose of advanced testing is not only to detect bugs in the software but also to identify their causes. This approach allows developers to operate with maximum efficiency, quickly eliminating any bugs that occur. Testing solves several major problems:

• it confirms that all the declared functional requirements have been implemented;
• it gives confidence in the quality of the final product;
• it confirms that the application has no errors in the code;
• it verifies how the user can intuitively work with the application and also ensures that your product is easy to use and does not contain obscure features that may mislead users;
• testing ensures that stored and processed data is protected from unauthorized access and hacking;
• it confirms that the application is able to operate in all the required conditions and on all supported operating systems or web browsers correctly. 

Importance of quality assurance

One of the major problems in the software development business is misunderstandings between the customer and the team and, of course, incomplete, contradictory specifications for the project. To avoid this, it is important to involve the test engineer in the project as early as possible. A qualified test engineer must be attentive, be able to analyze the requirements, ask a lot of questions of the customer, and try to find as many contradictions in the original documentation as possible. It is very important to keep constant communication with the customer and to document all the answers and any clarifications. In this way, a proper understanding of the project will be achieved, and you will also avoid possible conflicts in the future.

Also, it is very important to give recommendations for solving any disagreements. We must always keep in mind that not all customers are technically minded. Their solutions for some emerging problems are not always good and certainly not optimal. The customer often thinks about how much he likes the software and how much this software satisfies his own requirements. But the customer is not always thinking about how convenient the functionality of the product is or how attractive the user interface of this software will be to the end user. So the quality assurance management system is essential for solving all of these problems.

 The visualization of misunderstandings in development

When to start testing

In Appus QA tester is involved in the process of software product development at the stage of requirements analysis and wireframes creation. The earlier the test engineer is involved in the project, the cheaper any mistakes will be. Fixing the mistake in the design or specification stage takes much less time than correcting the mistake in the code, when the application is already functioning. Fixing even one mistake in the code may result in the emergence of new major errors, and debugging can take anywhere from several hours to several days. And as you know, time is money.

Price of mistakes on different stages of development

Testing process at Appus Studio

The mobile app testing process at our company is divided into several stages:

1. Requirements analysis and testing of flowchart/wireframes.
2. Drafting the test documentation (test cases, checklists, test plan).
3. Functional testing of application/website (iOS, Android, Web).
4. User interface testing.
5. Bugs reporting (bug reports creation).
6. Acceptance and regression testing, verification of bug fixes. 

Requirements analysis and testing of flowchart/wireframes

The example of the flowchart

The Quality Assurance engineer analyzes the requirements for completeness and any contradictions. The initial requirements of each project contain the conflicting information and errors, and we solve these before the start of development. Also, in each project, requirements are incomplete and lack layouts of secondary screens, have restrictions on the input fields, display error messages, and have buttons that lead to nowhere. Ambiguous things are unseen on the mockups: animations, images and content caching, work in non-standard situations.

If the map of application (flowchart/wireframes) was developed by one of our designers, the software test engineer will find all the errors, create a report about the testing of wireframes, and then make any corrections. All changes are then made available to the customer for approval. Any disadvantages of the requirements are discussed with the project manager, developers, and designers. After several iterations, the whole team will have a better understanding of the project.

Test documentation

All products have a need for quality assurance. To provide the correct testing process and to make sure that your product is truly high quality and meets any necessary requirements, we use the following documents:

• Test plan (a document describing the objectives, approach, resources, and schedule of planned test activities);
• Test cases (artifacts that describe a set of steps, the specific conditions and parameters, the expected results, and requirements to check the implementation of the tested function or a part of it);
• Checklists (analogy of test cases that briefly describe the test);
• Bug reports (documents that describe a situation or sequence of actions that led to the incorrect work of the test object).

Functional testing and user interface testing

We use such QA testing methodologies as the waterfall model and iterative model. Each model includes functional testing, user interface testing, performance testing, usability testing, and compatibility testing. During the functional testing of the application, the workability of the main functions, their suitability for use, and their compliance with the functional specifications must be checked and tested. For each function, both the positive and negative scenarios must be performed as well.

The scenario is considered to be a positive one if the user finally reaches their target (creates an item, sends a message, etc.). On the contrary, a negative scenario would have an entry error that is specifically permitted (the Internet connection is turned off, an invalid email has been entered, etc.) and the purpose cannot be achieved at some of the steps.

We use real devices (iPhone 4s – iPhone 6+ with iOS 9+ and Samsung, LG, HTC with Android 4.0+) for testing in our company. We also use the following quality testing tools: Genymotion (Android emulator), iOS Simulator (available only for Mac OS), xCode, and Android Studio.

Also, our developers build into our apps such crash reporting systems as Fabric Crashlytics. All this software allows for finding memory leaks, getting logs from the device, debugging, etc.  Basically, we check the following features:

• how the application works on different actual versions of firmware (iOS 8 – 10, Android 4 – 7, Chrome, Safari, Firefox browsers) and different generations of devices (e.g., iPhone 4s – iPhone 7, different Android devices);
• validation of mandatory text fields and validation of email/password fields. The user should be notified if one of the mandatory fields or the email/password fields aren’t filled with valid values;
• the response of the application to the privacy settings (location services are turned off, access to the camera or microphone, calendar is not allowed, etc.);
• the orientation (portrait/landscape) modes of the application;
• the application on different screen resolutions;
• that the user interface of the application corresponds to the mockups;
• the push notifications (if they exist);
• that the application works in multitasking mode;
• sharing to social networking (Facebook, Twitter, Instagram, etc.) if this functionality exists in the app;
• that the application supports payment transactions through payment systems such as Visa, MasterCard, PayPal, and others;
• scrolling and navigating through the app;
• existence of alert/error messages: “Network error. Please try again later”, “Are you sure you want to log out?” “Are you sure you want to delete the item?” etc;
• regression testing to identify the new programming mistakes in existing and already modified areas of the application as well as additional execution of all previous tests to check the behavior of the program after changes.

Compliance with the mockup

Bugs reporting

We use own bug-tracking system that was developed by Appus, called “Corporate” (previously Corporate was mentioned in the earlier post). Any bugs that were found during the mobile app testing process are added to the system, and each bug that was created then has the status of “New”. The test engineer or project manager also must to set the priority of the bug so others understand how quickly it should be fixed. Once the developer has started to fix the bug, he or she changes the status of the bug report to “In Progress” and begins to fix the bug. If the bug was reproduced and fixed by the developer, the status of the bug report is changed to “Completed”, and then a test engineer will have to check and verify the fix. If the bug was successfully fixed and the test engineer was unable to reproduce it again, the status is changed to “Closed”; if not, he or she writes a comment and changes the status to “Feedback”.

In order to get a screenshot from device, we use the “Android file transfer” or “Image Capture” tools on Mac OS. If we need to capture a video from a device, we use the “QuickTime” player and “Android Studio” or another device’s camera to manually record the video with steps to reproduce the bug. Then default image editing tools are used to highlight the problematic area of the app on the screenshot. Here’s an example of a bug report in Corporate:

Bug report in Appus corporate system

Acceptance and regression testing

Acceptance testing in our company is a formal testing process that verifies compliance with the requirements of the system and is aimed at the following:

1. Determining whether the system meets the acceptance criteria.
2. A decision by the customer or another authorized person concerning whether the application is accepted or not.

During the regression testing, we check that fixed bugs do not cause any new problems and do not influence the rest of the functionality in the different parts of the app.

At the end of the testing process, we move the application from our development server (if the web server is required) to the production server, where the customer can personally test the application on his own server and give us feedback. We also have a “Versions” tab in our corporate system. When we reach the milestone and development is finished, the developer generates a new build and uploads it to either Corporate or Testflight. The test engineer downloads the application and verifies the build to see if it matches the milestone description and works well. Then the test engineer approves or declines the build depending on the status of the build. Our clients get email notifications about new approved versions, and these versions also can be downloaded and tested by the client. 

In conclusion

If the code has not been tested, it is considered to be not working. Therefore, if you are developing, you should be aware that the importance of quality assurance is crucial, and you should not avoid the testing process. It is better to pay for testing and identify errors at the development stage than to have to pay several times more to fix errors during the release phase. According to experts, testing is a gradually evolving process, and it should confirm the reliability, usability, efficiency, and safety of the product.

Testing is not a single action, and it usually accompanies the entire process of product development. By constantly analyzing and evaluating the developed software with a variety of approaches, test engineers identify and correct defects in order to ensure the high quality of the final product. In this case, not only is the fact of detection of an error in the software product important but also the reason for its occurrence. This will prevent the occurrence of similar problems in the later stages of product development.