Sr.No. | Verification | Validation |
---|---|---|
1 | Verification addresses the concern: 'Are you building it right?' | Validation addresses the concern: 'Are you building the right thing?' |
2 | Ensures that the software system meets all the functionality. | Ensures that the functionalities meet the intended behavior. |
3 | Verification takes place first and includes the checking for documentation, code, etc. | Validation occurs after verification and mainly involves the checking of the overall product. |
4 | Done by developers. | Done by testers. |
5 | It has static activities, as it includes collecting reviews, walkthroughs, and inspections to verify a software. | It has dynamic activities, as it includes executing the software against the requirements. |
6 | It is an objective process and no subjective decision should be needed to verify a software. | It is a subjective process and involves subjective decisions on how well a software works. |
Quality Assurance | Quality Control | Testing |
---|---|---|
QA includes activities that ensure the implementation of processes, procedures and standards in context to verification of developed software and intended requirements. | It includes activities that ensure the verification of a developed software with respect to documented (or not in some cases) requirements. | It includes activities that ensure the identification of bugs/error/defects in a software. |
Focuses on processes and procedures rather than conducting actual testing on the system. | Focuses on actual testing by executing the software with an aim to identify bug/defect through implementation of procedures and process. | Focuses on actual testing. |
Process-oriented activities. | Product-oriented activities. | Product-oriented activities. |
Preventive activities. | It is a corrective process. | It is a preventive process. |
It is a subset of Software Test Life Cycle (STLC). | QC can be considered as the subset of Quality Assurance. | Testing is the subset of Quality Control. |
Sr.No | Standard & Description |
---|---|
1 | IEEE 829 A standard for the format of documents used in different stages of software testing. |
2 | IEEE 1061 A methodology for establishing quality requirements, identifying, implementing, analyzing, and validating the process, and product of software quality metrics. |
3 | IEEE 1059 Guide for Software Verification and Validation Plans. |
4 | IEEE 1008 A standard for unit testing. |
5 | IEEE 1012 A standard for Software Verification and Validation. |
6 | IEEE 1028 A standard for software inspections. |
7 | IEEE 1044 A standard for the classification of software anomalies. |
8 | IEEE 1044-1 A guide for the classification of software anomalies. |
9 | IEEE 830 A guide for developing system requirements specifications. |
10 | IEEE 730 A standard for software quality assurance plans. |
11 | IEEE 1061 A standard for software quality metrics and methodology. |
12 | IEEE 12207 A standard for software life cycle processes and life cycle data. |
13 | BS 7925-1 A vocabulary of terms used in software testing. |
14 | BS 7925-2 A standard for software component testing. |
Advantages | Disadvantages |
---|---|
Well suited and efficient for large code segments. | Limited coverage, since only a selected number of test scenarios is actually performed. |
Code access is not required. | Inefficient testing, due to the fact that the tester only has limited knowledge about an application. |
Clearly separates user's perspective from the developer's perspective through visibly defined roles. | Blind coverage, since the tester cannot target specific code segments or errorprone areas. |
Large numbers of moderately skilled testers can test the application with no knowledge of implementation, programming language, or operating systems. | The test cases are difficult to design. |
Advantages | Disadvantages |
---|---|
As the tester has knowledge of the source code, it becomes very easy to find out which type of data can help in testing the application effectively. | Due to the fact that a skilled tester is needed to perform white-box testing, the costs are increased. |
It helps in optimizing the code. | Sometimes it is impossible to look into every nook and corner to find out hidden errors that may create problems, as many paths will go untested. |
Extra lines of code can be removed which can bring in hidden defects. | It is difficult to maintain white-box testing, as it requires specialized tools like code analyzers and debugging tools. |
Due to the tester's knowledge about the code, maximum coverage is attained during test scenario writing. |
Advantages | Disadvantages |
---|---|
Offers combined benefits of black-box and white-box testing wherever possible. | Since the access to source code is not available, the ability to go over the code and test coverage is limited. |
Grey box testers don't rely on the source code; instead they rely on interface definition and functional specifications. | The tests can be redundant if the software designer has already run a test case. |
Based on the limited information available, a grey-box tester can design excellent test scenarios especially around communication protocols and data type handling. | Testing every possible input stream is unrealistic because it would take an unreasonable amount of time; therefore, many program paths will go untested. |
The test is done from the point of view of the user and not the designer. |
Black-Box Testing | Grey-Box Testing | White-Box Testing |
---|---|---|
The internal workings of an application need not be known. | The tester has limited knowledge of the internal workings of the application. | Tester has full knowledge of the internal workings of the application. |
Also known as closed-box testing, test and partly to provide a preview of the next release. In this phase, the audience will be testing the following −
Non-Functional TestingThis section is based upon testing an application from its non-functional attributes. Non-functional testing involves testing a software from the requirements which are nonfunctional in nature but important such as performance, security, user interface, etc. Some of the important and commonly used non-functional testing types are discussed below. Performance TestingIt is mostly used to identify any bottlenecks or performance issues rather than finding bugs in a software. There are different causes that contribute in lowering the performance of a software −
Performance testing is considered as one of the important and mandatory testing type in terms of the following aspects −
Performance testing can be either qualitative or quantitative and can be divided into different sub-types such as Load testing and Stress testing. Load TestingIt is a process of testing the behavior of a software by applying maximum load in terms of software accessing and manipulating large input data. It can be done at both normal and peak load conditions. This type of testing identifies the maximum capacity of software and its behavior at peak time. Most of the time, load testing is performed with the help of automated tools such as Load Runner, AppLoader, IBM Rational Performance Tester, Apache JMeter, Silk Performer, Visual Studio Load Test, etc. Virtual users (VUsers) are defined in the automated testing tool and the script is executed to verify the load testing for the software. The number of users can be increased or decreased concurrently or incrementally based upon the requirements. Stress TestingStress testing includes testing the behavior of a software under abnormal conditions. For example, it may include taking away some resources or applying a load beyond the actual load limit. The aim of stress testing is to test the software by applying the load to the system and taking over the resources used by the software to identify the breaking point. This testing can be performed by testing different scenarios such as −
Usability TestingUsability testing is a black-box technique and is used to identify any error(s) and improvements in the software by observing the users through their usage and operation. According to Nielsen, usability can be defined in terms of five factors, i.e. efficiency of use, learn-ability, memory-ability, errors/safety, and satisfaction. According to him, the usability of a product will be good and the system is usable if it possesses the above factors. Nigel Bevan and Macleod considered that usability is the quality requirement that can be measured as the outcome of interactions with a computer system. This requirement can be fulfilled and the end-user will be satisfied if the intended goals are achieved effectively with the use of proper resources. Molich in 2000 stated that a user-friendly system should fulfill the following five goals, i.e., easy to Learn, easy to remember, efficient to use, satisfactory to use, and easy to understand. In addition to the different definitions of usability, there are some standards and quality models and methods that define usability in the form of attributes and sub-attributes such as ISO-9126, ISO-9241-11, ISO-13407, and IEEE std.610.12, etc. UI vs Usability TestingUI testing involves testing the Graphical User Interface of the Software. UI testing ensures that the GUI functions according to the requirements and tested in terms of color, alignment, size, and other properties. On the other hand, usability testing ensures a good and user-friendly GUI that can be easily handled. UI testing can be considered as a sub-part of usability testing. Security TestingSecurity testing involves testing a software in order to identify any flaws and gaps from security and vulnerability point of view. Listed below are the main aspects that security testing should ensure −
Portability TestingPortability testing includes testing a software with the aim to ensure its reusability and that it can be moved from another software as well. Following are the strategies that can be used for portability testing −
Portability testing can be considered as one of the sub-parts of system testing, as this testing type includes overall testing of a software with respect to its usage over different environments. Computer hardware, operating systems, and browsers are the major focus of portability testing. Some of the pre-conditions for portability testing are as follows −
Testing documentation involves the documentation of artifacts that should be developed before or during the testing of Software. Documentation for software testing helps in estimating the testing effort required, test coverage, requirement tracking/tracing, etc. This section describes some of the commonly used documented artifacts related to software testing such as −
Test PlanA test plan outlines the strategy that will be used to test an application, the resources that will be used, the test environment in which testing will be performed, and the limitations of the testing and the schedule of testing activities. Typically the Quality Assurance Team Lead will be responsible for writing a Test Plan. A test plan includes the following −
Test ScenarioIt is a one line statement that notifies what area in the application will be tested. Test scenarios are used to ensure that all process flows are tested from end to end. A particular area of an application can have as little as one test scenario to a few hundred scenarios depending on the magnitude and complexity of the application. The terms 'test scenario' and 'test cases' are used interchangeably, however a test scenario has several steps, whereas a test case has a single step. Viewed from this perspective, test scenarios are test cases, but they include several test cases and the sequence that they should be executed. Apart from this, each test is dependent on the output from the previous test. Test CaseTest cases involve a set of steps, conditions, and inputs that can be used while performing testing tasks. The main intent of this activity is to ensure whether a software passes or fails in terms of its functionality and other aspects. There are many types of test cases such as functional, negative, error, logical test cases, physical test cases, UI test cases, etc. Furthermore, test cases are written to keep track of the testing coverage of a software. Generally, there are no formal templates that can be used during test case writing. However, the following components are always available and included in every test case −
Many test cases can be derived from a single test scenario. In addition, sometimes multiple test cases are written for a single software which are collectively known as test suites. Traceability MatrixTraceability Matrix (also known as Requirement Traceability Matrix - RTM) is a table that is used to trace the requirements during the Software Development Life Cycle. It can be used for forward tracing (i.e. from Requirements to Design or Coding) or backward (i.e. from Coding to Requirements). There are many user-defined templates for RTM. Each requirement in the RTM document is linked with its associated test case so that testing can be done as per the mentioned requirements. Furthermore, Bug ID is also included and linked with its associated requirements and test case. The main goals for this matrix are −
Estimating the efforts required for testing is one of the major and important tasks in SDLC. Correct estimation helps in testing the software with maximum coverage. This section describes some of the techniques that can be useful in estimating the efforts required for testing. Functional Point AnalysisThis method is based on the analysis of functional user requirements of the software with the following categories −
Test Point AnalysisThis estimation process is used for function point analysis for black-box or acceptance testing. The main elements of this method are: Size, Productivity, Strategy, Interfacing, Complexity, and Uniformity. Mark-II MethodIt is an estimation method used for analyzing and measuring the estimation based on end-user’s functional view. The procedure for Mark-II method is as follows −
MiscellaneousYou can use other popular estimation techniques such as −
|