We expect this product to be tested in several phases. We broke the integration testing into five phases which are listed as follow:
1. Mapping Function module
In this phase, we are testing the Mapping Function (MF) module only. An automated script will be used and will be executed to test the MF REST service which includes:
2. Administrative Web Application (including REST service & database)
In this phase, we are testing the web application only. The web application will be connected to a Mapping simulator to provide predefined walking tour and map simulation, so we can focus on testing all the functionalities of the web application, REST service and the database. Main test cases include account registration & login, user workflow and tour developer workflow.
3. Mobile Application
In this phase, we are testing the mobile application. The mobile application will be connected to the real REST service (in Phase 2) but will be connected to a Mapping simulator to provide predefined walking tour and map simulation, so we can focus on testing all the functionalities of the mobile application and the interaction with the REST service.
4. Web Application & Mapping Function
In this phase, we are doing real testing between web application and mapping function. We will cover the account registration & login, user workflow and tour developer workflow.
5. Mobile Application & Mapping Function
In this phase, we are doing real testing between mobile application and mapping function. Similar to phase 4, we will cover similar tests using the mobile application with an additional testing of downloading the package for offline use.
Once the integration testing is completed and WalkingStick system artifact is deployed in performance environment, the performance test will run to ensure the following:
The performance test will provide multiple types of metrics for each functional requirement, such as:
Load test, which will provide system performance metrics during simulation of estimated number of users using system concurrently
Unit performance test, which will provide metrics on each feature sub-element performance breakdown, such as database query execution, sorting functions, etc.
Soak test, which will provide metrics on the system performance during the gradual system load increase
Stress test, which will provide metrics on system performance during peak activity load
Spike test, which will provide metrics on the system during sudden and unexpected activity spikes.
Each performance test type will cover the following set of test cases:
PF-1. User login
PF-2. Select Tour
PF-3. Search Tour
PF-4. Follow Walking Directions
PF-5. Save Walking Tour
PF-6. Download Walking Tour
PF-7. Create Walking Tour
PF-8. Edit Walking Tour
The duration and number of requests (samples) will vary depending on test type. For example, load test will have a tolerable number of requests run over an extensive period of time and spike tests will have abnormally large number of requests run in a short period of time.
The test results will be analyzed and areas of concern will be addressed to ensure the product is up to required quality standards.
1. Mapping Function module
In this phase, we are testing the Mapping Function (MF) module only. An automated script will be used and will be executed to test the MF REST service which includes:
- Testing the conversion of address to GPS coordinates
- Testing the walking direction between coordinates.
- Testing the retrieval of map image tiles
2. Administrative Web Application (including REST service & database)
In this phase, we are testing the web application only. The web application will be connected to a Mapping simulator to provide predefined walking tour and map simulation, so we can focus on testing all the functionalities of the web application, REST service and the database. Main test cases include account registration & login, user workflow and tour developer workflow.
3. Mobile Application
In this phase, we are testing the mobile application. The mobile application will be connected to the real REST service (in Phase 2) but will be connected to a Mapping simulator to provide predefined walking tour and map simulation, so we can focus on testing all the functionalities of the mobile application and the interaction with the REST service.
4. Web Application & Mapping Function
In this phase, we are doing real testing between web application and mapping function. We will cover the account registration & login, user workflow and tour developer workflow.
5. Mobile Application & Mapping Function
In this phase, we are doing real testing between mobile application and mapping function. Similar to phase 4, we will cover similar tests using the mobile application with an additional testing of downloading the package for offline use.
Performance Testing
Once the integration testing is completed and WalkingStick system artifact is deployed in performance environment, the performance test will run to ensure the following:
- WalkingStick pages loading time
- Response/request time for the each of the user flows corresponding to listed functional requirements
- Ensure the tolerance of the system with the concurrent requests to ensure to support estimated number of users using system at the same time
- Validate the system resources consumption (CPU, RAM, etc) and insure correct error handling
The performance test will provide multiple types of metrics for each functional requirement, such as:
Load test, which will provide system performance metrics during simulation of estimated number of users using system concurrently
Unit performance test, which will provide metrics on each feature sub-element performance breakdown, such as database query execution, sorting functions, etc.
Soak test, which will provide metrics on the system performance during the gradual system load increase
Stress test, which will provide metrics on system performance during peak activity load
Spike test, which will provide metrics on the system during sudden and unexpected activity spikes.
Each performance test type will cover the following set of test cases:
PF-1. User login
PF-2. Select Tour
PF-3. Search Tour
PF-4. Follow Walking Directions
PF-5. Save Walking Tour
PF-6. Download Walking Tour
PF-7. Create Walking Tour
PF-8. Edit Walking Tour
The duration and number of requests (samples) will vary depending on test type. For example, load test will have a tolerable number of requests run over an extensive period of time and spike tests will have abnormally large number of requests run in a short period of time.
The test results will be analyzed and areas of concern will be addressed to ensure the product is up to required quality standards.