Difference Between UAT and SIT - 7 Challenges of UAT

#Rajeev Prabhakaran Nair

User acceptance Testing is performed by the Client of the application to determine whether the application is developed as per the requirements specified by him/her. It is performed within the development of the organization or at the client site. Alpha testing and Beta testing are the examples of Acceptance Testing.

System testing is performed by the Testers to determine whether the application is satisfying the requirements as specified in SRS. (System Requirement Specification)

#Peter the 'One'

System integration testing is testing performed when two systems, generally presumed stable themselves, are integrated with one another. For example, this could be when an inventory management system is integrated with a sales accounting system. Each system feeds into the other.

The goal of systems integration testing is to ensure that the data crossing the boundary between systems is received, stored and used appropriately by the receiving system. Until integration begins, testing of the isolated systems is done on mocked or replayed data and not on "live" data. Integration testing is the final step before customer acceptance testing.

What is User Acceptance Testing?

User Acceptance Testing is often the final step before rolling out the application.

Usually the end users who will be using the applications test the application before ‘accepting’ the application.

This type of testing gives the end users the confidence that the application being delivered to them meets their requirements.

This testing also helps nail bugs related to usability of the application.

User Acceptance Testing – Prerequisites:

Before the User Acceptance testing can be done the application is fully developed.
Various levels of testing (Unit, Integration and System) are already completed before User Acceptance Testing is done. As various levels of testing have been completed most of the technical bugs have already been fixed before UAT.

User Acceptance Testing – What to Test?

To ensure an effective User Acceptance Testing Test cases are created.
These Test cases can be created using various use cases identified during the Requirements definition stage.
The Test cases ensure proper coverage of all the scenarios during testing.

During this type of testing the specific focus is the exact real world usage of the application. The Testing is done in an environment that simulates the production environment.
The Test cases are written using real world scenarios for the application

User Acceptance Testing – How to Test?

The user acceptance testing is usually a black box type of testing. In other words, the focus is on the functionality and the usability of the application rather than the technical aspects. It is generally assumed that the application would have already undergone Unit, Integration and System Level Testing.

However, it is useful if the User acceptance Testing is carried out in an environment that closely resembles the real world or production environment.

The steps taken for User Acceptance Testing typically involve one or more of the following:

1. User Acceptance Test (UAT) Planning
2. Designing UA Test Cases
3. Selecting a Team that would execute the (UAT) Test Cases
4. Executing Test Cases
5. Documenting the Defects found during UAT
6. Resolving the issues/Bug Fixing
7. Sign Off 

#Software Testing Help

What is User Acceptance Testing (UAT)?
UAT is final testing performed when functional, system and regression testing is completed. The main purpose of UAT is to validate the software against the business requirements. This validation is carried out by end users who are familiar with the business requirements. UAT, alpha and beta testing are different types of acceptance testing.

As user acceptance testing is the last testing carried out before the software goes live, obviously this is the last chance for the customer to test the software and measure if it’s fit for the purpose.

Need of User Acceptance Testing
Developers and functional testers are technical people who validate the software against the functional specifications. They interpret the requirements according to their knowledge and develop/test the software (here is the importance of domain knowledge). This software is complete according to the functional specifications but there are some business requirements and processes which are known to end users only are either missed to communicate or misinterpreted. UAT plays an important role in validating if all the business requirements are fulfilled or not before releasing software for the market use. Use of live data and real use cases make UAT testing an important part of the release cycle.

Many businesses who suffered big losses due to post-release issues know the importance of successful UAT. The cost of fixing defects after release is many times greater than fixing it before.

Who should be part of the UAT team?
Mainly end users of the software should be involved in UAT. The team can be comprised of beta testers or customer should select UAT members internally from every group of organization so that each and every user role can be tested.

7 Major Challenges of UAT And Mitigation Plan To Overcome These

It doesn’t matter if you are part of a billion dollar release or a startup team, you should overcome these UAT challenges for delivering successful software for the end user.

1) UAT environment setup and deployment process
Carrying out UAT on same environment used by functional test team will certainly end up overlooking real world use cases. Also crucial testing activities like performance testing can’t be carried out on test environment with incomplete test data. Separate production like environment should be setup for UAT.

Once the UAT environment is separated from test environment you need to control the release cycle effectively. Uncontrolled release cycle may leads to different software versions on test and UAT environment. Valuable UAT time is wasted by not testing software on latest version. Not to mention the time required for issue tracking on incorrect software version.

2) UAT test planning
UAT should be planned with clear acceptance test plan in requirement analysis and design phase. In strategy planning, set of real world use cases should be identified for execution. It is very important to define test objectives for UAT testing as complete test execution for large application is not possible in UAT phase. Testing should be carried out by prioritizing critical business objectives first.

UAT is carried out at the end of the testing cycle. Obviously it is the most critical period for the software release. Delay in any of the previous stages of development and testing eat up UAT time. Improper test planning, in worst cases, leads to overlap between system testing and UAT. Due to less time for UAT and pressure to meet deadlines, software is deployed to UAT environment even though functional testing is not completed. UAT goals can’t be achieved in such situations.

UAT test plan (template) should be prepared and communicated to team well before beginning UAT. This will help them for test planning, writing test cases and test scripts and creating UAT environment.

3) Handling new business requirements as incidents/defects
Ambiguities in requirements get caught in UAT phase. UAT testers find issues arising due to ambiguous requirements (by looking at the complete UI which wasn’t available during requirement gathering phase) and log it as a defect. Customer expects these to be fixed in current release without considering the time for change requests. If timely decision is not taken by project management on these last-minute changes this could lead to the release failure.

4) Unskilled testers or testers without business knowledge
If there is no permanent UAT test team, company select UAT staff from various internal departments. Even if this staff is well familiar with business needs, if they are not trained for the new requirements being developed they can’t perform effective UAT. Also non-technical business team might face many technical difficulties executing UAT test cases. Also assigning testers at the end of the UAT cycle do not add any value to the project. Little time to train UAT staff can significantly increase the chances of UAT success.

5) Improper Communication channel
Communication between remote development, testing and UAT team is more difficult. Email communication is often very difficult when you have an offshore tech team. A small ambiguity in incident reports can delay its fix for a day. Proper planning and effective communication are critical to effective team collaboration. Project teams should use web based tool to log defects and questions. This will help to distribute work load evenly and avoid reporting duplicate issues.

6) Asking functional test team to perform UAT
There is no worse situation than asking functional test team to perform UAT. Customers offload their responsibility to test team due to lack of resources. The whole purpose of UAT gets compromised in such cases. Once the software goes live, end users will quickly spot the issues which are not considered as real world scenarios by functional testers. Solution to this is to assign UAT to dedicated and skilled testers having business knowledge.

7) The blame game
Sometimes business users just try to find reasons to reject the software. It might be their selfdom to show how superior they are or blame the development and testing team to get respect in business team. This is very rare but happens in teams with internal politics. It’s very difficult to deal with such situations. Building positive relationship with business team would definitely help to avoid blame game.