#manualregressiontesting

There are many reasons that motivate application owners to modernize the existing code base. Some of the objectives outlined by most of our top clients includes the following:

Making the codes safer to change in the future without breaking functionality

Ensuring the code becomes more container friendly and is easy to update

Moving to less expensive and easy-to-maintain application server

Moving off from a deprecated version of Java to the most updated version

Migrating from a hybrid codebase to traditional java code

Ensuring codes are testable and writing tests become faster, easier

Ensuring codes and configurations are more container friendly

For most of our clients, another key roadblock is finding the talent and skills to write manual test cases. When the state of the code undergoes many changes over the years, it can create problems in adding test coverage. Therefore, in many applications, it may also require manual creation of test cases.

In most cases, our clients come to us because their application teams have to deal with problems of legacy code modernization and testing. Legacy code is typically code that has been in use for a long time and may not have been well-documented, often-times lacking automated tests.

Here’s what makes preparation of manual test cases necessary:

Lack of Automated Tests

No Existing Test Coverage: Legacy code often lacks automated tests, which means that there is no existing suite of tests to rely on. Writing manual test cases helps ensure that the code behaves as expected before any changes are made.

Gradual Test Automation: While the long-term goal might be to automate testing, starting with manual test cases allows for immediate validation and helps in identifying critical areas for automated test development.

Understanding Complex and Untested Code

Code Complexity: Legacy systems can be complex and difficult to understand, especially if the code has evolved over time without proper refactoring. Manual testing allows testers to interact with the system in a way that automated tests may not easily facilitate, helping to uncover edge cases and unexpected behavior.

Exploratory Testing: Manual testing allows for exploratory testing, where testers can use their intuition and experience to find issues that are not covered by predefined test cases. This is particularly important in legacy systems where the code’s behavior might be unpredictable.

High Risk of Breaking Changes

Fragility of Legacy Systems: Legacy code is often fragile, and small changes can lead to significant issues elsewhere in the system. Manual test cases allow for careful and deliberate testing, reducing the risk of introducing breaking changes.

Regression Testing: Manual regression testing is often necessary to ensure that new changes do not negatively impact existing functionality, especially when automated regression tests are not available.

Lack of Documentation

Poor or Outdated Documentation: Legacy code is often poorly documented, if at all. Manual test cases can serve as a form of documentation, helping developers and testers understand the expected behavior of the system.

Knowledge Transfer: Manual test cases can also help in knowledge transfer, especially when working with code that was originally written by developers who are no longer with the organization.

Limited Tooling and Automation Compatibility

Incompatibility with Modern Tools: Legacy systems may not be compatible with modern testing frameworks and tools, making it difficult to implement automated testing without significant investment in refactoring or tool adaptation. In such cases, manual testing might be the most feasible option.

Custom or Proprietary Systems: If the legacy code is part of a custom or proprietary system, existing automated testing tools might not work out of the box, necessitating manual test case development.

Interdependencies with Other Legacy Systems

Complex Interactions: Legacy code often interacts with other legacy systems, and the complexity of these interactions may not be fully understood. Manual testing allows testers to observe and verify the behavior of the system as a whole, which can be difficult to achieve with automated tests alone.

End-to-End Testing: Manual end-to-end testing is often necessary in legacy environments to ensure that all components of the system work together as expected.

Identifying Test Scenarios for Automation

Test Case Identification: Writing manual test cases helps identify critical and high-value test scenarios that should be automated in the future. This can serve as a roadmap for gradually building an automated test suite.

Incremental Automation: Starting with manual tests allows teams to prioritize and incrementally automate the most important or frequently executed test cases.

Case Study: Executing Manual Regression Testing for a Legacy Collateral Management System of a Federal bank

Background:

A top Federal Bank had been using a legacy loan advances system for over a decade. The system was originally developed in COBOL and has undergone numerous small updates over the years. However, it lacked automated test coverage, and the codebase was complex, with many interdependencies. The system’s functionality was critical, as it handled the calculation of mortgages and distribution of loans.

A typical problem:

The bank’s application team decided to make a small but significant change to the system. This meant updating the tax calculation logic to comply with new government policy regulations. This change required modification in risk calculations of loan applicants.

Given the age and complexity of the system, there were concerns about the potential for unintended side effects. The loan advances system is tightly integrated with other legacy systems, used by auditors and risk compliance, and a bug could lead to incorrect calculations, which would be a serious issue.

The system had no automated test suite due to its age and the lack of modern testing practices when it was originally developed.

The codebase was poorly documented, making it difficult to fully understand the impact of the changes.

The potential risk of failure was high, as any error in tax calculations could lead to legal compliance issues and unhappy employees.

Decision:

Due to the lack of automated tests and the critical nature of the loan ad system, it was decided that a manual regression test would be conducted. The regression test would focus on ensuring that the recent changes to the tax calculation logic did not break any existing functionality.

Steps Taken:

Testing team identified key test cases that covered the most critical functionalities of the payroll system, including:

Accurate calculation of collateral value for different types of assets

Correct application of federal and state taxes

Proper handling of risk and compliance

Accurate generation of loan disbursal stubs and financial reports

Creation of Test Data: The testers created a set of test data that included employees from different tax brackets, states, and employment categories. This data was designed to cover various scenarios that might be affected by the mortgage calculation changes.

Manual Execution of Test Cases: The testing team manually executed the identified test cases. This involved:

Running the collateral calculations for different brackets as a separate test scenarios

Verifying that the calculations were correct according to the new regulations

Checking that no other parts of other interlinked process were affected by the changes (e.g., credit check, risk analysis, compliance)

Validation of Results: Testers cross-checked the results against expected outcomes. They manually calculated and validated results for each scenario and compared these with the outputs from the system.

Exploratory Testing: In addition to predefined test cases, testers performed exploratory testing to uncover any unexpected issues. This involved running the payroll process under various edge cases, such as unusual deduction combinations or high-income employees in multiple states.

Outcome:

The manual regression testing uncovered a few minor issues where the mortgage calculations were slightly off for specific edge cases. These issues were documented, fixed, and re-tested manually. The overall system was confirmed to be stable after the changes, with all critical functionalities working as expected.

The manual regression test provided confidence that the collateral system would function correctly in production. As a result, the company successfully updated the system to comply with the new tax regulations without any disruptions.

Lessons Learned:

Importance of Manual Testing: In environments where automated testing is not feasible, manual regression testing is essential to ensure that critical systems continue to function correctly after changes.

Documentation: The process highlighted the importance of documenting test cases and results, especially in legacy systems where knowledge is often scattered or lost over time.

Incremental Improvement: The Federal Bank recognized the need to gradually build an automated test suite for the collateral system to reduce reliance on manual testing in the future.