Test Data Management tools recommendations

Our organisation are adding new applications where we find data internally and externally to determine if a customer is eligible for a service. Our UAT testing requires finding data for 2 parts of the test:
1. The “Customer Persona” (e.g. Age, History, Current Status, etc)
2. The “Eligibility” record to see if that Customer can obtain the service based on the business rules and the record in “1” above.

Data is time sensitive on External and Internal systems and would mean we enter into a Refresh cycle of restoring Production (obfuscated) to UAT. The External system has a refresh cycle each month in place.

How we go about finding this data is as follows:
1. External System first accessed and “Customer Persona” in our tests determines what “Customer Record” we get.
2. The “Customer Record” given to us by this External Party may or may not have associated records on our “Internal Business Systems” due to the unknown relationship between this external DB and our internal systems.
3. We currently find records (see How we currently do this below!) on each of these “Internal Business Systems” to match the “Eligibility” part of our tests and assign it the “Customer Record” above.

What we currently do:
1. External system data is supplied to us by an External party based on our Test Case requirements.
2. Internal team create SQL scripts to change the “Customer Record” based on the “Eligibility” required data on each of the systems which is a large task.

What we would like to do:
1. Without having rights to the Test Data in the “External” system, we would like to amalgamate the data in our internal systems to meet the “Eligibility” part of the Test Data and then update the “Customer record” against that subset of data.

What Test Data Management tools could possibly complete this? Any experiences on managing of this test data across multiple internal and external systems would be appreciated.



I probably over complicated the framework I implemented in a way but it works!

  1. I source the customer data now in the excel system by executing several hundred SELECTS via Python and Python outputs the data to a test file.

  2. The Customer data is then manually added to the test Data Master File.

  3. In the Master Data file, there is reference to the particular “data persona
    a. Example: Customer X with Credit Rating of 0 and Age> 60 has Completed “all” “Pension Training” (DB1) and Submitted Partial “Revenue” documents (DB2)
    b. Example: Customer X with Credit Rating of 0 and Age> 60 has Completed "Partial " “Pension Training” (DB1) and Submitted Partial “Revenue” documents (DB2)
    i. The Stored procedure inputs as “CustomerID” and “Partial|All|None” for DB1
    ii. The Stored procedure inputs as “CustomerID” and “Partial|All|None” for DB2

  4. The Python script is again used to manipulate this data in the internal DB’s as last step
    a. This is overkill really and using Excel and CONCATENATE would have done the trick to have a few hundred EXECUTE SQL Scripts.

No OOTB solution was used.

Perhaps there’s an opportunity to seek inspiration from the latest round of challenge solutions featured as part of the Test.bash(); 2022 event.

Look out for the videos with “Data Management Challenge” in the title. :movie_camera: