In the past I always tested on data. Basically there are two categories:
- Actual data
This data contains information which must be processed. For example what does my order contain and what is my contact information like email address?
- Process data
This data is added to simplify the handling of the actual data. If an order picker has collected all items for an order, then it can be sent to the delivery service provider. In this case the status of the order is ready for delivery.
Notice that some process data is not always visible. E.g. customer information identification number.
During a regression test there is a product risk to focus on data which has not been processed much. I might enter an order and cancelled it immediately. But in production orders can be modified multiple times using different features which can add process data. In turn this data can be used by other features.
Deleting a feature might lead to different process data which might have negative consequences for the system.
My black box approach is to test the system using production data. With the current privacy laws this can be quite tricky. A solution is a deployment test before going live.
An extra test is to
- use some features of the current system based on a product risk analysis.
- update the system
- use the updated system to process the migrated data.
A white box approach is to model the state transition diagram of the data. This can require quite some time.
I also suggest to involve the users to explore the consequences. Some users might have workarounds which need some features which are to deleted.