What is the best way to test data migration from old platform to new platform

Discussions
,
1

I am looking for a tool that helps test the migration of data from an old platform to a new platform. What is the best tool to test Data migration?

4 Likes
2

Hi @meet_shimpy, welcome to The Club :wave:t2:

Is the following article on your radar to trigger some ideas? Five Things To Consider When Testing Database Migration

What extra context are you comfortable to share here to help others answer your question?

4 Likes
3

The article that Simon linked to looks very helpful. A few extra questions to consider.

Imagine the source system has a table with rows whose primary keys go 1-57. Then rows 1-50 are deleted. When the data’s migrated, should the destination system hold rows labelled 51-57, or the same rows but labelled 1-7?

Sometimes tables have a column on them that marks them as deleted, rather than physically removing the rows from the table. (Something like IsDeleted.) Should such rows be migrated?

Also there are tables that hold a history of something e.g. all versions of someone’s address over time. Do out of date rows need migrating? What about things like all orders a customer has placed - do 10 year old orders need to be migrated?

There are often practical considerations to check. For example, the source database might be a moving target i.e. it’s being updated all the time. One way to tackle this is to somehow take a subset of the data e.g. the most recent x%, during an overnight maintenance period and then gradually fill in the remaining data in less of a hurry. Will the destination system stay healthy all the time, by having a consistent set of data?

3 Likes
4

Migration testing is very important and plays a vital role when it comes to moving the data from old platform to new platform. In order to check that data is migrated correctly to new platform following parameters should be taken into consideration:

  1. Duplicate data is not created in Database.
  2. Existing data is not removed from database.
  3. Application features are working same as it was with old platform.
  4. Performance of the application should be intact.
  5. Test results is collected during pre migration testing and the same is validated during post migration testing and results are compared if something got modified.
  6. Migration may lead to application failure so ensure that you have required backups.
  7. Migrations steps should be documented so that it can be traced back if some failures occur.

I hope this information will be helpful to get a successful migration from old to new platform.

5

I coded a tool by myself. When you can not you maybe can ask for help?

The very basics what I did:

  1. Read the data from data base => create per table a CSV file and write the content sorted into that (on directory per database)
    1.1. About sorting: I first sort all columns with their content a-z (like you would move a column in Excel) and then I sort all lines a-z. This works no matter the content
    1.2. Maybe you also need some transformation of data to make the comparison easier because by their very nature databases treat data different (e.g. Oracle saves empty string (“”) always as null, Postgres saves them as the are).
  2. Depending on the content:
    2.1. Just a view tables: Compare the files/directories directly with e.g. WinMerge
    2.2. Many tables: Code a comparison. I did: compare all headers to be equal => compare number of lines are equal => compare content of columns is equal
    2.2.1. also add a exclusion list of columns (like internal IDs)
2 Likes
6

This topic was automatically closed after 280 days. New replies are no longer allowed.