Automated testing for dynamic content

How do you all go about automating tests for sites with dynamic content (eCommerce sites like Amazon or content sites like The Guardian)? Because the content you see today might not be there tomorrow. How do you handle that?

Hi @adityasg13 it really depends on the goal you are trying to achieve with the testing and the environment on which you are working on. In my experience with eCommerce I would say that it’s best to create all the content needed for the automations by yourself and when you finish the automation flows delete the testing data you’ve created. For instance if you are testing an order journey. First create the product -> publish it -> create new user account (or buy it anonymously) -> sync the order in the internal systems -> dispatch it -> etc. and when the whole journey is finished delete the product and the user account this way you can always rely on having a product and a user that you can use.

3 Likes

Hiya. This is a question of data control. Do you have access to the database?

At any point in time, your test is only as strong as the variables you are able to control. In this instance, you have identified a variable you seem unable to control - so this will naturally diminish the reliability of your test results.

If the purpose of the test is NOT to assert what should be displayed in content box y, then dimply don’t check it. I would suggest you build out a POM, or some means of referring to the particular content area, without making assertions about the contents itself. eg, assert the pixel width of the content box, but don’t assert the contents itself - make sense?

Hi adityasg13 , I think it can depend on your test cases by example if you try happy scenario you can implement test case to find first available product and then try to buy it , Check search results matches the criteria some cases like this
My idea to make your test case implementation dynamic not depend on specific product

Some options:

Model Based Testing using Finite State Machines may be what you want if you are concerned about an item having multiple states. Like the state of an SKU where you have a stock, an order, items in transit and so on. This can circumvent the dynamic nature by your test system doing a series of tests that know what the state to the item should be.

If you look up BDD you have the pattern (GIVEN, WHEN, THEN) where the first part is a predicate that needs to be fulfilled for the rest to be applicable. You can either inject the predicate like (GIVEN SKU with a stock of 0), WHEN buy item, THEN Out of stock information.) -> Add SKU with stock of 0, or if you have a lot of available data you can create a lookup service that find you SKU #12342 fo which all your conditions are true and use that specifically in the WHEN / THEN part. The first option with the injection is most popular in my experience, and it gives you higher predictability but makes your tests less stable, the second we finding items is the opposite.

If you have a CD/CI situation you typically have a data set for your automation that you bootstrap your system under test before each run to remove the dynamic portion of your system.

My favorite is the second option where you set predicates for your tests and if you have no means to inject data with the option of first finding the data on which you will work.

How do you all go about automating tests for sites with dynamic content (eCommerce sites like Amazon or content sites like The Guardian)? Because the content you see today might not be there tomorrow. How do you handle that?

You are starting from the wrong question.
Testing is about finding an answer to the question: is there something that threatens the value of our product?
What is the ‘something’ you are concerned about?
Are you looking into issues with the booking or purchasing, payments, carts, calculations of costs, logic of some feature, gaps in some flow of usage, data - localisation?, data - content fitting, data - meaning, data - existence, finding missing steps/buttons, reviewing entry points or exit points, etc,…etc…

Map what you need to test, analyze the risks, then think if there’s any need for tools and which tool can help you with testing.

Trying to keep it simple and not answer questions with a question, so here’s 2 approaches condensed from some of the above.

  1. If you have access/ability to create your own data, do so. This is the only way to guarantee a deterministic outcome for your tests. (Important side note: If you don’t, then always worth asking if you can).

  2. Alternatively find the patterns that you can predict. For example while you might not know the exact number that will be generated, you may at least know the expected range of that number, what format it should be in, it’s attributes etc. Then perhaps look to use assertions based on regex patterns. This option isn’t as ideal but might at least give you some confidence around data quality by ensuring the content is adhering to certain standards.

Good luck!

1 Like