30 Days of Ecommerce Testing Day 16: Busiest Times

šŸ“† 30 Days of Testing
,
1

Day 16 of 30 days of ecommerce testing is:

Identify which occasions in the year are busiest for ecommerce websites. Brainstorm some ideas on how youā€™d prepare for them?

So an obvious one is Black Friday. I was thinking of some other times of the year that there might be peaks but not as hectic, like Mothers day and Christmas.

While I think for busy times you need to think about load/performance, itā€™s also a time where your coverage becomes amplified. I had a discussion recently with @ash_winter about performance testing. A valuable thing I learned from this was while performance is useful, thereā€™s no point doing it unless you have a solid amount of testing done elsewhere before that.

With this in mind, before I focus on tools like Jmeter or locust.io, I would focus on the API and Unit test layers. If theyā€™re not up to scratch, performance testing wouldnā€™t give me useful information.

I took to SauceLabs website for some ideas here:

The tax example in the post above is an interesting one, sure to lose you even more money if itā€™s a bug thatā€™s introduced right before a busy season.

https://saucelabs.com/blog/when-to-automate-mobile-tests
I liked this one for its focus on automation and how exploration works well with that.

I think after this, I would focus on our logging. Logging would help determine where else we should focus efforts before we go spending too much money on those efforts.

5 Likes
2

Also depends on region where the website is deployed, like in India, Diwali is a festival where everyone purchase lot of gifts and new products (clothes, furniture and what not!), in NZ it is the boxing day, and any occasions like Valentines day etc.

Will surely test for load but also concentrate on stock qty. I have heard that a few times, people order on websites on these days, but when all the hustle & bustle is over, they get a message that the product is actually out of stock.

3 Likes
3

The busiest times depends on the world wide occasion and also on geographic location where the occasion take place, like @prachijain.me has mentioned

The ones which i have worked for is:
Christmas
Thanksgiving day

The things to concentrate on other than performance would be the

  1. Discounts and coupons valid for that occasion
  2. calculation regarding the same
  3. banners related to that occasion
1 Like
4

Yes, I also agree with @prachijain.me that the busiest time for ecommerce site will depend on events of different locations, which drives high demand of specified services and discount ā€“

  • Regional festivals when B2C websites likely to provide subtle discounts
  • Event when people get used to clear out old stuff from their home. C2C business may be busy
    • Auction pages where people will try to sell 2nd hand stuff
  • At the end of fiscal year for B2B
    • Ecommerce websites providing accounting, taxation services
  • Long holidays when people will travel around
    • Ecommerce websites providing transportation and logistic services

In term of preparation, a tester should keep an eye onā€“

  1. Performance test on web browsing and database access for example, planned with well estimated performance required, plus enough margin
  2. More test concentrated on purchasing with large amount of purchase in terms of quality and variety in each checkout. Should focus also if there will be discount on unit price/ free shipment due to large amount of purchase
  3. Test with using discounts, promoted coupons. In events use of these features in combination is likely to be occurred
  4. Test with price change before checkout. It can occur when selling price changes by the seller for clearance

Thanks,
Anthony

3 Likes
5

According to the national retail federation, who keeps numbers on this stuff and helps people plan for the year. (This is for the U.S. - does the U.K. have something similar?) these were the top 10 spending events for 2017:

image

Iā€™m glad I checked. I never even thought about ā€˜back to schoolā€™ and ā€˜back to collegeā€™ shopping, and never EVER did I think it was competitive with the winter holidays.

According to the article on back to school and college shopping, over half of the consumers plan to shop in a department store. Online shopping is 4th place after discount and clothing stores, and most people plan to start their shopping about a month in advance.

In fact back to college spending hit an all time high in 2017. So back to colllege and back to school spending combine seem to beat out holiday shopping by a lot. And itā€™s a loong season.

So weā€™re talking about a steady season from August through September, then maybe a drop in October, and then things start heating up again in November through the holidays.

I found a few relevant articles about analytics around the holidays:

Preparing for the Holidays ā€“ Part I: Connecting Your Load Testing with Revenue Metrics
3 Things eCommerce Sites Need to Do When Preparing for the Holidays

These articles focus on load testing, A/B testing, and optimization.

But look! When we get to back to school shopping, the needs might change a little:
Is your site ready for back-to-school shopping sprees?

Performance is still an issue of course. But to illustrate the difference, and make my point, I will cherry pick a few facts from the report card. :slight_smile:

ā€œThe top stressors are shipping costs (57%), out-of-stock inventory (51%) and not being able to see or test products before buying them (50%)ā€

On day 13 I talked about shipping costs and making sure there were no surprises.
Also, this type of shopping involves a lot of people buying the SAME THINGS, i.e. stationery, backpacks, etc. So the second complaint above makes sense. We need to make sure inventory count is working correctly. If X items are bought then X items should be deducted from inventory. If we go below a certain threshold, the merchant needs to know itā€™s time to get/order/make more of whatever it is they are selling.

The last item is a good one as well. There should be pictures of the item to purchase - and good ones. Preferably multiple from different angles or some sort of a rotatable view. We need to make sure all these work (if they exist).

ā€œNearly 1 in 5 (17%) plan to shop using their smartphones or tablets.ā€
ā€œMillennials are the most likely to shop online for back to school.ā€

So definitely mobile testing!

  • Dave K
3 Likes
6

As mentioned in the one of the post above, it really depends on the region.
All over the world we have:

  • Black Friday
  • Christmas Day
  • Women Day
  • Valentineā€™s Day
  • Easter
  • Cyber Monday (newer thing from what I observed)

The testers should keep an eye on

  • Performance. I truly believe that this is the most important thing for a website.
  • Campaigns, promotions are functional
  • Content changes to be in place (mostly for holidays)
  • Stock and prices for products.
1 Like
7

The peak times I could think of, both regular and one-off, were:

  • Christmas
  • Christmas gift return season
  • Valentineā€™s day
  • Chinaā€™s Singles Day (apparently this is a crrrrazy huge one)
  • back to school season
  • Black Friday / Cyber Monday
  • the big seasonal sales
  • in C2A ecommerce, last days when tax filings are due
  • big premieres (e.g. tickets, ebooks, a sensational interview in a paywalled newspaper)
  • certain big events (e.g. takeout ordering during big sports events that people watch at home)
  • major communication problems (e.g. people trying to rebook tickets all at once because a route is closed unexpectedly or there is a big strike)

Also possibly local special occasions where gifts are expected, although this can be limited to specific types of goods. Some that come to mind are:

  • April-May in Poland due to First Communion season
  • graduation season
  • Mothersā€™ Day and Fathersā€™ Day

Interestingly, I learned today that on the first non-shopping Sunday in Poland this year (itā€™s a newly introduced regulation), ecommerce traffic went up 30% in comparison to an average Sunday until now. The shitty part is that someone immediately requested that same-day delivery capacities are expanded :frowning:

As for ways to prevent clogging the service, the following come to mind but they are mostly process/business solutions:

  • extend the season (e.g. start reminders and cues early, extend the return period)
  • stage the sales (e.g. tickets in batches, promotion waves starting at various times of the day)
  • give certain groups of customers early access to promotions
  • create incentives to use the service outside of peak times (e.g. by processing early tax filings quickly so people get their returns early, too)
  • offer quick to buy (prominently displayed), maybe electronic gift cards (is also lighter on shipping and delivery)

In the technical realm, hmm, I donā€™t know much about this part:

  • general care about performance both in terms of technical components and user flows
  • extra quick paths to lead the user from arrival to checkout efficiently (e.g. compartmentalized Christmas gift pickers instead of wide searches and browsing entire categories)
  • load balancing
  • dynamic provisioning of resources to cover for peaks, maybe content delivery networks?
  • caching
  • maybe keeping things leaner such as temporarily downgrading heavy page elements?
  • soon maybe chatbots for quick customer support?
2 Likes
8

I reckon this are the most important events that an ecommerce website should take into account:

  • Black Friday
  • Christmas sales
  • Cyber Monday
  • Back to school time

Regarding Black Friday there is no much to say: It is a huge event that it has started in 1952 and it is considered one of the most biggest event in the world. A lot of people take advantage of the sales and start buying either in physical or in digital stores. So the stores should be prepared with the huge amount of requests that will come from such event.
As for the Christmas sales, there is a specific date that probably it will be more significant in Europe than US: 23 and 24th of December. Many people tend to be lazy and they wait the last minute sales or last minutes in general to buy the presents for the 25th of May. So this usually cause a bootleneck because of the last minute :stuck_out_tongue:
Then the cyber monday is an event that many people, specially freak of technology, gamers and so on. So as for the stores, it is a big opportunity to make more money and especially the ecommerce platforms.
Finnaly we have the ā€œback to school eventā€. A lot of materials, a lot of new technologies and no time. So parents normally wait for final days to buy everything and this might cause bottlenecks in the platforms.

So here I have my general thought about the event which have a big impact in the ecommerce. As for ā€œHow youā€™d prepare for them?ā€, the logical thought is to do some performace/load and stress testing as for those special days, platforms receive a huge amount of data and as a testers we should assure that it wonā€™t break nor slow down the systems.
Moreover I would suggest to prepare some accessibility testing and more in general, usability testing. If the page is not ready usable, a lot of sales will be lost along with the incomes.
And finnaly I would suggest the normal functional testing as to assure that everything is set up for the incoming customers.

9

Day 16 https://wp.me/p9EXXo-5k

Time of day should also be considered along with ā€œone off eventsā€
Concert ticket sales are a good example of this. The sales window opens at a set time and then a bajillion people all try to get in at once, just for it to calm down to normal later in the day.

1 Like
10

Iā€™d only heard of the 11th of November sales on Aliexpress last year, never knew that it was known as Singles Day or what the significance was, thanks! :slight_smile:

2 Likes
11

Timing is also important for events with changing pricesā€“for example, lower prices for earlier ticket sales, and possibly no sales after the time of the event.

1 Like
12

Iā€™d also consider the nature of the E-Commerce site for the busiest time. For example, if the site sells event tickets then the day tickets for the really high impact events go on sale will swamp the rest of the year (e.g. ticket sales for Glastonbury Festival in the UK). Culture and location of customer base also have a big impact.

13

From our friends joining in on Twitter:

@conorfi