Automated Testing of Blackboard VLE

Questions
#1

Fellow Testers,

I’m a software tester/coordinator at a UK university. We use Blackboard as our virtual learning environment, which I’m looking for advice on how to test.

Have any of my fellow testers had any experience with the Blackboard platform?

Blackboard is delivered to us on a SaaS modal which means we can get updates to the environment on a regular basis; however we have no control over this and sometimes updates are drop at very short notice.

The issue we are currently experiencing is that updates are introducing new settings that change things without us knowing. This is something we’re trying to stop but in the meantime I would like to use some automation to check for what has changed.

I’ve been exploring visual regression checkers like Applitools, which seems promising (this would enable me to have a baseline of before the update to compare). I’ve also built a framework using WebDriver in Visual Studio, but this is proving very laborious with hundreds of settings to check!

Even if you have no experience with Blackboard, perhaps you could offer advice on what tools I could look into.

Any help would be most appreciated!

Cheers,
Miles

#2

Hello @tester007!

Interesting topic! SaaS is becoming not only a method to deliver valuable services to businesses but a testing challenge on many levels. Updates to a SaaS platform is one of those levels.

We were implementing a SaaS solution and discovered SaaS updates during that implementation. It gave us pause and wondered how updates would impact business continuity in production. As we investigated, we found the provider schedules updates and preview periods on a regular basis. In this manner, all of the provider customers have an opportunity to prepare and evaluate updates to the SaaS product before it is deployed to production. While this seems like a mature approach, I believe it is should be a principle for SaaS providers. I applaud your effort in helping Blackboard understand this.

We also made plans to monitor our configurations in the production environment. Fortunately, the SaaS provider publishes APIs that allow us to retrieve the configurations we assigned for our implementation of their product. The plan was to retrieve those configurations periodically and compare them to the last known configurations we assigned. If there is not a match, then it is an indication of some change.

Joe

#3

Hi Joe,

Thank you for sharing your experiences with testing SaaS solutions.

As you have discovered with your product, I’m also starting to believe that an API approach is the best way forward. However, my current understanding is that Blackboard does not offer such a solution for settings config. I need to do more research!

It’s good to see that I’m not the only one thinking about how we test SaaS products.

Thanks,
Miles