Performance testing of Cloud Application Security agent on device

Questions
#1

Hi All,

The project I will be testing is installing a Cloud Application Security (CAS) agent on endpoints (CASB and proxy management). The agent is sold as a VERY low performance impact, but this needs to be proved.

The devices I will test on are:

  • PC (win7/10)
  • Managed devices (iPhone and Andriod)
  • Unmanaged Devices (iPhone and Andriod)
    [I can go into further detail on the devices matrix if needed]

On internal network, external (connected via VPN), and external network. There may also be a reverse proxy components for unmanaged devices on an external network.

Does any have any recommendations for tools I could use, which I can run on each test device to records the agent performance? I have noted we can probably just use Performance Monitor for windows devices, what is your opinion on this strategy?

Thanks :slight_smile:

#2

You have said that the application will be low performance impact but your going to prove it using windows monitor. I think that you are missing a number of things which could impact the user’s perception on the performance of the tool and you still need to answer these questions:

  • What impact does this have when the application loads as a user logging into their PC?

  • What impact does it have on applications running locally on the PC vs cloud based applications?

  • If a user is running on a low performance PC vs a high performance PC?

  • If the user is on the other side of the world with a high latency/low bandwidth scenario what impact does it have?

#3

Thanks Craig, some answers below to your points:

What impact does this have when the application loads as a user logging into their PC?

  • To be determined. The application is lightweight and only sends data to the tenant to determine actions if applicable, it does not do any ‘work’ itself. Another thread I have open questions baseline testing strategy, if you have any opinion on that:
    Page response baseline testing

What impact does it have on applications running locally on the PC vs cloud based applications?

  • We will have pilot groups across the business to confirm applications (cloud and on-prem) are not affected. Specific pilot group members will write and conduct test cases on high use/high risk applications.

If a user is running on a low performance PC vs a high performance PC?

  • Yes, you are right. There will be a range of performance levels, running this test across a range of test devices is to provide indicitive results only. It is noted in my Test Strategy that this is the case.

If the user is on the other side of the world with a high latency/low bandwidth scenario what impact does it have?

  • Another assumption, based on the vendor instruction, is the product will connect to different data centres depending on location. This testing will be out of scope at the moment.

So at its basic level, I just want to show the CPU/memory usage the application has, but need a tool across all devices to show this.