OpenAI recently launched ChatGPT Agent.
https://openai.com/index/introducing-chatgpt-agent/
The chatter on the interwebs is in the tone of ‘browser wars’.
My thoughts, of course, jump to quality and testing.
How will those change how we test?
Will these simply be another browser configuration to test, or is there more to it than that?
Oh this is such an interesting thing to think about!
My first reaction is that I would treat it just like any other browser and begin officially supporting once it reaches some baseline usage %. But… I think there’s more to it in these cases bc of the way these browsers will interact with websites.
It seems similar to testing a feature that has some functionality around sharing something to facebook et al - testing the way the thing is being shared on those platforms is part of the work (even though you don’t own those surfaces)
My initial reaction was that this is a nothingburger, but on reflection I concluded that some testing would indeed be valid. However, my conclusion further down may not be what you expect. ChatGPT itself says:
“ChatGPT Agent’s built-in web browsers use a custom, sandboxed rendering engine developed by OpenAI rather than standard browsers like Chromium, WebKit, or Gecko. The Visual Browser (which can click, scroll, and interact with pages) runs inside a secure virtual machine and is designed to simulate a real browser’s behavior …”
The agent’s web browser is going to be a black box, like embedded Java applets and Flash movies (showing my age - you youngsters should ask your grandparents what they are). It’s not going to have a DOM Inspector, so even if you see an issue you are not going to be able to investigate and diagnose it.
It’s very unlikely you will be able to fix any issues unless ChatGPT publish a list of HTML, CSS or JavaScript properties or coding patterns it doesn’t handle correctly. I think there is zero chance of them doing that.
The agent needs to be able to identify and interact with interactive controls in websites. The best way to ensure this happens is to make the website accessible. If you are already doing this properly (which extremely few companies do) everything should be fine. If you are not, you will be reliant on the agent’s visual heuristics, which are a lottery.
In my view, it would be far better to do high quality accessibility testing in a normal browser rather than trying to test your website in the agent where you can’t see any code and you can’t use any tools. The problem is that there are no more than a couple of dozen people in the UK capable of doing high quality accessibility testing. ChatGPT said the following:
"ChatGPT Agent’s web browser identifies interactive components using a combination of DOM analysis, accessibility cues, and visual heuristics within its controlled environment. Here’s how it generally identifies interactive components:
<button>, <a> (links), <input>, <select>, <textarea>, and other form controls.role="button", role="link", aria-haspopup, aria-expanded, and aria-pressed.onclick, onkeydown) which indicate interactivity even if the element isn’t a traditional interactive tag.cursor: pointer, tabindex attributes, or focusable elements to infer interactiveness.tabindex attribute and natural tab order) to identify elements that users can interact with via keyboard, reinforcing their interactivity.