as long as the test is “green”, you can be confident that what the test checks is still OK–no manual effort involved
Ish. If we grant that a system displaying “green” does mean that whatever it’s checking is OK then the manual effort involved is writing the check, running the check, looking at the green, and evaluating what that means for the product and the project, including all the weaknesses of that check. This also rests on presumptions about the quality of that check, and whether it’s still valuable in the new light of a shifting context, which leads to the subject of maintenance of the code base. There’s a lot of person hours put into an automation suite.
There are many subtleties in the testing vs checking discussion. For example the subconcious processes that run in our minds while we explore are considered “scripts” under the ET 3.0 definition. While automation checks are explicit scripts, there are also tacit scripts, and it’s only with the agency remaining after what we cannot choose for ourselves that we test in the “test vs check” sense. This, I think, one reason why the proponents of ET 3.0 and the check vs test concept don’t like the terms “automated testing” and “manual testing”, because testing cannot be automated. It is one test effort, done by a human (always, by definition), aided to one degree or another by tools, which may include an automatic check execution tool. From this position we can see that we don’t “automate” the testing of any functionality at all. We check one very specific thing, at one time, in one environment, with one setup, with one set of data, and we do it in a way that no user ever would. That’s why we need a good reason to write a check - one that supports the purposes of a wider test strategy in the mind of a human - and that’s why “automation” is a “manual” process (or, if you prefer, there is no automation or manual, just tool-assisted testing).
What I meant is not that regression checking is controlling changes to dynamic properties due to context changes. What I meant is that regression checking is controlling changes to dynamic properties due to changes to the software itself
I disagree.
Regression testing […] verifies that software which was previously developed and tested still performs the same way after it was changed.
I disagree with this less. This definition doesn’t completely exclude the idea that changes that are not represented programmatically can influence regression testing. Luckily I don’t need to disagree with Wikipedia because Mr Bolton’s beaten me to it with characteristic speed and clarity: http://www.developsense.com/presentations/2012-09-KWSQA-Regression.pdf (page 5).
Whether context changes are important for the software and should be regression checked is up to you
What if I think they are, though? The problem seems to me to be the difference between a programmatic approach where the checks have some innate intrinsics and a human-centric approach where everything is a relationship and testing is a social activity performed for humans by humans. It’s the difference between those views that gives automation checks such limitation.
Do you agree on that clarification?
If I were to say that I did, or that we could assume it to be true for the purposes of this discussion, then we’d hopefully come to some conclusion - but that conclusion would have to be subject to the problems I’ve mentioned. That’s not to say it’d be without value, just that we are aware of its limitations, like we should be with all the tools and oracles and heuristics we apply.