I’ve been using some of these spacex experiments recently to try and convey that a lot of testing is about unknowns and really about learning.
This can be a challenge as a lot of people think about testing in a pass or fail way a bit like this newspaper header.
I’ve even had comments that these launches are more like beta testing rather than testing in general that focuses on learning, investigation and experiments.
I’ve no doubt they were collecting thousands of bits of information as part of this test, that their focus was on learning and they will hopefully have learned a lot.
For me as a tester that is a somewhat successful test.
Failing an exam does not have the goal of testing or learning, its primary goal is a pass or fail scenario.
Here in those launches their goal is testing and learning, they have set up to learn, they accept there are things they don’t know about, that they are the first to do so so they set up to learn.
Perhaps even as a more important primary goal than to pass or fail.
Maybe that is a key element in the thinking, what is the primary goal of the test, is a pass or fail test or is the primary goal about learning so as long as you learn the test will be successful.
I suspect spaceX at least some of them will take this as success to that extent, one step forward to mars but a lot more testing to go
Different viewpoints in articles:
“With a test like this, success comes from what we learn, and today’s flight will help us improve Starship’s reliability.”
‘SpaceX’s Starship test flight ends in failure after spacecraft is destroyed’
A test can fail if the scope that it had wasn’t achieved, in that it didn’t bring any learning.
There can be a failure happening during or after a test: a software/hardware component failed, which is a learning opportunity that the test brought.