The Post Office Scandal

I watched The Post Office Scandal on the BBC yesterday. Itā€™s constantly on the news atm too. I recognise I may be late to the game.

I also stumbled upon this post by @jameschristie, which must be the most comprehensive testerā€™s review into the scandal?

Who else has been following along and what kind of thoughts do you have around it?

Itā€™s heartbreaking to see the damage it has caused to so many people.

6 Likes

I bought the book when it came out, and struggled for over a year to even finish it, itā€™s a disgusting story not only of IT project failure, but of corruption. Corruption in the way we think.


The story does not start with James, but with Computer Weekly uncovering the story, a story that first came to light on 2 June 1994, where 2 pilots of a chinook were blamed for a crash that killed high ranking officials, when in fact it was, as we all now knowā€¦

buy the book, itā€™s really hard to read, the podcast (on iplayer I think) is a doddle to listen to. As for the ITV dramatization, I cannot speak for I do not want to replay this story in my head.

5 Likes

I totally missed there was a book. You know itā€™s been going on for a long time if thereā€™s a book about it (!) :pensive:

Is The Great Post Office Trial the podcast?

2 Likes

Iā€™ve just finished the watching the drama show and had no idea about the podcast or the book! Will have a look at both.

Everything about this story infuriates me:

  • All the lies and cover up from the Post Office, Fujitsu and the Government
  • All the ruined lives
  • The fact that no one yet has been made accountable for what happened
  • The delays in getting all these families compensated
  • The fact it takes a TV drama to make the government talk about it
  • And so many more smaller details about it
2 Likes

Maybe a controversial but Iā€™m of the view that the Post Office Horizon scandal is more about people and money than software and testing.

Horizon had bugs (impossible not to say about any software) but it was also a Ā£1 billion Fujitsu system that processed hundreds and thousands of transactions correctly - so it was most likely tested, and tested reasonably well.

The issue was lack of proper investigation into the bugs when they came about, for whatever reason (poor management, lack of accountability, Post Office contracts, cover ups or whatever) is the real reason itā€™s dragged out.

As someone who tests software for a living, the ITV drama is a frustrating watch for certain.

2 Likes

I agree with @whitenoise , some good testing work was being done, but from the book, I got the impression that bugs were swept under the carpet, because the system was unstable, it failed most often when the internet connection to the terminals gave problems, and that would have been reported. But the project also suffered scope and funding churn, no doubt some new process problems got injected if there werenā€™t any to start with. Even the platform choice is bizarre.
But the post office also had a far more ancient weapon to use in itā€™s defence against fraud and theft, a tool even older than the London police force, yes, you guessed it. If not, Iā€™ll not spoil it, read the book.

Yes I have been following this in Private Eye for many, many years now.

At one point I was asked to work on some integration of our products with a Post Office system - my first question was ā€œis it Horizon?ā€.

Yep, that is the correct podcast.

I read the book a while back, and I think another human element that stood out was the sheer number of different people and roles that had incentives to cover things up. Things like the contract between Fujitsu and the Post Office making it in Fujitsuā€™s monetary interest to not acknowledge the presence of bugs, IIRC.

2 Likes

Any tips on watching the documentary/tv series from outside the UK?

I did find one episode here: https://www.dailymotion.com/video/x8r2p41

(edit: I figured out that I could create an ITV account and watch them once I set my VPN to a UK gateway)

Back to the original question, having followed the story for at least a year and having read the book and (now) watched the series, I can say that the TV series seems to have done a really good job of summarizing much of whatā€™s in the book. The book obviously goes into a lot more detail and there were some things, particularly the trial, where I felt like more detail would have helped the series, but overall I thought it was very well done.

Both the book and the tv series made me far more angry than anything Iā€™ve ever read or watched in my life, and brought me to tears more than a few times just thinking about both the extent of the damage done to so many individuals, but also the callousness and lack of consequences for those responsible. I sincerely hope the renewed publicity helps speed up addressing both of those areasā€“and it sounds like some things have happened even in the last couple days that seem like promising steps in those directions.

I think the Horizon scandal should end up in future software textbooks alongside older well-known software fiascos like the Therac-25 and Ariane 5 rocket as yet another cautionary tale of the human impact of putting too much trust in software systems.

5 Likes

The book and podcast made my so angry I was only able to read 1 chapter at a time and took, yes a year to get through reading. Have been following @jameschristie on the former bird platform for ages on the story. However, as a textbook chapter, it serves as an exact echo of the Chinook crash in 1994. Itā€™s an older old story, and my grasp of it starts with Vasa (ship) - Wikipedia which suffered the exact same problems but in their own time context of 1628 in Sweden.

1 Like

I raced through the book! It revealed some shocking practices at Fujitsu, but the real horror story has emerged at the public inquiry. When they built the original Horizon system Fujitsuā€™s development teams were working like a bunch of schoolkids on a school project. Coding quality was abysmal. There were no design specs. They were reverse engineered from what was built. The crude prototype hacked together for sales demos became the real system, rather than building it properly from scratch. Acceptance testing was carried out in live. Developers focused on the happy path and didnā€™t think about what could go wrong, or even understand accounting principles and the implications of losing accounting records . These are just a few of the problems. The revelations went on and on at the inquiry.

1 Like

Agree. Though I did feel it was failure or overlooking on part of testers but realise that every production system is manageable by IT guys in support teams otherwise how would they provide fixes, which would be data most of the times. I believe it is POā€™s problem of not holding Fujitsu properly accountable for issues and discrepancies it caused.

There are so many failure points in this tragedy that one struggles to know where to begin.

However, in the context of this group, it is interesting to consider the ramifications on the IT sector and the QA/Test community specifically.

There have been many discussions about the demise of our craft over the past years, either due to the introduction of AI, the belief that the developer community can 'mark their own homework, that the ā€˜fail early, fail fastā€™ approach can deliver what the business wants faster without testers involved ā€¦ The list goes on.

Iā€™ve worked in ā€œbig banksā€ where there are many 10s of interlinked systems dealing with massive individual inter-corporate and inter-bank transations running into 10 major digits of US Dollars, Sterling, Euros and others as well as FinTech startups where, if we got it wrong, real peopleā€™s real money was at risk. I can confidently say that there is no ā€œone size fits allā€ software development life cycle, team topology, project mamangement or other paradigm which can be considered ā€œbest practiceā€ and applicable across every domain.

Experientially though, across all these settings, I have found that every situation was improved by bringing the testers/QA team into the process prior to development commencing. The ability to identify ambiguities and omissions within requirements/user stories/feature files ā€¦ and then to address those prior to the code being developed has, again experientially, given a 14:1 return based on time saved vs time expended. Working collaboratively and early in the processes, the QA/Test members of any development organisation are, in reality, bringers of commercial advantage, not simply an irritating cost to be borne. I am, of course, happy to share the background to that statement. Iā€™m also happy to share those experiences and learnings with the community.

The outfall of the Post Office Inquiry will, I hope and in my opinion, lead to radical requirements for software suppliers to be able to ā€œshow their workingā€ when signing off systems for release, particularly given Lord Arbuthnotā€™s hope expressed to the Business and Trade Select Committee on Tuesday that the current legal "presumption that computer evidence is reliable"will be changed. There may also be hightened indemnity premiums demanded by insurance companies of software houses who are unable/unwilling to provide those ā€œworkingsā€ for inspection.

Putting all this together, I believe we may be at an inflection point for the software industry and I hope we, collectively, can demonstrate our worth.

1 Like

Changing Requirements and scope as well as a lot of funding being pulled when Camelot pulled out was probably the straw that broke itā€™s back - I secretly wonder if Camelot could see that the project was toast and pulled out for that reason. I mean who would not want to partner with a big trusted brand like the post office, unless Camelot knew that they were not really dealing with Royal Mail but with POL instead, an entirely separate organisation. Resources just moved :confused: