I may be wrong, but I keep getting this niggle at the back of my head that we are paying the supplier for delivery, but at the same time testing their product for them.
Welcome to the bleeding edge of the New World Order. For years mankind has written about fascist dystopias and inescapable hells born of accidents, destiny or a seizing of power. Nobody predicted that we’d build one ourselves, our faces etched with a twisted, plastic grin, smiling at the construction of our own downfall lest we upset someone with our frowns. But I hear it’s very popular, so what do I know?
I haven’t solved this problem myself, and I may be missing the full picture, but if someone sells me something that doesn’t work I’d take it back for a repair/replacement. Enough of those and I’d feel it time for a refund. By that point I’d be angry. Just saying. I’m sure there’s a diplomatic way to ask “why are we paying you for something that doesn’t work?”. Maybe it’s part of the discussion “by what point do you predict this will be usable?”, or “are there normally this many problems this far into the project?”. Maybe it’s part of the discussion “we’re worried about the number of problems in the software, and we’d like to examine your test strategy/processes/procedures.”. If you’re paying then you’re in the position of power, and these questions sound totally reasonable and understandable to me. Software is an investment. Asking risk-based questions of your supplier is protecting that investment, and clearing up communication problems.
Of course I might be wrong - maybe this is your first run-in with a heavily agile project that is frequently releasing minimum-viable software to you and the software is basically in alpha. Maybe it’s a way to get great feedback to build the perfect software. Or a way to say they’re doing that while giving you bad code. Either way, yes, you’re testing the software for them, which is sometimes not a Bad Thing - the question is are they doing their fair share? It might be that they are not communicating their expectations to you very well - maybe the software’s not supposed to work right now, it’s just to give you a feel of what it might look like in case you say “this is not at all what we’re looking for”, as opposed to “this table doesn’t sort properly when I click on the little arrow”. If they’re worried they wouldn’t tell you. You’re worried and you may not have told them. But you’re paying, so I’d go knock on the door. Bring someone tall.