OK, this is a simple Windows only exploration (linux is an option, but it’s less pressing)
i have an msi and cab files as emitted by the build system, i want to verify it.
Now aside from a uninstall and upgrade/downgrade tests running without errors, what approaches have people taken to verify that all the expected files and all the expected components do register?
The app is not automate-able at this point, so looking only at logs to check that the program starts and initializes correctly, my main question: anyone used some technique for checking all files and all registry entries do exist? I’m thinking to use orca and look at the manifest for some clues in order to detect where we might be hiding some fiels and some settings, but anyone used any algorithms for installer checking?
I can’t give you advice on that as our app was automate-able and just want to add that we used Windows VMs of which we had clean snapshots. At the start of every execution we set the VMs to the clean snapshots and let the installation of our program ran. By that we avoided that multiple un- and installing taint the system.
Yes @sebastian_solidwork . I want to go that route at some point, but it’s not that mission critical of an install, this is not consumer grade/mass distribution. But I’m keen to be sure no files are missed for install or for removal, and no obvious settings are missed. We don’t have that much great control and tracking because a lot of manual processing happens.
A clean VM is probably he only way to be sure you are not missing something non-specific or unknown. I’ve had a fair few of those creep past an installer test and have used clean VM’s as a antidote to things that go bump in the night. right now my approach is to write a script that will “manually” remove all software settings and traces, almost like a CClean registry cleaner or an install-tracker app might do. I might even evaluate one to see if it finds settings I missed. But right now the orca tool for windows has told me just about everything that the installer does do, so a simple check of which files and which components are registered is probably going to cover me for 90% of the cases. I also have more than one app installer to test later on, so a VM will probably be where I go in the end. Might share some code samples once I am happy with my install verifier.
Aside from a VM, you could also just use a clean image/install of the OS. There are OS imaging tools out there, most not free though, that you can use for this purpose, that IT departments typically use. You could even check with your IT if they have such tooling that you could “borrow” use of.
Basically, with imaging software, you can image/reimage/clone a system the same way you provision a VM and revert to saved snapshots. It works like VM image management but for physical hardware installs (provided your hardware doesn’t change or doesn’t change much). I’ve used this route at a previous employer. We used a native solution from Microsoft that used WinPE and the Windows related toolchain, I forget the exact name of the image tool.
Do you have a spec for what files and registry keys you are expecting from the installation process? If not, it becomes exploratory to find out and then confirm with developers if needed.
orca or some MSI inspector/extractor tool could be useful. I’m not aware the tooling and process for inspecting what an installer installs, etc.
One thing you can also for registry is export registry key branches (the root branches), and do that before and after (clean) install. You can then compare and diff the changes for monitoring what got added/modified, etc. - for what is expected and what is unexpected. Not sure if there’s good registry diff tools, but a file diff tool will suffice for diffing exported registry files.
A file system monitor also would be useful for comparing file changes from an installer. That or snapshot the directory tree of file paths you expect installer to modify. You can then compare against the before and after directory tree snapshots.
Yes. I’m unpacking it using the orca tool by exporting the installer database tables, which are tab-delimited CSV files, then parsing just the interesting tables into a map of where files go. The plan is to do this as a once-off and every time developers add ore remove files, to detect it and flag it as a test fail.
It’s damn fast to compare and sanity check an install this way, so it has value for now at least.
Does CAB/MSI files include signing/hashing to verify the integrity of the package?
A small thing sure - but a good base security measure, which reminds me…
Any concerns wrt confidentially of the contents of the package? Would you need to encrypt stuff before distributing? Any open source (share-a-like) or similar policies in frameworks that needs to be handled? Internal files that are not supposed be in the package etc.
Yes, @jesper the cab file does contain hashes - I did think about that I can test them, but as a basic “unexpected missing files” check I just want to verify there are no zero length files being randomly installed. We don’t encrypt and we don’t publish our msi hashes, in fact you have to login to our site to grab the installers, so that is less of a worry, it’s low volume.
But after 3 days I’m finding that there are a few more tricks to simple once-off export of the tables as a way to verify when developers change the installer or even if (this is possible) we decide to change our installer toolstack. It’s also an installer that installs a huge huge bunch of files, so that helps not.
And that brings me to your cartoon panel, and yes, I’m feeling like a lot of that cartoon panel is relevant and that I need to simplify the problem of un-mapping the msi database tables and unpicking how they work in a non-trivial installer. A little bit overwhelmed before I complete the task, because we also have post-installer tasks that unzip more files, and so I’m actually testing the “behaviour” of an installer and not testing the MSI packaging alone. You have inspired me to try a different angle now that I know more.
