You do not need to become an expert in assessment methodology to tell whether a vendor did real work. You need a short list of questions where a vague answer is itself the answer. These are the ones that separate the people who assessed an engineer from the people who read a CV and formed an impression.
Who assessed this engineer, and are they technical?
The single most useful question, and the one that most often produces an uncomfortable pause. A great deal of vetting is done by people who cannot evaluate an architecture answer, because they were never engineers. That is not a character failing, it is a structural limit: you cannot tell whether an answer about database sharding is good if you do not know what a good one sounds like. If the assessor was not technical, everything else in their report is a summary of words they could not grade.
How long was the assessment, and what format?
A number and a format. Forty minutes on a video call is a screen, and if that is what it was, you would rather be told so than have it described as a deep technical evaluation. Anything that produces a real conclusion about a senior engineer takes an hour at minimum, is live rather than asynchronous, and has an unscripted section. If the vendor cannot tell you the format without checking, no consistent format exists.
What did you test that was specific to this person?
Generic assessment is the easiest to fake and the least informative. You want to hear that the questions came out of the engineer's own claimed history: this system, this outage, this decision, this trade-off. A vendor who assessed properly will be able to tell you something specific and slightly odd about the engineer's actual work within a sentence or two. A vendor who did not will speak in adjectives.
What did the engineer do badly?
The best question on this list, because it has no comfortable evasion. Every real senior engineer has areas that are weaker, and anyone who genuinely spent an hour with them knows what those are. An answer of nothing significant means either the assessment did not reach far enough to find anything, or the vendor is managing you rather than informing you. Both are worth knowing before the first call.
What could you not conclude?
A close relative of the previous one, and the marker of a serious process. Real assessment has edges. There will be areas the session did not reach, claims that were plausible but unconfirmed, skills relevant to your specific role that were not covered because the vendor did not know your role in that detail. A vendor who names those unprompted is telling you where to spend your own interview time, which is genuinely useful. A vendor who claims complete coverage is claiming something no assessment achieves.
How many engineers did you turn down to send me this one?
You are testing whether a filter exists at all. Some vendors forward everyone and let the client do the filtering, which is a volume business dressed as a talent business. The answer does not need to be dramatic, and a vendor with a small deliberate pool is not automatically better than one with a large one. But a vendor who has never turned anyone away is not assessing, they are forwarding.
What happens if this engineer is not what you described?
Finally, ask what the vendor carries. Not because a replacement clause makes a bad hire painless, it never does, but because a vendor's willingness to attach a consequence to their own judgment tells you how confident they actually are. The specific terms matter less than whether they hesitate.
We answer all of these in writing before you take a call, including the parts we could not conclude. See how the assessment works.