Earlier this year I lead a QSA selection activity for a large PCI-related program I am the security lead for. Thanks to an email conversation this morning – with a friend who is crafting a QSA-related RFP – I want to share some points of consideration that I shared with her.
1. Carefully craft your RFP. Know what you want to get out of the engagement. Thus, when you read the responses – you may be able to quickly separate QSAs that did not take the time to tailor their response (and thus did not understand the engagement as a whole) from those that actually read it, understand your needs and want the business. In my case, before we allowed vendors to respond – we had a huge conference call. I allowed all the vendors to ask a few questions. In interesting observation from this call was that after the first four (of 12) vendors asked questions – there were no more questions. I guess they tend to ask the same questions. In addition, I think the conference call scared off some vendors from actually responding. They realized that we understood PCI-DSS and they were not going to be able to sell a shoddy engagement.
2. Specify your minimum experience expectations for vendor personnel that will be doing the actual work. The PCI SSC outlines minimum requirements. I tend to have higher expectations and have no problem forcing my expectations on vendors. I want a QSA assessor that has between 5-7 years of “information security” – not auditing – experience. In addition, I want someone that has a certification from the Society of Payment Security Professionals. Finally, I want a QSA assessor that has been doing PCI-related assessments / consulting for at least two years. Some QSAs will balk at these experience expectations – but again, it is my engagement and my choice and I will validate that they are meeting my experience expectations.
3. Request Resumes. Dictate that the QSA vendor provides resumes from the pool of individuals that could be performing the work. There will always be a chance they do a bait and switch on you – that is a different problem.
4. Interview the person(s) that the vendor foresees performing the engagement. The sales / account manager may also balk at this – which if they do – that should be a red flag. The serious QSA vendors should have no problem doing this. And guess what – if the vendor pulls a bait and switch on you after the work has been awarded– demand that you interview the replacement before the actual work begins. You need to be comfortable with the QSA assessor.
5. Validate Estimates. Make sure that the estimates the QSA vendor provides are realistic; this is a shared responsibility between the merchant and the QSA vendor. I cannot underscore this enough. Some vendors will low-ball their estimates for the hours needed to make themselves more appealing from a cost perspective or simply to provide a less then complete assessment. Each environment is unique so assessment times will vary. Regardless have another set of eyes review the estimates to make sure they are fairly realistic. Also, double check the hours needed for documentation. I am a big proponent of having ample documentation time. However, when vendors abuse the use of templates and do not take the time to do real, comprehensive documentation – that makes me really upset. This is probably a separate blog-post topic.
6. References. Have the QSA vendor provide references. Again, they may balk or drag their feet on this. Also, keep in mind that they will not provide references from unhappy customers. The way around this is to make sure you ask questions to the happy customers that give insight to things like timeliness, quality, business acumen, and skill sets of the QSA assessors themselves. Also, get references from clients of the QSA vendor that are in the same industry and the same merchant level as you (this should already be a requirement for in your RFP; that the QSA vendor has performed QSA-related work in your industry and at your merchant level).
7. QSA Feedback Forms. Make it known that you fully intent to provide the PCI SSC with a QSA Feedback form after the engagement with the QSA vendor. The form can be found here and can be submitted by the QSA vendor client directly to the PCI SSC. The QSA I chose never gave me a feedback form and I am debating whether or not I want to share my feedback – that I have already shared with the vendor – with the PCI SSC directly.
8. Be familiar with the QSA Agreements and QSA Requirements. You should expect to get responses from QSA vendors that are probably in violation of these two documents. I certainly did and guess what – those QSA vendors – yes, more then one – were removed from my consideration. You can find these documents here, here and here.
In summary, one way that the PCI SSC and QSA market can get better is by merchants better educating themselves on PCI-DSS and the QSA market. Merchants need to understand that they have resources to make sound QSA selection decisions as well feedback loops to help the PCI SSC perform some QA on the QSA vendors community as a whole.