When a GRC or risk management platform comes with built-in artificial intelligence, that AI starts reading the organisation's most sensitive material: the risk map, open vulnerabilities, supplier contracts and audit evidence. It therefore needs to be assessed like any other third-party component, with questions, evidence and contract clauses, and not just on the strength of the demo.
This article deals with one specific question: how to run due diligence on the AI that comes inside a GRC or risk product you are buying. The general buying criteria are in GRC Software: What It Is and How to Choose and Enterprise AI platform for GRC: what to evaluate. The difference between native and bolted-on AI is covered in AI-native platform: what it means for GRC, and governance of the models your own company builds or runs in AI Governance Software: What It Is and How to Choose It. The focus here is the vendor.
Why the vendor's AI is third-party risk
The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1), published in January 2023 and intended for voluntary use, lists risks from third-party software, hardware and data among the challenges of measuring risk: the metrics of whoever develops the system may not match those of whoever operates it, and the developer may not be transparent about the methods it used. The framework covers the subject in two categories. GOVERN 6 calls for policies and procedures for risks from third-party software, data and supply chains, including contingency processes for failures or incidents in third-party AI systems deemed high-risk. MANAGE 3 calls for those risks to be monitored regularly and for controls to be applied and documented, including for pre-trained models used in development.
In a GRC platform the chain usually has more than one link. The vendor may use a language model from another company, hosted in a third party's cloud, and each link receives part of your data. Note that, according to the NIST page, AI RMF 1.0 is being revised as part of the US government's AI Action Plan. Until a new version comes out, 1.0 is the published reference.
The generative AI risks that matter in GRC
In July 2024 NIST published NIST AI 600-1, the AI RMF's Generative AI Profile, which sets out 12 risks that generative AI creates or exacerbates. Not all of them apply to a GRC tool. Five are a good guide for the assessment:
- Confabulation. Wrong content stated with confidence, commonly called hallucination. In GRC it shows up as a control mapping that does not exist, a regulation cited with the wrong article number or an evidence summary that claims more than the document does.
- Data privacy. Leakage, unauthorised use or disclosure of personal or sensitive data. Audit evidence and incident records often contain both.
- Information security. The profile points out that generative AI itself widens the attack surface, because it is vulnerable to prompt injection, including the indirect kind, with instructions hidden in data the system will retrieve. In GRC, a questionnaire completed by a supplier or a PDF of evidence uploaded by a third party is exactly that kind of data.
- Value chain and component integration. Non-transparent or untraceable integration of third-party components and inadequate supplier vetting across the AI lifecycle.
- Human-AI configuration. Automation bias and over-reliance, when people start accepting the machine's suggestion without reviewing it.
For each risk, the question for the vendor is the same: which control is in place, and what evidence shows that it works.
Model and data governance: what to ask
- Which models and providers are in the chain. Ask for the list of models by feature, who supplies them and where they run. The NIST profile suggests keeping an inventory of third parties with access to organisational content and maintaining lists of approved AI providers (GV-6.1-007).
- Whether your data trains models. Ask whether customer data, prompts or outputs are used to train or fine-tune the vendor's or the provider's models, and where that is stated in the contract, not just in the privacy policy.
- Retention. How long prompts, outputs, logs and indexes derived from your documents are kept, and how they are deleted when the contract ends.
- Separation between customers. How the vendor ensures that the context retrieved to answer one customer never includes another customer's documents.
- Evaluation and testing. Which tests measure error in the tasks the AI performs, such as classifying, mapping and summarising, and whether the results are shared. The AI RMF asks for the model to be explained, validated and documented (MEASURE 2.9) and for security, resilience and privacy risk to be evaluated and documented (MEASURE 2.7 and 2.10).
- Change management. How, and how far in advance, the customer is told when the model or provider changes. The AI RMF includes change management, incident response and decommissioning in post-deployment monitoring plans (MANAGE 4.1).
- Defence against injection. How the platform handles instructions embedded in documents uploaded by third parties, and whether the AI can take actions or only make suggestions.
Transparency and human oversight you can verify
“Human-in-the-loop” has become a brochure phrase. The AI RMF asks for human oversight processes to be defined, assessed and documented (MAP 3.5) and for mechanisms and assigned responsibilities to supersede, disengage or deactivate AI systems whose performance is inconsistent with their intended use (MANAGE 2.4). In a demo, that becomes four checks:
- Each suggestion shows which data it started from and why it reached its conclusion.
- Rejecting or adjusting a suggestion is logged, with author and date.
- An administrator can switch off an AI feature without stopping the rest of the platform.
- The interface makes clear what was generated by AI and what was approved by a person, which helps to contain automation bias.
Architecture tests, such as changing a data point and seeing whether the priority changes, are in AI-native platform. The question here is whether oversight can be audited.
There is also a legal point. Article 20 of the LGPD, Brazil's General Data Protection Law, gives data subjects the right to request a review of decisions taken solely on the basis of automated processing of personal data that affect their interests, and paragraph 1 requires the controller to provide clear information on the criteria and procedures used, subject to commercial and industrial secrecy. Most AI use in GRC decides about controls and risks, not about people. If a feature starts deciding about individuals, the organisation, as controller, must be able to explain the decision, and that depends on what the vendor documents.
Data, residency and the LGPD
In most contracts the GRC vendor processes personal data on the customer's behalf and acts as a processor (operador), which under Article 39 of the LGPD must follow the controller's instructions. When the language model belongs to another company, that company becomes one more link in the processing chain, a sub-processor. Four points in the law carry more weight when AI is involved:
- International transfers (Article 33). If inference takes place outside Brazil, the transfer has to fit one of the cases in the article, such as a country with an adequate level of protection or standard contractual clauses. Resolution CD/ANPD No. 19/2024 of the ANPD, Brazil's National Data Protection Authority, approved the regulation and the text of the standard clauses. Ask where the model runs, not just where the database sits.
- Security by design (Article 46). Processing agents must adopt technical and administrative security measures, and paragraph 2 says these must be observed from the design stage of the product or service.
- Records of processing (Article 37). Controller and processor must keep records of the processing operations they carry out, which includes what the AI receives and returns.
- Incidents (Article 48). It is the controller that notifies the ANPD and data subjects. The contract must require the vendor to report promptly any incident involving the AI or the model provider, so that notification can be made on time.
Hosting, sub-processors and transfers for the GRC platform as a whole are covered in GRC platform in Brazil: how to choose the right one, and ANPD deadlines and resolutions in ANPD regulations. In the financial sector, Resolution No. 4,893/2021 of the CMN (Brazil's National Monetary Council) has its own rules for contracting data processing, data storage and cloud computing services, discussed in Third-party risk and CMN Resolution 4,893.
ISO/IEC 42001: what a certificate proves
ISO/IEC 42001:2023 specifies requirements for establishing, implementing, maintaining and continually improving an AI management system in organisations that develop, provide or use AI systems. It is a voluntary, certifiable standard that follows the common structure of ISO management system standards, the same as ISO/IEC 27001. A certificate says the management system has been audited. It does not say that every AI output is correct. When you receive one, check:
- the scope: whether the product you are buying and its AI features are inside it;
- who issued it and whether the certification body is accredited;
- its validity and the date of the last audit;
- whether the language model provider, when it is another company, has been left out of scope.
The NIST AI RMF has no certification programme. “Aligned with the NIST AI RMF” is the vendor's own claim, which you can test by asking for the mapping between the GOVERN, MAP, MEASURE and MANAGE functions and the controls it applies.
Clauses worth putting in the contract
NIST's Generative AI Profile includes suggested actions that turn into clauses with little adaptation:
- contracts and service level agreements that set out content ownership, usage rights, quality standards, security requirements and content provenance expectations (GV-6.1-004);
- the right to evaluate the third party's generative AI processes and standards (GV-6.1-006);
- an incident response plan for third-party generative AI technologies, with defined owners, regular rehearsals and review against breach notification and data protection laws (GV-6.2-003);
- records of incidents involving third-party AI data and systems (GV-6.2-002).
Add the privacy clauses that apply to any processor, detailed in Third-party risk and the LGPD: how to protect your data, and align incident notice with your incident response plan.
Short questionnaire for the vendor
| Topic | Question | Evidence to request | Reference |
|---|---|---|---|
| Model chain | Which models and providers, by feature? | List of providers and sub-processors, with locations | AI RMF GOVERN 6; AI 600-1 GV-6.1-007 |
| Data use | Do data, prompts or outputs train models? | Contract clause | AI 600-1 GV-6.1-004; LGPD Art. 39 |
| Retention | How long are prompts, outputs and logs kept? | Retention policy and deletion procedure | AI 600-1, data privacy |
| Residency | Where does inference take place? | Processing regions and transfer mechanism | LGPD Art. 33; ANPD Resolution No. 19/2024 |
| Quality | How is AI error measured in GRC tasks? | Test results and known limitations | AI RMF MEASURE 2.9; AI 600-1, confabulation |
| Security | How does the platform resist prompt injection in third-party documents? | Security testing that covers the AI | AI 600-1, information security; LGPD Art. 46 |
| Oversight | Are rejections logged? Can the AI be switched off? | Demo and audit trail | AI RMF MAP 3.5 and MANAGE 2.4 |
| Changes | How is the customer told about a change of model or provider? | Change management procedure | AI RMF MANAGE 4.1 |
| Incidents | How quickly does the vendor report an incident involving the AI? | Contract clause and response plan | AI 600-1 GV-6.2-003; LGPD Art. 48 |
| Certification | Does the ISO/IEC 42001 certificate cover this product? | Certificate with scope | ISO/IEC 42001:2023 |
Where Eagle stands on these questions
Eagle, Futture's Enterprise Risk Intelligence platform, should go through the same checklist. According to the website, Eagle's AI supports classification, identification of relationships, contextual prioritisation and recommendations; the user reviews, approves, rejects or adjusts every relevant result; and each recommendation keeps its context, data source, rationale, confidence level and decision history (how AI works in Eagle). On security and governance, the site lists role-based access control, environments segregated by organisation and an audit trail, with the caveat that capabilities may vary with the environment and contracted scope (security and governance). The models and providers used, where inference runs and the retention policy are not on the site. They are questions to ask in writing, including of us.
Next step
Before your next demo, turn the table above into an annex to your due diligence questionnaire and send it alongside the information security one. The written answers, not the demo, are what goes into the supplier file and what an auditor will ask for later. For the full third-party assessment cycle, see Vendor risk management: best practices.
Further reading: AI Governance Software: What It Is and How to Choose It, AI-native platform: what it means for GRC and GRC platform in Brazil: how to choose the right one.
Frequently asked questions
Why assess a GRC vendor's AI separately?
Because it reads sensitive data such as risks, vulnerabilities, contracts and evidence, and often relies on a model from another company. The NIST AI RMF treats this as third-party risk (GOVERN 6 and MANAGE 3), with its own policies, monitoring and contingency plans.
Which questions must be in the due diligence?
Which models and providers are used, whether your data trains models, how long prompts and outputs are kept, where inference takes place, how human oversight is recorded and how the customer is told about model changes and incidents.
Is an ISO/IEC 42001 certificate enough?
Not on its own. It shows the vendor has an audited AI management system within its stated scope. Check that the product and its AI features are in scope, who issued the certificate and whether the model provider was left out. The certificate does not attest that every AI output is correct.
Does using a vendor's AI with servers outside Brazil breach the LGPD?
Not necessarily. International transfers of personal data are allowed in the cases set out in Article 33 of the LGPD, such as the standard contractual clauses approved by ANPD Resolution No. 19/2024. What matters is knowing where inference takes place and which mechanism covers the transfer.
Official references
- NIST AI 100-1 — Artificial Intelligence Risk Management Framework (AI RMF 1.0)
- NIST — AI Risk Management Framework
- NIST AI 600-1 — AI RMF: Generative Artificial Intelligence Profile
- ISO/IEC 42001:2023 — AI management systems
- Law No. 13,709/2018 — Brazil’s General Data Protection Law (LGPD)
- ANPD Resolution No. 19/2024 — International Data Transfer Regulation
About the author
Founder, Futture.ai | CISSP
Cybersecurity, governance and risk management specialist with over 25 years of industry experience. CISSP-certified, founder of Network Secure and Futture.ai.