Round 46

Vendor Risk Escalation

A mid-sized company is redesigning how it handles third-party vendor risk. The goal is to reduce exposure to security, compliance, and operational failures without slowing procurement so much that business teams start bypassing the process. The company has a limited risk team, several critical vendors in finance and customer support, and a mix of low-risk SaaS tools and high-impact infrastructure providers. The decision must balance detection speed, review effort, false positives, contractual leverage, and the ability to scale as vendor count grows. Choose the governance model that best fits the organization’s risk tolerance and operating capacity.

Dashboard

Status

DECIDED

Winner

E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools.

Majority

E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools.

Duration

5541 ms

Topic

Options

E Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools.
3.00 of 3 PPV winner
A Use continuous monitoring after onboarding as the main control, combining external security ratings, breach alerts, and periodic automated reassessments, with minimal upfront review except for the most sensitive vendors.
0.00 of 3
B Delegate first-line risk decisions to the purchasing or business owner, supported by concise policy guardrails and a risk team that only reviews exceptions, escalations, or vendors flagged by predefined triggers.
0.00 of 3
C Shift primary control into contracting by mandating stronger baseline terms, audit rights, insurance requirements, and indemnities for all vendors, while using spot checks for risk review rather than full pre-approval.
0.00 of 3
D Require a central risk review for every new vendor and every material renewal, with a standardized questionnaire, manual evidence checks, and approval gates before procurement can proceed.
0.00 of 3

Checks

Baselines

PPV no gold answer
Majority no gold answer
Comparison PPV matched majority
k samples8
Agents3
Parsed samples 24 / 24
Quality unanimous

PPV

Top Influences

Claude Sonnet 5 · delegate 1.000
GPT 5.4 mini · delegate 1.000
Gemini 3 Flash Preview · delegate 1.000

Samples

Answer Distribution

Claude Sonnet 5 E: 8
GPT 5.4 mini E: 8
Gemini 3 Flash Preview E: 8

PPV

Influences

Label Role Score
Claude Sonnet 5 delegate 1.000
GPT 5.4 mini delegate 1.000
Gemini 3 Flash Preview delegate 1.000

Voting matrix

Constructed Delegation Matrix

To / From Claude Sonnet 5 GPT 5.4 mini Gemini 3 Flash Preview
Claude Sonnet 5 0.0000.0000.000
GPT 5.4 mini 0.0000.0000.000
Gemini 3 Flash Preview 0.0000.0000.000
A 0.0000.0000.000
B 0.0000.0000.000
C 0.0000.0000.000
D 0.0000.0000.000
E 1.0001.0001.000

Embeddings

Sample Embeddings

24 samples
Claude Sonnet 5 sample #0 · E · 1536 dims E Claude Sonnet 5 sample #1 · E · 1536 dims E Claude Sonnet 5 sample #2 · E · 1536 dims E Claude Sonnet 5 sample #3 · E · 1536 dims E Claude Sonnet 5 sample #4 · E · 1536 dims E Claude Sonnet 5 sample #5 · E · 1536 dims E Claude Sonnet 5 sample #6 · E · 1536 dims E Claude Sonnet 5 sample #7 · E · 1536 dims E GPT 5.4 mini sample #0 · E · 1536 dims E GPT 5.4 mini sample #1 · E · 1536 dims E GPT 5.4 mini sample #2 · E · 1536 dims E GPT 5.4 mini sample #3 · E · 1536 dims E GPT 5.4 mini sample #4 · E · 1536 dims E GPT 5.4 mini sample #5 · E · 1536 dims E GPT 5.4 mini sample #6 · E · 1536 dims E GPT 5.4 mini sample #7 · E · 1536 dims E Gemini 3 Flash Preview sample #0 · E · 1536 dims E Gemini 3 Flash Preview sample #1 · E · 1536 dims E Gemini 3 Flash Preview sample #2 · E · 1536 dims E Gemini 3 Flash Preview sample #3 · E · 1536 dims E Gemini 3 Flash Preview sample #4 · E · 1536 dims E Gemini 3 Flash Preview sample #5 · E · 1536 dims E Gemini 3 Flash Preview sample #6 · E · 1536 dims E Gemini 3 Flash Preview sample #7 · E · 1536 dims E
Claude Sonnet 5 GPT 5.4 mini Gemini 3 Flash Preview

Samples

Agent Responses

Claude Sonnet 5 openai_compatible · anthropic/claude-sonnet-5
#0 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

Given a limited risk team, growing vendor count, and a mix of low-risk SaaS and high-impact infrastructure vendors, a one-size-fits-all gate (D) would create bottlenecks and encourage bypassing, while purely reactive monitoring (A) or contract-only controls (C...

#1 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered model directly matches the described constraints: it concentrates the limited risk team's effort on critical finance/support vendors and high-impact infrastructure, while letting low-risk SaaS tools move through lightweight self-attestation, preservin...

#2 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered model directly matches the company’s constraints: limited risk team capacity, a mix of low-risk SaaS and high-impact infrastructure vendors, and the need to avoid slowing procurement. It focuses deep review effort on truly critical/high-access vendors...

#3 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

Tiered assessment (E) directly matches the stated constraints: a limited risk team, a mix of low-risk SaaS and high-impact vendors, and the need to avoid slowing procurement enough to cause bypassing. It concentrates scarce review effort on critical finance/su...

#4 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered approach directly addresses the stated constraints: a limited risk team, mixed vendor risk profiles, and the need to scale without slowing procurement, by concentrating deep review effort only on high-impact vendors (finance, customer support, infrast...

#5 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered model directly matches the stated constraints: limited risk team capacity, a mix of low-risk SaaS and high-impact infrastructure vendors, and the need to avoid procurement bypass from excessive friction. By concentrating deep review effort only on hig...

#6 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

Given a limited risk team, a growing vendor base, and a mix of low-risk SaaS and high-impact infrastructure providers, a uniform full-review process (D) would create bottlenecks and encourage bypassing, while pure monitoring (A) or contract-only (C) approaches...

#7 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered model directly targets the stated constraint—limited risk team capacity plus a mix of low-risk SaaS and high-impact infrastructure vendors—by scaling scrutiny to actual risk instead of applying uniform effort. It preserves review depth and contractual...

GPT 5.4 mini openai_compatible · openai/gpt-5.4-mini
#0 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered assessment model best balances speed and rigor: low-risk SaaS can move quickly with lighter self-attestation, while critical finance and customer support vendors receive deeper review. It also scales better than reviewing every vendor manually, reduce...

#1 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The company needs a model that scales review effort to vendor risk, since it has limited staff, many low-risk tools, and a few high-impact critical vendors. A tiered assessment approach best balances speed and control by applying deeper scrutiny only where dat...

#2 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered assessment model best matches the company’s needs because it reduces effort for low-risk SaaS tools while reserving deeper review for critical finance, customer support, and infrastructure vendors. It also scales better than full pre-approval for ever...

#3 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The company needs a model that scales across many vendors while focusing the limited risk team on the highest-impact relationships. A tiered approach balances speed and rigor by giving low-risk SaaS tools a lighter touch and reserving deeper assessment for cri...

#4 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The company needs a model that scales with many vendors while preserving deeper scrutiny for finance and customer-support providers, which points to risk-based segmentation rather than universal heavy review or mostly-post-onboarding monitoring. A tiered asses...

#5 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered model best matches the need to protect critical finance and customer-support vendors while keeping low-risk SaaS from clogging the process. It scales better than reviewing every vendor centrally, reduces false positives and effort, and preserves speed...

#6 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The company needs a model that scales across many vendors while preserving speed for low-risk SaaS tools and applying deeper scrutiny to critical finance and support providers. A tiered assessment approach best balances effort and control by matching review de...

#7 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The best fit is a tiered assessment model because it scales review effort to vendor risk: low-risk SaaS tools get lighter controls, while critical finance and customer support vendors receive deeper scrutiny. This balances speed and false positives better than...

Gemini 3 Flash Preview openai_compatible · google/gemini-3-flash-preview
#0 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The best fit for this organization is a tiered assessment model because it efficiently allocates limited risk team resources by prioritizing high-impact vendors while allowing low-risk tools to move through procurement quickly. This approach balances the need ...

#1 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The company requires a solution that balances rigorous oversight for high-impact vendors with efficiency to prevent business teams from bypassing procurement. Option (E) provides a scalable, risk-based framework that concentrates the limited risk team's manual...

#2 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

This scenario requires balancing a limited risk team's capacity with the need to avoid business bottlenecks (procurement bypass) while maintaining security for critical vendors. Option (E) provides a scalable, risk-based approach that focuses the team's scarce...

#3 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

This scenario requires balancing the limited capacity of the risk team with the need for rigorous oversight of critical infrastructure and finance vendors. A tiered assessment model (Option E) addresses this by focusing manual resources on high-impact provider...

#4 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

A tiered assessment model balances thoroughness with efficiency by allocating limited risk team resources to the most critical threats. By categorizing vendors based on sensitivity and impact, the company prevents bottlenecks on low-risk tools while ensuring h...

#5 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

The ideal governance model balances thoroughness for critical assets with speed for low-risk purchases, making a tiered assessment the most efficient and scalable approach. Option (E) allows the limited risk team to focus their manual efforts on high-impact ve...

#6 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

This scenario requires balancing a limited risk team's capacity with the need to prevent business teams from bypassing procurement due to delays. Option (E) provides a scalable framework by filtering effort based on criticality, ensuring high-impact infrastruc...

#7 E · Adopt a tiered assessment model where vendors are classified by data sensitivity, business criticality, and access level, with deeper review only for high-risk tiers and lighter self-attestation for low-risk tools. 1536 dims

This scenario requires a balance between rigorous oversight for critical vendors and efficiency for low-risk tools to prevent process bypassing. Option (E) effectively manages limited team capacity by tiering vendors based on criticality and data sensitivity, ...

Timings

Stages

Sampling4514 ms
Embedding1015 ms
Signals12 ms
PPV0 ms

Errors

Agent and Round Errors

No errors recorded.