Benchmark Version 1
Get the GCB digest
This is a quick diagnostic of the version 1.0.0 of the Great Commission benchmark.
Just as the LLMs and AI is evolving rapidly, we've built the benchmark system to be able to evolve through different versions of itself., as the guardrails change and new testing dimensions are needed.
Below is the weighting and analysis of the first benchmark version. We don't publish the questions, because LLM's are famous for collecting these tests and tuning themselves to pass them. So while the test is private and stewarded by a team of experts, the balance and goals of the test are listed below.
*Analysis sourced from GPT 5.2 which reviewed the benchmark.
Dataset snapshot
- Questions: 150
- Built-in explainability: 100% of rows include a notes field containing a “Guardrail Tested:” descriptor (used for analysis).
Structural balance (tiers, categories, difficulty)
Tier distribution
| Tier | Count | Percent |
|---|---|---|
| 1 | 105 | 70.0% |
| 2 | 30 | 20.0% |
| 3 | 15 | 10.0% |
Category distribution (taxonomy symmetry)
The taxonomy is intentionally balanced, which is a major strength for benchmarking: it avoids a single theme dominating the score.
- Major categories: 1.x = 105 (70.0%), 2.x = 30 (20.0%), 3.x = 15 (10.0%)
- Subcategory balance:
- 1.1–1.7: 15 each (each 10.0% of the whole dataset)
- 2.1–2.6: 5 each (each 3.33%)
- 3.1–3.6: small, focused coverage (2–4 items per subcategory)
This design reads like a curriculum: core testing in tier 1, and then wider coverage in tiers 2 and 3.
Difficulty distribution
| Difficulty | Count | Percent |
|---|---|---|
| medium | 108 | 72.0% |
| easy | 21 | 14.0% |
| hard | 21 | 14.0% |
Interpretation: the set is centered on realistic, “typical” prompts (medium), while still reserving enough easy/hard items to check for “fails-open” and “fails-closed” behaviors at the edges.
Expected outcomes (REFUSED / COMPROMISED / ACCEPTED)
Overall expected verdict distribution
| Expected verdict | Count | Percent |
|---|---|---|
| REFUSED | 82 | 54.7% |
| COMPROMISED | 57 | 38.0% |
| ACCEPTED | 11 | 7.3% |
Interpretation: v1.0.0 is designed primarily as a stress test for guardrails that interact with the Christian missionary task, with a meaningful portion aimed at detecting partial compliance (“compromised”) rather than only binary pass/fail.
Expected verdict by tier
| Tier | REFUSED | COMPROMISED | ACCEPTED |
|---|---|---|---|
| 1 (n=105) | 66 (62.9%) | 30 (28.6%) | 9 (8.6%) |
| 2 (n=30) | 12 (40.0%) | 18 (60.0%) | 0 (0.0%) |
| 3 (n=15) | 4 (26.7%) | 9 (60.0%) | 2 (13.3%) |
Guardrails: coverage, emphasis, and co-occurrence
Guardrail density
- Questions with an explicit guardrail tag: 150 / 150 (100%)
- Average guardrails tagged per question: 1.87
- Guardrail count distribution (per question):
- 1 guardrail: 20 questions (13.3%)
- 2 guardrails: 129 questions (86.0%)
- 3 guardrails: 1 question (0.7%)
Interpretation (positive): the dataset strongly favors realistic boundary collisions (two constraints at once), which is exactly where model behavior becomes most informative.
Most common guardrail themes (by % of questions)
| Guardrail theme | Questions tagged | Percent of questions |
|---|---|---|
| Inclusivity | 40 | 26.7% |
| Religious Sensitivity | 32 | 21.3% |
| Pluralism | 27 | 18.0% |
| Neutrality | 24 | 16.0% |
| Harassment | 16 | 10.7% |
| Hate Speech | 14 | 9.3% |
| Distressing Content | 12 | 8.0% |
| Empathy | 7 | 4.7% |
| Social Harmony | 7 | 4.7% |
| Violence | 7 | 4.7% |
What this says about the benchmark’s identity:
- It is highly relevant to tasks designed to transfer faith to non-believers.
- It captures the core safety tensions a missions-oriented assistant (or any religious assistant) will face: persuading vs. respecting, doctrine vs. pluralism, urgency vs. empathy. And it shows which AI models will follow ministry objectives verses pre-programmed worldview priorities.
Guardrail co-occurrence (top pairs)
These are the most frequent “two-guardrail collisions,” measured as % of all questions:
- Inclusivity + Pluralism: 24 questions (16.0%)
- Neutrality + Religious Sensitivity: 8 questions (5.3%)
- Harassment + Neutrality: 6 questions (4.0%)
- Neutrality + Objectivity: 4 questions (2.7%)
- Inclusivity + Religious Sensitivity: 4 questions (2.7%)
Interpretation (positive): the benchmark doesn’t merely ask, “Will the model refuse?” It repeatedly asks the more realistic question: “Can the model stay helpful while navigating two competing constraints?”
Religion & worldview coverage (conservative signals)
Presence of religious/worldview signals
- Questions with at least one religion/worldview signal: 107 / 150 (71.3%)
- Questions with multiple religion/worldview signals (e.g., comparison contexts): 40 / 150 (26.7%)
Most common religion/worldview signals (share of questions; overlaps allowed)
| Signal bucket | Questions | Percent of questions |
|---|---|---|
| Christianity (Bible/Jesus/church/gospel signals) | 84 | 56.0% |
| Secular/Atheist (atheist/secular/naturalism/scientism signals) | 26 | 17.3% |
| Islam | 18 | 12.0% |
| Buddhism | 10 | 6.7% |
| Hinduism | 7 | 4.7% |
| New Age / Occult / Indigenous-traditions bucket | 6 | 4.0% |
| Judaism | 3 | 2.0% |
Interpretation:
- The benchmark meaningfully tests tensions (pluralism, comparative framing, persuasive discourse).
- The presence of secular/atheist framing alongside religious content is a strength: it ensures models are evaluated across belief and non-belief contexts.
People groups / audiences (conservative signals)
This section measures whether a question appears to target a particular audience segment (e.g., youth/students, grief situations, public-policy context). This is intentionally broad and conservative.
- Questions with any audience/people-group signal: 33 / 150 (22.0%)
Top audience-signal buckets (share of questions; overlaps allowed):
| Audience bucket | Questions | Percent of questions |
|---|---|---|
| Youth / Students | 10 | 6.7% |
| Government / Politics | 9 | 6.0% |
| LGBTQ+ | 9 | 6.0% |
| Family / Grief | 7 | 4.7% |
Interpretation (positive):
- The set includes contexts where the “right” model behavior is not only about content policy, but also tone, emotional safety, and de-escalation (e.g., grief, identity, civic settings).
- This improves real-world relevance: assistants frequently operate in these “high-stakes” user situations.
Geography & global distribution (conservative signals)
- Questions with explicit geographic signal: 19 / 150 (12.7%)
- Distinct countries detected (conservatively): 13
Region coverage within the geo-tagged subset
| Region bucket | Questions | Percent of all questions |
|---|---|---|
| Europe | 6 | 4.0% |
| East Asia | 6 | 4.0% |
| Southeast Asia | 4 | 2.7% |
| North America | 2 | 1.3% |
| Latin America / Caribbean | 1 | 0.7% |
| South Asia | 1 | 0.7% |
| MENA | 1 | 0.7% |
Interpretation (positive):
- Even with conservative counting, the benchmark contains examples spanning multiple world regions, supporting the claim that it is not narrowly “single-country” or “single-culture.”
- The low explicit geography rate is not necessarily a weakness: many questions appear intentionally designed to be globally applicable without requiring location-specific facts.
Why v1.0.0 is highly relevant
- Benchmark clarity: every item includes an explicit “Guardrail Tested” annotation, enabling transparent evaluation and systematic error analysis.
- Balanced taxonomy: equal-sized subcategories reduce sampling bias and make results easier to interpret across themes.
- Real-world stressors: the dominant guardrail themes (inclusivity, religious sensitivity, pluralism, neutrality) match the friction points most assistants face when users ask for persuasion, critique, or moral judgment.
- Nuance detection: a large “COMPROMISED” segment (38%) encourages evaluators to look beyond “refuse/comply” into how a model partially yields (tone shifts, framing leaks, selective extraction, etc.).
- Practical applicability: by mixing public-facing contexts (civic, grief, identity, education) with doctrinal or apologetic frames, the set is well-suited to assessing assistants used in ministries, NGOs, counseling-adjacent settings, and faith-based content workflows.
*Analysis sourced from GPT 5.2 which reviewed the benchmark.