Identity Boundary v5: OAuth/OIDC Account Takeover Repair
A local Docker coding eval for agents repairing a synthetic OAuth/OIDC relying party. It tests account binding, web and mobile sign-in, explicit identity linking, session isolation, token signature/issuer/audience/authorized-party checks, state and nonce binding, and legitimate recovery. Includes 107 behavioral checks and separate candidate and evaluator ZIPs. Buyers run grading locally.
Coding By Matthew Keeley
What you get
- Contents
- 1 files · Custom folder / seller-provided setup
- Made for
- AppSec and security engineering teams, coding-agent builders, and evaluation leads comparing authentication remediation across agent versions.
Setup and compatibilityHow to run the pack with your agent
This custom-format eval is supplied as the seller’s original folder. Download the ZIP and follow the seller’s instructions. EvalMarket provides no execution or grading integration for these files.
Python 3, Docker Engine or Docker Desktop, and Docker Compose on a machine that can run five disposable containers. No live OAuth provider or cloud account is needed. Initial Docker image builds may require network access. The agent edits one Python challenge/app.py file.
1. Extract the buyer ZIP and verify SHA256SUMS.txt. 2. Give only identity-boundary-candidate-v5.zip to the coding agent. 3. Have the agent follow the challenge instructions and return its edited challenge/app.py. 4. Separately extract identity-boundary-evaluator-v5.zip; keep it away from the agent. 5. With Docker running, run python3 scripts/grade_submission.py /path/to/submitted/app.py --output report.json from the evaluator directory. 6. Review the JSON score, per-case outcomes, hashes, and isolation probes. The runner generates fresh keys and removes its containers.
Scope and limitationsKnow what this pack does and does not measure
Limitations
Focused synthetic account-boundary remediation task with one issuer and one Python relying party. It does not test a real provider or full OAuth/OIDC interoperability, PKCE, consent, resource scopes, refresh tokens, revocation, browser redirects, multi-provider linking, or organization offboarding. Scores are task results, not a security certification. Buyer runs all code locally; EvalMarket does not host grading. Independent blind v5 trials and external security review have not yet been completed.
Review covers version 1 of this pack, not the readiness of your agent.
Reviews and questionsBuyer feedback and answers from the seller
Buyer reviews
How well does this eval measure what it claims? Reviews come from people who acquired it through EvalMarket.
Loading reviews…
Checking review access…
Questions
Ask about compatibility, coverage or setup. The seller can answer here.
Loading questions…
Checking sign-in…