RLTasksby ulam.ai
Sign inAll tasks
Operations & simulationsCloudFinOps

CloudFinOps

A production-remediation benchmark where agents reduce a generated cloud estate's spend without sacrificing performance, resilience, retention, governance, or reproducibility. It reconciles provider-shaped billing, telemetry, inventory, dependencies, incidents, commitments, and Terraform state, then tests open-ended multi-resource changes through staged rollout, recovery, approval, and evidence-backed savings verification.

Decision makingStateful environmentRLVR
Version2.0.0
Environments1
RewardScalar · 0–1
DeliveryPrivate ZIP
01 Task contract

A real, versioned RL environment.

A production-remediation benchmark where agents reduce a generated cloud estate's spend without sacrificing performance, resilience, retention, governance, or reproducibility. It reconciles provider-shaped billing, telemetry, inventory, dependencies, incidents, commitments, and Terraform state, then tests open-ended multi-resource changes through staged rollout, recovery, approval, and evidence-backed savings verification.

Agent objective

  • Interact with the supplied stateful environment.
  • Produce verifier-checkable actions or artifacts.
  • Maximise scalar reward under the package contract.

Evaluation

  • Private verifier.
  • Reported reward range 0–1.
  • Package-specific public and private checks.

Delivery boundary

  • Private object stored in Cloudflare R2.
  • Authenticated entitlement required.
  • Short-lived signed URL per download.
02 What you will work on

Distinct environments, one demanding research contract.

Enough detail to understand the intellectual terrain; generated instances, hidden mechanisms, and solution paths remain inside the private package.

EnvironmentMathematical or technical frontierAdaptive research problem
SaaS growthForecast-aware cloud optimizationRightsize and autoscale without mistaking a quiet window for permanent capacity evidence; size commitments only after remediation.
Retail promotionBursty capacity and scale-outCapture savings around promotional spikes while preserving fast scale-out, headroom, and staged rollback.
Fintech disaster recoveryResilience, retention, legal holdDistinguish idle waste from failover dependencies and preserve restore, retention, approval, and attestation requirements.
Media batchPeriodic compute and storageCoordinate scheduled capacity, storage tiers, batch-worker scaling, commitments, and performance verification.
Marketplace variantsCyclical and regional operationsHandle month-end amplification, cache/database effects, provider-shaped evidence, regional failover, drift, and recovery.
03 Why it is interesting

What the supplied evaluation reveals.

A conservative evidence-only policy passed all 16 development and private cases with 0.9975 mean reward and 41.77% mean verified savings. Every hard gate passed, including SLO preservation, deletion safety, governance, IaC approval, and recovery. The result is package-local rather than a hosted blind leaderboard, but it demonstrates that the benchmark rewards safe, evidenced remediation rather than raw cost cutting. Validation reports 25 package and 84 workspace tests.

We publish aggregate behavior and task structure, while withholding generated instances, hidden labels, exact successful probes, private checks, and solution trajectories.

04 Supplied evaluation

Observed evaluation result.

Shown with its provenance and limitations; it is not a performance guarantee.

i
Methodology matters

Reported result from the evaluation artifact supplied with this package.

Evaluated system / policyGPT-5.6 Pro

As identified by the supplied artifact.

Mean reward0.9975

4 reported runs.

Result artifactIncluded

cloudfinops_gpt56_score_report.md

Public result record

Machine-readable provenance and the exact displayed metric are available in results.json.

Open result JSON
05 Private delivery

The package stays off the public website.

The paid ZIP will live in a private R2 bucket. Vercel authorizes the buyer and issues a 2–5 minute object URL; R2 serves the bytes directly.

Included with purchase

  • Exact package version 2.0.0
  • Environment and task contracts
  • Verifier or scoring interface
  • Supplied reference/evaluation artifacts
  • Purchase record and licence v1.1
delivery flow
Authenticated buyer + entitlement check
+ private R2 object + 2–5 minute signed URL
= direct, auditable download

Package SHA-256
1e98ca2e3bd17639b4ec6aa4ef91b0e4c8af3a0c309651ac978512b10a53fcc6
06 Licence v1.1

Commercial use, without exclusivity.

One purchase licenses this identified item to one legal organisation for worldwide, perpetual commercial model training, evaluation, research and development. Redistribution and resale of the package are not permitted.

Read the full licenceYotta Content LTD · business customers only
CloudFinOps

Ready to add this environment?

Back to marketplace
Stripe checkout

Business purchase confirmation

Sign in or create an account, then complete secure Stripe Checkout. Access is granted only by the verified payment webhook.