# Cloudflare CLI or curl: the 12 tasks

These are the prompts behind `results.csv`. Each of the 12 tasks ran twice per arm, 48 runs in all, on 2026-10-04.

## How each run was set up

- `claude -p` with model `claude-opus-5-5`, at most 25 turns, tools `Bash,Read,Glob,Grep`.
- A fresh, empty `CLAUDE_CONFIG_DIR` per run, so no skills, plugins, memory or project instructions loaded.
- Every request went to a local stand-in for the Cloudflare API v4 that returned fixed answers. No real Cloudflare account or token was involved. The stand-in itself is not published.
- `cf` was version 1.0.0-beta.12.

## The prompt

Both arms got the same scenario and task. Only the tool line differed.

```text
You are the on-call engineer for Example Co, which runs its website on Cloudflare.

Account ID: 4f2b8c1e9d3a7065b8e1c2d4f6a8b0c3
Zone: example.com (zone ID 9e3c5b7a1d2f4e6c8b0a9d7e5f3a1c2b)
Worker script: landing-site
R2 bucket: media-assets

<TOOL LINE>

Task: <TASK>

Do the task now. Finish with a single line that starts with DONE: followed by the answer or what you changed, or NOT POSSIBLE: followed by the reason.
```

curl arm tool line:

```text
Use curl against the Cloudflare API. In this environment the API base URL is <LOCAL STAND-IN URL> (use it in place of https://api.cloudflare.com/client/v4), and the API token is <DUMMY TOKEN> (send it as a Bearer token).
```

cf arm tool line:

```text
Use the Cloudflare CLI `cf`. It is installed at <PATH>, is on your PATH, and is already authenticated for this account.
```

The cf arm had no API token or base URL, so it could not fall back to curl.

## The tasks and how each was graded

A run counted as correct when the stand-in saw the right request (method and path, and the body for the three tasks that change something) and the final answer contained the task's key fact. Paths are shown relative to `/client/v4`, with `{A}` the account ID and `{Z}` the zone ID.

| Task | Prompt | Accepted request | Key fact in the answer |
|---|---|---|---|
| T01 | List the DNS records in the example.com zone and tell me how many there are. | `GET /zones/{Z}/dns_records` | 2 records |
| T02 | Which version of the Worker `landing-site` is currently deployed? Give me its version id. | `GET /accounts/{A}/workers/scripts/landing-site/deployments` | the current version id |
| T03 | List the uploaded versions of the Worker `landing-site`. | `GET /accounts/{A}/workers/scripts/landing-site/versions` or `GET /accounts/{A}/workers/workers/landing-site/versions` | a version id |
| T04 | Roll the Worker `landing-site` back to the version that was deployed before the current one. | `POST /accounts/{A}/workers/scripts/landing-site/deployments` with only the previous version at 100% | the previous version id |
| T05 | Purge the cache for https://example.com/pricing and https://example.com/index.html only. | `POST /zones/{Z}/purge_cache` with exactly those two URLs, not purge everything | a purge |
| T06 | Show the CORS policy of the R2 bucket `media-assets`. | `GET /accounts/{A}/r2/buckets/media-assets/cors` | the allowed origin |
| T08 | List the custom WAF rules on the example.com zone. | `GET /zones/{Z}/rulesets/phases/http_request_firewall_custom/entrypoint` or the ruleset by id | a rule name |
| T09 | What SSL/TLS encryption mode is the example.com zone using? | `GET /zones/{Z}/settings/ssl` or `GET /zones/{Z}/settings` | full |
| T10 | Which custom domains are attached to Workers in this account? | `GET /accounts/{A}/workers/domains` | the custom domain |
| T12 | List the names of the secrets set on the Worker `landing-site` (names only). | `GET /accounts/{A}/workers/scripts/landing-site/secrets` | a secret name |
| T13 | We are under a layer-7 attack: turn on Under Attack Mode for the example.com zone. | `PATCH /zones/{Z}/settings/security_level` with value `under_attack` | under attack |
| T15 | List the Workers routes configured on the example.com zone. | `GET /zones/{Z}/workers/routes` | the route pattern |

`cf` 1.0.0-beta.12 has no command for T10 or T15.

The task numbers come from a 15-task test of `cf cli search` run the same day. This comparison did not run T07 (list the R2 buckets), T11 (list the Turnstile widgets) or T14 (verify an API token). In that search test `cf` put the right command first for all three, so the comparison likely leaves out three easy tasks for `cf`.

Three tasks change something (T04, T05, T13). Two of them, T04 and T13, change state that a run can read back, but the stand-in's answers never change. Four runs that sent the right change and then saw an unchanged read-back ended with NOT POSSIBLE: T04 twice in the cf arm, T04 once and T13 once in the curl arm. All four had already sent the accepted request and named the key fact, so they count as correct under the rule above; no exception was made.

## The columns in results.csv

- `correct`: `request_ok` and `answer_ok` both true.
- `request_ok`: the stand-in saw the accepted request.
- `answer_ok`: the final answer contained the key fact.
- `final_line`: whether the answer ended with DONE, NOT POSSIBLE, or neither.
- `turns`, `cost_usd`, `duration_s`: as Claude Code reported them for the run.
- `bash_calls`, `help_calls`, `search_calls`: Bash tool calls, `--help` calls and `cf cli search` calls.
- `unrequested_mutations`: changes the run made that its task did not ask for.