Datasets

Upload historical ticker data and use it in the standard backtesting workflow.

Backtest against a CSV you upload instead of a managed exchange: create a dataset, PUT the file to a presigned URL, finalize it to trigger ingest, then prepare/execute exactly as normal but with the reserved exchangeId: user.

v1 is ticker data only — type is always "ticker". instrument must be a plain spot pair (BASE/QUOTE, exactly one /); derivative forms (BTC/USDT:USDT) are rejected.

Creating a dataset

POST /datasets — creates the dataset and its first upload session in one call: a presigned URL your client PUTs the file to directly, no API credentials involved in that PUT.

curl -X POST https://api.qtsurfer.net/v1/datasets 
  -H "Authorization: Bearer $TOKEN" 
  -H "Content-Type: application/json" 
  -d '{"name":"My BTC ticks","instrument":"BTC/USDT"}'

DatasetCreated (201) — the metadata available immediately after creation plus the upload session. It is not yet the full Dataset: lifecycle fields such as createdAt, currentVersionId, range, and cadence are obtained from GET /datasets/{datasetId} after the relevant lifecycle stages.

{
  "datasetId": "ds_3f9a1c2e7b0d4a5f", "name": "My BTC ticks",
  "type": "ticker", "instrument": "BTC/USDT",
  "uploadId": "up_1a2b3c4d5e6f7a8b",
  "upload": {
    "url": "https://storage.qtsurfer.com/.../uploads/up_1a2b3c4d5e6f7a8b/raw.csv?X-Amz-...",
    "expiresInMinutes": 15
  }
}

uploadId is what you pass to finalize; upload.url is the presigned target — PUT the raw CSV there directly, no Authorization header.

Lost this response? Nothing lost — call POST .../uploads on this dataset’s id and you get the very same upload session back, as long as you haven’t finalized it yet.

Errors: 400 invalid request, or instrument isn’t a plain spot pair · 409 dataset name already taken · 429 your tier’s dataset count limit is reached — delete one, or upgrade.

Opening a new upload session

POST /datasets/{datasetId}/uploads — get a fresh upload session for a dataset you already have: a corrected file, or the next chunk of history. Same idempotency contract as POST /datasets’s own upload half: at most one session is open per dataset at a time, so calling this again before finalizing just hands back that same session — safe to retry if a response gets lost. Once a session has been finalized (successfully or not), the next call here opens a genuinely new one.

curl -X POST https://api.qtsurfer.net/v1/datasets/$DATASET_ID/uploads 
  -H "Authorization: Bearer $TOKEN"

201 — the same {uploadId, upload} shape POST /datasets returns, without the dataset metadata around it:

{
  "uploadId": "up_1a2b3c4d5e6f7a8b",
  "upload": {
    "url": "https://storage.qtsurfer.com/.../uploads/up_1a2b3c4d5e6f7a8b/raw.csv?X-Amz-...",
    "expiresInMinutes": 15
  }
}

Errors: 404 no such dataset for this user.

Uploading the file

CSV format. Header row required. timestamp (ISO-8601, or numeric epoch seconds / millis / micros — detected from the first row, then enforced for every later row) and close are required columns. Optional: open, high, low, volume, quoteVolume, bid, bidSize, ask, askSize. Cadence and timestamp unit are discovered from the data, not declared.

curl -X PUT "$UPLOAD_URL" --data-binary @my-btc-ticks.csv

Finalizing an upload (triggering ingest)

POST /datasets/{datasetId}/uploads/{uploadId}/finalize — call once the PUT above completes. Enqueues ingest and returns immediately; poll GET .../uploads/{uploadId} below. Idempotent while the upload is still open — a repeat finalize before it has produced a version returns the same jobId rather than enqueueing a second ingest. Once it HAS produced a version, uploadId is spent: finalizing it again is a 409, even with different bytes freshly PUT to the same URL — open a new upload session instead of reusing a spent one.

curl -X POST https://api.qtsurfer.net/v1/datasets/$DATASET_ID/uploads/$UPLOAD_ID/finalize 
  -H "Authorization: Bearer $TOKEN"
# → 202 {"jobId": "dataset-upload:.../ds_3f9a1c2e7b0d4a5f:up_1a2b3c4d5e6f7a8b"}

Errors: 404 no such dataset for this user; uploadId wasn’t issued for this dataset (never minted, or minted for a different one); or nothing was PUT to upload.url yet — a finalize with nothing to finalize · 409 uploadId already produced a version (the error message names it) · 413 the uploaded file exceeds your tier’s size limit for a dataset.

Polling ingest

GET /datasets/{datasetId}/uploads/{uploadId} — poll after finalize until status is ready or failed. Also reports uploading (finalize not called yet, but the file was PUT) before you finalize at all. Postgres-backed once a version exists, so ready/failed are permanent answers; uploading/ingesting reflect in-flight state that can itself age out (see the 404 case below).

Response — DatasetUploadState

DatasetVersion — one successfully ingested upload

curl https://api.qtsurfer.net/v1/datasets/$DATASET_ID/uploads/$UPLOAD_ID 
  -H "Authorization: Bearer $TOKEN"

{
  "uploadId": "up_1a2b3c4d5e6f7a8b",
  "status": "ready",
  "version": {
    "datasetId": "ds_3f9a1c2e7b0d4a5f", "id": "dsv_8e2b4f19c6a03d7e",
    "bytes": 4831022, "rows": 86400, "cadence": "1s",
    "timestampUnit": "iso", "gaps": 0, "largestGapSteps": 0
  }
}

Errors: 404 no such dataset for this user, or genuinely nothing known about this uploadId — no version, no in-flight job, nothing was ever PUT to its upload URL.

Dataset shape

Both GET /datasets and GET /datasets/{datasetId} return this — from/to/cadence mirror the current version’s own discovered range and cadence, so you don’t need a second call to see what a dataset covers.

Listing your datasets

GET /datasets — every dataset you’ve created and not deleted, most recently created first. Never a 404 — an empty array if you have none, same convention as GET /strategies.

curl https://api.qtsurfer.net/v1/datasets -H "Authorization: Bearer $TOKEN"

Getting a dataset

GET /datasets/{datasetId} — a Dataset plus a _links.self.

curl https://api.qtsurfer.net/v1/datasets/$DATASET_ID -H "Authorization: Bearer $TOKEN"

Errors: 404 no such dataset for this user.

Deleting a dataset

DELETE /datasets/{datasetId} — soft delete. Stops appearing in the list/get endpoints and can no longer be prepared from, but its object data is reclaimed later rather than purged inline, so a backtest already running against one of its versions isn’t disrupted.

curl -X DELETE https://api.qtsurfer.net/v1/datasets/$DATASET_ID -H "Authorization: Bearer $TOKEN"
# → {"datasetId": "ds_3f9a1c2e7b0d4a5f", "deleted": true}

Errors: 404 no such dataset for this user, or already deleted.

Backtesting against a dataset

Once a version is ready, prepare and execute exactly as against a managed exchange, but with exchangeId: user and datasetId in place of instrument:

curl -X POST https://api.qtsurfer.net/v1/backtest/user/ticker/prepare 
  -H "Authorization: Bearer $TOKEN" 
  -H "Content-Type: application/json" 
  -d '{"datasetId":"ds_3f9a1c2e7b0d4a5f","from":"2026-03-14","to":"2026-03-15"}'
# → 202 {"jobId":"5ikYAMIO...","datasetId":"ds_3f9a1c2e7b0d4a5f","datasetVersionId":"dsv_8e2b4f19c6a03d7e"}

execute is unchanged — same request body as against a managed exchange, since the instrument and range are recovered from prepareJobId either way. See docs/backtest_execute.md for the full prepare/execute reference, including PrepareRequest’s datasetId/datasetVersionId fields and the dataset-backed coverage shape on PrepareJobState (cadence/gaps/largestGapSteps in place of the hour-walked totalHours/hoursWithData/hoursWithoutData).