Skip to content

Uploads

An observation is uploaded before it is submitted, as its own resource. Create an upload, send the bytes in parts, complete it, and then submit the completed upload against a model.

Splitting it in two is what makes a large archive survive a bad network: a part that fails is a part you re-send, not a gigabyte you send again.

The shape of it

  1. Create — declare what you are uploading and how big it is. You get an upload identifier and where to put the bytes.
  2. Send the parts — each part is sent to a signed URL that is bound to one method and expires. Parts can be re-sent; a part whose bytes do not hash to the digest declared for them is rejected with checksum_mismatch rather than stored.
  3. Complete — the parts are assembled and the container is validated against its contract. This is where a malformed archive is caught, not later inside a job.
  4. Submit — reference the completed upload from a submission. An upload that is not complete answers upload_incomplete.

Validation happens before the bytes are trusted

The container is checked before anything decodes it: member names, the manifest, tile geometry, declared sizes against the cap. An archive whose members claim to expand to more than the cap is refused within the bound rather than while expanding — a compressed archive can declare a very small size and a very large content, and a validator that finds out by unpacking has already lost.

Size caps are per input kind and are listed in Limits. A request over the cap is refused with payload_too_large before the bytes are written, so an oversized upload costs you a rejected request and not a half-written file.

Signed URLs

The URLs you are given for parts are signed, expiring, and bound to the method they were minted for. A URL for a PUT cannot be used to read, and neither can be used after it expires. They are not secrets to keep — they are secrets that stop mattering.

What happens to an upload afterwards

It is deleted on a schedule after the job that read it finishes, and an upload that is never submitted is cleaned up as abandoned. See Privacy and retention.