Media pipeline

Uploads are validated from their bytes (allowed raster formats only, a decompression-bomb pixel budget), re-encoded with all metadata stripped (EXIF/GPS/device — privacy #215), and stored in object storage (local dev or S3/MinIO). A background worker (Media.VariantWorker) then derives the processing outputs; originals of non-raster uploads are simply served as-is.

The library also holds documents (PDF, #481) — see below; everything through "Alt text and usage tracking" describes the image side, which is unchanged.

Derived variants

Downscales never upscale; the crop is skipped when the source is smaller than its box. Public delivery builds responsive srcsets from the downscales plus the original — cropped variants are excluded (a different aspect ratio in an srcset would let the browser pick the wrong framing); consumers ask for crops by label from the variants map (JSON:API/GraphQL expose it).

Modern formats (#473)

Every variant is written once per output format: the source's own format, plus each configured alternate. WebP is on by default; AVIF is opt-in because encoding it costs roughly an order of magnitude more CPU per image, which is a real bill on a bulk regeneration.

config :kiln_cms, :image_variants,
  formats: [:webp],     # add :avif to opt in
  webp_quality: 82,
  avif_quality: 50,     # AVIF's scale isn't JPEG's — 50 ≈ WebP 80
  jpg_quality: 82

Quality applies to the lossy formats only. There is no png_quality: libvips has no quality knob for PNG (it has compression) and Image.write discards :quality for it outright, so offering the setting would mean a config value that silently does nothing. A quality outside 1..100, or a non-integer (System.get_env/1 returns strings), falls back to the default rather than being passed through — a rejected write produces no variant, so the alternative to clamping is a config typo emptying the library.

Keys in the variants map name exactly one file. The bare label is the source format — that is the <img src> fallback, and it is how every map written before #473 is keyed — and alternates take a <label>.<format> suffix:

thumb        image/jpeg   /uploads/…-thumb.jpg
thumb.webp   image/webp   /uploads/…-thumb.webp
card         image/jpeg   /uploads/…-card.jpg
card.webp    image/webp   /uploads/…-card.webp
full.webp    image/webp   /uploads/…-full.webp

full is a full-size re-encode written in the alternate formats only — the source-format equivalent is the original itself. It exists because a matching <source> replaces the <img>'s srcset rather than adding to it: without a candidate at the original's width, a WebP-capable browser could never reach anything wider than the largest downscale, and every content image would quietly render smaller than it did before.

Each entry carries its own content_type, which is what a <picture> <source type=…> needs. A source format that is also a configured alternate (a WebP upload with WebP variants) is written once, under the bare label — two identical files under two keys would double storage and put the same bytes in a <picture> twice. Animated sources (GIF) get no alternates: variants are already flattened stills, so transcoding would spend encoder time producing a second still of an image whose animation is the point.

An unknown format name in config is dropped rather than raising — a typo should cost the site its WebP variants, not its uploads. A format whose encoder is missing from the libvips build fails that one write and leaves the rest, including the source-format fallback.

A failed full-size write is additionally recorded, on the item's variant_failures map (#1000) — full-size only, for now: a failed thumb.avif is still logged and nothing more, so an item can still be re-enqueued on every run when a responsive label's encoder is missing (#1036). Without that record, "no full.webp" is ambiguous: it means either "the write has not happened yet" — which a regeneration should repair — or "this source can never be encoded to WebP", which it should leave alone. A 17000px panorama is past libvips' 16383px WebP ceiling and will never gain one, and re-decoding it on every run is the standing tax Regeneration exists to avoid. The map is rewritten on every run rather than merged, so a libvips upgrade that gains an encoder, or a re-crop that brings the source under the ceiling, clears it. It is not part of the headless surface.

Delivery renders <picture>

KilnCMS.Media.Presentation exposes the two halves separately, and the split is the point:

• srcset/1 — the source-format downscales plus the original. A browser picks from a srcset on width alone, so mixing encodings there would hand a WebP-less client a WebP, and offer two entries at the same width to choose between arbitrarily.
• sources/1 — one srcset per alternate encoding (a key with a format suffix), each tagged with its type, most efficient first (AVIF, then WebP). That ordering is the whole contract of <picture>: the browser takes the first type it supports and stops looking. A WebP upload therefore gets no <source> at all — its variants are already the <img> fallback, and offering them again would only drop the original from consideration.

A format is offered only when its ladder reaches the item's own width (#919). The replacement semantics above cut both ways: a <source> whose alternates stop short of the original does not merely miss the top rung, it takes the original out of consideration for every browser supporting that format. full.<format> normally carries that rung, and a write that fails is dropped with only a log line — so a 17000×2000 panorama whose full.webp exceeds libvips' WebP dimension limit would otherwise keep emitting <source type="image/webp" srcset="…thumb 400w, …medium 1024w">, and every WebP-capable browser would upscale a 1024px render of it.

Suppression is not free, and costs more than the codec: blocks render sizes="(max-width: 768px) 100vw, 768px", so a 2x display wants ~1536px and picks the original off the <img> ladder — a full-resolution download where the buggy <source> served a 1024px WebP. It is still the right trade (it is what every WebP-less browser already gets, and the alternative is a visibly upscaled image), but the real remedy is making sure full.<format> exists. The check is per format, so losing WebP's top rung does not cost AVIF its <source>.

Image and gallery blocks render a <picture> wrapping the existing <img>. An item with no alternates produces no <source> elements, so the markup degrades to exactly the <img> that was there before.

Bulk regeneration

A configuration change only ever reaches images uploaded after it. To roll it out over the existing library:

mix kiln.media.regenerate_variants          # only what's missing (a format rollout)
mix kiln.media.regenerate_variants --all    # everything (a quality/width change)

Admins can do the same from /media → Regenerate variants.

An image whose recorded size is over the pixel cap (max_pixels, 50 MP) is skipped and counted, not enqueued: its job would read the original only to refuse it, and for an image backend's item that read is an API download, repeated on every run.

Both enqueue KilnCMS.Media.VariantWorker on the throttled :media Oban queue rather than processing inline, and at the lowest Oban priority — :media is a concurrency-3 queue shared with upload processing and video probes, and jobs are otherwise fetched in id order, so an un-deprioritised bulk run would sit ahead of every subsequent upload for hours. The admin button runs the scan itself in a supervised task, because Oban's unique inserts are one transaction per row and a large library would otherwise block the LiveView past its heartbeat.

Jobs are unique per item for an hour, keyed with a "source" => "regenerate" marker so they dedupe against other regeneration runs only: Oban's default unique states include :completed, so without it every image uploaded in the previous hour would collide with its own upload job and be silently skipped.

Each run replaces the variant map wholesale and then deletes the blobs the old map named — every other deletion path in Kiln reads the current map, so without that a single run over a large library would orphan tens of thousands of files nothing could ever find again. Originals are never rewritten — published snapshots and fired artifacts point at them by key.

"Missing only" means nothing a run would add, not has every format: an animated GIF (which gets no alternates by design) and an image narrower than every responsive target (which produces no variants at all) both count as current, or the rollout mode would re-decode them on every run for ever.

Focal point

Every image carries a focal point (focal_x/focal_y, fractions of the dimensions, default center). Editors set it by clicking the preview in the media library (/media); changing it regenerates the focal-aware crops, and public delivery emits object-position on image blocks so any theme cropping via object-fit keeps the subject in frame. Both fields ride the public media APIs for headless consumers.

In-admin editing

The media detail panel offers rotate left/right and flip horizontal/vertical (Media.Transform). Edits write the result under a new storage key and repoint the item — the previous file is deliberately kept, because published content embeds media snapshots captured at write time and fired artifacts keep serving the old URL until re-publish. The focal point is carried through the geometry (rotating the image rotates the point), and variants regenerate from the edited original.

On-the-fly transforms

The fixed variants above cover Kiln's own templates. Anything else — a 16:9 card at 1080px, a square avatar, an AVIF for a browser that takes one — is a transform URL, rendered on first request and cached:

GET /media/<id>/t/w_1080,ar_16:9,fm_auto,v_3f2a9c01

The <ops> segment is key_value pairs joined by commas:

Output is never upscaled: a box larger than the source (or than the largest crop window of the requested ratio) comes back at the largest size the source can supply, in the requested shape. Animated GIFs become a still of their first frame. Only JPEG, PNG, WebP and GIF originals transform; anything else is a 422.

Don't hand-build these URLs. The builders do the snapping, versioning and signing: KilnCMS.Media.ImageTransform.url/2 / srcset/3 and the <KilnCMSWeb.MediaComponents.transform_img> component inside Kiln, transformPath / client.imageUrl in the JS SDK and KilnClient.image_url/2 in the Elixir client. All three are held to the same checked-in test vectors (clients/js/test/fixtures/image_transform_vectors.json).

Bounds

Every distinct parameter set is a decode, a resize and an encode, so an open transform endpoint is a CPU and storage amplifier. It is bounded at every layer:

• Unsigned URLs are held to an allowlist. w/h must come from the size ladder (16 32 48 64 96 128 256 384 640 750 828 1080 1200 1920 2048 3840), ar from 1:1 4:3 3:4 3:2 2:3 4:5 5:4 16:9 9:16 21:9, q from 50 75 90. Anything else is a 400 that names the allowed values. The SDK builders snap up to the ladder, so a browser-side caller never meets it.
• Signed URLs may use any value within the hard limits. s is HMAC-SHA256 over "<id>/<canonical ops>", truncated to 16 bytes and base64url-encoded. The key is KILN_IMAGE_TRANSFORM_KEY when set (share it with a server-side frontend so it can sign), else derived from SECRET_KEY_BASE — which Kiln's own templates can use and nobody else can. A bad signature is a 403. KILN_IMAGE_TRANSFORM_UNSIGNED=false turns the allowlist path off, so only signed URLs serve.
• Hard limits apply to both. No output side over 4000px after dpr (a box is shrunk as a whole, keeping its ratio), and no source over the upload pixel cap (config :kiln_cms, :media, max_pixels:, 50 MP) — refused from the recorded dimensions before anything is decoded, and re-checked on the decoded header.
• Refusals are cheap. Parameters and signature are checked before the item is read at all, so a malformed or unauthorized request costs no database or storage work.
• Renders are metered. Every request spends the per-IP :media_transform bucket (1,200/min); a cache miss also spends :media_render (120/min) — the actual cost bound — and waits for a slot in KilnCMS.Media.TransformGate, which caps concurrent renders per node (half the schedulers, at least 2). A request that can't get a slot within 10 s is a 503 with Retry-After.
• Storage is budgeted per item. An item keeps at most 200 derivatives; past that a transform is still rendered and served but not kept.

All of it is tunable:

config :kiln_cms, :image_transforms,
  allow_unsigned: true,                 # KILN_IMAGE_TRANSFORM_UNSIGNED
  signing_key: nil,                     # KILN_IMAGE_TRANSFORM_KEY
  auto_avif: false,                     # KILN_IMAGE_TRANSFORM_AUTO_AVIF
  sizes: [16, 32, 48, 64, 96, 128, 256, 384, 640, 750, 828, 1080, 1200, 1920, 2048, 3840],
  aspect_ratios: ~w(1:1 4:3 3:4 3:2 2:3 4:5 5:4 16:9 9:16 21:9),
  unsigned_qualities: [50, 75, 90],
  max_dimension: 4000,
  max_source_pixels: 50_000_000,        # default: the upload cap
  max_derivatives_per_item: 200,
  max_concurrency: 4,                   # default: half the schedulers, min 2
  queue_timeout: 10_000,
  max_queue: 64

Change sizes and the SDKs need the same ladder (sizes: option on each builder), or their snapped URLs will 400.

Caching

A transform is cached twice, once per layer:

• Derivatives, in blob storage next to the originals (t-<digest>.<ext>, private storage for an item outside the :public audience). The key is a digest of what is rendered — the original's storage key, the crop window, the output size, format and quality — so w_800,dpr_2 and w_1600 share one file, as does every width past the source's own. A rotate or replace (new storage key) or a focal move (new crop window) simply misses, so a derivative is never stale. KilnCMS.CMS.MediaDerivative rows record each one, so the budget can be counted, derivatives the item can no longer produce are deleted before they count against it, and a permanent delete from the media library removes them with the original.
• HTTP. The response carries a strong ETag (the digest; If-None-Match is answered from the plan alone, without touching storage). v is a hash of the item's url and focal point — the two things that change a transform's pixels for a given URL — computed by every builder from fields the public APIs already return. A URL whose v matches is served public, max-age=31536000, immutable; one with no v or a stale one gets the current image with public, max-age=300. An item outside the :public audience is private, no-store, like its download.

Put a CDN in front as for originals (below). fm_auto responses vary on Accept; a CDN that ignores Vary should be given explicit fm_webp/fm_jpg URLs instead.

Visibility

/media/:id/t/… reads the item through the same policy-checked read as /media/:id/download (MediaDownloadController.readable_item/2), under the session's actor: a gated item is a 404 to anyone without its audience, and a quarantined upload is a 404 to everyone, editors included.

Alt text and usage tracking

MediaItem.alt has always existed and has always been optional, which means it is missing on exactly the images nobody thought about. Two things address that (#403).

Alt text is enforced at publish, not at upload

Off by default:

config :kiln_cms, :media, require_alt_text: true

With it on, :publish and :publish_scheduled are refused when the document contains an image block whose alt is blank, and the error names every offending image at once so an editor can fix them in one pass.

It checks the alt that actually renders — the block's alt field. That is what the image block's renderer and KilnCMSWeb.BlockComponents emit; the library item's MediaItem.alt is the editor's default when inserting an image, not what ships. Checking the library row instead would get it wrong in both directions: refusing a page whose block carries a perfectly good description because the library row is blank, and publishing a page that renders alt="" because some library row happens to be filled in. An image pasted in by URL, with no library item at all, is checked the same way.

Publish rather than upload, deliberately: what matters is whether the published page is readable. A required field on upload blocks a bulk import, has nothing sensible to say about decorative images, and makes every item already in the library retroactively invalid.

decorative is an answer, not an omission. A divider, a texture, an image that only repeats the sentence beside it — those correctly have no alt text, which HTML spells alt="". The flag on the media item records that as a decision, so a blank block alt is accepted when the block points at an item marked decorative. Without somewhere to say it, "deliberately silent" is indistinguishable from "nobody got round to it".

Editing an already published document does not re-run this check — the gate is on the publish actions. Tightening that is tracked separately.

"Used by"

The media detail drawer lists the documents that reference an item, and the delete confirmation says how many there are, so an editor can see what a delete or a replace affects before doing it.

This reads the same Firing.ReferenceEdge graph the re-fire wave already maintains, extended to record content -> media edges — an exact answer from one indexed lookup rather than a scan of every document's block tree. Media is a reference target only: a media item is not a document, has no artifacts and never fires, so it is never a from_type.

Three reference sites are tracked: image-shaped block fields (any field named …media_id or …image_id), a document's featured_image_id, and :media custom fields (every image of a gallery field). Edges are rebuilt from scratch on every fire, so removing a reference removes the usage.

Published references only. Edges are written when a document fires, which happens on publish — so an image used only by never-published drafts reports as unused. Deletes are soft (AshArchival), so a restore covers the mistake either way.

The drawer lists at most 25 referrers plus a total, because a site logo can be referenced by every document on the site and each one costs its own fetch.

Documents (#481, #808)

The media library also accepts documents — PDF in v1, joined by office formats and zip archives in #808 — uploaded through the same /media picker (.pdf/.docx/.xlsx/.pptx/.doc/.xls/.ppt/.zip alongside the image extensions), byte-validated by KilnCMS.DocumentProcessor the same way images are by ImageProcessor (a magic-byte check, never the client's claimed filename/MIME — deny-by-default). A document has no width/height (that's the library's own image/document discriminator: content_type LIKE 'image/%', with a NULL content_type defaulting to "image" for backward compatibility with rows written before #481) and no responsive variants.

Office documents and zip archives (#808)

.docx/.xlsx/.pptx are recognized as OOXML: a zip signature (PK\x03\x04) plus an internal [Content_Types].xml entry and a format-specific main part (word/document.xml, xl/workbook.xml, ppt/presentation.xml) — that combination is what distinguishes an Office document from an arbitrary zip, which is accepted too (as plain .zip) when the zip signature matches but [Content_Types].xml doesn't. Legacy .doc/.xls/.ppt are recognized by the OLE2 compound-file signature plus the application's own root stream name (WordDocument, Workbook, PowerPoint Document).

Because a zip's own central directory can declare whatever compressed/uncompressed sizes it likes, DocumentProcessor reads that declared metadata via :zip.list_dir/1 — which touches only the central directory, never any entry's compressed data — and refuses the upload as a decompression bomb before anything is stored: over 500 MB of declared uncompressed content, a declared compression ratio past 100:1, or more than 10,000 entries. Nothing in the validation path ever inflates archive content.

Office/zip uploads are not metadata-stripped. #807's qpdf-based strip only understands PDF; it cannot open a zip or an OLE2 file at all, so KilnCMS.Media.Ingest stores these formats as uploaded, the same posture A/V had before #820. A .docx still carries its author/revision history and a legacy .doc/.xls/.ppt still carries the authoring machine's path — a real gap, tracked separately rather than assumed away.

PDF metadata stripping (#807)

An uploaded PDF is rewritten without its /Info dictionary (title, author, creator, producer), its XMP metadata stream, and every per-object /Metadata packet and /PieceInfo dictionary before the blob is stored — the document-library counterpart to the EXIF stripping images get. Outlines, form fields, attachments and page content are untouched.

The per-object half needs saying, because it is the half that is easy to miss (#918). --remove-info --remove-metadata clears the two document-level fields and nothing else — a page-level XMP packet and a /PieceInfo private blob both survive them, and /PieceInfo is where Illustrator, InDesign and Acrobat park the author name and the authoring machine's filesystem path (C:\Users\jane\...). qpdf has no flag for it, so the strip runs in two passes: dump the object dictionaries with --json-output, prune those keys, and apply the result with --update-from-json.

qpdf is required, and a PDF that cannot be stripped is refused. This is a deliberate departure from how ffmpeg is treated (see below): a missing ffmpeg costs you enrichment — no duration, no poster — while a missing stripper costs you a privacy guarantee, and a control that silently does not apply is worse than no control, because the operator believes it did. The editor sees "PDF metadata stripping isn't available on this server" rather than a misleading "unsupported format".

A/V metadata stripping (#820)

An MP4 or M4A is remuxed through ffmpeg -map_metadata -1 -map_chapters -1 -c copy before it is stored: a stream copy, so the bitstreams pass through untouched and only the container's metadata atoms are dropped. Cheap enough to run on every upload.

What that removes, on the file type where the recording is most likely to be personal:

• ©xyz GPS coordinates, which iOS writes on every phone recording
• com.apple.quicktime.model / .software — device and OS version
• creation-date atoms, often in local wall-clock
• the original filename, in some encoders' ©nam
• editing-application metadata from the export

Unlike the PDF strip above, this is best-effort by default, and that is a weaker guarantee. The argument in the previous section — that a control which silently does not apply is worse than no control — applies here too, and the only reason the default differs is that ffmpeg is optional today: requiring it would stop A/V upload working on every deployment that does not have it, on upgrade, with no warning. That is a migration, not a default.

So the behaviour is stated exactly rather than implied:

Note the middle rows: having ffmpeg is not the same as the strip succeeding. A container ffmpeg cannot remux under -c copy (pcm_s16le audio in MP4, from some screen recorders) fails the same way a missing binary does, and the default stores it. The log line names which of the two happened, because telling an operator to install ffmpeg on a host that already has it sends them after the one thing that is not broken.

And note the row that ignores the setting entirely (#1100). The strip writes a second full copy of the upload to the temp filesystem while the original is still there, so peak usage is roughly twice the file — a gigabyte for one 500 MB video, three gigabytes for three of them at once. When that runs out, ffmpeg fails with No space left on device, and under the default every row above would have stored the file unstripped: the privacy guarantee lapsing precisely under disk pressure, which is when nobody is looking at it.

So a full temp disk is refused whatever the setting says. Every other failure in the table is a standing property of the host or the file — ffmpeg is absent, or this container will never remux — where storing the upload is the better of two bad answers, because the alternative is that it can never be uploaded at all. A full disk is neither: it is transient, retrying works, and the editor is told exactly that. Free space up, or point TMPDIR at a larger filesystem.

The check runs before ffmpeg starts, but it cannot be the whole answer — concurrent uploads can each see enough room and then exhaust the disk together — so an ENOSPC that happens anyway is recognised in ffmpeg's output and refused identically. If free space cannot be measured at all (no df, or output we cannot parse), the strip proceeds rather than refusing: refusing on an unknown would be an outage in exchange for a guess, and the ENOSPC path still catches the real thing one step later.

If you rely on the privacy guarantee, set REQUIRE_AV_METADATA_STRIP=true and install ffmpeg. That gets you the same contract PDFs already have. The :warning exists so the gap is visible in logs rather than assumed away — but a log line is not a control, and nobody should treat the default as one.

The deferred strip, behind a quarantine (#1122)

The table above is the synchronous path: the remux runs on the upload request, inside consume_uploaded_entries, so a 500 MB video holds the editor's media page for a full read+write (bounded at two minutes, #1112) and peaks at ~2× the file on TMPDIR. Set KILN_AV_STRIP_MODE=deferred and none of that happens on the request:

1. Ingest stores the upload as it arrived, in private storage under its final key, and creates the MediaItem with quarantined: true.
2. While quarantined, the row is unreachable from every non-editor read surface — that is written into the read policy (SiteEmbed-style: a second condition next to audience == :public, and inside Checks.MediaInAudience for gated audiences), not a UI filter, so JSON:API and GraphQL reads are covered too; /media/:id/download and /stream answer 404 to everyone, editors included, since both routes read private storage for a gated item; and the row's url points at a public key that has nothing behind it. Editors see the item in the library so they know it exists.
3. KilnCMS.Media.AVStripWorker (queue :media) copies the private blob to a temp file, runs the same strip_metadata/2, re-checks the size cap, stores the stripped copy at the public key, releases the quarantine (a system-only action), deletes the private blob, and only then enqueues derivation (AVWorker — probe, poster), which fetches from public storage and must not run before promotion.
4. Failure outcomes are the table's, one step later: a transient one (insufficient_space, timeout) is retried by Oban; a standing one (no ffmpeg, un-remuxable container) is refused under REQUIRE_AV_METADATA_STRIP=true — row purged, private blob deleted, error logged naming the item — and otherwise promoted as it arrived with the same :warning.
5. KilnCMS.Media.QuarantineReaper (hourly, KILN_MEDIA_QUARANTINE_REAPER_CRON) removes any item still quarantined after KILN_MEDIA_QUARANTINE_MAX_AGE_HOURS (24): a strip job that was never queued or never succeeded must not leave a private blob and a permanently held row.

The fallback is deliberate for the reason the PDF strip refuses rather than degrades: a control that silently does not apply is worse than no control, and "stored public and unstripped for a while" would be exactly that. Deferred needs somewhere to hold the unstripped original that no delivery route serves from, and that is what private storage is — a private blob has no url/1 at all. Not viable, and recorded so it is not re-proposed: streaming the strip (ffmpeg on stdin, output to the adapter) — -movflags +faststart requires a seekable output, and -i pipe:0 cannot read a non-faststart MP4 whose moov is at the tail, which is what a phone export often is.

A password-protected PDF is refused, with is password-protected, so its metadata can't be removed — upload an unlocked copy. qpdf cannot open a user-password-encrypted file, so it cannot strip one, and storing it unstripped would defeat the guarantee above. This is a real narrowing: such a file was storable before #807 and opens fine in every reader, so a signed contract or a bank statement has to be unlocked before it can be added to the library. It is the one refusal here the uploader can act on, which is why it gets its own message rather than the generic "couldn't have its metadata removed".

Owner-password-only PDFs are not affected — the "restrict printing/editing" export that opens with no password anywhere. Encryption is classified from the strip failure, not probed up front, because qpdf --is-encrypted exits 0 for those too: probing would have refused a large class of ordinary documents and told the uploader to remove a password that does not exist.

Three operational notes:

• qpdf ≥ 11.10 is required — --remove-info/--remove-metadata arrived in that release. KilnCMS.DocumentProcessor.available?/0 checks the capability, not just that a qpdf binary exists, because Debian bookworm's 11.3.0 would pass the latter and fail every strip. The release image runs Debian trixie (qpdf 12.2.0) for exactly this reason.
• The prune covers stream objects and nested dictionaries, not just page dictionaries. qpdf's JSON gives a stream object a different shape from a plain one, and image and Form XObjects are streams — which is where a placed asset's original XMP (photographer, GPS) and Photoshop's /PieceInfo live. /Metadata and /PieceInfo are also legal at any depth, so the prune recurses. The object dump is written to a file rather than read from stdout: qpdf merges stderr into stdout, and any file whose xref it reconstructs prints WARNING: lines first, which would otherwise break the JSON parse on exactly the files most likely to need the prune.
• Streams are copied, not decoded (--decode-level=none). A ≤25 MB PDF can hold a FlateDecode stream that inflates to tens of gigabytes, and qpdf's default generalized decode level would expand it — a decompression bomb by another name, and the reason the 30-second cap used to be reachable at all. Nothing in the strip inspects stream contents, so decoding them was only ever cost. If the cap is hit, the qpdf process is killed by pid: closing an Erlang port shuts the pipes without signalling the child, which used to leave an orphan burning CPU and then writing an unreferenced file into the temp dir minutes after the upload had already been refused.
• Not exiftool. exiftool -all= on a PDF writes an incremental update that marks the metadata deleted while leaving the original bytes in the file, recoverable by anything that reads it with a parser rather than a viewer. It looks like it worked. qpdf rewrites the document.

Existing PDFs uploaded before this are not retroactively stripped; re-upload one to strip it.

A document is placed on content with the file block (title, description, and a download link) — separate from the image block, which never renders a document.

Bounding ffmpeg

Four limits, covering different failure modes:

The last one is not redundant with -timelimit (#1100). A -c copy remux is I/O-bound: it burns almost no CPU, so the rlimit essentially never fires while a stalled disk can hold the process for hours. And nothing outside ffmpeg could stop it either — neither System.cmd/3 nor the enclosing Oban job timeout, because closing an Erlang port shuts the pipes but sends the child no signal. Only the OS pid a port hands back can be signalled, which is what KilnCMS.ExternalCommand exists to do. It is shared with the qpdf path, which needed the same thing first (#918).

Two minutes, and that number is doing two jobs: a 500 MB stream copy finishes inside 100 seconds even at 10 MB/s, so anything past it is stuck rather than slow — and because the strip runs inline on the LiveView handling the upload, it is also the longest an editor's media page can be blocked.

Gated documents

A document (not an image — v1 scopes the gate to the document library the issue asked for) can be restricted to a consumer-facing audience, the same tier published content already uses (KilnCMS.CMS.Audiences). Setting a non-:public audience — from the media library's item detail panel — relocates the underlying blob to private storage:

• Local adapter: a second directory (priv/private_uploads by default) that KilnCMSWeb.Endpoint has no Plug.Static mount for — nothing serves it over HTTP at all. Always available, no configuration needed.
• S3 adapter: a separate bucket (S3_PRIVATE_BUCKET) this app's own AWS credentials read directly — it needs no public-read config, no CDN, and no S3_PUBLIC_BASE_URL equivalent. Not configuring it refuses gating outright rather than silently leaving the blob in the public bucket: the public bucket is public at the bucket level (see "Production storage and CDN" below), so an object sitting there is reachable at its plain URL regardless of what the app ever links to — gating can't be faked on top of that.

Every download — gated or not — goes through KilnCMSWeb.MediaDownloadController (GET /media/:id/download), never a direct storage URL: it's the one place that checks the reader's audience (CMS.get_media_item/2's ordinary policy-checked read — a gated document simply isn't in an unauthorized reader's result set, so a denied/missing item both 404, never 403, and a gated document's existence isn't confirmed to a reader without its audience), serves the original filename rather than the UUID storage key, and bumps the aggregate, privacy-first download_count (no per-viewer identity — consistent with every other counter in this codebase).

An image can never be gated (its responsive-variant pipeline — Media.VariantWorker — assumes public storage throughout). Documents and A/V both can; gating an existing item requires private storage to be configured first, same as a fresh upload.

Public media URLs are capability URLs

Warning — anyone holding a public media URL can fetch the file. Keep sensitive files out of public storage.

A media item with the :public audience (every image, and every document or A/V file nobody has gated) lives in public storage, and its bytes answer at a plain URL with no authentication, no audience check, no tenant check and no rate limit:

• Local adapter — /uploads/<key>, served by Plug.Static straight from priv/uploads, on any host the app answers.
• S3 adapter — S3_PUBLIC_BASE_URL + <key>, served by the bucket or the CDN in front of it; the request never reaches Kiln.

The same holds for every derived variant and poster frame. What the app does or does not link to changes nothing: an item attached to an unpublished draft, or to no content at all, is as reachable as one on the home page.

What stands between such a file and the world is the key: a random UUID (Storage.generate_key/1), which cannot be guessed or enumerated, so the URL works as a capability — whoever has it can read the file, and nobody else can find it. That also means the URL is the secret. It escapes wherever a URL does: a shared link, a forwarded email, a page's HTML once the item is used on published content, browser history, proxy or CDN logs. It cannot be revoked short of deleting the item, and the immutable cache headers above let a CDN and browsers keep serving it for up to a year after that.

For anything that must not be public:

• Gate it. Give the document or A/V item a non-:public audience (see Gated documents). That moves the blob to private storage — priv/private_uploads, which has no static mount, or the S3_PRIVATE_BUCKET bucket, which needs no public read — and from then on it is reachable only through GET /media/:id/download and the A/V stream route, both of which run the ordinary policy-checked read on every request. On S3, configure S3_PRIVATE_BUCKET first; without it gating is refused. Gate before the URL is shared: gating moves the bytes, but it cannot recall a copy someone already downloaded or a CDN already cached.
• Don't upload sensitive images. Images can't be gated (their variant pipeline assumes public storage), so an image is public for as long as it exists.
• Don't let the public bucket be listed. On S3 the bucket's public policy should grant anonymous GetObject only, never ListBucket — a listable bucket hands out every key, and with them every capability. Plug.Static never lists a directory.

This is an accepted residual, not a gap waiting for a fix: public media is meant to be served by a CDN without a round trip through the app. See the Media (`/uploads/`) entry under Per-surface risks & mitigations* in threat-model.md (security audit 2026-09-27, finding 15, #1666).

Video and audio (#494)

The library accepts self-hosted video, audio and WebVTT caption tracks, uploaded through the same /media picker and byte-validated by KilnCMS.AVProcessor on the same deny-by-default, magic-bytes-only terms as images and PDFs.

Upload web-ready files — there is no transcoding

Kiln stores exactly what you give it. A QuickTime .mov, a Matroska .mkv and an HEIF-branded MP4 are all rejected at upload, even renamed to .mp4, because the browser could not play them and nothing here will convert them. Export H.264/AAC in an MP4 (or VP9/Opus in a WebM) first. Adaptive streaming (HLS/DASH), in-app transcoding and DRM are deliberately out of scope — that is a video platform, not a CMS; the Kiln.Plugin seam is where an external transcoder (Mux-style) would attach.

Duration and poster frames need ffmpeg — and work without it

KilnCMS.Media.AVWorker runs after upload (off the request, like VariantWorker), re-fetches the original from storage, and shells out to ffprobe/ffmpeg for the playback duration, the video's intrinsic dimensions, and a poster frame taken one second in.

ffmpeg is an optional system dependency, not a Mix dep. Install it in your image (apk add ffmpeg / apt-get install -y ffmpeg) to get those three things. Without it every step no-ops: the upload still stores and still plays, it simply has no duration and no generated poster, and an editor picks a poster image by hand — which the video block supports either way.

Note that ffprobe writes width/height for a video exactly as libvips does for an image, so width alone does not mean "this is an image" anywhere in this codebase. KilnCMS.MediaKind.of/1 is the one place that decides an item's kind, from its content_type.

Blocks

A/V is placed on content with the video and audio blocks — distinct from embed, which points at YouTube/Vimeo. The video block carries a poster image reference, a WebVTT caption track reference, and autoplay / loop flags (autoplay always renders muted, since no browser will autoplay sound).

Neither block ever stores a storage URL. Their src is always /media/<media_id>/stream, for the same reason the file block's href is always /media/<id>/download: a gated item has no public URL to store, and a public one can be gated later, which would leave a baked URL pointing at a blob that has since moved. A pasted url field remains for media hosted somewhere else entirely, and is used only when no library item is set.

Streaming and Range

GET /media/:id/stream (MediaDownloadController.stream/2) serves playback bytes under the same authorization as the download route, and differs from it on three points:

• Inline, not attachment — a <video> cannot play a response the browser is told to save. Only content types on KilnCMS.MediaKind.inline_streamable?/1's exact allowlist are served that way; anything else falls back to the attachment path, so an editor-supplied content_type (it is writable through the API) can never make this route an inline host for arbitrary content on the app's origin.
• Range requests — seeking is a Range: request, and a player shows no scrub bar without Accept-Ranges. Answered by Storage.fetch_range/3, which reads only the requested slice (:file.pread locally, a ranged GET on S3). Every response is capped at 8 MB, so a seek never pulls a whole film into memory — and the cap holds on the un-ranged path too, which streams the body in 8 MB chunks rather than buffering it. That matters because omitting Range (or sending an unparseable one) would otherwise be a one-header way to ask for a 500 MB allocation.
• No download counter — one scrub is dozens of ranged requests; download_count stays a download-only measure.

Gating A/V

A video or audio item gates exactly like a document. One extra step applies: gating discards the generated poster frame, row and blob. A poster is written to public storage (it renders as a plain <img>), and a still from a members-only video should not stay world-readable once the video isn't. Un-gating re-derives it (#821), so a re-published video does not open on a black frame. A poster picked by hand on the block still wins — it is a different field, and Blocks.Video.poster_src/1 prefers it.

If you serve media from a CDN on another hostname, note that CSP_IMG_SRC widens the browser CSP's media-src as well as its img-src; without it, default-src 'self' blocks a cross-host <video>.

Large objects

Nothing in the pipeline holds a whole video in memory. Uploads above 16 MB go to S3 as a streamed multipart upload (smaller ones keep the single PUT, so a 4 KB thumbnail doesn't pay for three round trips); Storage.copy_to_file/3 streams a blob to a temp file for Media.AVWorker and for the gating relocation; and the stream route chunks its responses as described above. The one deliberate exception is GET /media/:id/download, which is a whole-file response by definition — keep the document caps in mind if you raise them.

Uploading over the API

POST /api/media, POST /api/media/import-url and the direct-upload pair (documented in api.md → Uploading media) are the same pipeline as the library's upload form — KilnCMS.Media.Ingest, via KilnCMS.Media.Upload — so everything on this page applies to a file that arrives that way: the byte sniff, the per-kind caps, every metadata strip and its refusals, the #1122 quarantine and the derivation jobs. The API adds only what a remote caller needs: authorization before a large body is read, the editor-settable metadata (alt, caption, decorative, focal point, tags) in the same request, and a media_upload rate-limit bucket.

Direct uploads

The presigned flow (KilnCMS.Media.DirectUpload) lets a client PUT a large file straight to the bucket instead of through the app. It needs the S3 adapter and S3_PRIVATE_BUCKET: the client's bytes land in the private bucket under direct-uploads/<uuid> — never the public one, because nothing has sniffed or stripped them yet — and completion copies them down (8 MB at a time), ingests them like any upload, and deletes the staged object. The declared size is signed into the URL as content-length, so the store itself refuses a different body.

Two pieces of bucket configuration, on the private bucket:

• CORS, if browsers upload directly — allow PUT from your front end's origin with the content-length header. Server-side clients need none.

  [{ "AllowedOrigins": ["https://app.example.com"], "AllowedMethods": ["PUT"],
     "AllowedHeaders": ["content-length", "content-type"], "MaxAgeSeconds": 3000 }]

• A lifecycle rule expiring direct-uploads/ objects after a day, as a backstop. Kiln already queues a delete for every issued upload once its token expires (KilnCMS.Media.StagedUploadCleanup); the rule covers a job lost with its node.

Presigned PUT rather than a presigned POST form, so the flow works on R2 and B2, which do not implement S3's POST Object.

Production storage and CDN

Development uses the Local adapter (priv/uploads, served by the app's own Plug.Static mount). Production should serve media from object storage with a CDN in front, so image bytes never touch the app.

Switch adapters by setting S3_BUCKET; S3_PUBLIC_BASE_URL then becomes required (the app refuses to boot without it) and is the CDN hostname — every media URL the CMS emits, original and variant alike, is that base plus the storage key. Non-AWS stores also need S3_ENDPOINT_HOST. Full table: environment-variables.md.

Caching contract

Uploads carry Cache-Control: public, max-age=31536000, immutable, written as object metadata at PUT time by KilnCMS.Storage.S3, so the CDN and the browser both see it on every GET.

Caching that hard is safe because storage keys are write-once UUIDs. A blob is never overwritten in place: an in-admin edit writes a new key and repoints the item (above), and regenerated variants get fresh keys too. A URL therefore always denotes the same bytes — there is no invalidation step to run after an edit, and no CDN purge to wire up.

Putting a CDN in front

The pattern is the same everywhere: make the bucket readable, attach a CDN to it on a hostname you control, and set that hostname as S3_PUBLIC_BASE_URL (including the bucket path, if the provider's URLs carry one).

Public read is configured at the bucket level, not per object — that is what R2, B2, Wasabi and modern AWS all expect. Only set S3_ACL=public_read for a bucket that still relies on per-object canned ACLs.

Leave the CDN's own cache TTL on "respect origin headers"; the max-age above is already the intended lifetime. If you also enable the CDN's image resizing, note that Kiln has already derived the variants listed at the top — the two will duplicate work.

Finally: if the CDN hostname differs from the site's own origin, add it to CSP_IMG_SRC (space-separated, read in config/runtime.exs) or the browser's img-src policy will block every image.

Security headers

Local-adapter responses carry two defense-in-depth headers, set by secure_upload_headers/2 in KilnCMSWeb.Endpoint on every /uploads/* request. S3-served media bypasses Phoenix entirely, so it has to get them elsewhere — and only one of the two can ride along on the object:

Neither header affects rendering: disposition is ignored for <img> and other subresource loads, and every media URL Kiln emits is a subresource. They matter only when someone navigates directly at a media URL, where together they stop a stored file from being interpreted as active content rather than served as a download.

nosniff is not settable as S3 object metadata. S3 persists a fixed set of system headers (Content-Type, Content-Disposition, Cache-Control, Content-Encoding, Content-Language, Expires); anything else you attach is returned prefixed as x-amz-meta-*, which no browser acts on. Add it at the edge instead:

Two S3-specific caveats on the disposition header, both absent on the Local adapter, where it is a per-request plug:

• It is written once, at upload time. Changing it later does not follow from a deploy — you have to rewrite the existing objects (aws s3 cp s3://bucket s3://bucket --recursive --metadata-directive REPLACE).
• url/1 emits a plain public URL, not a presigned one, so there is no per-request response-content-disposition override to reach for.

The practical consequence: pasting a media URL straight into the address bar downloads the file instead of displaying it — on both adapters. That is the intended behavior, not a misconfiguration. To eyeball a stored image, view it in the media library at /media, which renders it as an <img>.

Per-site buckets (#1559)

Everything above is the operator's storage, set once for the deployment. A site admin can instead keep their site's files in their own S3-compatible bucket, at Configure → Integrations → Object storage (/editor/site-storage): an endpoint (blank for AWS), region, bucket, an optional private bucket for gated documents and direct uploads, the public URL the bucket is served from, and a key pair. The secret is stored encrypted and never shown again. Test the saved bucket writes, reads back and deletes a small file, so a wrong bucket name or a read-only key shows up before the first upload does.

Only new uploads move. Each media item records the store its file went to (storage_profile_id; empty means the operator's storage, which is every item uploaded before the site had its own). Everything that happens to an item later — its variants, poster frame, on-the-fly transforms, gating and un-gating, downloads, streaming, deletion — goes to that store, whatever the site has set since. So switching the setting off, or moving to another bucket, strands nothing, and needs no migration. It also moves nothing: files stay where they were stored. Keep an old bucket and its key working until you have copied its files yourself; there is no background copy job yet.

An unusable bucket refuses uploads. If the site's settings can't be read, its secret can't be decrypted (SECRET_KEY_BASE was rotated — see secrets-rotation.md) or its endpoint now resolves to a private address, the upload fails with a message saying so. It is never written to the operator's bucket instead.

The site's bucket is set up like the operator's — public read at the bucket level, a CDN if you want one (see Putting a CDN in front), a lifecycle rule on the private bucket's direct-uploads/ prefix. The site's public URL origin is added to that site's img-src and media-src automatically, so it needs no CSP_IMG_SRC entry.

What a site admin cannot do: point the endpoint at a private, loopback, link-local or metadata address (checked at save and on every connection, and the connection goes to the address that was checked), use plain http://, or have the operator's credentials used for their bucket — a request to a site's bucket is built from the site's settings alone.

Image backends (#1955)

An image can live in an external image service — Cloudflare Images, imgix, Bunny Optimizer — instead of in the bucket above. The service comes from a plugin, which contributes a Kiln.ImageBackend from its image_backends/0; the operator picks it in config/runtime.exs:

config :kiln_cms, :image_backend, MyPlugin.ImageBackend

Installing the plugin changes nothing until then, and a selected backend that is missing its own settings (configured?/0 false) is never offered an upload. mix kiln.plugins.doctor fails on a selected backend no installed plugin declares.

System → Image backend (/editor/system/image-backend, platform admins; linked from System → Plugins) shows the state in one place: which backend is selected and whether it is receiving uploads, whether its service answers, and how much of the current site's library it holds. The connection check is the backend's optional check/0, run when the page opens and on Check again. A backend without one says it can't check. A revoked token shows up here rather than as an editor's failed upload.

What goes to it. A new image upload bound for the operator's store, after its metadata is stripped, when the backend accepts?/1 it. Not documents, video or audio; not an upload to a site's own bucket (above) — the backend is the operator's choice, the bucket is the site's; and not an image whose metadata strip failed. That one is kept in the operator's store, as every image upload already is, rather than handed to a third party unstripped. Images can never be gated, so a backend needs no private storage.

Every later operation follows the row. The item records the backend's name (image_backend), and everything that happens to it — downloads, rotation, variants, transforms, deletion — goes to that backend, whichever is selected later. An item whose backend has been uninstalled fails those operations rather than reaching for the operator's store, which doesn't have the file; reinstall the plugin to reach it again.

Variants and transforms come from the backend. The variant worker records the backend's URL for each variant the encoder would have written — the same labels, sizes and content types — so the variants map and every srcset built from it read as they always have. /media/:id/t/<ops> checks the signature, the allowlist, the read policy and the pixel cap as usual, then redirects (302) to the backend's rendering with the same Cache-Control and ETag the image would have carried. A backend that can't render a given output answers :unsupported, and Kiln encodes it as before, storing the result in the backend.

CSP. The selected backend's delivery origin is added to img-src automatically. If you move to another backend while the old one still holds images, keep the old origin in CSP_IMG_SRC, or its images render blank.

Adopting images a backend already holds. A library imported from another CMS can arrive as rows carrying only a URL into the backend — no file in Kiln, so no variants, rotation or transforms, and deleting the row leaves the image behind. Once the backend is selected and configured,

mix kiln.media.adopt_image_backend --dry-run   # count, write nothing
mix kiln.media.adopt_image_backend             # --org <uuid> for another site

hands it every row with no stored file whose URL it recognises as its own (key_for_url/1), and enqueues each one's variants, which read its dimensions back from the backend. A row Kiln holds a file for is never touched. Content already published keeps the URL its snapshot recorded, which the backend still serves.

System → Image backend counts the same rows before you run it: those the backend holds, the imported links it recognises but doesn't hold yet (what the task would adopt), and the imported links it doesn't recognise. When there is something to adopt it prints the command with the site's --org filled in, and the bin/kiln_cms rpc form for a release, which has no mix.