amethyst/commons/plans/2026-10-17-hand-drawn-posts.md

Hand-drawn posts

Draw by hand in the composer and post the drawing as a looping GIF that replays the strokes.

What the user gets

• A Draw button in the short-note composer opens a full-window editor (DrawingEditorDialog) in two steps:
  • Draw: the paper fills the window. A floating dock (DrawingDock) holds a tray of illustrated pens tipped with the current ink (the one in hand springs up), mirror, undo/redo, the inks and a size button with a live stroke preview. A chip in the top bar opens the paper sheet (PaperSheet), which shows real samples of every paper and the user's saved pictures.
  • Preview: the drawing replays exactly as the posted GIF will (ReplayPreview, using DrawingReplay.replayTimeAt). The alt text and "Add to post" live here.
• Tools: brush (follows stylus pressure), pencil, spray, eraser, mirror; eight inks; pen size.
• Backgrounds: plain paper, four sticky notes (square, with a glue strip), a notebook page, grid paper, a dot grid, a chalkboard, or the user's own picture to draw on. Pictures are kept on the device (DrawingBackgroundStore) and come back in the row next time, so a user can reuse their own "stationery".
• An alt-text field in the preview step, since a picture of handwriting is unreadable to a screen reader without it. Where the platform has an on-device describer (AppServices.createAltTextSuggester: Gemini Nano on the Play build), the preview fills it with a description of the finished drawing, marked "AI-suggested" like the upload screen; the user's own words are never overwritten.
• Add to post renders the replay GIF and a PNG of the finished drawing, uploads both and puts the GIF's URL in the note, with an imeta the uploader could not have built on its own.

Why a GIF

The post has to work in every client, not only in Amethyst. GIF is the one animated format that every Nostr client and browser plays inline, with no player controls and no buffering. Amethyst decodes it on Android (GifDecoder) and on desktop (SkiaGifDecoder); animated WebP only animates on Android here, and an MP4 would show up as a video player. Drawings fit GIF's limits: ink on paper needs far fewer than 256 colours, and each replay frame changes a small patch of the picture.

What was considered and set aside:

• SVG: Amethyst renders it, but most other clients and many media servers refuse SVG.
• Stroke data inside the event (replay, remix, draw-on-reply): it needs a new kind or tag, which is a protocol change for the maintainer to approve, and other clients would still need an image. The editor keeps strokes in a form (DrawingStroke) that could be serialised later.
• Simple Nostr Objects (kind 33331) in lines mode: one polyline with no pen lifts, no width, no timing, and other clients would show it as a 3D object.

Pieces

Piece	Module	Notes
DrawingStroke, DrawingPaper, DrawingBackground	commons	Points as fractions of the canvas width, with timestamps
DrawingEditorState	commons	Snapshot state: background, pen, strokes, undo/redo
DrawingReplay	commons	Replay timing: cuts pauses (150 ms) and gaps (300 ms), fits ~3.2 s, 0.4 s lead-in, 2.5 s hold
AnimatedGifEncoder, GifPalette, GifLzw	commons	Pure Kotlin GIF89a writer. Later frames store only the changed rectangle, with unchanged pixels transparent
DrawingRenderer (drawDrawing, BackgroundPainter)	commonsUI	One renderer for the live canvas, every GIF frame and the PNG
ReplayFrameRenderer	commonsUI	Replay frames in constant time: finished strokes in a layer that only grows, the background drawn once, one half-drawn stroke on top. Used by the export and the preview
DrawingExporter	commonsUI	Renders frames offscreen (ImageBitmap + CanvasDrawScope + readPixels) at 540 px wide
DrawingBackgroundStore	commons	Picked pictures as cropped PNGs, one folder per account under the host's drawingBackgroundsDir, newest 12 kept
PhotoBackgroundLoader	commonsUI	Coil decode (upright on Android), centre crop to 4:3..2:3, 1080 px wide
encodeArgbAsPng	commonsUI	Bitmap.compress on Android, Skia on desktop/iOS
ShortNotePostViewModel.uploadDrawing	commonsUI	Uploads both files, builds the GIF's imeta

Strokes over backgrounds

Strokes are drawn on their own layer above the background (inStrokeLayer, a saveLayer), and the eraser is BlendMode.Clear in that layer. That is what lets the eraser reveal a photo rather than paint over it. The live canvas keeps finished strokes in a transparent bitmap that only repaints on undo, clear or resize, and draws the stroke under the finger on top.

Palette

GifPalette.Builder counts the colours of the finished drawing plus about 12 frames sampled across the replay, because ink that is erased or painted over never reaches the last frame. When it has at most 255 colours (5-bit buckets), the palette is exact and nothing is dithered. Over a photo it uses median cut plus a 4x4 Bayer dither. The dither depends only on a pixel's position and colour, so an unchanged pixel maps to the same index in every frame and the per-frame patches stay small.

Upload

The composer's batch upload is not used, for three reasons:

• It would re-encode the PNG as JPEG at the default quality.
• On Android it sends the GIF through EXIF stripping, which fails on GIF.
• It measures blurhash and dims from the first frame, which is blank paper.

uploadDrawing instead uploads each file UNCOMPRESSED with stripMetadata = false. It then writes the GIF's imeta with:

• the dims and the blurhash/thumbhash of the finished drawing;
• image = the PNG URL (NIP-94's preview field), for clients that show a still;
• alt.

A failed PNG upload does not block the post. NIP-95 servers get the GIF as an embedded event, as other uploads do.

Measured on desktop: a two-stroke sticky note replays in 61 frames as a 16 KB GIF (the PNG is 27 KB). A drawing over a 1080 px photo makes a 140 KB GIF (the PNG is 390 KB).

Saved backgrounds

A picture the user draws on is saved, already cropped and scaled, as <drawingBackgroundsDir>/<account pubkey>/<saved at ms>_<sha256 prefix>.png:

• Location: filesDir/drawing_backgrounds on Android, the desktop data dir on desktop.
• One folder per account, so one account's pictures never show in another account's editor on the same device.
• Name: the timestamp orders the row and the hash avoids duplicates. Only the newest 12 are kept.
• Removing: a badge on the selected thumbnail removes the picture.
• Device only: saved pictures are never uploaded unless they end up in a posted drawing.

Not done yet

• Syncing saved backgrounds across devices (e.g. on the user's Blossom server plus a list event), which needs a NIP decision.

• Reading imeta image as a still preview for GIFs in Amethyst's own feed (it already animates them).

• Stroke data in the event (see above), which needs a NIP decision.

• Saving an unfinished drawing; closing the editor asks before discarding.