riddle
nicheA creative hardware demo integrating vision LLM with e-ink for a magical interactive diary, but niche and device-specific.
What it is
An interactive diary app for the reMarkable Paper Pro that uses a vision LLM to read handwritten notes and generate handwritten replies, all on the e-ink display.
How it differs from vanilla Claude
A vanilla agent cannot replicate this because it requires direct hardware access to the e-ink display, real-time pen input with pressure sensitivity, and the ability to render stroke-by-stroke handwriting. The project also includes a low-level display driver (quill) that bypasses the vendor UI for instant ink latency.
Skill, plugin, or workflow shift?
The project consists of a Rust app (riddle) for pen input, handwriting synthesis, and LLM communication; a C/C++ display driver (quill) that interposes the vendor waveform engine; and integration with the reMarkable's custom launcher (AppLoad). It also supports two oracle backends (OpenAI-compatible API or local pi RPC).
Devil's advocate โ is this just complexity?
This is a clever demo, but it's a solution in search of a problem. The reMarkable is designed for distraction-free note-taking; adding an LLM that writes back is a gimmick. The setup is complex, requires a paid API key, and only works on one specific device and OS version. Most AI engineers will find it fun for five minutes but not useful. The real value is as a proof-of-concept for hardware-LLM integration, but that's not why most people visit AIx.
The honest case for it
The project demonstrates a deep integration of LLMs with a niche hardware platform, achieving low-latency interaction (0.9-1.1s to first ink) through careful engineering: Rust for performance, direct evdev input, and a custom display driver. The handwriting synthesis (rasterize โ skeletonize โ trace) is computationally lightweight. It's a well-architected reference implementation for anyone building AI-powered physical interfaces, and the code is open-source under MIT.
Who it's for
Audience fit
Depth and leverage for a technical engineer who wants to understand it and level up their workflow โ not just offload work.
Value for someone who wants a more capable tool without the technical depth โ accessible, does-it-for-you.
The project is a well-engineered hardware demo that showcases an interesting use of vision LLMs, but it's tied to a niche device and doesn't provide reusable AI infrastructure.