I Built an Obsidian Plugin to Convert Handwritten Notes to Markdown
In my previous article (https://www.dsebastien.net/i-built-an-obsidian-plugin-to-sync-my-remarkable-notes), I showed how to sync reMarkable notebooks into Obsidian as images. That was step one of my handwriting-to-text pipeline. This is step two.
I built the Transcriber plugin for Obsidian to convert those images into structured Markdown using local AI models. Right-click any image in your vault, select "Transcribe Image", and get a .md file back with headings, lists, quotes, tables, code blocks, and even Mermaid diagrams. All extracted by a vision AI running on your own machine. No data leaves your computer.
How it works
The plugin uses Ollama to run vision models locally. Ollama is a tool that lets you download and run AI models on your own hardware. You install it once (one command on Windows, Linux, or macOS), pull a model, and the plugin handles the rest.
Here's the workflow:
- Select an image in your vault (any image; handwritten notes, diagrams, screenshots)
- Right-click and choose "Transcribe Image" from the context menu
- The plugin calls Ollama, loads the vision model, and sends the image for conversion
- A Markdown file appears alongside the original image with the transcribed content
The first transcription takes a bit longer because the model needs to be loaded into memory. Subsequent ones are fast because the model stays loaded for a few minutes before being automatically unloaded to save resources.
Batch transcription
You don't have to go one image at a time. Right-click on a folder and select "Transcribe all images in folder". The plugin processes every image and saves the corresponding Markdown files. Handy when you've just synced a whole notebook from your reMarkable.
The prompt is customizable
The plugin includes a default prompt that instructs the AI to output Obsidian-flavored Markdown. It preserves the original document structure, formats headings and lists properly, converts diagrams to Mermaid syntax, and transcribes handwritten text as accurately as possible.
If the results aren't what you want, you can tweak the prompt in the plugin settings. Different models respond differently to prompting. You can also switch between multiple installed models to compare results.
Recommended models
The plugin settings list recommended vision models that I've tested for this task. I've been using glm-ocr and getting solid results. You can install models directly from the plugin settings; no terminal needed.
Any Ollama vision model works. Install multiple ones and compare.
Privacy
I don't want to send my handwritten notes to some cloud service. You might not care about that, or you might care a lot. Either way, running everything locally means zero data leaves your machine. Ollama processes everything on your CPU/GPU.
The full pipeline
My handwriting-to-Markdown pipeline is now two steps:
- Import notebooks as images using the reMarkable Sync plugin for Obsidian
- Convert images to Markdown using the Transcriber plugin for Obsidian
Both plugins I built. Both run locally. Both open source.
The results aren't perfect. You still need to review and clean up the output. But it saves a massive amount of time compared to manual transcription.
Demo
I recorded a 10-minute video walking through the full setup and workflow: https://youtu.be/uD5FcY1fx-s
Get started
The plugin isn't in the community plugin directory yet, but you can install it manually from GitHub.
- GitHub: https://github.com/dsebastien/obsidian-transcriber
- Documentation: https://developassion.gitbook.io/obsidian-transcriber
- Ollama: https://ollama.com
Going Further
If you want a ready-made Obsidian vault with the best structure, plugins, and templates already set up, check out my Obsidian Starter Kit: https://www.store.dsebastien.net/product/obsidian-starter-kit
And if you want weekly tips on PKM, note-taking, and knowledge work, subscribe to my newsletter (free): https://dsebastien.net/newsletter
That's it for today!