Moebius 0.2B: Image Inpainting Comes to Your Browser

Discovery of Moebius on Hacker News

This morning, while browsing Hacker News, a user came across an image inpainting model named Moebius: 0.2B Lightweight Image Inpainting Framework. This model, although lightweight, promises performance comparable to much larger models, reaching the 10B level. It is designed to let users select areas of an image to remove, then generate what should fill the empty space. Initially, the model required PyTorch and NVIDIA CUDA. However, intrigued by its description as a 0.2B model, this user decided to try to get it running via WebGPU in a browser. After a few attempts, he succeeded, and you can now try the demo at simonw.github.io/moebius-web/.

Presentation of the Final Tool

A demonstration video is available to illustrate how the finished tool works. You can load any image, even if it is not square, as it will be adjusted automatically. Then, simply highlight the areas you want to remove, click the "Run inpaint" button and wait for the model to do its job.

A Side Project with Datasette

The user's main project for the day was to finalize a major feature in Datasette: a user interface for creating and modifying tables. This followed the row insertion and editing feature released the previous week. He was working on this project in Codex Desktop and often spent 5 to 10 minutes waiting for the system to finish a medium-sized refactoring or add the finishing touches to a user interface change.

Exploration with Claude Code

During these waiting periods, the user decided to launch Claude Code in a terminal window to see how far he could get in porting Moebius to the web. His first step was to ask Claude about the feasibility of this project. In Claude.ai, which can clone repositories from GitHub, he asked:

• Clone https://github.com/hustvl/Moebius/ and tell me if they have published the code and weights to run this model somewhere.
• For Moebius, what are the options to run it right now: only Python and NVIDIA CUDA, or other options as well?
• Think about the feasibility of porting it to Transformers.js or similar and running it in a browser.

The user is in the habit of asking models to "think about X", as it is a concise way of saying that he wants them to consider a problem without giving them a concrete goal. Claude suggested using ONNX Runtime Web on the WebGPU backend, a more suitable solution than the Transformers.js library initially considered.

Setting Up the Project

Convinced by this approach, the user decided to pursue the project. He began by gathering as much information as possible that the coding agent might need. Since he did not expect this project to actually work, he did everything in his /tmp folder:

• Retrieve the Python code for Moebius

  git clone https://github.com/hustvl/Moebius
  

• And the model weights (Claude figured this out):

  GIT_LFS_SKIP_SMUDGE=0 git clone https://huggingface.co/hustvl/Moebius Moebius-weights
  

• Finally, a few libraries that might be used:

  git clone https://github.com/huggingface/transformers.js
  git clone https://github.com/microsoft/onnxruntime
  

Launching Claude Code

The user created a directory for the rest of the project and ran git init so that Claude could start recording code notes:

mkdir /tmp/Moebius/moebius-web
cd /tmp/Moebius/moebius-web

• Copy the research.md from earlier

  git add research.md
  git commit -m "Initial research by Claude Opus 4.8"
  

He launched an instance of Claude in the /tmp/Moebius folder, the level above all the research materials prepared for it. He prompted it:

• Read ./moebius-web/research.md. Your goal is to port this model to ONNX and WebGPU so that we can run it directly in a browser, with a simple user interface.

As it started working, the user added this follow-up (typos included):

• Build this in /tmp/Moebius/moebius-web and commit often. Also maintain a notes.md file in there with notes on what you discover along the way, and start by writing a plan.md in there and update that plan as you work.

He often asks agents to keep notes like this: the end result is often interesting, both for him and for the next agent session that touches the same project. Here is what this notes.md file looked like at the end of the project.

Publishing on Hugging Face

How could this be published on Hugging Face so that the model weights are there and the HTML demo appears in Hugging Face Spaces? Claude Code knows how to use the hf CLI tool, so the user created a model repository on Hugging Face, then created a token that could write to this repository and placed it in a /tmp/Moebius/token.txt file for Claude to use.

Claude published the converted 1.24 GB of ONNX weights at huggingface.co/simonw/Moebius-ONNX for him. The user had already seen other demos load weights in the browser from Hugging Face, so he knew it was possible. He decided to host his own frontend code on GitHub Pages, so he said:

• I want to publish the moebius-web folder on GitHub, except for the large files (so maybe except for the models/ folder), so that when I enable GitHub Pages for this repository, navigating to https://simonw.github.io/moebius-web/ serves the user interface.

Specifying the final URL was important in case Claude needed to correct the URLs in the demos it was building so that they would work when deployed to production.

Loading Optimization

After a few more iterations, in between work on his main project, the user got a deployed and working version! Except that... every time he reloaded the page, it seemed to download about 1.3 GB of model weights. The browser cache seemed very important for this!

• Is there something smart we can do with service workers or similar to help cache these items? It seems to reload every time, and he is worried there might be something odd about how the HF redirects work that means we are not benefiting from the browser cache.

The user knew that Transformers.js projects could handle this properly, so he took a copy of the Whisper Web demo, placed it in /tmp/Moebius/whisper-web and said:

• Look in /tmp/Moebius/whisper-web (with a sub-agent) and see how they do this.

This project was entirely obfuscated, built from JavaScript files, so he thought that using a sub-agent would avoid spending the rest of his top-level token context deciphering those files.

Claude figured out that it used caches.open("transformers-cache") (the CacheStorage API) and added that to the project.

Final Thoughts

This definitely counts as vibe coding: the user did not look at a single line of code in the project, limiting his input to testing, suggesting small feature improvements (like a progress bar for large file downloads) and steering the model toward examples of how he wanted things to work.

Since he did not write any code, the amount he learned about the underlying technologies (WebGPU, ONNX and the Moebius model itself) was very limited. As is often the case with this kind of project, the most important things he learned concerned what was possible:

• Claude Opus 4.8 is capable of converting a PyTorch model to ONNX, publishing the result on Hugging Face and then building a web application and interface that can load and run this model.

• Chrome, Firefox and Safari are all now capable of running this kind of model, and he tried it in all three.

• The CacheStorage API works with model files of about 1.3 GB.

... which means inpainting can be a feature of a client-only web application! (If users can tolerate downloading 1.3 GB.)

The user felt he should probably try to learn a bit more about his project. He launched Claude.ai and prompted it:

• Clone https://github.com/simonw/moebius-web/ and use it to teach me everything about the model and ONNX and the process of converting a model to ONNX and WebGPU and basically everything I would need to know to fully understand this repository.

Here is the transcript and the understanding.md file in Markdown that it created, which he has now added to the GitHub repository. He found the explanation of ONNX particularly enlightening:

ONNX (Open Neural Network Exchange) is a portable, framework-neutral file format for neural networks. An .onnx file is essentially two things bundled together:

• A computation graph: a directed graph of nodes, where each node is an operator (Conv, MatMul, Add, Einsum, Softmax, Gather, Resize, …) connected by named tensors flowing between them. This is the "recipe" for the forward pass.

• The weights: the learned parameter tensors (the convolution kernels, the embedding table, etc.), stored as initializers in that same graph.

Crucially, ONNX describes what to compute, abstractly, without saying how or on what hardware. The set of operators is versioned by an opset number (this repository uses opset 18), which specifies exactly which operators exist and what their semantics are.

It turns out that PyTorch has built-in mechanisms for exporting to ONNX, as seen here in export_onnx.py:

torch.onnx.export(
    dec, (lat,), dec_path, opset_version=args.opset,
    input_names=["latent"], output_names=["image"],
    dynamic_axes={"latent": {0: "B"}, "image": {0: "B"}},
)

Claude also included a handy glossary and a barely broken ASCII diagram showing how the model pipeline fits together.