The Artifact: Why Screen Photos Produce Moire
Photo moire appears when a camera photographs a display. The screen has its own pixel and subpixel structure. The camera sensor has its own sampling grid. The image then passes through demosaicing, denoising, sharpening, compression, and other ISP stages.
When these systems interact, structured artifacts can appear: colored bands, repetitive ripples, false textures, and local patterns that do not belong to the original displayed content.
This is not just ordinary noise. It is an imaging artifact caused by the interaction between display structure, camera sampling, optics, and image processing.
That makes photo moire removal a good small project for an AI camera restoration portfolio.
Why This Belongs to AI Camera Restoration
AI camera restoration is about improving images produced by real camera pipelines. The important word is “real.” Real images contain artifacts that come from hardware, optics, sensors, ISP decisions, and usage conditions.
Photo demoireing connects several of my interests:
- smartphone photography;
- computational imaging;
- ISP-aware image restoration;
- display-camera sampling artifacts;
- practical before/after visual demos;
- deployable restoration systems.
It is also small enough to finish. That matters. This project should not become a new paper direction or a full AI camera platform. It should become a polished, reproducible demo.
Baseline First: UHDM / ESDNet
The first rule of this project is simple:
Do not start by inventing a new model.
The current baseline is UHDM/ESDNet. I selected it first because it targets practical high-resolution screen photographs, provides pretrained weights, and is suitable for building a reproducible inference pipeline.
The current foundation includes:
- a separate WSL code repository;
- a dedicated code repository for the demo pipeline;
- a modern CUDA-enabled PyTorch environment;
- imported and modernized UHDM/ESDNet inference code;
- official baseline checkpoint loading;
- strict state-dict verification;
- first GPU inference on a sample image;
- a measured first forward pass around the sub-second range on the current GPU setup.
This is enough to justify building a result gallery before considering training or architecture changes.
Demo Pipeline
The minimum demo should be clear:
moire-corrupted photo -> ESDNet inference -> restored image -> before/after comparisonThe portfolio version should include:
- upload or select an input image;
- run the demoireing model;
- show input and output side by side;
- show synchronized crop or zoom comparisons;
- report inference time;
- include several sample images;
- mark failure cases honestly.
The goal is not to claim that the baseline solves every case. The goal is to build a trustworthy demo that shows what works, what fails, and what should be improved next.
What Makes a Result Convincing
A good demoireing result is not just a smoother image.
It should remove visible moire patterns while preserving text edges, UI lines, local contrast, and meaningful texture. It should avoid over-smoothing, color shifts, and new artifacts near high-frequency regions.
For the first evaluation set, I want a small sample manifest rather than a huge unorganized folder. The sample set should cover:
- text and UI screenshots;
- photographs displayed on screens;
- flat colors;
- dark backgrounds;
- fine grids;
- mixed content;
- several moire scales and shooting conditions.
Every sample should have known provenance and a clear public/private status. A portfolio demo should not depend on unclear sample licensing.
Failure Cases I Want To Keep
Failure cases are useful. They keep the project honest and make later improvements more meaningful.
The first gallery should label results such as:
good;partial;failed;over-smoothed.
I want to record residual color bands, text damage, texture loss, color shifts, and boundary artifacts. If training or fine-tuning becomes necessary later, it should be motivated by repeated observed failures, not by the vague desire to “improve the model.”
Next Steps
The next milestone is an evidence-backed baseline gallery.
The immediate tasks are:
- define a sample manifest with image metadata and license status;
- collect a small real-photo set;
- add reproducible batch inference;
- measure runtime correctly with warm-up and repeated runs;
- select strong examples and honest failure cases;
- generate standardized before/after and crop comparison assets;
- build a lightweight local demo or static result gallery;
- write a GitHub-facing README.
Only after that should I decide whether training, synthetic moire generation, ONNX export, or a second baseline is worth the time.
For now, the best version of this project is small, visual, reproducible, and easy to understand.