SIGGRAPH Asia 2026
Video refocusing enables the focal plane of a shot to be adjusted after capture, even when the original video was recorded with arbitrary and time-varying focus. Existing video restoration methods assume static blur kernels and become unstable under dynamic focus, while video bokeh methods require restrictive inputs. We present Refocusing Any Video (RAV), a video refocusing framework that renders temporally smooth bokeh at user-specified dynamic focal planes. RAV introduces temporal sharpness and face-aware correspondence constraints for stable video refocusing under severe defocus. Experiments show stronger temporal consistency, identity preservation, and overall refocus quality than image refocusing baselines.
This example shows the original defocused video, the input and target focal-plane maps, and the final refocused video.
Compare RAV with DiffCamera, GenFocus, and Learn2Refocus across six scenes. Each sample includes the input video, input and target focus maps, and ground truth. Focus maps use grayscale, with white indicating the focal plane.
The zoom-inset videos show local detail and temporal behavior under the same target focus setting.
We evaluate on 100 held-out videos, each 60 frames long: 50 from UltraVideo-AIF and 50 from CelebV-HQ-AIF. Each dataset includes 25 constant-focus and 25 time-varying-focus clips. Image refocusing baselines are applied frame by frame.
↓ Lower is better. ↑ Higher is better. In method comparisons, red bold marks the best result and blue underlined marks the second best. IDS measures identity similarity on the CelebV-HQ-AIF face subset only.
Scroll tables horizontally to view all columns.
| Method | LPIPS ↓ | FVD ↓ | tOF ↓ | tLP ↓ | IDS ↑ |
|---|---|---|---|---|---|
| DiffCamera | 0.2457 | 413.47 | 3.0925 | 0.0543 | 0.5523 |
| Learn2Refocus | 0.2464 | 478.66 | 3.2605 | 0.0434 | 0.5206 |
| Genfocus | 0.1545 | 323.62 | 3.9520 | 0.0588 | 0.7347 |
| RAV (ours) | 0.1598 | 264.89 | 2.2586 | 0.0095 | 0.7766 |
RAV achieves the best FVD, tOF, tLP, and identity similarity, while Genfocus achieves the lowest LPIPS.
Inputs and target refocused videos are synthesized with BokehMe. These metrics measure agreement with the synthesized references; the user study below complements them with direct perceptual judgments.
95 participants compare anonymized outputs on 12 videos for temporal consistency and spatial refocusing accuracy. Percentages below include all valid responses, including “None / difficult to judge.”
| Comparison | Criterion | RAV | Baseline | None / difficult to judge |
|---|---|---|---|---|
| vs. DiffCamera | Temporal consistency | 78.24% | 11.51% | 10.25% |
| vs. DiffCamera | Spatial refocusing accuracy | 71.44% | 16.53% | 12.03% |
| vs. Learn2Refocus | Temporal consistency | 66.98% | 22.01% | 11.01% |
| vs. Learn2Refocus | Spatial refocusing accuracy | 65.30% | 25.16% | 9.54% |
| vs. Genfocus | Temporal consistency | 78.97% | 8.26% | 12.76% |
| vs. Genfocus | Spatial refocusing accuracy | 64.54% | 15.27% | 20.19% |
Each row sums to approximately 100% due to rounding. Bold marks the most selected response.
| Method | LPIPS ↓ | FVD ↓ | tOF ↓ | tLP ↓ | IDS ↑ |
|---|---|---|---|---|---|
| DiffCamera | 0.2507 | 430.54 | 3.5819 | 0.0639 | 0.6776 |
| Genfocus | 0.2096 | 389.03 | 9.7415 | 0.0962 | 0.7491 |
| RAV (ours) | 0.2098 | 263.60 | 2.6953 | 0.0077 | 0.7846 |
Learn2Refocus is omitted because it does not produce an explicit all-in-focus output.
Both ablations evaluate Stage 1 on CelebV-HQ-AIF. Lower is better for all metrics; bold marks the better variant.
| Variant | MMD ↓ | tLP ↓ | FVD ↓ |
|---|---|---|---|
| Without LTSC | 0.00605 | 0.0158 | 262.76 |
| With LTSC | 0.00575 | 0.0080 | 263.60 |
| Variant | LD ↓ | LVD ↓ |
|---|---|---|
| Without both | 2.563 | 0.719 |
| With both | 2.503 | 0.700 |
MMD is the squared maximum mean discrepancy between per-frame Laplacian-variance sequences. LTSC improves sharpness-distribution alignment and tLP, with a small increase in FVD.
LD is Landmark Distance; LVD is Landmark Velocity Distance. Removing LFC also removes its associated Refiner module.
| Method | Resolution | Time (s) ↓ |
|---|---|---|
| DiffCamera† | 512 × 512 | 1283 |
| Learn2Refocus | 512 × 896 | 2104 |
| Genfocus | 512 × 896 | 1772 |
| RAV (ours) | 512 × 896 | 731 |
† DiffCamera is evaluated at 512 × 512 because its released model does not support 512 × 896 inputs; its runtime uses a different resolution.
We adapted Ultra Video to create Ultra Video All in Focus (UltraVideo-AIF), which we use for both qualitative and quantitative evaluation.
@inproceedings{gan2026refocusinganyvideo,
title = {Refocusing Any Video: Temporally Consistent Video Refocusing for Dynamic Focal Planes},
author = {Gan, Chee Kim and Chiu, Pin-Yen and Fang, I-Sheng and Chen, Jun-Cheng},
year = {2026},
booktitle = {SIGGRAPH Asia 2026 Conference Papers},
}