REAL Video Enhancer

NOTICE

This project is under a massive refactor, the current version will not be updated for a while.

Download on Steam!

Table of Contents

• Introduction
• Features
• Hardware Requirements
• Models
  • Interpolate Models
  • Upscale Models
• Backends
• FAQ
  • General App Usage
  • TensorRT
  • ROCm
  • NCNN
• Cloning
• Building
• Colab Notebook
• Credits
  • People
  • Software

Introduction

REAL Video Enhancer is a redesigned and enhanced version of the original Rife ESRGAN App for Linux. This program offers convenient access to frame interpolation and upscaling functionalities on Windows, Linux and MacOS, and is an alternative to outdated software like Flowframes or enhancr.

Features:

• Windows support. !!! NOTICE !!! The bin can be detected as a trojan. This is a false positive caused by pyinstaller.
• Ubuntu 22.04+ support on Executable and Flatpak. (20.04 can work but is now legacy)
• MacOS 15+ arm/x86 support
• Discord RPC support for Discord system package and Discord flatpak.
• Scene change detection to preserve sharp transitions.
• Preview that shows latest frame that has been rendered.
• TensorRT and NCNN for efficient inference across many GPUs.

Hardware/Software Requirements

	Minimum	Recommended
CPU	Dual Core x64 bit	Quad Core x64 bit
GPU	Vulkan 1.3 capable device	Nvidia RTX GPU (20 series and up)
VRAM	4 GB - NCNN	8 GB - TensorRT (Nvidia, why keep making 8gb cards?)
RAM	16 GB	32 GB
Storage	1 GB free - NCNN	16 GB free - TensorRT
Operating System	Windows 10/11 64bit / MacOS 14+	Any modern Linux distro (Ubuntu 22.04+)

Models:

Interpolate Models:

Model	Author	Link
RIFE 4.6,4.7,4.15,4.18,4.22,4.22-lite,4.25	Hzwer	Practical-RIFE
GMFSS	98mxr	GMFSS_Fortuna
IFRNet	ltkong218	IFRnet

Upscale Models:

Model	Author	Link
4x-SPANkendata	Crustaceous D	4x-SPANkendata
4x-Nomos8k-SPAN series	Helaman	4x-Nomos8k-SPAN series
2x-OpenProteus	SiroSky	OpenProteus
2x-AnimeJaNai V2 and V3 Sharp	The Database	AnimeJanai
2x-AniSD	SiroSky	AniSD
AnimeSR	Tencent ARC	AnimeSR

Decompression Models:

Model	Author	Link
DeH264	Helaman	1xDeH264_realplksr

Denoise Models:

Model	Author	Link
DRUnet	cszn	DRUnet
DnCNN	czsn	DnCNN

Backends

Backend	Hardware
TensorRT	NVIDIA RTX GPUs
PyTorch	CUDA 12.6 and ROCm 6.2 capable GPUs
NCNN	Vulkan 1.3 capable GPUs

FAQ

General Application Usage

Question	Answer
What does this program attempt to accomplish?	Fast, efficient and easily accessable video interpolation (Ex: 24->48FPS) and video upscaling (Ex: 1920->3840)
Why is it failing to recognize installed backends?	REAL Video Enhancer uses PIP and portable python for inference, this can sometimes have issues installing. Please attempt reinstalling the app before creating an issue.

TensorRT related questions

Why does it take so long to begin inference?	TensorRT uses advanced optimization at the beginning of inference based on your device, this is only done once per resolution of video inputed.
Why does the optimization and inference fail?	The most common way an optimization can fail is Limited VRAM There is no fix to this except using CUDA or NCNN instead.

ROCm related questions

Why am I getting (Insert Error here)?	ROCM is buggy, please take a look at ROCm Help.

NCNN related questions

Why am I getting (Insert Vulkan Error here)?	This usually is an OOM (Out Of Memory) error, this can indicate a weak iGPU or very old GPU, I recommeding trying out the Colab Notebook instead.

Cloning:

# Nightly
git clone --recurse-submodules https://github.com/TNTwise/REAL-Video-Enhancer 

# Stable
git clone --recurse-submodules https://github.com/TNTwise/REAL-Video-Enhancer --branch 2.4.1

Building:

3 supported build methods:

- pyinstaller (recommended for Win/Mac)
- cx_freeze (recommended for Linux)
- nuitka (experimental)

supported python versions:

- 3.10 3.11, 3.12

python3 build.py --build BUILD_OPTION --copy_backend

Colab Notebook

Colab Notebook

Credits:

People:

Person	For	Link
NevermindNilas	Some backend and reference code and working with me on many projects	https://github.com/NevermindNilas/
Styler00dollar	RIFE ncnn models (4.1-4.5, 4.7-4.12-lite), Sudo Shuffle Span, and Scene Detect Models	https://github.com/styler00dollar
HolyWu	TensorRT engine generation code, inference optimizations, and RIFE jagged lines fixes	https://github.com/HolyWu/
Rick Astley	Amazing music	https://www.youtube.com/watch?v=dQw4w9WgXcQ

Software:

Software Used	For	Link
FFmpeg	Multimedia framework for handling video, audio, and other media files	https://ffmpeg.org/
QT	GUI framework	https://qt.io/
FFMpeg Builds	Pre-compiled builds of FFMpeg.	Windows/Linux: https://github.com/BtbN/FFmpeg-Builds, MacOS: https://github.com/eko5624/mpv-mac
PyTorch	Neural Network Inference (CUDA/ROCm/TensorRT)	https://pytorch.org/
NCNN	Neural Network Inference (Vulkan)	https://github.com/tencent/ncnn
RIFE	Real-Time Intermediate Flow Estimation for Video Frame Interpolation	https://github.com/hzwer/Practical-RIFE
rife-ncnn-vulkan	Video frame interpolation implementation using NCNN and Vulkan	https://github.com/nihui/rife-ncnn-vulkan
rife ncnn vulkan python	Python bindings for RIFE NCNN Vulkan implementation	https://github.com/media2x/rife-ncnn-vulkan-python
GMFSS	GMFlow based Anime VFI	https://github.com/98mxr/GMFSS_Fortuna
GIMM	Motion Modeling Realistic VFI	https://github.com/GSeanCDAT/GIMM-VFI
ncnn python	Python bindings for NCNN Vulkan framework	https://pypi.org/project/ncnn
Real-ESRGAN	Upscaling	https://github.com/xinntao/Real-ESRGAN
SPAN	Upscaling	https://github.com/hongyuanyu/SPAN
Spandrel	CUDA upscaling model architecture support	https://github.com/chaiNNer-org/spandrel
ChaiNNer	Model Scale Detection	https://github.com/chaiNNer-org/chainner
cx_Freeze	Tool for creating standalone executables from Python scripts (Linux build)	https://github.com/marcelotduarte/cx_Freeze
PyInstaller	Tool for creating standalone executables from Python scripts (Windows/Mac builds)	https://github.com/pyinstaller/pyinstaller
Feather Icons	Open source icons library	https://github.com/feathericons/feather
PySceneDetect	Transition detection library for python	https://github.com/Breakthrough/PySceneDetect/
Python Standalone Builds	Backend inference using portable python, helps when porting to different platforms.	https://github.com/indygreg/python-build-standalone

Star History