Awesome ML Kit

Awesome ML Kit gives Unity developers easy way to employ optimized real-time ML models centered around computer vision.

Documentation

Forum thread

Background matting demo video

The current set of models includes

1. High-quality background removal/background matting based on RobustVideoMatting. Optimized for Unity, both horizontal and vertical/portrait video aspect ratio supported

2. Depth estimation by single image

Features

1. Sample scene for each model. Specially made visualizations with different post effects. Additional 3d scene, background blur and glitch scenes

2. Select solid color / image / video as background replacement; put Unity content between foreground & background layers

3. Different model versions for Quality / Speed tradeoff

4. Linear/Gamma color spaces supported

5. WebCamTexture support

6. Unity VideoPlayer integration

7. AVPro Live Camera integration (recommended instead of WebCamTexture)

8. AVPro Video 3 integraion

Ideal for

+ Video processing

+ Media installations

+ Apps with video streaming (e.g. can be used with Agora.io)

+ Kiosks

Performance (barracuda)

NVidia RTX3070

+ Background removal: 300 FPS

+ Depth estimation: 400 FPS

NVidia GTX1050Ti (mobile)

+ Background removal: 50 FPS

+ Depth estimation: 80 FPS

Note: only desktop platforms currently supported (tested on Windows DX11 and Linux Vulkan)

Unity versions: 2021 and above (Unity 6 now supported)