[10.15]Thru3D媒体服务器（原VR视频透视服务器）V1.5.2版（支持字幕、离线鱼眼转透视）

pwzdby

zelefans 发表于 2026-5-27 00:31
SAM3对显卡要求太高，能用得起的人不多也。虽然我的16G显存也能跑。 我这几天专攻离线，明天看有啥进展。 ...

我是开不开加速都一样，绿幕我没试过，反正不影响我，我用跳舞视频vr离线如果sam3开加速会离线不了，通道问题已经解决了。我现在用sam3 p4跳舞视频可以37这样还行，能接受，格式不一样，我要是离普通岛国vr就快了，就是拖影太严重了

zelefans

pwzdby 发表于 2026-5-27 01:37
我是开不开加速都一样，绿幕我没试过，反正不影响我，我用跳舞视频vr离线如果sam3开加速会离线不了，通道 ...

有空帮我试一下加速模式下的绿幕模式，如果这个没问题就证明40系列显卡在FP16上有问题了。

拖影找到优化点了，通过模型论文深入挖到了拖影根源，大幅提高了效果，看看有能力往工业级靠不。。。

pwzdby

zelefans 发表于 2026-5-27 08:16
有空帮我试一下加速模式下的绿幕模式，如果这个没问题就证明40系列显卡在FP16上有问题了。

拖影找到优化 ...

大佬威武，目前迫在眉睫需要处理拖影问题，加速我来试一下后，给你反馈

pwzdby

实测 开启加速离线绿幕模式和不开并没有提升，跳舞类视频rv1快速p4下依旧37fps左右。p1下 95fps。sam3开启加速前置跳完，后续依然卡住离不了，关掉加速则可以。大佬拖影什么时候更新，我得把视频重新离一遍

zelefans

pwzdby 发表于 2026-5-27 10:59
实测 开启加速离线绿幕模式和不开并没有提升，跳舞类视频rv1快速p4下依旧37fps左右。p1下 95fps。sam3开启 ...

我还是无法定位你现在的问题。。。。
1、RVM模式下打开TensorRT加速不卡住并能跑完出最终视频吗? 能否跑个15s给复制日志我看下tensorrt有没有启用。如果速度不变感觉没有启用。
2、如果RVM模式下打开TensorRT加速并能跑完，绿幕模式效果是否正常，Alpha模式四个角和中间上下有没有红色的，我问了好几次都没说。。。。
2、MatAnyone2的卡住的情况，只有SAM3前置有问题，还是yoloworld+efficientsam也会卡住。我要判断是否是显存占用的问题

pwzdby

这是直通RVM快速 P4 开了加速跑出的 和不开速度一样，开了反而画面有点跳动
[offline] GPU runtime initialization may take a while on first use. If this is the first offline run after installation or a driver/model update, the window can stay quiet for 1-3 minutes while CUDA/ONNX Runtime caches are built.
[offline] run: D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\pt_core.exe tool offline_alpha_passthrough D:\Downloads\59306_3840p.mp4 --engine rvm --out D:\59306_3840p_rvm1_S000000_15S_LR_180_FISHEYE_F180_alpha.mp4 --audio copy --model D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\models\rvm_mobilenetv3_fp32.onnx --duration 15.0 --bitrate source --alpha-stride 1 --preset P4 --cq -1 --input-size 1024 --sbs-batch
[offline] gpu warmup: state=cold reason=marker_missing eta=45s gpu=NVIDIA GeForce RTX 4090 cc=8.9 ort=1.25.1
[offline] gpu warmup: first offline run can appear idle while CUDA/CuPy/ONNX Runtime load libraries and build caches. Please wait for the first [matting] or [offline-*] progress line.
[offline] env: decoder=simple batch=1 buffer=2 preset=P4 model= providers=TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider offline_rvm_trt_enable=1 offline_matanyone2_trt_enable=0 offline_rvm_trt=1 offline_matanyone2_trt=0
[offline-alpha] input codec=av1 profile=Main pix_fmt=yuv420p bit_depth=8 size=7680x3840
[offline-alpha] cuda gpu_id=0 name=NVIDIA GeForce RTX 4090 cc=8.9 vram=24.0GB free=22.5GB driver=13000 runtime=12090
[offline-alpha] RVM TensorRT enabled providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
[matting] CuPy 14.0.1 detected, GPU composite enabled.
[matting] options: input_size=1024 square=False split_sbs=True sbs_batch=True alpha_stride=1 alpha_mode=reuse alpha_cutoff=0.0 alpha_hard_edge=False alpha_contrast=1.0
[matting] model kind=rvm input=src shape=['batch_size', 3, 'height', 'width'] type=tensor(float) batch2=True model_batch2=True rvm_iobinding=True static_trt=True providers=['CUDAExecutionProvider', 'CPUExecutionProvider'] inputs=['src', 'r1i', 'r2i', 'r3i', 'r4i', 'downsample_ratio'] outputs=['fgr', 'pha', 'r1o', 'r2o', 'r3o', 'r4o']
[offline-alpha] RVM runtime provider_diag={'main': ['CUDAExecutionProvider', 'CPUExecutionProvider'], 'static_trt': True, 'iobinding': True}
[offline-alpha] decoder=simple batch=1 buffer=2 owned_copy=False backend_reason=8K+ output should use HEVC; H.264 NVENC failed in probe
[offline-alpha] src=D:\Downloads\59306_3840p.mp4 out=D:\59306_3840p_rvm1_S000000_15S_LR_180_FISHEYE_F180_alpha.mp4 engine=rvm 7680x3840 output=7680x3840 source_fps=60.000000 output_fps=60.000000 target=900 audio=copy bitrate=23466599 maxbitrate=28159918 vbvbufsize=46933198 rc=vbr cq=-1 preset=P4
[offline-alpha] projection=sbs half-equirect 180 -> fisheye sbs radius_scale=1
[offline-alpha] packing=DeoVR-style red-channel alpha blocks scale=0.4 layout=alpha-packer-6block
[offline] mux=.\ffmpeg.EXE -hide_banner -loglevel warning -y -fflags +genpts -f hevc -framerate 60.000000 -i - -map 0:v:0 -c:v copy -an -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -movflags +faststart -f mp4 D:\59306_3840p_rvm1_S000000_15S_LR_180_FISHEYE_F180_alpha._video_only.mp4
[offline]      30/900 fps=  32.35 dec= 22.15ms mat=  7.19ms sync= 0.00ms dec+mat+sync= 29.34ms enc= 0.28ms mux= 1.40ms
[offline]      60/900 fps=  35.12 dec= 22.40ms mat=  4.90ms sync= 0.00ms dec+mat+sync= 27.30ms enc= 0.21ms mux= 0.98ms
[offline]      90/900 fps=  36.12 dec= 22.68ms mat=  4.14ms sync= 0.00ms dec+mat+sync= 26.83ms enc= 0.19ms mux= 0.66ms
[offline]     120/900 fps=  36.66 dec= 22.82ms mat=  3.76ms sync= 0.00ms dec+mat+sync= 26.59ms enc= 0.18ms mux= 0.50ms
[offline]     150/900 fps=  36.95 dec= 22.93ms mat=  3.53ms sync= 0.00ms dec+mat+sync= 26.46ms enc= 0.17ms mux= 0.41ms
[offline]     180/900 fps=  37.19 dec= 22.98ms mat=  3.38ms sync= 0.00ms dec+mat+sync= 26.36ms enc= 0.16ms mux= 0.35ms
[offline]     210/900 fps=  37.32 dec= 23.03ms mat=  3.27ms sync= 0.00ms dec+mat+sync= 26.31ms enc= 0.16ms mux= 0.31ms
[offline]     240/900 fps=  37.45 dec= 23.06ms mat=  3.19ms sync= 0.00ms dec+mat+sync= 26.25ms enc= 0.16ms mux= 0.27ms
[offline]     270/900 fps=  37.53 dec= 23.09ms mat=  3.13ms sync= 0.00ms dec+mat+sync= 26.22ms enc= 0.16ms mux= 0.24ms
[offline]     300/900 fps=  37.62 dec= 23.11ms mat=  3.07ms sync= 0.00ms dec+mat+sync= 26.18ms enc= 0.16ms mux= 0.22ms
[offline]     330/900 fps=  37.67 dec= 23.13ms mat=  3.03ms sync= 0.00ms dec+mat+sync= 26.16ms enc= 0.15ms mux= 0.21ms
[offline]     360/900 fps=  37.73 dec= 23.13ms mat=  3.00ms sync= 0.00ms dec+mat+sync= 26.14ms enc= 0.15ms mux= 0.19ms
[offline]     390/900 fps=  37.76 dec= 23.15ms mat=  2.97ms sync= 0.00ms dec+mat+sync= 26.12ms enc= 0.15ms mux= 0.18ms
[offline]     420/900 fps=  37.81 dec= 23.15ms mat=  2.95ms sync= 0.00ms dec+mat+sync= 26.10ms enc= 0.15ms mux= 0.17ms
[offline]     450/900 fps=  37.83 dec= 23.17ms mat=  2.92ms sync= 0.00ms dec+mat+sync= 26.09ms enc= 0.15ms mux= 0.17ms
[offline]     480/900 fps=  37.86 dec= 23.17ms mat=  2.90ms sync= 0.00ms dec+mat+sync= 26.08ms enc= 0.15ms mux= 0.16ms
[offline]     510/900 fps=  37.88 dec= 23.18ms mat=  2.89ms sync= 0.00ms dec+mat+sync= 26.07ms enc= 0.15ms mux= 0.15ms
[offline]     540/900 fps=  37.91 dec= 23.18ms mat=  2.87ms sync= 0.00ms dec+mat+sync= 26.06ms enc= 0.15ms mux= 0.15ms
[offline]     570/900 fps=  37.93 dec= 23.19ms mat=  2.86ms sync= 0.00ms dec+mat+sync= 26.05ms enc= 0.15ms mux= 0.14ms
[offline]     600/900 fps=  37.95 dec= 23.19ms mat=  2.85ms sync= 0.00ms dec+mat+sync= 26.04ms enc= 0.15ms mux= 0.14ms
[offline]     630/900 fps=  37.96 dec= 23.20ms mat=  2.84ms sync= 0.00ms dec+mat+sync= 26.04ms enc= 0.15ms mux= 0.13ms
[offline]     660/900 fps=  37.98 dec= 23.20ms mat=  2.83ms sync= 0.00ms dec+mat+sync= 26.03ms enc= 0.15ms mux= 0.13ms
[offline]     690/900 fps=  37.99 dec= 23.21ms mat=  2.82ms sync= 0.00ms dec+mat+sync= 26.03ms enc= 0.15ms mux= 0.12ms
[offline]     720/900 fps=  38.01 dec= 23.21ms mat=  2.81ms sync= 0.00ms dec+mat+sync= 26.02ms enc= 0.15ms mux= 0.12ms
[offline]     750/900 fps=  38.02 dec= 23.21ms mat=  2.80ms sync= 0.00ms dec+mat+sync= 26.02ms enc= 0.15ms mux= 0.12ms
[offline]     780/900 fps=  38.03 dec= 23.21ms mat=  2.80ms sync= 0.00ms dec+mat+sync= 26.01ms enc= 0.15ms mux= 0.11ms
[offline]     810/900 fps=  38.04 dec= 23.21ms mat=  2.79ms sync= 0.00ms dec+mat+sync= 26.01ms enc= 0.15ms mux= 0.11ms
[offline]     840/900 fps=  38.05 dec= 23.22ms mat=  2.78ms sync= 0.00ms dec+mat+sync= 26.00ms enc= 0.15ms mux= 0.11ms
[offline]     870/900 fps=  38.05 dec= 23.22ms mat=  2.78ms sync= 0.00ms dec+mat+sync= 26.00ms enc= 0.15ms mux= 0.11ms
[offline]     900/900 fps=  38.06 dec= 23.22ms mat=  2.77ms sync= 0.00ms dec+mat+sync= 26.00ms enc= 0.15ms mux= 0.10ms
---- summary ----
rc = 0
frames = 900
video_bytes = 53937672
elapsed = 23.754 s
throughput = 37.89 fps
source_index_rewinds = 0
decode_avg = 23.219 ms
decode_p99 = 24.616 ms
matting_avg = 2.774 ms
matting_p99 = 3.373 ms
sync_avg = 0.003 ms
sync_p99 = 0.005 ms
decode_matting_avg = 25.993 ms
decode_matting_sync_avg = 25.996 ms
rvm_upload_nv12_avg = 0.054 ms n=900
rvm_preprocess_avg = 0.046 ms n=900
rvm_ort_avg = 2.579 ms n=900
rvm_alpha_pack_avg = 0.033 ms n=900
encode_avg = 0.146 ms
encode_p99 = 0.211 ms
mux_write_avg = 0.105 ms
mux_write_p99 = 0.256 ms
mem_start = 18.9/18.9 MB
mem_end = 151.5/229.4 MB
[ffmpeg stderr]
[mp4 @ 00000205defc5040] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[offline] audio_mux=.\ffmpeg.EXE -hide_banner -loglevel warning -y -i D:\59306_3840p_rvm1_S000000_15S_LR_180_FISHEYE_F180_alpha._video_only.mp4 -i D:\Downloads\59306_3840p.mp4 -map 0:v:0 -map 1:a:0? -c:v copy -c:a copy -t 15.000000 -map_metadata 1 -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -movflags +faststart D:\59306_3840p_rvm1_S000000_15S_LR_180_FISHEYE_F180_alpha.mp4
audio_mux_rc = 0
audio_mux_elapsed = 0.096 s
[ffprobe]
codec_name=hevc
width=7680
height=3840
color_range=tv
color_space=bt709
color_transfer=bt709
color_primaries=bt709
avg_frame_rate=1000000/16667
nb_frames=900
codec_name=aac
avg_frame_rate=0/0
nb_frames=705
format_name=mov,mp4,m4a,3gp,3g2,mj2
duration=15.018000
size=54418964
[offline] child process exited rc=0
[offline] process completed rc=0

pwzdby

这是M2 慢速SAM3 P4 开了加速跑的15秒居然现在又可以了，但是速度依然和不开没有区别，离线全部会卡在[offline] MatAnyone2 segment reset at frame=0 src_idx=0 segment_start=0这里不动，关掉加速则可以，后边的不发了太长字节限制了
[offline] GPU runtime initialization may take a while on first use. If this is the first offline run after installation or a driver/model update, the window can stay quiet for 1-3 minutes while CUDA/ONNX Runtime caches are built.
[offline] MatAnyone2 requires an NVIDIA GPU with at least 16GB VRAM.
[offline] run: D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\pt_core.exe tool offline_alpha_passthrough D:\Downloads\59306_3840p.mp4 --engine matanyone2_onnx --out D:\59306_3840p_matanyone2_S000000_15S_LR_180_FISHEYE_F180_alpha.mp4 --audio copy --duration 15.0 --bitrate source --alpha-stride 1 --matanyone2-size 512 --matanyone2-batch 1 --no-sbs-batch --sam3-prompt person --preset P4 --cq -1
[offline] gpu warmup: state=cold reason=marker_missing eta=45s gpu=NVIDIA GeForce RTX 4090 cc=8.9 ort=1.25.1
[offline] gpu warmup: first offline run can appear idle while CUDA/CuPy/ONNX Runtime load libraries and build caches. Please wait for the first [matting] or [offline-*] progress line.
[offline] env: decoder=simple batch=1 buffer=2 preset=P4 model= providers=TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider offline_rvm_trt_enable=0 offline_matanyone2_trt_enable=1 offline_rvm_trt=0 offline_matanyone2_trt=1
[offline-alpha] input codec=av1 profile=Main pix_fmt=yuv420p bit_depth=8 size=7680x3840
[offline-alpha] cuda gpu_id=0 name=NVIDIA GeForce RTX 4090 cc=8.9 vram=24.0GB free=22.5GB driver=13000 runtime=12090
[offline-alpha] decoder=simple batch=1 buffer=2 owned_copy=False backend_reason=8K+ output should use HEVC; H.264 NVENC failed in probe
[offline] SAM3 prepass subprocess=D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\pt_core.exe tool offline_alpha_passthrough D:\Downloads\59306_3840p.mp4 --engine matanyone2_onnx --model D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\models\matanyone2_onnx_512_bs1 --matanyone2-size 512 --matanyone2-batch 1 --sam3-model-dir D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\models\sam3_onnx --sam3-prompt person --sam3-score-threshold 0.5 --sam3-min-area-ratio 0.0005 --sam3-max-area-ratio 0.95 --sam3-top-k 0 --sam3-scan hybrid --sam3-scan-interval-sec 1.0 --sam3-active-min-area-ratio 0.001 --sam3-provider cuda --sam3-decoder-provider cuda --sam3-cuda-memory-limit-mb 8192 --sam3-decoder-cuda-memory-limit-mb 4096 --sam3-min-vram-gb 15.5 --sam3-release-interval 0 --no-sam3-low-memory --sam3-low-memory-batch 1 --sam3-log-vram --matanyone2-segment-frames 300 --matanyone2-min-segment-sec 3.0 --sam3-gap-fill-frames 300 --frames 900 --fps 60.0 --sam3-prepass-out D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\debug_output\_sam3_prepass\sam3_prepass_1779857195596_2395746612960.npz
[offline-alpha] input codec=av1 profile=Main pix_fmt=yuv420p bit_depth=8 size=7680x3840
[offline-alpha] cuda gpu_id=0 name=NVIDIA GeForce RTX 4090 cc=8.9 vram=24.0GB free=22.5GB driver=13000 runtime=12090
[offline-alpha] decoder=simple batch=1 buffer=2 owned_copy=False backend_reason=8K+ output should use HEVC; H.264 NVENC failed in probe
[offline] SAM3 prepass prompt='person' samples=18 scan=hybrid max_segment_frames=300 low_memory=False encoder=[CUDAExecutionProvider(arena_cap=8192MiB), CPUExecutionProvider] decoder=[CUDAExecutionProvider(arena_cap=4096MiB), CPUExecutionProvider]
[0;93m2026-05-27 12:46:43.7317864 [Wnnxruntime:, transformer_memcpy.cc:111 onnxruntime::MemcpyTransformer::ApplyImpl] 2 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
[0;93m2026-05-27 12:46:43.9107090 [Wnnxruntimeefault, scatter_nd.h:53 onnxruntime::cuda::ScatterNDWithAtomicReduction::ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
[offline] SAM3 prepass 1/18 frame=0 src_idx=0 ms=5629.7 active=True objects=1 L=[0] area=0.0145 R=[0] area=0.0144 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 2/18 frame=60 src_idx=60 ms=3936.2 active=True objects=2 L=[0, 1] area=0.0161 R=[0, 1] area=0.0159 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 3/18 frame=120 src_idx=120 ms=3943.4 active=True objects=2 L=[0, 1] area=0.0163 R=[0, 1] area=0.0163 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 4/18 frame=180 src_idx=180 ms=3949.3 active=True objects=2 L=[0, 1] area=0.0145 R=[0, 1] area=0.0138 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 5/18 frame=240 src_idx=240 ms=3945.8 active=True objects=2 L=[0, 1] area=0.0190 R=[0, 1] area=0.0186 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 6/18 frame=250 src_idx=250 ms=3946.4 active=True objects=2 L=[0, 1] area=0.0189 R=[0, 1] area=0.0184 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 7/18 frame=300 src_idx=300 ms=3935.5 active=True objects=2 L=[0, 1] area=0.0177 R=[0, 1] area=0.0175 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 8/18 frame=360 src_idx=360 ms=3939.8 active=True objects=2 L=[0, 1] area=0.0164 R=[0] area=0.0152 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 9/18 frame=420 src_idx=420 ms=3941.6 active=True objects=2 L=[0, 1] area=0.0150 R=[0, 1] area=0.0145 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 10/18 frame=480 src_idx=480 ms=3938.2 active=True objects=2 L=[0, 1] area=0.0206 R=[0, 1] area=0.0203 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 11/18 frame=500 src_idx=500 ms=3948.1 active=True objects=2 L=[0, 1] area=0.0240 R=[0, 1] area=0.0231 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 12/18 frame=540 src_idx=540 ms=3942.0 active=True objects=2 L=[0, 1] area=0.0265 R=[0, 1] area=0.0264 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 13/18 frame=600 src_idx=600 ms=3948.7 active=True objects=2 L=[0, 1] area=0.0241 R=[0, 1] area=0.0236 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 14/18 frame=660 src_idx=660 ms=3946.8 active=True objects=2 L=[0, 1] area=0.0248 R=[0] area=0.0234 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 15/18 frame=720 src_idx=720 ms=3945.2 active=True objects=2 L=[0, 1] area=0.0208 R=[0, 1] area=0.0207 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 16/18 frame=750 src_idx=750 ms=3945.1 active=True objects=2 L=[0, 1] area=0.0211 R=[0] area=0.0194 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 17/18 frame=780 src_idx=780 ms=3950.2 active=True objects=2 L=[0, 1] area=0.0231 R=[0, 1] area=0.0226 stereo=paired gpu=23968/24564MiB
[offline] SAM3 prepass 18/18 frame=840 src_idx=840 ms=3961.8 active=True objects=2 L=[0, 1] area=0.0233 R=[0, 1] area=0.0231 stereo=paired gpu=23968/24564MiB
[offline] MatAnyone2 segment plan starts=[0, 60, 240, 420, 600, 780] active=[0, 60, 240, 420, 600, 780]
[offline-sam3] child process exited rc=0
[matting] CuPy 14.0.1 detected, GPU composite enabled.
[matting] options: input_size=1024 square=False split_sbs=True sbs_batch=False alpha_stride=1 alpha_mode=reuse alpha_cutoff=0.0 alpha_hard_edge=False alpha_contrast=1.0
[offline-alpha] MatAnyone2 TensorRT enabled model=matanyone2_step_update.onnx cache=D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\runtime_cache\trt_engines\matanyone2_onnx_512_bs1
[matting] utility mode: ONNX matting model not loaded
[offline] MatAnyone2 ONNX loaded dir=D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\models\matanyone2_onnx_512_bs1 input=512x512 sbs=per-eye bootstrap=sam3 sam3_prompt='person' bootstrap_erode=1 bootstrap_dilate=0 bootstrap_soft=False segment_frames=300 alpha_stride=1 batch2=False dtype=float32 fused_update=False step_update=True providers={'matanyone2_image_key.onnx': ['CUDAExecutionProvider', 'CPUExecutionProvider'], 'matanyone2_mask_memory.onnx': ['CUDAExecutionProvider', 'CPUExecutionProvider'], 'matanyone2_first_frame_refine.onnx': ['CUDAExecutionProvider', 'CPUExecutionProvider'], 'matanyone2_propagate.onnx': ['CUDAExecutionProvider', 'CPUExecutionProvider'], 'matanyone2_step_update.onnx': ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']}
[offline-alpha] src=D:\Downloads\59306_3840p.mp4 out=D:\59306_3840p_matanyone2_S000000_15S_LR_180_FISHEYE_F180_alpha.mp4 engine=matanyone2_onnx 7680x3840 output=7680x3840 source_fps=60.000000 output_fps=60.000000 target=900 audio=copy bitrate=23466599 maxbitrate=28159918 vbvbufsize=46933198 rc=vbr cq=-1 preset=P4
[offline-alpha] projection=sbs half-equirect 180 -> fisheye sbs radius_scale=1
[offline-alpha] packing=DeoVR-style red-channel alpha blocks scale=0.4 layout=alpha-packer-6block
[offline] mux=.\ffmpeg.EXE -hide_banner -loglevel warning -y -fflags +genpts -f hevc -framerate 60.000000 -i - -map 0:v:0 -c:v copy -an -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -movflags +faststart -f mp4 D:\59306_3840p_matanyone2_S000000_15S_LR_180_FISHEYE_F180_alpha._video_only.mp4
[offline] MatAnyone2 segment reset at frame=0 src_idx=0 segment_start=0
[offline]      30/900 fps=  23.18 dec= 12.87ms mat= 28.50ms sync= 0.00ms dec+mat+sync= 41.37ms enc= 0.27ms mux= 1.62ms
[offline]      60/900 fps=  28.92 dec= 13.46ms mat= 19.87ms sync= 0.00ms dec+mat+sync= 33.32ms enc= 0.21ms mux= 1.06ms
[offline] MatAnyone2 segment reset at frame=60 src_idx=60 segment_start=60
[offline]      90/900 fps=  31.47 dec= 13.46ms mat= 17.40ms sync= 0.00ms dec+mat+sync= 30.86ms enc= 0.19ms mux= 0.71ms
[offline]     120/900 fps=  32.94 dec= 13.75ms mat= 15.86ms sync= 0.00ms dec+mat+sync= 29.61ms enc= 0.18ms

pwzdby

至于四个角没有红点，我把原先的离线16模型清空了，重新覆盖了下又好了，绿幕模式下离线的视频都是正常的

pwzdby

这是绿幕 RVM快速 P4 开了加速的
[offline] GPU runtime initialization may take a while on first use. If this is the first offline run after installation or a driver/model update, the window can stay quiet for 1-3 minutes while CUDA/ONNX Runtime caches are built.
[offline] run: D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\pt_core.exe tool offline_passthrough D:\Downloads\59306_3840p.mp4 --engine rvm --out D:\59306_3840p_rvm1_S000000_15S_LR_180_SBS_passthrough.mp4 --audio copy --model D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server\models\rvm_mobilenetv3_fp32.onnx --duration 15.0 --bitrate source --alpha-stride 1 --preset P4 --cq -1 --input-size 1024 --sbs-batch
[offline] gpu warmup: state=cold reason=marker_missing eta=45s gpu=NVIDIA GeForce RTX 4090 cc=8.9 ort=1.25.1
[offline] gpu warmup: first offline run can appear idle while CUDA/CuPy/ONNX Runtime load libraries and build caches. Please wait for the first [matting] or [offline-*] progress line.
[offline] env: decoder=simple batch=1 buffer=2 preset=P4 model= providers=TensorrtExecutionProvider,CUDAExecutionProvider,CPUExecutionProvider offline_rvm_trt_enable=1 offline_matanyone2_trt_enable=0 offline_rvm_trt=1 offline_matanyone2_trt=0
[offline] input codec=av1 profile=Main pix_fmt=yuv420p bit_depth=8 size=7680x3840
[offline] cuda gpu_id=0 name=NVIDIA GeForce RTX 4090 cc=8.9 vram=24.0GB free=22.5GB driver=13000 runtime=12090
[offline] RVM TensorRT enabled providers=['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']
[matting] CuPy 14.0.1 detected, GPU composite enabled.
[matting] options: input_size=1024 square=False split_sbs=True sbs_batch=True alpha_stride=1 alpha_mode=reuse alpha_cutoff=0.0 alpha_hard_edge=False alpha_contrast=1.0
[matting] model kind=rvm input=src shape=['batch_size', 3, 'height', 'width'] type=tensor(float) batch2=True model_batch2=True rvm_iobinding=True static_trt=True providers=['CUDAExecutionProvider', 'CPUExecutionProvider'] inputs=['src', 'r1i', 'r2i', 'r3i', 'r4i', 'downsample_ratio'] outputs=['fgr', 'pha', 'r1o', 'r2o', 'r3o', 'r4o']
[offline] RVM runtime provider_diag={'main': ['CUDAExecutionProvider', 'CPUExecutionProvider'], 'static_trt': True, 'iobinding': True}
[offline] decoder=simple batch=1 buffer=2 backend_reason=8K+ output should use HEVC; H.264 NVENC failed in probe
[offline] audio_extract=.\ffmpeg.EXE -hide_banner -loglevel warning -y -i D:\Downloads\59306_3840p.mp4 -map 0:a:0? -vn -c:a copy -t 15.000000 -f adts D:\59306_3840p_rvm1_S000000_15S_LR_180_SBS_passthrough._audio.aac
audio_extract_rc = 0
audio_extract_elapsed = 0.031 s
[offline] src=D:\Downloads\59306_3840p.mp4 out=D:\59306_3840p_rvm1_S000000_15S_LR_180_SBS_passthrough.mp4 engine=rvm 7680x3840 source_fps=60.000000 output_fps=60.000000 target=900 audio=copy bitrate=23466599 maxbitrate=28159918 vbvbufsize=46933198 rc=vbr cq=-1 preset=P4
[offline] mux=.\ffmpeg.EXE -hide_banner -loglevel warning -y -fflags +genpts -f hevc -framerate 60.000000 -i - -map 0:v:0 -c:v copy -an -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -movflags +faststart -f mp4 D:\59306_3840p_rvm1_S000000_15S_LR_180_SBS_passthrough._video_only.mp4
[offline]      30/900 fps=  32.96 dec= 22.33ms mat=  6.96ms sync= 0.33ms dec+mat+sync= 29.61ms enc= 0.28ms mux= 0.46ms
[offline]      60/900 fps=  35.27 dec= 22.79ms mat=  4.80ms sync= 0.32ms dec+mat+sync= 27.91ms enc= 0.20ms mux= 0.22ms
[offline]      90/900 fps=  36.23 dec= 22.86ms mat=  4.08ms sync= 0.32ms dec+mat+sync= 27.25ms enc= 0.18ms mux= 0.15ms
[offline]     120/900 fps=  36.94 dec= 22.74ms mat=  3.72ms sync= 0.32ms dec+mat+sync= 26.78ms enc= 0.16ms mux= 0.11ms
[offline]     150/900 fps=  37.37 dec= 22.66ms mat=  3.51ms sync= 0.32ms dec+mat+sync= 26.49ms enc= 0.16ms mux= 0.09ms
[offline]     180/900 fps=  37.66 dec= 22.62ms mat=  3.36ms sync= 0.32ms dec+mat+sync= 26.30ms enc= 0.15ms mux= 0.07ms
[offline]     210/900 fps=  37.88 dec= 22.58ms mat=  3.26ms sync= 0.32ms dec+mat+sync= 26.16ms enc= 0.15ms mux= 0.06ms
[offline]     240/900 fps=  38.04 dec= 22.56ms mat=  3.18ms sync= 0.32ms dec+mat+sync= 26.06ms enc= 0.14ms mux= 0.06ms
[offline]     270/900 fps=  38.16 dec= 22.55ms mat=  3.12ms sync= 0.32ms dec+mat+sync= 25.99ms enc= 0.14ms mux= 0.05ms
[offline]     300/900 fps=  38.26 dec= 22.54ms mat=  3.07ms sync= 0.32ms dec+mat+sync= 25.93ms enc= 0.14ms mux= 0.04ms
[offline]     330/900 fps=  38.34 dec= 22.53ms mat=  3.03ms sync= 0.32ms dec+mat+sync= 25.88ms enc= 0.14ms mux= 0.04ms
[offline]     360/900 fps=  38.41 dec= 22.52ms mat=  3.00ms sync= 0.32ms dec+mat+sync= 25.83ms enc= 0.14ms mux= 0.04ms
[offline]     390/900 fps=  38.48 dec= 22.51ms mat=  2.97ms sync= 0.32ms dec+mat+sync= 25.79ms enc= 0.14ms mux= 0.03ms
[offline]     420/900 fps=  38.52 dec= 22.50ms mat=  2.95ms sync= 0.32ms dec+mat+sync= 25.77ms enc= 0.14ms mux= 0.03ms
[offline]     450/900 fps=  38.57 dec= 22.50ms mat=  2.92ms sync= 0.32ms dec+mat+sync= 25.74ms enc= 0.14ms mux= 0.03ms
[offline]     480/900 fps=  38.61 dec= 22.49ms mat=  2.90ms sync= 0.32ms dec+mat+sync= 25.71ms enc= 0.14ms mux= 0.03ms
[offline]     510/900 fps=  38.64 dec= 22.49ms mat=  2.89ms sync= 0.32ms dec+mat+sync= 25.69ms enc= 0.14ms mux= 0.03ms
[offline]     540/900 fps=  38.66 dec= 22.49ms mat=  2.88ms sync= 0.32ms dec+mat+sync= 25.68ms enc= 0.13ms mux= 0.03ms
[offline]     570/900 fps=  38.68 dec= 22.49ms mat=  2.86ms sync= 0.32ms dec+mat+sync= 25.67ms enc= 0.13ms mux= 0.02ms
[offline]     600/900 fps=  38.69 dec= 22.50ms mat=  2.85ms sync= 0.32ms dec+mat+sync= 25.67ms enc= 0.13ms mux= 0.02ms
[offline]     630/900 fps=  38.70 dec= 22.50ms mat=  2.84ms sync= 0.32ms dec+mat+sync= 25.66ms enc= 0.13ms mux= 0.02ms
[offline]     660/900 fps=  38.71 dec= 22.50ms mat=  2.83ms sync= 0.32ms dec+mat+sync= 25.65ms enc= 0.13ms mux= 0.02ms
[offline]     690/900 fps=  38.72 dec= 22.51ms mat=  2.83ms sync= 0.32ms dec+mat+sync= 25.65ms enc= 0.13ms mux= 0.02ms
[offline]     720/900 fps=  38.73 dec= 22.51ms mat=  2.82ms sync= 0.32ms dec+mat+sync= 25.64ms enc= 0.13ms mux= 0.02ms
[offline]     750/900 fps=  38.74 dec= 22.51ms mat=  2.81ms sync= 0.32ms dec+mat+sync= 25.64ms enc= 0.13ms mux= 0.02ms
[offline]     780/900 fps=  38.75 dec= 22.51ms mat=  2.80ms sync= 0.32ms dec+mat+sync= 25.63ms enc= 0.13ms mux= 0.02ms
[offline]     810/900 fps=  38.75 dec= 22.51ms mat=  2.80ms sync= 0.32ms dec+mat+sync= 25.63ms enc= 0.13ms mux= 0.02ms
[offline]     840/900 fps=  38.76 dec= 22.52ms mat=  2.79ms sync= 0.32ms dec+mat+sync= 25.63ms enc= 0.13ms mux= 0.02ms
[offline]     870/900 fps=  38.77 dec= 22.52ms mat=  2.79ms sync= 0.32ms dec+mat+sync= 25.62ms enc= 0.13ms mux= 0.02ms
[offline]     900/900 fps=  38.77 dec= 22.52ms mat=  2.78ms sync= 0.32ms dec+mat+sync= 25.62ms enc= 0.13ms mux= 0.02ms
---- summary ----
rc = 0
frames = 900
video_bytes = 3145716
elapsed = 23.295 s
throughput = 38.64 fps
source_index_rewinds = 0
decode_avg = 22.520 ms
decode_p99 = 23.754 ms
matting_avg = 2.784 ms
matting_p99 = 3.341 ms
sync_avg = 0.318 ms
sync_p99 = 0.621 ms
decode_matting_avg = 25.303 ms
decode_matting_sync_avg = 25.621 ms
rvm_preprocess_avg = 0.048 ms n=900
rvm_ort_avg = 2.565 ms n=900
rvm_composite_avg = 0.089 ms n=900
encode_avg = 0.130 ms
encode_p99 = 0.223 ms
mux_write_avg = 0.016 ms
mux_write_p99 = 0.207 ms
mem_start = 18.9/18.9 MB
mem_end = 299.0/317.8 MB
[ffmpeg stderr]
[mp4 @ 0000016b1a45b280] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
[offline] audio_mux=.\ffmpeg.EXE -hide_banner -loglevel warning -y -i D:\59306_3840p_rvm1_S000000_15S_LR_180_SBS_passthrough._video_only.mp4 -f aac -i D:\59306_3840p_rvm1_S000000_15S_LR_180_SBS_passthrough._audio.aac -map 0:v:0 -map 1:a:0? -c:v copy -c:a copy -t 15.000000 -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -movflags +faststart D:\59306_3840p_rvm1_S000000_15S_LR_180_SBS_passthrough.mp4
audio_mux_rc = 0
audio_mux_elapsed = 0.066 s
[audio mux stderr]
[aac @ 00000259a612db00] Estimating duration from bitrate, this may be inaccurate
[ffprobe]
codec_name=hevc
width=7680
height=3840
color_range=tv
color_space=bt709
color_transfer=bt709
color_primaries=bt709
avg_frame_rate=1000000/16667
nb_frames=900
codec_name=aac
avg_frame_rate=0/0
nb_frames=704
format_name=mov,mp4,m4a,3gp,3g2,mj2
duration=15.018667
size=3626255
[offline] child process exited rc=0
[offline] process completed rc=0

zelefans

pwzdby 发表于 2026-5-27 13:00
这是绿幕 RVM快速 P4 开了加速的
[offline] GPU runtime initialization may take a while on first use. I ...

非常感谢。我正在替换YOLO识别模型，弄好了研究你这些日志。这几天应该能出个新版本.

一眼看到速度问题在你选了P4画质。
[offline]      30/900 fps=  32.35 dec= 22.15ms mat=  7.19ms sync= 0.00ms dec+mat+sync= 29.34ms enc= 0.28ms mux= 1.40ms
dec=22.15ms，这个是读取原始视频到抠图前的时间，这个是大头
mat=  7.19ms，切图时间是小头，TensorRT优化的是这个时间，所以对你顶配显卡来说区别不大了。

[offline] RVM runtime provider_diag={'main': ['CUDAExecutionProvider', 'CPUExecutionProvider'], 'static_trt': True, 'iobinding': True}
其中'static_trt': True表明了已经启用了TensorRT了

抖动问题我怀疑是FP16的问题，这是神经网络的算法导致的，下个版本改FP32