[10.15]Thru3D媒体服务器（原VR视频透视服务器）V1.5.2版（支持字幕、离线鱼眼转透视）

shuaige007

这是个技术贴

yz2931

唉，电脑配置有点低，只能等升级配置了试试

zelefans

项目官方一上午搭好了，启用了买了20多年没用过的域名：

http://wapok.com

defalt456

zelefans 发表于 2026-5-16 18:58
找一下错误信息。如果是软件奔溃可以在windows事件查看器里看到。
显卡也是1060吗？ ...

现在可以正常使用了，大佬有没有兴趣引入torch.compile给自己的两个项目，这个东西加速效果还是挺强的，n卡直接可以用的

5x5x5x4n

4070 8G显存测试跑YOLOWorld报错
[offline] GPU runtime initialization may take a while on first use. If this is the first offline run after installation or a driver/model update, the window can stay quiet for 1-3 minutes while CUDA/ONNX Runtime caches are built.
[offline] run: E:\VR_Video_Passthrough_Server\pt_core.exe tool offline_alpha_passthrough E:\VR_Video_Passthrough_Server\videos\nkwmr-018_2_8k.mp4 --engine matanyone2_onnx --out E:\VR_Video_Passthrough_Server\videos\nkwmr-018_2_8k_matanyone2m_S000000_1M_FISHEYE_alpha.mp4 --audio copy --duration 60.0 --alpha-stride 1 --matanyone2-size 512 --matanyone2-batch 1 --no-sbs-batch --matanyone2-prepass yoloworld_efficientsam
[offline] gpu warmup: state=cold reason=marker_missing eta=45s gpu=NVIDIA GeForce RTX 4070 Laptop GPU cc=8.9 ort=1.25.1
[offline] gpu warmup: first offline run can appear idle while CUDA/CuPy/ONNX Runtime load libraries and build caches. Please wait for the first [matting] or [offline-*] progress line.
[offline] env: decoder=simple batch=1 buffer=2 preset=P4 model=
[offline-alpha] input codec=hevc profile=Main 10 pix_fmt=yuv420p10le bit_depth=10 size=8192x4096
[offline-alpha] cuda gpu_id=0 name=NVIDIA GeForce RTX 4070 Laptop GPU cc=8.9 vram=8.0GB free=6.9GB driver=13020 runtime=12090
[offline-alpha] decoder=simple batch=1 buffer=2 owned_copy=False backend_reason=experimental SDR Main10/P016 route enabled
[offline] YOLO-World+EfficientSAM prepass subprocess=E:\VR_Video_Passthrough_Server\pt_core.exe tool offline_alpha_passthrough E:\VR_Video_Passthrough_Server\videos\nkwmr-018_2_8k.mp4 --engine matanyone2_onnx --model E:\VR_Video_Passthrough_Server\models\matanyone2_onnx_512_bs1 --matanyone2-size 512 --matanyone2-batch 1 --matanyone2-prepass yoloworld_efficientsam --ywes-model-dir E:\VR_Video_Passthrough_Server\models\yoloworld_efficientsam --ywes-txt-feats E:\VR_Video_Passthrough_Server\models\person_txt_feats.npy --ywes-provider cuda --ywes-yolo-model yolov8l-worldv2.onnx --ywes-sam-model efficientsam_s.onnx --ywes-yolo-size 1280 --ywes-score-threshold 0.03 --ywes-nms-threshold 0.6 --ywes-box-expand 0.08 --ywes-top-k 1 --ywes-scan hybrid --ywes-scan-interval-sec 1.0 --ywes-active-min-area-ratio 0.001 --ywes-gap-fill-frames 300 --matanyone2-segment-frames 300 --matanyone2-min-segment-sec 3.0 --frames 3596 --fps 59.94005994005994 --ywes-prepass-out E:\VR_Video_Passthrough_Server\debug_output\_ywes_prepass\ywes_prepass_1779170120049_2762912356768.npz
[offline-alpha] input codec=hevc profile=Main 10 pix_fmt=yuv420p10le bit_depth=10 size=8192x4096
[offline-alpha] cuda gpu_id=0 name=NVIDIA GeForce RTX 4070 Laptop GPU cc=8.9 vram=8.0GB free=6.9GB driver=13020 runtime=12090
[offline-alpha] decoder=simple batch=1 buffer=2 owned_copy=False backend_reason=experimental SDR Main10/P016 route enabled
2026-05-19 13:56:37.6899766 [Wnnxruntime:, transformer_memcpy.cc:111 onnxruntime::MemcpyTransformer::ApplyImpl] 4 Memcpy nodes are added to the graph main_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
2026-05-19 13:56:37.6908204 [Wnnxruntime:, transformer_memcpy.cc:111 onnxruntime::MemcpyTransformer::ApplyImpl] 1 Memcpy nodes are added to the graph sub_graph for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
2026-05-19 13:56:37.6914623 [Wnnxruntime:, transformer_memcpy.cc:111 onnxruntime::MemcpyTransformer::ApplyImpl] 2 Memcpy nodes are added to the graph sub_graph2 for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
2026-05-19 13:56:38.0443235 [Wnnxruntimeefault, scatter_nd.h:53 onnxruntime::cuda::ScatterNDWithAtomicReduction::ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
2026-05-19 13:56:38.0447332 [Wnnxruntimeefault, scatter_nd.h:53 onnxruntime::cuda::ScatterNDWithAtomicReduction::ScatterNDWithAtomicReduction] ScatterND with reduction=='none' only guarantees to be correct if indices are not duplicated.
[offline] YOLO-World+EfficientSAM loaded dir=E:\VR_Video_Passthrough_Server\models\yoloworld_efficientsam yolo=yolov8l-worldv2.onnx sam=efficientsam_s.onnx provider=['CUDAExecutionProvider', 'CPUExecutionProvider'] yolo_size=1280 score=0.03 nms=0.6 expand=0.08 top_k=1 txt_classes=5
[offline] YOLO-World+EfficientSAM prepass samples=60 scan=hybrid max_segment_frames=300
Traceback (most recent call last):
  File "main.py", line 350, in <module>
  File "main.py", line 132, in main
  File "main.py", line 67, in _run_legacy_tool
  File "tools\offline_alpha_passthrough.py", line 1558, in main
  File "offline\yoloworld_efficientsam.py", line 475, in precompute_segment_masks
cv2.error: OpenCV(4.13.0) D:\a\opencv-python\opencv-python\opencv\modules\imgproc\src\color.simd_helpers.hpp:94: error: (-2:Unspecified error) in function '__cdecl cv::impl::`anonymous-namespace'::CvtHelper<struct cv::impl::`anonymous namespace'::Set<1,-1,-1>,struct cv::impl::A0xabc0dcb0::Set<3,4,-1>,struct cv::impl::A0xabc0dcb0::Set<0,-1,-1>,1>::CvtHelper(const class cv::_InputArray &,const class cv::_OutputArray &,int)'
> Unsupported depth of input image:
>     'VDepth::contains(depth)'
> where
>     'depth' is 2 (CV_16U)

[PYI-45312:ERROR] Failed to execute script 'main' due to unhandled exception!
Traceback (most recent call last):
  File "main.py", line 350, in <module>
  File "main.py", line 132, in main
  File "main.py", line 67, in _run_legacy_tool
  File "tools\offline_alpha_passthrough.py", line 1558, in main
  File "offline\yoloworld_efficientsam.py", line 442, in precompute_segment_masks
  File "offline\yoloworld_efficientsam.py", line 617, in precompute_segment_masks_subprocess
  File "subprocess.py", line 571, in run
subprocess.CalledProcessError: Command '['E:\\VR_Video_Passthrough_Server\\pt_core.exe', 'tool', 'offline_alpha_passthrough', 'E:\\VR_Video_Passthrough_Server\\videos\\nkwmr-018_2_8k.mp4', '--engine', 'matanyone2_onnx', '--model', 'E:\\VR_Video_Passthrough_Server\\models\\matanyone2_onnx_512_bs1', '--matanyone2-size', '512', '--matanyone2-batch', '1', '--matanyone2-prepass', 'yoloworld_efficientsam', '--ywes-model-dir', 'E:\\VR_Video_Passthrough_Server\\models\\yoloworld_efficientsam', '--ywes-txt-feats', 'E:\\VR_Video_Passthrough_Server\\models\\person_txt_feats.npy', '--ywes-provider', 'cuda', '--ywes-yolo-model', 'yolov8l-worldv2.onnx', '--ywes-sam-model', 'efficientsam_s.onnx', '--ywes-yolo-size', '1280', '--ywes-score-threshold', '0.03', '--ywes-nms-threshold', '0.6', '--ywes-box-expand', '0.08', '--ywes-top-k', '1', '--ywes-scan', 'hybrid', '--ywes-scan-interval-sec', '1.0', '--ywes-active-min-area-ratio', '0.001', '--ywes-gap-fill-frames', '300', '--matanyone2-segment-frames', '300', '--matanyone2-min-segment-sec', '3.0', '--frames', '3596', '--fps', '59.94005994005994', '--ywes-prepass-out', 'E:\\VR_Video_Passthrough_Server\\debug_output\\_ywes_prepass\\ywes_prepass_1779170120049_2762912356768.npz']' returned non-zero exit status 1.
[PYI-39608:ERROR] Failed to execute script 'main' due to unhandled exception!

joe123

有个问题pico生成的多段透视文件，第一段只能播放不到十秒钟，后面的没问题。
还有个问题就是家里的环境光和视频的光线不太一样，如果给软件添加一个自定义环境光，不知道能不能实现

zelefans

5x5x5x4n 发表于 2026-5-19 13:59
4070 8G显存测试跑YOLOWorld报错
[offline] GPU runtime initialization may take a while on first use. I ...

视频源10bit输入的问题，其他模式下修了，这个新的模式是新增的 MatAnyone2 前置分支没有复用之前的 10-bit 转换逻辑。我测试的视频都是8bit的。我先下载同名视频测一下报错， 晚上打个补丁。

现在变成维护两个软件了，去马赛克哪里为啥突然用的人多了……晚上那里还要更新，正在测改动……

zelefans

joe123 发表于 2026-5-19 14:22
有个问题pico生成的多段透视文件，第一段只能播放不到十秒钟，后面的没问题。
还有个问题就是家里的环境光 ...

第一段问题更像是首次播放的内部缓存导致的，哪个播放器我可以测一下。按理说缓存问题我做过大量工作，比如前置垫片啥的……
环境光的问题我想了想，视频可以加滤镜，耗费成本不清楚，但这个怎么定义呢……我心里记一下，现在最关键的是我想去掉分段式的直播流，实现像普通文件那样拖动可以模拟定位

pwzdby

[offline] GPU runtime initialization may take a while on first use. If this is the first offline run after installation or a driver/model update, the window can stay quiet for 1-3 minutes while CUDA/ONNX Runtime caches are built.
[offline] run: D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server_v0.1.0-beta.3\pt_core.exe tool offline_alpha_passthrough D:\BaiduNetdiskDownload\75114_3840p.mp4 --engine rvm --out D:\BaiduNetdiskDownload\75114_3840p_rvm1_S000000_ALL_FISHEYE_alpha.mp4 --audio copy --model D:\BaiduNetdiskDownload\VR_Video_Passthrough_Server_v0.1.0-beta.3\models\rvm_mobilenetv3_fp32.onnx --alpha-stride 1 --input-size 1024 --sbs-batch --bitrate live --preset P7 --cq -1
[offline] gpu warmup: state=cold reason=marker_missing eta=45s gpu=NVIDIA GeForce RTX 2060 cc=7.5 ort=1.25.1
[offline] gpu warmup: first offline run can appear idle while CUDA/CuPy/ONNX Runtime load libraries and build caches. Please wait for the first [matting] or [offline-*] progress line.
[offline] env: decoder=simple batch=1 buffer=2 preset=P7 model=
[offline-alpha] input codec=av1 profile=Main pix_fmt=yuv420p bit_depth=8 size=7680x3840
[offline-alpha] cuda gpu_id=0 name=NVIDIA GeForce RTX 2060 cc=7.5 vram=6.0GB free=5.0GB driver=13020 runtime=12090
[offline-alpha] decoder=simple batch=1 buffer=2 owned_copy=False backend_reason=8K+ output should use HEVC; H.264 NVENC failed in probe
[matting] CuPy 14.0.1 detected, GPU composite enabled.
[matting] options: input_size=1024 square=False split_sbs=True sbs_batch=True alpha_stride=1 alpha_mode=reuse alpha_cutoff=0.0 alpha_hard_edge=False alpha_contrast=1.0
2026-05-19 14:50:08.4243754 [Wnnxruntime:, transformer_memcpy.cc:111 onnxruntime::MemcpyTransformer::ApplyImpl] 1 Memcpy nodes are added to the graph torch-jit-export for CUDAExecutionProvider. It might have negative impact on performance (including unable to run CUDA graph). Set session_options.log_severity_level=1 to see the detail logs before this message.
[matting] model kind=rvm input=src shape=['batch_size', 3, 'height', 'width'] type=tensor(float) batch2=True model_batch2=True rvm_iobinding=True providers=['CUDAExecutionProvider', 'CPUExecutionProvider'] inputs=['src', 'r1i', 'r2i', 'r3i', 'r4i', 'downsample_ratio'] outputs=['fgr', 'pha', 'r1o', 'r2o', 'r3o', 'r4o']
[offline-alpha] src=D:\BaiduNetdiskDownload\75114_3840p.mp4 out=D:\BaiduNetdiskDownload\75114_3840p_rvm1_S000000_ALL_FISHEYE_alpha.mp4 engine=rvm 7680x3840 output=7680x3840 source_fps=60.000000 output_fps=60.000000 target=11879 audio=copy bitrate=33280218 maxbitrate=39936261 vbvbufsize=66560436 rc=vbr cq=-1 preset=P7
[offline-alpha] projection=sbs half-equirect 180 -> fisheye sbs radius_scale=1
[offline-alpha] packing=DeoVR-style red-channel alpha blocks scale=0.4 layout=alpha-packer-6block
[offline] mux=.\ffmpeg.EXE -hide_banner -loglevel warning -y -fflags +genpts -f hevc -framerate 60.000000 -i - -map 0:v:0 -c:v copy -an -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -movflags +faststart -f mp4 D:\BaiduNetdiskDownload\75114_3840p_rvm1_S000000_ALL_FISHEYE_alpha._video_only.mp4
Traceback (most recent call last):
  File "main.py", line 350, in <module>
  File "main.py", line 132, in main
  File "main.py", line 67, in _run_legacy_tool
  File "tools\offline_alpha_passthrough.py", line 1633, in main
  File "pipeline\pynv_io.py", line 292, in frame_at
  File "yNvVideoCodec\decoders\SimpleDecoder.py", line 86, in __getitem__
_PyNvVideoCodec.PyNvVCExceptionUnsupported: NvDecoder::HandleVideoSequence :
Error code : 801
Error Type : Codec not supported on this GPU
at G:\build\sw\devrel\PyNvVideoCodec\rel_2_1\src\VideoCodecSDKUtils\helper_classes\NvCodec\NvDecoder\NvDecoder.cpp:260

[PYI-25816:ERROR] Failed to execute script 'main' due to unhandled exception!
这些视频还是离线和实时不了

joe123

zelefans 发表于 2026-5-19 14:39
第一段问题更像是首次播放的内部缓存导致的，哪个播放器我可以测一下。按理说缓存问题我做过大量工作，比 ...

4xvr，玄学，有时候退出重新打开就正常了，不影响