# nanobsd-alike for Linux?



## Bobi B. (Sep 7, 2018)

Greetings, All! Can someone point me in the right direction: I'm looking for a way to build a nanobsd-alike install image for Linux.

My goal is to prepare a pre-built Linux environment for GPU video transcoding since, sadly, Nvidia chose not to support FreeBSD for CUDA toolkit. Therefore I'll need to pick either of Nvidia-supported Linux distributions and make an image that includes CUDA runtime, ffmpeg and a handful of packages for management & monitoring, able to run from a read-only root filesystem with support for /etc persistency. Most tutorials are about how to make a bootable Linux from install media, which is not what I seek. Can anyone point me in the right direction or maybe propose better search terms? Thanks in advance!

PS: Nvidia supports OpenSUSE, CentOS and Ubuntu (plus few commercial distributions).


----------



## SirDice (Sep 10, 2018)

Bobi B. said:


> I'm looking for a way to build a nanobsd-alike install image for Linux.


Ask on a Linux forum.

Rule #7: FreeBSD Forums Rules


----------



## kpedersen (Sep 10, 2018)

Are you able to work with OpenCL instead? Then Intel and AMDs GPUs should be fine on FreeBSD.

If you are stuck with Nvidia, a fellow user might have a workaround for an ICD (on the UE4 thread):



shkhln said:


> Relevant bits: ICD loader and Mesa patches. Nvidia doesn't provide a native ICD for FreeBSD at the moment and it's not clear whether they are planning to do it; there is a possible workaround (more on that later).



I wonder if this is able to work with OpenCL or is it Vulkan specific?


----------



## shkhln (Sep 10, 2018)

kpedersen said:


> I wonder if this is able to work with OpenCL or is it Vulkan specific?



OpenCL is likely implemented on top of CUDA, so I suppose it wouldn't work without _nvidia-uvm_ kernel module. Worth a try though.


----------



## Bobi B. (Sep 11, 2018)

Nvidia implements video decoding and encoding in hardware, same goes for deinterlacing and video scaling. A low-level Quadro can handle real-time adaptive encoding of several HD channels with ease, all for less than USD 500 / 75 Watts. Not sure if Intel/AMD have something similar, but frankly, I haven't looked into it, either.

I was thinking about using PXE/TFTP/NFS for Linuxes. Even if it doesn't work, being able to boot Linux from a read-only NFS is a half win.


----------



## shkhln (Sep 11, 2018)

Bobi B. said:


> Nvidia implements video decoding and encoding in hardware, same goes for deinterlacing and video scaling.



Are you saying you don't actually need CUDA?


----------



## SirDice (Sep 11, 2018)

Bobi B. said:


> Nvidia implements video decoding and encoding in hardware, same goes for deinterlacing and video scaling.


Isn't this possible with multimedia/libvdpau or multimedia/libvdpau-va-gl?


----------



## Bobi B. (Sep 11, 2018)

shkhln said:


> Are you saying you don't actually need CUDA?


Not sure. Compiling hardware-accelerated ffmpeg requires Nvidia's CUDA toolkit (seems to depend on Video Codec SDK) plus plenty of X11; currently 12 Gigs are used on an Ubuntu build machine. Only runtime support for that ffmpeg (without development packages) weights 3,4 GB. Its a bit hard to tell what is used and what is not. And, by some reason, installing Nvidia runtime installs half of X11, as well. As far as I understood, hardware acceleration uses Nvidia's NVDEC, NVENC and some whatever-it-is-called GPU core being able to accelerate some video filters.



SirDice said:


> Isn't this possible with multimedia/libvdpau or multimedia/libvdpau-va-gl?


Seems that VDPAU takes care for decoding only. But it is encoding that needs a lot of resources. And being a transcoding server, there is really no need to display the video anywhere.


----------



## shkhln (Sep 12, 2018)

Bobi B. said:


> Not sure. Compiling hardware-accelerated ffmpeg requires Nvidia's CUDA toolkit



I think FFmpeg only requires https://github.com/FFmpeg/nv-codec-headers.



Bobi B. said:


> As far as I understood, hardware acceleration uses Nvidia's NVDEC, NVENC and some whatever-it-is-called GPU core being able to accelerate some video filters.



My point is that CUDA is a relatively general purpose programming toolkit and NVENC appears to make use of some fixed-function hardware, a use case where CUDA wouldn't be needed. This doesn't necessarily mean NVENC isn't _partially_ implemented in CUDA, hence the question.

FFmpeg at least calls cuInit in libavcodec/nvenc.c.


----------



## Bobi B. (Sep 12, 2018)

shkhln said:


> I think FFmpeg only requires https://github.com/FFmpeg/nv-codec-headers.


Seems like this takes care for loading CUDA runtime dynamically, in run-time. Ubuntu's ffmpeg is not linked against CUDA runtime, only libnppig.so.9.2 and libnppicc.so.9.2; loads libcuda.so.1, libnvcuvid.so.1 and libnvidia-encode.so.1 dynamically with dlopen(3), however...



shkhln said:


> My point is that CUDA is a relatively general purpose programming toolkit and NVENC appears to make use of some fixed-function hardware, a use case where CUDA wouldn't be needed. This doesn't necessarily mean NVENC isn't _partially_ implemented in CUDA, hence the question.
> 
> FFmpeg at least calls cuInit in libavcodec/nvenc.c.


... seems like x11/nvidia-driver takes care for OpenGL only and contains just libcuda.so.1.


----------



## shkhln (Mar 27, 2020)

I think I'll just throw this link here without any explanation: https://github.com/shkhln/nvshim/issues/1#issuecomment-600358438. Make what you will of it.


----------

