# Laptop with AMD 'Cezanne' graphics + Nvidia RTX 3060



## yha (Mar 2, 2022)

So I have this Acer Nitro 5 running FreeBSD 13.0-RELEASE (fresh install), and I'm wondering if it's possible to get nvidia-driver to work on it. The BIOS has no option to disable the onboard graphics. If I startx with nvidia.ko loaded and 'device "nvidida"' in the config, I get "no screens found". If I load 'amdgpu.ko' and remove the 'device' config, then X works just fine. Is it possible to get the nvidia driver working? Also, I apologize for my ignorance, but if I were to use the integrated graphics, I'm a bit confused as to whether I need the xf86-video-ati pkg as well (in addition to the loaded kernel module). I obviously have X working without xf86-video-ati, but will it make any difference if I install it?

One side question, out of curiosity, is there any architectural barrier in X11's design that prevents the use of multi-GPU setups? You always hear about how you can't get the dedicated GPU to work on Optimus systems, but what's the technical reason behind it if the manufacturer (Nvidia in this case) is releasing drivers for it (even if it's a blob)?

See attachments.

My touchpad/mouse isn't working either, but I'll start another thread for that once I have the "correct" graphics setup.


----------



## Phishfry (Mar 2, 2022)

yha said:


> but what's the technical reason behind it if the manufacturer (Nvidia in this case) is releasing drivers for it (even if it's a blob)?


Windows garbage. Probably some backroom skullduggery to make sure it never works on anything but Windows.
Optimus was a Windows thing.


----------



## shkhln (Mar 2, 2022)

Can't you just google it? Both of you.


----------



## skunk (Mar 2, 2022)

yha said:


> If I startx with nvidia.ko loaded and 'device "nvidida"' in the config, I get "no screens found".


You need to turn the switcheroo muxer first.
Check /var/run/dmesg.boot. I think the amd is the boot vga, right?



yha said:


> I'm a bit confused as to whether I need the xf86-video-ati pkg as well (in addition to the loaded kernel module). I obviously have X working without xf86-video-ati, but will it make any difference if I install it?


Yes.
The majority of the driver options for xorg.conf are handled by the ati driver, have no effect if only using the amdgpu kms driver.



yha said:


> One side question, out of curiosity, is there any architectural barrier in X11's design that prevents the use of multi-GPU setups? You always hear about how you can't get the dedicated GPU to work on Optimus systems, but what's the technical reason behind it if the manufacturer (Nvidia in this case) is releasing drivers for it (even if it's a blob)?


No. It depends on the hardware (offloader or switcheroo). Both are different.
Multi-GPU (by the same manufacturer) is not supported by AMD, only by Nvidia. But multiple Nvidia graphics cards obviously are no laptop issue.



yha said:


> My touchpad/mouse isn't working either, but I'll start another thread for that once I have the "correct" graphics setup.


The multitude of issues with that lappy is impressive.



shkhln said:


> Can't you just google it? Both of you.


I love the Russians' bitchiness bluntness.


----------



## shkhln (Mar 2, 2022)

Too busy doomscrolling nowadays…


----------



## monwarez (Mar 3, 2022)

Looking at your Xorg logs, neither amd or nvidia gpus are working (you are using scfb which is software rendering).
You could try to use this config (still software rendering, but you may be able to use the nvidia driver with prime)

```
Section "ServerLayout"
  Identifier "layout"
  Screen     0 "iGPU"
  Screen     1 "dGPU"
EndSection

Section "Device"
  Identifier "iGPU"
  Driver     "scfb"
  BusID      "PCI:6:0:0"
EndSection

Section "Device"
  Identifier "dGPU"
  Driver     "nvidia"
  BusID      "PCI:1:0:0"
EndSection

Section "Screen"
  Identifier "iGPU"
  Device     "iGPU"
EndSection

Section "Screen"
  Identifier "dGPU"
  Device     "dGPU"
  Option  "UseDisplayDevice" "None"
EndSection
```
And of course try it with the 510.47.03 version of nvidia driver that you can get the patch here
PR 261666


----------



## checkpoint (Mar 12, 2022)

Hello guys. 

I have a laptop with similar setup (Lenovo IP 3), AMD Cezanne + nVidia RTX 3050. Whatever I tried I could not get Xorg working neither with internal AMD nor with nVidia graphics, except using "scfb" driver.

If I load nvidia modules and configure X to use "nvidia" driver, the X stucks with a black screen (and white cursor in the corner) during mode switch. Tried it with nvidia-driver 470. The latest 510.54 driver make Xorg 1.20.13 to crash. Proposed patch does not solve this issue.


If I unload `nvidia` and `nvidia-modeset` modules and try loading `amdgpu` instead, it seems the latter does not load properly. I can see only this line in dmesg and no GPU specific modules are loaded.


```
[drm] amdgpu kernel modesetting enabled.
```

Starting X this way gets me to:


```
(II) AMDGPU(0): [KMS] drm report modesetting isn't supported.
(WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
(EE) Screen 0 deleted because of no matching config section.
(II) UnloadModule: "amdgpu"
(EE) Device(s) detected, but none match those in the config file.
(EE)
Fatal server error:
(EE) no screens found(EE)
```

Tried with graphics/drm-fbsd13-kmod.

What do I do wrong with `amdgpu` and how to get any of the graphics to work ?


----------



## grahamperrin@ (Mar 12, 2022)

checkpoint said:


> … AMD Cezanne …



I'm not familiar with Cezanne, is there a more specific identification? 

<https://bsd-hardware.info/?id=pci:1002-1638-1002-1636> not hopeful, for FreeBSD. 

<https://bsd-hardware.info/?probe=d77aae8064&d=OpenBSD#pci:1002-1638-1002-1636> shows AMD Cezanne working with `amdgpu` with OpenBSD.


----------



## checkpoint (Mar 12, 2022)

grahamperrin said:


> I'm not familiar with Cezanne, is there a more specific identification?


Here's what `pciconf -l` shows:


```
vgapci1@pci0:5:0:0:     class=0x030000 rev=0xc6 hdr=0x00 vendor=0x1002 device=0x1638 subvendor
=0x17aa subdevice=0x3a5d
```

I compiled and installed graphics/drm-fbsd13-kmod port (which is drm-kmod 5.4-stable). Then I did `kldload amdgpu` and could see it did not instantiate and did not upload firmware. I read somewhere that my device_id=0x1638 could be missing in amdgpu_drv.c and it really was, so I added it manually, recompiled and got this:


```
This hardware requires experimental hardware support.
                         See modparam exp_hw_support.
```

I edited amdgpu_drv.c again to set amdgpu_exp_hw_support to 1. Now it does load firmware successfully, I could see lots of messages from amdgpu drivers...


```
[drm] amdgpu kernel modesetting enabled.
drmn1: <drmn> on vgapci1
VT: Replacing driver "efifb" with new "dummy".
vgapci1: child drmn1 requested pci_enable_io
vgapci1: child drmn1 requested pci_enable_io
sysctl_warn_reuse: can't re-use a leaf (hw.dri.debug)!
[drm] initializing kernel modesetting (RENOIR 0x1002:0x1638 0x17AA:0x3A5D 0xC6).
[drm] register mmio base: 0xD1400000
[drm] register mmio size: 524288
[drm] add ip block number 0 <soc15_common>
[drm] add ip block number 1 <gmc_v9_0>
[drm] add ip block number 2 <vega10_ih>
[drm] add ip block number 3 <psp>
[drm] add ip block number 4 <smu>
[drm] add ip block number 5 <gfx_v9_0>
[drm] add ip block number 6 <sdma_v4_0>
[drm] add ip block number 7 <vcn_v2_0>
drmn1: successfully loaded firmware image 'amdgpu/renoir_gpu_info.bin'
drmn1: kfd not supported on this ASIC

ATOM BIOS: 113-CEZANNE-018
drmn1: successfully loaded firmware image 'amdgpu/renoir_sdma.bin'
[drm] VCN decode is enabled in VM mode
[drm] VCN encode is enabled in VM mode
[drm] VCN jpeg decode is enabled in VM mode
[drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
drmn1: VRAM: 4096M 0x000000F400000000 - 0x000000F4FFFFFFFF (4096M used)
drmn1: GART: 1024M 0x0000000000000000 - 0x000000003FFFFFFF
drmn1: AGP: 267419648M 0x000000F800000000 - 0x0000FFFFFFFFFFFF
Successfully added WC MTRR for [0xfc10000000-0xfc1fffffff]: 0;
[drm] Detected VRAM RAM=4096M, BAR=256M
[drm] RAM width 128bits UNKNOWN
[TTM] Zone  kernel: Available graphics memory: 10393078 KiB
[TTM] Zone   dma32: Available graphics memory: 2097152 KiB
[TTM] Initializing pool allocator
[drm] amdgpu: 4096M of VRAM memory ready
[drm] amdgpu: 4096M of GTT memory ready.
[drm] GART: num cpu pages 262144, num gpu pages 262144
[drm] PCIE GART of 1024M enabled (table at 0x000000F400900000).
[drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[drm] Driver supports precise vblank timestamp query.
drmn1: successfully loaded firmware image 'amdgpu/renoir_asd.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_pfp.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_me.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_ce.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_rlc.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_mec.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_mec2.bin'
[drm] use_doorbell being set to: [true]
drmn1: successfully loaded firmware image 'amdgpu/renoir_vcn.bin'
[drm] Found VCN firmware Version ENC: 1.7 DEC: 4 VEP: 0 Revision: 17
[drm] PSP loading VCN firmware
sos fw version = 0x110068.
[drm] reserve 0x400000 from 0xf4ff800000 for PSP TMR
amdgpu: [powerplay] smu driver if version = 0x0000000a, smu fw if version = 0x0000000e, smu fw version = 0x00402d00 (64.45.0)
amdgpu: [powerplay] SMU driver if version not matched
amdgpu: [powerplay] dpm has been disabled
amdgpu: [powerplay] SMU is initialized successfully!
[drm] VCN decode and encode initialized successfully(under DPG Mode).
drmn1: ring gfx uses VM inv eng 0 on hub 0
drmn1: ring comp_1.0.0 uses VM inv eng 1 on hub 0
drmn1: ring comp_1.1.0 uses VM inv eng 4 on hub 0
drmn1: ring comp_1.2.0 uses VM inv eng 5 on hub 0
drmn1: ring comp_1.3.0 uses VM inv eng 6 on hub 0
drmn1: ring comp_1.0.1 uses VM inv eng 7 on hub 0
drmn1: ring comp_1.1.1 uses VM inv eng 8 on hub 0
drmn1: ring comp_1.2.1 uses VM inv eng 9 on hub 0
drmn1: ring comp_1.3.1 uses VM inv eng 10 on hub 0
drmn1: ring kiq_2.1.0 uses VM inv eng 11 on hub 0
drmn1: ring sdma0 uses VM inv eng 0 on hub 1
drmn1: ring vcn_dec uses VM inv eng 1 on hub 1
drmn1: ring vcn_enc0 uses VM inv eng 4 on hub 1
drmn1: ring vcn_enc1 uses VM inv eng 5 on hub 1
drmn1: ring vcn_jpeg uses VM inv eng 6 on hub 1
[drm] Initialized amdgpu 3.35.0 20150101 for drmn1 on minor 0
```

Yet it does not work. System goes instant reboot soon as Xorg accesses the driver (not panic).

So I conclude that drm-kmod 5.4-stable is NOT capable. I `git clone` latest drm-kmod from Github and am trying to build it now. I'm facing lots of "file not found" compile errors related to linux kernel includes. How do they build drm-kmod ? I have freebsd src installed, what else do I need ?

I'm brain-fscking with this amdgpu issue for almost a week now. I hope I could get it resolved, I definitely do not want Linux on my laptop, which works "out of the box", darn!  :-(


----------



## grahamperrin@ (Mar 13, 2022)

checkpoint said:


> … I `git clone` latest drm-kmod from Github and am trying to build …



Easier to *build from ports* i.e. /usr/ports, have you done so before?

If not – and if you have not previously fetched the ports collection:

use bectl(8) to create, activate then boot a new boot environment
uninstall any installed DRM
`mkdir -p /usr/local/etc/pkg/repos`
`ee /usr/local/etc/pkg/repos/FreeBSD.conf`
put the content below in the file, save the file
`pkg upgrade`
`pkg install --yes gitup`
`gitup ports`
`grep releng /usr/local/etc/gitup.conf`
edit the file if necessary
`gitup release`
`cd /usr/ports/graphics/drm-devel-kmod`
`make install clean`
`kldload amdgpu`
… something like that, I'm writing quite lazily at the moment.

If the module will not load with FreeBSD 13.0-RELEASE-p7, then (edit gitup.conf et cetera and) try with 13.1-BETA1.


```
FreeBSD: {
    url: "http://pkg0.bme.freebsd.org/${ABI}/latest",
    priority: 3
}
```

Afterthought: if you're confident with git-clone(1) etc., then you can use git commands in lieu of gitup.


----------



## Alexander88207 (Mar 13, 2022)

AMD Cezanne got support around 5.14, our drm drivers are currently at 5.4-5.7.


----------



## astyle (Mar 13, 2022)

According to the FreeBSD Graphics wiki page, you can try either VESA or SCFB if you're struggling with GPU-specific drivers like `amdgpu`. From personal experience (albeit back in 2017 with PC-BSD, which used `scfb` by default), I can tell you that it works pretty well. YMMV, though.


----------



## grahamperrin@ (Mar 13, 2022)

Alexander88207 said:


> AMD Cezanne got support around 5.14,



Thanks, that seems to fit with the mention of Cezanne at Linux 5.14-rc7 [LWN.net] (2021-08-22) 

Postscript: please see below, the comment from checkpoint.



Alexander88207 said:


> our drm drivers are currently at 5.4-5.7.



No timeline, but <https://gitter.im/FreeBSDDesktop/Lobby/archives/2022/02/18?at=620fda5f4164105ab0a48254> mentions 5.10 and the plan to move most things to base.


----------



## Alexander88207 (Mar 13, 2022)

grahamperrin said:


> but <https://gitter.im/FreeBSDDesktop/Lobby/archives/2022/02/18?at=620fda5f4164105ab0a48254> mentions 5.10 and the plan to move most things to base.



Damn. Thanks for the information. My new card got support at 5.11


----------



## grahamperrin@ (Mar 13, 2022)

Alexander88207 said:


> 𡀦… My new card got support at 5.11








… was there recently a case where someone found a modern card working with something _lower_ than the 'expected' version of Linux DRM? 

(<https://forums.freebsd.org/posts/559136> came to mind, but there's no mention of DRM versions.)



grahamperrin said:


> No timeline,



Correction: <https://forums.freebsd.org/posts/541407> reminds me of the timeline that was envisaged last year.


----------



## checkpoint (Mar 13, 2022)

Alexander88207 said:


> AMD Cezanne got support around 5.14, our drm drivers are currently at 5.4-5.7.



That's not true, Cezanne support was backported to 5.7 in Linux. I have built and installed drm-kmod 5.7.19 successfully from 13.1-BETA1 ports, it has device_id 0x1638 listed. The amdgpu driver now loads ok on my laptop, see dmesg below. But I stumbled against another problem: Xorg 1.20.13 just segfaults when trying to access amdgpu driver. I have posted some info here: https://github.com/freebsd/drm-kmod/issues/118


```
[drm] amdgpu kernel modesetting enabled.
drmn1: <drmn> on vgapci1
vgapci1: child drmn1 requested pci_enable_io
vgapci1: child drmn1 requested pci_enable_io
[drm] initializing kernel modesetting (RENOIR 0x1002:0x1638 0x17AA:0x3A5D 0xC6).
[drm] register mmio base: 0xD1400000
[drm] register mmio size: 524288
[drm] add ip block number 0 <soc15_common>
[drm] add ip block number 1 <gmc_v9_0>
[drm] add ip block number 2 <vega10_ih>
[drm] add ip block number 3 <psp>
[drm] add ip block number 4 <smu>
[drm] add ip block number 5 <gfx_v9_0>
[drm] add ip block number 6 <sdma_v4_0>
[drm] add ip block number 7 <dm>
[drm] add ip block number 8 <vcn_v2_0>
[drm] add ip block number 9 <jpeg_v2_0>
drmn1: successfully loaded firmware image 'amdgpu/renoir_gpu_info.bin'
ATOM BIOS: 113-CEZANNE-018
drmn1: successfully loaded firmware image 'amdgpu/renoir_sdma.bin'
[drm] VCN decode is enabled in VM mode
[drm] VCN encode is enabled in VM mode
[drm] JPEG decode is enabled in VM mode
[drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
drmn1: VRAM: 4096M 0x000000F400000000 - 0x000000F4FFFFFFFF (4096M used)
drmn1: GART: 1024M 0x0000000000000000 - 0x000000003FFFFFFF
drmn1: AGP: 267419648M 0x000000F800000000 - 0x0000FFFFFFFFFFFF
[drm] Detected VRAM RAM=4096M, BAR=4096M
[drm] RAM width 128bits DDR4
[TTM] Zone  kernel: Available graphics memory: 10393088 KiB
[TTM] Zone   dma32: Available graphics memory: 2097152 KiB
[TTM] Initializing pool allocator
iwlwifi0: base HW address: 80:38:fb:dc:30:1b
[drm] amdgpu: 4096M of VRAM memory ready
[drm] amdgpu: 4096M of GTT memory ready.
[drm] GART: num cpu pages 262144, num gpu pages 262144
[drm] PCIE GART of 1024M enabled (table at 0x000000F400900000).
drmn1: successfully loaded firmware image 'amdgpu/renoir_asd.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_pfp.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_me.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_ce.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_rlc.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_mec.bin'
drmn1: successfully loaded firmware image 'amdgpu/renoir_mec2.bin'
[drm] use_doorbell being set to: [true]
drmn1: successfully loaded firmware image 'amdgpu/renoir_dmcub.bin'
[drm] Loading DMUB firmware via PSP: version=0x00000000
drmn1: successfully loaded firmware image 'amdgpu/renoir_vcn.bin'
[drm] Found VCN firmware Version ENC: 1.7 DEC: 4 VEP: 0 Revision: 17
[drm] PSP loading VCN firmware
sos fw version = 0x110068.
[drm] reserve 0x400000 from 0xf4ff800000 for PSP TMR
ugen0.3: <Azurewave Integrated Camera> at usbus0
smu driver if version = 0x0000000b, smu fw if version = 0x0000000e, smu fw version = 0x00402d00 (64.45.0)
SMU driver if version not matched
SMU is initialized successfully!
[drm] kiq ring mec 2 pipe 1 q 0
[drm] Display Core initialized with v3.2.76!
[drm] DMUB hardware initialized: version=0x01020003
[drm] Connector eDP-1: get mode from tunables:
[drm]   - kern.vt.fb.modes.eDP-1
[drm]   - kern.vt.fb.default_mode
[drm] Connector HDMI-A-1: get mode from tunables:
[drm]   - kern.vt.fb.modes.HDMI-A-1
[drm]   - kern.vt.fb.default_mode
[drm] Supports vblank timestamp caching Rev 2 (21.10.2013).
[drm] VCN decode and encode initialized successfully(under DPG Mode).
[drm] JPEG decode initialized successfully.
[drm] fb mappable at 0x530C8D000
[drm] vram apper at 0x530000000
[drm] size 8294400
[drm] fb depth is 24
[drm]    pitch is 7680
VT: Replacing driver "efifb" with new "fb".
start FB_INFO:
type=11 height=1080 width=1920 depth=32
pbase=0x530c8d000 vbase=0xfffffe012028d000
name=drmn1 flags=0x0 stride=7680 bpp=32
end FB_INFO
drmn1: ring gfx uses VM inv eng 0 on hub 0
drmn1: ring comp_1.0.0 uses VM inv eng 1 on hub 0
drmn1: ring comp_1.1.0 uses VM inv eng 4 on hub 0
drmn1: ring comp_1.2.0 uses VM inv eng 5 on hub 0
drmn1: ring comp_1.3.0 uses VM inv eng 6 on hub 0
drmn1: ring comp_1.0.1 uses VM inv eng 7 on hub 0
drmn1: ring comp_1.1.1 uses VM inv eng 8 on hub 0
drmn1: ring comp_1.2.1 uses VM inv eng 9 on hub 0
drmn1: ring comp_1.3.1 uses VM inv eng 10 on hub 0
drmn1: ring kiq_2.1.0 uses VM inv eng 11 on hub 0
drmn1: ring sdma0 uses VM inv eng 0 on hub 1
drmn1: ring vcn_dec uses VM inv eng 1 on hub 1
drmn1: ring vcn_enc0 uses VM inv eng 4 on hub 1
drmn1: ring vcn_enc1 uses VM inv eng 5 on hub 1
drmn1: ring jpeg_dec uses VM inv eng 6 on hub 1
sysctl_warn_reuse: can't re-use a leaf (hw.dri.debug)!
[drm] Initialized amdgpu 3.37.0 20150101 for drmn1 on minor 0
```


----------



## checkpoint (Mar 13, 2022)

grahamperrin said:


> Easier to *build from ports* i.e. /usr/ports, have you done so before?


Neither graphics/drm-kmod nor graphics/drm-devel-kmod can be built on 13.0-REALEASE, `make` reports some missing kernel support. I could built graphics/drm-kmod successfully only after I upgrade to 13.1-BETA1.

Now another issue, packages in 13.1-BETA1 are three-to-four months lag behind those in 13.0-RELEASE. WTF ?


----------



## astyle (Mar 13, 2022)

checkpoint said:


> Neither graphics/drm-kmod nor graphics/drm-devel-kmod can be built on 13.0-REALEASE, `make` reports some missing kernel support. I could built graphics/drm-kmod successfully only after I upgrade to 13.1-BETA1.
> 
> Now another issue, packages in 13.1-BETA1 are three-to-four months lag behind those in 13.0-RELEASE. WTF ?


That's probably because you needed to install kernel sources (src.txz) when you installed the system. If you don't install kernel sources, then graphics/drm-kmod will fail to build. I'd know, I'm running 13.0-RELEASE with graphics/drm-kmod and `amdgpu`.


----------



## checkpoint (Mar 13, 2022)

astyle said:


> According to the FreeBSD Graphics wiki page, you can try either VESA or SCFB if you're struggling with GPU-specific drivers like `amdgpu`. From personal experience (albeit back in 2017 with PC-BSD, which used `scfb` by default), I can tell you that it works pretty well. YMMV, though.



Yes, I can run Xorg with scfb or modesetting drivers quite ok, but I need hardware GLE because I'm heavy KiCAD and FreeCAD user, KiCAD since 6.0.2 does not run at all if there's no GLE support. That was my aim or purchasing this new laptop.


----------



## checkpoint (Mar 13, 2022)

astyle said:


> That's probably because you needed to install kernel sources (src.txz) when you installed the system.



I'm FeeBSD user since around '96, I'm used to always install kernel source codes along with usr.bin src same time I install FreeBSD on a new machine.


----------



## astyle (Mar 13, 2022)

checkpoint said:


> I'm FeeBSD user since around '96, I'm used to always install kernel source codes along with usr.bin src same time I install FreeBSD on a new machine.


FreeBSD plays decent with Lenovo laptops, but after all this time, it's still a hit and miss, more miss with newer laptops. I'd suggest building a PC from aftermarket components, then you can do your homework on what's actually compatible. Laptops are not that flexible with hardware. I would know, I have a few.


----------



## grahamperrin@ (Mar 13, 2022)

checkpoint said:


> … `make` reports some missing kernel support. …



koobs in IRC drew attention to a tweet by Kevin *fully_patched* Sweeney


----------



## checkpoint (Mar 13, 2022)

astyle said:


> Laptops are not that flexible with hardware. I would know, I have a few.


You are absolutely right. I have had to switch from Mediatek WiFi adapter to Intel AX200, thankfully it's M.2 formfactor. Till now, GPU issue is the only left unresolved on this laptop. And since I upgraded to 13.1-BETA1 it suspends/resumes perfectly! 

Just in case someone is interested, this Lenovo IP 3 equipped with I2C-HID Touchpad made by ELAN. To make it work one has to put both `ig4` and `iichid` modules in kld_list and configure `evdev` input in Xorg.conf.


----------



## bmeneg (Jul 22, 2022)

Hi all,

Any news since March?
I bought a brand new Asus laptop with RTX 3050 + AMD Cezzane which, unfortunately, has no option to turn off the onboard card, and even with the latest NVidia driver (from its own website - I also tested with the ports version) is segfaulting during the start:


```
(==) Log file: "/var/log/Xorg.0.log", Time: Fri Jul 22 17:59:42 2022
(==) Using config file: "/etc/X11/xorg.conf"
(==) Using system config directory "/usr/local/share/X11/xorg.conf.d"
(EE)
(EE) Backtrace:
(EE) 0: /usr/local/bin/Xorg (?+0x0) [0x41e2aa]
(EE) 1: /lib/libthr.so.3 (_pthread_sigmask+0x54e) [0x82961258e]
(EE) 2: /lib/libthr.so.3 (_pthread_setschedparam+0x82f) [0x829611b3f]
(EE) 3: ? (?+0x0) [0x7ffffffff8a3]
(EE) 4: ? (?+0x0) [0x0]
(EE) 5: /usr/local/lib/xorg/modules/drivers/nvidia_drv.so (nvidiaUnlock+0x4201f) [0x83c6a8c1f]
(EE) 6: ? (?+0x0) [0x0]
(EE) unw_step failed: unspecified (general) error [-1]
(EE)
(EE) Segmentation fault at address 0x0
(EE)
Fatal server error:
(EE) Caught signal 11 (Segmentation fault). Server aborting
(EE)
(EE)
```


----------



## checkpoint (Jul 28, 2022)

Hi bmeneg,

Yes it now works fine on Cezanne. You have to git clone drm-kmod manually, switch to branch 5.10-lts, apply below simple patch to disable HDCP, build and install new modules.


```
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
index b0ee77ee8..58ba7bf60 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c
@@ -49,9 +49,14 @@ static int psp_v12_0_init_microcode(struct psp_context *psp)
 {
        struct amdgpu_device *adev = psp->adev;
        const char *chip_name;
+#ifdef __linux__
        char fw_name[30];
        int err = 0;
        const struct ta_firmware_header_v1_0 *ta_hdr;
+#elif defined(__FreeBSD__)
+       /* We do not support HDCP in drm-kmod yet */
+       int err = 0;
+#endif
        DRM_DEBUG("\n");
 
        switch (adev->asic_type) {
@@ -69,6 +74,7 @@ static int psp_v12_0_init_microcode(struct psp_context *psp)
        if (err)
                return err;
 
+#ifdef __linux__
        snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_ta.bin", chip_name);
        err = request_firmware(&adev->psp.ta_fw, fw_name, adev->dev);
        if (err) {
@@ -113,6 +119,7 @@ out:
                        "psp v12.0: Failed to load firmware \"%s\"\n",
                        fw_name);
        }
+#endif
 
        return err;
 }
```


----------



## checkpoint (Jul 28, 2022)

Still have no idea how to use nVidia GPU. PRIME support was added to Xorg 1.21 which is not supported on FreeBSD at the moment.


----------



## shkhln (Jul 28, 2022)

I'm not quite sure, but I think the Xorg's PRIME support has reached a reasonably complete state somewhere around 2014-2016: https://github.com/freedesktop/xorg-xserver/search?o=asc&q=PRIME&s=committer-date&type=commits.

Now, the offloading method that concerns us was added to the Nvidia's driver in 2019. There are two of them actually: http://download.nvidia.com/XFree86/Linux-x86_64/515.57/README/randr14.html and http://download.nvidia.com/XFree86/Linux-x86_64/515.57/README/primerenderoffload.html. The latter works, the former does not.


----------



## laufdi (Jul 30, 2022)

checkpoint said:


> Hi bmeneg,
> 
> Yes it now works fine on Cezanne. You have to git clone drm-kmod manually, switch to branch 5.10-lts, apply below simple patch to disable HDCP, build and install new modules.
> 
> ...



This patch was already committed on May 31st?

```
commit f7cc9dd58ca097a428ffcacad1686d970ca633a8 (tag: drm_v5.10.113_2)
Author: ivan-volnov <51086293+ivan-volnov@users.noreply.github.com>
Date:   Tue May 31 14:01:33 2022 +0800

    drm/amdgpu: Disable HDCP on green_sardine and renoir

    We do not support HDCP in drm-kmod yet
```


... I can load that module without crash and Xorg runs, but shows a colorful pattern and no content, and a working mousepointer ...
... I have a Renoir chip ...



... this patch is even included in the drm-510-kmod port (5.10.113_2)


----------



## elgrande (Jul 30, 2022)

checkpoint said:


> Still have no idea how to use nVidia GPU. PRIME support was added to Xorg 1.21 which is not supported on FreeBSD at the moment.


Have you tried x11/nvidia-hybrid-graphics?


----------



## checkpoint (Jul 31, 2022)

shkhln said:


> I'm not quite sure, but I think the Xorg's PRIME support has reached a reasonably complete state somewhere around 2014-2016: https://github.com/freedesktop/xorg-xserver/search?o=asc&q=PRIME&s=committer-date&type=commits.
> 
> Now, the offloading method that concerns us was added to the Nvidia's driver in 2019. There are two of them actually: http://download.nvidia.com/XFree86/Linux-x86_64/515.57/README/randr14.html and http://download.nvidia.com/XFree86/Linux-x86_64/515.57/README/primerenderoffload.html. The latter works, the former does not.


Just gave it one more try - none of these two methods works to me, I cannot get two providers listed. If I enable both GPUs in xorg.conf, it just crashes when accessing NVIDIA GPU.


----------



## checkpoint (Jul 31, 2022)

elgrande said:


> Have you tried x11/nvidia-hybrid-graphics?


Just tried. It does not work because secondary Xorg crashes soon as it accesses NVIDIA driver, see log below. I strongly believe all our problems running FreeBSD on such hybrid systems come from an outdated Xorg (1.20.14), it's simply too old and miss some important features. Couple of months ago I spent a lot of time investigating it and came to conclusion that we need at least Xorg 21.1 which incorporates all the necessary patches and bugfixes. I even tried to port it to FreeBSD, but failed dues lack of understanding how it's made and how it's build system works.


```
[    26.228]
X.Org X Server 1.20.14
X Protocol Version 11, Revision 0
[    26.228] Build Operating System: FreeBSD 13.1-RELEASE amd64
[    26.228] Current Operating System: FreeBSD butterfly 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd64
[    26.228] Build Date: 19 June 2022  12:55:14AM
[    26.228] 
[    26.228] Current version of pixman: 0.40.0
[    26.228]     Before reporting problems, check http://wiki.x.org
    to make sure that you have the latest version.
[    26.228] Markers: (--) probed, (**) from config file, (==) default setting,
    (++) from command line, (!!) notice, (II) informational,
    (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[    26.228] (==) Log file: "/var/log/Xorg.8.log", Time: Sun Jul 31 04:59:25 2022
[    26.228] (++) Using config file: "/var/cache/nvidia-headless/xorg.conf"
[    26.228] (EE) Unable to locate/open config directory: "xorg-nvidia-headless.conf.d"
[    26.228] (==) Using system config directory "/usr/local/share/X11/xorg.conf.d"
[    26.228] (==) ServerLayout "nvidia"
[    26.228] (**) |-->Screen "Screen0" (0)
[    26.228] (**) |   |-->Monitor "Monitor0"
[    26.229] (**) |   |-->Device "Device0"
[    26.229] (**) |-->Input Device "fake"
[    26.229] (**) Option "AutoAddDevices" "false"
[    26.229] (**) Not automatically adding devices
[    26.229] (==) Automatically enabling devices
[    26.229] (==) Not automatically adding GPU devices
[    26.229] (==) Max clients allowed: 256, resource mask: 0x1fffff
[    26.229] (==) FontPath set to:
    /usr/local/share/fonts/misc/,
    /usr/local/share/fonts/TTF/,
    /usr/local/share/fonts/OTF/,
    /usr/local/share/fonts/Type1/,
    /usr/local/share/fonts/100dpi/,
    /usr/local/share/fonts/75dpi/,
    catalogue:/usr/local/etc/X11/fontpath.d
[    26.229] (**) ModulePath set to "/usr/local/lib/xorg/modules-NVIDIA,/usr/local/lib/xorg/modules"
[    26.229] (II) Loader magic: 0x433270
[    26.229] (II) Module ABI versions:
[    26.229]     X.Org ANSI C Emulation: 0.4
[    26.229]     X.Org Video Driver: 24.1
[    26.229]     X.Org XInput driver : 24.1
[    26.229]     X.Org Server Extension : 10.0
[    26.229] (--) PCI: (1@0:0:0) 10de:25a2:17aa:3a5d rev 161, Mem @ 0xd0000000/16777216, 0xfb00000000/4294967296, 0xfc00000000/33554432, I/O @ 0x00003000/128
[    26.229] (--) PCI:*(5@0:0:0) 1002:1638:17aa:3a5d rev 198, Mem @ 0xfc10000000/268435456, 0xfc20000000/2097152, 0xd1400000/524288, I/O @ 0x00001000/256, BIOS @ 0x????????/65536
[    26.229] (WW) "efifb" will not be loaded unless you've specified it to be loaded elsewhere.
[    26.229] (II) "glx" will be loaded. This was enabled by default and also specified in the config file.
[    26.229] (II) LoadModule: "dri3"
[    26.229] (II) Module "dri3" already built-in
[    26.229] (II) LoadModule: "glx"
[    26.229] (II) Loading /usr/local/lib/xorg/modules/extensions/libglx.so
[    26.229] (II) Module glx: vendor="X.Org Foundation"
[    26.229]     compiled for 1.20.14, module version = 1.0.0
[    26.229]     ABI class: X.Org Server Extension, version 10.0
[    26.229] (II) LoadModule: "nvidia"
[    26.229] (II) Loading /usr/local/lib/xorg/modules/drivers/nvidia_drv.so
[    26.229] (II) Module nvidia: vendor="NVIDIA Corporation"
[    26.229]     compiled for 1.6.99.901, module version = 1.0.0
[    26.229]     Module class: X.Org Video Driver
[    26.229] (II) NVIDIA dlloader X Driver  515.57  Wed Jun 22 22:24:02 UTC 2022
[    26.229] (II) NVIDIA Unified Driver for all Supported NVIDIA GPUs
[    26.229] (--) Using syscons driver with X support (version 2.0)
[    26.229] (++) using VT number 9

[    26.230] (II) Loading sub module "fb"
[    26.230] (II) LoadModule: "fb"
[    26.230] (II) Loading /usr/local/lib/xorg/modules/libfb.so
[    26.230] (II) Module fb: vendor="X.Org Foundation"
[    26.230]     compiled for 1.20.14, module version = 1.0.0
[    26.230]     ABI class: X.Org ANSI C Emulation, version 0.4
[    26.230] (II) Loading sub module "wfb"
[    26.230] (II) LoadModule: "wfb"
[    26.230] (II) Loading /usr/local/lib/xorg/modules/libwfb.so
[    26.230] (II) Module wfb: vendor="X.Org Foundation"
[    26.230]     compiled for 1.20.14, module version = 1.0.0
[    26.230]     ABI class: X.Org ANSI C Emulation, version 0.4
[    26.230] (II) Loading sub module "ramdac"
[    26.230] (II) LoadModule: "ramdac"
[    26.230] (II) Module "ramdac" already built-in
[    26.230] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
[    26.230] (**) NVIDIA(0): Depth 24, (--) framebuffer bpp 32
[    26.230] (==) NVIDIA(0): RGB weight 888
[    26.230] (==) NVIDIA(0): Default visual is TrueColor
[    26.230] (==) NVIDIA(0): Using gamma correction (1.0, 1.0, 1.0)
[    26.230] (**) NVIDIA(0): Option "AllowEmptyInitialConfiguration" "True"
[    26.231] (**) NVIDIA(0): Option "UseEDID" "False"
[    26.231] (**) NVIDIA(0): Option "ConnectedMonitor" "DFP"
[    26.231] (**) NVIDIA(0): Enabling 2D acceleration
[    26.231] (**) NVIDIA(0): ConnectedMonitor string: "DFP"
[    26.231] (**) NVIDIA(0): Ignoring EDIDs
[    26.231] (II) Loading sub module "glxserver_nvidia"
[    26.231] (II) LoadModule: "glxserver_nvidia"
[    26.231] (II) Loading /usr/local/lib/xorg/modules/extensions/libglxserver_nvidia.so
[    26.294] (II) Module glxserver_nvidia: vendor="NVIDIA Corporation"
[    26.294]     compiled for 1.6.99.901, module version = 1.0.0
[    26.294]     Module class: X.Org Server Extension
[    26.294] (II) NVIDIA GLX Module  515.57  Wed Jun 22 22:21:01 UTC 2022
[    26.295] (II) NVIDIA: The X server supports PRIME Render Offload.
[    27.035] (EE)
[    27.035] (EE) Backtrace:
[    27.036] (EE) 0: /usr/local/bin/Xorg (OsInit+0x38a) [0x41c96a]
[    27.038] (EE) unw_get_proc_name failed: no unwind info found [-10]
[    27.038] (EE) 1: /lib/libthr.so.3 (?+0x0) [0x80093158e]
[    27.038] (EE) unw_get_proc_name failed: no unwind info found [-10]
[    27.038] (EE) 2: /lib/libthr.so.3 (?+0x0) [0x800930b3f]
[    27.039] (EE) 3: ? (?+0x0) [0x7ffffffff8a3]
[    27.039] (EE) 4: ? (?+0x0) [0x0]
[    27.039] (EE) 5: /usr/local/lib/xorg/modules/drivers/nvidia_drv.so (nvidiaUnlock+0x4201f) [0x801ca8c1f]
[    27.040] (EE) 6: ? (?+0x0) [0x0]
[    27.040] (EE) unw_step failed: unspecified (general) error [-1]
[    27.040] (EE)
[    27.040] (EE) Segmentation fault at address 0x0
[    27.040] (EE)
Fatal server error:
[    27.040] (EE) Caught signal 11 (Segmentation fault). Server aborting
[    27.040] (EE)
[    27.040] (EE)
Please consult the The X.Org Foundation support
     at http://wiki.x.org
 for help.
[    27.040] (EE) Please also check the log file at "/var/log/Xorg.8.log" for additional information.
[    27.040] (EE)
[    27.040] (EE) Server terminated with error (1). Closing log file.
```


----------



## checkpoint (Jul 31, 2022)

laufdi said:


> ... I can load that module without crash and Xorg runs, but shows a colorful pattern and no content, and a working mousepointer ...


Please show us content of your Xorg.0.log.


----------



## shkhln (Jul 31, 2022)

checkpoint said:


> If I enable both GPUs in xorg.conf, it just crashes when accessing NVIDIA GPU.


https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=261666#c34?


----------



## laufdi (Jul 31, 2022)

Here is my Xorg.0.log. It's practically identical to that of a Manjaro live stick (but where playing videos also takes 70% cpu, that means no accel?)  ...


----------



## Alexander88207 (Jul 31, 2022)

checkpoint said:


> Xorg 1.21 which is not supported on FreeBSD at the moment.








						⚙ D35661 xorg: upgrade to 21.1.3
					






					reviews.freebsd.org


----------



## laufdi (Jul 31, 2022)

Alexander88207 said:


> ⚙ D35661 xorg: upgrade to 21.1.3
> 
> 
> 
> ...


doesn't change anything for me ...


----------



## checkpoint (Aug 1, 2022)

shkhln said:


> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=261666#c34?


My Goddess, it worked! The *Option "UseDisplayDevice" "None"* in *Device* section for nvidia driver did the trick, it's not crashing any more and I'm able to use hybrid setup (two Xorgs one per each GPU).

Here's a brief benchmark of NVIDIA GPU on my laptop:


```
rz@butterfly:~ % nvrun-vgl  glxgears -info -fullscreen
GL_RENDERER   = NVIDIA GeForce RTX 3050 Laptop GPU/PCIe/SSE2
GL_VERSION    = 4.6.0 NVIDIA 515.57
GL_VENDOR     = NVIDIA Corporation
VisualID 33, 0x21
1766 frames in 5.0 seconds = 353.034 FPS
1777 frames in 5.0 seconds = 355.243 FPS
1767 frames in 5.0 seconds = 353.321 FPS
^C
```

Same test with AMD GPU:


```
rz@butterfly:~ % glxgears -info -fullscreen
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.
GL_RENDERER   = AMD RENOIR (DRM 3.40.0, 13.1-RELEASE, LLVM 13.0.1)
GL_VERSION    = 4.6 (Compatibility Profile) Mesa 21.3.8
GL_VENDOR     = AMD
VisualID 1276, 0x4fc
401 frames in 5.0 seconds = 79.980 FPS
301 frames in 5.0 seconds = 60.046 FPS
301 frames in 5.0 seconds = 60.055 FPS
301 frames in 5.0 seconds = 60.047 FPS
^C
```

This is so cool! Thank you shkhln for the hint.


----------



## checkpoint (Aug 1, 2022)

laufdi said:


> Here is my Xorg.0.log. It's practically identical to that of a Manjaro live stick (but where playing videos also takes 70% cpu, that means no accel?)  ...


I looked through your log and to me it seems ok. What if you run `startx` command, will it switch to graphics and produce xterm windows ?

As for video decoding acceleration, what browser do you use ? In Firefox there are some hardware acceleration settings AFAIK which may be not enabled in your case.


----------



## shkhln (Aug 1, 2022)

checkpoint said:


> ```
> rz@butterfly:~ % nvrun-vgl  glxgears -info -fullscreen
> ```


I would generally advice you to follow the Nvidia's documentation, rather than use that nvidia-hybrid-graphics port (which is basically a wrapper for VirtualGL).


----------



## checkpoint (Aug 1, 2022)

shkhln said:


> I would generally advice you to follow the Nvidia's documentation, rather than use that nvidia-hybrid-graphics port (which is basically a wrapper for VirtualGL).


Sure I gave it a try even before hybrid setup. Unfortunately PRIME (and Reverse PRIME as described in the docs) did not work - I was not able to obtain two providers despite the fact that both GPUs were initialized by Xorg. I shuffled options in xorg.conf back and forth, I tried without xorg.conf, all with no luck. Seems the problem is some missing feature in Xorg 1.20.14.

Personally I find hybrid setup more convenient, it gives me obvious control on which GPU app is using.

BTW, nVidia driver still does not support suspend/resume. Attempting to suspend while GPU is in use either crashes the OS or just hangs forever - there's a note about this in their README. Only once it came back (resumed), but GPU was stuck in unusable state. So, this discrete GPU has to be used with care - allways stop the app before suspending. AMD GPU works flawlessly. 

Anyway, I'm very much delighted with the result.

Just in case someone is interested in reproducing my setup:

First, install wrapper scripts for hybrid setup from ports:

`rz@butterfly:~ % uname -a
FreeBSD butterfly 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 GENERIC amd6

rz@butterfly:~ % cd /usr/ports/x11/nvidia-hybrid-graphics

rz@butterfly:/usr/ports/x11/nvidia-hybrid-graphics % sudo make && sudo make install`

Configure Xorg.conf for primary GPU (AMD):

```
rz@butterfly:~ % cat /etc/X11/xorg.conf

Section "Files"
        FontPath "/usr/local/share/fonts/jmk-x11-fonts"
        FontPath "/usr/local/share/fonts/cyrillic"
        FontPath "/usr/local/share/fonts/Liberation"
        FontPath "/usr/local/share/fonts/dejavu"
        FontPath "/usr/local/share/fonts/Caladea"
        FontPath "/usr/local/share/fonts/Carlito"
        FontPath "/usr/local/share/fonts/gnu-unifont-ttf"
EndSection

Section "ServerLayout"
      Identifier "layout"
      Screen 0 "iGPU"
      #Screen 1 "dGPU"
      #inactive "dGPU"
      Option "AllowNVIDIAGPUScreens"
      InputDevice    "Keyboard0" "CoreKeyboard"
      InputDevice    "Mouse0" "CorePointer"
EndSection

Section "ServerFlags"
        Option  "AutoAddGPU"      "1"
EndSection

Section "Device"
        Identifier "dGPU"
        Driver "nvidia"
        BusID "PCI:1:0:0"
        Option  "UseDisplayDevice" "None"
EndSection

Section "Screen"
      Identifier "dGPU"
      Device "dGPU"
EndSection


Section "Device"
      Identifier "iGPU"
      Driver "amdgpu"
      #Driver "modesetting"
      BusID "PCI:5:0:0"
EndSection

Section "Screen"
      Identifier "iGPU"
      Device "iGPU"
EndSection

Section "InputDevice"
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/sysmouse"
    Option         "Emulate3Buttons" "yes"

Section "InputDevice"
    Identifier     "Touchpa0"
    Driver         "evedv"
EndSection

Section "InputClass"
        Identifier "libinput touchpad catchall"
        MatchIsTouchpad "on"
        MatchDevicePath "/dev/input/event*"
        Driver "libinput"
        Option "Tapping" "on"
        Option "MiddleEmulation" "on"
        Option "AccelProfile" "adaptive"
EndSection

Section "InputClass"
        Identifier "keyboard defaults"
        MatchIsKeyboard "on"
        Option "XkbLayout" "us,ru"
        Option "XKbOptions" "terminate:ctrl_alt_bksp,numpad:microsoft,grp:caps_t
oggle"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "keyboard"
EndSection

:q
rz@butterfly:~ % cat /etc/X11/xorg.conf
Section "Files"
    FontPath "/usr/local/share/fonts/jmk-x11-fonts"
    FontPath "/usr/local/share/fonts/cyrillic"
    FontPath "/usr/local/share/fonts/Liberation"
    FontPath "/usr/local/share/fonts/dejavu"
    FontPath "/usr/local/share/fonts/Caladea"
    FontPath "/usr/local/share/fonts/Carlito"
    FontPath "/usr/local/share/fonts/gnu-unifont-ttf"
EndSection

Section "ServerLayout"
      Identifier "layout"
      Screen 0 "iGPU"
      #Screen 1 "dGPU"
      #inactive "dGPU"
      Option "AllowNVIDIAGPUScreens"
      InputDevice    "Keyboard0" "CoreKeyboard"
      InputDevice    "Mouse0" "CorePointer"
EndSection

Section "ServerFlags"
    Option    "AutoAddGPU"      "1"
EndSection

Section "Device"
    Identifier "dGPU"
    Driver "nvidia"
    BusID "PCI:1:0:0"
    Option    "UseDisplayDevice" "None"
EndSection

Section "Screen"
      Identifier "dGPU"
      Device "dGPU"
EndSection


Section "Device"
      Identifier "iGPU"
      Driver "amdgpu"
      #Driver "modesetting"
      BusID "PCI:5:0:0"
EndSection

Section "Screen"
      Identifier "iGPU"
      Device "iGPU"
EndSection

Section "InputDevice"
    Identifier     "Mouse0"
    Driver         "mouse"
    Option         "Protocol" "auto"
    Option         "Device" "/dev/sysmouse"
    Option         "Emulate3Buttons" "yes"
    Option         "ZAxisMapping" "4 5"
EndSection

Section "InputDevice"
    Identifier     "Touchpa0"
    Driver         "evedv"
EndSection

Section "InputClass"
        Identifier "libinput touchpad catchall"
        MatchIsTouchpad "on"
        MatchDevicePath "/dev/input/event*"
        Driver "libinput"
        Option "Tapping" "on"
        Option "MiddleEmulation" "on"
        Option "AccelProfile" "adaptive"
EndSection

Section "InputClass"
    Identifier "keyboard defaults"
    MatchIsKeyboard "on"
    Option "XkbLayout" "us,ru"
    Option "XKbOptions" "terminate:ctrl_alt_bksp,numpad:microsoft,grp:caps_toggle"
EndSection

Section "InputDevice"
    # generated from default
    Identifier     "Keyboard0"
    Driver         "keyboard"
EndSection
```

Configure Xorg.conf for secondary GPU (nVidia):

```
rz@butterfly:~ % cat /usr/local/etc/X11/xorg-nvidia-headless.conf

Section "ServerLayout"
    Identifier     "nvidia"
    Screen      0  "Screen0"
    InputDevice    "fake" "CorePointer" "CoreKeyboard"
    Option         "AutoAddDevices" "false"
EndSection

Section "Files"
    ModulePath      "/usr/local/lib/xorg/modules-NVIDIA"
    ModulePath      "/usr/local/lib/xorg/modules"
EndSection

Section "Module"
    Load           "dri3"
    Load           "glx"
    Disable        "efifb"
EndSection

Section "InputDevice"
    Identifier     "fake"
    Driver         ""
EndSection

Section "Monitor"
    Identifier     "Monitor0"
EndSection

Section "Device"
    Identifier     "Device0"
    Driver         "nvidia"
    BusID     "PCI:1:0:0
    Option  "UseDisplayDevice" "None"
EndSection

Section "Screen"
    Identifier     "Screen0"
    Device         "Device0"
    Monitor        "Monitor0"
EndSection
```

Run secondary service:
`service nvidia_xorg restart`

Log in to your X environment, then run a test:
`rz@butterfly:~ % nvrun-vgl glxgears -info -fullscreen
GL_RENDERER   = NVIDIA GeForce RTX 3050 Laptop GPU/PCIe/SSE2
GL_VERSION    = 4.6.0 NVIDIA 515.57
GL_VENDOR     = NVIDIA Corporation`


----------



## shkhln (Aug 1, 2022)

checkpoint said:


> Unfortunately PRIME (and Reverse PRIME as described in the docs) did not work - I was not able to obtain two providers despite the fact that both GPUs were initialized by Xorg.


You are not supposed to have _two_ providers. In fact, you don't even need to look at that information, directly testing whether __NV_PRIME_RENDER_OFFLOAD works should be enough.



checkpoint said:


> I shuffled options in xorg.conf back and forth, I tried without xorg.conf, all with no luck.


The explicit config is definitely required, xorg.conf-less Nvidia setup works only with Linux at the moment.



checkpoint said:


> Seems the problem is some missing feature in Xorg 1.20.14.


No.



checkpoint said:


> Personally I find hybrid setup more convenient, it gives me obvious control on which GPU app is using.


VirtualGL obviously doesn't work with Vulkan apps and, in nvidia-hybrid-graphics implementation, it's also doesn't work with 32-bit apps or Linux apps. Considering there aren't really any common use cases for a dedicated GPU other than gaming, being unable to run Wine or Steam games is pretty significant limitation.


----------



## checkpoint (Aug 2, 2022)

Just tried setting __NV_PRIME_RENDER_OFFLOAD env variable, it does not change anything. The problem is that I do not have NVIDIA-G0 provider together with modesetting provider.  nVidia docs explicitly says that there should be two providers listed to get things working. I do not know how to achieve this. I can have two separate screens one per each GPU, but that's not what is needed.


----------



## shkhln (Aug 2, 2022)

checkpoint said:


> Just tried setting __NV_PRIME_RENDER_OFFLOAD env variable, it does not change anything.


That's a bit too vague.



checkpoint said:


> The problem is that I do not have NVIDIA-G0 provider together with modesetting provider.


What exactly `xrandr --listproviders` prints?



checkpoint said:


> I can have two separate screens one per each GPU, but that's not what is needed.


I'm pretty sure you can't, unless you mean an external display.


----------



## checkpoint (Aug 2, 2022)

> I'm pretty sure you cant, unless you mean an external display.


Well, it allows me to.  NVIDIA driver gets loaded and I have black text mode screen of course which is expected, but technically I can have two screens.



> What exactly xrandr --listproviders prints?



Following nVidia docs I have to set iGPU to "modesetting" and dGPU to "nvidia", then this is what I get from `xrandr`:



```
rz@butterfly:~ % xrandr --listproviders
Providers: number : 1
Provider 0: id: 0x45 cap: 0xa, Sink Output, Sink Offload crtcs: 4 outputs: 2 associated providers: 0 name:modesetting
```

According to the Xorg.0.log it does NOT instantiate NVIDIA driver.

If I set "amdgpu" driver for iGPU, then I get this:


```
rz@butterfly:~ % xrandr --listproviders
Providers: number : 1
Provider 0: id: 0x55 cap: 0xf, Source Output, Sink Output, Source Offload, Sink Offload crtcs: 4 outputs: 2 associated providers: 0 name:Unknown AMD Radeon GPU @ pci:0000:05:00.0
```


----------



## shkhln (Aug 2, 2022)

checkpoint said:


> According to the Xorg.0.log it does NOT instantiate NVIDIA driver.


Well, if you see the driver being unloaded in the log (or missing from `sudo procstat -v `pgrep X` | grep drv`), you should tweak the config file until Xorg actually starts using it. You should probably start from scratch with the config from this post: https://forums.freebsd.org/threads/...rated-dedicated-nvidia-gpu.81994/#post-530613.


----------



## shkhln (Aug 2, 2022)

Oh, and don't test this simultaneously with nvidia-hybrid-graphics being installed. You need the normal nvidia-driver port.


----------



## laufdi (Aug 4, 2022)

checkpoint said:


> I looked through your log and to me it seems ok. What if you run `startx` command, will it switch to graphics and produce xterm windows ?
> 
> As for video decoding acceleration, what browser do you use ? In Firefox there are some hardware acceleration settings AFAIK which may be not enabled in your case.


I see some colorful little squares in the lower region, a mouse pointer which works ok, but no windows. My .initrc opens an xterm of which I see nothing, but I can close it blindly with Ctrl-D to end the session.
So the browser doesn't matter. 
Video is mpv or vlc (Accel off). That uses about 180% cpu vs. 60% on linux.


----------



## checkpoint (Aug 5, 2022)

> I see some colorful little squares in the lower region, a mouse pointer which works ok, but no windows. My .initrc opens an xterm of which I see nothing, but I can close it blindly with Ctrl-D to end the session.



It looks like a problem with Xorg driver. Try to install latest x11-drivers/xf86-video-amdgpu from ports.


----------



## checkpoint (Aug 5, 2022)

shkhln said:


> Oh, and don't test this simultaneously with nvidia-hybrid-graphics being installed. You need the normal nvidia-driver port.


I use hybrid setup with the latest nVidia driver 515.57, works pretty well.

As I said I can have two screens configured in xorg.conf, one screen per each GPU. In this case two GPU drivers load well, but this setup does not allow render offloading. No matter what I try, I cannot fit two GPUs into one screen setup. I can have either "nvidia" driver or "amdgpu" driver loaded by Xorg at a moment, but not both. If I set "modesetting" driver for iGPU, then "nvidia" driver for dGPU never gets probed. I tried a lot many combinations of options in xorg.conf and I conclude that something in Xorg itself does not allow this.

Can you please share your Xorg.0.log with working setup of two GPUs ?


----------



## laufdi (Aug 5, 2022)

checkpoint said:


> It looks like a problem with Xorg driver. Try to install latest x11-drivers/xf86-video-amdgpu from ports.


I have the newest (22.0.0_1) already.
And I tried many versions of drm-kmod: master, 5.10-lts, 5.5, 5.4 etc.
Do I need CURRENT? Will it work there? -- I built and installed the 14.0 kernel. No difference.


----------



## shkhln (Aug 5, 2022)

checkpoint said:


> I use hybrid setup with the latest nVidia driver 515.57, works pretty well.


With what? There are _zero_ plausible use cases for it. Well, maybe Blender with Eevee, but that's really niche.



checkpoint said:


> As I said I can have two screens configured in xorg.conf, one screen per each GPU. In this case two GPU drivers load well, but this setup does not allow render offloading. No matter what I try, I cannot fit two GPUs into one screen setup.


What is this "one screen setup"?



checkpoint said:


> I can have either "nvidia" driver or "amdgpu" driver loaded by Xorg at a moment, but not both. If I set "modesetting" driver for iGPU, then "nvidia" driver for dGPU never gets probed.


Did you notice nvidia-hybrid-graphics puts nvidia_drv.so outside of default search path?



checkpoint said:


> Can you please share your Xorg.0.log with working setup of two GPUs ?


I only have one GPU at the moment.


----------



## laufdi (Aug 5, 2022)

I get some vt backtrace when loading amdgpu on 14.0:


```
amdsmn0: <AMD Family 17h System Management Network> on hostb0
amdtemp0: <AMD CPU On-Die Thermal Sensors> on hostb0
[drm] amdgpu kernel modesetting enabled.
drmn0: <drmn> on vgapci0
vgapci0: child drmn0 requested pci_enable_io
vgapci0: child drmn0 requested pci_enable_io
[drm] initializing kernel modesetting (RENOIR 0x1002:0x1636 0x17AA:0x507F 0xC3).
drmn0: Trusted Memory Zone (TMZ) feature disabled as experimental (default)
[drm] register mmio base: 0xFD300000
[drm] register mmio size: 524288
[drm] add ip block number 0 <soc15_common>
[drm] add ip block number 1 <gmc_v9_0>
[drm] add ip block number 2 <vega10_ih>
[drm] add ip block number 3 <psp>
[drm] add ip block number 4 <smu>
[drm] add ip block number 5 <gfx_v9_0>
[drm] add ip block number 6 <sdma_v4_0>
[drm] add ip block number 7 <dm>
[drm] add ip block number 8 <vcn_v2_0>
[drm] add ip block number 9 <jpeg_v2_0>
drmn0: Fetched VBIOS from VFCT
amdgpu: ATOM BIOS: 113-RENOIR-025
drmn0: successfully loaded firmware image 'amdgpu/renoir_sdma.bin'
[drm] VCN decode is enabled in VM mode
[drm] VCN encode is enabled in VM mode
[drm] JPEG decode is enabled in VM mode
[drm] vm size is 262144 GB, 4 levels, block size is 9-bit, fragment size is 9-bit
drmn0: VRAM: 512M 0x000000F400000000 - 0x000000F41FFFFFFF (512M used)
drmn0: GART: 1024M 0x0000000000000000 - 0x000000003FFFFFFF
drmn0: AGP: 267419648M 0x000000F800000000 - 0x0000FFFFFFFFFFFF
[drm] Detected VRAM RAM=512M, BAR=512M
[drm] RAM width 128bits DDR4
[TTM] Zone  kernel: Available graphics memory: 7955644 KiB
[TTM] Zone   dma32: Available graphics memory: 2097152 KiB
[TTM] Initializing pool allocator
[drm] amdgpu: 512M of VRAM memory ready
[drm] amdgpu: 3072M of GTT memory ready.
[drm] GART: num cpu pages 262144, num gpu pages 262144
[drm] PCIE GART of 1024M enabled (table at 0x000000F400900000).
drmn0: successfully loaded firmware image 'amdgpu/renoir_asd.bin'
drmn0: successfully loaded firmware image 'amdgpu/renoir_pfp.bin'
drmn0: successfully loaded firmware image 'amdgpu/renoir_me.bin'
drmn0: successfully loaded firmware image 'amdgpu/renoir_ce.bin'
drmn0: successfully loaded firmware image 'amdgpu/renoir_rlc.bin'
drmn0: successfully loaded firmware image 'amdgpu/renoir_mec.bin'
drmn0: successfully loaded firmware image 'amdgpu/renoir_mec2.bin'
drmn0: successfully loaded firmware image 'amdgpu/renoir_dmcub.bin'
[drm] Loading DMUB firmware via PSP: version=0x01000000
drmn0: successfully loaded firmware image 'amdgpu/renoir_vcn.bin'
[drm] Found VCN firmware Version ENC: 1.7 DEC: 4 VEP: 0 Revision: 17
drmn0: Will use PSP to load VCN firmware
[drm] reserve 0x400000 from 0xf41f800000 for PSP TMR
drmn0: SMU is initialized successfully!
[drm] kiq ring mec 2 pipe 1 q 0
[drm] Display Core initialized with v3.2.104!
[drm] DMUB hardware initialized: version=0x01000000
lkpi_iic0: <LinuxKPI I2C> on drmn0
iicbus0: <Philips I2C bus> on lkpi_iic0
iic0: <I2C generic I/O> on iicbus0
ugen1.3: <vendor 0x8087 product 0x0029> at usbus1
lkpi_iic1: <LinuxKPI I2C> on drmn0
iicbus1: <Philips I2C bus> on lkpi_iic1
iic1: <I2C generic I/O> on iicbus1
lkpi_iic2: <LinuxKPI I2C> on drmn0
iicbus2: <Philips I2C bus> on lkpi_iic2
iic2: <I2C generic I/O> on iicbus2
[drm] VCN decode and encode initialized successfully(under DPG Mode).
[drm] JPEG decode initialized successfully.
drmn0: SE 1, SH per SE 1, CU per SH 8, active_cu_number 6
[drm] fb mappable at 0x410CE0000
[drm] vram apper at 0x410000000
[drm] size 8294400
[drm] fb depth is 24
[drm]    pitch is 7680
VT: Replacing driver "efifb" with new "fb".
taskqueue_drain with the following non-sleepable locks held:
exclusive sleep mutex vtdev (vtdev) r = 0 (0xffffffff81aea698) locked @ /usr/src/sys/dev/vt/vt_core.c:3051
stack backtrace:
#0 0xffffffff80c5c815 at witness_debugger+0x65
#1 0xffffffff80c5d97a at witness_warn+0x3fa
#2 0xffffffff80c4f113 at taskqueue_drain+0x33
#3 0xffffffff84451933 at vt_kms_postswitch+0x73
#4 0xffffffff80a314fd at vt_fb_init+0xfd
#5 0xffffffff80a36b48 at vt_replace_backend+0x118
#6 0xffffffff80a31603 at vt_fb_attach+0x13
#7 0xffffffff844635bb at linux_register_framebuffer+0xeb
#8 0xffffffff84464a62 at __drm_fb_helper_initial_config_and_unlock+0x492
#9 0xffffffff8401d9a6 at amdgpu_fbdev_init+0xe6
#10 0xffffffff840125e0 at amdgpu_device_init+0x1d70
#11 0xffffffff84029536 at amdgpu_driver_load_kms+0x16
#12 0xffffffff8401ca5f at amdgpu_pci_probe+0x1ef
#13 0xffffffff80e635f1 at linux_pci_attach_device+0x431
#14 0xffffffff80c27021 at device_attach+0x3c1
#15 0xffffffff80c26bd0 at device_probe_and_attach+0x70
#16 0xffffffff80c28be7 at bus_generic_driver_added+0x67
#17 0xffffffff80c246a9 at devclass_driver_added+0x39
Sleeping on "tq_drain" with the following non-sleepable locks held:
exclusive sleep mutex vtdev (vtdev) r = 0 (0xffffffff81aea698) locked @ /usr/src/sys/dev/vt/vt_core.c:3051
stack backtrace:
#0 0xffffffff80c5c815 at witness_debugger+0x65
#1 0xffffffff80c5d97a at witness_warn+0x3fa
#2 0xffffffff80bf6e05 at _sleep+0x55
#3 0xffffffff80c4f1db at taskqueue_drain+0xfb
#4 0xffffffff84451933 at vt_kms_postswitch+0x73
#5 0xffffffff80a314fd at vt_fb_init+0xfd
#6 0xffffffff80a36b48 at vt_replace_backend+0x118
#7 0xffffffff80a31603 at vt_fb_attach+0x13
#8 0xffffffff844635bb at linux_register_framebuffer+0xeb
#9 0xffffffff84464a62 at __drm_fb_helper_initial_config_and_unlock+0x492
#10 0xffffffff8401d9a6 at amdgpu_fbdev_init+0xe6
#11 0xffffffff840125e0 at amdgpu_device_init+0x1d70
#12 0xffffffff84029536 at amdgpu_driver_load_kms+0x16
#13 0xffffffff8401ca5f at amdgpu_pci_probe+0x1ef
#14 0xffffffff80e635f1 at linux_pci_attach_device+0x431
#15 0xffffffff80c27021 at device_attach+0x3c1
#16 0xffffffff80c26bd0 at device_probe_and_attach+0x70
#17 0xffffffff80c28be7 at bus_generic_driver_added+0x67
lock order reversal: (Giant after non-sleepable)
 1st 0xffffffff81aea698 vtdev (vtdev, sleep mutex) @ /usr/src/sys/dev/vt/vt_core.c:3051
 2nd 0xffffffff81a02b40 Giant (Giant, sleep mutex) @ /usr/src/sys/kern/kern_synch.c:232
lock order Giant -> vtdev established at:
#0 0xffffffff80c5bb0d at witness_checkorder+0x32d
#1 0xffffffff80bc51b4 at __mtx_lock_flags+0x94
#2 0xffffffff80a36115 at vt_upgrade+0x365
#3 0xffffffff80b71a08 at mi_startup+0x1e8
#4 0xffffffff8038b023 at btext+0x23
lock order vtdev -> Giant attempted at:
#0 0xffffffff80c5c3dd at witness_checkorder+0xbfd
#1 0xffffffff80bc51b4 at __mtx_lock_flags+0x94
#2 0xffffffff80bf712f at _sleep+0x37f
#3 0xffffffff80c4f1db at taskqueue_drain+0xfb
#4 0xffffffff84451933 at vt_kms_postswitch+0x73
#5 0xffffffff80a314fd at vt_fb_init+0xfd
#6 0xffffffff80a36b48 at vt_replace_backend+0x118
#7 0xffffffff80a31603 at vt_fb_attach+0x13
#8 0xffffffff844635bb at linux_register_framebuffer+0xeb
#9 0xffffffff84464a62 at __drm_fb_helper_initial_config_and_unlock+0x492
#10 0xffffffff8401d9a6 at amdgpu_fbdev_init+0xe6
#11 0xffffffff840125e0 at amdgpu_device_init+0x1d70
#12 0xffffffff84029536 at amdgpu_driver_load_kms+0x16
#13 0xffffffff8401ca5f at amdgpu_pci_probe+0x1ef
#14 0xffffffff80e635f1 at linux_pci_attach_device+0x431
#15 0xffffffff80c27021 at device_attach+0x3c1
#16 0xffffffff80c26bd0 at device_probe_and_attach+0x70
#17 0xffffffff80c28be7 at bus_generic_driver_added+0x67
start FB_INFO:
type=11 height=1080 width=1920 depth=32
pbase=0x410ce0000 vbase=0xfffff80410ce0000
name=drmn0 flags=0x0 stride=7680 bpp=32
end FB_INFO
drmn0: ring gfx uses VM inv eng 0 on hub 0
drmn0: ring comp_1.0.0 uses VM inv eng 1 on hub 0
drmn0: ring comp_1.1.0 uses VM inv eng 4 on hub 0
drmn0: ring comp_1.2.0 uses VM inv eng 5 on hub 0
drmn0: ring comp_1.3.0 uses VM inv eng 6 on hub 0
drmn0: ring comp_1.0.1 uses VM inv eng 7 on hub 0
drmn0: ring comp_1.1.1 uses VM inv eng 8 on hub 0
drmn0: ring comp_1.2.1 uses VM inv eng 9 on hub 0
drmn0: ring comp_1.3.1 uses VM inv eng 10 on hub 0
drmn0: ring kiq_2.1.0 uses VM inv eng 11 on hub 0
drmn0: ring sdma0 uses VM inv eng 0 on hub 1
drmn0: ring vcn_dec uses VM inv eng 1 on hub 1
drmn0: ring vcn_enc0 uses VM inv eng 4 on hub 1
drmn0: ring vcn_enc1 uses VM inv eng 5 on hub 1
drmn0: ring jpeg_dec uses VM inv eng 6 on hub 1
vgapci0: child drmn0 requested pci_get_powerstate
sysctl_warn_reuse: can't re-use a leaf (hw.dri.debug)!
lkpi_iic3: <LinuxKPI I2C> on drm1
iicbus3: <Philips I2C bus> on lkpi_iic3
iic3: <I2C generic I/O> on iicbus3
lkpi_iic4: <LinuxKPI I2C> on drm3
iicbus4: <Philips I2C bus> on lkpi_iic4
iic4: <I2C generic I/O> on iicbus4
[drm] Initialized amdgpu 3.40.0 20150101 for drmn0 on minor 0
```


----------



## checkpoint (Aug 6, 2022)

laufdi said:


> Do I need CURRENT? Will it work there? -- I built and installed the 14.0 kernel. No difference.


I'm using 13.1-RELEASE now and it works fine with 5.10-lts amdgpu driver. I noticed that my GPU has a different ChipID which is 0x1638 where yours is 0x1636.

Have your tried some Linux on your machine ? Can it handle GPU properly ? If so, with what DRM driver and Xorg version ?


----------



## laufdi (Aug 6, 2022)

As I said, Manjaro linux is running fine. Here the Xorg.0.log


----------



## laufdi (Aug 6, 2022)

checkpoint said:


> I noticed that my GPU has a different ChipID which is 0x1638 where yours is 0x1636.


Strange, the 1636 is listed in `drivers/gpu/drm/amd/display/include/dal_asic_id.h`, while the 1638 isn't. (I found it)
Searching for 0x1636 I found this who apparently succeeded with drm-kmod _5.5-stable_, but that version doesn't compile with today's 13.1-STABLE.

... I built the commits of kernel (13.0) and drm-kmod (5.5) he advised and get exactly the same result as before ...


----------



## checkpoint (Aug 6, 2022)

laufdi said:


> Strange, the 1636 is listed in `drivers/gpu/drm/amd/display/include/dal_asic_id.h`, while the 1638 isn't. (I found it)
> Searching for 0x1636 I found this who apparently succeeded with drm-kmod _5.5-stable_, but that version doesn't compile with today's 13.1-STABLE.
> 
> ... I built the commits of kernel (13.0) and drm-kmod (5.5) he advised and get exactly the same result as before ...


You should use drm-kmod 5.10-lts from the git repo. The one that is in FreeBSD repo does not work with our GPUs.

Here's what I have installed:


```
rz@butterfly:~/drm-kmod % git status
On branch 5.10-lts
Your branch is up to date with 'origin/5.10-lts'.

nothing to commit, working tree clean
rz@butterfly:~/drm-kmod % git remote -v
origin    https://github.com/freebsd/drm-kmod.git (fetch)
origin    https://github.com/freebsd/drm-kmod.git (push)

rz@butterfly:~/drm-kmod % git log
commit ecbad2e93ea96fab832b1d144b7dcf839b32f36e (HEAD -> 5.10-lts, origin/5.10-lts)
Author: Emmanuel Vadot <manu@FreeBSD.org>
Date:   Wed Jul 27 11:40:47 2022 +0200

    linuxkpi: Use power_supply.h from base
    
    It's present since 13.1
    
    Sponsored by:   Beckhoff Automation GmbH & Co. KG
    
    (cherry picked from commit 833208195d4880b95ce4fb01a09e48af50813ab1)
```


----------



## laufdi (Aug 6, 2022)

checkpoint said:


> ```
> rz@butterfly:~/drm-kmod % git status
> commit ecbad2e93ea96fab832b1d144b7dcf839b32f36e (HEAD -> 5.10-lts, origin/5.10-lts)
> Author: Emmanuel Vadot <manu@FreeBSD.org>
> ...


All the same here, but I think you are one commit behind  
And whether I use the master branch or 5.10-lts doesn't change the results for me ... 

Is there a comprehensive list of Options I can play with? Are there more than:

```
Section "Device"
#Section "OutputClass"
        Identifier "Card0"
       Driver "amdgpu"
#Driver "scfb"
#       Driver "modesetting"
#    BusID "PCI:4:0:0"

#      Option "AccelMethod" "none"
#    Option "MigrationHeuristic" "greedy"
#       Option "SWcursor" "Off"
#       Option "Accel" "Off"
#       Option "ZaphodHeads" "string"
       Option "DRI" "3"
#       Option "EnablePageFlip" "Off"
#       Option "TearFree" "On"
#       Option "VariableRefresh" "On"
#       Option "AsyncFlipSecondaries" "Off"
#      Option "ShadowPrimary" "On"
EndSection
```


----------



## laufdi (Aug 9, 2022)

I made a diff of the linux and freebsd Xorg.0.log.
The only difference I see is the smaller(!) gart and vram size on linux - does it mean anything?
The linux kernel module has an option to restrict `gartsize` ... can I do something like this on FreeBSD?

... There is `hw.amdgpu.gartsize`, but setting it doesn't change anything.
And setting the graphics memory in BIOS changes the vram amounts, also in linux, but doesn't help.


```
--- xorg.fbsd    2022-08-09 14:48:33.322869000 +0200
+++ xorg.linux    2022-08-09 14:48:46.885883000 +0200
@@ -1,7 +1,8 @@
 
 r 1.21.1.4
 rsion 11, Revision 0
-Current Operating System: FreeBSD thinkpad 13.1-STABLE FreeBSD 13.1-STABLE #13 stable/13-n252050-9a0d922f57f: Sun Aug  7 13:02:24 CEST 2022     root@thinkpad:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64
+Current Operating System: Linux manjaro 5.15.55-1-MANJARO #1 SMP PREEMPT Fri Jul 15 09:45:39 UTC 2022 x86_64
+Kernel command line: BOOT_IMAGE=/boot/vmlinuz-x86_64 lang=en_US keytable=de tz=UTC misobasedir=manjaro misolabel=MANJARO_XFCE_2135 quiet systemd.show_status=1 apparmor=1 security=apparmor driver=free nouveau.modeset=1 i915.modeset=1 radeon.modeset=1
 
 (II) Module ABI versions:
@@ -43,31 +43,59 @@
     X.Org Video Driver: 25.2
     X.Org XInput driver : 24.4
     X.Org Server Extension : 10.0
-(--) PCI:*(4@0:0:0) 1002:1636:17aa:507f rev 195, Mem @ 0x460000000/268435456, 0x470000000/2097152, 0xfd300000/524288, I/O @ 0x00001000/256, BIOS @ 0x????????/65536

+(--) PCI:*(4@0:0:0) 1002:1636:17aa:507f rev 195, Mem @ 0x460000000/268435456, 0x470000000/2097152, 0xfd300000/524288, I/O @ 0x00001000/256
[ICODE]1e2e5000[/ICODE]

 (II) Module amdgpu: vendor="X.Org Foundation"
-    compiled for 1.21.1.4, module version = 22.0.0
+    compiled for 1.21.1.3, module version = 22.0.0
     Module class: X.Org Video Driver
     ABI class: X.Org Video Driver, version 25.2

@@ -79,11 +107,11 @@


-(II) AMDGPU(0): glamor X acceleration enabled on AMD RENOIR (DRM 3.40.0, 13.1-STABLE, LLVM 13.0.1)
+(II) AMDGPU(0): glamor X acceleration enabled on AMD RENOIR (LLVM 14.0.6, DRM 3.42, 5.15.55-1-MANJARO)
 (II) AMDGPU(0): glamor detected, initialising EGL layer.
 (==) AMDGPU(0): TearFree property default: auto
 (==) AMDGPU(0): VariableRefresh: disabled
@@ -142,19 +170,21 @@

 (II) AMDGPU(0): Output eDP using initial mode 1920x1080 +0+0
-(II) AMDGPU(0): mem size init: gart size :bfa7e000 vram size: s:1e2e5000 visible:1e2e5000
+(II) AMDGPU(0): mem size init: gart size :bf6ea000 vram size: s:1df2d000 visible:1df2d000
```


----------



## checkpoint (Aug 9, 2022)

laufdi said:


> I made a diff of the linux and freebsd Xorg.0.log.
> The only difference I see is the smaller(!) gart and vram size on linux - does it mean anything?
> The linux kernel module has an option to restrict `gartsize` ... can I do something like this on FreeBSD?


I'm not an expert in these matters, but what version of DRM driver do you run on Linux (`uname -r`) ?


----------



## laufdi (Aug 10, 2022)

FreeBSD:
`[drm] Initialized amdgpu 3.40.0 20150101 for drmn0 on minor 0`

Linux:
`[drm] Initialized amdgpu 3.42.0 20150101 for 0000:04:00.0 on minor 0`

an older gparted iso works with 3.40:

`AMDGPU(0): glamor X acceleration enabled on AMD RENOIR (DRM 3.40.0, 5.10.0-8-amd64, LLVM 11.0.1)`

it even works with 3.39 (gparted 1.1):
`AMDGPU(0): glamor X acceleration enabled on AMD RENOIR (DRM 3.39.0, 5.9.0-3-amd64, LLVM 11.0.0)`


----------



## laufdi (Aug 17, 2022)

The problems are gone after I removed /boot/device.hints. Is it the `hint.sc` entry there?
It comes from /usr/src/sys/amd64/conf/GENERIC.hints


----------



## checkpoint (Aug 19, 2022)

I have default /boot/device.hints on my system. Here blow is its content:


```
# $FreeBSD$
hint.fdc.0.at="isa"
hint.fdc.0.port="0x3F0"
hint.fdc.0.irq="6"
hint.fdc.0.drq="2"
hint.fd.0.at="fdc0"
hint.fd.0.drive="0"
hint.fd.1.at="fdc0"
hint.fd.1.drive="1"
hint.atkbdc.0.at="isa"
hint.atkbdc.0.port="0x060"
hint.atkbd.0.at="atkbdc"
hint.atkbd.0.irq="1"
hint.psm.0.at="atkbdc"
hint.psm.0.irq="12"
hint.sc.0.at="isa"
hint.sc.0.flags="0x100"
hint.uart.0.at="isa"
hint.uart.0.port="0x3F8"
hint.uart.0.flags="0x10"
hint.uart.0.irq="4"
hint.uart.1.at="isa"
hint.uart.1.port="0x2F8"
hint.uart.1.irq="3"
hint.ppc.0.at="isa"
hint.ppc.0.irq="7"
hint.atrtc.0.at="isa"
hint.atrtc.0.port="0x70"
hint.atrtc.0.irq="8"
hint.attimer.0.at="isa"
hint.attimer.0.port="0x40"
hint.attimer.0.irq="0"
hint.acpi_throttle.0.disabled="1"
hint.p4tcc.0.disabled="1"
```


----------



## laufdi (Aug 19, 2022)

Nonsense again ... I recopied the GENERIC.hints as device.hints and it's still working. So there must have been some other change, maybe I recompiled an xf86 driver or deinstalled/reinstalled the correct one ... or STABLE changes of the last 3 days (I doubt it) ... I can't reproduce it.


----------



## cy@ (Aug 19, 2022)

Phishfry said:


> Windows garbage. Probably some backroom skullduggery to make sure it never works on anything but Windows.
> Optimus was a Windows thing.


I have a couple of acer laptops here. Acer tends not to fully implement ACPI functions that FreeBSD expects to be in the BIOS resulting in some things not working. Acer makes up for that lack using WMI enabled drivers in Windows, implementing such functions in the Windows drivers. It's certainly not ideal but a person can work with those laptops if a person is willing to overlook the deficiencies. My first Acer laptop (20 years ago) worked flawlessly because the functions were implemented in the BIOS only and were not advertised as ACPI functions. My next one had problems with screen dim using the keyboard buttons (though xbacklight and xbrightness work because they talk directly to the hardware), volume control using the keyboard buttons (though mixer works because it talks directly to the hardware). Had they implemented those functions in BIOS only and not attempted to implement those same functions in ACPI in a broken manner requiring a Windows driver to make up for the bugs in their ACPI implementation their laptops would work perfectly with FreeBSD. IMO: vendor incompetence.

If a person wants full FreeBSD support I would not recommend an Acer. I suspect any laptop that uses WMI might also have the same problems.

I would not buy another Acer again.  Though I suspect there are other laptops similarly with incorrect or buggy ACPI too. A person should build a memory stick or clone enough of their boot disk, including /usr/local, to kick the tires in the shop. Shopping for laptops has always been a painful experience and something I keep putting off because of the work and pain involved.


----------



## laufdi (Aug 19, 2022)

Speaking of BIOS ... I switched on virtualization in BIOS, can this be related (that it was off)?


----------



## laufdi (Dec 15, 2022)

Now since a week or so (13.1-STABLE #33, drm-510-kmod-5.10.113_8, and  gpu-firmware-amd-kmod-renoir-20221207_1 updates), the problem is back ...


----------

