libcamera: prefer udmabuf over dma-heaps - #464

kernel config PR: qualcomm-linux/kernel-topics#1805

Test run workflow

Test jobs for commit fe7e479

Test Results

1 files   -  18  1 suites   - 77   2m 14s ⏱️ - 1h 33m 33s
7 tests  -  72  7 ✅  -  68  0 💤 ±0  0 ❌  - 4 
9 runs   - 509  9 ✅  - 505  0 💤 ±0  0 ❌  - 4 

Results for commit fe7e479. ± Comparison against base commit 2c0f42b.

0_BT_FW_KMD_Service ‑ BT_FW_KMD_Service
0_hotplug ‑ hotplug
10_AudioRecord ‑ AudioRecord
10_Docker_Kernel_Config ‑ Docker_Kernel_Config
11_EFI_Variable_Validation ‑ EFI_Variable_Validation
11_Ethernet ‑ Ethernet
12_DSP_AudioPD ‑ DSP_AudioPD
12_Kubernetes_Kernel_Config ‑ Kubernetes_Kernel_Config
13_Docker_Kernel_Config ‑ Docker_Kernel_Config
13_Logging_Journalctl_Validation ‑ Logging_Journalctl_Validation
…

I think this is a more generic problem, to be solved with libcamera itself. If the allocator can't provide enough memory, the libcamera should be able to fallback to the other allocators. This is especially true for the CMA allocators where we know that memory space is limited.

The upstream linux-media (e.g. gstreamer) direction seems to be to use udmabuf more and more where ever possible now that it is generally available in most distro configurations.

I think trying relatively unlimited udmabuf memory before CMA heaps makes sense from a not wasting scarce resources like CMA. OTOH IIRC the try CMA first approach is there for platforms where e.g. the video hw encoder which may sit after libcamera can only operate on CMA memory.

So I think that rather then carrying a downstream patch the following should be done:

1. Make the order in which allocators are tried configurable through libcamera's configfile mechanism allowing changing of the order by dropping a /etc/libcamera/configuration.yaml file. This alone should be sufficient for meta-qcom for now, by meta-qcom using this new config mechanism.
2. Allow pipeline-handlers to provide a pipeline-handler specific default order which to use when the config file does not specify any order. pipeline-handlers is how libcamera deals with platform differences. So this will allow the camss pipeline-handler I'm working on to make trying udmabuf first the default on Qualcomm platforms.

Dmitry Baryshkov (@lumag)'s idea of falling back to another allocator is interesting but I think that will complicate things quite a bit. E.g. in case of platforms where we actually want CMA to be tried first, returning a udmabuf backed dmabuf may cause errors elsewhere (e.g. the video-encoder), so I'm not sure how helpful this will be.

I think that a configurable setting in which order to try the allocators should solve this nicely.

Wenmeng Liu (@wenmliu) I think, Hans de Goede (@jwrdegoede) 's reply means: please open a merge request against libcamera, once agreed there, please backport it to QLI.

Ack, I was about to write: to be clear my suggestion to fix this should be done upstream first.

p.s. :

1. For more info on the config mechanism see: https://docs.libcamera.org/master/runtime_configuration.html and also please update runtime_configuration.rst when adding a setting for this.
2. Note upstream does not take pull-requests they use an email based patch-workflow closely mirroring the kernel's. So subscribe to the libcamera-devel list: https://lists.libcamera.org/listinfo/libcamera-devel and after writing the patches git send-email them to the libcamera-devel list.

Sounds good. I'll work on the changes and submit them upstream.

The upstream linux-media (e.g. gstreamer) direction seems to be to use udmabuf more and more where ever possible now that it is generally available in most distro configurations.

Just an FYI but you probably already know: UDMABUF is not enabled in defconfig

The upstream linux-media (e.g. gstreamer) direction seems to be to use udmabuf more and more where ever possible now that it is generally available in most distro configurations.

Just an FYI but you probably already know: UDMABUF is not enabled in defconfig

Interesting, I'm not really a defconfig user, but that is a good point and one which is probably worth fixing... Let me discuss this with Robert Mader who has been pushing this from the gstreamer side.

The upstream linux-media (e.g. gstreamer) direction seems to be to use udmabuf more and more where ever possible now that it is generally available in most distro configurations.

Just an FYI but you probably already know: UDMABUF is not enabled in defconfig

Yes, that's why I submitted a PR to enable UDMABUF for Qualcomm platforms:
qualcomm-linux/kernel-topics#1805