Hello,
I would like to encode an H.264 bitstream with v4l2h264enc element on GStreamer with various encoding parameters, so I guess I should specify parameters with “extra-controls=”, but I don’t know the specification of it.
Where is the description of it in 96Boards.org Documentation or Qualcomm’s Developers Site?
Thank you in advance.
Regards,
Mitsuteru
Hello,
Is there anyone who can tell me about “extra-controls=” option details?
I have checked keywords “video_bitrate_mode”, “video_gop_size”, “h264_entropy_mode”, “h264_loop_filter_mode”, these seems can be accepted by GStreamer because there are no warning messages output.
But I don’t know these options are correct.
I guess there is options list where in the Linaro kernnel or GStremaer source codes.
Best regards,
Mitsureru
This “extra-controls” field is in some way passed directly to the v4l2 subsystem by gstreamer.
An exhaustive raw list of all v4l2 control IDs (CID) is available in the v4l2-controls uapi linux header.
However at runtime It’s possible to list all supported controls with the following command:
$ v4l2-ctl -L
Note that v4l2-ctl can be obtained by installing v4l-utils package.
Hi Loic,
Thank you for your information, and then, I have tried the command “v4l2-ctl -L”:
linaro@linaro-alip:~$ v4l2-ctl -L
Codec Controls
mpeg4_loop_filter_enable (bool) : default=0 value=0
h264_level (menu) : min=0 max=15 default=0 value=0 flags=volatile
0: 1
1: 1b
2: 1.1
3: 1.2
4: 1.3
5: 2
6: 2.1
7: 2.2
8: 3
9: 3.1
10: 3.2
11: 4
12: 4.1
13: 4.2
14: 5
15: 5.1
h264_profile (menu) : min=0 max=16 default=0 value=0 flags=volatile
0: Baseline
1: Constrained Baseline
2: Main
4: High
15: Stereo High
16: Multiview High
mpeg4_level (menu) : min=0 max=7 default=0 value=0 flags=volatile
0: 0
1: 0b
2: 1
3: 2
4: 3
5: 3b
6: 4
7: 5
mpeg4_profile (menu) : min=0 max=4 default=0 value=0 flags=volatile
0: Simple
1: Advanced Simple
vpx_profile (int) : min=0 max=3 step=1 default=0 value=0 flags=volatile
I certainly got some information, but it does not include detailed parameters such as “video_bitrate” which specifies the bitrate.
“v4l2-ctl” is not displayed all parameters?
Best regards,
Mitsureru
Sorry, I could check the parameter details as follows;
v4l2-ctl -d /dev/video1 -l
This is what I want to do. Thank you.
Excuse me, I was able to set extra-controls in the way you taught me the other day,
but unfortunately most of the parameters seem to have no effect on the output bitstream even if I actually set it.
In order to obtain higher quality encoding results,
I would like to be able to set the parameters in practice as follows;
(1) Entropy Mode : CABAC
parameter : h264_entropy_mode
purpose : Improve quality
(2) Deblocking Filter
parameter : (maybe) h264_loop_filter_mode
purpose : Noise reduction
(3) IDR Interval
parameter : video_gop_size
purpose : Improve compression ratio
(4) The number of B pictures (Increase)
parameter : video_b_frames , (2 - 4 if possible)
purpose : Improve compression ratio
(5) CPB Size
parameter : h264_cpb_size
purpose : Improve quality
((5) is not exist in v4l2-ctl -L, but it is defined as V4L2_CID_MPEG_VIDEO_H264_CPB_SIZE in v4l2-ctrls.c.)
Of course, I do not think that these parameters can be used in a short space of time,
but are there plans to be able to use these parameters in the future?
if you upgraded to the latest software release, there is now v4l2 support in ffmpeg for encoding/decoding; just select the encoder that you want (-c:v h264_v4l2m2m for instance)
An example to encode an NV12 video stream to h264:
$ ffmpeg -loglevel debug -f rawvideo -pix_fmt nv12 -s:v 1280:720 -r 25 -i ~/Videos/raw/freeway.yuv -c:v h264_v4l2m2m out/out.h264.mp4
There is additional information about h264 on this link:
https://trac.ffmpeg.org/wiki/Encode/H.264
For more detailed information you can check the code itself:
Thank you for your information.
It seems that v4l2_m2m_enc.c is passthroughing parameters with v4l2_set_ext_ctrl(), is it covering the parameters indicated by so-called V4L2_CID_?
Also, I’d like to incorporate the encoder code into my program, but do I have to access the encoder via the libavcodec API to use this FFmpeg implementation?
medianoche http://discuss.96boards.org/u/medianoche
October 24Thank you for your information.
It seems that v4l2_m2m_enc.c is passthroughing parameters with
v4l2_set_ext_ctrl(), is it covering the parameters indicated by
so-called V4L2_CID_?
yes; line 181 dumps to the console all the parameters that are
programmable (bit rate, profile, frame rate, gov size…)
Also, I’d like to incorporate the encoder code into my program, but do
I have to access the encoder via the libavcodec API to use this FFmpeg
implementation?
I guess it depends what you wan to do but generally speaking yes you
need to use the libavcodec API to select the codec and so on.
or if you just want to encode a file present in the file system I guess
you could just execve ffmpeg from your main.
I thought to try encoding with FFmpeg.
Therefore, I upgraded Debian using Build # 283 image.
However, after installing the # 283 image (Linux linaro-alip 4.9.56-linaro-lt-qcom) I noticed that there are some problems as follows;
(a) can not find the binary of FFmpeg.
Do I need to obtain FFmpeg package separately with apt-get etc.?
(b) can not find /dev/video*
I think that v4l2h264enc will access these devices, but has the device file to be accessed changed?
(c) can not use v4l2-ctl.
I think that is the same reason as (b).
(d) can not find v4l2h264enc with gst-launch-1.0
The following error message is displayed.
“WARNING: erroneous pipeline: no element “v4l2h264enc””
I may be missing the necessary steps for video encoding.
Please let me know the solution.
I am sorry, I forgot to say that you have to use the OE release:
https://builds.96boards.org/releases/dragonboard410c/linaro/openembedded/latest/
All right, I’d like to try out various with the OE.
However, I do not understand that hardware acceleration can not be used on Debian Build # 283 with FFmpeg or GStreamer.
Is there any information of these?
Ffmpeg was just not included in the Debian release (but you can always pull it from the sources master branch and build it yourself, there is nothing impeding it from working).
As to why you don’t have the video nodes I can’t comment without some logs- can you share the boot console or look for any error strings that mention “Venus” o “venus”?
I see, I’ll try it later.
I’ve checked the dmesg and grep “venus”, the result is as folllows;
linaro@linaro-alip:~$ dmesg | grep venus
[ 1.975764] msm_iommu_ctx 1e25000.venus_ns: context venus_ns using bank 5
[ 1.977940] msm_iommu_ctx 1e33000.venus_sec_pixel: context venus_sec_pixel using bank 19
[ 1.978327] msm_iommu_ctx 1e34000.venus_sec_bitstream: context venus_sec_bitstream using bank 20
[ 1.978700] msm_iommu_ctx 1e35000.venus_sec_non_pixel: context venus_sec_non_pixel using bank 21
[ 1.979076] msm_iommu_ctx 1e36000.qcom,iommu-ctx: context venus_fw using bank 22
[ 9.659969] Modules linked in: venus_core(+) msm_rng(+) qcom_camss(+) rng_core mdt_loader videobuf2_dma_sg v4l2_mem2mem videobuf2_memops videobuf2_v4l2 videobuf2_core videodev media ip_tables x_tables i2c_qcom_cci
[ 9.743083] qcom-venus 1d00000.video-codec: Direct firmware load for qcom/venus-1.8/venus.mdt failed with error -2
[ 9.743244] venus_core(+)
[ 9.743441] [<ffff000000b8a014>] qcom_venus_driver_init+0x14/0x1000 [venus_core]
[ 9.743571] qcom-venus: probe of 1d00000.video-codec failed with error -2
linaro@linaro-alip:~$
Surely some errors are occurred. Did I failed downloading the #283 image?
I think the venus firmware files should be placed now in /lib/firmware/qcom/venus-1.8 and not in /lib/firmware.
Could you check that you have
root@linaro-alip:/lib/firmware/qcom/venus-1.8# ls -la
total 1964
drwxr-xr-x 2 root root 4096 Sep 28 00:52 .
drwxr-xr-x 3 root root 4096 Sep 28 00:52 …
-rw-r–r-- 1 root root 212 Sep 28 00:52 venus.b00
-rw-r–r-- 1 root root 6600 Sep 28 00:52 venus.b01
-rw-r–r-- 1 root root 975088 Sep 28 00:52 venus.b02
-rw-r–r-- 1 root root 5568 Sep 28 00:52 venus.b03
-rw-r–r-- 1 root root 32 Sep 28 00:52 venus.b04
-rw-r–r-- 1 root root 988880 Sep 28 00:52 venus.mbn
-rw-r–r-- 1 root root 6812 Sep 28 00:52 venus.mdt
Sorry, I’ve missed the workaround.
mkdir -p /lib/firmware/qcom/venus-1.8
cd /lib/firmware/qcom/venus-1.8/
for i in /lib/firmware/venus.*; do ln -s $i; done
Then, these error messages are disappeared.
makes sense. Gstreamer should work. and if you install ffmpeg from sources, it will work as well.
Thank you for the other day.
BTW, I am planning to experiment with various encode parameters for FFmpeg, but I can not find parameters equivalent to “extra-controls” in GStreamer.
I think that parameters such as H.264 profile, level, and quality can not be specified with the FFmpeg standard commands, but how can I specify it?
Can I use FFmpeg to grab frames directly from a camera?
In the example:
ffmpeg -loglevel debug -f rawvideo -pix_fmt nv12 -s:v 1280:720 -r 25 -i ~/Videos/raw/freeway.yuv -c:v h264_v4l2m2m out/out.h264.mp4
It appears that the encoder is encoding frames from an already recorded video. Can I do this with live video?
I tried this and FFmpeg cannot tell that the source is a video capture device, so I think I’m missing something here:
ffmpeg -loglevel debug -f v4l2 -pix_fmt nv12 -s:v 1280:960 -r 30 -i /dev/video3 -c:v h264_v4l2m2m encoded.mp4
Edit: Here’s a log from an unsuccessful run of the above command
dragonboard-410c:~$ ffmpeg -loglevel debug -f v4l2 -pix_fmt nv12 -s:v 1280:720 -r 25 -i /dev/video3 -c:v h264_v4l2m2m out.h264.mp4 ffmpeg version 3.3.3 Copyright (c) 2000-2017 the FFmpeg developers built with gcc 6.2.1 (Linaro GCC 6.2-2016.11) 20161016 configuration: --disable-stripping --enable-pic --enable-shared --enable-pthreads --disable-libxcb --disable-libxcb-shm --disable-libxcb-xfixes --disable-libxcb-shape --cross-prefix=aarch64-linaro-linux- --ld='aarch64-linaro-linux-gcc --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --cc='aarch64-linaro-linux-gcc --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --cxx='aarch64-linaro-linux-g++ --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --arch=aarch64 --target-os=linux --enable-cross-compile --extra-cflags=' -O2 -pipe -g -feliminate-unused-debug-types -fdebug-prefix-map=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/work/aarch64-linaro-linux/ffmpeg/3.3.3-r0=/usr/src/debug/ffmpeg/3.3.3-r0 -fdebug-prefix-map=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/x86_64-linux= -fdebug-prefix-map=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c= --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c' --extra-ldflags='-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed' --sysroot=/media/rob/a72581e8-3ca3-4dc1-b3b8-6db5464de098/qc_openEmbedded/apps_proc/build/tmp-glibc/sysroots/dragonboard-410c --enable-hardcoded-tables --libdir=/usr/lib64 --shlibdir=/usr/lib64 --datadir=/usr/share/ffmpeg --disable-mipsdsp --disable-mipsdspr2 --cpu=generic --pkg-config=pkg-config --enable-avcodec --enable-avdevice --enable-avfilter --enable-avformat --disable-avresample --enable-bzlib --enable-gpl --disable-libgsm --disable-indev=jack --disable-libvorbis --enable-lzma --disable-libmp3lame --disable-openssl --enable-postproc --disable-libschroedinger --disable-sdl2 --disable-libspeex --enable-swresample --enable-swscale --enable-libtheora --disable-vaapi --disable-vdpau --disable-libvpx --enable-libx264 --enable-outdev=xv libavutil 55. 58.100 / 55. 58.100 libavcodec 57. 89.100 / 57. 89.100 libavformat 57. 71.100 / 57. 71.100 libavdevice 57. 6.100 / 57. 6.100 libavfilter 6. 82.100 / 6. 82.100 libswscale 4. 6.100 / 4. 6.100 libswresample 2. 7.100 / 2. 7.100 libpostproc 54. 5.100 / 54. 5.100 Splitting the commandline. Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'. Reading option '-f' ... matched as option 'f' (force format) with argument 'v4l2'. Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'nv12'. Reading option '-s:v' ... matched as option 's' (set frame size (WxH or abbreviation)) with argument '1280:720'. Reading option '-r' ... matched as option 'r' (set frame rate (Hz value, fraction or abbreviation)) with argument '25'. Reading option '-i' ... matched as input url with argument '/dev/video3'. Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'h264_v4l2m2m'. Reading option 'out.h264.mp4' ... matched as output url. Finished splitting the commandline. Parsing a group of options: global . Applying option loglevel (set logging level) with argument debug. Successfully parsed a group of options. Parsing a group of options: input url /dev/video3. Applying option f (force format) with argument v4l2. Applying option pix_fmt (set pixel format) with argument nv12. Applying option s:v (set frame size (WxH or abbreviation)) with [ 3623.130796] 1b0c000.qcom,cci supply qcom,gdscr-vdd not found, using dummy regulator argument 1280:720. Applying option r (set frame rate (Hz value, fraction or abbreviation)) with argument 25. Successfully parsed a group of options. Opening an input file: /dev/video3. [video4linux2,v4l2 @ 0x1f8b6460] fd:3 capabilities:85201000 [video4linux2,v4l2 @ 0x1f8b6460] Not a video capture device. /dev/video3: No such device
did you check this link?
https://trac.ffmpeg.org/wiki/Capture/Webcam
it seems your input device is not a video capture device.
