This is a cache of https://discuss.96boards.org/t/imx219-sony-image-sensor-with-dragonboard/8157. It is a snapshot of the page at 2024-11-04T11:18:56.393+0000.
Imx219 Sony image sensor with dragonboard - DragonBoard410c - 96Boards Forum

Imx219 Sony image sensor with dragonboard

Products Support DragonBoard410c
#1

Hi,

We are trying to interface a imx219 camera module with the Dargonboard410c (Kernel Version:4.14.96).

We got error: “Failed to allocate required memory”

sudo media-ctl -d /dev/media0 -l ‘“msm_csiphy0”:1->“msm_csid0”:0[1],“msm_csid0”:1->“msm_ispif0”:0[1],“msm_ispif0”:1->“msm_vfe0_rdi0”:0[1]’

sudo media-ctl -d /dev/media0 -V '"imx219 4-0010":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_csiphy0":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_csid0":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_ispif0":0[fmt:SBGGR8_1X8/1920x1080 field:none],"msm_vfe0_rdi0":0[fmt:SBGGR8_1X8/1920x1080 field:none]'

linaro@linaro-alip:~$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-bayer,format=bggr,framerate=30/1,width=1920,height=1080 ! bayer2rgb ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Got context from element 'autovideosink0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Failed to allocate required memory.
Additional debug info:
gstv4l2src.c(658): gst_v4l2src_decide_allocation (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Buffer pool activation failed
Execution ended after 0:00:00.467249103
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...   

[  643.681303] ------------[ cut here ]------------
[  643.681357] WARNING: CPU: 2 PID: 3870 at drivers/media/v4l2-core/videobuf2-core.c:1347 vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[  643.684989] Modules linked in: rfcomm aes_neon_blk bnep arc4 wcn36xx mac80211 btqcomsmd btqca bluetooth cfg80211 ecdh_generic qcom_q6v5_pil qrtr_smd venus_dec venus_enc snd_soc_hdmi_codec imx219 v4l2_common qcom_wcnss_pil qcom_common qcom_glink_smem msm qcom_sysmon remoteproc qmi_helpers adv7511 drm_kms_helper qcom_camss venus_core videobuf2_dma_sg drm v4l2_mem2mem v4l2_fwnode mdt_loader videobuf2_memops leds_qcom_lpg snd_soc_lpass_apq8016 videobuf2_v4l2 snd_soc_lpass_cpu snd_soc_msm8916_digital rtc_pm8xxx videobuf2_core snd_soc_lpass_platform snd_soc_apq8016_sbc snd_soc_msm8916_analog msm_rng videodev rng_core media i2c_qcom_cci qrtr crc32_ce rpmsg_char rmtfs_mem ip_tables x_tables
[  643.735060] CPU: 2 PID: 3870 Comm: v4l2src0:src Not tainted 4.14.96 #1
[  643.757287] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[  643.763886] task: ffff800033616200 task.stack: ffff000009598000
[  643.770758] PC is at vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[  643.776398] LR is at vb2_start_streaming+0xa0/0x1a0 [videobuf2_core]
[  643.782982] pc : [<ffff000000c11cd8>] lr : [<ffff000000c11c68>] pstate: 60000145
[  643.789405] sp : ffff00000959bb80
[  643.796779] x29: ffff00000959bb80 x28: ffff800035598b00 
[  643.799993] x27: 0000000000000001 x26: ffff000000b81448 
[  643.805375] x25: ffff80003a737d00 x24: ffff00000959bd30 
[  643.810671] x23: 0000000000000000 x22: ffff800035424580 
[  643.815965] x21: ffff8000354244e0 x20: ffff800035424350 
[  643.821261] x19: 00000000ffffffe0 x18: 000000000000001f 
[  643.826556] x17: 0000ffff90d94eb8 x16: ffff000008236060 
[  643.831852] x15: 0000000000000002 x14: 000000000000001f 
[  643.837146] x13: 0000000000000000 x12: 0000000000000000 
[  643.842441] x11: 0000000000000000 x10: 0000000000000000 
[  643.847736] x9 : 0000000000000000 x8 : ffff80003af8c188 
[  643.853032] x7 : 0000000000000000 x6 : 000000000000003f 
[  643.858329] x5 : 0000000000000000 x4 : 0000000000000000 
[  643.863623] x3 : ffff800033616200 x2 : 0000000000000000 
[  643.868917] x1 : ffff000000c18400 x0 : 0000000000000004 
[  643.874222] Call trace:
[  643.879506] Exception stack(0xffff00000959ba40 to 0xffff00000959bb80)
[  643.881682] ba40: 0000000000000004 ffff000000c18400 0000000000000000 ffff800033616200
[  643.888280] ba60: 0000000000000000 0000000000000000 000000000000003f 0000000000000000
[  643.896092] ba80: ffff80003af8c188 0000000000000000 0000000000000000 0000000000000000
[  643.903905] baa0: 0000000000000000 0000000000000000 000000000000001f 0000000000000002
[  643.911717] bac0: ffff000008236060 0000ffff90d94eb8 000000000000001f 00000000ffffffe0
[  643.919530] bae0: ffff800035424350 ffff8000354244e0 ffff800035424580 0000000000000000
[  643.927343] bb00: ffff00000959bd30 ffff80003a737d00 ffff000000b81448 0000000000000001
[  643.935155] bb20: ffff800035598b00 ffff00000959bb80 ffff000000c11c68 ffff00000959bb80
[  643.942968] bb40: ffff000000c11cd8 0000000060000145 0000000000000000 ffff800035424350
[  643.950781] bb60: 0000ffffffffffff ffff800035424580 ffff00000959bb80 ffff000000c11cd8
[  643.958606] [<ffff000000c11cd8>] vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[  643.966415] [<ffff000000c13ecc>] vb2_core_streamon+0x134/0x1b8 [videobuf2_core]
[  643.973878] [<ffff000000c54264>] vb2_streamon+0x3c/0x80 [videobuf2_v4l2]
[  643.980993] [<ffff000000c542f0>] vb2_ioctl_streamon+0x48/0x58 [videobuf2_v4l2]
[  643.987991] [<ffff000000b5db70>] v4l_streamon+0x20/0x28 [videodev]
[  643.995018] [<ffff000000b607b0>] __video_do_ioctl+0x2a8/0x2e0 [videodev]
[  644.001179] [<ffff000000b6024c>] video_usercopy+0x1e4/0x480 [videodev]
[  644.008036] [<ffff000000b604f8>] video_ioctl2+0x10/0x20 [videodev]
[  644.014372] [<ffff000000b5c64c>] v4l2_ioctl+0x7c/0x128 [videodev]
[  644.020487] [<ffff000008235854>] do_vfs_ioctl+0xa4/0x8b0
[  644.026646] [<ffff0000082360d8>] SyS_ioctl+0x78/0x98
[  644.032025] Exception stack(0xffff00000959bec0 to 0xffff00000959c000)
[  644.036977] bec0: 0000000000000014 0000000040045612 0000aaaae8271450 0000ffff90c21eb0
[  644.043314] bee0: 0000000000000000 000000000000017f 0000000000000000 0000000000000001
[  644.051127] bf00: 000000000000001d 0000000000000001 0000000000000000 0000000000000000
[  644.058940] bf20: 0000000000000000 0000000000000000 000000000000001f 0000000000000002
[  644.066753] bf40: 0000ffff90e908c8 0000ffff90d94eb8 000000000000001f 0000000000000000
[  644.074564] bf60: 0000aaaae828aed0 0000aaaae8271440 0000ffff90701000 0000ffff90702000
[  644.082378] bf80: 0000000000000002 0000aaaae8271440 0000ffff7c00daf0 0000000000000001
[  644.090196] bfa0: 0000aaaae80aac80 0000ffff8d4c35d0 0000ffff906c75cc 0000ffff8d4c35c0
[  644.098003] bfc0: 0000ffff90c21ebc 0000000060000000 0000000000000014 000000000000001d
[  644.105814] bfe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[  644.113628] [<ffff000008083ac0>] el0_svc_naked+0x34/0x38
[  644.121435] ---[ end trace 50c5407b420c97a1 ]---

linaro@linaro-alip:~$ yavta -B capture-mplane -c10 -I -n 4 --requeue-last -f SBGGR8 -s 1920x1080 /dev/video0 -F
Device /dev/video0 opened.
Device Qualcomm Camera Subsystem' on platform:1b0ac00.camss’ (driver ‘qcom-camss’) supports video, capture, with mplanes.
Video format set: SBGGR8 (31384142) 1920x1080 field none, 1 planes:

  • Stride 1920, buffer size 2073600
    Video format: SBGGR8 (31384142) 1920x1080 field none, 1 planes:
  • Stride 1920, buffer size 2073600
    4 buffers requested.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 0/0 mapped at address 0xffffa16ba000.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 1/0 mapped at address 0xffffa14bf000.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 2/0 mapped at address 0xffffa12c4000.
    length: 1 offset: 4178921768 timestamp type/source: mono/EoF
    Buffer 3/0 mapped at address 0xffffa10c9000.
    Unable to start streaming: Broken pipe (32).
    4 buffers released.

[ 794.759234] ------------[ cut here ]------------
[ 794.759290] WARNING: CPU: 0 PID: 3894 at drivers/media/v4l2-core/videobuf2-core.c:1347 vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[ 794.762916] Modules linked in: rfcomm aes_neon_blk bnep arc4 wcn36xx mac80211 btqcomsmd btqca bluetooth cfg80211 ecdh_generic qcom_q6v5_pil qrtr_smd venus_dec venus_enc snd_soc_hdmi_codec imx219 v4l2_common qcom_wcnss_pil qcom_common qcom_glink_smem msm qcom_sysmon remoteproc qmi_helpers adv7511 drm_kms_helper qcom_camss venus_core videobuf2_dma_sg drm v4l2_mem2mem v4l2_fwnode mdt_loader videobuf2_memops leds_qcom_lpg snd_soc_lpass_apq8016 videobuf2_v4l2 snd_soc_lpass_cpu snd_soc_msm8916_digital rtc_pm8xxx videobuf2_core snd_soc_lpass_platform snd_soc_apq8016_sbc snd_soc_msm8916_analog msm_rng videodev rng_core media i2c_qcom_cci qrtr crc32_ce rpmsg_char rmtfs_mem ip_tables x_tables
[ 794.812992] CPU: 0 PID: 3894 Comm: yavta Tainted: G W 4.14.96 #1
[ 794.835220] Hardware name: Qualcomm Technologies, Inc. APQ 8016 SBC (DT)
[ 794.842510] task: ffff80002a85aa00 task.stack: ffff00000f968000
[ 794.849296] PC is at vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[ 794.854935] LR is at vb2_start_streaming+0xa0/0x1a0 [videobuf2_core]
[ 794.861521] pc : [] lr : [] pstate: 60000145
[ 794.867944] sp : ffff00000f96bb80
[ 794.875317] x29: ffff00000f96bb80 x28: ffff800035f69500
[ 794.878532] x27: 0000000000000001 x26: ffff000000b81448
[ 794.883915] x25: ffff800035f69700 x24: ffff00000f96bd30
[ 794.889211] x23: 0000000000000000 x22: ffff800035424580
[ 794.894505] x21: ffff8000354244e0 x20: ffff800035424350
[ 794.899800] x19: 00000000ffffffe0 x18: 000000000000021f
[ 794.905095] x17: 0000ffffa19abeb0 x16: ffff000008236060
[ 794.910390] x15: 00000000000002b0 x14: 0000000000000001
[ 794.915685] x13: 0000ffffa10c9000 x12: 0000000000000000
[ 794.920979] x11: 0000000000000000 x10: 0000000000000000
[ 794.926275] x9 : 0000000000000000 x8 : ffff8000335d4088
[ 794.931571] x7 : 0000000000000000 x6 : 000000000000003f
[ 794.936865] x5 : 0000000000000000 x4 : 0000000000000000
[ 794.942161] x3 : ffff80002a85aa00 x2 : 0000000000000000
[ 794.947455] x1 : ffff000000c18400 x0 : 0000000000000004
[ 794.952752] Call trace:
[ 794.958044] Exception stack(0xffff00000f96ba40 to 0xffff00000f96bb80)
[ 794.960220] ba40: 0000000000000004 ffff000000c18400 0000000000000000 ffff80002a85aa00
[ 794.966818] ba60: 0000000000000000 0000000000000000 000000000000003f 0000000000000000
[ 794.974630] ba80: ffff8000335d4088 0000000000000000 0000000000000000 0000000000000000
[ 794.982444] baa0: 0000000000000000 0000ffffa10c9000 0000000000000001 00000000000002b0
[ 794.990257] bac0: ffff000008236060 0000ffffa19abeb0 000000000000021f 00000000ffffffe0
[ 794.998069] bae0: ffff800035424350 ffff8000354244e0 ffff800035424580 0000000000000000
[ 795.005881] bb00: ffff00000f96bd30 ffff800035f69700 ffff000000b81448 0000000000000001
[ 795.013694] bb20: ffff800035f69500 ffff00000f96bb80 ffff000000c11c68 ffff00000f96bb80
[ 795.021506] bb40: ffff000000c11cd8 0000000060000145 0000000000000000 ffff800035424350
[ 795.029319] bb60: 0000ffffffffffff ffff800035424580 ffff00000f96bb80 ffff000000c11cd8
[ 795.037147] [] vb2_start_streaming+0x110/0x1a0 [videobuf2_core]
[ 795.044956] [] vb2_core_streamon+0x134/0x1b8 [videobuf2_core]
[ 795.052418] [] vb2_streamon+0x3c/0x80 [videobuf2_v4l2]
[ 795.059534] [] vb2_ioctl_streamon+0x48/0x58 [videobuf2_v4l2]
[ 795.066531] [] v4l_streamon+0x20/0x28 [videodev]
[ 795.073561] [] __video_do_ioctl+0x2a8/0x2e0 [videodev]
[ 795.079727] [] video_usercopy+0x1e4/0x480 [videodev]
[ 795.086585] [] video_ioctl2+0x10/0x20 [videodev]
[ 795.092915] [] v4l2_ioctl+0x7c/0x128 [videodev]
[ 795.099027] [] do_vfs_ioctl+0xa4/0x8b0
[ 795.105185] [] SyS_ioctl+0x78/0x98
[ 795.110565] Exception stack(0xffff00000f96bec0 to 0xffff00000f96c000)
[ 795.115514] bec0: 0000000000000003 0000000040045612 0000fffff91545a4 0000000000000000
[ 795.121855] bee0: 0000ffffa12c3400 0000000000000004 0000000000000000 000000000000000a
[ 795.129667] bf00: 000000000000001d ffffff80ffffffc8 0000000000000000 0000000000000000
[ 795.137478] bf20: 0000000000000000 0000ffffa10c9000 0000000000000001 00000000000002b0
[ 795.145291] bf40: 0000aaaab57e8118 0000ffffa19abeb0 000000000000021f 0000aaaab57e7000
[ 795.153104] bf60: 0000000000000001 0000fffff91547e8 0000aaaae551182c 0000000000000001
[ 795.160915] bf80: 0000000000000000 0000aaaab57d69c8 0000000000000000 00000000001fa400
[ 795.168728] bfa0: 00000000ffffffff 0000fffff9154570 0000aaaab57d4e68 0000fffff9154570
[ 795.176541] bfc0: 0000ffffa19abebc 0000000060000000 0000000000000003 000000000000001d
[ 795.184353] bfe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 795.192167] [] el0_svc_naked+0x34/0x38
[ 795.199974] —[ end trace 50c5407b420c97a2 ]—

We checked the clock and the MIPI lines, We can see data transfer
Thanks,
Osama

#2

I’m also getting similar issue with raspberry pi camera v1.3 (OV5647) connected using MIPI adapter mezzanine. While using yavta, I’m also getting VFE timeouts in dmesg:

[  622.517402] qcom-camss 1b0ac00.camss: VFE sof timeout
[  623.029290] qcom-camss 1b0ac00.camss: VFE reg update timeout

Does this mean the frame is not received properly from camera sensor? SOF timeout looks suspicious as it corresponds to Start of Frame.

Tagging @Loic if he can help with this!

Thanks,
Mani

#3

Pipeline is not able to start stream.
Could you attach ouput of media-ctl -p after setting the pipeline ?
Which sensor driver are you running? a custom one? a downstream one? any link?

#4

Also please note that the camera sensor driver should support pixelclock v4l2 ctrl in order to work with camss.

#5

Custom driver based on ov5640.c

Media controller API version 4.14.96

Media device information
------------------------
driver          qcom-camss
model           Qualcomm Camera Subsystem
serial          
bus info        
hw revision     0x0
driver version  4.14.96

Device topology
- entity 1: msm_csiphy0 (2 pads, 3 links)
            type Node subtype V4L flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "imx219 4-0010":0 [ENABLED,IMMUTABLE]
	pad1: Source
		-> "msm_csid0":0 [ENABLED]
		-> "msm_csid1":0 []

- entity 4: msm_csiphy1 (2 pads, 2 links)
            type Node subtype V4L flags 0
            device node name /dev/v4l-subdev1
	pad0: Sink
	pad1: Source
		-> "msm_csid0":0 []
		-> "msm_csid1":0 []

- entity 7: msm_csid0 (2 pads, 4 links)
            type Node subtype V4L flags 0
            device node name /dev/v4l-subdev2
	pad0: Sink
		<- "msm_csiphy0":1 [ENABLED]
		<- "msm_csiphy1":1 []
	pad1: Source
		-> "msm_ispif0":0 [ENABLED]
		-> "msm_ispif1":0 []

- entity 10: msm_csid1 (2 pads, 4 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev3
	pad0: Sink
		<- "msm_csiphy0":1 []
		<- "msm_csiphy1":1 []
	pad1: Source
		-> "msm_ispif0":0 []
		-> "msm_ispif1":0 []

- entity 13: msm_ispif0 (2 pads, 6 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev4
	pad0: Sink
		<- "msm_csid0":1 [ENABLED]
		<- "msm_csid1":1 []
	pad1: Source
		-> "msm_vfe0_rdi0":0 [ENABLED]
		-> "msm_vfe0_rdi1":0 []
		-> "msm_vfe0_rdi2":0 []
		-> "msm_vfe0_pix":0 []

- entity 16: msm_ispif1 (2 pads, 6 links)
             type Node subtype V4L flags 0
             device node name /dev/v4l-subdev5
	pad0: Sink
		<- "msm_csid0":1 []
		<- "msm_csid1":1 []
	pad1: Source
		-> "msm_vfe0_rdi0":0 []
		-> "msm_vfe0_rdi1":0 []
		-> "msm_vfe0_rdi2":0 []
		-> "msm_vfe0_pix":0 []

- entity 19: msm_vfe0_rdi0 (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev6
	pad0: Sink
		[fmt:SBGGR8_1X8/1920x1080 field:none colorspace:srgb]
		<- "msm_ispif0":1 [ENABLED]
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:SBGGR8_1X8/1920x1080 field:none colorspace:srgb]
		-> "msm_vfe0_video0":0 [ENABLED,IMMUTABLE]

- entity 22: msm_vfe0_video0 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		<- "msm_vfe0_rdi0":1 [ENABLED,IMMUTABLE]

- entity 28: msm_vfe0_rdi1 (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev7
	pad0: Sink
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		<- "msm_ispif0":1 []
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		-> "msm_vfe0_video1":0 [ENABLED,IMMUTABLE]

- entity 31: msm_vfe0_video1 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Sink
		<- "msm_vfe0_rdi1":1 [ENABLED,IMMUTABLE]

- entity 37: msm_vfe0_rdi2 (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev8
	pad0: Sink
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		<- "msm_ispif0":1 []
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb]
		-> "msm_vfe0_video2":0 [ENABLED,IMMUTABLE]

- entity 40: msm_vfe0_video2 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Sink
		<- "msm_vfe0_rdi2":1 [ENABLED,IMMUTABLE]

- entity 46: msm_vfe0_pix (2 pads, 3 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev9
	pad0: Sink
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb
		 compose.bounds:(0,0)/1920x1080
		 compose:(0,0)/1920x1080]
		<- "msm_ispif0":1 []
		<- "msm_ispif1":1 []
	pad1: Source
		[fmt:UYVY8_2X8/1920x1080 field:none colorspace:srgb
		 crop.bounds:(0,0)/1920x1080
		 crop:(0,0)/1920x1080]
		-> "msm_vfe0_video3":0 [ENABLED,IMMUTABLE]

- entity 49: msm_vfe0_video3 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video3
	pad0: Sink
		<- "msm_vfe0_pix":1 [ENABLED,IMMUTABLE]

- entity 87: imx219 4-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev10
	pad0: Source
		[fmt:unknown/0x0@1/30]
		-> "msm_csiphy0":0 [ENABLED,IMMUTABLE]
#6

Unknown format? I think it’s a problem.

I suggest you to add some debugging code in sensor and camss (driver/media/platform/qcom/camss vide/vfe…) source in order to highlight issues in the pipeline, mainly in streamon/set_power/set_fmt code…

Also, camss requests some controls to be implemented, like ‘pixel_clock’ (e.g. [1]), check you correctly implemented them.

https://git.linaro.org/people/loic.poulain/linux.git/commit/?h=qcomlt-4.14-ov5640&id=c3fb3b5b6ea78247a5d485cd620f8d5d1b4ab71c