Hello,
Is it possible to use SONY IMX377 (which is the image sensor used in NEXUS 6P AND 5X) with dragonboard? On the documentation it says that dragonboard supports image sensors up to 13 MP and SONY IMX377 is less than 13MP.
Hi @edurmus,
Yes, the ISP of the SoC of the dragonboard 410c is written to support in the documentation.
Normally it requires binary provided by the SoC vendor, to make the Camera image sensor to work with the ISP.
The ISP is not supported with current build images for dragonboard 410C at the moment.
We are improving every release, so the future releases might be supported some time.
Hi emre,
I believe the answer provided here is sort of misleading. Do not let the fact that the dragonboard 410c board has MIPI-CSI2 interface, fool you into thinking that you can take any MIPI-CSI2 image sensor and simply interface it to the board. The whole process of getting a new CMOS image sensor to work with the board is very complicated, from the firmware/driver/software side of things. While it is true that the Qualcomm Snapdragon 410 processor is able to support the SONY IMX377 sensor, 96 boards including dragonboard 410c, will more then likely never support this type of CMOS image sensor. All CMOS image sensors without integrated ISP, will have all of the data go through the SOCs ISP, which is priority. Qualcomm will not give you access to the ISP documentation unless you are buying Million of SOCs from them. While I am not an expert on the topic of CMOS image sensor, I know this is true since at my work, the company which I work for makes their own SOCs and some of the IP comes from Marvell. We always have to sign a NDA to get any access to the register documentation, for the specific piece of IP. As well I have a friend, which used to work for the former Motorola Phone division and he said that Qualcomm did not do a good job at providing them with all of the needed support and documentation for some of their IP used in their products.
Unless Qualcomm will directly get involved and provide both firmware and drivers for the specific CMOS image sensor for the dragonboard 410c, you might as well say the image sensor is not usable. Here is a link that I found useful explaining the whole process for CMOS image sensor integration and tuning.
Now if you want to use an omnivision 5MP image sensor, you probably will be able to do that in the future once Linaro community will enable the support. Thous types of sensors do have an integrated ISP and the data bypasses the SOC’s ISP. Again even the documentation for the MIPI-CSI2 bus is not released and only available to Linaro members.
There are hybrid solutions for now, but not at the 13MP level.
Alternatives:
- Use an USB solution could be 13MP. But this is sort of impractical and not usable for real time video streaming and processing.
- wait for the 5MP Omnivision sensor support.(probably the best solution)
- Use (at $5)Raspberry Pi zero Ver 1.3 with the Sony imx219 8MP sensor module and transfer the data over the LAN/WLAN/USB etc to the dragonboard 410c and do the processing on the dragonboard. (I have tested this and it works well)
- Buy millions of SOCs from Qualcomm and they will provide you with the documentation more then likely.
This is the conclusion which I came to after doing months worth of research on what would work for a project that I am working on, which needs video streaming.
regards,
Daniel
Frustrated Engineer looking for a solution.
Hi Daniel
Thanks for sharing your insights. I’m also inmersed in the same frustating research work…trying to decide whether if the dragonboard 410 is an option for a Machine Vision project.
What is your opinion about receiving RAW images from a APTINA CMOS sensor trhough the MIPI-CSI 2 port? I can imagine several applications that could benefit from the RAW image data…even without the RGB conversion activated in the Qualcomm’s ISP.
Hi Cloudbreak,
I have looked at some of the APTINA Sensors in the past and have seen some of the great reviews. As well they have global shutters, which is a big feature for anything with real time image processing related. I have as well looked at other Images sensors which I was pretty excited about in the past as well, imagining all of the possibilities for different applications and designs.
After lots of looking into what image sensors would work, I came to the conclusion that the only viable option for me would be to use a platform that has one or multiple camera module fully supported, including firmware, drivers, software, availability of camera module and cost. The reason for this is, I don’t have lots of time or resources or expertise to get the camera stack working, nor the financial power to buy large quantities of a certain image sensor or soc.
As of right now dragonboard 410 is NOT an option for anything image processing/robotics related, unless you want to go the USB/WAN/LAN route. I think latency will be a very big issues then. As well I think USB will be CPU intensive and not use the GPU as with the MIPI-CSI2 sensor.
Your option for using something with an Aptina image sensor would be use a Wandboard:
http://forums.wandboard.org/viewtopic.php?f=7&t=228
The Wandboard has an “NXP”(Freescale) I.MX6 processor which has a fully documented ISP. The spec is a little over 5000 pages BUT, the SOC is fully documented, none of the IP protection garbage. As well the ISP or GPU are not as powerful as the Snapdragon 410 either.
Texas Instruments as well makes a few SOCs that have a MIPI-CSI2 interface. They was well fully document almost everything. They as well do point you to a APTINA sensor on their website here:
It might be worth asking them what is your best option in-terms of what board to use for he APTINA support.
Right now for me, the project that I am working on requires that I have a little weight as possible, and a very small form factor, so Wandboard, and TIs solution don’t work. The dragonboard 410 does NOT work either, as of right now. I think that the Wandboard though do have a new form factor, I will look into it further. But I found something that is very low cost and is available, which was the Raspberry pi. All of the models work with the same camera modules. The new PI Zero ver 1.3 was the best temp solution I could find.
They as well do have some work for getting data straight from the MIPI-CSI2 bus here:
https://www.raspberrypi.org/forums/viewtopic.php?f=43&t=109137
but this is for the Omnivision module.
As well I for my project I needed low latency video streaming for which I found this project:
Overall my advice would be to as follows.
- wait for the 5MP Omnivision sensor support on dragonboard.(But is not an option if you want something working now)
- use the Wandboard with the APTINA module with a much lower resolution and not Global Shutter.
- use the Raspberry pi(any model) with the Sony imx219 8MP sensor module or The Omnivision 5MP camera module.(I have personally tested this and works right out of the box)
- Talk to TI and see what your options are.
- Talk to e-consystems or leopardimaging and see if they have any suggestions for board to use with their products.
- Nvidia Jetson TX1 pretty much a super computer on a module. I think they are very good about support. As well it is pretty much made for Robotics an computer vision. But there is very steep price.
One really interesting thing that I did find out about the Raspberry Pi is that Element14 will customize the board for you. But I think you have to buy at least I think it is either 1000 or 5000 units.
https://www.element14.com/community/videos/17859/l/an-introduction-to-customizing-the-raspberry-pi
https://www.element14.com/community/docs/DOC-76955/l/raspberry-pi-customization-service
The customizing might not be something for right now but in the future if you have a project or product that becomes really popular that would be something that you might consider.
If you look at the schematics for the PI you will see a second CSI2 port. This is usable and could be brought out. That means you could do stereoscopic vision processing. Something like this:
Well I hope you find the information useful and find a solution for your product/project.
Daniel
To continue on the previous comment, I forgot to mention that there are quite a few solutions for the I.MX6 processor which will work with the OV564X Cmos image sensor. Solid-Run makes a few low cost boards around this SOC https://www.solid-run.com/freescale-imx6-family/imx6-som/.
So I think one can just use the PI cam module with this if they are looking for a low cost solution.
As well NXP(Freescale) the company that makes the SOC is better with regards to support and open documentation of their IP as stated above. I think ISP is called an IPU(Image processing Unit). I sort of felt like I left this off in the previous, if you are looking for an I.MX6 solution.
So you might be able to write some sort of driver for the APTINA sensor, maybe(Don’t quote me on this).
Daniel
Hi Daniel
Wow…thanks for this so clever and elaborated answer…lots of useful hints here!
Yes, we are using the Aptinas CMOS due to its higher sensibility and Global Start features…due to specific requirements in our machine vision application we need to stick to the MT9J006, which is a 5MP RAW sensor.
Our application requires to meet small form factor, so options like Wandaboard and Raspberry are not an option so far. Price is also a limiting factor, so Nvidia Jetson is neither a possibility.
The SolidRun modules were our first option, but the ARMv7 core has nothing to do against the A53 inside the Snapdragon, and our application requests using computationally intensive algorithms (the benchmark showed that the Snapdragon410 was 40% faster than the quad core i.MX6) even if we are allowed to work under low FPSs (1 fps is ok) and latency of a couple of seconds wouldn’t be a big issue
So, due to the high performance and competitive price, we will give a grace period to the Snapdragon option; we will keep the HW development on stand-by waiting for the ‘announced’ new Linux Release in June…just looking forward to see if the MIPI-CSI interface is finally enabled there, at least for accessing YUV-RGB smart CMOS sensors. Based on that hypothetic SW feature, we would try to accommodate the driver to get access to the MT9J006 video sequence in RAW format.
However, to me, looks like the Snapdragon option could eventually become a dead end for LINUX users, maybe perhaps due to the lack of interest of Qualcomm in the Industrial/Robotics/Machine vision market niches.
Besides, I will definitively check the Raspberry Pi customization (our targeted volume is around 5,000 units so it might be a clever option for accessing a A53 CPU)
Thanks a lot Daniel!!
