Instruction

MIPI-CSI Camera, based on Sony IMX477 sensor, 1230M pixels. Supports Raspberry Pi Compute Module series and Jetson Nano developer Kit.

Specification

• Sensor
  • IMX477R
  • 12.3 million pixels: 4056(H) x 3040(V)
  • CMOS diagonal length: 7.9mm
  • Single pixel size: 1.55μm(H) x 1.55μm (V)
• Output format: RAW12/10/8, COMP8
• Lens standard
  • C-mount
  • CS-mount (including C/CS connection ring)
• IR filter: integrated

Specification of Lens

Notice

IMX477 12.3MP Camera can only support the Raspberry Pi Compute Module series, it cannot be used with Raspberry Pi boards like Raspberry Pi Model 4B.
IMX477 12.3MP Camera can be used with Jetson Nano Developer Kit, if you want to use it with Jetson Nano, you need to install the driver. Please backup your OS before you install the driver because the driver is not suitable for IMX219.

• Jetson Nano free driver by RidgeRun

Hardware Connection

To test the Camera, you need to connect a HDMI display or a DIS display for previewing.
The connectors of the DSI interface (display) and the CSI interface (camera) look the same, please take care of it when you connect the camera. The CSI interface is placed between the Audio jack and the HDMI port. the CSI connector of Pi zero is beside the Power interface. If you use Compute Module, please check the actual place of the carrier board.

• Connect to Pi Zero Series

The metal surface of the FPC is downward, then connect to the CSI interface.

• Other Raspberry Pi motherboards

Place the metal side of the cable towards the HDMI port and connect to the camera port.

About the model

Enable Camera Interface

If you are using the latest bullseye image, the camera interface has been enabled by default, so you can skip the modification steps.

• Open the Raspberry Pi terminal and use the command to enter the setting interface

sudo raspi-config

• Set up the camera

Select Interface Options -> Camera -> Yes -> Finish -> Yes

• Reboot the Raspberry Pi

Test Camera(Bullseyes OS)

Configure
If you use OV9281, IMX290, IMX378, or non-Raspberry Pi official IMX219 and IMX477 cameras, you need to configure the config.txt file separately.

sudo nano /boot/config.txt

Find "camera-auto-detect=1" and modify it to "camera_auto_detect=0"

At the end of the file, add the following setting statements according to the camera model.

Open the Raspberry Pi terminal and enable the camera to preview.

sudo libcamera-hello -t 0

If you want to close the preview window, you can directly combine the keys Alt-F4, or click x to close. You can also go back to the terminal interface and terminate the program with Ctrl+c.

Test Camera（Buster OS）

Open a terminal and test the camera with the following command:

sudo raspistill -t 0

If you want to close the preview windows, you can use Ctrl-C to stop the appliation.

User Guides for libcamera/libcamera

Introduction

After the Bullseye version of the Raspberry Pi image, the underlying Raspberry Pi driver was switched from Raspicam to libcamera. Libcamera is an open-source software stack (will be called driver later, which is easy to understand), which is convenient for third-party porting and development of their own camera drivers. As of 2021-12-20, there are still many bugs in libcamera, and the current libcamera does not support python, so the Raspberry Pi official still provides a method for installing and downloading Raspicam. For users who are difficult to switch to libcamera but need to use the latest system, please directly refer to the Raspicam instructions.

Call Camera

The libcamera software stack provides six instructions for users to preview and test the camera interface.

libcamera-hello

This is a simple "hello world" program that previews the camera and displays the camera image on the screen.

Example

libcamera-hello

This command will preview the camera on the screen for about 5 seconds. The user can use the "-t <duration>" parameter to set the preview time, where the unit of <duration> is milliseconds. If it is set to 0, it will keep previewing all the time. For example:

libcamerahello -t 0

Tune File

The libcamera driver of the Raspberry Pi will call a tuning file for different camera modules. The tuning file provides various parameters. When calling the camera, libcamera will call the parameters in the tuning file, and process the image in combination with the algorithm. The final output is the preview screen. Since the libcamera driver can only automatically receive the signal of the chip, the final display effect of the camera will also be affected by the entire module. The use of the tuning file is to flexibly handle the cameras of different modules and adjust to improve the image quality.
If the output image of the camera is not ideal after using the default tuning file, the user can adjust the image by calling the custom tuning file. For example, if you are using the official NOIR version of the camera, the NOIR camera may require different white balance parameters compared with the regular Raspberry Pi Camera V2. In this case, you can switch by calling the tuning file.

libcamera-hello --tuning-file /usr/share/libcamera/ipa/raspberrypi/imx219_noir.json

Users can copy the default tuning files and modify them according to their needs.
Note: The use of tuning files is applicable to other libcamera commands, and will not be introduced in subsequent commands.

Preview Window

Most libcamera commands will display a preview window on the screen. Users can customize the title information of the preview window through the --info-text parameter, and can also call some camera parameters through %directives and display them on the window.
For example, if you use HQ Camera: You can display the focal length of the camera on the window through --info-txe "%focus".

libcamera-hello --info-text "focus %focus".

Note: For more information on parameter settings, please refer to the following chapters.

libcamera-jpeg

libcamera-jpeg is a simple still picture shooting program, different from the complex functions of libcamera-still, libcamera-jpeg code is more concise, and has many of the same functions to complete picture shooting.

Take JPEG image of full pixel

libcamera-jpeg -o test.jpg

This shooting command will display a preview serial port for about 5 seconds, and then shoot a full-pixel JPEG image and save it as test.jpg.
Users can set the preview time through the -t parameter, and can set the resolution of the captured image through --width and --height. E.g:

libcamera-jpeg -o test.jpg -t 2000 --width 640 --height 480

Exposure control

All libcamera commands allow the user to set the shutter time and gain themselves, such as:

libcamera-jpeg -o test.jpg -t 2000 --shutter 20000 --gain 1.5

This command will capture an image with 20ms exposure and camera gain set to 1.5x. The gain parameter set will first set the analog gain parameter inside the photosensitive chip. If the set gain exceeds the maximum built-in analog gain value of the driver, the maximum analog gain of the chip will be set first, and then the remaining gain multiples will be used as numbers. gain to take effect.
Remarks: The digital gain is realized by ISP (image signal processing), not directly adjusting the built-in register of the chip. Under normal circumstances, the default digital gain is close to 1.0, unless there are the following three situations.

1. Overall gain parameter requirements, that is, when the analog gain cannot meet the set gain parameter requirements, digital gain will be needed for compensation.
2. One of the color gains is less than 1 (the color gain is achieved by digital gain), in this case, the final gain is stabilized at 1/min(red_gain, blue_gain), that is, a uniform number is actually applied gain, and is the gain value for one of the color channels (not the green channel).
3. AEC/AGC was modified. If there is a change in AEC/AGC, the numerical gain will also change to a certain extent, where does it come from to eliminate any fluctuations, but this change will be quickly restored to the "normal" value.

The Raspberry Pi's AEC/AGX algorithm allows the program to specify exposure compensation, which is to adjust the brightness of the image by setting the aperture value, for example:

libcamera-jpeg --ev -0.5 -o darker.jpg
libcamera-jpeg --ev 0 -o normal.jpg
libcamera-jpeg --ev 0.5 -o brighter.jpg

libcamera-still

libcamera-still and libcamera-jpeg are very similar, the difference is that libcamera inherits more functions of raspistill. As before, the user can take a picture with the following command.

Test Command

libcamera-still -o test.jpg

Encoder

libcamea-still supports image files in different formats, can support png and bmp encoding, and also supports saving binary dumps of RGB or YUV pixels as files without encoding or in any image format. If you save RGB or YUV data directly, the program must understand the pixel arrangement of the file when reading such files.

libcamera-still -e png -o test.png
libcamera-still -e bmp -o test.bmp
libcamera-still -e rgb -o test.data
libcamera-still -e yuv420 -o test.data

Note: The format of image saving is controlled by the -e parameter. If the -e parameter is not called, it will be saved in the format of the output file name by default.

Raw image capture

The raw image is the image output by the direct image sensor without any ISP or CPU processing. For color camera sensors, the output format of the raw image is generally Bayer. Note that the raw image is different from the bit-encoded RGB and YUV images we said earlier, and RGB and YUV are also ISP-processed images.
Instructions to take a raw image:

libcamera-still -r -o test.jpg

The original image is generally saved in DNG (Adobe Digital Negative) format, which is compatible with most standard programs, such as dcraw or RawTherapee. The original image will be saved as a file of the same name with the .dng suffix, for example, if you run the above command, it will be saved as a test.dng, and generate a jpeg file at the same time. The DNG file contains metadata related to image acquisition, such as white balance data, ISP color matrix, etc. The following is the metadata encoding information displayed by the exiftool:

File Name                       : test.dng
Directory                       : .
File Size                       : 24 MB
File Modification Date/Time     : 2021:08:17 16:36:18+01:00
File Access Date/Time           : 2021:08:17 16:36:18+01:00
File Inode Change Date/Time     : 2021:08:17 16:36:18+01:00
File Permissions                : rw-r--r--
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Raspberry Pi
Camera Model Name               : /base/soc/i2c0mux/i2c@1/imx477@1a
Orientation                     : Horizontal (normal)
Software                        : libcamera-still
Subfile Type                    : Full-resolution Image
Image Width                     : 4056
Image Height                    : 3040
Bits Per Sample                 : 16
Compression                     : Uncompressed
Photometric Interpretation      : Color Filter Array
Samples Per Pixel               : 1
Planar Configuration            : Chunky
CFA Repeat Pattern Dim          : 2 2
CFA Pattern 2                   : 2 1 1 0
Black Level Repeat Dim          : 2 2
Black Level                     : 256 256 256 256
White Level                     : 4095
DNG Version                     : 1.1.0.0
DNG Backward Version            : 1.0.0.0
Unique Camera Model             : /base/soc/i2c0mux/i2c@1/imx477@1a
Color Matrix 1                  : 0.8545269369 -0.2382823821 -0.09044229197 -0.1890484985 1.063961506 0.1062747385 -0.01334283455 0.1440163847 0.2593136724
As Shot Neutral                 : 0.4754476844 1 0.413686484
Calibration Illuminant 1        : D65
Strip Offsets                   : 0
Strip Byte Counts               : 0
Exposure Time                   : 1/20
ISO                             : 400
CFA Pattern                     : [Blue, Green][Green, Red]
Image Size                      : 4056x3040
Megapixels                      : 12.3
Shutter Speed                   : 1/20

Long exposure

If we want to take a super long exposure picture, we need to disable AEC/AGC and white balance, otherwise these algorithms will cause the picture to wait for a lot of frame data when it converges. Disabling these algorithms requires another explicit value to be set. Additionally, the user can skip the preview process with the --immediate setting.
Here is the instruction to take an image with an exposure of 100 seconds:

libcamera-still -o long_exposure.jpg --shutter 100000000 --gain 1 --awbgains 1,1 --immediate

Remarks: Reference table for the longest exposure time of several official cameras.

libcamera-vid

libcamera-vid is a video recording program that uses the Raspberry Pi hardware H.264 encoder by default. After the program runs, a preview window will be displayed on the screen, and the bitstream encoding will be output to the specified file. For example, record a 10s video.

libcamera-vid -t 10000 -o test.h264

If you want to view the video, you can use vlc to play it

vlc test.h264

Note: The recorded video stream is unpackaged. Users can use --save-pts to set the output timestamp to facilitate subsequent conversion of the bit stream to other video formats.

libcamera-vid -o test.h264 --save-pts timestamps.txt

If you want to output the mkv file, you can use the following command:

mkvmerge -o test.mkv --timecodes 0:timestamps.txt test.h264

Encoder

Raspberry Pi supports JPEG format and YUV420 without compression and format:

libcamera-vid -t 10000 --codec mjpeg -o test.mjpeg
libcamera-vid -t 10000 --codec yuv420 -o test.data

The --codec option sets the output format, not the output file extension.
Use the --segment parameter to split the output file into segments (unit is ms), which is suitable for JPEG files that need to split the JPEG video stream into separate short (about 1ms) JPEG files.

libcamera-vid -t 10000 --codec mjpeg --segment 1 -o test%05d.jpeg

libcamera-raw

Libcamera-raw is similar to a video recording program. In different places, libcamera-raw records the Bayer format data output by the direct sensor, that is, the original image data. Libcamera-raw doesn't show a preview window. For example, record a 2-second clip of raw data.

libcamera-raw -t 2000 -o test.raw

The program will directly dump the original frame without format information, the program will directly print the pixel format and image size on the terminal, and the user can view the pixel data according to the output data.
By default, the program will save the original frame as a file, the file is usually large, and the user can divide the file by the --segement parameter.

libcamera-raw -t 2000 --segment 1 -o test%05d.raw

If the memory is large (such as using SSD), libcamera-raw can write the official HQ Camera data (about 18MB per frame) to the hard disk at a speed of about 10 frames per second. In order to achieve this speed, the program writes the unformatted raw frames, there is no way to save them as DNG files like libcamera-still does. If you want to ensure that there are no dropped frames, you can use --framerate to reduce the frame rate.

libcamera-raw -t 5000 --width 4056 --height 3040 -o test.raw --framerate 8

Common Command Setting Options

Common command setting options apply to all libcamera commands:

--help, -h

Print program help information, you can print the available setting options for each program command, and then exit.

--version

Print the software version, print the software version of libcamera and libcamera-app, then exit.

--timeout, -t

The "-t" option sets the running time of the libcamera program. If the video recording command is run, the timeout option sets the recording duration. If the image capture command is run, the timeout sets the preview time before the image is captured and output.
If the timeout is not set when running the libcamera program, the default timeout value is 5000 (5 seconds). If the timeout is set to 0, the program will continue to run.
Example: libcamera-hello -t 0

--preview, -p

"-p" sets the size and position of the preview window (the qualified settings are valid in both X and DRM version windows), and the format is --preview <x. y, w, h> where "x, y" sets the preview window coordinate, "w" and "h" set the width and length of the preview window.
The settings of the preview serial port will not affect the resolution and aspect ratio of the camera image preview. The program will scale the preview image to display in the preview window and adapt it according to the original image aspect ratio.
Example: libcamera-hello -p 100,100,500,500

--fullscreen, -f

The "-f" option sets the preview window full screen, the preview window, and the border in full-screen mode. Like "-p", it does not affect the resolution and aspect ratio, and will automatically adapt.
Example: libcamera-still -f -o test.jpg

--qt-preview

Using the preview window based on the QT framework, this setting is not recommended under normal circumstances, because the preview program will not use zero-copy buffer sharing and GPU acceleration, which will occupy more resources. The QT preview window supports X forwarding (the default preview program does not).
The Qt preview serial port does not support the "--fullscreen" setting option. If the user wants to use the Qt preview, it is recommended to keep a small preview window to avoid excessive resource usage and affecting the normal operation of the system.
Example: libcamera-hello --qt-preview

--nopreview, -n

Images are not previewed. This setting will turn off the image preview function.
Example: libcamera-hello -n

--info-text

Set the title and information display of the preview window (only available in the X graphics window) using the format --info-text <string>. When calling this option, there are multiple parameters that can be set, and the parameters are usually called in the % command format. The program will call the corresponding value in the graphics metadata according to the instruction.
If no window info is specified, the default --info-text is set to "#%frame (%fps fps) exp %exp ag %ag dg %dg"
Example: libcamera-hello --info-test "Focus measure: %focus Available parameters:

--width
--height

These two parameters set the width and height of the image, respectively. For libcamera-still, libcamera-jpeg and libcamera-vid commands, these two parameters can set the resolution of the output image/video.
If the libcamera-raw command is used, these two parameters affect the size of the obtained metadata frame. The camera has a 2 x 2 block reading mode. If the set resolution is smaller than the split mode, the camera will obtain the metadata frame according to the 2 x 2 block size.
libcamera-hello cannot specify the resolution.
Example:
libcamera-vid -o test.h264 --width 1920 --height 1080 Record 1080p video
libcamera-still -r -o test.jpg --width 2028 --height 1520 Takes a 2028 x 1520 JPEG image.

--viewfinder-width
--viewfinder-height

This setting option is also used to set the resolution of the image, the difference is only the image size of the preview. It does not affect the final output image or video resolution. The size of the preview image will not affect the size of the preview window and will be adapted according to the window.
Example: libcamera-hello --viewfinder-width 640 --viewfinder-height 480

--rawfull

This setting forces the sensor to use the --width and --height settings to output still images and video in full-resolution read mode. This setting libcamera-hello has no effect.
With this setting, the framerate is sacrificed. In full-resolution mode, frame reading will be slower.
Example: libcamera-raw -t 2000 --segment 1 --rawfull -o test%03d.raw The example command captures multiple full resolution images Metadata frame in rate mode. If you are using HQ camera. The size of each frame is 18MB, and if --rawfull is not set, the HQ camera defaults to 2 x 2 mode, and the data size of each frame is only 4.5MB.

--lores-width
--lores-height

These two options set low-resolution images. The low-resolution data stream compresses the image, causing the aspect ratio of the image to change. When using libcamera-vid to record video, if a low resolution is set, functions such as color denoising will be disabled.
Example: libcamera-hello --lores-width 224 --lores-height 224 Note that low resolution settings are often used in conjunction with image postprocessing, otherwise it has little effect.

--hflip #Flip the image horizontally
--vflip #Flip the image vertically
--rotation #Flip the image horizontally or vertically according to the given angle <angle>

These three options are used to flip the image. The parameters of --rotation currently only support 0 and 180, which are actually equivalent to --hflip and --vflip.
Example: libcamera-hello --vflip --hflip

--roi #crop image <x, y, w, h>

"--roi" allows the user to crop the image area they want according to the coordinates from the complete image provided by the sensor, that is, digital scaling, pay attention to the coordinate value if it is in the valid range. For example --roi 0, 0, 1, 1 is an invalid instruction.
Example: libcamera-hello --roi 0.25,0.25,0.5,0.5
The example command will crop 1/4 of the image from the center of the image.

--sharpness #Set the sharpness of the image <number>

Adjust the sharpness of the image by the value of <number>. If set to 0, no sharpening is applied. If you set a value above 1.0, an extra sharpening amount will be used.
Example: libcamera-still -o test.jpg --sharpness 2.0

--contrast #Set image contrast <number>

Example: libcamera-still -o test.jpg --contrast 1.5

--brightness #Set image brightness <number>

The setting range is -1.0 ~ 1.0
Example: libcamera-still -o test.jpg --brightness 0.2

--saturation #Set image color saturation <number>

Example: libcamera-still -o test.jpg --saturation 0.8

--ev #Set EV compensation <number>

Set the EV compensation of the image in aperture units, the setting range is -10 ~ 10, the default value is 0. The program works by improving the target method of the AEC/AGC algorithm.
Example: libcamera-still -o test.jpg --ev 0.3

--shutter #Set the exposure time, the unit is ms <number>

Note: If the frame rate of the camera is too fast, it may not work according to the set shutter time. If this happens, you can try to use --framerate to reduce the frame rate.
Example: libcamera-hello --shutter 30000

--gain #Set gain value (combination of numerical gain and analog gain) <number>
--analoggain #--gain synonym

--analoggain is the same as --gain, the use of analoggain is only for compatibility with raspicam programs.

--metering #Set metering mode <string>

Set the metering mode of the AEC/AGC algorithm, the available parameters are:

• centre - Center metering (default)
• spot - spot metering
• average - average or full frame metering
• custom - custom metering mode, can be set via tuning file

Example: libcamera-still -o test.jpg --metering spot

--exposure #set exposure profile <string>

The exposure mode can be set to normal or sport. The report profile for these two modes does not affect the overall exposure of the image, but in the case of sports mode, the program will shorten the exposure time and increase the justice to achieve the same exposure effect.
Example: libcamera-still -o test.jpg --exposure sport

--awb #Set white balance mode <string>

Available white balance modes:

Example: libamera-still -o test.jpg --awb tungsten

--awbgains #Set a fixed color gain <number,number>

Set red and blue gain.
Example: libcamera-still -o test.jpg --awbgains 1.5, 2.0

--denoise #Set denoising mode <string>

Supported denoising modes:

• auto - default mode, use standard spatial denoising, if it is video, it will use fast color noise reduction and use high-quality color noise reduction when taking still pictures. The preview image will not use any color denoising.
• off - turn off spatial denoising and color denoising
• cdn_off - turn off color denoising
• cdn_fast - use fast color denoising
• cdn_hq - use high-quality color denoising, not suitable for video recording.

Example: libcamera-vid -o test.h264 --denoise cdn_off

--tuning-file #Specify camera tuning file <string>

For more instructions on tuning files, you can refer to official tutorial
Example: libcamera-hello --tuning-file ~/my~camera-tuning.json

--output, -o #output filename <string>

Set the filename of the output image or video. In addition to setting the file name, you can also specify the output udp or tcp server address to output the image to the server. If you are interested, you can check the relevant setting instructions of the subsequent tcp and udp.
Example: libcamera-vid -t 100000 -o test.h264

--wrap #Wrap the output file counter <number>

Example: libcamera-vid -t 0 --codec mjpeg --segment 1 --wrap 100 -o image%d.jpg

--flush # Flush the output file immediately

--flush will immediately update each frame of the image to the hard disk at the same time as it is written, reducing latency.
Example: libcamera-vid -t 10000 --flush -o test.h264

Still photo shooting setting parameters

--qiality, -q       #Set JPEG image quality <0 ~ 100>
--exif, -x          #Add extra EXIF flags
--timelapse         #Time interval of time-lapse photography, the unit is ms
--framestart        #start value of frame count
--datetime          #name output file with date format
--timestamp         #name the output file with the system timestamp
-- restart          #Set the JPEG restart interval
--keypress, -k      # Set the enter button photo mode
--signal, -s        #Set the signal to trigger the photo
--thumb             #Set thumbnail parameters <w:h:q>
--ebcoding, -e      #Set the image encoding type. jpg/png/bmp/rgb/yuv420
--raw, -r           #Save raw image
--latest            #Associate symbols to the latest saved file

Video recording image setting parameters

--quality, -q   # Set JPEG commands <0 - 100>
--bitrate, -b   # Set H.264 bitrate
--intra, -g     #Set the internal frame period (only supports H.264)
--profile       #Set H.264 configuration
--level         #Set H.264 level
--codec         #Set encoding type h264 / mjpeg / yuv420
--keypress, -k  #Set carriage return to pause and record
--signal, -s    #Set signal pause and record
--initial       #Start the program in the recording or paused state
--split         #Split video and save to another file
--segment       #Split video into multiple video segments
--circular      #Write video to circular buffer
--inline        #Write header in each I frame (H.264 only)
--listen        #Wait for a TCP connection

• For more camera setup instructions, please refer to the official documentation Official camera documentation

• User manual
• Camera Guide
• Raspberry Pi High Quality Camera Getting Started
• HQ Camera Drawing
• C-Mount Lens Guide
• CS-Mount Lens Guide
• FCC
• CE

To：

Dear Customers, There may be a lag of several hours before we could process your order, depending on the time difference between your location and ours. Sincerely hoping your understanding!

Email(support)

services01@spotpear.com