Building Dataset: Sensor Configuration

The model that I want to deploy is Camera Radar Net (CRN).

Obviously, I need a camera and radar sensor to deploy it. In addition, LiDAR and DGPS are also desired for ground truth, annotation, and evaluation.

Fortunately, the camera, LIDAR, and DGPS are already installed in the lab autonomous vehicle platform and I couldn’t change LiDAR and DGPS sensors. As a result, the focus was on selecting the most suitable camera and radar for our needs.

Camera

When we initially installed the camera and its housing, we designed the camera mount to be changed to another camera. (Of course, we had to make sure that the camera position should not be changed by the vibration of the vehicle.) The cameras originally installed in the platform were USB cameras, which have disadvantages in limited dynamic range, lack of PTP synchronization, and slow data transfer rates.

I was especially concerned that the slow data transfer rates of USB cameras might become a possible bottleneck when using multiple cameras, so I considered switching to the automotive camera with a GMSL interface. However, as I fixed the field of view (FoV) of our MVP model to the front view using a single camera and radar, I decided to keep using a USB camera from Leopard Imaging.

Radar

My camera radar fusion model was developed and tested on nuScenes that uses Continental ARS408-21 radar, which provides point-like data. Fortunately (or unfortunately), we didn’t have many options when it came to radar sensors that provide point cloud, so we settled on the Continental ARS430 radar.

ARS430 has the same hardware as ARS408-21 but offers an improved I/O interface (upgraded from CAN to Ethernet). This means that two radars will have less domain gap from the hardware, and we can better utilize nuScenes for pre-training. I designed the mounting position and angle considering the radar FoV and to cover 360-degree vehicles, while obeying the Korean traffic laws.

Here is the final sensor configuration for cameras (green), radars (red), LiDAR (orange), and DGPS (blue).

You can check the detailed information on the VDC lab homepage.