Over the period of May 2014 to December 2015 we traversed a route through central Oxford twice a week on average using the Oxford RobotCar platform, an autonomous Nissan LEAF. This resulted in over 1000km of recorded driving with almost 20 million images collected from 6 cameras mounted to the vehicle, along with LIDAR, GPS and INS ground truth.
Data was collected in all weather conditions, including heavy rain, night, direct sunlight and snow. Road and building works over the period of a year significantly changed sections of the route from the beginning to the end of data collection.
By frequently traversing the same route over the period of a year we enable research investigating long-term localisation and mapping for autonomous vehicles in real-world, dynamic urban environments.
The RobotCar is equipped with the following sensors:
Intrinsic calibration parameters and undistortion look-up-tables for the cameras are provided on the downloads page. For those wishing to perform their own calibrations, we also provide original images of a chequerboard sequence.
The below image shows the location and orientation of each sensor on the vehicle. Precise extrinsic calibrations for each sensor are included in the development tools.
For distribution we have divided up the datasets into individual routes, each corresponding to a single traversal. To reduce download file sizes, we have further divided up each traversal into chunks, where each chunk corresponds to an approximately 6 minute segment of the route. Within one traversal, chunks from different sensors will overlap in time (e.g. left stereo chunk 2 covers the same time period as LD-MRS chunk 2); however, chunks do not correspond between different traversals.
Each chunk has been packaged as a
tar archive for downloading. It is intended that all
tar archives are extracted in the same directory: this will preserve the folder structure indicated above when multiple chunks and/or traversals are downloaded. Each chunk archive also contains a full list of all sensor timestamps for the traversal in the
<sensor>.timestamps file, as well as a list of condition tags as illustrated in the
tags.csv file. The timestamps file contains ASCII formatted data, with each line corresponding to the UNIX timestamp and chunk ID of a single image or LIDAR scan.
All images are stored as lossless-compressed PNG files in unrectified 8-bit raw Bayer format. The files are structured as
stereo for Bumblebee XB3 images or
mono for Grasshopper2 images,
<sensor> is any of
left | centre | right for Bumblebee XB3 images and
left | rear | right for Grasshopper2 cameras, and
<timestamp> is the UNIX timestamp of the capture, measured in microseconds.
The top left 4 pixel Bayer pattern for the Bumblebee XB3 images is GBRG, and for Grasshopper2 images is RGGB. The raw Bayer images can be converted to RGB using the MATLAB
demosaic function, the OpenCV
cvtColor function or similar methods.
The 2D LIDAR returns for each scan are stored as double-precision floating point values packed into a binary file, similar to the Velodyne scan format the KITTI dataset. The files are structured as
Each 2D scan consists of 541 triplets of
(x, y, R), where
x, y are the 2D Cartesian coordinates of the LIDAR return relative to the sensor (in metres), and
R is the measured infrared reflectance value.
No correction for the motion of the vehicle has been performed when projecting the points into Cartesian coordinates; this can be optionally performed by interpolating a timestamp for each point based on the 15ms rotation period of the laser.
For a scan with filename
(x, y, R) triplet at index 0 was collected at
<timestamp> and the triplet at index 540 was collected at
The 3D LIDAR returns from the LD-MRS are stored in the same packed double-precision floating point binary format as the 2D LIDAR scans.
The files are structured as
ldmrs/<timestamp>.bin. Each 3D scan consists of triplets of
(x, y, z), which is the 3D Cartesian coordinates of the LIDAR return relative to the sensor (in metres). The LD-MRS does not provide infrared reflectance values for LIDAR returns.
GPS and inertial sensor data from the SPAN-CPT are provided in an ASCII-formatted csv file. Two separate files are provided:
gps.csv contains the GPS-only solution of latitude (deg), longitude (deg), altitude (m) and uncertainty (m) at 5Hz, and
ins.csv contains the fused GPS+Inertial solution, consisting of 3D UTM position (m), velocity (m/s), attitude (rad) and solution status at 50Hz.
ins.csvgive the units of attitude as degrees. They are in fact in radians.
Local errors in the GPS+Inertial solution (due to loss or reacquisition of satellite signals) can lead to discontinuites in local maps built using this sensor as a pose source. For some applications a smooth local pose source that is not necessarily globally accurate is preferable.
We have processed the full set of Bumblebee XB3 wide-baseline stereo imagery using our visual odometry system and provide the relative pose estimates as a reference local pose source. The file
vo.csv contains the relative pose solution, consisting of the source and destination frame timestamps, along with the six-vector Euler parameterisation
(x, y, z, α, β, γ) of the SE(3) relative pose relating the two frames. Our visual odometry solution is accurate over hundreds of metres (suitable for building local 3D pointclouds) but drifts over larger scales, and is also influenced by scene content and exposure levels; we provide it only as a reference and not as a ground truth relative pose system.
We provide a set of simple MATLAB and Python development tools for easy access to and manipulation of the dataset. The MATLAB and Python functions provided include simple functions to load and display imagery and LIDAR scans, as well as more advanced functions involving generating 3D pointclouds from push-broom 2D scans, and projecting 3D pointclouds into camera images.
LoadImage.m reads a raw Bayer image from a specified directory and at a specified timestamp, and returns a MATLAB format RGB colour image. The function can also take an optional look-up table argument (provided by the function
ReadCameraModel.m) which will then rectify the image using the undistortion look-up tables.
load_image function in the
image module will load and image, perform Bayer demosaicing, and optionally undistort the image. A camera model for undistortion can be built with the
BuildPointcloud.m (MATLAB) or
build_pointcloud (Python) combines a 6DoF trajectory from the INS with 2D LIDAR scans to produce a local 3D pointcloud. The resulting pointcloud is coloured using LIDAR reflectance information, highlighting lane markings and other reflective objects. The size of the pointcloud can be controlled using the
BuildPointcloud can be called without outputs to visualise the pointcloud. The Python
build_pointcloud.py script can be run from the command line to display the 3D pointcloud.
ProjectLaserIntoCamera.m combines the above two tools as follows:
LoadImage.m is used to retrieve and undistort a raw image from a specified directory at a specified timestamp, then
BuildPointcloud.m is used to generate a local 3D pointcloud around the vehicle at the time of image capture. The 3D pointcloud is then projected into the 2D camera image using the camera intrinsics.
The same functionality is provided by the
project_laser_into_camera.py script. The
CameraModel class provides both image undistortion and pointcloud projection, using a pinhole camera model.