CvSBA: an OpenCV wrapper for sba library

cvsba is an OpenCV wrapper for the well-known Sparse Bundle Adjustment library (sba) by M. Lourakis.

The main features are:

  • Based on sba-1.6, one of the most popular and robust bundle adjustment implementation, which is extensively used and tested by the community
  • sba installation is not needed since it is included in cvsba
  • New CMake structure which makes the library compilation, installation and linkage easier
  • Similar interface than Bundle Adjustment implementation on cv::LevMarqSparse::bundleAdjust()
  • Include examples to test the library on synthetically generated data
  • GPL licence

 

Download

Sourceforge download cvsba

Installation

Unix:

sudo apt-get install liblapack-dev libf2c2-dev 
tar -zxvf cvsba-1.0.0.tgz 
mkdir cvsba-1.0.0.tgz/build
cd cvsba-1.0.0.tgz/build
cmake ..
make
make install

Contact info:

i52gajus at uco dot es
rmsalinas at uco dot es

How to use

The method Sba::run() executes the bundle adjustment optimization. For a scenerio with M cameras and N 3d points, the parameters description is the following:

double Sba::run (  std::vector<cv::Point3d>& points,
                   const std::vector<std::vector<cv::Point2d> >& imagePoints,
                   const std::vector<std::vector<int> >& visibility,
                   std::vector<cv::Mat>& cameraMatrix,
                   std::vector<cv::Mat>& R,
                   std::vector<cv::Mat>& T,
                   std::vector<cv::Mat>& distCoeffs );
  • points: vector of estimated 3d points (size N).
  • imagePoints: (input/[output]) vector of vectors of estimated image projections of 3d points (size MxN). Element imagePoints[i][j] refers to j 3d point projection over camera i.
  • visibility: [input] same structure and size than imagePoints (size MxN). Element  visibility[i][j] is 1 if points[j] is visible on camera i. Otherwise it is 0. No-visible projections are ignored in imagePoints structure.
  • cameraMatrix: (input/[output]) vector of camera intrinsic matrixes (size N). Each matrix consists in 3×3 camera projection matrix.
  • R: (input/[output]) vector of estimated camera rotations (size N). Each rotation is stored in Rodrigues format (size 3).
  • T: (input/[output]) vector of estimated camera traslations (size N).
  • distCoeffs: (input/[output]) vector of camera distortion coefficients (size N). Each element is composed by 5 distortion coefficients.

Return value:  the projection error obtained after the optimization.

 

Other  parameters can be established through the Sba::Params structure and using the corresponding Sba::getParams() and Sba::setParams() methods. The parameters on Sba::Params are:

  • TYPE type: type of bundle adjustment optimization. Posible values:
    • MOTIONSTRUCTURE: 3d points positions and camera extrinsics (R and T) are optimized. Input is used as initial values.
    • MOTION: just camera extrinsics are optimized, 3d points are fixed.
    • STRUCTURE: just 3d points are optimized, camera extrinsics are fixed.
  • int iterations: maximun number of process iterations.
  • double minError: minimun error to stop the process.
  • int fixedIntrinsics: number of intrinsics parameters that are kept fixed [0-5] in the following order: fx cx cy fy/fx s.
  • int fixedDistortion: number of distortion parameters that are kept fixed [0-5] in the following order: k1 k2 p1 p2 k3.
  • bool verbose: if enabled, some extra information is printed during optimization.

Types of optimization

The sba library can run  diferent types of optimizations (MOTION,STRUCTURE,MOTIONSTRUCTURE). Each type of optimization affect to some of the input parameters. For instance, MOTIONSTRUCTURE is aimed at modifying both the struture (3d points), and the motion (Intrinsics,Distortion , R and T). However, you can control which of these elements to be effectively optimized using fixedIntrinsics and
fixedDistortion. If you set this parameters to 5, then Intrinsics and Distortion will not be modified at all. Then, the algorithm will only modify the 3d points. However, if you set  fixedIntrinsics and fixedDistortion to 0, then,  the the elements in cameraMatrix,R,T, distCoeffs will also be modified with the new optimized values.

Code Example

Simple code that shows how to use cvsba in your own project. It includes the test program cvsba_simple_test.cpp and a basic CMakeLists.txt for easy compilation. Download full code example.

CMakeLists.txt

project(cvsba_simple_test)
SET(CMAKE_MODULE_PATH ${CMAKE_INSTALL_PREFIX}/lib/cmake/ )
find_package(OpenCV REQUIRED )
find_package(cvsba REQUIRED )
add_executable(cvsba_simple_test cvsba_simple_test.cpp)
target_link_libraries(cvsba_simple_test ${OpenCV_LIBS} ${cvsba_LIBS})

cvsba_simple_test.cpp

#include <cvsba/cvsba.h>

int main(int argc, char **argv) {

  /****** INPUT DATA *****************/
     std::vector< cv::Point3d > points3D;
     std::vector< std::vector< cv::Point2d > > pointsImg;
     std::vector< std::vector< int > > visibility;
     std::vector< cv::Mat > cameraMatrix, distCoeffs, R, T;

     int NPOINTS=32; // number of 3d points
     int NCAMS=2; // number of cameras

     // fill 3D points
     points3D.resize(NPOINTS);
     points3D[0] =cv::Point3d(-0.9991,-0.6847,-0.9559);
     points3D[1] =cv::Point3d(-0.009,0.4921,0.9615);
     points3D[2] =cv::Point3d(-0.7931,0.5179,-0.366);
     //...


     // fill image projections
     pointsImg.resize(NCAMS);
     for(int i=0; i<NCAMS; i++) pointsImg[i].resize(NPOINTS);
     pointsImg[0][0] =cv::Point2d(112.468,68.9535);
     pointsImg[1][0] =cv::Point2d(1087.52,731.054);
     pointsImg[0][1] =cv::Point2d(598.747,522.755);
     //...

     //... FILL THE REST OF DATA: visibility, intrinsics, extrinsics, distortion...
     //...
     //...
  /***********************************/


  /****** RUN BUNDLE ADJUSTMENT ******/
     cvsba::Sba sba;
     sba.run(points3D, pointsImg, visibility, cameraMatrix, R, T, distCoeffs);

     std::cout<<"Initial error="<<sba.getInitialReprjError()<<". "<<

                "Final error="<<sba.getFinalReprjError()<<std::endl;

  /***********************************/

     return 0;
}