sksurgerysurfacematch.pipelines.register_cloud_to_stereo_reconstruction module

Pipeline to register 3D point cloud to 2D stereo video

class sksurgerysurfacematch.pipelines.register_cloud_to_stereo_reconstruction.Register3DToStereoVideo(video_segmentor: sksurgerysurfacematch.interfaces.video_segmentor.VideoSegmentor, surface_reconstructor: sksurgerysurfacematch.interfaces.stereo_reconstructor.StereoReconstructor, rigid_registration: sksurgerysurfacematch.interfaces.rigid_registration.RigidRegistration, left_camera_matrix: numpy.ndarray, right_camera_matrix: numpy.ndarray, left_to_right_rmat: numpy.ndarray, left_to_right_tvec: numpy.ndarray, left_mask: numpy.ndarray = None, z_range: list = None, radius_removal: list = None, voxel_reduction: list = None)[source]

Bases: object

Class for single-shot, registration of 3D point cloud to stereo video. Uses Dependency Injection for each pluggable component.

Parameters:
  • video_segmentor – Optional class to pre-segment the video.
  • surface_reconstructor – Mandatory class to do reconstruction.
  • rigid_registration – Mandatory class to perform rigid alignment.
  • left_camera_matrix – [3x3] camera matrix.
  • right_camera_matrix – [3x3] camera matrix.
  • left_to_right_rmat – [3x3] left-to-right rotation matrix.
  • left_to_right_tvec – [1x3] left-to-right translation vector.
  • left_mask – a static mask to apply to stereo reconstruction.
  • z_range – [min range, max range] to limit reconstructed points.
  • radius_removal – [radius, number] to reject points with too few neighbours
  • voxel_reduction – [vx, vy, vz] parameters for PCL Voxel Grid reduction.
register(reference_cloud: numpy.ndarray, left_image: numpy.ndarray, right_image: numpy.ndarray, initial_ref2recon: numpy.ndarray = None) → Tuple[numpy.ndarray, numpy.ndarray, numpy.ndarray, numpy.ndarray][source]

Main method to do a single 3D cloud to 2D stereo video registration.

Camera calibration parameters are in OpenCV format.

Parameters:
  • reference_cloud – [Nx3] points, each row, x,y,z, e.g. from CT/MR.
  • left_image – undistorted, BGR image
  • right_image – undistorted, BGR image
  • initial_ref2recon – [4x4] of initial rigid transform.
Returns:

residual, [4x4] transform, of reference_cloud to left camera space, [Mx3] downsampled xyz points and [Mx6] reconstructed point cloud, as [x, y, z, r, g, b] rows.