Innovative computer vision approach to 3D bladder model reconstruction from flexible cystoscopy

Bladder cancer is the fourth most common cancer in men and is considered to have the highest rate of recurrence of all cancers at ∼70%, and transitional cell carcinoma (TCC) is the most common form of intrabladder malignancy. Current standard-of-care for Stages 2 or higher is radical cystectomy, which involves removal of the urinary bladder and nearby lymph nodes. Alternative, organ-sparing treatments such as chemo- or radiotherapy are relatively ineffective against these cancers. The latter is effective when precisely targeted, but suffers from accuracy issues due to low contrast from computed tomography guidance. These motivate an innovative approach to more precisely visualize and spatially pinpoint TCC. This manuscript presents a novel non-invasive computer vision pipeline that can extract 3D structural information from 2D images obtained during routine flexible cystoscopy. The pipeline utilized camera calibration, adaptive thresholding, Scale Invariant Feature Transform (SIFT), and a Structure from Motion (SFM) implementation to reconstruct 3D point clouds of the inner surface of organ phantoms and an ex vivo porcine bladder. 3D point clouds were processed by Poisson reconstruction to generate a textured, triangle meshed 3D surface. The reconstruction pipeline generated a visually recognizable, qualitative 3D representation of the bladder from 2D video captured via flexible cystoscopy. Once further developed, this approach will enhance the targeting precision of external beam radiotherapy, providing clinicians with better organ-sparing methods to treat TCC.