Dataset & Labels - SAGES CVS Challenge (CVS-Challenge)

SAGES CVS Challenge (CVS-Challenge) Banner

DISCLAIMER: The SAGES CVS Challenge took place at MICCAI 2024 in Marrakech and is no longer open for submissions - stay tuned for the CVS Lighthouse Challenge at MICCAI 2025! The SAGES CVS Challenge dataset is currently not publicly available. Any use of the dataset outside the official MICCAI challenge is strictly prohibited. Participants have agreed not to publish the data or any methods developed using the data until the final challenge manuscript has been released by the organizers.

The Dataset¶

The SAGES CVS challenge offers a global and diverse dataset of 1000 videos of laparoscopic cholecystectomy procedures, a routinely performed minimally invasive surgical procedure to remove the gallbladder due to inflammation or other pathologies. The dataset was assembled through data contributions from 67 surgeons from 57 institutions in 24 countries and 6 continents and encompasses a wide diversity of patient demographics and procedural quality to reflect the worldwide diversity in patients and surgeons. Moreover, the video data is acquired using a variety of laparoscopes from different manufacturers and displays an array of optical properties, which adds to the diversity of the data and ensures adequate representation of the real-world population. The dataset was carefully curated and annotated with respect to the three CVS criteria, following consensus-based protocols incorporating the expertise of worldwide leading clinicians, computer scientists, and industry through structured multidisciplinary Advisory Committees (AC).

Map of countries represented in the dataset:¶

The Labels¶

Each video in the dataset represents a single patient's case, spanning a 90-second segment. This segment starts 90 seconds before one of the relevant anatomic structures composing the CVS (cystic duct or artery) is clipped ("point of no return"). Within each video (90-second segment), additional frames are extracted at five-second intervals (= 18 frames per video) to provide additional training data.

Extraction of 90-second video segment:¶

The three criteria defining the CVS are annotated as binary labels (criterion achieved yes/no) for each video and each of the 18 frames per case. No semantic segmentation labels are provided. Moreover, we provide an overall rating of the annotation difficulty for each video, reflecting the annotators' subjective assessment. This does not reflect the surgical/operative difficulty of the case. The annotation complexity is provided as a 0-1 value, representing the annotator's confidence ( 0 = very difficult to annotate/ not confident at all; 1 = very easy to annotate/ very confident). For each video, we provide annotations:

Per Video (Were the 3 CVS criteria achieved in the overall video? Yes / No) = 3 Binary Labels x 3 annotators
Per Frame (Were the 3 CVS criteria achieved in this frame? Yes / No) = 3 Binary Labels x 18 Frames x 3 annotators
Annotation Confidence (How sure was the annotation of the 3 CVS criteria for this case? 0-1 / not confident-confident) = 1 probabilistic label x 3 annotators

Definition of the 3 CVS Criteria:¶

The Metadata¶

Besides the Labels (per video / per frame / annotation difficulty) we provide demographic and selected clinical metadata.

Country of data origin (anonymized) = the dataset was composed with contributions from 6 continents, to preserve data privacy the country name is anonymized
Laparoscope Type (category) = The type and manufacturer with which the video was recorded, which can influence the resolution, is provided in pseudonymized categories.
Robotic Surgery (yes/no) = If the video was performed robotically (as opposed to conventional laparoscopy) is provided as a binary (yes/no) label.
Indocyanine Green (ICG) Use (yes/no) = During any laparoscopic cholecystectomy procedure, ICG can be used to better visualize and identify the relevant anatomy under fluorescent light. While ICG is routinely used in some institutions, in others, surgeons might selectively opt to use ICG for an enhanced overview of the operative field. In this case, ICG may indirectly indicate a more difficult case. While you may not see ICG or fluorescence in the provided 90-second video, the label indicates whether it was applied throughout the case.
Intraoperative Cholangiography (IOC) (yes/no) = An intraoperative cholangiography is a measure to visualize the anatomy of the biliary ducts more clearly. During an IOC, the surgeon opens the structure identified as the cystic duct and introduces a catheter into the common bile duct for drainage or to inject dye. This can improve the understanding of the anatomic relations of the structures (cystic duct vs common bile duct) to each other and help to prevent accidental clipping of the common bile duct. Usually, an IOC indicates a more complex case (i.e., due to the clinical history of the patient, complex anatomy, an increased degree of inflammation of the gallbladder). Similarly to ICG, the IOC may be performed outside the 90-second video segment.

Dataset Release¶

The training data will be released in three intervals: 1) First batch of 250 videos (mid April) 2) Second batch of 250 videos (mid June) 3) Third batch of 200 videos (mid July). Thus, the dataset will be released in a traditional 70 / 30 split, with 70% of the dataset available to participants for model development and 30% of the dataset retained for validation. Participants will only have access to the intraabdominal endoscopic video and corresponding labels. The specific demographics associated with the donated video data will remain proprietary for evaluation and data split purposes, and to ensure privacy compliance. Check out Important Dates to find out more about the challenge timeline.