🔍 Introduction and Goal
The Cell Behavior Video Classification Challenge (CBVCC) is a challenge designed to develop/adapt computer vision methods for classifying videos capturing cell behavior through Intravital Microscopy (IVM).
IVM is a powerful imaging technique that allows for non-invasive visualization of biological processes in living animals. Platforms such as two-photon microscopes exploit multiple low-energy photons to deliver high-resolution, three-dimensional videos depicting tissues and cells deep within the body. IVM has been used to visualize a wide range of biological processes, including immune responses, cancer development, and neurovascular function.
The primary goal of the CBVCC challenge is to create models that can accurately classify videos based on the movement patterns of cells. Specifically, the models should be able of:
- Identifying videos where cells exhibit sudden changes in their direction of movement.
- Distinguishing these from videos where cells show consistent, linear movement, stationary cells as well as from videos containing only background.
The CBVCC challenge aims to provide a platform for researchers to develop innovative methods for classifying IVM videos, potentially leading to new insights into biological processes.
👩🔬👨🔬 The CBVCC Challenge
The CBVCC challenge will be open to researchers from all over the world. Participants will be asked to develop computational models to classify IVM videos. The models will be evaluated based on their accuracy to classify videos.
The challenge will be divided into two phases:
- Phase 1: In the first phase, participants will be provided with a training dataset and a test dataset of IVM images and labels. They will use this data to develop and evaluate their models.
- Phase 2: In the second phase, participants will submit their results obtained on a distinct test dataset of IVM images. The models will be ranked based on their performance on the test dataset.
The challenge results will be decided based on the performance of the Phase 2 submission.
🏅 The Challenge timeline
The CBVCC challenge will begin on September 15th, 2024, and end on December 20th, 2024. The timeline of the key events is organized as follows:
- September 15th 2024 to November 15th 2024: Participants are invited to join the challenge by registering on the website.
Registration link: https://forms.gle/cQSHJmEFZ5bRBJ328
- November 15th 2024: The train dataset will be published, officially launching the challenge. Additionally, a validation leaderboard will be made available for participants.
- December 13th 2024: Test dataset (not annotated) released.
- December 20th 2024: Challenge closes. By this date participants are required to submit their predictions on the test set, an abstract describing the methodology, code or docker container to replicate results.
- January 7th 2025: Results and winners will be announced.
Dataset
The CBVCC dataset consists of 2D video-patches extracted from IVM videos. These videos capture the behavior of regulatory T cells in the abdominal flank skin of mice undergoing a contact sensitivity response to the sensitizing hapten, oxazolone. The videos are acquired either 24 or 48 hours after the initial skin challenge with oxazolone. Each video sequence lasts for 30 minutes, with images captured at one-minute intervals, resulting in a total of 31 acquisitions per video.
The primary goal of this challenge is to classify the provided video-patches into two distinct categories:
- Video-patches where cells suddenly change their direction of movement (class 1): These videos contain cells that demonstrate sudden changes in their migratory paths.
- Video-patches without sudden changes in cell direction (class 0): These videos either show cells moving in a linear manner, remain stationary, or contain video with only the background and no visible cells.
The dataset comprises a total of 300 2D video-patches extracted from 48 different videos, representing both classes (n=180 class 0 and n=120 class 1). Each video-patch is a 2D projection along the z-axis of a 3D video sequence, carefully adjusted to a common contrast range to enhance the visibility of the biological processes. Additionally, all videos have been preprocessed to ensure a uniform pixel size of 0.8 µm. Video-patches are saved as RGB .avi files.
The dataset consists of 300 pairs of video-patches and their corresponding labels, divided as follows:
Each subset includes video patches extracted from different and independent IVM videos. The organization of the training folder and of the Phase 1 test set folder is as follows:
Training_set/
├── 01
│ ├── 0/ 01_1.avi, 01_2.avi, … (Class 0)
│ ├── 1/ 01_5.avi, 01_6.avi, … (Class 1)
├── 02
│ ├── 0/ 02_1.avi, 02_2.avi, … (Class 0)
│ ├── 1/ 02_5.avi, 02_6.avi, … (Class 1)
├── ...
The folder numbers (e.g. 01/, 02/, …) correspond to different IVM videos from which the video-patches were extracted. The subfolder numbers 0/ and 1/ correspond to the ground truth labels for the contained video patches.
In class 1 the change of direction occurs in the middle time point and in the center of the video patches.
Evaluation
The evaluation metrics for the challenge are designed to comprehensively assess the models' performance in classifying video-patches. The metrics include:
- Area Under the ROC Curve (AUC)
- Precision
- Recall
- Balanced Accuracy
The final score will be calculated as:
score = 0.4*AUC + 0.2*(Precision+Recall+Balanced Accuracy)
NEW: Submission & Leaderboard
You can submit the results of your method in a CSV file indicating the video id and the cass-1 probability [from 0 to 1].
(e.g.
02_12.avi,0.48
02_4.avi,0.65
02_3.avi,0.84
...
)
An example can be found here https://www.dp-lab.info/cbvcc/sample_file.csv
Link to submit your results (user and password received via email are required to login): https://www.dp-lab.info/cbvcc/data/submitresults.php
Link to view the current leaderboard: https://www.dp-lab.info/cbvcc/
NEW: Submission to Phase 2 (final)
please submit your results on the test set of the phase 2 at the following link
Submit: https://www.dp-lab.info/cbvcc/data/submitphase2.php
Rules of Participation
- The data used to train algorithms may be restricted to the data provided by the challenge.
- Participants must not manually annotate the test dataset to train a supervised model for phase 1 or 2 submissions.
- All results will be made publicly available.
- During the submission phase, participants will be asked to upload their model predictions (in terms of probabilities) and an abstract describing the methodology used (in Phase 2).
- There is no monetary prize for the winners of the challenge.
- Both AI-based and non-AI-based methods are allowed.
- Note: We will soon provide you with instructions for the citation of the provided data.
We will collaborate with participants to create a comprehensive journal article summarizing the key results and analyses from this challenge. Participants who submit valuable work are welcome to contribute to the publication, with up to three authors from each team being acknowledged.
In order for us to include you in our paper:
- Please submit a detailed description of your solution with your final test phase submission.
- You are welcome to submit additional paragraphs and figures about your submission via email (see the Contact info on the Organizers page).
Besides, we encourage all participants to independently submit their results without imposing any publication embargo.
👥 Organizers
- Diego Ulisse Pizzagalli - Faculty of Biomedical Sciences, Università della Svizzera Italiana, Lugano, Switzerland
- Rolf Krause - Euler Institute, Università della Svizzera Italiana, Lugano, Switzerland
- Santiago Fernandez Gonzalez - Institute for research in biomedicine, Bellinzona, Switzerland
- Raffaella Fiamma Cabini - Euler Institute, Università della Svizzera Italiana, Lugano, Switzerland
- Elisa Palladino - Institute for research in biomedicine, Bellinzona, Switzerland
- Enrico Moscatello - Euler Institute, Università della Svizzera Italiana, Lugano, Switzerland
For further details please contact This email address is being protected from spambots. You need JavaScript enabled to view it.
