Submission Guidelines¶

Algorithms are submitted to the PRISM-AI Challenge as inference containers on the Grand Challenge platform. This page describes the requirements and how to get from a trained model to a valid submission.

At a glance¶

What is an inference container?¶

An inference container is a Docker container that encapsulates your trained AI algorithm (architecture + model weights) together with everything needed to:

1. load a mammographic examination,
2. generate predictions: for example, 5-year future breast cancer risk, and
3. store the corresponding output file for evaluation.

To help participants prepare their algorithm for submission, a demo inference container is available on GitHub, which you can adapt to your needs.

It may also be helpful to consult general Grand Challenge documentation, including but not limited to:

• Creating an algorithm
• Making a submission

Example scripts¶

The example repository contains the files that together form a minimal working submission. Key files are:

inference.py¶

The main inference script for running an algorithm on the Grand Challenge platform. It reads the stacked mammograms and their header information, runs a prediction per participant, and writes the output file. The example algorithm predicts breast cancer risk at multiple time points (years 1–5), based on the four views per participant.

Helper code is included to convert the stacked .mha files to individual DICOM files and to populate the required DICOM header fields from the .json files. If your algorithm expects DICOM input, this is the part to reuse; if it works directly on pixel arrays, you can read from the stacks and skip the conversion.

Dockerfile¶

Defines the Docker container used for deployment on Grand Challenge. All required system and Python dependencies are installed during the image build, so no separate local Python or conda environment is needed.

See the following pages for documentation on how to build and test the Docker image (and this page to set things up on Windows), and how to upload your algorithm.

Input interface¶

Each case is a screening examination consisting of the four standard views, in MetaImage (.mha) format, accompanied by the relevant DICOM header information. Examinations are not passed to your container one at a time:

• Mammograms are provided as stacked .mha files, with separate stacks for each of the four standard views: CC-L, CC-R, MLO-L and MLO-R.
• DICOM header information is provided as stacked .json files, again with separate stacks per view, synchronized with the mammograms: the n-th entry of a metadata stack belongs to the n-th image of the corresponding image stack.
• Each stack holds at most 100 images, so your container is called repeatedly (once for every stacked .mha file) to cover the full evaluation set.

Output interface¶

Your container writes a JSON file containing a breast cancer probability score:

• The score must be a probability between 0.0 and 1.0 (or between 0 and 100).
• The 5-year probability is the primary outcome of the Challenge and the basis for the leaderboards.
• Algorithms that additionally output 2-, 3- or 4-year probabilities are evaluated in secondary analyses.

Configuration¶

Before building or running the container, open inference.py and verify the following paths:

INPUT_PATH and OUTPUT_PATH must keep their default values: Grand Challenge mounts the data at these locations. RESOURCE_PATH is yours to change, as long as it matches the location your Dockerfile copies the model weights to.

Compute and runtime¶

When you submit an algorithm, you can choose GPU or no GPU access. If GPU access is selected, the algorithm will have access to an NVIDIA T4 GPU (16 GB VRAM) instance. The runtime limit for processing a single case is three minutes.

Because all weights must be present inside the container and there is no internet access at runtime, load your model once at container start-up rather than once per case.

Testing your container locally¶

To test your algorithm locally before submission, run:

./do_test_run.sh

This script launches the Docker container, mounts the required input and output directories, and runs inference.py inside the container. Input is read from ./test/input and output is written to ./test/output.

Building and submitting your container¶

To build and export the Docker container for upload to Grand Challenge, run:

./do_save.sh

This produces a .tar.gz file that can be uploaded directly to Grand Challenge. For more information on testing and submitting your container, see the this and this documentation.

Trial run and debugging¶

Before making a final submission, it is possible to run your algorithm on a small set of mammograms on Grand Challenge to ensure it will run error-free. Most problems that surface at this stage are input-parsing issues that take minutes to fix and would otherwise invalidate a submission.

The Challenge is structured with two rounds:

1. Initial submission — a first working version. The Challenge organizers review the logs of all submissions and report back on any errors.
2. Final submission — the version that is evaluated on the hidden PRISMA set.

Evaluation¶

After submission and processing, performance metrics are computed automatically on Grand Challenge, based on the pre-defined and publicly available PRISM-AI evaluation scripts. No manual steps by the organizers sit between your predictions and the leaderboard.