Problem

Developing robust and generalizable artificial intelligence models for medical image segmentation faces significant hurdles. Medical data is often fragmented and siloed across different hospitals, making it difficult to centralize for training due to strict privacy regulations like HIPAA and GDPR. Furthermore, imaging modalities vary widely (e.g., CT vs. MRI), leading to "cross-modality domain shifts" where models trained on one modality fail to generalize to another. Existing solutions often assume the availability of all imaging modalities for each patient during training, which is rare in clinical practice, or involve complex architectures that limit adoption. Federated learning (FL) offers a privacy-preserving way to collaborate by sharing model updates instead of raw data, but it does not inherently address these inter-institutional domain shifts, especially when each client only possesses data from a single modality and lacks paired cross-modality scans.

Method

The authors propose FedGIN, a federated learning framework designed for cross-modality medical image segmentation. This framework integrates on-the-fly Global Intensity Nonlinear (GIN) augmentation during local training at each participating institution. The core idea behind GIN is to introduce random intensity transformations at the pixel level using a shallow, randomly-initialized convolutional network. These transformations encourage the segmentation model to learn anatomical features that are consistent across different modalities, rather than relying on modality-specific intensity patterns. The augmented images effectively simulate cross-modality variations, exposing each client's model to diverse synthetic styles resembling unseen modalities. During each communication round, clients apply GIN augmentation locally, train their models, and then send only their updated model parameters to a central server, which aggregates them using the FedAvg algorithm. A key advantage of FedGIN is its compatibility with existing segmentation architectures without requiring modifications and its ability to operate without the need for paired cross-modality data, making it highly suitable for real-world federated settings. The study also systematically evaluated other domain generalization strategies, including frequency-domain augmentations (FMAug, RaffeSDG), other spatial-domain methods (ProRandConv, RC-Unet), and network-level adaptation (DSBN), comparing their effectiveness in the federated cross-modality context.

Results

The experimental evaluation covered two main use cases: binary abdominal organ segmentation and multiclass whole-heart segmentation. For abdominal organ segmentation, incorporating CT data from federated clients significantly improved performance for MRI-scarce institutions, particularly for challenging organs like the pancreas (a 498% increase in Dice score from a near-failure baseline) and gallbladder (a 151.9% increase). FedGIN consistently achieved 93–98% of the segmentation accuracy obtained by centralized training, demonstrating strong cross-modality generalization while preserving data privacy. In multiclass whole-heart segmentation, FedGIN achieved a mean Dice Similarity Coefficient (DSC) of 0.6297, showing an absolute gain of 0.0592 over an MRI-only baseline and retaining 93% of the centralized GIN performance. Across both tasks and various configurations, GIN consistently outperformed network-level adaptation (DSBN), frequency-domain approaches (FMAug, RaffeSDG), and other spatial-domain methods (ProRandConv, RC-Unet). Other augmentation methods often failed significantly in the federated setting due to sensitivity to cross-client distribution shifts or conflicting batch normalization statistics. GIN's spatial-domain approach, by contrast, proved robust by applying simple random convolutional transformations locally before aggregation.

Implications

The study demonstrates that cross-modality federated learning, particularly with GIN augmentation, offers a viable and privacy-preserving solution for multi-institutional collaboration in medical image segmentation. It effectively addresses challenges posed by data silos, privacy regulations, and modality heterogeneity, enabling hospitals with diverse imaging equipment to collectively train robust models without centralizing patient data. The significant performance gains observed in low-data settings and for difficult-to-segment organs highlight the practical value of this approach in clinical scenarios where baseline performance is poor. While the absolute performance numbers are still below state-of-the-art centralized models (due to the low-data regime and inherent challenges of unpaired cross-modality generalization), the method provides a foundational step towards deployable federated AI in healthcare. Future work includes exploring filtering for low-quality augmentations, extending FedGIN to modern deep learning architectures, utilizing full 3D volumes, and conducting more rigorous multi-center variability analyses.