Ultrasound imaging is pivotal in clinical diagnostics due to its affordability, portability, safety, real-time capability, and non-invasive nature. It is widely utilized for examining various organs, such as the breast, thyroid, ovary, cardiac, and more. However, the manual interpretation and annotation of ultrasound images are time-consuming and prone to variability among physicians. While single-task

Few-shot medical image segmentation typically uses a joint model for registration and segmentation. The registration model aligns a labeled atlas with unlabeled images to form initial masks, which are then refined by the segmentation model. However, inevitable spatial misalignments during registration can lead to inaccuracies and diminished segmentation quality. To address this, we developed EFS-MedSeg

The fetal standard plane acquisition task aims to detect an Ultrasound (US) image characterized by specified anatomical landmarks and appearance for assessing fetal growth. However, in practice, due to variability in human operator skill and possible fetal motion, it can be challenging for a human operator to acquire a satisfactory standard plane. To support a human operator with this task, this paper

Cancer survival prediction based on multimodal data (e.g., pathological slides, clinical records, and genomic profiles) has become increasingly prevalent in recent years. A key challenge of this task is obtaining an effective survival-specific global representation from patient data with highly complicated correlations. Furthermore, the absence of certain modalities is a common issue in clinical practice

Tertiary lymphoid structures (TLS) are ectopic lymphoid aggregates that form under specific pathological conditions, such as chronic inflammation and malignancies. Their presence within the tumor microenvironment (TME) is strongly correlated with patient prognosis and response to immunotherapy, making TLS detection in whole-slide pathological images (WSIs) crucial for clinical decision-making. Although

Invasive breast cancer (IBC) is a prevalent malignant tumor in women, and precise grading plays a pivotal role in ensuring effective treatment and enhancing survival rates. However, accurately grading IBC presents a significant challenge due to its heterogeneous nature and the need to harness the complementary information from multiple nuclei sources in histopathology images. To tackle this critical

Magnetic Resonance Imaging (MRI) produces images with different contrasts, providing complementary information for clinical diagnoses and research. However, acquiring a complete set of MRI sequences can be challenging due to limitations such as lengthy scan time, motion artifacts, hardware constraints, and patient-related factors. To address this issue, we propose a novel method to learn Contrast and

Cardiovascular magnetic resonance imaging is emerging as a crucial tool to examine cardiac morphology and function. Essential to this endeavour are anatomical 3D surface and volumetric meshes derived from CMR images, which facilitate computational anatomy studies, biomarker discovery, and in-silico simulations. Traditional approaches typically follow complex multi-step pipelines, first segmenting images

Difficulty in resisting gravity is a common leg motor impairment in stroke patients, significantly impacting daily life. Automated clinical-level quantification of motor-leg videos based on the National Institutes of Health Stroke Scale is crucial for consistent and timely stroke diagnosis and assessment. However, real-world applications are challenged by interference impacting motion representation

Multiple instance learning (MIL) is a powerful approach for whole-slide pathological image (WSI) analysis, particularly suited for processing gigapixel-resolution images with slide-level labels. Recent attention-based MIL architectures have significantly advanced weakly supervised WSI classification, facilitating both clinical diagnosis and localization of disease-positive regions. However, these methods

Automatic radiology report generation can alleviate the workload for physicians and minimize regional disparities in medical resources, therefore becoming an important topic in the medical image analysis field. It is a challenging task, as the computational model needs to mimic physicians to obtain information from multi-modal input data (i.e., medical images, clinical information, medical knowledge

Recently, large pre-trained models (LPM) have achieved great success, which provides rich feature representation for downstream tasks. Pre-training and then fine-tuning is an effective way to utilize LPM. However, the application of LPM in the medical domain is hindered by the presence of a large number of parameters and a limited amount of labeled data. In clinical practice, there exists a substantial

Lung infections are the leading cause of death among infectious diseases, and accurate segmentation of the infected lung area is crucial for effective treatment. Currently, segmentation methods that rely solely on imaging data have limited accuracy. Incorporating text information enriched with expert knowledge into the segmentation process has emerged as a novel approach. However, previous methods

Image synthesis from Magnetic Resonance (MR) to Computed Tomography (CT) can estimate the electron density of tissues, thereby facilitating Positron Emission Tomography (PET) attenuation correction in whole-body PET/MR imaging. Whole-body MR-to-CT synthesis faces several challenges including the spatial misalignment caused by tissue variety and respiratory movements, and the complex intensity mapping

State-of-the-art research of traditional computer vision is increasingly leveraged in the surgical domain. A particular focus in computer-assisted surgery is to replace marker-based tracking systems for instrument localization with pure image-based 6DoF pose estimation using deep-learning methods. However, state-of-the-art single-view pose estimation methods do not yet meet the accuracy required for

Late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) imaging is considered the in vivo reference standard for assessing infarct size (IS) and microvascular obstruction (MVO) in ST-elevation myocardial infarction (STEMI) patients. However, the exact quantification of those markers of myocardial infarct severity remains challenging and very time-consuming. As LGE distribution patterns can

Accurate and efficient group-wise registration for medical images is fundamentally important to construct a common template image for population-level analysis. However, current group-wise registration faces the challenges posed by the algorithm’s efficiency and capacity, and adaptability to large variations in the subject populations. This paper addresses these challenges with a novel Nested Hierarchical

Endoscopic Submucosal Dissection (ESD) constitutes a firmly well-established technique within endoscopic resection for the elimination of epithelial lesions. Dissection trajectory prediction in ESD videos has the potential to strengthen surgical skills training and simplify surgical skills training. However, this approach has been seldom explored in previous research. While imitation learning has proven

Quality control (QC) of structures segmentation in volumetric medical images is important for identifying segmentation errors in clinical practice and for facilitating model development by enhancing network performance in semi-supervised and active learning scenarios. This paper introduces SegQC, a novel framework for segmentation quality estimation and segmentation error detection. SegQC computes

Medical segmentation models are evaluated empirically. As such an evaluation is based on a limited set of example images, it is unavoidably noisy. Beyond a mean performance measure, reporting confidence intervals is thus crucial. However, this is rarely done in medical image segmentation. The width of the confidence interval depends on the test set size and on the spread of the performance measure

Confounding factors inherent in medical images can significantly impact the causal exploration capabilities of deep learning models, resulting in compromised accuracy and diminished generalization performance. In this paper, we present an innovative methodology named CausalMixNet that employs query-mixed intra-attention and key&value-mixed inter-attention to probe causal relationships between input

In the domain of medical imaging, many supervised learning based methods for segmentation face several challenges such as high variability in annotations from multiple experts, paucity of labelled data and class imbalanced datasets. These issues may result in segmentations that lack the requisite precision for clinical analysis and can be misleadingly overconfident without associated uncertainty quantification

The absence of MRI sequences is a common occurrence in clinical practice, posing a significant challenge for prediction modeling of non-invasive diagnosis of glioma (GM) via fusion of multi-sequence MRI. To address this issue, we propose a novel unified reciprocal assistance imputation-representation learning framework (namely REPAIR) for GM diagnosis modeling with incomplete MRI sequences. REPAIR

Medical image segmentation is a pivotal task within the realms of medical image analysis and computer vision. While current methods have shown promise in accurately segmenting major regions of interest, the precise segmentation of boundary areas remains challenging. In this study, we propose a novel network architecture named CTO, which combines Convolutional Neural Networks (CNNs), Vision Transformer

This work presents a framework for monocular 6D pose estimation of surgical instruments in open surgery, addressing challenges such as object articulations, specularity, occlusions, and synthetic-to-real domain adaptation. The proposed approach consists of three main components: (1) synthetic data generation pipeline that incorporates 3D scanning of surgical tools with articulation rigging and physically-based

Virtual Imaging Trials (VIT) offer a cost-effective and scalable approach for evaluating medical imaging technologies. Computational phantoms, which mimic real patient anatomy and physiology, play a central role in VITs. However, the current libraries of computational phantoms face limitations, particularly in terms of sample size and heterogeneity. Insufficient representation of the population hampers

The recent popularity of foundation models and the pre-train-and-adapt paradigm, where a large-scale model is transferred to downstream tasks, is gaining attention for volumetric medical image segmentation. However, current transfer learning strategies devoted to full fine-tuning for transfer learning may require significant resources and yield sub-optimal results when the labeled data of the target

In this paper, we introduce the Visual Query-based task of Video Clip Localization (VQ-VCL) for medical video understanding. Specifically, we aim to retrieve a video clip containing frames similar to a given exemplar frame from a given input video. To solve the task, we propose a novel visual query-based video clip localization model called TIER-LOC. TIER-LOC is designed to improve video clip retrieval

Clinical data warehouses, which have arisen over the last decade, bring together the medical data of millions of patients and offer the potential to train and validate machine learning models in real-world scenarios. The quality of MRIs collected in clinical data warehouses differs significantly from that generally observed in research datasets, reflecting the variability inherent to clinical practice

Deep Learning in Image Registration (DLIR) methods have been tremendously successful in image registration due to their speed and ability to incorporate weak label supervision at training time. However, existing DLIR methods forego many of the benefits and invariances of optimization methods. The lack of a task-specific inductive bias in DLIR methods leads to suboptimal performance, especially in the

In digital pathology, precise nuclei segmentation is pivotal yet challenged by the diversity of tissue types, staining protocols, and imaging conditions. Recently, the segment anything model (SAM) revealed overwhelming performance in natural scenarios and impressive adaptation to medical imaging. Despite these advantages, the reliance on labor-intensive manual annotation as segmentation prompts severely

Augmented reality (AR) has significant potential to enhance the identification of critical locations during endoscopic surgeries, where accurate endoscope calibration is essential for ensuring the quality of augmented images. In optical-based surgical navigation systems, asynchrony between the optical tracker and the endoscope can cause the augmented scene to diverge from reality during rapid movements

Recently, the state space model Mamba has demonstrated efficient long-sequence modeling capabilities, particularly for addressing long-sequence visual tasks in 3D medical imaging. However, existing generative self-supervised learning methods have not yet fully unleashed Mamba’s potential for handling long-range dependencies because they overlook the inherent causal properties of state space sequences

The reliance on computationally intensive U-Net and Transformer architectures significantly limits their accessibility in low-resource environments, creating a technological divide that hinders global healthcare equity, especially in medical diagnostics and treatment planning. This divide is most pronounced in low- and middle-income countries, primary care facilities, and conflict zones. We introduced

Precise segmentation and uncertainty estimation are crucial for error identification and correction in medical diagnostic assistance. Existing methods mainly rely on pixel-wise uncertainty estimations. They (1) neglect the global context, leading to erroneous uncertainty indications, and (2) bring attention interference, resulting in the waste of extensive details and potential understanding confusion

Traditional medical image generation often lacks patient-specific clinical information, limiting its clinical utility despite enhancing downstream task performance. In contrast, medical image translation precisely converts images from one modality to another, preserving both anatomical structures and cross-modal features, thus enabling efficient and accurate modality transfer and offering unique advantages

Surface-based analysis of the cerebral cortex is ubiquitous in human neuroimaging with MRI. It is crucial for tasks like cortical registration, parcellation, and thickness estimation. Traditionally, such analyses require high-resolution, isotropic scans with good gray–white matter contrast, typically a T1-weighted scan with 1 mm resolution. This requirement precludes application of these techniques

Idiopathic Pulmonary Fibrosis (IPF) is a progressive lung disease that continuously scars and thickens lung tissue, leading to respiratory difficulties. Timely assessment of IPF progression is essential for developing treatment plans and improving patient survival rates. However, current clinical standards require multiple (usually two) CT scans at certain intervals to assess disease progression. This

Deep learning-based fundus image enhancement has attracted extensive research attention recently, which has shown remarkable effectiveness in improving the visibility of low-quality images. However, these methods are often constrained to specific datasets and degradations, leading to poor generalization capabilities and having challenges in the fine-tuning process. Therefore, a general method for fundus

Aortic stenosis (AS) is a prevalent heart valve disease that requires accurate and timely diagnosis for effective treatment. Current methods for automated AS severity classification rely on black-box deep learning techniques, which suffer from a low level of trustworthiness and hinder clinical adoption. To tackle this challenge, we propose ProtoASNet, a prototype-based neural network designed to classify

Nuclei instance segmentation and classification of hematoxylin and eosin (H&E) stained digital pathology images are essential for further downstream cancer diagnosis and prognosis tasks. Previous works mainly focused on bottom-up methods using a single-level feature map for segmenting nuclei instances, while multilevel feature maps seemed to be more suitable for nuclei instances with various sizes

Unsupervised deformable multimodal medical image registration often confronts complex scenarios, which include intermodality domain gaps, multi-organ anatomical heterogeneity, and physiological motion variability. These factors introduce substantial grayscale distribution discrepancies, hindering precise alignment between different imaging modalities. However, existing methods have not been sufficiently

Radiogenomic research provides a deeper understanding of breast cancer biology by investigating the correlations between imaging phenotypes and genetic data. However, current radiogenomic research primarily focuses on the correlation between imaging phenotypes and single-genomic data (e.g., gene expression data), overlooking the potential of multi-genomics data to unveil more nuances in cancer characterization

Diffusion-weighted imaging (DWI) is a key modality for studying brain microstructure, but its signals are highly susceptible to noise due to the thermal motion of water molecules and interactions with tissue microarchitecture, leading to significant signal attenuation and a low signal-to-noise ratio (SNR). In this paper, we propose a novel approach, a Cycle-Conditional Diffusion Model (Cycle-CDM) using

Simulation-Based Inference (SBI) has recently emerged as a powerful framework for Bayesian inference: Neural networks are trained on simulations from a forward model, and learn to rapidly estimate posterior distributions. We here present an SBI framework for parametric spherical deconvolution of diffusion MRI data of the brain. We demonstrate its utility for estimating white matter fibre orientations

Magnetic resonance imaging (MRI) is a widely used radiological modality renowned for its radiation-free, comprehensive insights into the human body, facilitating medical diagnoses. However, the drawback of prolonged scan times hinders its accessibility. The k-space undersampling offers a solution, yet the resultant artifacts necessitate meticulous removal during image reconstruction. Although deep

Sparse-view computed tomography (CT) reduces radiation exposure by acquiring fewer projections, making it a valuable tool in clinical scenarios where low-dose radiation is essential. However, this often results in increased noise and artifacts due to limited data. In this paper we propose a novel 3D Gaussian representation (3DGR) based method for sparse-view CT reconstruction. Inspired by recent success

Large-scale generative models have garnered significant attention in the field of medical imaging, particularly for image editing utilizing diffusion models. However, current research has predominantly concentrated on pathological editing involving single or a limited number of labels, making it challenging to achieve precise modifications. Inaccurate alterations may lead to substantial discrepancies

Since the outbreak of the COVID-19 pandemic in 2019, medical imaging has emerged as a primary modality for diagnosing COVID-19 pneumonia. In clinical settings, the segmentation of lung infections from computed tomography images enables rapid and accurate quantification and diagnosis of COVID-19. Segmentation of COVID-19 infections in the lungs poses a formidable challenge, primarily due to the indistinct

We present a methodology to personalize the stiffness response of a biomechanical model of the torso and the spine. In high contrast to previous work, the proposed methodology uses controlled force–deformation data that mimic the conditions of spinal bracing for scoliosis, which leads to personalized biomechanical models that are suitable for computational brace design. The novel methodology relies

Radiologists are in desperate need of automatic radiology report generation (RRG) for alleviating the workload and preventing the inexperienced from making mistakes in diagnosis. From our perspective, each radiology report can be viewed as a mixture of topics, where the topics extend from the disease annotations. Taking into account the abundance of clinical details in radiology reports, harnessing

Deep learning-based medical image segmentation and surface mesh generation typically involve a sequential pipeline from image to segmentation to meshes, often requiring large training datasets while making limited use of prior geometric knowledge. This may lead to topological inconsistencies and suboptimal performance in low-data regimes. To address these challenges, we propose a data-efficient deep

Capturing rich multi-scale features is essential to address complex variations in medical image segmentation. Multiple hybrid networks have been developed to integrate the complementary benefits of convolutional neural networks (CNN) and Transformers. However, existing methods may suffer from either huge computational cost required by the complicated networks or unsatisfied performance of lighter networks

Three-dimensional (3D) spinal ultrasound imaging has demonstrated its promising potential in measuring spinal deformity through recent studies, and it is more suitable for massive early screening and longitudinal follow-up of adolescent idiopathic scoliosis (AIS) compared with X-ray imaging due to its radiation-free superiority. Moreover, some deformities with low observability, such as vertebral rotation

Despite their effectiveness, current deep learning models face challenges with images coming from different domains with varying appearance and content. We introduce SegCLR, a versatile framework designed to segment images across different domains, employing supervised and contrastive learning simultaneously to effectively learn from both labeled and unlabeled data. We demonstrate the superior performance

The coronary angiography-derived fractional flow reserve (FFR) curve, referred to as the Angio-FFR curve, is crucial for guiding percutaneous coronary intervention (PCI). The invasive FFR is the diagnostic gold standard for determining functional significance and is recommended to complement coronary angiography. The invasive FFR curve can quantitatively define disease patterns. The Angio-FFR curve

High-throughput imaging techniques acquire large amounts of images efficiently. These images contain rich biological information including cellular processes. A common method to analyse them is to encode them into quantitative representation vectors. Generally, there are two ways to extract cell biological information into representations, hand-crafted and machine-learning. Although representations

Medical imaging maps tissue characteristics into image intensity values, enhancing human perception. However, comprehending this data, especially in high-stakes scenarios such as surgery, is prone to errors. Additionally, current multimodal methods do not fully leverage this valuable data in their design. We introduce “From Tissue to Sound,” a new paradigm for medical sonic interaction design. This