The routine use of a digital tool for the tumor cell fraction quantification in molecular pathology: an international validation of QuANTUM

Objective. The absolute and relative quantification of tumor cell fraction (TCF) in tissue samples for molecular pathology testing is time-consuming and poorly reproducible.
Methods. Here we report the results of an international survey on non-small cell lung cancer (NSCLC), validating the Qupath Analysis of Nuclei from Tumor to Uniform Molecular tests (QuANTUM) automated computational pipeline for TCF quantification.
Results. The TCF obtained with QuANTUM is reliable, as demonstrated by the comparison with the manual counting of cells (ground truth, GT) in cell blocks, small biopsies and surgical specimens (overall correlation of 0.89). The visual evaluation of QuANTUMprocessed images increased the pathologists’ agreement with GT and QuANTUM of +0.16, +0.21, +0.09 and +0.17, +0.29, +0.21 across the three sample types, respectively. An overall increase in cases classified as containing ≥100 tumor cells for all sample types was noted after QuANTUM (from 75 cases, 63% to 96 cases, 80% among cell blocks, p = 0.003).
Conclusions. QuANTUM is an easy-to-use and reliable tool for the TCF assessment and its employment significantly modifies the visual estimation by pathologists, improving the assessment of NSCLC cases for molecular analysis

Introduction

The molecular analysis of solid tumors is changing the pathology lab routine practice ¹, even if pre-analytical steps such as tumor cell fraction (TCF) counting may be time-consuming and poorly reproducible ^2-4. The standard morphological evaluation of slides by the human eyes gives a semiquantitative final score, which is usually part of the report as an adequacy parameter of the results of next-generation sequencing (NGS) ^5-9. Even if conventionally accepted, this method is notoriously affected by a significant degree of inter-observer variability ^5-8, which can potentially be faced by the adoption of digital pathology in the diagnostic workflow ^9-14. The association of artificial intelligence (AI) tools for the simplification of repetitive tasks ^15,16 has already proved its usefulness in the molecular characterization of non-small cell lung cancer (NSCLC) ^17,18. Recently, a Working Group of the Italian Society of Pathology (Gruppo di Patologia Molecolare e Medicina di Precisione - PMMP) proposed a computational QuPath-based pipeline to obtain the TCF (Qupath Analysis of Nuclei from Tumor to Uniform Molecular tests - QuANTUM), demonstrating its diagnostic reliability and robustness with consequent potential changes in the final NGS report ¹⁹.

Now we validate the QuANTUM computational pipeline thanks to an international survey to test the introduction of this digital tool for the routine evaluation of TCF in NGS analysis.

Materials and methods

CASES

We randomly extracted 30 samples from the NSCLC cases that underwent molecular analysis in 2023 at the Oncological Molecular Pathology Unit of Fondazione IRCCS San Gerardo dei Tintori, Università degli Studi di Milano-Bicocca (UNIMIB) in Monza, Italy. The study was approved by the local Ethics Committee (prot. 35859, 24/10/2022). The three types of samples, i.e. cell blocks (CB), small biopsies (SB) and surgical specimens (SS), were equally represented. The Hematoxylin and Eosin (H&E) slides retrieved from the archives were all already pen-marked by the molecular pathologist (DS) to delineate the tumor-containing regions on which the routine TCF counting and the subsequent dissection for molecular analysis were performed. These slides were scanned at 20x magnification (0.4416 MPP, Nanozoomer S60, Hamamatsu, Shizuoka, Japan) and the obtained whole slide images (WSIs) were imported in QuPath v0.4.4²⁰. On these WSIs, one screenshot per case from representative areas (magnification x10, 1496x934 pixels static images) were taken in. jpg format, containing an admixture of neoplastic and non neoplastic cells to reproduce as well as possible the evaluation of TCF in the real setting. For each picture, a cell-by-cell visual count and classification into “Tumor” or “Non-neoplastic”categories (e.g. immune cells, stromal cells and normal epithelial cells) were obtained by two expert lung pathologists (FB and FaPa) after consensus for each cell in the sample and considered like the reference/ground truth (GT), as previously suggested ⁸. Then, the ground truth tumor cell fraction (gtTCF) was calculated. Subsequently, the recently introduced QuANTUM pipeline was applied on each image ¹⁹ and additional screenshots were taken in the same area at the same magnification (post-QuANTUM images,. jpg). The absolute number of total, tumor (red detections) and other (green detections) cells extracted from the region captured were recorded (Supplementary Fig. 1), and the computed tumor cell fraction (cTCF) was calculated as the ratio of tumor and total cells (Supplementary Tab. I). The application of QuANTUM was performed on a standard workstation equipped with an Intel Core i5-8265U processor (1.6 GHz), 8 GB of memory and an integrated Intel UHD graphics card.

SURVEY

The obtained images were used to build two different surveys (Google Forms, Mountain View, USA), the first containing the original images (H&E slides without detections) and the second with the same images post-QuANTUM. In the second survey, post-QuANTUM images were provided to pathologists only as visual representations of cell detection results, with tumor cells marked in red and other cells in green, without displaying the exact cTCF values obtained with QuANTUM. This approach was intended primarily to avoid any bias from the cTCF values on the pathologists’ judgment and to assess whether they could achieve greater convergence toward a more precise TCF even without the exact tumor cell percentage.

For each image, the following information were requested:

1. pathologist TCF (pTCF, %), in 10% increments (from 0 to 100%);
2. number of vital tumor cells (< or ≥100).

Surveys were submitted sequentially, with a 2 weeks washout period, to a panel of 12 international experienced pathologists, who independently scored the cases in both the surveys. Results were afterward extracted in Google Sheets for statistical analysis.

STATISTICAL ANALYSIS

Continuous variables were summarized using mean ± standard deviation (SD), while qualitative variables were presented as counts and relative frequencies. To compare means and qualitative variables, t-tests, chi square and Fisher’s exact tests were employed, depending on the nature of the data. The significance level was set at 0.05. To investigate the agreement between QuANTUM and GT in quantifying TCF in the 3 different sources, the method of Bland and Altman was used and the 95% limits of agreement were calculated. For assessing the agreement of the pTCF observations pre and post-QuANTUM both the Bangdiwala’s B-statistic and Cohen’s kappa have been used, with values ranging from 0 (no agreement) to 1 (perfect agreement). Due to the design of the study (multiple images assessed by multiple raters for ordinal/semi-continuous data, e.g. TCF), the former metrics was preferred for its higher suitability for multi-observer studies, the capacity to handle agreement beyond chance, the better performances with ordinal data and the possibility to have visual and intuitive interpretation through an agreement chart ²¹. A TCF ≥ 20% and a value of viable tumor cells ≥100 were considered as the minimum desirable prerequisite for the reliability of the subsequent molecular analysis (clinically relevant cutoffs) ²²,²³. Statistical analyses were performed using the open-source R software v.3.6.0 (R Foundation for Statistical Computing, Vienna, Austria).

Results

GT VS QUANTUM

The GT estimation resulted in an overall average absolute and % tumor cellularity (gtTCF) of 1625 ± 1058 and 45% ± 19, with values ranging from 170 to 4491 and from 8% to 84%, respectively. No statistically significant differences were observed among the different sample types for gtTCF in % (p = 0.995, 0.332 and 0.345 for CB vs SB, SB vs SS and CB vs SS, respectively). None of the specimens were considered inadequate based on absolute tumor cellularity (< 100) and only 2 cases had a gtTCF < 20% (both in the SS group: #26 with 8% and #21 with 15%). The QuANTUM pipeline required an average of 5 min per case (mean time: 308 ± 98 sec) and produced an overall average absolute and % tumor cellularity (cTCF) of 1260 ± 888 and 39% ± 17%, with values ranging from 264 to 3950 and from 7% to 79%, respectively. No statistically significant differences were found among the different sample types for cTCF in % (p = 0.130, 0.862 and 0.101 for CB vs SB, SB vs SS and CB vs SS, respectively). As with GT, none of the specimens were inadequate (< 100 tumor cells) and only 2 cases had a cTCF < 20% (1 in the SB group: #17 with 17% and 1 in the SS group: #26 with 7%). A comparison of the average TCF values obtained with GT and QuANTUM showed no statistically significant differences (Tab. I). A strong correlation (0.89, Fig. 1a) was observed between the two methods, with good overall accuracy (-6.42, 95%CI 10.76, -23.61) and similar accuracy within the CB, SB and SS groups (CB: -0.81, 95%CI 13.33, -14.95; SB: -12.98, 95%CI -2.02, -23.93; SS: -5.48, 95%CI 12.13, -23.09). Analysis at the individual case level revealed slight discrepancies in TCF values between GT and QuANTUM, with differences of up to 20% in a minority of cases. However, the results remained comparable around the clinically relevant 20% TCF cutoff (Fig. 1b).

COMPARISON OF PATHOLOGIST EVALUATIONS PRE- AND POST- QUANTUM

The average pTCF values assigned by the pathologists before (pre) and after (post) QuANTUM processing are reported in Supplementary Table II. Individual pathologist evaluations for each case are graphically represented in Figure 2, while the overall trend is shown in Figure 3. A general trend of pTCF overestimation is evident in the pre-QuANTUM phase (red line), with greater variability among pathologists compared to the post-QuANTUM assessment (green line). Post-QuANTUM evaluations generally align more closely with both GT and QuANTUM values, particularly for SS and CB, and show a reduction in inter-pathologist variability. The joint distribution of values obtained pre and post-QuANTUM, compared with QuANTUM itself, is reported in Supplementary Table III. The agreement between pathologists’ pre-QuANTUM assessments and reference values (GT and QuANTUM) was generally poor across all sample types (0.25 and 0.48, respectively, see Tab. II and Fig. 4 for the B-statistic values and Supplementary Tab. IV for Cohen’s kappa). However, Bangdiwala’s B index improved in the post-QuANTUM phase, reaching substantial agreement, often exceeding 0.50.

IMPACT OF QUANTUM ON PTCF AT CLINICALLY RELEVANT CUTOFFS

A comparison of the pre- and post-QuANTUM average pTCF values showed an overall increase in cases classified as containing ≥100 tumor cells across all sample types. The most notable and statistically significant difference was observed in the CB group (from 75 cases, 63%, to 96 cases, 80%, p = 0.003, Tab. III). Conversely, a slight decrease in cases classified above the ≥20% cutoff was noted in the post-QuANTUM assessment across all sample types, although none of these changes reached statistical significance.

Discussion

AI reduces pathologists’ effort for tedious and time-consuming tasks, and this is relevant especially for those evaluations still affected by relatively low interobserver reproducibility ²⁴, as already well established for specific use cases (e.g. in breast cancer for the immunohistochemical staining assessment of Ki-67, ER, PR and HER2, as well as for grading) ²⁵. This progressive shift did not spare the lung pathology domain, in which the application of AI tools already proved the capability of differentiating cancer histotypes ²⁶, assessing the tumor grade of adenocarcinomas ²⁷, and quantifying the expression of PD-L1 ²⁸. The recent description of QuANTUM as a computational assistant to quantify the absolute and relative tumor cellularity in biopsy and surgical NSCLC samples further filled the gap of adequacy assessment for molecular analysis, which is paramount in the precision medicine era ¹⁹. This tool already showed significant differences between the pTCF of molecular and general pathologists and a good user-friendliness due to its implementability within the open-access software QuPath ¹⁹.

In the current paper, we examined its impact on the visual assessment of real cases by a panel of experienced international pathologists to test its introduction in routine daily practice. This confirmed the reliability of QuANTUM, demonstrating substantial overlap with the GT on the different sample types (strong correlation of 0.89), with minor fluctuations of the percentage, still irrelevant around the clinically used cutoff of 20%. Moreover, the comparative analysis of pathologists evaluations pre and post-QuANTUM showed a progressive approximation of the pTCF values on the computationally processed image, as demonstrated by the increase in the agreement measures and already described in the literature ²⁹. This experience confirms the overestimation of TCF by pathologists in the pre-QuANTUM images, even more relevant for high percentages of tumor cells, which reinforces the hypothesis that human eyes can be influenced by the surface area occupied by the tumor more than by the actual number of tumor cells in the sample ^6,7,30. The introduction of QuANTUM processed images also reduced the inter-observer variability, as shown by the post-QuANTUM curves vs the GT and QuANTUM ones (pathologists’ agreement with GT and QuANTUM of +0.16, +0.21, +0.09 and +0.17, +0.29, +0.21 across the three sample types), allowing a standardization of TCF quantification. Finally, the application of QuANTUM increases the number of cases eligible for the molecular analysis as per absolute tumor cellularity ≥100 ²³ in all sample types, with a statistically significant improvement in the cell block group (from 75 cases, 63% to 96 cases, 80%, p = 0.003), which is paramount in the assessment of advanced NSCLC cases, representing a potential barrier criterion for the access to life-saving molecular analysis. On the other hand, a slight but not significant reduction of cases above the 20% cutoff ²² was noted post-QuANTUM in all categories, suggesting the need to define more appropriate TCF adequacy values with the introduction of these computational tools.

This study aimed to analyze the impact of introducing a computational tool to help pathologists converge on a more reliable TCF value for molecular analysis, providing valuable insights into the reliability of this tool in a survey scenario designed to closely resemble the clinical setting. However, certain limitations remain to be addressed, such as the selection of static regions/images within the pen-marked areas of the WSIs for each case. This approach simplified analysis while still retaining sufficient detail for a reliable TCF assessment by pathologists, providing them with access to the entire WSI – or at least the pen-marked area for molecular analysis – with the ability to freely navigate the slide at different magnifications would offer an even more realistic representation of how the tool would function in a clinical setting. Furthermore, the enrollment of cases/images and the number of pathologists involved in the present survey were not determined by a statistical power analysis but rather aimed to strike a balance between assembling a robust cohort of international colleagues and selecting a diverse set of images that could be easily submitted for analysis within the survey.

This approach may limit the statistical significance of some analyses performed, and with a larger cohort of cases, some non-significant trends observed here could potentially reveal more meaningful insights into the differences identified. Finally, involving specialists from different international centers represents a crucial first step in assessing the generalizability of QuANTUM. However, a larger cohort of cases and images from multiple centers is still needed to account for potential pre-analytical variability and to evaluate its impact on the reliability of cTCF assessment as future perspective.

Conclusions

QuANTUM is an easy-to-use and reliable tool for the TCF assessment and its employment significantly modifies visual estimation by pathologists, improving the agreement on TCF assessment of NSCLC cases for molecular analysis. Further studies are needed to unveil the impact of cTCF on the final molecular results (e.g. copy number variation, CNV) and to establish whether tumor cellularity cutoffs need to be revised in the digital and computational pathology era.

ACKNOWLEDGEMENTS

*Patologia Molecolare e Medicina di Precisione (PMMP), Società Italiana di Anatomia Patologica e Citologia (SIAPEC) Working Group.

CONFLICTS OF INTEREST STATEMENT

PP received personal fees (as consultant and/or speaker bureau) from Novartis unrelated to the current work. FraPe received personal fees (as consultant and/or speaker bureau) from Menarini, Roche unrelated to the current work. UM received personal fees (as consultant and/or speaker bureau) from Amgen, Boehringer Ingelheim, Diaceutics, Eli Lilly, GSK, Merck, MSD, Roche, Thermo Fisher Scientific; and from AstraZeneca, Diatech, Hedera, Janssen, Novartis unrelated to the current work. FP received personal fees (as consultant and/or speaker bureau) from Amgen, Eli Lilly, GSK, Merck, MSD, Roche, AstraZeneca, Diatech, Janssen, Novartis, Diapath, unrelated to the current work. GT reports personal fees (as consultant and/or speaker bureau) from Boehringer Ingelheim, Roche, MSD, Pfizer, Eli Lilly, BMS, GSK, Menarini, AstraZeneca, Amgen and Bayer unrelated to the current work. VL received personal fees (as consultant and/or speaker bureau) from Eli Lilly, Roche, Novartis unrelated to the current work. EGR received advisory fees, honoraria, travel accommodation and expenses, grants and/or non-financial support from AstraZeneca, Exact Sciences, GSK, Illumina, MSD, Novartis, Roche, StemlineMenarini, Sophia Genetics, ThermoFisher Scientific. SAK is a shareholder in Histofy, an AI startup based in Birmingham, UK. CE received personal fees from Sakura, Diapath, Diaceutics, Epredia and MSD. JT received advisory fees from MSD, unrelated to the current work.

FUNDING

The work has been funded by the project in the Italian Ministry of the University MUR Dipartimenti di Eccellenza 2023-2027 (l. 232/2016, art. 1, commi 314-337). This work was partially funded by the National Plan for NRRP Complementary Investments (PNC, established with the decree-law 6 May 2021, n. 59, converted by law n. 101 of 2021) in the call for the funding of research initiatives for technologies and innovative trajectories in the health and care sectors (Directorial Decree n. 931 of 06-06-2022) - project n. PNC0000003 - AdvaNced Technologies for Human-centrEd Medicine (project acronym: ANTHEM).

AUTHORS’ CONTRIBUTIONS

VL, GioCa, DS and MM created the survey and selected cases adopting QuANTUM to obtain pictures for the second survey; GiuCa performed the statistical analysis and SG supervised the analysis; GT, UM and EB provided specialized perspective in molecular and digital pathology; FB and FaPa performed the manual annotation for the ground truth; CE, JP, EGR, MF, PP, FraPe, LP, JT, AP, SAK, AM, GP were involved in the two surveys and in the evaluation of cases in blind before and after the application of QuANTUM; VL, SG and FaPa critically revised the manuscript in its final version; FaPa provided the funding acquisition and administrative support. All authors were involved in writing the paper and had final approval of the submitted and published versions.

ETHICAL CONSIDERATION

Approval was obtained from the local ethics committee (prot. 35859, 24/10/2022).

Supplementary material

Supplementary Figure 1. Examples of figures used for the survey. Cell blocks (first row), small biopsies (central row) and surgical specimens (last row). “Tumor cells” are highlighted in red and “Other cells” are reported in green. On the left column, images in their original form (H&E), on the right column the same images post-QuANTUM processing.

History

Received: October 4, 2024

Accepted: March 18, 2025

Figures and tables

Figure 1. In (a) the Bland-Altman graph showing an strong correlation between the GT and QuANTUM (all the points converge to the diagonal), with only mild underestimation of TCF values by QuANTUM as compared to GT (about -6% in average). In (b) graphical representation of the divergence of TCF obtained with QuANTUM and GT for each single case per sample category. On the X-axis is reported the average TCF value between GT and QuANTUM, on the Y-axis the QuANTUM-GT TCF difference values. For lower values especially around the clinically relevant cutoff of 20% (yellow bands of the left graphics), the differences are < 10%, as demonstrated by exemplificative cases reported on the right columns (blue circles).

Figure 2. The graphical representation illustrates the pTCF values (% and absolute values around the 100 cutoff) for each pathologist across all cases, both before (a and c) and after (b and d) QuANTUM processing. In the figure, TCF% values are ordered by ascending gtTCF, while absolute values are arranged by progressive case ID. This figure highlights the systematic overestimation of pTCF% by certain pathologists (e.g., Path #6) and demonstrates the convergence of pTCF% from the pre- to post-QuANTUM assessment toward the QuANTUM and gtTCF values (e.g., cases 3, 11, and 22, as further detailed in Figure 3). Additionally, the application of QuANTUM-processed images reduced the number of pTCF < 100 (e.g. for cases 5, 6, 9, and 10), aiding some pathologists in significantly decreasing the number of cases deemed inadequate for molecular analysis (e.g., Path #10 and #12).

Figure 3. Distribution of the TCF pre (red), post (green) as compared to GT (yellow) and QuANTUM (blue) values divided per sample type and distributed in order of rising % (Y-axis), with ID case reported in the X-axis. Examples of significant changes in the attributed TCF value after QuANTUM are reported for each sample type (red rectangles).

Figure 4. Concordance plots of the different comparisons (pre/post vs GT and QuANTUM) divided per sample type. The increase in agreement post-QuANTUM with both GT and QuANTUM is demonstrated by the higher proximity of the boxes to the red diagonal. White rectangles are determined by the marginal totals, and in the case of perfect agreement, they are all perfect squares, and the area of the blackened squares completely corresponds to the area of the rectangles. Black squares correspond to the exact observed agreement, and lesser agreement is visualized by comparing the areas of the blackened squares to the rectangles. Gray squares indicate partial agreement by including a weighted contribution from off-diagonal cells.

Table I. Comparison of the average TCF values obtained with QuANTUM and GT.

	GT (%, mean ± SD)			QuANTUM cTCF (%, mean ± SD)			p-value
Cell blocks	48%	±	20%	47%	±	20%	0.9
Small biopsies	48%	±	19%	35%	±	14%	0.1
Surgical specimens	39%	±	20%	34%	±	15%	0.5

Table II. Agreement measured with B-statistic of the different correlations (pre and post vs GT and QuANTUM) of the assessment by pathologists divided in the different sample type group.

	B-statistic Weighted
	PRE vs GT	POST vs GT	PRE vs QuANTUM	POST vs QuANTUM
Cell blocks	0.41	0.57	0.48	0.65
Small biopsies	0.25	0.46	0.38	0.67
Surgical specimens	0.46	0.55	0.27	0.48

Table III. Comparison of the distribution of the pathologists observations (absolute tumoral cell counts and pTCF) around clinically relevant cutoffs pre- and post-QuANTUM in the different sample types, with relative p-values.

	Tumor cells (n, %)			TCF (n, %)
	< 100	≥100	p-value	< 20%	≥20%	p-value
Cell blocks (Pre)	45 (38%)	75 (63%)	0.003	7 (5.8%)	113 (94%)	0.11
Cell blocks (Post)	24 (20%)	96 (80%)		14 (12%)	106 (88%)
Small biopsies (Pre)	12 (10%)	108 (90%)	0.14	11 (9.2%)	109 (91%)	0.7
Small biopsies (Post)	6 (5.0%)	114 (95%)		13 (11%)	107 (89%)
Surgical specimens (Pre)	4 (3.3%)	116 (97%)	0.12	14 (12%)	106 (88%)	0.2
Surgical specimens (Post)	0 (0%)	120 (100%)		21 (18%)	99 (83%)