The controlled "VASARI" terminology for describing the MR features of human gliomas was devised based upon prior work (REMBRANDT project). This comprehensive featureset consists of 24 observations familiar to neuroradiologists to describe the morphology of brain tumors on routine contrast-enhanced MRI. De-identified baseline MRI studies for 88 glioblastomas were collected from subjects analyzed as part of The Cancer Genome Atlas (TCGA) initiative. Neuroradiologists in disparate geographic locations were recruited and trained in the use of the featureset using a visual guidebook. Training cases were employed to assess competency and to ensure agreement. A open-source PACS workstation (Clear Canvas) was customized for clinical imaging research evaluation and deployed at each of the centers. As studies were evaluated, scores were uploaded from Clearcanvas to a central AIM (Annotation and Image Markup) Data Service for QC checks and interim analysis. Case assignments were deliberately staged in a staggered fashion to ensure that a minimum of three evaluations were efficiently obtained. Administrative tools were employed by coordinators in a fourth location. Qualitative assessments included: (1) effectiveness of training, (2) ease of deployment & functionality of the informatics solutions and (3) efficiency of the process. Inter-observer variation for each feature was assessed with the generalized kappa statistic of Berry&Miekle.
Training, deployment of resources and completion of three evaluations per case were accomplished in 30 days. Functionality of the IT solutions was rated superior in qualitative assessment. The results indicated strong overall average inter-observer agreement among all six readers. Agreement was highest for tumor side (generalized kappa statistic k=0.943, 95% CI 0.915-0.982) and tumor location (k=0.837, 95% CI 0.807-0.902). Other features with high agreement included proportion enhancing tumor (k=0.656, 95% CI 0.596-0.757), presence of satellites (k=0.663, 95% CI 0.591-0.780), and diffusion (k=0.730, 95% CI 0.664-0.828). Of the remaining, only three features (12.5%) showed low agreement (k<0.4): presence of calvarial remodeling (k=0.366, 95% CI 0.124-0.626), cortical involvement (k=0.167, 95% CI 0.157-0.335), and definition of non-enhancing margin (k=0.374, 95% CI 0.347-0.514).
Inclusion of vetted, tested and validated controlled terminologies into imaging arms of clinical trials is essential in adding value of imaging as a biomarker in cross-cutting correlative studies. Controlled terminologies such as the one described herein for assessment of gliomas can be effectively used by domain experts following a relatively short training period. Technologies developed through the caBIG initiative provide an effective and efficient framework for federated imaging assessments that can expedite cross-correlative analysis with other data repositories (e.g. genomics / proteomics / pathology).
Updates to the TCGA-GBM imaging data set are being stored within TCIA and the research group is continuing to expand on their work.
Supporting Documentation and Metadata
The VASARI project has been conducted in multiple rounds. In the first round 88 subjects were reviewed using the feature set, resulting in 75 finalized scores after removal of cases missing the appropriate scans required to score the case. Similarly in the second round an additional 41 cases were reviewed resulting in 36 new cases being added. Minor changes were made to the VASARI feature definitions between rounds 1 and 2.
- Round 1 VASARI Feature Scores: Multiple reads per case.
- Round 1 VASARI Feature Scores 1 read per case consolidated by majority vote.
- Round 1 TCIA Shared Lists (see Section 3.7 of TCIA User Guide for help with Shared Lists)
- TCGA-GBM Round 1: Consists of all 88 subjects imaging studies who were reviewed for scoring
- VASARI Round 1 Final: Consists of only the 75 subjects whose scores were utilized in the final data sets referenced above
- Vasari MR Feature Guide v1.1: Image based examples of how the VASARI feature set was utilized in Round 1 analysis
- Round 2 VASARI Feature Scores: Multiple reads per case.
- Round 2 VASARI Feature Scores: 1 read per case consolidated by majority vote.
- Round 2 TCIA Shared Lists (see Section 3.7 of TCIA User Guide for help with Shared Lists)
- VASARI Round 2 Disqualified: Consists of the 5 cases reviewed in round 2 that were not utilized
- VASARI Round 2 Final: Consists of only the 36 subjects whose scores were utilized in the final data sets referenced above
- Round 2 Google Form: In Round 2 the readers scored the features for the cases using Google Forms (with exception to measuring the mass). This link allows you to view and test the form. If you would like to re-use this Google Form in your own research we can clone a copy for you upon request to firstname.lastname@example.org.
- Round 2 Clearcanvas AIM Template: This template was used in conjunction with AIM Clearcanvas v3.0.2 to collect the readers' markup.
- Louis, D.N., Ohgaki, H., Wiestler, O.D., Cavenee, W.K., WHO Classification of Tumours of the Central Nervous System.4th ed. 2007, Lyon: Intl. Agency for Research. 309.
- CBTRUS, CBTRUS Statistical Report: Primary Brain and Central Nervous System Tumors in the United States in 2004-2006. Published by the Central Brain Tumor Registry of the United States. 2010: Hinsdale, IL.
- Stupp, R., et al., Radiotherapy plus concomitant and adjuvant temozolomide for glioblastoma. N Engl J Med, 2005.352(10): p. 987-96.
- Bonavia, R., et al., Heterogeneity Maintenance in Glioblastoma: A Social Network. Cancer Research, 2011. 71(12): p. 4055-4060.
- Park, J.K., et al., Scale to Predict Survival After Surgery for Recurrent Glioblastoma Multiforme. Journal of Clinical Oncology, 2010. 28(24): p. 3838-3843.
- Pope, W.B., et al., MR Imaging Correlates of Survival in Patients with High-Grade Gliomas. American Journal of Neuroradiology, 2005. 26(10): p. 2466-2474.
- Pope, W.B., et al., Relationship between Gene Expression and Enhancement in Glioblastoma Multiforme: Exploratory DNA Microarray Analysis, Radiology. 2008 October; 249(1): 268--277.
- Lacroix, M., et al., A multivariate analysis of 416 patients with glioblastoma multiforme: prognosis, extent of resection, and survival. Journal of Neurosurgery, 2001. 95(2): p. 190-198.
- Hammoud, M.A., et al., Prognostic significance of preoperative MRI scans in glioblastoma multiforme. Journal of Neuro-Oncology, 1996. 27(1): p. 65-73.
- Verhaak, R.G., et al., Integrated genomic analysis identifies clinically relevant subtypes of glioblastoma characterized by abnormalities in PDGFRA, IDH1, EGFR, and NF1. Cancer Cell, 2010. 17(1): p. 98-110.
- Phillips, H.S., et al., Molecular subclasses of high-grade glioma predict prognosis, delineate a pattern of disease progression, and resemble stages in neurogenesis. Cancer Cell, 2006. 9(3): p. 157-73.
- New, A.S., et al., Laboratory induced aggression: A PET study of borderline personality disorder. Biological Psychiatry, 2007. 61(8): p. 14s-14s.
- Channin, D.S., et al., The Annotation and Image Mark-up project. Radiology, 2009. 253(3): p. 590-2.
- Channin, D.S., et al., The caBIG annotation and image Markup project. J Digit Imaging, 2010. 23(2): p. 217-25.
- Rubin, D.L., P. Mongkolwat, and D.S. Channin, A semantic image annotation model to enable integrative translational research. Summit on Translat Bioinforma, 2009. 2009: p. 106-10.
- Showalter, T.N., et al., Multifocal glioblastoma multiforme: prognostic factors and patterns of progression. Int J Radiat Oncol Biol Phys, 2007. 69(3): p. 820-4.
- Scott, C.B., et al., Validation and predictive power of Radiation Therapy Oncology Group (RTOG) recursive partitioning analysis classes for malignant glioma patients: a report using RTOG 90-06. Int J Radiat Oncol Biol Phys, 1998. 40(1): p. 51-5.
- Curran, W.J., Jr., et al., Recursive partitioning analysis of prognostic factors in three Radiation Therapy Oncology Group malignant glioma trials. J Natl Cancer Inst, 1993. 85(9): p. 704-10.
- Krippendorff, K. Computing Krippendorf's Alpha-Reliability. 2011 (cited 2011 12/12/2011); Available from:http://repository.upenn.edu/asc_papers/43/.
- Burnham, K.P. and D.R. Anderson, Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach 2nd ed. 2002: Springer-Verlag.