This user's guide provides the basic guidance for Image Provider sites to submit new collections to The Cancer Imaging Archive (TCIA). It summarizes the overall workflow for the entire process and covers in detail the installation and use of the Clinical Trials Processor (CTP) Client software. This is the primary tool used to de-identify and upload images into The Cancer Imaging Archive.
Use this guide in conjunction with The Cancer Imaging Archive User's Guide. A collections facilitator at the University of Arkansas for Medical Sciences (UAMS) will be assigned to assist the staff at the Image Provider site in installing the necessary software and dealing with technical issues. Questions about the use of this guide should be sent to The Cancer Imaging Archive User Help Desk at email@example.com.
As a provider of image data to The Cancer Imaging Archive, you can expect a collections facilitator from The Cancer Imaging Archive (TCIA) team to provide all the required tooling and answer your questions throughout the process.
You will be asked to:
You will receive:
The Cancer Imaging Archive team will work with you every step of the way to make sure the upload process goes smoothly and your valuable contribution is accurately and completely incorporated into The Cancer Imaging Archive.
The Cancer Imaging Archive is intended to be a resource to the wider imaging research community. Proposals for additional data submissions are welcomed and encouraged. The CIP informatics team will be glad to work with you to evaluate the suitability of your data to this archive.
If you feel that you have data at your institution that meets one or more of these objectives, please fill out our application form so that we can evaluate the suitability of your data to the archive. Questions about filling out the application may be directed to the help desk.
Our goal in providing this service is to ensure that every image collection in the archive is one that would be of value to one of our target audiences. These include researchers with the following objectives:
To determine if your image collection meets these requirements or to discussion potential submissions, contact the TCIA Help Desk.
In order to properly de-identify your collection during the submission process it is essential that The Cancer Imaging Archive team completely understands the unique characteristics of your data. Since all images uploaded to The Cancer Imaging Archive use the DICOM standard defined image formats, it is essential to understand any vendor-specific private data elements that may contain Protected Health Information (PHI). The Private Tag Research Procedure and TagSniffer software package have been created to support a complete analysis of your unique image collection.
The Clinical Trials Processor (CTP) is the primary means for uploading data into The Cancer Imaging Archive. CTP is a client-server software package with the server installed on a special intake system at UAMS. You will be given a pre-configured CTP client, containing de-identification scripts customized for your site and collection. You will receive instructions on the proper installation and usage of CTP, including those below as well as additional instructions provided by the collections facilitator.
The CTP software is written in the Java programming language so that it can be supported on all common computing platforms and operating systems.
Before installing CTP Client, the Java 1.6.0_24 (or better) Java Runtime Environment (JRE) must be present on the system. Do the following to check for the current version on your PC.
Only the JRE is required, not the JDK.
If the response indicates JRE is not present or the version is older than 1.6.0_24, install Java or a new version of Java.
Certain CTP pipeline stages (FileStorageService, BasicFileStorageService) require that the Java Advanced Imaging ImageIO Tools version 1.1 be present on the system. These are only necessary if you are retaining a copy of the images on your local computer. The assigned collections facilitator for your site will inform you if Image IO Tools are required. Do the following to check for the current version installed on your PC.
If your installed version of JAI Tools is 1.0, install the latest version of JAI Tools.
Certain CTP pipeline stages (FileStorageService, BasicFileStorageService) require that the Java Advanced Imaging ImageIO Tools be present on the system. It is critically important that version 1.1 of the ImageIO Tools be installed rather than version 1.0. Parenthetically, note that the Java Advanced Imaging component is not the same as the Java Advanced Imaging ImageIO Tools. Only the latter component is required. The ImageIO Tools need not be present in order to run the installer, and they may be installed later if required, without having to re-install CTP. From the download page linked to above, select the appropriate JRE version for your operating system and download to your hard drive. Install and reboot if requested.
The collections specialist will provide each Image Provider site with a compressed file containing the CTP program files and configuration files customized for each site and collection. This file should be copied to a directory on the computer that will run the CTP program. Follow these steps to install the CTP Client software.
Follow these steps to start CTP Client:
FileSender is a utility program for sending files to CTP. It is installed as part of the CTP installation process. It is used when image files are in multiple directories. Follow these steps to use FileSender:
The left pane in FileSender is used to navigate to a file or a folder of interest. The pull-down menu in the header of the left pane lets you choose the root (drive) file system. The button in the footer bar lets you specify the extensions to accept. Enter an asterisk if you wish to accept all files, regardless of their extensions.
Check (by clicking on it) the "Include subdirectories" option, if you want FileSender to drill down into all the subdirectories and send all files.
The UAMS collections facilitator will work with you to facilitate a limited test-set submission of your collection once the CTP client has been installed. This test will consist of one series of images using either a DICOM 'send' via your PACS or using FileSender to transmit the data from a separate storage system on which you have pre-loaded your data to be submitted. Once the limited test-set has been uploaded you will be asked to send an email to UAMS with the image count. The collections facilitator will verify successful submission of data and provide assistance if there were any errors. Optionally this test will be performed interactively (via web conferencing) by a UAMS collections facilitator.
Follow the image transmission process after selecting the folder/image in FileSender (step #5). Image provider sites should monitor progress of the transmissions by checking the CTP Status Page during the transmission.
The files processed count should increase during the transmission. Refresh the browser to check on progress.
After the transmission is complete several pages should be checked to ensure the transmission was completed successfully. For problems, please consult Troubleshooting and Error Reporting.
This is critical! The submission process cannot be completed without this step. At the end of a successful transmission, please email your collections facilitator the image count from the status page; a screenshot of the database verifier results is most welcome! Also in the email you are encouraged to include comments, observations, suggestions, or criticisms. These will help us make this process and documentation better.
FileSender is the most versatile method of file transmission and will be used in most cases. CTP also provides three additional methods of image transfer. Your submissions facilitator will determine the most appropriate method based on information you provide during initial telephone discussions. Please refer to the following information for other transmission methods.
If the image files are in a single location the 'treeRoot' method of image transmission can be used with no additional steps required. The disadvantage of this method is not being able to select only certain images for transmission.
CTP Client is configured to receive images directly from a PACS. Your PACS administrator will need to configure the PACS to send to the ipaddress of the computer that is running CTP. CTP is configured to listen on port 11112 and will accept all AET connections. Before pushing images from the PACS, start the CTP Client and monitor the submission.
Image Providers are strongly encouraged to preserve the ID Mapping table for future reference if necessary.
When something goes awry, the reason(s) may not be readily apparent. The sections that follow describe procedures for resolving more common issues. If the procedures do not apply to your issue or apply but do not work, please consult your collection facilitator.
The cause of unverified images may be hard to discern. At times, it may be that transmissions and client-server handshakes are still in progress and the unverified counts are merely temporary. If it has only been a few minutes since the last images were transmitted, you should continue to check the unverified counts until they reach zero or stabilize. If the count stabilizes greater than zero, you should check the export queue in the CTP-Client User Interface (UI) to make sure all images have been sent (count = 0). If greater than zero and stable, you should shutdown the CTP-Client and restart it to see if this gets transmissions moving again. If the export queue is zero, but the unverified number is non-zero, then the images either got lost in transmission, or are stuck in a queue or quarantine on the server side. You will need to dialog with your submissions specialist. At other times, unverified images may be related to the submission of duplicate images, for which there may be clues in quarantines. You should provide the collections facilitator with whatever information is known about the unverified images so UAMS can examine server-side logs and quarantines in order to advise submitter how to proceed. One solution may require the submitter to re-transmit images.
There are two situations in which images might need to be retransmitted. There may have been serious problems with your transmission that require starting over from scratch, or alternatively you may need to retransmit a subset of images to fix a smaller problem.
First your collections facilitator will completely delete your images from TCIA. Then they will request you to perform these steps before you retransmit your images:
If you are only fixing a subset of images you can simply re-send the images. This could include:
Transmission errors that can't be resolved by retransmitting the files should be reported to your assigned collections facilitator at UAMS. In an emergency please contact The Cancer Imaging Archive User Support Center @ +1 385-275-8242 (ASK-TCIA).
Generally, this will not be required. If your assigned collections facilitator deems it necessary, you will be asked to send the appropriate log files for analysis.
You may encounter an error message suggesting an issue with the HTTP Export process. The message appears as the following:
12:04:49 WARN HttpExportService CT XML HTTP Export: export failed: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target.
Your first response should be to restart the CTP client (refer to Starting CTP Client). If that fails, reboot your computer and then restart the CTP client.
The CTP program uses a configuration file - config.xml. This file is configured for each individual Image Provider site and is automatically installed during the CTP installation process. Should changes be required after the installation of CTP the UAMS collections specialist will provide a revised file. You will need to replace the config.xml file created during the install by following these steps.
The CTP program requires a customized anonymizer file for each collection. This file is configured for each individual Image Provider site and is automatically installed during the CTP installation process. Should changes be required after the installation of CTP the assigned collections facilitator will provide a revised file. You will need to replace the "project anonymizer".script file created during the install by following these steps.