We developed a multi-modality DICOM image dataset that can be used to evaluate the performance of automated de-identification pipelines and protocols. Previously de-identified radiology cases (426) were selected from the Cancer Imaging Archive (TCIA) to use as a validation dataset. The set includes CT, MRI, PET, and radiograph images of most body parts and from various imaging system vendors.  The DICOM Standard Security and System Profile was used to create the validation image dataset along with audit logs from TCIA curation of the images. Synthetic PHI/PII and standardized patient IDs were added to DICOM tags in the validation image dataset to mimic non-de-identified images. The validation test dataset and associated de-identified test dataset for 5% of 426 subjects are being released with this publication. This paper describes the validation image dataset creation process, location of associated tables and datasets, and guides for using the dataset. We believe this is the first multi-modality image validation dataset available to the public for use in testing automated image de-identification algorithms.