X-ray scanners are commonly used to help prevent contraband from being smuggled into prisons. Analyzing every image to look for contraband can be a time consuming task. This research explores deep learning networks that could possibly aid security inspectors at a prison with localizing and classifying anomalies. The popular object detection models YOLOv5 and Faster R-CNN are explored in this research to determine which model is most suited for this task. The results show that YOLOv5s with a learning rate of 0.0001 and a random tiling collator yields the best results with a recall score of 0.65 and an F1 score of 0.71. Interestingly, the experiments suggest that the model has a hard time detecting objects in dark areas. An attempt to solve this shows that using a RandomBrightnessDecrease method on lighter images when training the model results in a 0.08 increase in mean average precision.