The inspection of ship-board structures by humans is a time-consuming and expensive activity, creating the need to search for better solutions. We have realized that as the implementation of segmentation on common problems becomes more realistic, there is an opportunity to apply this to the detection of ship defects. In this study, we investigated how transfer learning and data augmentation interact. work in the segmentation of defects. Semantic segmentation models, specifically U-Net, are trained using five data sets in total. The five datasets used were manually created by combining different percentages of a concrete crack dataset and 73 ship break and rust images. We also studied how the performance of these models changes when applying data augmentation (grayscaling, rotating, as well as random contrast).To evaluate each experiment, we used the IoU, the precision, the recall, and the F1-score. The results of the experiments indicate that without transfer learning, a precision above 50% can be achieved. Recall, however, would be too low for practical use. Using transfer learning, recall can be increased above 90% at the cost of precision. or degrade performance This research shows that a U-Net model with transfer learning can segment damage in ships. It would be of interest to see if few-shot learning can be used to improve the precision of segmentation.