Error Message "CUDNN_STATUS_NOT_SUPPORTED" Displayed in Logs

Symptom

RuntimeError: cuDNN error: CUDNN_STATUS_NOT_SUPPORTED. This error may appear if you passed in a non-contiguous input.

Possible Causes

The input data is not of contiguous type, which is not supported by cuDNN.

Solution

1. Disable cuDNN before training.

   torch.backends.cudnn.enabled = False

2. Convert the input data into contiguous data.

   images = images.cuda()  
   images = images.permute(0, 3, 1, 2).contigous()

Summary and Suggestions

Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.