Error Message "pandas.errors.ParserError: Error tokenizing data. C error: Expected .* fields" Displayed in Logs

Symptom

When pandas is used to read CSV data, the following error is displayed in logs, and the training job failed:

pandas.errors.ParserError: Error tokenizing data. C error: Expected 4 field

Possible Causes

The number of columns in each row of the CSV file is different.

Solution

Use either of the following methods to resolve this issue:

Check the CSV file and delete the lines with extra columns.
Run the following commands to ignore the lines with extra columns:
```
import pandas as pd
pd.read_csv(filePath,error_bad_lines=False)
```

Summary and Suggestions

Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.

Parent topic: Service Code Issues