vpruthi/doc-exports

forked from docs/doc-exports

Lai, Weijian 4e4b2d5f6d ModelArts UMN 23.3.0 Version.

Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com>
Co-authored-by: Lai, Weijian <laiweijian4@huawei.com>
Co-committed-by: Lai, Weijian <laiweijian4@huawei.com>

2024-06-26 07:03:02 +00:00

2.5 KiB

Raw Blame History

Error Message "pandas.errors.ParserError: Error tokenizing data. C error: Expected .* fields" Displayed in Logs

Symptom

When pandas is used to read CSV data, the following error is displayed in logs, and the training job failed:

pandas.errors.ParserError: Error tokenizing data. C error: Expected 4 field

Possible Causes

The number of columns in each row of the CSV file is different.

Solution

Use either of the following methods to resolve this issue:

Check the CSV file and delete the lines with extra columns.
Run the following commands to ignore the lines with extra columns:
```
import pandas as pd
pd.read_csv(filePath,error_bad_lines=False)
```

Summary and Suggestions

Before creating a training job, use the ModelArts development environment to debug the training code to maximally eliminate errors in code migration.

Parent topic: Service Code Issues