Data tables can be used to define basic attributes and parameters of source tables, dimension tables, and output tables.
Parameter |
Description |
Remarks |
---|---|---|
Stream/Table Name |
Stream/Table name, which can contain 1 to 64 characters. Only letters, digits, and underscores (_) are allowed. |
Example: flink_sink |
Description |
Stream/Table description information, which can contain 1 to 1024 characters. |
- |
Mapping Table Type |
Flink SQL does not provide the data storage function. Table creation is actually the creation of mapping for external data tables or storage. The value can be Kafka, HDFS. |
- |
Type |
Includes the data source table Source, data result table Sink. Tables included in different mapping table types are as follows:
|
- |
Data Connection |
Name of the data connection. |
- |
Topic |
Kafka topic to be read. Multiple Kafka topics can be read. Use separators to separate topics. This parameter is available when Mapping Table Type is set to Kafka. |
- |
File Path |
HDFS directory or a single file path to be transferred. This parameter is available when Mapping Table Type is set to HDFS. |
Example: /user/sqoop/ or /user/sqoop/example.csv |
Code |
Codes corresponding to different mapping table types are as follows:
|
- |
Prefix |
When Mapping Table Type is set to Kafka, Type is set to Source, and Code is set to JSON, this parameter indicates the hierarchical prefixes of multi-layer nested JSON, which are separated by commas (,). |
For example, data,info indicates that the content under data and info in the nested JSON file is used as the data input in JSON format. |
Separator |
Meanings of this parameter corresponding to different mapping table types are as follows: This parameter is used to specify the separator between CSV fields. This parameter is available when Code is set to CSV. |
Example: comma (,) |
Row Separator |
Line break in the file, including \r, \n, and \r\n. This parameter is available when Mapping Table Type is set to HDFS. |
- |
Column Separator |
Field separator in the file. This parameter is available when Mapping Table Type is set to HDFS. |
Example: comma (,) |
Stream Table Structure |
Stream/Table structure, including Name and Type. |
- |
Proctime |
System time, which is irrelevant to the data timestamp. That is, the time when the calculation is complete in Flink operators. This parameter is available when Type is set to Source. |
- |
Event Time |
Time when an event is generated, that is, the timestamp generated during data generation. This parameter is available when Type is set to Source. |
- |