Expression Overview

Node parameter values in a DataArts Factory job can be dynamically generated based on the running environment by using Expression Language (EL). You can determine whether to execute this node based on the input parameters of the pipeline and the output of the upstream node. EL uses simple arithmetic and logic to calculate and references embedded objects, including job objects and tool objects.

Job object: provides properties and methods of obtaining the output message, job scheduling plan time, and job execution time of the previous node in a job.

Tool job: Provides methods of operating character strings, time, and JSON. For example, truncating a substring from a string or formatting time.

Syntax

Expression syntax:

#{expr}

In the preceding information, expr indicates an expression. # and {} are common operators used in EL, allowing you to access job properties using embedded objects.

Example

In the URL parameter of the Rest Client node, use expression tableName=#{JSONUtil.path(Job.getNodeOutput("get_cluster"),"tables[0].table_name")}.

Expression description:

  1. Job.getNodeOutput("get_cluster") is used to obtain the execution result of the get_cluster node in the job. The execution result is a JSON character string.
  2. tables[0].table_name is used to obtain the value of a field in the JSON character string.

Debugging Methods

You can debug EL expressions using the following methods.

This section uses the #{DateUtil.now()} expression as an example.

  1. Use the DIS Client node.
    • Prerequisites: A DIS stream is available.
    • Method: Select the DIS Client node, write the EL expression in the data to be sent, and click Test. Then right-click the node to view the log. The value of the EL expression is printed in the log.

  2. Use the Kafka Client node.
    • Prerequisites: An MRS cluster with the Kafka component is available.
    • Method: Select the Kafka Client node, write the EL expression in the data to be sent, and click Test. Then right-click the node to view the log. The value of the EL expression is printed in the log.

  3. Use the shell node.
    • Prerequisites: An ECS is available.
    • Method: Create a host connection, print the EL expression using echo, and click Test. Then view the log. The value of the EL expression is printed in the log.

  4. Use the Create OBS node.

    If none of the preceding methods is available, use the Create OBS node and create an OBS path with the value of the EL expression as its name. You can click Test and go to the OBS console to view the name of the created path.

Parent topic: EL Expression Reference