How Do I Set the AK/SK for a Queue to Operate an OBS Table?

Setting Up a Spark Jar Job to Obtain the AK/SK

• To obtain the AK/SK, set the parameters as follows:
  • Create a SparkContext using code.

    val sc: SparkContext = new SparkContext()
    sc.hadoopConfiguration.set("fs.obs.access.key", ak)
    sc.hadoopConfiguration.set("fs.obs.secret.key", sk)

  • Create a SparkSession using code.

    val sparkSession: SparkSession = SparkSession
          .builder()
          .config("spark.hadoop.fs.obs.access.key", ak)
          .config("spark.hadoop.fs.obs.secret.key", sk)
          .enableHiveSupport()
          .getOrCreate()

• To obtain the AK/SK and security token and use them together for authentication, set the parameters as follows:
  • Create a SparkContext using code.

    val sc: SparkContext = new SparkContext()
    sc.hadoopConfiguration.set("fs.obs.access.key", ak)
    sc.hadoopConfiguration.set("fs.obs.secret.key", sk)
    sc.hadoopConfiguration.set("fs.obs.session.token", sts)

  • Create a SparkSession using code.

    val sparkSession: SparkSession = SparkSession
          .builder()
          .config("spark.hadoop.fs.obs.access.key", ak)
          .config("spark.hadoop.fs.obs.secret.key", sk)
          .config("spark.hadoop.fs.obs.session.token", sts)
          .enableHiveSupport()
          .getOrCreate()