com.baidubce.services.bmr.model

Class SparkStepConfig

java.lang.Object

com.baidubce.services.bmr.model.StepConfig

com.baidubce.services.bmr.model.SparkStepConfig

public class SparkStepConfig
extends StepConfig

Represent configuration for a spark step.

A spark step can be configured with name, actionOnFailure, jar, submitOptions and arguments. The essential options are mapper, input, output and actionOnFailure, and the optional ones are name and arguments.

Constructor Summary

Constructors

Constructor and Description
SparkStepConfig()

Method Summary

Modifier and Type	Method and Description
SparkStepConfig	withActionOnFailure(String actionOnFailure) Configure the action on failure for the java step.
SparkStepConfig	withAdditionalFile(String remote, String local) Configure the additional file for the step.
SparkStepConfig	withArguments(String arguments) Configure the arguments for the step.
SparkStepConfig	withJar(String jar) Configure the BOS path for step's .jar file.
SparkStepConfig	withName(String name) Configure the name of the step.
SparkStepConfig	withSubmitOptions(String submitOptions) Configure the submit options for the step.

Methods inherited from class com.baidubce.services.bmr.model.StepConfig

addAdditionalFile, addProperty, getActionOnFailure, getAdditionalFiles, getName, getProperties, getType, setActionOnFailure, setAdditionalFiles, setName, setProperties, setType, withAdditionalFiles, withProperties, withType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkStepConfig

public SparkStepConfig()

Method Detail

withJar

public SparkStepConfig withJar(String jar)

Configure the BOS path for step's .jar file.

Parameters:

jar - The BOS path for the step's .jar file.

Returns:

SparkStepConfig

withSubmitOptions

public SparkStepConfig withSubmitOptions(String submitOptions)

Configure the submit options for the step.

Parameters:

submitOptions - The main class for the step.

Returns:

SparkStepConfig

withArguments

public SparkStepConfig withArguments(String arguments)

Configure the arguments for the step.

Parameters:

arguments - The arguments for the step.

Returns:

SparkStepConfig

withActionOnFailure

public SparkStepConfig withActionOnFailure(String actionOnFailure)

Configure the action on failure for the java step. This property is set to enum value: "Continue": continue to execute other steps. "TerminateCluster": terminate the cluster when this step fails. "CancelAndWait": cancel the other pending steps and set the cluster's status to WAITING.

Overrides:

withActionOnFailure in class StepConfig

Parameters:

actionOnFailure - The action on step's failure.

Returns:

SparkStepConfig

withName

public SparkStepConfig withName(String name)

Configure the name of the step.

Overrides:

withName in class StepConfig

Parameters:

name - The name of the step.

Returns:

SparkStepConfig

withAdditionalFile

public SparkStepConfig withAdditionalFile(String remote,
                                          String local)

Configure the additional file for the step.

Parameters:

remote - The remote file of the additional file.

local - The local file of the additional file.

Returns:

SparkStepConfig