MLWriter¶

class pyspark.ml.util.MLWriter[source]¶

Utility class that can save ML instances.

New in version 2.0.0.

Methods

`option`(key, value)	Adds an option to the underlying MLWriter.
`overwrite`()	Overwrites if the output path already exists.
`save`(path)	Save the ML instance to the input path.
`saveImpl`(path)	save() handles overwriting and then calls this method.
`session`(sparkSession)	Sets the Spark Session to use for saving/loading.

Attributes

`sc`	Returns the underlying SparkContext.
`sparkSession`	Returns the user-specified Spark Session or the default.

Methods Documentation

option(key: str, value: Any) → pyspark.ml.util.MLWriter [source]¶: Adds an option to the underlying MLWriter. See the documentation for the specific model’s writer for possible options. The option name (key) is case-insensitive.

overwrite() → pyspark.ml.util.MLWriter [source]¶: Overwrites if the output path already exists.

save(path: str) → None[source]¶: Save the ML instance to the input path.

saveImpl(path: str) → None[source]¶: save() handles overwriting and then calls this method. Subclasses should override this method to implement the actual saving of the instance.

session(sparkSession: pyspark.sql.session.SparkSession) → RW¶: Sets the Spark Session to use for saving/loading.

Attributes Documentation

sc¶: Returns the underlying SparkContext.

sparkSession¶: Returns the user-specified Spark Session or the default.

MLWritable Spark Streaming (Legacy)