pyspark.Broadcast.dump

Broadcast.dump(value: T, f: BinaryIO) → None

Write a pickled representation of value to the open file or socket. The protocol pickle is HIGHEST_PROTOCOL.

Parameters

valueT

Value to write.

fBinaryIO

File or socket where the pickled value will be stored.

Examples

>>> import os
>>> import tempfile

>>> b = spark.sparkContext.broadcast([1, 2, 3, 4, 5])

Write a pickled representation of b to the open temp file.

>>> with tempfile.TemporaryDirectory() as d:
...     path = os.path.join(d, "test.txt")
...     with open(path, "wb") as f:
...         b.dump(b.value, f)

pyspark.Broadcast.destroy pyspark.Broadcast.load