pyspark.Broadcast.load_from_path

Broadcast.load_from_path(path: str) → T

Read the pickled representation of an object from the open file and return the reconstituted object hierarchy specified therein.

Parameters

pathstr

File path where reads the pickled value.

Returns

T

The object hierarchy specified therein reconstituted from the pickled representation of an object.

Examples

>>> import os
>>> import tempfile

>>> b = spark.sparkContext.broadcast([1, 2, 3, 4, 5])
>>> c = spark.sparkContext.broadcast(1)

Read the pickled representation of value fron temp file.

>>> with tempfile.TemporaryDirectory() as d:
...     path = os.path.join(d, "test.txt")
...     with open(path, "wb") as f:
...         b.dump(b.value, f)
...     c.load_from_path(path)
[1, 2, 3, 4, 5]

pyspark.Broadcast.load pyspark.Broadcast.unpersist