SparkSession.
read
Returns a DataFrameReader that can be used to read data in as a DataFrame.
DataFrameReader
DataFrame
New in version 2.0.0.
Changed in version 3.4.0: Supports Spark Connect.
Examples
>>> spark.read <...DataFrameReader object ...>
Write a DataFrame into a JSON file and read it back.
>>> import tempfile >>> with tempfile.TemporaryDirectory() as d: ... # Write a DataFrame into a JSON file ... spark.createDataFrame( ... [{"age": 100, "name": "Hyukjin Kwon"}] ... ).write.mode("overwrite").format("json").save(d) ... ... # Read the JSON file as a DataFrame. ... spark.read.format('json').load(d).show() +---+------------+ |age| name| +---+------------+ |100|Hyukjin Kwon| +---+------------+