pyspark.sql.DataFrameWriter.text

DataFrameWriter.text(path: str, compression: Optional[str] = None, lineSep: Optional[str] = None) → None

Saves the content of the DataFrame in a text file at the specified path. The text files will be encoded as UTF-8.

New in version 1.6.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

pathstr

the path in any Hadoop supported file system

Other Parameters

Extra options

For the extra options, refer to Data Source Option for the version you use.

Notes

The DataFrame must have only one column that is of string type. Each row becomes a new line in the output file.

Examples

Write a DataFrame into a text file and read it back.

>>> import tempfile
>>> with tempfile.TemporaryDirectory() as d:
...     # Write a DataFrame into a text file
...     df = spark.createDataFrame([("a",), ("b",), ("c",)], schema=["alphabets"])
...     df.write.mode("overwrite").text(d)
...
...     # Read the text file as a DataFrame.
...     spark.read.schema(df.schema).format("text").load(d).sort("alphabets").show()
+---------+
|alphabets|
+---------+
|        a|
|        b|
|        c|
+---------+

pyspark.sql.DataFrameWriter.sortBy pyspark.sql.DataFrameWriterV2.using