pyspark.sql.functions.to_csv¶

pyspark.sql.functions.to_csv(col: ColumnOrName, options: Optional[Dict[str, str]] = None) → pyspark.sql.column.Column[source]¶

Converts a column containing a StructType into a CSV string. Throws an exception, in the case of an unsupported type.

New in version 3.0.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str: name of column containing a struct.
options: dict, optional: options to control converting. accepts the same options as the CSV datasource. See Data Source Option for the version you use.

Returns

Column: a CSV string converted from given StructType.

Examples

>>> from pyspark.sql import Row
>>> data = [(1, Row(age=2, name='Alice'))]
>>> df = spark.createDataFrame(data, ("key", "value"))
>>> df.select(to_csv(df.value).alias("csv")).collect()
[Row(csv='2,Alice')]

pyspark.sql.functions.schema_of_csv pyspark.sql.functions.years