pyspark.sql.functions.input_file_name

pyspark.sql.functions.input_file_name() → pyspark.sql.column.Column

Creates a string column for the file name of the current Spark task.

New in version 1.6.0.

Changed in version 3.4.0: Supports Spark Connect.

Returns

Column

file names.

Examples

>>> import os
>>> path = os.path.abspath(__file__)
>>> df = spark.read.text(path)
>>> df.select(input_file_name()).first()
Row(input_file_name()='file:///...')

pyspark.sql.functions.coalesce pyspark.sql.functions.isnan