pyspark.sql.functions.element_at

pyspark.sql.functions.element_at(col: ColumnOrName, extraction: Any) → pyspark.sql.column.Column

Collection function: Returns element of array at given index in extraction if col is array. Returns value for the given key in extraction if col is map. If position is negative then location of the element will start from end, if number is outside the array boundaries then None will be returned.

New in version 2.4.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colColumn or str

name of column containing array or map

extraction :

index to check for in array or key to check for in map

Returns

Column

value at given position.

See also

get()

Notes

The position is not zero based, but 1 based index.

Examples

>>> df = spark.createDataFrame([(["a", "b", "c"],)], ['data'])
>>> df.select(element_at(df.data, 1)).collect()
[Row(element_at(data, 1)='a')]
>>> df.select(element_at(df.data, -1)).collect()
[Row(element_at(data, -1)='c')]

>>> df = spark.createDataFrame([({"a": 1.0, "b": 2.0},)], ['data'])
>>> df.select(element_at(df.data, lit("a"))).collect()
[Row(element_at(data, a)=1.0)]

pyspark.sql.functions.array_position pyspark.sql.functions.array_append