pyspark.sql.DataFrame.rollup

DataFrame.rollup(*cols: ColumnOrName) → GroupedData

Create a multi-dimensional rollup for the current DataFrame using the specified columns, so we can run aggregation on them.

New in version 1.4.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters

colslist, str or Column

Columns to roll-up by. Each element should be a column name (string) or an expression (Column) or list of them.

Returns

GroupedData

Rolled-up data by given columns.

Examples

>>> df = spark.createDataFrame([(2, "Alice"), (5, "Bob")], schema=["age", "name"])
>>> df.rollup("name", df.age).count().orderBy("name", "age").show()
+-----+----+-----+
| name| age|count|
+-----+----+-----+
| null|null|    2|
|Alice|null|    1|
|Alice|   2|    1|
|  Bob|null|    1|
|  Bob|   5|    1|
+-----+----+-----+

pyspark.sql.DataFrame.replace pyspark.sql.DataFrame.sameSemantics