pyspark.RDD.coalesce

RDD.coalesce(numPartitions, shuffle=False)

Return a new RDD that is reduced into numPartitions partitions.

Examples

>>> sc.parallelize([1, 2, 3, 4, 5], 3).glom().collect()
[[1], [2, 3], [4, 5]]
>>> sc.parallelize([1, 2, 3, 4, 5], 3).coalesce(1).glom().collect()
[[1, 2, 3, 4, 5]]

pyspark.RDD.checkpoint pyspark.RDD.cogroup