What is a shuffle in a distributed processing engine?
- A Redistributing data across partitions, usually involving network transfer and disk writes
- B Randomising record order
- C Compressing data
- D Replicating blocks
Answer
Redistributing data across partitions, usually involving network transfer and disk writes
Wide transformations such as groupBy and join trigger shuffles, and minimising them is the main performance lever in Spark tuning.





