Why is a Spark DataFrame usually preferred over an RDD?
- A The Catalyst optimiser can optimise DataFrame operations because it understands the schema
- B RDDs cannot be partitioned
- C DataFrames use less storage
- D RDDs are deprecated
Answer
The Catalyst optimiser can optimise DataFrame operations because it understands the schema
With RDDs, Spark sees opaque functions and cannot reorder or push down operations the way it can with structured queries.





