Question: Explain the concept of 'checkpointing' in PySpark.Answer: 'Checkpointing' is a mechanism in PySpark to truncate the lineage of a RDD or DataFrame by saving it to a reliable distributed file system.Example:
|
Is it helpful?
Yes
No
Most helpful rated by users: