EXECUTOR MEMORY: Amount of memory to use per executor process, in the

same format as JVM memory strings (e.g. 512m, 2g).

DRIVER MEMORY – depends on the job what we are doing like

1. If the job is based purely on transformations and terminates on some

distributed output action like [Link], [Link], ...
then the memory needs of the driver will be very low. Few 100's of MB will do.

2. If the job requires the driver to participate in the computation, like e.g. some
ML algo that needs to materialize results and broadcast them on the next
iteration, then your job becomes dependent of the amount of data passing
through the driver. Operations like .collect,.take,.takeSample