Hadoop: code sent from master to slave
I started the hadoop cluster and submitted the job to the master. The jar file is only contained in the main file. Does hadop send jar to all slave machines at the start of work? Is there a chance the slave machine will run with the previous version of the code posted during the last run?
Thanks, Bala
+2
a source to share
1 answer
From the mapreduce tutorial:
The framework will copy the required files to the slave node before tasks for the job are executed on that node. Its effectiveness stems from the fact that files are only copied once per job and the ability to cache archives that are not archived to slaves.
More details here:
http://hadoop.apache.org/common/docs/current/mapred_tutorial.html
+2
a source to share