matlab parallel processing on several nodes -
i have studied pages , discussion on matlab processing, still don't know how distribute program on several nodes(not cores). in cluster using, there 10 nodes available, , inside each node there 8 cores available. when using "parfor" inside each node (locally between 8 cores), parallel-ization works fine. when using several nodes, think (not sure how verify this) doesn't work well. here piece of program run on cluster:
function testpool2() disp('this comment') disp(['matlab number of cores : ' num2str(feature('numcores'))]) matlabpool('open',5); disp('this comment!!') tic; i=1:10000 b = rand(1,1000); end; toc tic; parfor i=1:10000 b = rand(1,1000); end; toc end
and outputs :
this comment matlab number of cores : 8 starting matlabpool using 'local' profile ... connected 5 labs. comment!! elapsed time 0.165569 seconds. elapsed time 0.649951 seconds. {warning: objects of distcomp.abstractstorage class exist - not clearing class or of super-classes} {warning: objects of distcomp.filestorage class exist - not clearing class or of super-classes} {warning: objects of distcomp.serializer class exist - not clearing class or of super-classes} {warning: objects of distcomp.fileserializer class exist - not clearing class or of
super-classes}
the program first compiled using "mcc -o out testpool2.m" , transferred scratch drive of server. submit job using microsoft hpc pack 2008 r2. note don't have access graphical interface of matlab installed on each of nodes. can submit jobs using msr hpc job manager (see this: http://blogs.technet.com/b/hpc_and_azure_observations_and_hints/archive/2011/12/12/running-matlab-in-parallel-on-a-windows-cluster-using-compiled-matlab-code-and-the-matlab-compiler-runtime-mcr.aspx )
based on above output can see that, number of available cores 8; infer "matlabpool" works local cores in machine; not between nodes (separate computers connected each other)
so, ideas how can generalize loop ("parfor") nodes ?
ps. have no idea warnings @ end of output !
in order run matlab on multiple nodes, distributed computing server needed in addition parallel computing toolbox. distributing computing server must installed , correctly configured on of nodes in cluster. matlab distributed server comes shell scripts launching parallel matlab jobs on, multiple nodes based on scheduler , cluster setup.
without access distributed computing server, matlab can run on single node. valuable verify cluster administrator distributed computing server setup , running correctly; in cases administrators of these servers have example scripts launching , running jobs common user base, e.g. matlab
here link documentation on distributed computing server: http://www.mathworks.com/help/mdce/index.html?searchhighlight=distributed%20computing%20server
Comments
Post a Comment