Snippet:Batch run CPlex models on Hypatia from Matlab

From HUES Platform Wiki
Description This code manages the simulation of multiple CPlex jobs on Hypatia. It could also be adapted to other programs.

The steps are:

  • Log in via SSH
  • Delete old files
  • Copy .mps file locally, rename
  • Copy to Hypatia using PSCP
  • Submit to qsub, passing job number to script to be sued in file names
  • Get array of job numbers
  • Wait until all jobs complete
  • Copy back .sol files
Author(s) Ralph Evins
Language Matlab
Tags Hypatia
Snippet code empty space
remotefolder = '/mnt/scratch/USR/cplex/';
conn = sshfrommatlab('USR','hypatia.empa.emp-eaw.ch','PWD'); % log in
[conn, result] = sshfrommatlabissue(conn,['cd ' remotefolder '; rm output*; rm *.mps; rm *.log']); % move to remotefolder; delete all files output* *.mps *.log
copyfile([projectpath 'cpx00000.mps'],[projectpath 'cpx' num2str(j) '.mps']); % copy cpx00000.mps created by AIMMS ready to move, numbering by ID j
cd(projectpath) % move to project folder
tic
[status,cmdout] = system(['pscp *.mps evr@hypatia:' remotefolder]); % copy to Hypatia using PSCP
[conn, result] = sshfrommatlabissue(conn,['qsub -v myjob=' num2str(j) ' pbs_cplex']); % queue the job, passing j to script pbs_cplex
jobno(j) = str2double(result{1}(1:5)); % get Hypatia queue number
pause(0.5); % 0.5 sec lag for queuing
jobno=sort(jobno);
fprintf(['Running on Hypatia: %g to %g...'],jobno(1),jobno(end)); % report job numbers queued
esc=0;
while esc==0 % check if done
   [conn, result] = sshfrommatlabissue(conn,['cd ' remotefolder '; ls test_cplex.*']); % get list of files
   ended=[]; ended_list=zeros(size(jobno));
   if size(result{1,1},2) > 0
       for i=1:size(result,1) % for each file
            ended(end+1)=str2double(result{i,1}(end-4:end)); % get filename without extension
        end
        ended=unique(ended);
        ended=sort(ended);
        ended_list(1:size(ended,2))=ended; % get unique sorted list of jobs
    end
    if sum(ended_list~=jobno)==0 % if identical to list of jobs
        esc=1;
    else
        pause(1)% wait 1 sec before retrying
    end
end    
[status,cmdout] = system(['pscp evr@hypatia:' remotefolder '*.sol ./']); % copy from hypatia
hypatiatime = toc;
fprintf(' Done (%g secs).\n',hypatiatime); % report number of replacements
Notes Replace USR and PWD with your 3 letter username and password.