Translations:META-Farm/4/en
Jump to navigation
Jump to search
META vs. GLOST[edit]
There are three important advantages of the META package over the other approaches (like GLOST) where farm processing is done by bundling up all the jobs into a large parallel (MPI) job:
- As the scheduler has full flexibility to start individual meta-jobs when it wants, the queue wait time can be dramatically shorter with the META package than with GLOST. Consider a large farm where 1000 CPU cores need to be used for 3 days. With META, some meta-jobs start to run and produce the first results within minutes. With GLOST, with a 1000-way MPI job, queue wait time can be weeks, so it'll be weeks before you see your very first result.
- With GLOST, at the end of the farm computations, some MPI ranks will finish earlier and will sit idle until the very last -- the slowest -- MPI rank ends. In META package there is no such waste at the end of the farm -- individual meta-jobs exit earlier if they have no more workload to process.
- GLOST and other similar packages do not support automated resubmission of the cases which failed or never ran. META has this feature, and it is very easy to use.