Translations:META-Farm/142/en: Difference between revisions

From Alliance Doc
Jump to navigation Jump to search
(Importing a new version from external source)
 
m (FuzzyBot a déplacé la page Translations:META: A package for job farming/142/en vers Translations:META-Farm/142/en sans laisser de redirection: Part of translatable page "META: A package for job farming")
(No difference)

Revision as of 15:06, 21 March 2023

Information about message (contribute)
This message has no documentation. If you know where or how this message is used, you can help other translators by adding documentation to this message.
Message definition (META-Farm)
The META package has important advantages over other approaches like [[GLOST]] where farm processing is done by bundling up all the jobs into a large parallel (MPI) job:
* As the scheduler has full flexibility to start individual metajobs when it wants, the queue wait time can be dramatically shorter with the META package than with GLOST. Consider a large farm where 1000 CPU cores need to be used for 3 days;
** with GLOST, with a 1000-way MPI job, queue wait time can be weeks, so it'll be weeks before you see your very first result;
** with META, some metajobs start to run and produce the first results within minutes.
* At the end of the farm computations;
** with GLOST, some MPI ranks will finish earlier and will sit idle until the very last—the slowest—MPI rank ends;
** with META, there is no such waste at the end of the farm: individual metajobs exit earlier if they have no more workload to process. 
* GLOST and other similar packages do not support automated resubmission of the cases which failed or never ran. META has this feature, and it is very easy to use.

There are three important advantages of the META package over other approaches like GLOST where farm processing is done by bundling up all the jobs into a large parallel (MPI) job:

  1. As the scheduler has full flexibility to start individual meta-jobs when it wants, the queue wait time can be dramatically shorter with the META package than with GLOST. Consider a large farm where 1000 CPU cores need to be used for 3 days. With META, some meta-jobs start to run and produce the first results within minutes. With GLOST, with a 1000-way MPI job, queue wait time can be weeks, so it'll be weeks before you see your very first result.
  2. With GLOST, at the end of the farm computations, some MPI ranks will finish earlier and will sit idle until the very last -- the slowest -- MPI rank ends. In META package there is no such waste at the end of the farm -- individual meta-jobs exit earlier if they have no more workload to process.
  3. GLOST and other similar packages do not support automated resubmission of the cases which failed or never ran. META has this feature, and it is very easy to use.

Message Passing Interface

Retrieved from "https://docs.alliancecan.ca/mediawiki/index.php?title=Translations:META-Farm/142/en&oldid=132180"