cc_staff
4
edits
Job arrays: Difference between revisions
Jump to navigation
Jump to search
Marked this version for translation
(→Example: Data-parallel Python script: Fix some typos) |
(Marked this version for translation) |
||
| Line 86: | Line 86: | ||
== Example: Data-parallel Python script == | == Example: Data-parallel Python script == <!--T:19--> | ||
<!--T:20--> | |||
Suppose you have a Python script doing certain calculations with some parameters defined in a Python list or a NumPy array such as | Suppose you have a Python script doing certain calculations with some parameters defined in a Python list or a NumPy array such as | ||
{{File | {{File | ||
| Line 96: | Line 97: | ||
import numpy as np | import numpy as np | ||
<!--T:21--> | |||
def calculation(x, beta): | def calculation(x, beta): | ||
time.sleep(2) #simulate a long run | time.sleep(2) #simulate a long run | ||
return beta * np.linalg.norm(x**2) | return beta * np.linalg.norm(x**2) | ||
<!--T:22--> | |||
if __name__ == "__main__": | if __name__ == "__main__": | ||
x = np.random.rand(100) | x = np.random.rand(100) | ||
| Line 107: | Line 110: | ||
print(res) #show the results on screen | print(res) #show the results on screen | ||
<!--T:23--> | |||
# Run with: python my_script.py | # Run with: python my_script.py | ||
}} | }} | ||
<!--T:24--> | |||
The above task can be processed in a job array so that each value of the beta parameter can be treated in parallel. | The above task can be processed in a job array so that each value of the beta parameter can be treated in parallel. | ||
The idea is to pass the <code>$SLURM_ARRAY_TASK_ID</code> to the Python script and get the beta parameter based on its value. | The idea is to pass the <code>$SLURM_ARRAY_TASK_ID</code> to the Python script and get the beta parameter based on its value. | ||
| Line 121: | Line 126: | ||
import sys | import sys | ||
<!--T:25--> | |||
def calculation(x, beta): | def calculation(x, beta): | ||
time.sleep(2) #simulate a long run | time.sleep(2) #simulate a long run | ||
return beta * np.linalg.norm(x**2) | return beta * np.linalg.norm(x**2) | ||
<!--T:26--> | |||
if __name__ == "__main__": | if __name__ == "__main__": | ||
x = np.random.rand(100) | x = np.random.rand(100) | ||
| Line 133: | Line 140: | ||
print(res) #show the results on screen | print(res) #show the results on screen | ||
<!--T:27--> | |||
# Run with: python my_script_parallel.py $SLURM_ARRAY_TASK_ID | # Run with: python my_script_parallel.py $SLURM_ARRAY_TASK_ID | ||
}} | }} | ||