How to submit a job to a specific node in PBS

Question

How do I send a job to a specific node in PBS/TORQUE? I think you must specify the node name after nodes.

#PBS -l nodes=abc

However, this doesn't seem to work and I'm not sure why. This question was asked here on PBS and specify nodes to use

Here is my sample code

#!/bin/bash
#PBS nodes=node9,ppn=1,
hostname
date 
echo "This is a script"
sleep 20    # run for a while so I can look at the details
date

Also, how do I check which node the job is running on? I saw somewhere that $PBS_NODEFILE shows the details, but it doesn't seem to work for me.

Solution 1

You can do it like this:

#PBS -l nodes=<node_name>

You can also specify the number of processors:

#PBS -l nodes=<node_name>:ppn=X

Or you can request additional nodes, specified or unspecified:

#PBS -l nodes=<node_name1>[:ppn=X][+<node_name2...]

That gives you multiple specific nodes.

#PBS -l nodes=<node_name>[:ppn=X][+Y[:ppn=Z]]

This requests the specific node with X execution slots from that node, plus an additional Y nodes with Z execution slots each.

Edit: To simply request a number of nodes and execution slots per node:

PBS -l nodes=X:ppn=Y

NOTE: this is all for TORQUE/Moab. It may or may not work for other PBS resource managers/schedulers.

Solution 2

The above answer doesn't work for PBS Pro. The following works for including a list of nodes (node1 and node2).

#PBS -l select=1:host=node1+1:host=node2

For also including the number of processors,

#PBS -l select=1:ncpus=24:host=node1+1:ncpus=24:host=node2

Comments

[iamauser]

Have you tried removing the #!/bin/bash. I am wondering if bash interprets #PBS as a comment...

[Ashwin]

I didnt try that. so what do you think I should write in the first line?

[Ashwin]

Also is there a command to know on which node a job is running, I mean to be displayed by echo. I know from the terminal I can do that using qstat -n

[Pixelchemist]

qstat -n1 gives you the information on which node your jobs are running.

[Keith Thompson]

@iamauser: Yes, bash interprets #PBS as a comment, but qsub recognizes it as a directive. That's exactly why that syntax was chosen. Removing the #!/bin/bash won't help.

[dtlussier]

This answer is not applicable to PBS Pro > v10 or v11. PBS Pro has changed the syntax for this type of selection. See Ch 5 of the User Guide for info. I'm only getting familiar with it myself so don't want to try to do it justice here.

[dbeer]

True, this answer is meant specifically for TORQUE.

[becko]

How can I know what PBS resource manager my cluster has? (Without asking the admin, is there a command at the terminal that I can use?)

[dbeer]

qstat --version might help. PBSPro versions are much higher than Torque versions.

[Nike]

Do you know how to submit to a specific node in loadleveler?

[soungalo]

I am getting "Job violates queue and/or server resource limits" on PBS Pro 14.0.1. Any idea what this means?

[rashid]

@soungalo It usually means the queue in which you are running the job doesn't allow you to submit jobs with the resources you have specified. You can get the various queues available and corresponding resource limits of the cluster you use by typing qstat -Qf in terminal. And you can specify the queue in which a job should run, by adding a line #PBS -q queuename in your jobscript.