(此文说明需要在pbs server发出qsub)PBS server host---Compute host---User hosts
PBS/SCore(Open PBS/SCore)Administrator's Guide
Index
- Introduction
- Host Configuration
- Common file setting
- File setup on PBS server host
- File setup on compute hosts
- File setup on user hosts
- File setup on the host to submit SCore jobs
- Initiating PBS servers
- The difference between PBS and PBS/SCore
1. Introduction
PBS/SCore must be configired independently from the SCore configuration. This document describes hot to configure PBS/SCore. For further deials, please refer to the Administrator Guide attached to PBS(/SCore). In this document, it is assumed that SCore is already installed and configured.
If SCore system has been installed by using EIT, all the configuration procedure described in this document has been automatically done by EIT.
2. Host Configuration
There are 4 host types in PBS/SCore. An administrator must decide which host is which type.
- PBS server host
One of the hosts must be a PBS server host on which PBS server (
pbs_server
) and PBS scheduler (pbs_sched
) are running.It is possible that
msgbserv
and/orscoreboard
server programs of SCore can also run on the PBS server host. - Compute host
Parallel jobs run on compute hosts. On each compute host,
pbs_mom
server must be running to monitor host resource and invoke user parallel jobs. To run SCore jobs, compute hosts must be registered in scorehosts.db file, andscoreboard
server is up and running on some host. - User hosts
To submit parallel jobs from user hosts, PBS commands, such as
qsub
andqdel
, must be installed on the user hosts. - Host to invoke SCore jobs
An SCore administrator may specify a host from which SCore jobs are invoked via the
scout
command. This host should not be a compute host. Like a computer host,pbs_mom
must be initiated on this host too. If this host is not allocated, one of compute hosts which are allocated to a submitted job is used.
In this document, it is assumed that above hosts are cofigured like the following.
- PBS server host
server.pccluster.org
- Compute hosts
comp0.pccluster.org
comp1.pccluster.org
comp2.pccluster.org
comp3.pccluster.org
- User hosts
server.pccluster.org
- Host to invoke SCore jobs
server.pccluster.org
3. File setup on all hosts
Every (possible) host must be configured as in the following.
/opt/score/etc/pbs_server_name
Default PBS server host name must be specified in the /opt/score/etc/pbs_server_name
file.
server.pccluster.org
4. File setup on PBS server host
/var/scored/pbs/
directory
Execute the following command to setup the /var/scored/pbs/
. This operation is not needed if SCore system has been installed by using the binary rpms or EIT.
# cd /opt/score/install # ./setup -pbs_server
/var/scored/pbs/server_priv/nodes
All compute hosts must be listed in the /var/scored/pbs/server_priv/nodes
file. If scorehosts.db
file is already created and scoreboard
server is up and running, then the scbd2pbs
command creates the nodes
file via the scoreboard
database.
% /opt/score/sbin/scbd2pbs pcc > /var/scored/pbs/server_priv/nodes
In this example, all hosts belong to pcc host group are listed, and then the nodes
file is created. The follwoing is an example of the created nodes
file.
comp0.pccluster.org pcc score comp1.pccluster.org pcc score comp2.pccluster.org pcc score comp3.pccluster.org pcc score
Each line consists of hostname and property names of the host. As a PBS property name, host group name, "pcc" in this example, is used. In scorehosts.db
file, if a host record has an attribute named "pbs", then its associated value(a) is added to the line as property name(s).
For more detail on the nodes
file, please refer to Section 3.2.2. Declaring Nodes of PBS Administrator Guide
5. File setup on compute hosts
/var/scored/pbs/
directory
Type the following command to setup the /var/scored/pbs/
. This operation is not needed if SCore system has been installed by using the binary rpms or EIT.
# cd /opt/score/install # ./setup -pbs_comp
/var/scored/pbs/mom_priv/config
In the /var/scored/pbs/mom_priv/config
file, PBS server hostname must be included as in the following.
$logevent 0x1ff $clienthost server.pccluster.org
For more detail on the config
file, please refer to Section 9.2. pbs_mom of PBS Administrator Guide
6. File setup on user hosts
There is nothing to do with user hosts.
7. File setup on the host to submit SCore jobs
/var/scored/pbs/
directory
Type the following command to setup the /var/scored/pbs/
. This operation is not needed if SCore system has been installed by using the binary rpms or EIT.
# cd /opt/score/install/ # ./setup -pbs_comp
/var/scored/pbs/mom_priv/config
In the /var/scored/pbs/mom_priv/config
file, PBS server hostname and all compute hostnames must be listed.
$logevent 0x1ff $clienthost server.pccluster.org $clienthost comp0.pccluster.org $clienthost comp1.pccluster.org $clienthost comp2.pccluster.org $clienthost comp3.pccluster.org
For more detail on the config
file, please refer to Section 9.2. pbs_mom of PBS Administrator Guide
8. Initiating PBS Servers
pbs_mom
Initiate the pbs_mom
program on all compute host. If there is a host to submit SCore jobs, then the pbs_mom
program must be initiate on the host too.
% su # /etc/rc.d/init.d/pbs_mom startSuSE Linux:
% su # /etc/init.d/pbs_mom start
pbs_sched
Initiate PBS scheduler on PBS server host.
Red Hat or Turbo Linux:% su # /etc/rc.d/init.d/pbs_sched startSuSE Linux:
% su # /etc/init.d/pbs_sched start
pbs_server
Initiate the pbs_server
program on the server host. If there is a host to submit SCore jobs, add -m
option followed by the name of the host. If this is the first time to initiate PBS server, add -t
option to setup databse.
% su # /opt/score/sbin/pbs_server -t create -m server.pccluster.orgOtherwise,
Red Hat or Turbo Linux:
% su # /etc/rc.d/init.d/pbs_server startSuSE Linux:
% su # /etc/init.d/pbs_server start -m
Confirmation
To confirm if the PBS server processes are running, invoke the following command.
% su # /opt/score/bin/qmgr -c 'p n @active'
If they are running properly, then the following output is obtained.
# # Create nodes and set their properties. # # # Create and define comp0.pccluster.org # # create node comp0.pccluster.org # unsuppored operation set node comp0.pccluster.org state = free set node comp0.pccluster.org properties = any set node comp0.pccluster.org properties += score set node comp0.pccluster.org properties += score-pcc set node comp0.pccluster.org ntype = cluster # # Create and define comp1.pccluster.org # # create node comp1.pccluster.org # unsuppored operation set node comp1.pccluster.org state = free set node comp1.pccluster.org properties = any set node comp1.pccluster.org properties += score set node comp1.pccluster.org properties += score-pcc set node comp1.pccluster.org ntype = cluster # # Create and define comp2.pccluster.org # # create node comp2.pccluster.org # unsuppored operation set node comp2.pccluster.org state = free set node comp2.pccluster.org properties = any set node comp2.pccluster.org properties += score set node comp2.pccluster.org properties += score-pcc set node comp2.pccluster.org ntype = cluster # # Create and define comp3.pccluster.org # # create node comp3.pccluster.org # unsuppored operation set node comp3.pccluster.org state = free set node comp3.pccluster.org properties = any set node comp3.pccluster.org properties += score set node comp3.pccluster.org properties += score-pcc set node comp3.pccluster.org ntype = cluster
Creation of default queue
Create a default queue. In the following example, the created queue is named as "default."
# qmgr -c "create queue default queue_type=execution" # qmgr -c "set queue default enabled=true" # qmgr -c "set queue default started=true" # qmgr -c "set server default_queue=default"
For more detail on the config
file, please refer to Section 3.5.2. Queue Configuration of PBS Administrator Guide
Start of scheduling
Finally, activate PBS scheduling queue.
# qmgr -c "set server scheduling=true"
If PBS/SCore is installed from a binary package, then C based scheduler is implemented. For more detail on the C based scheduler, please refer to Section 9.9. C Based Scheduler of PBS Administrator Guide
9. The difference between PBS and PBS/SCore
score
property
In PBS/SCore, the property name, score
is treated as a special name to distinguish not-SCore jobs and SCore jobs.
Host to invoke SCore jobs
Any SCore jobs must be invoked via the scout
and scrun
programs. The scout
program creates a parallel execution environment in a cluster, and the scrun
program invokes user parallel progam in the parallel environment. The scout
program can be invoked outside of a cluster.
In contrast, parallel program invokation scheme of PBS is different. PBS assumed user parallel program is firstly invoked on a compute host. And then the program itself spawns remote processes on the other compute hosts.
"-m" option is added to pbs_server
to specify the host to invoke SCore jobs. As described above, pbs_mon
daemon process must be running on this host. However, this host cannot be allocated to a designated compute host. By default, the host is allocated dynamically one of the compute hosts.
pbs_server -m <hostname>[:<portno>]
![]() |
PC Cluster Consotium |