GridSAM User Guide

Introduction

The User Guide documents the tools available in GridSAM for submitting, controlling and monitoring jobs submitted to a GridSAM Web Service. If you are a service administrator interested in deploying and configuring a GridSAM Web Service, please consult the Deployment Guide.

Concepts

A job in GridSAM is defined as an executable entity (e.g. binary) described by a JSDL document. A job is an atomic unit to be executed and monitored through GridSAM. A job in GridSAM can undergo the following state transition.

GridSAM Job State Transition

Command-line tools

gridsam-submit

SYNOPSIS
usage: gridsam-submit (-s service-endpoint-address | -sn service-name)
        [-j path-to-jsdl] [-suspended] 
        [-myproxy [-myproxypassword password] -myproxyhost hostname 
        [-myproxyport port] [-myproxyuser user]]
        path-to-jsdl [path-to-jsdl..]
DESCRIPTION

Command-line tool for submitting a JSDL document to a GridSAM Web Service. Upon successful submission, a Job Identifier would be written to the standard output. The Job Identifier can be used subsequently in other GridSAM commands for querying or monitoring the state of the job.

  • -j: (Required) File location to the JSDL document to be submitted. This can be a path relative to the current directory or an absolute path pointing to the JSDL file.
  • -suspended: Submit the job as suspended. The job will remain at the pending state. The job can be moved into the execution pipeline by using the gridsam-start command. The default is the job will be moved into the execution stage upon submission.
  • -s: (Optional) URL denoting the endpoint of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -sn: (Optional) Logical name of the service defined in the ~/.gridsam/services.properties. The file is in Java properties file format. It contains key-value pairs denoting the logical name and the corresponding endpoint address of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -myproxy: Enable MyProxy support. MyProxy details would be automatically inserted into the JSDL document as extension. These extension elements are interpreted by the GridSAM service as credentials to interact with Globus based compute and file resources.
  • -myproxypassword (Optional): The MyProxy passphrase for GridSAM to unlock the proxy. If omitted, the user would be prompted for the password. The use of this option is strongly discouraged because of the potential security risk.
  • -myproxyhost: The hostname of the MyProxy server holding the user's credential.
  • -myproxyport: (Optional) The port of the MyProxy server holding the user's credential. The default is "7512".
  • -myproxyuser: (Optional) The user name of the MyProxy credential held in the MyProxy server (-myproxyhost). This is defaulted to the user name of the current user.
  • path-to-jsdl: Same as -j. Can specify more than one JSDL documents to be submitted.
RESULTS

Upon successful submission, the command outputs a Job Identifier for each JSDL document specified in the parameters and exits with 0. Exit code other than 0 indicates an error. An error message would be printed to the standard error stream.

EXAMPLES
$> gridsam-submit -j path/to/myjob.jsdl -s "http://myhost:18080/gridsam/services/gridsam" 
$> gridsam-submit -sn myservice path/to/myjob.jsdl
$> gridsam-submit -suspended path/to/myjob.jsdl another/job.jsdl

gridsam-status

SYNOPSIS
usage: gridsam-status (-s service-endpoint-address | -sn service-name) [-j job-id] [-file job-id-file] [-x] job-id [job-id..]
DESCRIPTION

Command-line tool for query the status of a job previously submitted to a GridSAM Web Service. Upon successful submission, a job status report is written to the standard output for inspection. By default, the command will produce a human-readable output unless the -x parameter is present. The -x parameter instructs the command to produce a machine-parsable XML output.

  • -j: (Required) A Job Identifier denoting a job previously submitted to a GridSAM Web Service. This is the job identifier produced by the gridsam-submit command.
  • -s: (Optional) URL denoting the endpoint of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -sn: (Optional) Logical name of the service defined in the ~/.gridsam/services.properties. The file is in Java properties file format. It contains key-value pairs denoting the logical name and the corresponding endpoint address of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -file: (Optional) A file containing a list of job IDs to be queried. Each line of the file should be a file ID collected from a call to gridsam-submit.
  • -x: (Optional) Produce a gridsam:JobStatus XML output that can be parsed by any XML compliant tool. The XML output satisfies the following pseudo-schema.
    <gridsam:JobStatus>
    
        <!-- stage of the job in chronological order -->
        <gridsam:Stage> +
    
            <!-- the state of the job -->
            <gridsam:State>
                pending|staging-in|staged-in|active|executed|
                staging-out|staged-out|done|failed|undefined
            </gridsam:State>
    
            <!-- human-readable description associated with the state -->
            <gridsam:Description>xsd:string</gridsam:Description>
    
            <!-- the date and time the state is reached -->
            <gridsam:Time>xsd:DateTime</gridsam:Time>
    
        </gridsam:Stage>
    
        <!-- properties associated with the job -->
        <gridsam:Property name="xsd:anyURI">?
            xsd:string
        </gridsam:Property>?
    
    </gridsam:JobStatus>
  • job-id: Same as -j. Can specify more than one job ID.
RESULTS

Upon successful submission, the command outputs the job status for each job ID to the standard output and exits with 0. Exit code other than 0 indicates an error. An error message would be printed to the standard error stream.

EXAMPLES
$> gridsam-status -s "http://myhost:18080/gridsam/services/gridsam" -x urn:gridsam:1231241241414

gridsam-terminate

SYNOPSIS
usage: gridsam-terminate (-s service-endpoint-address | -sn service-name) 
    [-j job-id] [-file job-id-file] job-id [job-id..]
DESCRIPTION

Command-line tool for terminating a job previously submitted to a GridSAM Web Service. Upon successful execution, the job will start to move to the terminating state, all subsequent stages of the job will be passed. Use the gridsam-status to determine whether the job has completely terminated.

  • -j: (Required) A Job Identifier denoting a job previously submitted to a GridSAM Web Service. This is the job identifier produced by the gridsam-submit command.
  • -s: (Optional) URL denoting the endpoint of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -sn: (Optional) Logical name of the service defined in the ~/.gridsam/services.properties. The file is in Java properties file format. It contains key-value pairs denoting the logical name and the corresponding endpoint address of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -file: (Optional) A file containing a list of job IDs to be terminated. Each line of the file should be a file ID collected from a call to gridsam-submit.
  • job-id: Same as -j. Can specify more than one job ID.
RESULTS

Upon successful submission, the command exits with 0. Exit code other than 0 indicates an error. An error message would be printed to the standard error stream.

$> gridsam-terminate -sn myservice urn:gridsam:1231241241414

gridsam-start

SYNOPSIS
usage: gridsam-start (-s service-endpoint-address | -sn service-name) 
    [-j job-id] [-file job-id-file] job-id [job-id..]
DESCRIPTION

Command-line tool for starting jobs previously submitted as 'suspended'. Upon successful execution, the job will start to enter the execution pipeline.

  • -j: (Required) A Job Identifier denoting a job previously submitted to a GridSAM Web Service. This is the job identifier produced by the gridsam-submit command.
  • -s: (Optional) URL denoting the endpoint of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -sn: (Optional) Logical name of the service defined in the ~/.gridsam/services.properties. The file is in Java properties file format. It contains key-value pairs denoting the logical name and the corresponding endpoint address of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -file: (Optional) A file containing a list of job IDs to be terminated. Each line of the file should be a file ID collected from a call to gridsam-submit.
  • job-id: Same as -j. Can specify more than one job ID.
RESULTS

Upon successful submission, the command exits with 0. Exit code other than 0 indicates an error. An error message would be printed to the standard error stream.

$> gridsam-start -sn myservice urn:gridsam:1231241241414

gridsam-groovy

SYNOPSIS
usage: gridsam-groovy (-s service-endpoint-address | -sn service-name) 
    -groovy groovy-script-url [args..]
DESCRIPTION

Command-line tool for scripting job submission using the Groovy Language. A variable of type org.icenigrid.gridsam.core.JobManager is made available through the variable "jobManager". This can be used to submit jobs to the GridSAM service identified by the -s or -sn parameters.

  • -s: (Optional) URL denoting the endpoint of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -sn: (Optional) Logical name of the service defined in the ~/.gridsam/services.properties. The file is in Java properties file format. It contains key-value pairs denoting the logical name and the corresponding endpoint address of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -groovy: The script to be executed.
  • args: Arguments to be passed to the script as the java.util.List variable named "args".
RESULTS

Upon successful submission, the command exits with 0. Exit code other than 0 indicates an error. An error message would be printed to the standard error stream.

$> more /home/myuser/myscript.groovy
import java.io.File

jsdl = (new File(args[0])).getText()


job = jobManager.submitJob(jsdl)

println(job.jobStages)

while(true) {
        job = jobManager.findJobInstance(job.ID)
        println(job.jobStages)
        if(job.jobStages.reverse()[0].state.toString() == "done") break
}

println(job.jobStages[0].state)
$> gridsam-groovy -sn myservice -groovy file:///home/myuser/myscript.groovy data/examples/uname.jsdl

gridsam-ftp-server

SYNOPSIS
usage: gridsam-ftp-server -p <ftp-port> -d <root-directory> [-l]
DESCRIPTION

Anonymous FTP server daemon. It exposes a chosen directory on the user's file system as a FTP file system. This can be used for anonymous FTP file staging.

  • -p: (Required) The FTP port
  • -d: (Required) The path to the directory to be exposed as the root of the FTP server.
  • -l: (Optional) The FTP server is ONLY available to the localhost (i.e. bound to the loopback interface) for testing purpose.
EXAMPLES
$> gridsam-ftp-server -p 19245 -d /my/public/directory

gridsam-file-transfer

SYNOPSIS
usage: gridsam-file-transfer from-url to-url
DESCRIPTION

Command-line tool for transferring file between remote servers.

  • from-url: (Required) The source location of the file to be transferred. Supported URL schemes are file://, http://, ftp://, sftp://, webdav:// and gsiftp://. * to-url: (Required) The target location of the file to be transferred. Supported URL schemes are file://, ftp://, sftp://, webdav:// and gsiftp://.
EXAMPLES
$> gridsam-ftp-server gsiftp://gridftp.host.com/file.txt webdav://webdav.host.com/webdavfile.txt

gridsam-stress-test

SYNOPSIS
usage: gridsam-stress-test (-s service-endpoint-address | -sn service-name) [-j
    path-to-jsdl] [-o omii-conf-dir] [-wssec on|off] [-users no-of-users]
    [-jobs repetition] [-delay milliseconds] path-to-jsdl
DESCRIPTION

Performs a stress test on a GridSAM web service. It simulates a number of users submitting a series of jobs to a GridSAM web service concurrently.

  • -j: (Required) File location to the JSDL document to be submitted. This can be a path relative to the current directory or an absolute path pointing to the JSDL file.
  • -s: (Optional) URL denoting the endpoint of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -sn: (Optional) Logical name of the service defined in the ~/.gridsam/services.properties. The file is in Java properties file format. It contains key-value pairs denoting the logical name and the corresponding endpoint address of the GridSAM service. If both -sn or -s are not specified and the ~/.gridsam/services.properties is non-empty, the first entry would be used by default.
  • -o: (Optional) A path to a directory containing OMII client configuration files. This is defaulted to $OMII_CLIENT_HOME/conf. This path is used to look up WS-Security configuration used to to invoke the GridSAM service.
  • -users: (Required) number of concurrent users to simulate.
  • -jobs: (Required) number of jobs to be submitted per user. The jobs are submitted sequentially delayed by the parameter -delay.
  • -delay: (Required) number of milliseconds to wait before a user submits another job.
  • path-to-jsdl: Same as -j
RESULTS

The command will write a XML report to the standard output. The XML report conforms to the following pseudo-schema

    <!--
        concurrency: number of concurrent users
        jobs: number of jobs per user
    -->
    <test concurrency="xsd:integer" jobs="xsd:integer">

        <!-- For each submission (i.e. GridSAM.submitJob() is successful)
            id: the GridSAM ID of the job
            responseTime: the time(ms) taken to invoke GridSAM.submitJob() to
                the time the ID is returned to the user.
        -->
        <submittedJob id="xsd:string" responseTime="xsd:integer" /> +

        <!-- For each successful submission (i.e. GridSAM.submitJob() is successful),
            a job that is completed successfully (i.e. job state = Done)
            id: the GridSAM ID of the job
            submittedAt: the time (ms since the epoch) the job is submitted by the user (client's clock)
            enteredAt: the time (ms since the epoch) the job is entered the GridSAM service (server's clock)
            submittedAt: the time (ms since the epoch) the job reached the done stage (server's clock)
        -->
        <completedJob id="xsd:string"
            submittedAt="xsd:integer"
            enteredAt="xsd:integer"
            completedAt="xsd:integer"
            > +

        <!-- For each successful submission (i.e. GridSAM.submitJob() is successful),
            a job that is failed (i.e. job state = Failed)
            id: the GridSAM ID of the job
            submittedAt: the time (ms since the epoch) the job is submitted by the user (client's clock)
            enteredAt: the time (ms since the epoch) the job is entered the GridSAM service (server's clock)
            submittedAt: the time (ms since the epoch) the job reached the failed | undefined stage (server's clock)
            content: the failure message
        -->
        <failedJob id="xsd:string"
            submittedAt="xsd:integer"
            enteredAt="xsd:integer"
            completedAt="xsd:integer"
            >+
            xsd:string
        </failedJob>

        <!-- aggregated statistics -->
        <statistics>
                <!-- number of failed submissions (i.e. GridSAM.submitJob() failed to return an ID) -->
                <totalFailedSubmissions>xsd:integer</totalFailedSubmissions>
                <!-- number of failed job (i.e. job state = failed) -->
                <totalFailedJobs>xsd:integer</totalFailedJobs>
                <!-- the average response time (ms) of GridSAM.submitJob() for the number
                of successful samples
                -->
                <averageResponseTime samples="xsd:integer">xsd:float</averageResponseTime>
                <!-- the average turnaround time (ms) for the job to be completed (reached the done state)
                -->
                <averageTurnaroundTime samples="xsd:integer">xsd:float</averageTurnaroundTime>
        </statistics>

    </test>
EXAMPLES
$> gridsam-stress-test -sn localhost -users 10 -jobs 10 -delay 1000 data/examples/sleep.jsdl

gridsam-version

SYNOPSIS
usage: gridsam-version
DESCRIPTION

Prints the GridSAM version number to the console

myproxy

SYNOPSIS
usage: myproxy [common options] command [command options]
DESCRIPTION

Command-line tool for managing MyProxy credential on a MyProxy server.

    Common Options:
    -help
            Displays usage
    -v | -version
            Displays version

    -h <host> | -host <host>
            Hostname of the myproxy-server
    -p <port> | -port <port>
            Port of the myproxy-server
            (default 7512)
    -s <subject> | -subject <subject>
            Performs subject authorization
    -l <username> | -username <username>
            Username for the delegated proxy
    -d | -dn_as_username
            Use the proxy certificate subject (DN) as the default
            username instead of the "user.name" system property.

    Commands:
     put     - put proxy
     get     - get proxy
     anonget - get proxy without local credentials
     destroy - remove proxy
     info    - credential information
     pwd     - change credential password

    Specify -help after a command name for command-specific help.
EXAMPLES
$> myproxy -l myusername -host myproxy.ncsa.uiuc.edu put