Using Job Control

Introduction

A Job is a box if it does not define a FunctionExpr and a task if it does. Both tasks and boxes may have a TimerExpr, which determines the time the job will next run.

Top Level Jobs

A top level job (typically a box) has its schedule time determined by its timer (if it defines one) or by the earliest timer defined by its children. When this timer fires that job tree is executed. Sub-level boxes and tasks are run according to any time they themselves may define or by their JobOrder within their parent.

Box Types

Boxes have two attributes that determine how they execute their child jobs. In the GUI client there are tool bar buttons for setting these.

Box Type boxtype

When a box is of type timed then it runs its children at times they specify. Those that do not define a TimerExpr are run after any that do, according to their JobOrder.

A box of type sequence runs its children according to their job order, regardless of any timers they may define.

Child Exit Status boxerror

A task's FunctionExpr must return an exit status, zero indicates success, non-zero means an error has occurred. A box can be configured to stop executing its children should the current child return an error, or to continue to

The first error status from a child becomes the status for its box parent.

The Test Jobs

If we expand the job tree created by testSetup.inq then we can see examples of these:

jobtree

The Timer Expression

Job Control uses an Inq script fragment to create and return an instance of the data type timer. In the example job Periodic the script is as follows:

// Run in 10 seconds time
timer t;
date d = getdate();
t.properties.nextRuns = dateadd(SECOND, 10, d);

// return the timer
t;

This example fires 10 seconds after the box last completed. The script must set the timer's nextRuns property for the timer to be schedulable.

You can change a job's timer by clicking on the magnifying glass in the table cell to bring up the expression editor. Then click on the Edit button to enable editing. In order to be able to commit the edit by clicking OK the script must parse. This example shows what happens when it does not:

editor

Enabling A Job

testSetup.inq leaves the jobs it creates disabled. In the GUI any number of jobs can be selected and enabled using the toolbar button enable. If the top-level job Periodic is enabled then it is executed and its Last Ran value is updated each time the timer fires. However as none of its child jobs are enabled nothing else happens. If we enable the remaining jobs in this sub-tree, Box, gcc error and no-op ok then Job Control will attempt to execute them.

Abort On Error

Notice that job Box is set to abort on error, that is if a child returns error status any subsequent children are not executed. We can see that no-op ok does not run:

abortonerror

You can use the expression editor to look at gcc error's function expression to see that it does its best to return an error. Furthermore, if Box is selected and changed to Continue On Error we can see that no-op ok then runs.

Timed Boxes

The top-level job Periodic defines the sub-tree timer itself and there are no timers elsewhere below. An alternative is to use a timed box and have children define timers according to their needs. This is demonstrated by the top-level job Absolute whose children foo and bar both define a timer expression. Here is the timer expression for foo:

// Run at a fixed time
timer t;
date d = getdate();
dateset(HOUR_OF_DAY, d, 22);
dateset(MINUTE, d, 56);
dateset(SECOND, d, 0);
if (d <= getdate())
  d = dateadd(DAY_OF_YEAR, 1, d);

t.properties.nextRuns = d;

// return the timer
t;

This expression returns a timer that fires when the time is next 22:56:00. The expression for bar is the same other than it fires three minutes later.

When these jobs are enabled Job Control evaluates the timers and assigns the earliest one as that which fires the top-level job. To experiment, adjust the timer expressions to fire shortly in the future. We see that foo runs and when complete, execution waits until the Next Runs time of bar and the top-level job has status Waiting:

timedbox

Logging

Job Output

When Job Control executes a task it prepares a file stream and places it at $process.system.out. This can be used by script to write any diagnostic output to, or passed to Inq's syscmd function if the task executes host commands. By default these files are placed at $INQHOME/tmp and are named after the job's ShortName combined with a time stamp. In fact, this line generates the file name:

// Pattern is Java MessageFormat
renderf("{0}_{1,date,yyyyMMdd_HHmmss}.log", filename, getdate());

where filename is the job's short name with any spaces removed. Job Control limits the number of files kept according to its properties configuration.

Job Control System Log

The Inq server logging configuration, defined in $INQHOME/etc/server.log.properties, includes the named logger inq.jobcontrol. This logger is used by Job Control for all its logging output.

Stopping and Starting Job Control

When the app/jobcontrol/jobBoot.inq has been parsed Job Control is running. A detached process called jobdispatcher hosts the timers. This process can be stopped (via the GUI client's System menu), thus halting the Job Control application.

The next section covers how Job Control is implemented in more detail.

API Functions

The following functions are available to other applications wishing to use Job Control programmatically.

Note
Inq code can always throw an exception which, if not caught, will be handled by a process's default exception handler. If an application has local code flow requirements such as a monitor notification, it can use a try/catch/finally construct.

function setLogLevel(any logLevel)

Sets the logging level used by the logger inq.jobcontrol.

Example:

call inq.jobcontrol:setLogLevel(LOG_FINER);

function getJob(any ShortName)

Find a job by its ShortName. Returns the Job instance or null if there is no job with the given ShortName.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "foo");

function createJob(any Job, any parent)

Create a new job. Job is the candidate instance, requiring at least the ShortName and JobOrder fields to have been initialised. If parent is supplied it must be a Job that is a box. Otherwise the new job will be at the top level.

If the job's FunctionExpr is null then the new instance will be a box. Otherwise it will be a task. Once the enclosing transaction has committed and the instance enters the managed state this field cannot be changed between null and non-null, that is a box cannot later become a task and vice versa.

Example:

// Make a top-level box
any topLevelPeriodic = new(inq.jobcontrol:Job);
topLevelPeriodic.ShortName = "Periodic";
setblob(topLevelPeriodic.TimerExpr, "// Run in 10 seconds time
timer t;
date d = getdate();
t.properties.nextRuns = dateadd(SECOND, 10, d);

// return the timer
t;
");

topLevelPeriodic.JobOrder = 0;

any Job = call inq.jobcontrol:createJob(Job = topLevelPeriodic);

function deleteJob(any Job)

Delete a single Job. If a job is a box then its children are deleted also.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "foo");

call inq.jobcontrol:deleteJob(Job);

function deleteJobs(any jobs)

Delete a list of Jobs. If any job is a box then its children are deleted also.

This example deletes the children of job foo without deleting foo itself:

any Job = call inq.jobcontrol:getJob(ShortName = "foo");

any root = call inq.jobcontrol:getJobsBelow(Job);

call inq.jobcontrol:deleteJobs(jobs = root.tree.childJobs);

function getJobsBelow(any Job, any root)

Creates a tree structure rooted at the specified Job. The root argument is optional. If specified it will contain the tree and must be a map type that supports vector access. If not supplied a omap is created and returned.

The structure conforms to that of a nested node set whose paths are root.tree.Job and root.tree.childJobs[].Job and recursively downwards.

See function deleteJobs(any jobs) for an example

Mutation Functions

As we discuss further when examining the implementation, Job instances can only be modified by specific functions provided for the purpose. When certain fields are changed, in particular TimerExpr and Active, Job Control re-evaluates the timer for the affected sub-tree.

If Job Control is running, mutating a Job instance directly causes an exception.

function setJobExpression(any Job, any FunctionExpr)

Set the FunctionExpr of the specified Job. This function can only be called for jobs that are already tasks, that is they were created with a non-null FunctionExpr. Violating this condition causes an exception to be thrown during transaction commit.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "foo");

call inq.jobcontrol:setJobExpression(Job, "
call myFoo(arg = \"something\");
");

function setJobTimer(any Job, any TimerExpr)

Set the TimerExpr of the specified Job. If Job Control is running this function causes the affected sub-tree timer to be reevaluated.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "foo");

call inq.jobcontrol:setJobTimer(Job, "// Run at a fixed time
timer t;
date d = getdate();
dateset(HOUR_OF_DAY, d, 22);
dateset(MINUTE, d, 56);
dateset(SECOND, d, 0);
if (d <= getdate())
  d = dateadd(DAY_OF_YEAR, 1, d);

t.properties.nextRuns = d;

// return the timer
t;
");

function setJobShortName(any Job, any ShortName)

Set the specified job's ShortName.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "foo");

call setJobShortName(Job, "Fetch Prices");

function setJobOrder(any Job, any JobOrder)

Set the specified job's JobOrder. In the absence of any timer, the JobOrder field determines the order of the child jobs within a box.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "foo");

call inq.jobcontrol:setJobOrder(Job, 2);

function restartJob(any Job)

The specified Job must at the top-level. The sub-tree rooted at Job has its timer reevaluated and scheduled.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "Master");

call inq.jobcontrol:restartJob(Job);

function runJobNow(any Job)

Executes the specified Job immediately. The job can be any job in the tree, not necessarily a top-level one. Timers that the job or any of its descendents define are ignored and children are run in their defined order.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "bar");

call inq.jobcontrol:runJobNow(Job);

function killJob(any Job)

Kills a job, if it is running, otherwise has no effect. If the specified Job or any ancestor is executing its execution is halted immediately and any subsequent children are not run. The Job.LastRan field is not updated.

Example:

any Job = call inq.jobcontrol:getJob(ShortName = "bar");

call inq.jobcontrol:killJob(Job);

function startJobControl()

Starts Job Control, scheduling timers of active jobs. Throws an exception if Job Control is already running.

Example:

call inq.jobcontrol:startJobControl();

function stopJobControl()

Stops Job Control, cancelling all timers. Throws an exception if Job Control is not running.

When Job Control is not running jobs can still be created, mutated and deleted, however calling restartJob or runJobNow throws an exception.

Example:

call inq.jobcontrol:stopJobControl();