Skip to main content

AutoSys Jobs with JIL

Create, update and delete AutoSys jobs with JIL, with working examples for command, box and file watcher jobs.

Updated 7 min readMarkdown
AutoSys Jobs with JIL

If you run the same jobs every night, backups, reports, file loads, you need a scheduler you can trust. AutoSys is Broadcom Workload Automation AE, a scheduler that runs jobs on the machines you name, in the order you define, and tells you when one fails.

This post is for a beginner who has access to AutoSys and can run commands on the scheduler. By the end you will be able to create a job, check it, update it, and delete it safely.

What is AutoSys?

AutoSys has three parts you need to know.

  • Jobs: the work to run, usually a script or command.
  • Machines: where the job runs. Each machine runs an AutoSys agent.
  • Scheduler: the service that decides when a job starts, based on its schedule and its dependencies.

You define jobs in Job Information Language, or JIL. JIL is a plain text file with one attribute per line. You load it with the jil command, and AutoSys stores the definition.

Job types

There are three job types. Getting this right matters, because the type decides how AutoSys treats the job.

Type Use it for
c A command job. Runs a script or command once.
b A box job. A container that groups other jobs and controls when they can run.
f A file watcher job. Waits for a file to arrive, then runs a command.

There is no script job type. A script is run by a command job.

The running example

We will use one example for the whole post: a nightly backup.

  • database_backup runs first.
  • daily_backup runs after it succeeds, at 02:00 every day, on backup_server.
  • It runs /usr/bin/backup.sh, logs its output, and raises an alarm if it fails or runs too long.

Here is the complete JIL for daily_backup:

insert_job: daily_backup
job_type: c
box_name: nightly_jobs
command: /usr/bin/backup.sh
machine: backup_server
owner: admin
permission: gx,wx
description: "Runs the daily backup script"
std_out_file: /logs/backup_out.log
std_err_file: /logs/backup_err.log
alarm_if_fail: 1
max_run_alarm: 120
days_of_week: all
start_times: "02:00"
condition: s(database_backup)

What each line does:

  • insert_job: creates a new job. Use update_job to change an existing one, not insert_job.
  • job_type: c: this is a command job.
  • box_name: nightly_jobs: puts this job inside the nightly_jobs box. The box must exist first.
  • command: the script to run, with its full path.
  • machine: the machine where the agent runs the command. The machine name must already be defined in AutoSys.
  • owner: the user the job runs as.
  • permission: gx,wx: group execute and world execute. This controls who can run or edit the job.
  • std_out_file and std_err_file: where normal output and error output are written.
  • alarm_if_fail: 1: raise an alarm if the job fails. 0 means no alarm.
  • max_run_alarm: 120: raise an alarm if the job is still running after 120 minutes.
  • days_of_week: all: run every day.
  • start_times: "02:00": earliest start time.
  • condition: s(database_backup): start only if database_backup finished successfully. s() means success. f() means failure and d() means done, whatever the result.

Save this as daily_backup.jil, then load it:

jil < daily_backup.jil

If JIL reports an error, fix that line and run it again. It will not load a file with a syntax error.

Check the job

Do not assume it loaded correctly. Check it:

autorep -J daily_backup -d

This shows the definition AutoSys stored. Check the machine, command, start time and condition match what you wrote.

To see its current status:

autorep -J daily_backup

Common statuses are SU for success, FA for failure, RU for running, and IN for inactive.

To check the machine is reachable:

autoping -m backup_server

If autoping fails, the job will fail too. Fix the agent or the machine name first.

Update a job

To change an existing job, use update_job and list only the attributes you want to change. AutoSys keeps the rest.

This example changes the script and the start time:

update_job: daily_backup
command: /usr/bin/backup_v2.sh
start_times: "03:00"

Load it the same way:

jil < update_daily_backup.jil

Then run autorep -J daily_backup -d again to confirm the change.

Delete a job

Delete with delete_job:

delete_job: daily_backup

Check two things before you delete:

  1. No other job depends on it. Run autorep -J daily_backup and check for jobs with a condition that names it.
  2. It is not inside a box that is still in use. Deleting a box job does not delete the jobs inside it, but it removes the grouping and the control the box gave you.

Delete is immediate. There is no undo, so confirm the job name first.

Group jobs in a box

A box groups related jobs. Jobs in a box do not run until the box itself starts, and you can start, stop or hold the whole box at once.

insert_job: nightly_jobs
job_type: b
owner: admin
days_of_week: all
start_times: "01:00"
description: "Nightly jobs"

Put database_backup and daily_backup inside this box with box_name: nightly_jobs. The box starts at 01:00, then its jobs run in the order their conditions allow.

Watch for a file

A file watcher waits for a file, then runs a command. Its job type is f, not c.

insert_job: process_datafile
job_type: f
machine: data_server
watch_file: /data/incoming/datafile_*.csv
command: /apps/process_data.sh
owner: admin
alarm_if_fail: 1
max_run_alarm: 60

This watches for any file matching /data/incoming/datafile_*.csv. When one arrives, it runs /apps/process_data.sh. Use a file watcher when the trigger is a file arriving, not a time of day.

Run jobs from the command line

You can start, stop and force a job without waiting for its schedule:

sendevent -E STARTJOB -J daily_backup
sendevent -E KILLJOB -J daily_backup
sendevent -E FORCE_STARTJOB -J daily_backup

Use FORCE_STARTJOB only when you know the dependencies do not matter for this run. It bypasses the condition.

When a job fails

Work through this list in order:

  1. Run autorep -J daily_backup to confirm it failed, and when.
  2. Read std_err_file, in this example /logs/backup_err.log. Then read std_out_file.
  3. Check the machine with autoping -m backup_server.
  4. Check the condition. Did database_backup succeed? If it failed, daily_backup never started, and that is correct behaviour.
  5. Fix the cause, then start the job again with sendevent -E STARTJOB -J daily_backup.

Most failures are one of four things: the script path is wrong, the machine is down, a permission is missing, or a dependency did not succeed.

Run more than one JIL file

To load several JIL files from one file, use include:

include: "nightly_jobs.jil"
include: "daily_backup.jil"

Order matters. Define the box before the jobs that go inside it. If one included file has an error, JIL stops there, so load the rest separately after you fix it.

Checklist

  • Use c for command, b for box, f for file watcher.
  • Define the machine and the box before you use them.
  • Give every job a log file and alarm_if_fail: 1.
  • Check with autorep after every create or update.
  • Check dependencies before you delete a job.

Start with one real job, get it running and failing loudly, then add the next one. That is how you build a schedule you can trust.

About the author

  • I am an Engineer in love with software and writing code. I am super passionate about the reach of technology, and when it comes to conquering new frontiers, I want to be there in the forefront.

No comments yet. Be the first to comment.