Sidebar 11: Csironet Development Environment and Sample Cyber 76 job
Sidebar 11:
Csironet Development Environment and Sample Cyber 76 job
Started 15 Jan 2024.
Last updated – first pass done: 23 Jan 2024
Updated 06 Feb 2024 to add sample dayfile.
Robert C. Bell
Top
Introduction and aim
This page seeks to give some insight into the development and run environment for users of the Csironet services in the late 1970s and early 1980s, when the sole user host was the CDC Cyber 76.
By this stage, most users had switched from using punched card decks for jobs to using the on-line interactive system which DCR built on the Cyber 76, and which was accessible to all CSIRO Divisions and other sites over the telecommunications network. Nodes of the network typically had a DEC PDP-11 as the node host, and supported terminals, card readers, printers, plotters and other peripherals.
By the early 1980s, most users had a terminal on their desk (I had an HP2621A – see http://www.hpmuseum.net/display_item.php?hw=242). I had a microfiche reader, an HP 15C calculator (which could do complex arithmetic!), manuals (printed), an automatic pencil, an eraser, a highlighter pen, and a special ruler (see https://csiropedia.csiro.au/csiro-computing-history-appendix-7-artefacts/) for counting rows and columns – fully equipped. 
Sample job – access
Below is a sample job for the DCR CDC Cyber 76 from the early 1980s. The job illustrates the editor Ed commands and capabilities of the interactive CYI system, the SCOPE 2 job control, Cyber Command Language, and the use of the UPDATE source code utility and the FTN5 Fortran compiler.
(This page contains a minimal summary of Ed commands, and the document Summary_of_Commands_for_CSIRONET_Terminals_and_the_Subsystems_of_CYI.ENTO.report.10.1980 is a useful companion to CYI and Ed.)
A user would log into Csironet from a terminal with a command like:
*CYI,pid,,,
This specified a project id, and used all the defaults (e.g. user id, node and the Ed editor.)
A user might then copy a partition from an Ed library into the workspace with a command like:
‘c/rob01,bpel/
where the apostrophe specified mixed-case (ASCII) (later not needed as the type was taken from the partition’s metadata), and bpel is the name of an Ed library.
The metadata for the partition was:
NAME SIZE T CREATION RB01 1 A 841123 0918 JOB TO DO PRODUCTION RUN RB01.
(The ability to store a comment of up to 70 characters with each partition was far more valuable than anyone envisaged I think! )
Far below is the full text of the job. Immediately below is a breakdown of the lines and commands in the job.
The job was stored as a partition of an Ed library – DCR under the inspiration of Henry Hudson adapted the capabilities of the CDC binary libraries to provide a facility for storing multiple text partitions in a single permanent file, thus vastly reducing the load on the permanent file system, and in particular, its metadata performance.
Job analysis
Set-up and job initiation with the Ed editor
The first line is a line of commands for the Ed text editor. Once the partition (called RB01) was loaded into the Ed workspace (and possibly edited and saved again), then the job would be run by executing an Ed command like
21<21$
This command instructed the editor to suck the current line into a box numbered 21, and then execute the command in box 21, which would be the following line.
^;E?;Enter priority, Pxxxx;O'D%pI'0'?;Enter passwords;D@pI'0'\KPo@e
Boxes were a set of numbered locations for storing text; one of the genius characteristics of Ed was that command strings could be stored in boxes and executed, with support for recursion, branching and looping! Ed provided a simple conversational facility as well as very powerful editing and text manipulation capabilities.
The ^; instructed Ed to enter relaxed mode, where commands that expected a delimited text string (delimited with ; / ! or ‘) would instead take the next single character as the text string if a delimiter was not present. This saved a few characters of typing and box space!
The E command erased the current line from the workspace (but it was still held in box 21).
The ?;Enter priority, Pxxxx; displayed the text “Enter priority, Pxxxx” to the user, asking for a response. Priorities were important – they both influenced how quickly a job might be started, but also what the charging rate for the job would be – with quite large ranges.
The O command moved the pointer to the origin of the workspace.
The D% command, using relaxed mode, deleted the first % character from the workfile and moved the pointer to there – that is the original second line of the workfile, but now the first line.
The p command then printed the current line. The i’0′ command then inserted what the user typed into the workfile.
The ?;Enter passwords; command displayed the text “Enter passwords”. Asking for passwords obviated the need to store passwords in files.
The D@ command deleted the first @ character from the workfile and moved the pointer there. The p command then printed the current line. The i’0’ command then inserted what the user typed into the workfile.
The \kP command then instructed the editor to submit the workfile to the Cyber 76 batch system to be run non-interactively, and to direct the output to the RIO (remote input/output) system for dealing with the output of the job. (The KP command would run the job interactively, with output directed to an alternate workfile.)
Finally, the o@e command returned the pointer to the origin, and erased all the lines of the workfile – that was done so that the password that had been entered was no longer in the workfile. The @ character was a repetition count of 10000. The @e command erased as many lines as were available up to 10000 lines. (If the @ was preceded by a number, e.g. 22@, then the count in box 22 would be used instead of 10000. Mind you, a file of more than 10000 lines would have been huge in those days!)
Here endeth line 1!
SCOPE 2 job control statements
Note that the job statements included both lower case and upper case characters. The Ed editor converted all the lower case letters in the job to upper case prior to submission to the Cyber 76, which used a 6-bit character coding called DISPLAY code which did not include lower case letters. (Lower case could be handled by the editor and applications.) Job commands were terminated by a full stop or a closing bracket.
pbarotr(%,T200,dbp00) <<<<< Robert Bell >>>>>
The first line of the job is a job statement. It gave a name to the job (pbarotr), and parameters for the job in brackets. These specified the priority (inserted by Ed), a time limit (200 CPU seconds), and also contained a dependency flag. In this case, a set of jobs could share an identifying string (dbp), and each job could specify how many jobs need to be completed and signalled to this job before it would start. In this case, the count is zero (00), and the job could be started as soon as the scheduler wished. This facility was very useful when setting up a suite of jobs – perhaps one to prepare or fetch data, another to run a model, and later ones to archive or plot output data.
Specifying the priority was very important, because the charge for the job depended dramatically on the priority selected. The Csironet News from October 1981 shows that the charging factor applied to the Cyber 76 CPU time used ranged from 0.01 for priority P10 (overnight and weekends only) to 1.955 for priority P7000 which is the break-in facility – immediate execution using the Ed K command. A time limit of 200 seconds (CPU time) may seem a small chunk, but for some years, CSIRO’s Cyber 76’s one CPU was the only processor available to users in the whole of CSIRO’s central computing facility until the VM/CMS, FACOM and lower Cyber systems arrived.
After the closing bracket, comments could be included.
Comment. Job to run program barotr.
The next line is a comment, which in this case gives a brief note about the purpose of the job.
GETSET,CMP9007.
This instructed SCOPE 2 to mount a permanent file subset. As well as the default system permanent file set, DCR provided subsets for cheaper storage (without backup), typically managed on a CSIRO Division basis. This set was for CSIRO Meteorological Physics (CMP).
dcr.
This was a system command added to SCOPE 2, presumably to enable access to various local utilities.
attach(lib,sk2,id=cmpxsk,sn=cmp9007)
This was an instruction to attach a permanent file to a logical file with name ‘lib’. The file sk2 belonged to the user Steve Kepert (now sadly deceased), and was on the CMP subset.
borro(p=ztlpp15,c=olddata)
This was a DCR utility to make a copy of a partition called ztlpp15 from the library lib onto a logical file name olddata. The file olddata was rewound both before and after the copy was made.
(This raises the issue of the underlying file system in SCOPE 2. The default record structure for sequential files on disc was a type called W-record. This mirrored the Fortran conventions for sequential files, with each line (or record) preceded and followed by a control word containing a count of bits in the record. This allowed operations like backspace and skipping, just like tape-based files. The control words could also contain the equivalent of tape marks, and there was a range of separators for end-or-record, end-of-section, end-of-partition and end-of-file.)
return(lib)
This command released the logical file name ‘lib’, and thus allowed the next commands to re-use that and copy a partition to the logical file name tape42 (this would be used in the Fortran program as unit number 42.
attach(lib,bz21,id=cmpxjf,sn=cmp9007)
borro(p=toppuri,c=tape42)
begin,,input,tk=[@],docompile,runno=rb01,id=cmpxrb.
This is the last explicit SCOPE 2 control statement for this job, and is followed by
*EOR
This was a pseudonym for an end-of-section delimiter, which Ed created before submitting a job. The SCOPE 2 Users’ Guide contains:
“End-of-section cards are often referred to as end-of-record cards as a holdover from SCOPE 3.3 terminology.” (page 2-8).
The begin command uses the CDC CYBER Command Language (CCL) to set up and invoke a procedure with default name from the input. The begin command starts reading lines from the next *EOR to the following *EOR, starting with the line
.proc,run,docompile=f/t,tk=,runno=,id=cmpxrb,sysid=cmpxqf.
to build the procedure. Parameters are passed to the procedure. The first parameter, the procedure name, is here omitted and defaults to that on the .proc line, i.e. ‘run’. The second parameter, ‘input’, defines where the procedure is to be found. After that, parameters can be set in any order. In this case, the turnkey password will be passed. Parameters could have two defaults – the first being when the procedure was called without specifying the corresponding parameter, the second being when the parameter was specified without giving a value (with an = sign). In this case, the begin call specifies ‘docompile’, and so its value is set to ‘t’. A run number ‘rb01’ is specified, along with my ident ‘cmpxrb’.
attach(oldpl,$bp.pl$,#id=sysid,sn=cmp9007,pw=tk)
This attached an UPDATE program library to the local file name oldpl. The $ signs delimit a string, and the # sign inhibits value substitution. The strings “sysid” and “tk” will be replaced by the values specified in the call or by defaults, i.e. cmpxqf and the turnkey password entered via the Ed box program.
update(l=124,q)
This command invoked the UPDATE utility – the l parameter specified various listings (1 selects errors to be listed, 2 selects listing of directives encountered in the input and the old program library, and 4 selects listing of all input text lines). The q parameter selects quick mode – only decks selected by *COMPILE directives in the input to UPDATE would be processed. By default, text would be written to a file called ‘compile’, and no new program library would be created.
UPDATE processing
The UPDATE utility reflected its origin in replacing card decks. An initial run of UPDATE would create a program library from an input file containing ‘images’ of punched cards – in fact they were just lines of text, but divided by directives such as
*DECK MAIN
and
*COMDECK COMMON1
By default, only the first 72 columns of a ‘card’ would be processed, with the remaining columns being reserved for an identifier for each line. The identifiers were made up of the deck name and a line number, e.g. MAIN.1, MAIN.2, MAIN.3, COMDECK.1, etc.
Later runs of UPDATE would access the stored text, would make modifications to the decks in groups called IDENTs, would optionally save the changes in a new program library, and would optionally output a compile file containing the source code to be passed to a compiler.
In the example below, there are two groups of directives. Here is the first set.
*IDENT GETDATA
*COMPILE RBAROTR,TSTBAROTR
*COMPILE BAROTR3
*COMPILE MUTTOP
*DECLARE MUTTOP
*i muttop.24
C
C Zero the first 16 elements of the topography.
Do 3458 il = 1, 16
phis(il) = (0.0, 0.0)
3458 continue
*DECLARE,,
This defines an ident called GETDATA – changed lines are labelled with this ident and a line number. The *COMPILE lines declare which decks are to be copied to the compile file. The *COMPILE BAROTR3 forces the main program to be added to the compile file, which makes the load sequence simpler. The *DECLARE directive declares that the following changes must apply to the named deck, i.e. MUTTOP – this prevents some stray changes. The insert command specifies that the following lines up to the next directive are to be inserted after the line muttop.24. Finally, a null DECLARE directive prevents further changes until another DECLARE directive.
Here is an extract from the second set of directives.
*IDENT -INDATA
*/ Ident -indata. Replace the current input data for Barotr.
*COMPILE TSTBAROTR
*DECLARE TSTBAROTR
*d 20-dynarr.237,42-files.223
/ irarry (default 15). maximum ir value, to be used to set up arrays.
/ nlevel (default 1). The number of model levels.
/ nlinzd (default 1). Linearization flag. 1 for nonlin, 0 for linearized.
120, , , , 14, / ktotal, idelt, ir, imp, istart, iend.
…
*DECLARE,,
Again, there is an IDENT, COMPILE and DECLARE directives, and a delete directive with replacement lines. In this case, the deck contains input data to the Fortran program.
The compile file will contain lines for the compiler, plus after an end-of-section marker, a procedure to run the program (from deck RBAROTR), plus after EOS, data for the program from deck TSTBAROTR.
Compiling, loading and processing
The procedure now has a conditional, depending on whether compilation has been specified with the docompile flag. If so, the CDC FTN5 compile is invoked – it was a Fortran 77 compiler.
.ife,docompile,docomp.
ftn5(ansi,et,i,lo=m/r,md,opt=2)
The ansi flag specified that any non-standard Fortran usage would be flagged. The et option specified that the job will skip to an exit statement if there is a fatal compilation error. The i option selects i=compile for input to the compiler. The lo=m/r specified a memory allocation and cross-reference map, the md flag specified that machine dependencies are to be treated as trivial, and the opt-2 flag specifies optimisation, but not any potentially unsafe optimisation.
The next line specified the attachment of a library of compiled subroutines, using the turnkey password.
attach(newlib,$bp.lib$,#id=sysid,sn=cmp9007,pw=tk)
(At the time, there was a concern about other parties apart from the research group at Aspendale getting access to this particular code.)
The next three lines are part of a load sequence, and were treated by the loader as one command.
ldset(lib=newlib,compres=all,err=all,map=b,preset=nginf)
load(lgo)
nogo(abs)
The first line specified a library for finding unsatisfied externals. The compres=all specified compression of all program and common blocks in the absolute file. The err=all instructed the loader to not proceed on encountering any errors, the map=b specified a statistics and blocks map, and preset=nginf specified that any memory that was not otherwise specified was to be filled with a value representing negative infinity, but had the bonus of loading the address of each word of memory into its lower bits – this helped debugging.
The next line specified the loader to load compiled code from the file lgo which the compiler had created. The last line specified saving of the absolute file to a file called abs, and not to proceed to execution of the file.
The next lines returned files not needed, and ended the conditional branch.
return(lgo,newlib)
.endif,docomp.
The next line invoked the procedure rbarotr from the file compile, with added parameter pl=5000 (which was an option to be passed to Fortran programs to limit the number of lines in the standard OUTPUT file), and others which were passed into this procedure via the call and the begin statement.
begin,rbarotr,compile,pl=5000,#id=id,#runno=runno,#sysid=sysid.
The last line returned the abs file, which would have been done at program termination anyway, but might have been put here in preparation for adding any further steps.
return(abs)
A copy of the rbarotr procedure is shown far below – it may be an earlier version than the one used in 1984.
Sample Job output
I don’t have the output from the above job! I do have copies of outputs from a few sample jobs. Here is one. (This shows only extracts – the first, and last two dayfile pages.)
Printed job output was on fan-fold 11 inch by 15 inch paper with sprocket holes on both edges. The first line showed information about the network output – a sequence number (0130), the type of output (LP = line printer), a time of day, a node mnemonic (MA), a username (CMPXRB), a job name (FMAINT); then more information about the peripheral device (1-part paper), another time, a date, a fan-out node mnemonic (East Melburne), and then the last digit of the sequence number, for easier identification of the boundaries between printouts.
Then followed the Cyber 76 dayfile, and then after that any output from job steps.
The dayfile was such a useful concept. It showed for every command the time of day at which it was executed, the cumulative CPU time so far, and the origin of each command. Then followed an echo of the command, preceded by a minus sign. Subsequent lines optionally showed messages about the execution of a command.
The last line of the second page shown is the beginning of the accounting information. The first part is a list of the sizes of the files the job read or wrote. Then came a message about how many large-core/small-core swaps. (Code had to be in small-core to be executed, but with a heavily over-committed machine, multiple jobs and interactive sessions were run simultaneously, and inactive jobs would be swapped from small core to large core.)
Then came the accounting information, with the components of the charge, and the final charge. This job was run a priority 1000, which was about the lowest priority to be selected to make the job a candidate to run during prime time. The total job cost was $3.87, for less than 4 seconds of CPU time at a rate of 81.38 cents per unit. After the charge line, the next line is the first line of standard output, reporting a failure to attach a file.
The job was submitted from ED at 16.08.13. Execution commenced at 16.08.24, and finished at 16.11.26, and used 3.932 seconds of CPU time in 182 elapsed seconds, i.e. a CPU utilisation of 2.16%. This particular job was a file maintenance job, so a heavy user of the file system no doubt subject to many waits.
Full text of job
^;E?;Enter priority, Pxxxx;O'D%pI'0'?;Enter passwords;D@pI'0'\KPo@e pbarotr(%,T200,dbp00) <<<<< Robert Bell >>>>> Comment. Job to run program barotr. GETSET,CMP9007. dcr. attach(lib,sk2,id=cmpxsk,sn=cmp9007) borro(p=ztlpp15,c=olddata) return(lib) attach(lib,bz21,id=cmpxjf,sn=cmp9007) borro(p=toppuri,c=tape42) begin,,input,tk=[@],docompile,runno=rb01,id=cmpxrb. *EOR .proc,run,docompile=f/t,tk=,runno=,id=cmpxrb,sysid=cmpxqf. attach(oldpl,$bp.pl$,#id=sysid,sn=cmp9007,pw=tk) update(l=124,q) .ife,docompile,docomp. ftn5(ansi,et,i,lo=m/r,md,opt=2) attach(newlib,$bp.lib$,#id=sysid,sn=cmp9007,pw=tk) ldset(lib=newlib,compres=all,err=all,map=b,preset=nginf) load(lgo) nogo(abs) return(lgo,newlib) .endif,docomp. begin,rbarotr,compile,pl=5000,#id=id,#runno=runno,#sysid=sysid. return(abs) *EOR *IDENT GETDATA *COMPILE RBAROTR,TSTBAROTR *COMPILE BAROTR3 *COMPILE MUTTOP *DECLARE MUTTOP *i muttop.24 C C Zero the first 16 elements of the topography. Do 3458 il = 1, 16 phis(il) = (0.0, 0.0) 3458 continue *DECLARE,, *IDENT -INDATA */ Ident -indata. Replace the current input data for Barotr. *COMPILE TSTBAROTR *DECLARE TSTBAROTR *d 20-dynarr.237,42-files.223 / irarry (default 15). maximum ir value, to be used to set up arrays. / nlevel (default 1). The number of model levels. / nlinzd (default 1). Linearization flag. 1 for nonlin, 0 for linearized. 120, , , , 14, / ktotal, idelt, ir, imp, istart, iend. , 2, 24, , , / ibal, neuler, kountp, ilat, ilong, iglobe. 192, 24, , , , / ireset, idumpt, ibalw, kostar, kostop, kntrs. 1, 1.0, , , , , / toptyp,topfct, ap0(1:nlevel), Rgas,zerodg, alpha, Fpv(1:nlevel). 24, 24, 120/ kpdiag, kavno, kavto. , , , / vsctyp, fvsc(1:3,1:nlevel). Viscosity parameters. , , , , / frctyp, frcfmt, frcprm(1:3,1:nlevel). Forcing parameters. / Title (character*80). , , , , , / uin(40), uimn(41), uitpg(42), uifrc(43), uieng(40), uiavz(40). , , , , , / uout(30), uomn(31), uotpg(32), uofrc(33), uoeng(30), uoavz(30). *DECLARE,,
Back to top
Copy of procedure rbarotr
(Some of the indentation may be lost.)
.proc,rbarotr,test=f/test,name=barotr,sysname=bp,mods=0,ncar=f/t,tek=f/t,id=cmpxrb,sn=cmp9007,pl=5000,upin=#data,m77=/$m77$,static=f/t,oldname=0/$barotr.data$,newsubname=. ife,.not.file(compile,as),doupd1. attach(oldpl,$sysname.#pl$,#id=id,#sn=sn) update(i=upin,l=124,q) endif,doupd1. attach(in,$oldname$,#id=id,#sn=sn) exit,u. rewind(in) ife,$_test_$.ne.$_f_$,dotest1. ife,.not.file(abs,as),doabs1. attach(abs,$name.test_mods.abs_m77$,#id=id,#sn=sn) endif,doabs1. ife,ncar,doncart2. request(tape8,*pf,#sn=sn) endif,doncart2. request(iout,*pf,#sn=sn) ife,$m77$.eq.$_#m77$.or.static,dom771. abs(,,compile,#pl=pl) else,dom771. abs(*#pl=pl,i=compile) endif,dom771. catalog(iout,$name.test_mods_newsubname.data$,#id=id,rp=999,#sn=sn) ife,ncar,doncart3. catalog(tape8,$name.test_mods.metacode$,#id=id,#sn=sn,rp=999) ife,tek,dotekt1. setname(cmp9007) tektran(d=0,chtyp=1,pf=$name.test_mods.#tek$,rp=9) setname. endif,dotekt1. endif,doncart3. else,dotest1. ife,.not.file(abs,as),doabs2. attach(abs,$name.abs_m77$,#id=id,#sn=sn) endif,doabs2. ife,ncar,doncar2. kill,$name.#test_mods.metacode$,#id=id,#sn=sn. kill,$name.#test_mods.#tek$,#id=id,#sn=sn. request(tape8,*pf,#sn=sn) endif,doncar2. kill,$name.#test_mods.data$,#id=id,#sn=sn. request(iout,*pf,#sn=sn) ife,$m77$.eq.$_#m77$.or.static,dom772. abs(,,compile,#pl=pl) else,dom772. abs(*#pl=pl,i=compile) endif,dom772. catalog(iout,$name.test_mods_newsubname.data$,#id=id,rp=999,#sn=sn) ife,ncar,doncar4. catalog(tape8,$name.metacode$,#id=id,#sn=sn,rp=999) ife,tek,dotek1. setname(cmp9007) tektran(d=0,chtyp=1,pf=$name.#tek$,rp=9) setname. endif,dotek1. endif,doncar4. endif,dotest1. endif,end1.
Finale
To the novice, the SCOPE 2 (and other similar CDC) operating systems provided a simple job control syntax. This should be compared with the IBM OS/360 operating system at the time. Below is a sample extract from a job to be run on an IBM mainframe.
//T11CBHLM JOB (M11,CB,D11B),BELL.2479,TIME=(,20),PRTY=8 //*MAIN SYSTEM=P81 //COMP EXEC FORTVCLG,FVPOPT=2,FVREGN=1500K //FORT.SYSIN DD * C PROGRAM HELM (INPUT,OUTPUT,TAPE60=INPUT) COMMON U(75,75), F(75,75), A(75), B(75) CHARACTER*4 NO, NULL ... 1000 STOP END /* //LKED.CPUUSE DD DSN=M11.D11B.OBJ,DISP=SHR //LKED.SYSIN DD * INCLUDE CPUUSE(ZMCPUUSE) /* //GO.SYSIN DD * 75 75 150 0.0 1.E-4 0.0 1.0 1.0 /* // //
The cryptic nature of this still astounds me!