• reads a list of sites to be polled.
• connects to each site.
• executes a BLAST transfer command file to transfer files.
• disconnects.
• scans the Log File to determine which transfers were successful.
• builds retry files as required.
• adds the results to a status file.

Autopoll checks carefully for errors while polling. If an error is detected, the problem site is scheduled to be retried. Only the file transfer commands that failed are attempted again.

Installing Autopoll

Autopoll consists of ten scripts that were copied into your Blast\Scripts\Autopoll directory when Data Pump was installed in your system. The scripts are:

Autopoll.scr	master script
Autoinit.scr	initializes variables and files
Autoierr.scr	reports initialization errors
Autodisp.scr	draws screen displays
Autoline.scr	reads site information
Autopsnd.scr	checks log for status of SENDs
Autoprcv.scr	checks log for status of GETs
Autosw.scr	strips switches off @filename
Autoparx.scr	updates status files
Autosum.scr	displays session summary

In addition to these script files, you will need a new phonebook for specifying the sites you will be polling. The phonebook can have any appropriate name, but it must contain all the listings that will pertain to a single polling session--Autopoll cannot process listings from more than one phonebook. In the new phonebook, create a listing called "Autopoll." This listing loads when the Autopoll script begins and serves as a default listing for the polling session. For convenience in starting the Autopoll script, the Autopoll listing should include the entry "Autopoll\Autopoll.scr" in the Script File field as shown in the figure below.

Create additional listings in the new phonebook as necessary to specify the sites in your polling network. See The Site File for a discussion of a method that allows more than one site to share the same listing if the only difference between the sites is the phone number.

The last step in installing Autopoll is to create a directory to hold control files, log files, and other files that support a polling session. Autopoll's control files include a site file, which specifies the order of polling as well as other information, and one or more transfer command files ( TCFs) that specify file transfer actions. The support directory can be created in any convenient path in your system.

Starting Autopoll

Autopoll must be started from the directory in which the support files (site and transfer command files) are located. If Autopoll.scr with its relative path within the Script directory is entered in the Script File field of the Autopoll phonebook listing, the format for invoking Autopoll is:

blast -pphonebook -lAutopoll max_cycles site_file [start_time] [TRACE]

If Autopoll.scr has not been entered in the Script File field of the Autopoll phonebook listing, the command line must include the script explicitly. If you are not starting Autopoll from within the Autopoll directory, you must include the Autopoll directory as the relative path in the command line:

blast -pphonebook -lAutopoll -sAutopoll\Autopoll max_cycles site_file [start_time] [TRACE]

The command line parameters have the following meaning:

Here are some example command lines:

blast -pStorepoll -lAutopoll 3 retail 10:45 blast -pDatastations -1Autopoll 2 daily TRACE

In the first example, a maximum of three attempts will be made to poll the sites listed in the file Retail.dat, starting at 10:45 am. Notice that the command line specifies just the stub "Retail" of the site filename Retail.dat. (Autopoll appends a variety of extensions to the filename stub to determine the names of special files.)

The second example illustrates the use of TRACE. A start time can be specified with TRACE.

The Site File

The site file is one of the control files that you must create before starting Autopoll. It is the "master list" of information about the sites to be polled. Each line in the site file holds the parameters needed to connect to and transfer files to and/or from a site. Each line, or site record, consists of five fields separated by exclamation marks ("!," also called "bangs"), in the form:

phonebook_listing!site_name!phone_number!baud_rate!TCF_name

where

Each line must contain four bangs. Any fields that are to be skipped must be indicated by consecutive bangs ("!!"). Blank lines and lines beginning with a pound sign ("#"), space, or tab will be skipped, so you may freely comment your site file. Lines may not exceed 100 characters in length.

Use a standard text editor to create the site file. Save the file in the Autopoll support directory with the extension ".dat." Be aware that some text editors (such as Windows Notepad) may automatically append a different filename extension to files that are saved. You may have to rename the file manually to give it the required ".dat" extension.

Examples:

....|....1....|....2....|....3....|....4....|....5 !Blaster!1(919)542-0939!! store06!!!!nightly.tcf NewYork!Albany!782-8311!19.2!ny.tcf

In the first example, no phonebook listing is specified, so Autopoll.pbl will be loaded. The site name will be "Blaster." The phone number will be 1(919)542-0939. The baud rate will be taken from the phonebook listing since the field is blank, and the transfer command file defaults to Autopoll.tcf.

In the second site record, the phonebook listing Store06.su will be loaded. The site name, phone number, and baud rate will default to the values given in Store06.su. The command file will be Nightly.tcf.

In the last site record, the phonebook listing NewYork.su will be loaded. The site name will be "Albany," the phone number will be 782-8311, the baud rate will be set to 19.2 kbps, and the command file will be Ny.tcf.

Transfer Command File (TCF)

Autopoll uses a standard BLAST Transfer Command File (TCF) to specify the files to be sent and received. A TCF is a text file that contains line-by-line instructions describing functions to be performed during a BLAST protocol transfer. Any word processor or editor can create a TCF, but it must be saved in text only or ASCII format, under any name that you wish.

The TCF contains an unlimited number of filetransfer commands. Each command is entered as a single line with the source and destination specifiers separated by a space. Any desired transfer options are entered following the file specifier(s). Each line of text in the TCF must begin in the first column.

NOTE: Because a space separates the source and destination specifiers, filenames with spaces cannot be used--Data Pump will read the space in the filename as an indicator of the beginning of the destination specifiers.

Data Pump supports the following TCF command syntax for sending and receiving files:

Sending Files
No special character is required, just type the name of the local file to send and the name for the file on the remote system with a space in between. If no remote name is given, Data Pump will use the local name. Any options must be typed immediately following the filename:

localfilename[options] [remotefilename[options]]

Getting Files
The first character in the line must be a plus sign (+). Immediately following the "+," enter the name of the file to be received from the remote system and immediately after that any transfer options. If a different name is desired for the local file, type a single space after the remote filename and then type the local filename with any options immediately following:

+remotename[options] [localname[options]]

Please note that it is more efficient to put all GETs (lines beginning with "+") first, so that the remote file requests queue up on the remote. This allows for true bi-directional transfer during command file operations.

You may use a unique TCF for each site listed in your site file, or you may use one TCF for all sites. Autopoll treats wildcards or templates transfers as "try once" specifications. These transfers are attempted during the first cycle only. Even if errors occur, Autopoll does not attempt the transfers again.

Configuration Example

Suppose you are setting up a polling network with a central host machine and two UNIX sites. The sites are configured as follows:

Site Name	Phone	Login, Password
1. Sam's Discount Mart	542-0307	buz, apollo11
2. Metro Army Surplus	542-5694	neil, saturn5

Because the logins are different, different phonebook listings are created for each site ("sam" and "metro"). The site file for the polling network, Retail.dat, then becomes:

....|....1....|....2....|....3....|....4....|....5....    Retail Site List for My Polling Network sam!Sam's Discount!542-0307!!sams.tcf metro!Metro Army Surplus!542-5694!!metro.tcf

The first line of the file is treated as a comment, since it begins with a space. The last two lines are the actual site records. Actually, the site records may be duplicating information already in the listings since a listing contains the field for the Phone Number. If so, the site records could be simplified:

....|....1....|....2....|....3....|....4....|....5....    Retail Site List for My Polling Network    (Phone number from phonebook listing) sam!Sam's Discount!!sams.tcf metro!Metro Army Surplus!!metro.tcf

The site file now has an additional comment line and four lines altogether; otherwise it is equivalent to the previous site list.

According to the site list, a command file called "Sams.tcf" will be executed when Autopoll connects to Sam's Discount Mart. The command file Metro.tcf will be executed when Autopoll connects to Metro Army Surplus. Suppose you needed to get two files from Sam and send him one. The TCF Sams.tcf might look like this:

....|....1....|....2....|....3....|....4....|....5.... +/usr/buz/acq12.txt C:\tmp\client\sam1 +/usr/buz/wk_82/usr/sam2 C:\tmp\client\buz1/OVW C:\tmp\clients\readme readme/OVW

The "+" sign in column 1 of the first two lines signifies that Data Pump will perform a GET. Thus, Data Pump will get /usr/buz/acq12.txt and /usr/buz/wk_82 from Sam. The files will be called "C:\tmp\client\sam1" and "C:\tmp\client\buz1" respectively on the local system. The last line of the TCF signifies a SEND because it does not begin with "+." Thus, Data Pump will send \tmp\clients\readme to the remote system. The /OVW switch signifies that Data Pump may overwrite an existing file of the same name on the remote system.

Luckily, Metro.tcf is similar to Sams.tcf:

....|....1....|....2....|....3....|....4....|....5.... +/usr/neil/acq12.txt C:\tmp\client1\metro1 +/usr/neil/wk_82 /usr/client/metro2

These files--Retail.dat, Sams.tcf, and Metro.tcf--are created using a standard text editor (i.e., Notepad) and saved in the support directory. With the required files ready, the Data Pump command line to start Autopoll could then be:

blast -pdefault -1autopoll 3 retail

which specifies a maximum of three attempts to complete the polling session with Retail.dat. running the Autopoll script using the Autopoll phonebook listing in the default phonebook a maximum of three times.

Other Files That Use the Site Filename Stub

Autopoll distinguishes several special files by appending different extensions to the site filename stub. The extensions are shown below.

Extension	Created by	Meaning	Example*
dat	user (required)	site file	Retail.dat
stp	user (optional)	stop file	Retail.stp
hdr	user (optional)	banner file	Retail.hdr
log	Autopoll	short summary file	Retail.log
prn	Autopoll	long summary file	Retail.prn
cap	Autopoll	capture file	Retail.cap

*where the site filename stub is "Retail."

Site File.
The site file (e.g., Retail.dat) is the master list of information about the sites to be polled. The format of the site file is discussed under The Site File.

Stop File.
The stop file (e.g., Retail.stp) is used to exit prematurely but gracefully from a polling session. Autopoll checks for the existence of the stop file in the Autopoll support directory before connecting to every site. If the file is found, the polling session is terminated. The contents of the file are irrelevant. Thus, the file may be created by any application. Autopoll deletes the stop file before exiting.

Banner File.
The banner file (e.g., Retail.hdr) is an optional file that Autopoll will print before the summary file is printed at the end of polling. Printing is performed by the BLASTscript LPRINT statement.

Summary Files.
The short summary file (e.g., Retail.log) is printed at the end of polling. The long summary file (Retail.prn) is prepared by Autopoll but not printed. The file is saved in the Autopoll support directory. More information about the summary files is given below.

Capture File.
The capture file is created when tracing is requested. Autopoll uses a standard BLAST capture file, which will contain the complete incoming serial stream (BLAST protocol packets excepted). A tag is inserted into the file to simplify searching for the captured data for a particular cycle/site. This feature is primarily used for troubleshooting.

Long and Short Summaries

The Autopoll status files summarize the activities of a polling session. Two summary files are generated. The short summary, which is printed at the end of polling, is most helpful when polling goes well, since a quick glance is all it takes to see the happy outcome of the polling session. The long summary is helpful for troubleshooting.

A typical short summary file looks like this:

********************   02/09/98   11:15:29   ********************
Cycle 1
    1.  FAILED:  Sam's Discount < Error transferring 3 file(s). >
    2.  success: Metro Army Surplus
 
Cycle 2
    1.  success: Sam's Discount
*****************************************************************
Note: check Retail.prn for complete session information.

A long summary file looks like this:

02/09/98  **************************************************
11:15:33  * Cycle: 1    Site: 1
          *
          *  Name:  Sam's Discount
          * Phone:  542-0307
          *   TCF:  sams.tcf
          *   Log:  C1S001.log
          *
          *------------- SESSION INFORMATION ----------------
          * Filetransfer error -8: DCD lost during transfer
          * Error transferring 3 file(s). Log file follows:
          *
          * **** BLAST Professional Unix 10.7.3 on remote system [uov]
          * LOSS OF CARRIER, ending Filetransfer
          *   File transfer interrupted, 12% of file acq12.txt received
          ****************************************************
 
02/09/98  ****************************************************
11:16:30  * Cycle: 1    Site: 2
          *
          *  Name:  Metro Army Surplus
          * Phone:  542-5694
          *   TCF:  metro.tcf
          *   Log:  C1S002.log
          *
          *------------- SESSION INFORMATION -----------------
          * No errors encountered.
          * Log file has been deleted.
          ****************************************************
 
02/09/98  ****************************************************
11:18:49  * Cycle: 2    Site: 1
          *
          *  Name:  Sam's Discount
          * Phone:  542-0307
          *   TCF:  C1S001.tcf
          *   Log:  C2S001.log
          *
          *------------- SESSION INFORMATION -----------------
          * No errors encountered.
          * Log file has been deleted.
          ****************************************************
  
02/09/98  ****************************************************
11:20:41  * Polling complete: all sites polled successfully.
          ****************************************************

Tips and Tricks

Keep It Simple
Polling sessions can quickly become complicated if there are many remote sites and several file transfers that must be performed with each one. Create simple but sensible directory structures to support the polling network. As a rule of thumb, command files should contain lines no longer than 80 characters so they can be easily viewed and edited on standard terminals.

Problems Don't "Just Go Away"
In a large polling network, it is not uncommon to have a few remote sites that initially stumble. Especially frustrating are intermittent difficulties. Take some time to study problem sites carefully:

• Are the phone lines reliable?
• Can a phone line be dedicated to polling (at least during the polling session)?
• Are the modems compatible with each other?
• Is BLAST or BHost being initiated correctly on the remote?
• Are the expected files consistently present?
• Are directory and file permissions set appropriately?

Logon Timeout: 20
Inactivity Timeout: 20
DCD Loss Response: ABORT

These settings permit Autopoll to react more quickly to lost connections than the default settings allow. You may also wish to experiment with compression levels and packet size to find settings for best throughput.

Modifying Autopoll

Because Autopoll is written in BLAST's scripting language, it is very easy to modify. Feel free to customize Autopoll to your exact needs.

User-Supplied Script
The behavior of Autopoll can also be changed by writing one or more user-supplied scripts. Autopoll checks for the existence of these scripts at various points during execution. If it finds a user-supplied script, the script is CALLed. When the script returns, Autopoll tests the value of @STATUS. Polling continues normally if @STATUS equals "0"; otherwise the site is marked as failed.

User-supplied scripts reside in the Autopoll support directory. They are CALLed at the following places during execution:

Autousr0.scr	before the first site is polled
Autousr1.scr	before every attempt to CONNECT
Autousr2.scr	before every attempt to start FILETRANSFER
Autousr3.scr	before every attempt to DISCONNECT
Autousr4.scr	before Autopoll terminates

Because BLASTscript variables are global, a user-supplied script must be careful not to disturb the contents of any variables needed by Autopoll. The following variables may be changed freely by any user-supplied script:

@EFERROR
@STATUS
@input
@msg
@start
@temp


You can also create new variables if you wish. To prevent confusion, it is suggested that new variables begin with u, e.g., @uvar2.

File I/O with User-Supplied Scripts
Autopoll opens file handles 1-7 at various points during execution. The handles have the following functions:

1	read-only	current site (or retry) file
2		utility I/O
3		utility I/O
4		utility I/O
5	write-only	complete polling results
6	write-only	retry file for next cycle
7	write-only	brief polling results (printed out)

Any of the handles reserved for utility I/O may be opened by user-supplied scripts as long as the handles are freed before the scripts return to Autopoll (i.e., each user-supplied script must close its own files). User-supplied scripts may also write to the status files (handle 5 and handle 7) if they wish. An example of this technique is shown in the sample user-supplied script shown below.

Autopoll closes the log file before calling user-supplied scripts. If a user script opens a log of its own, the log must be closed before execution returns to Autopoll.

Note that if tracing is enabled, a user-supplied script cannot open a capture file since TRACE uses the BLAST capture buffer for its own use.

Sample User-Supplied Script
The following user-supplied script creates a command file on a remote UNIX system consisting of all the files in a given target directory. The file is retrieved for use by Autopoll in the normal way. This script provides a way to GET an unspecified number of files from the remote system without sacrificing Autopoll's ability to recover from file transfer errors.

# autousr2.scr # # Autopoll user-supplied script # # This script assumes that Autopoll is logged into a UNIX-based system! # # # The script creates a command file on the remote UNIX machine # consisting of all files in the given directory. The # command file is then retrieved so Autopoll can use it # in the normal way. # # To create the command file, awk processes the # output from ls. Since we will be getting each file in # the listing, a '+' is inserted at the front of each # filename. The /FWD switch is appended to the filename # so the file will be deleted from the remote system if the # transfer is successful. On the local (Autopoll) side, /OVW is # appended to the filename so the file will overwrite an existing # file. Assuming the target directory is /u/out and the command # file is x.tcf, the necessary command is: # # ls /u/out | awk '{printf("+/u/out/%s/FWD %s/OVW\n", $1, $1)}' > x.tcf # # The script returns the following error codes: # #      0 - no error #      1 - unable to detect remote shell prompt ($, %, or #) #      2 - target directory is empty #      3 - can't start BLAST on remote system # # set @upath = "/u/out"                  # adjust path to suit!   set @temp = "ls "   strcat @temp, @upath, " | "            # e.g, "ls /u/out | "   strcat @temp, "awk '{printf(\"+"       # attach awk command   strcat @temp, @upath   strcat @temp, "/%s/FWD %s/OVW\\n\", $1, $1)}' > "   strcat @temp, @worktcf                 # worktcf is Autopoll variable   tsend cr                               # get the attention of the remote   ttrap 2, "$", "#", "%"   if @STATUS = "0" return                # return error   wait 1 idle   tsend "ls ", @upath, " | wc -l", cr    # how many files?   ttrap 2, " 0"                          # no files in directory?   if @STATUS = "1"     fwrite 5, @b, " Remote directory ", @upath, " is empty."     write "remote directory ", @upath, " is empty!"     return 2                             # dir is empty, nothing to do!   end   write "preparing command file remotely"  # inform user   wait 1 idle tsend @temp, cr            # run the awk script   ttrap 2, "$", "#", "%"   if @STATUS = "0" return 1   filetransfer                           # now retrieve the command file   if @STATUS not = "0" return 3          # can't start BLAST on remote   get   @worktcf   @worktcf   to   esc   return @EFERROR                        # returns 0 if no error # # End of script.

Configuration Worksheets

The following "worksheets" may help you organize the large amount of information needed to set up a polling network successfully.

List the machines in your polling network. For completeness, include information for the central site as well.

Site	Name	Phone	Modem	Port	BLAST Ver.	Log In, Password
Central						X     X
1.
2.
3.

Decide whether different phonebook listings will be needed for each site. Remember, Autopoll loads the listing "Autopoll" by default.

Site	Name	Listing
1.
2.
3.

Set up the remote sites and test each connection with a simple script.

Name the site file. Build the entries in the site file with any standard text editor (e.g., Notepad), selecting appropriate name(s) for the TCFs.

Site Filename: __________.dat
Phonebook Listing	Name	Phone	Baud	TCF

Write the site file and save it in the Autopoll support directory.

List the files to be transferred with each site and the direction of transfer (S=send, G=get):

Site	S/G	Remote Name	Local Name	Options
1.
2.
3.

Write the various TCFs. Put them in the Autopoll support directory.

Decide how many cycles to allow for polling and when to start:

Cycles:           Start:

Build the command line to start Autopoll:

blast -pdefault -lautopoll -sautopoll\autopoll max_cycles site_file [start_time] [TRACE]