     [file version.texi not included]

This file documents sic.

Copyright (C) 2000 Gary V. Vaughan

Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.

Permission is granted to copy and distribute translations of this manual into another language, under the above conditions for modified versions, except that this permission notice may be stated in a translation approved by the Foundation.

Dynamic Modular Interpreter Prototyping
***************************************

This is Sic.

Introduction
************

Sic Is Cumulative.

Syntax
******

The key to writing syntax extensions for sic is understanding how the syntax analyser tokenises its input.  As each line of the input stream is examined, each character is looked up in the syntax table -- if there is a handler function registered for that character, the handler is called, otherwise the character is simply copied from the input buffer to the output buffer.

Without any handlers, the tokeniser will return a single token consisting of the entire input line.  Handlers can be associated with any character or characters that might arrive in the input buffer:

 -- Function Pointer: int SyntaxHandler (Sic *SIC, BufferIn *IN, BufferOut *OUT)
     Syntax handler functions must be of this type in order to be registerd in the syntax table.  Function of this type have access to both the input buffer, IN, and the output buffer, OUT as well as the parent interpreter state data, SIC.

     Each of the buffers has LIM bytes of memory allocated to the BUF.START field.  The BUF.I field of IN is the offset to the character which triggered the handler, and the BUF.I field of OUT is the offset to the next unused character.

     A simple null handler would have the following body:

          out->buf.start[out->buf.i] = in->buf.start[in->buf.i];

     This copies the current character into the output buffer.  The calling loop takes care of incrementing the offsets for you before dispatching the next character.

     A handler can manipulate OUT in whatever way it pleases, but the goal is to populate the OFFV field with an array of OFFC integers which are the offsets at which to start each token of the generated char * vector that will be passed to the builtins.

     The macro returned by the handler affects what the process does with the rest of the input:

     SIC_CONTINUE
          Continue processing the characters remaining (i.e. beyond out->buf.i offset) in the input buffer, copying and calling handlers as appropriate.

     SIC_INCOMPLETE
          The input buffer is not a complete command.  Fetch another line from the input stream and add it to the end of the input buffer, before continuing to process the buffer.

     SIC_OKAY
          No more processing is necessary, convert OUT->OFFV to a char * vector and evaluate it.

 -- structure: struct BufferIn { Buffer BUF; char *START; }
     Use START to point to the first character of interest in the current token, when skipping leading whitespace for example.

 -- structure: struct BufferOut {Buffer BUF; int *OFFV; size_t OFFC; size_t ARGIND;}
     You must populate OFFV with the offsets into BUF of the start of each of the tokens to use when building the char * vector for evaluation.  OFFC is simply a count of the number of elements allocated to OFFV to prevent buffer overruns. You can use ARGIND to store interesting offsets -- say the first character after a run of whitespace for later addition to OFFV.

     BufferIn:  start     --------------.
                                        v
                             ,--------------------------.
                buf.start -> |f|o|o| | |b|a|r| |b|a|z|\0|
                             `--------------------------'
                                                ^
                buf.i     ----------------------'

     BufferOut: argind    -------------.
                                       v
                             ,---------------------------.
                buf.start -> |f|o|o|\0|b|a|r|\0|b| | | | |
                             `---------------------------'
                              ^        ^        ^
                         ,--. |        |        |
                offv   ->| ---'        |        |
                         |--|          |        |
                         | ------------'        |
                         `--'                   |
                                                |
                buf.i   ------------------------'

In the example above, the non-space characters have been copied into the output buffer, and strings of spaces have been replaced by ascii 0, so that when the char * vector is generated, each entry can point directly into the same block of memory.

 -- Function Pointer: int SyntaxInit (Sic *SIC)
     In addition to the processing you want to do in your syntax handlers, sic will call any number of initialisation functions that have been registered by the module loader before tokenising a line.

 -- Function Pointer: int SyntaxFinish (Sic *SIC, BufferIn *IN, BufferOut *OUT)
     Similarly any number of registered finishing functions will be called each time a complete line has been tokenised.