NAME

docidx_fmt -
Specification of a simple Tcl Markup Language for Keyword Indices

SYNOPSIS

comment text
include filename
vset varname value
vset varname
lb 
rb 
index_begin text title
index_end 
key text
manpage file text
url url label

DESCRIPTION

This document specifies version 1 of a text format for keyword indices. The name of this format is docidx and it provides all the necessary commands to write a keyword index for a group of documents. It is intended to complement both the doctools format for writing documentation and the doctoc format for writing tables of contents. The formal specifications for these two formats can be found in the companion documents doctools_fmt and doctoc_fmt. A third companion document describes the package doctools::idx, which provides commands for the processing of text in docidx format.

Like for the formats doctools and doctoc a generic framework for the conversion of docidx to any number of different output formats is provided. This framework is provided by the package doctools::idx.

Anyone who wishes to write an index formatting engine which plugs into this framework has to read the document docidx_api. This is the formal specification of the API between the framework and its engines.

FORMAT SPECIFICATION

OVERVIEW

While doctools is similar to LaTeX docidx is only superficially so. Input written in this format consists of a series of markup commands, which may be separated by whitespace. Other text is not allowed. The best comparison would be to imagine a LaTeX document with all regular text removed.

The format used to mark commands is different from LaTeX however. All text between matching pairs of [ and ] is a command, possibly with arguments. Note that both brackets have to be on the same line for a command to be recognized.

GRAMMAR

The overall syntax of a keyword index is best captured in a formal context-free grammar. Our notation for the grammar is EBNF. Strings will stand for markup commands, however their arguments (if they have any) are not part of the grammar. Our grammar contains lexical elements as well.

First we specify the whitespace at the lexical level, which also includes comments.

    COMMENT  ::= "comment"
    WHITE    ::= { '\n' | '\t' | ' ' | '\r' | COMMENT }

Then we define rules for all the keywords. Here we introduce our knowledge that all commands can be separated by whitespace, and also that the inclusion of other files may happen essentially everywhere, like the definition of document variables. The content of any included file has to fit into the including file according to the location in the grammar the inclusion is at.

    BEGIN    ::= "index_begin" WHITE DEFUN
    END      ::= "index_end"   WHITE
    KEY      ::= "key"         WHITE DEFUN
    MANPAGE  ::= "manpage"     WHITE DEFUN
    URL      ::= "url"         WHITE DEFUN

    INCLUDE  ::= "include"     WHITE
    VSET     ::= "vset"        WHITE

    DEFUN    ::= { INCLUDE | VSET }

At last we can specify the whole index.

    INDEX    ::= DEFUN BEGIN CONTENT END

    CONTENT  ::= KEYWORD { KEYWORD }
    KEYWORD  ::= KEY REF { REF }
    REF      ::= MANPAGE | URL

COMMANDS

Here we specify the commands used in the grammar. Some commands specified here were not used in the grammar at all. The usage of these commands is confined to the arguments of other commands.

comment text

This command declares that the argument text is a comment.

include filename

This command loads the contents of the file filename for processing at its own place.

vset varname value

This form of the command sets the document variable varname to the specified value. It does not generate output. It is this form the grammar is refering to.

vset varname

This form of the command returns the value associated with the document variable varname. This form is not used by the grammar and restricted to usage in the arguments of other commands.

lb

This command adds a left bracket to the output. Its usage is restricted to the arguments of other commands.

rb

This command adds a right bracket to the output. Its usage is restricted to the arguments of other commands.

index_begin text title

This is the command to start a keyword index. The text argument provides a label for the whole group of documents the index refers to. Often this is the name of the package (or extension) the documents belong to. The title argument provides the overall title text for the index.

index_end

This is the command to close a keyword index.

key text

This command adds the keyword text to the index.

manpage file text

This command adds an element to the index which refers to a document. The document is specified through the symbolic name file. The text argument is used to label the reference.
The symbolic names are used to preserve the convertibility of this format to any output format. The actual name of the file will be inserted by the chosen formatting engine when converting the input. This will be based on a mapping from symbolic to actual names given to the engine.

url url label

This is the second command to add an element to the index. To refer to a document it is not using a symbolic name however, but a (possibly format-specific) url describing the exact location of the document indexed here.

NOTES

It is possible to generate a keyword index in docidx format automatically from a collection of documents in doctools format. All is needed is a special doctools formatting engine which extracts the relevant metadata and some glue code to convert this data to docidx.

The application dtplite is an example for this.

EXAMPLE

The example is a keyword index for all manpages in the module base64.

[index_begin tcllib/base64 {De- & Encoding}]
    [key base64]
        [manpage base64]
    [key encoding]
        [manpage base64]
        [manpage uuencode]
        [manpage yencode]
    [key uuencode]
        [manpage uuencode]
    [key yEnc]
        [manpage yencode]
    [key ydecode]
        [manpage yencode]
    [key yencode]
        [manpage yencode]
[index_end]

SEE ALSO

doctools::idx, docidx_api, doctools_fmt, doctoc_fmt

KEYWORDS

docidx, index, keyword index, keywords, documentation, markup, generic markup, TMML, HTML, nroff, LaTeX