\font\sfviii cmss10 at8pt
\def\LaTeXsf{{\sf L\kern-.36em\raise.35ex\hbox{\sfviii A}\kern-.15em
    T\kern-.1667em\lower.7ex\hbox{E}\kern-.125emX}}


\title{\LaTeXbf2.09$\rightarrow$\LaTeXbf3}
Asked to speak about \LaTeX2.09$\rightarrow$\LaTeX3 I will
try to give a you a picture of the history, the current
state, and the future of the \LaTeX3 project.

\section{Historical Remarks}
Whenever the future is somewhat unpredictable it
seems wise to take a look into history -- to find out
what has already been achieved and what remains to be
tackled.

From the history of the \LaTeX3 project we
will first take a look at the growing bulk of syntax
descriptions and (partial) implementations that are
the results of three years' work.

\setbox0\vbox{\sf
\halign{\vrule~#~\hfil&\strut#\hfil~\vrule\cr
\noalign{\hrule}
\multispan2{\vrule\strut\hfil Milestones:\hfil\vrule }\cr
\multispan2{\vrule\strut\hfil Syntax and implementation\hfil\vrule}\cr
\noalign{\hrule}
1988&Some bug fixes send to Dr Lamport\cr
&Four page sketch of NFSS\cr
&\hfill(New Font Selection Scheme)\cr
1989&First implementation of NFSS\cr
1990&New tabular implementation by D Duchier\cr
&First attribute prototype (thrown
away)\cr
&First kernel prototype\cr
&First recovery/help prototype\cr
1991&Second kernel prototype\cr
&Sketches for style designer interface\cr
&Second description of the attribute
concept\cr
&Extended description of the help
facility\cr
&Syntax for extended NFSS\cr
&Third kernel prototype\cr
&Release of \LaTeXsf2.09 international with\cr
&\hfill  NFSS support\cr
\noalign{\hrule}}}

\smallskip
\hbox to\hsize{\hfil\copy0\hfil}
\smallskip
The \LaTeX3 project was initiated at the Stanford
annual meeting in 1989. But the first vague plans were
already formulated in 1988 when Rainer Sch\"opf and
I, after sending several pages of bug fixes for \LaTeX2.09
to Leslie Lamport, received a positive answer.

Given the original goals for a reimplementation
described in the Stanford paper, nearly everything
seems to have been achieved.
\bitem NFSS  provides a far more general font
selection. The extended syntax
also provides for font scaling (a prototype
implementation has been completed).
\bitem The mathematical capabilities 
of \LaTeX\ have reached the standard
of \AmSTeX\ with {\tt amstex.sty}.
\bitem  Tabular processing has reached very high
quality with the new tabular implementation by
Denys Duchier (superceding {\tt array.sty})
and valuable suggestions by several others.
\bitem The new help/recovery concepts give a safe
and easy-to-learn environment 
for novice users.
\bitem The partially finished concept for
specifying attributes to environments and
functions, provides a more flexible input language. This
also allows easy conversion from SGML to \LaTeX3 DTDs.

How did this happen?

\setbox0\vbox{\sf\halign{\vrule~#~\hfil&#~\strut\hfil\vrule\cr
\noalign{\hrule}
\multispan2{\vrule\strut\hfil Milestones:\hfil\vrule}\cr
\multispan2{\vrule\strut\hfil Meetings, Workshops and
Correspondence\hfil\vrule}\cr 
\noalign{\hrule}
1988&Non-flame answer from Dr. Lamport\cr
1989&Talk in Stanford\cr
&Meetings with Leslie in Stanford\cr
&Talks in Karlsruhe\cr
1990&Mega-bytes of email correspondence\cr
&Working week in Mainz with Leslie\cr
&Talk in Cork\cr
1991&More mega-bytes of email correspondence\cr
&Workshop in London\cr
&Meeting with Leslie and Chris Rowley\cr
&\hfill in London\cr
&Workshop in Dedham\cr
&Working week in Providence with\cr
&\hfill Chris and Michael Downes\cr
&Working week in Mainz with Chris\cr
\noalign{\hrule}}}

\smallskip
\hbox to\hsize{\hfil\copy0\hfil}
\smallskip
All this work has been carried out in the free time
of several individuals and involves, as you can see,
some enthusiasm to keep the project alive. So far,
more than thirty people have contributed in one
way or the other.

One of the major problems is to bring people
together to discuss the open questions and find new
solutions. This must also involve people outside the
project since we need the opinion and experience
of typesetters, publishers, etc.\ to eliminate the flaws
in the system and find new and better
solutions.\looseness-1

In this regard both the London and the Dedham
workshops have been a great success but further
workshops of this kind are definitely necessary to
provide \LaTeX3 with a suitable designer interface.

So why is it that we feel that this project is still 
at its beginning? Because we have learned that our
original goals have not yet touched the real problems as
we now see them.

\section{Change of focus}
We now feel that one does not gain much by providing
further specialized style files to solve this
or that special problem. Instead, we think that
the major effort in the future has to go into the
design of a suitable style interface to allow easy
implementation of various layouts. (Easy, of course,
is relative: easy compared to the complexity of the
task.)

This change of focus implies:
\bitem The development of a new internal language 
that is more suited to
express visual components of the layout process.
\bitem The development of high-level generic functions 
that allow expression of most
commonly used layout components in an easy way.
\bitem The development of a model for specifying
and modifying parameters that influence
the layout.

Since the syntax for this internal language is
still changing on a daily basis and generic functions
mostly depend on it, I like to concentrate here on
the model for parameter setting.

\medskip
{\offinterlineskip
\halign to\hsize{\sl#\hfil\tabskip0pt plus1fil
 &\tabskip0pt\vrule\quad\hfil\it#%
 &\quad\sf#\enspace\hfil\strut\vrule\cr
&\multispan2{\hrulefill}\cr
&i&document context\cr
&ii&section context\cr
Context&iii&heading context\cr
&iv&title context\cr
&v&number context\cr
&\multispan2{\hrulefill}\cr
}}
\smallskip

\noindent The `context' concept encapsulates our main idea for 
maintaining  parameters in the \LaTeX3 system.

On every point in the document we are in
some context that is given by the the nesting and
sequencing of entities processed so far.

The major idea of the new system is to allow
the specification of parameters within such contexts
in a very general way. For example, it is possible
to redefine the behavior of lists within footnotes
by specifying the values of list-parameters in the
context of `footnotes' differently from those applied
in the context of, say `floats'.

Parameter is meant in a very general way,
for example, the code that some entity runs is internally a
parameter, so that via this concept different generic
functions can be run in different contexts.
\smallskip
\setbox0\vbox{\sf\hsize0.9\hsize 
\hbox to\hsize{\hfil The concept of a context\strut\hfil}
\hbox to\hsize{\hfil Some observations\hfil}
\bitem The nesting of entities forms the major component 
for describing layout via contexts. 
\bitem The specification of layout by a sequence
of contexts is important.
\bitem The context of some entity in a document is
not simply given by nesting and sequencing
of surrounding entities.
\bitem The context of some entity has a logical and
a visual component. The visual component
depends on the formatting of other entities.\strut}

\smallskip
\hbox to\hsize{\sf \hfil\vbox{\halign{\vrule\enspace#\enspace\vrule\cr
\noalign{\hrule}
\copy0\cr
\noalign{\hrule}}}\hfil}
\smallskip

As a further example, the layout attributes for a
table entity in a float may be different than for
tables in the main text.

Sequencing is, for example, important in
heading$\rightarrow$heading situations where intermediate
spacing and penalties change if headings follow
directly after each other, in list$\rightarrow$text/par situations
and many other places.

Take, for example, the situation of some
footnote or float that appears in a list and itself
contains a list. Because of an improper handling
of contexts in the current \LaTeX\ the inner list is
typeset as a second level list. In other words an
entity must be able to (partially) forget about
its context, or more generally must be able to
manipulate its context.

The fourth point is of a theoretical nature. All of
the currently available formatters format document
entities in a predetermined visual context, that is, they
assume that the visual context can be determined
by the logical nesting and sequencing of entities. To
a certain extent \TeX\ is an exception as it applies
dynamic programming to the process of paragraph
formatting which involve recomputation of contexts
for ligatures etc.
As an example for the wrong visual context
consider a hyphen at the end of a page, that is
avoided by \TeX\ by moving the line instead of
recomputing the paragraph.

\setbox0\vbox{\sf 
\hsize0.9\hsize
\hbox to \hsize{\hfil The concept of a context\strut\hfil}
\hbox to \hsize{\hfil Some problems\hfil}
\bitem The user input is not at normalized
document -- it may contain hidden entities
inside of user-defined shorthands that can
not be prescanned easily.
\bitem The specification of contexts by sequencing
is important but partially restricted by the
underlying \TeX\ engine.
\bitem Taking the visual component of contexts
into account requires the use of a multi-pass
system.\strut}

\smallskip
\hbox to\hsize{\hfil\vbox{\halign{\vrule\enspace#\enspace\vrule\cr
\noalign{\hrule}
\copy0\cr
\noalign{\hrule}}}\hfil}
\smallskip

The problem here is that we do not deal with
normalized documents (where every entity is fully
tagged) and therefore can not scan for further begin
or end tags before we start typesetting. This means
that certain decisions have to be taken without
knowing what follows. The only solutions to this
problem are
\bitem dissallow the use of user defined shorthands

\bitem the use of a two-pass system that normalize
the document in the first pass

\bitem the use of a multi-pass system that use
sequencing information from the last run.

None of the solutions seem to be feasible for a
system that uses \TeX\ as the input language but
should be explored further.

Deferring  the typesetting process is generally
possible in \TeX's vertical mode where we can wait
for the next |\everypar| to regain control.

But \TeX\ has no built in mechanism to detect
whether plain character material after a given
point is about to be contributed to some horizontal
list. Only after material has been contributed to
the horizontal list can one deduce this fact by
`dirty tricks' with special kerns. But this can be
used only for interrupting a context sequence --
the contributed material can not be manipulated
further.

This will draw the boundary (beside processing
time) between the ideal model and the real world.
So let us now turn to the question of how this
context-model is placed within the \LaTeX\ system.

\section{The Structure of the \LaTeXsl3 System}
There are many interwoven structures of the
\LaTeX3 system that are worth talking about. In
the following I will show how the results already achieved
and the ideas about contexts fit together in
an extensible modular system.

The \LaTeX3 system will consist of a kernel
system that provides the basic data structures such
as lists, stacks, etc.\ to program higher modules. It
will also contain arithmetic functions for integers
and dimensions, for example, it will be possible to express
relationships between individual parameters by
specifying assignments that contain expressions.
\smallskip

{\hrule
\sf\centerline{\strut The Structure of the \LaTeXsf3 System}
\centerline{System}
\smallskip
{\font\sfbig cmss17\sf
\input l_pic
\input l_frame
\unitlength4mm
\picture(16,12)(-0.5,0)

\put(0,9){\framebox(4,3){\shortstack[l]{Help\\system}}}
\put(6,9){\framebox(4,3){\shortstack[l]{Generic-\\functions}}}
\put(12,9){\framebox(4,3){\shortstack[l]{Style-\\designer\\language}}}

\put(2,4){\framebox(6,4){\shortstack[l]{\sfbig Kernel\\\sfbig system}}}
\put(10,4.5){\framebox(6,3){\shortstack[l]{Parameter\\database}}}

\put(0,0){\dashbox(4,3){}}
\put(6,0){\dashbox(4,3){}}
\put(12,0){\dashbox(4,3){}}

\endpicture}
\smallskip
\hrule}
\smallskip
On top of the kernel are built manipulation functions for the
parameter database and generic functions.
They form the platform for the style designer
language.

One important component of the system will
be an interactive help system that allows extensive
help texts as well as the possibility to define system
reaction depending on user action. Help messages
and such additional error-correcting code will be
held in external files that are read in when an
error is detected by the system. In this way an
elaborate help and error correcting mechanism
will be available while keeping the \LaTeX3 kernel
compact.

We distinguish between document styles that
are written in the style designer language (and
will probably contain nearly no \TeX\ code in the
traditional sense) and additional modules that
provide entities for specialized documents. This will
include, for example, higher math and we hope that
we can provide  a programming interface with the new kernel,
one that makes the development of further
modules a possible task.

Let me close with this quotation from some
unknown novice of \LaTeX.\smallskip

\setbox0\vbox{\hsize0.9\hsize\sf\narrower\noindent
\strut Dear Sir,

\noindent
I have successfully installed \LaTeXsf\ from the
distribution -- the file {\tt sample} has just been printed.
However, somewhere in the {\tt readme} files
a similar program called \TeX\ is mentioned.
Could you please explain to me how to install
this program?\dots\smallskip}

\hbox to\hsize{\sf \hfil\vbox{\halign{\vrule\enspace#\enspace\vrule\cr
\noalign{\hrule}
\copy0\cr
\noalign{\hrule}}}\hfil}

\noindent
At first this might sound funny, but this extreme is
not so far from reality.

\bitem Today, there are not many users who have a
deep understanding of the underlying
system structure.

\bitem Today, the majority of users use \LaTeX\
only. They usually have no knowledge of
the \TeX book. This class of users can be
nicely classified as `has heard of {\sl macros},
but has never seen one'.

\TeX\ and \LaTeX\ as its major front-end has
to compete with the so called Desktop Publishing
systems. To keep them alive we have to bridge
the gap between the `implementor\slash wizard' type
of user of the '80s and the new type who uses
the system just as one tool out of many, without
understanding its internals.

With the \LaTeX3 project we hope to achieve
this goal -- as far as the front end is concerned.
\author{Frank Mittelbach}