Content negotiating user agent extension
========================================

 v2.0, 28 June 1998

This package contains an implementation of the user-agent size of the
transparent content negotiation (TCN) specifications (RFC 2295 and RFC
2296, see (http://gewis.win.tue.nl/~koen/conneg/ for html versions of
these specifications).

It implements the example local variant selection algorithm in
appendix 19 of RFC 2295, and includes a dynamic accept header
generator which follows the method suggested in section 4.2.3 of RFC
2296.

The package is useful as an example implementation of the above RFCs
and can also be used for testing or debugging TCN implementations in
CGI scripts or servers.

Introduction
------------

The conneg-uax program is a HTTP/1.0 user agent which supports
transparent content negotiation.  It can be run from the command
line, but it is mainly intended to work as a wrapper around a normal
HTTP/1.0 user agent:

    -----------------------------
   |Transparently negotiating    |
   |user agent	                 |
   |                             |
   |   ----------------------    |
   |  | Normal user agent    |   |
   |   ----------------------    |
   |         ^                   |
   |         | HTTP/1.0          |
   |         V                   |
   |   ----------------------    |    
   |  | conneg-uax           |<------- preferences/capabilities database
   |   ----------------------    |     (capdb file)
   |         ^                   |
    ---------|-------------------
             |
             | HTTP/1.0 + transparent content negotiation
             |
             V
       Proxy caches, if present
             ^
             |
             | 
             V
       Origin server, maybe with transparently negotiable resources

            
Seen from the normal user agent, conneg-uax acts as a HTTP/1.0 proxy.

Seen from the origin server, conneg-uax looks like a monolithic user
agent system capable of transparent content negotiation.

From the user viewpoint, conneg-uax only becomes visible if a
transparently negotiated resource is accessed.  In this case, if the
response is a text/html page, conneg-uax will add diagnostic HTML text
to the bottom of the page, like this:

 <end of normal HTML information>
 ==========================================================================
 List response, chosen variant is: stats.tables.html. [pick other variant] 
 --------------------------------------------------------------------------
 Local variant selection algorithm transcript:
 q=1.000 for {"stats.tables.html" 1.0 {type text/html} {features tables}}
 q=0.800 for {"stats.html" 0.8 {type text/html}}
 q=0.760 for {"stats.ps" 0.95 {type application/postscript}}
 Choice: stats.tables.html


Typical use
-----------

A user agent extended with conneg-uax is typically used for
transparent content negotiation experiments, but it can also access
normal, un-negotiated resources.

Besides adding diagnostic html text, conneg-uax itself also prints
diagnostic information to the standard output, which is useful when
graphical content is being negotiated.

While conneg-uax is running, the file capdb, which contains the
capabilities and preferences database, can be edited to try out the
results of different user agent profiles.  The file is will be read
whenever a transparenntly negotiated resource is accessed.  Common use
is

  1. access a negotiated resource with the user agent
  2. view the result
  3. edit and save the capdb file
  4. press reload in the user agent and view the new result
  5. goto 3.

See the comments in the capdb file for information on the file format.

Use of conneg-uax
-----------------

   conneg-uax  : starts a transparently negotiating user agent extension
                 as a proxy on port 4444

   conneg-uax <URL> : retrieves URL with transparent content
                      negotiation, prints result to stdout.

Some parameters at the start of the conneg-uax file can be edited to
change the standard behavior.

Dynamic generation of Accept headers
------------------------------------

conneg-uax uses the database file ./serverprofiledb to cache profiles
of the type of negotiable content, for servers which have
transparently negotiable resources on them.  These profiles are used
in the construction of short Accept headers which can be sent to these
servers so that a server-side remote variant selection algorithm, if
present, might be able to choose the best variant and return it in a
choice response.

It is possible to delete the ./serverprofiledb file while conneg-uax
is running: this will have the effect of clearing the server profile
database.  This can be useful if one wants to study the performance of
contacting a `previously unknown' server.

For documentation on the ./serverprofiledb file format and the
heuristics used for building Accept headers, see the lvsa.awk file.

Use of lvsa.awk
------------------

Though normally called by conneg-uax, lvsa.awk can also be used as
a stand-alone program, see the comments at the start of the
lvsa.awk file.

Limitations
-----------

conneg-uax can only relay GET requests from the user agent, no HEAD
and POST requests.

The implementation of quoted-string handling for feature tag values is
incomplete: % HEX HEX encodings do not get decoded before comparison.
Also, conneg-uax will delete any ' character from the strings it
receives in variant lists before further processing.

Quoted-string parsing follows the HTTP/1.0 rules in that the next "
terminates a quoted string, and that the \ is not interpreted as an
escape character.

conneg-uax makes no attempt to disable any internal cache present in
the user agent.  Such an internal cache may cause intransparencies in
the negotiation process.

The heuristics in the dynamic Accept header generator can probably be
improved.  Also, the ./serverprofiledb database format is not very
compact.

Very little syntax checking is done on HTTP headers and on the capbd
database.  Headers with an incorrect syntax will usually be processed
(in some fashion) without much error reporting.

conneg-uax has only been tested as a wrapper for two versions of
Netscape and for the libwww-perl `get' program.

When adding HTML text to the end of a response, conneg-uax will
usually break some HTML-DTD structure rules.

conneg-uax does little network exception checking.  If I understand my
perl reference material correctly, it is theoretically possible that
certain network-related error events (like the user aborting the
transfer by pressing `stop' in the user agent at a bad point in time)
can cause a core dump.

Security
--------

To minimize the security risk due to the lack of error checking,
conneg-uax only accepts connections from the localhost interface
(127.0.0.1) in its standard configuration.

conneg-uax (or rather the libwww-perl routine it uses) adds a From
header to any outgoing request; this may be a privacy problem.

Included CGI scripts
--------------------

The CGI scripts included in the cgi/ directory implement a
transparently negotiable resource and its variants.  They provide a
very minimal, but compliant, implementation.  This negotiable
resource is also made available at

   http://gewis:81/conneg-bin/stats


Updates
-------

If this package is updated, the new version will appear under
http://gewis.win.tue.nl/~koen/conneg/.

Please send bug reports and questions/comments about this software
directly to me, and not to the http-wg mailing list.

Koen Holtman (koen@win.tue.nl)