xenv -- history of user-visible changes. 2025-05-01
See the end of file for copying conditions.

Please send xenv bug reports to <gray@gnu.org>

Version 4.2, 2025-05-01

* New option: -o FILE

Directs output to the named FILE, instead of the standard output.

* Options -n and -m can be meaningfully used together

* Support for POSIX prefix and suffix removal constructs

   ${VAR#PATTERN}
   ${VAR##PATTERN}
   ${VAR%PATTERN}
   ${VAR%%PATTERN}

* Support for pattern substitution

   ${VAR/PATTERN/SUBST}

Replace one or all occurrences of PATTERN in the expansion of VAR with
the string SUBST.  Both PATTERN and SUBST are subject to expansion.
By default, only the first occurrence of PATTERN is replaced.  If
PATTERN starts with /, all occurrences are replaced.  If it starts
with #, the match is replaced only if it occurs at the beginning of
the value.  If PATTERN starts with %, the match is replaced only if it
occurs at the end of the value.  To match a special character (/,
%, or #) literally, it should be escaped with a backslash.

* New directive: $$dnl

This directive causes all characters, up to and including the next
newline to be discarded.

* New directive: $$ignore

Discards all text up to and including the next $$end directive.

* New directive: $$evaldivert

This directive is similar to $$undivert, except that undiverted text
is passed to the scanner and preprocessed again.  This makes it
possible to delay text expansion until later.

* New directive: $$require

This directive takes a version number as its argument.  If the version
of xenv is greater than or equal to that version, the directive is
ignored.  Otherwise, a diagnostic message is printed and xenv
terminates with status code 78.

* New directive: $$defmacro

Defines a macro.  Syntax:

  $$defmacro NAME PARAM...
  TEXT
  $$end

NAME is the name identifying that macro, PARAM... is a list of formal
parameter names, and TEXT is an arbitrary text.  The text is read
verbatim and saved away in the macro definition.  A macro is
expanded whenever it name appears after a $$ prefix at the beginning
of a line (with optional whitespace at either side of the prefix).  It
can also be expanded explicitly using the $$expmacro directive.

* New directive: $$expmacro

Invokes a macro by its name.  Syntax:

  $$expmacro NAME ARG...

The effect is the same as when invoking macro by its name as 

  $$NAME ARG...

* Arguments to directives can be quoted

Arguments enclosed in a pair of single or double quotes can contain
whitespace characters.  Within such quoted arguments a backslash can
be used to escape the character that follows it.  Quoting parts of
an argument is allowed as well.  Adjacent quoted strings are
concatenated, as in shell, so that

   text='an unusually '"quoted "argument

is equivalent to

   'text=an unusually quoted argument'

* New option: -Wrpe

This instructs xenv to retain partially expanded constructs on output.

By default, when a syntax error occurs when expanding arguments to
complex constructs (e.g. $$range), entire faulty construct is omitted
from the output.  Enabling this feature instructs xenv to retain the
construct as is.

This may provide additional information about error context, helpful
in fixing the cause of the error.

* EX_DATAERR exit code

The program terminates with code 65 (EX_DATAERR), if it encountered
any syntax errors in the input, including, but not limited to, the
use of undefined variables when the -u option is in effect.

Version 4.1, 2023-02-23

* Diagnostic directives: $$warning and $$error

The $$warning directive emits warning message.  It does not alter exit
status in any way.

The $$error directive reports a fatal error and sets exit status to
65 (or any other, if supplied as argument).

After both directives, processing is resumed at the next line.

* $$exit

New directive $$exit causes immediate termination of the program.
Decimal exit code may be supplied as argument.

* New directive: $$eval

The text between $$eval and $$end is expanded and the resulting
expansion is scanned again, producing the actual output.  This makes
it possible to create variable names on the fly and obtain their values.
Useful in loops, e.g.:

  $$loop I 0 1 2 3 4 5 6 7
  $$  eval
  \$\$ ifset VAR_$I
  Expand \$VAR_$I;
  \$\$ endif
  $$ end
  $$end

* Bugfixes

** Fix closing the $$range loop.


Version 4.0, 2022-08-25

* Stash storage

Stash is a special kind of storage to hold temporary data.  It is
used to implement several preprocessor features: diversions,
conditionals and loops.  Data are written to the memory for as long
as their amount fits the memory buffer allotted for stash.  The size
of this buffer is controlled by the feature value "stashsize".  When
the buffer gets full, surplus data are written to a temporary
file.  Number of temporary files that can be open simultaneously is
controlled by the feature value "stashfiles".  This works as follows:
when a new stash file needs to be open and the number of already open
files hits the limit, the program will close the least recently used
stash file, thereby freeing a file descriptor for use in the new
stash.  The closed file will eventually be reopened when it is needed,
using the same process.

Default values are: for "stashsize" - the size of the system memory
page, and for "stashfiles" - 2/3 of the soft limit for open files.

* Sink streams

On systems where fopencookie(3) call is available, xenv uses special
"sink" streams for text that's going to be discarded.  This speeds up
the operation and spares open file descriptors.

* New directives: $$ifcom, $$ifncom

  $$ifcom COMMAND
  TEXT1
  $$else
  TEXT2
  $$endif

Runs COMMAND and substitutes TEXT1 if it exits with code 0 and TEXT2
otherwise.  $$incom is similar, but expands to TEXT1 if COMMAND exits
with a non-zero status and to TEXT2 otherwise.

COMMAND is passed to the shell verbatim, i.e. any variable an command
references in it are expanded by the shell and not by xenv (this is
similar to how command expansion behaves).

* Bugfixes

** $$set and $$unset ignored if evaluated in a false conditional branch

Version 3.2, 2022-08-13

* Conditional directives for boolean evaluation

Two new directives are provided: $$iftrue and $$iffalse:

  $$iftrue VAR
  text1
  $$else
  text2
  $$endif

If environment variable VAR evaluates to boolean true, this construct
expands to text1.  If VAR evaluates to boolean false or is unset, the
construct expands to text2.

$$iffalse works similarly, but with logics inverted.

A variable evaluates to true if its value is 1 and to false, if its
value is 0.  These defaults can be changed using the -Wbooleans
option.  For example, to consider 1, t, and true as true values and
0, f, and false as false ones:

  -Wbooleans=1/0,true/false,t/f

* Conditional directive in alternative variable expansion

The following now works:

  ${X:+
  $$ifset Y
  $$include a.inc
  $$else
  D
  $$endif
  }

I.e. xenv directives between ${ } are processed as usual.

* Diversions

Two new directives $$divert and $$undivert make it possible to stash
away portions of text and to generate output in a different order
than the input was read.

* Loops

Two directives are added to expand a portion of text several times.
The $$loop directive implements a "foreach" loop and the $$range
directive implements a "for" loop.

* Bugfixes

** Memory allocation error in stack handling

** $$include directive in false branch of a conditional

** Relative file name processing in $$include and $$sinclude


Version 3.1, 2022-04-01

* New feature-control options

** -Wno-escape

The '-Wno-escape' option disables special handling of backslash in
free context.  For example, with this option in effect, the variable
$HOME will be expanded in the following text

  The \$HOME always expands

The '-Wno-escape' option does not affect expansion of default values
in variable substitutions.  For example, in the text below, the backslash
prevents the 

  ${HOME:+"The variable \$HOME is set"}

** -Wminimal

This option disables all other features and enforces '$ENV' as environment
meta-variable.  In terms of xenv options, it is a shortcut for:

    -Wno-command -Wno-comment -Wno-directive -Wno-quote \
    -Wno-escape -e ENV

Version 3.0, 2022-02-09

* The -S option and $$sigil directive are withdrawn


Version 2.3, 2022-02-09

* Bugfix in scanner, 2022-02-09


Version 2.2, 2022-01-16

* Detailed error locations

Diagnostic messages include as much information about error location
as possible.  At most, two locations are printed, corresponding
to the start and end of the erroneous portion of text:

  FILE1:LINE1.COLUMN1-FILE2:LINE2.COLUMN2

If FILE1 and FILE2 is the same, the location is rendered as:

  FILE1:LINE1.COLUMN1-LINE2.COLUMN2

If LINE1 is the same as LINE2 as well, the location is rendered as:

  FILE1:LINE1.COLUMN1-COLUMN2

If end location is not applicable (e.g. at end of file), only the
start location is output:

  FILE:LINE.COLUMN

* Environment meta-variable

Environment meta-variable is an optional feature that makes variable
indirections more visible.  Use it if your input text can contain
considerable number of indirection characters (sigils) that can be
falsely identified as variable references.  The meta-variable is
defined using the -e option, e.g.:

  xenv -e ENV

Once you define it, variable reference syntax becomes $ENV{NAME},
instead of just $NAME.  Tests for variable that is unset or null are
modified accordingly.  Thus, e.g. $ENV{NAME:-text} substitutes
"text" if the variable NAME is unset or null and so on.  The construct
for verbatim quotations becomes $ENV[...], and comments become
$ENV{* ... *}.

* The use of the -S option and $$sigil directive is deprecated.

Both will be removed in future versions.  You are advised to
use -e option instead.

Currently, a warning message is issued each time a -S or $$sigil is
used.  To suppress the message, use the -Wsigil option.  To turn -S
and $$sigil off, use the -Wno-sigil option.

* New option -W controls various xenv features

The -W option can be used to disable certain xenv features.  There are
currently five features:

  command     - command expansion
  comment     - multi-line comments (${* ... *})
  directive   - preprocessor directives: $$set, $$ifset, and the like
  quote       - $[ ... ] constructs
  sigil       - the deprecated sigil-changing feature (-S and $$sigil)

All of them are enabled by default.  To disable a particular feature,
use the -Wno-FEATURE option.  For example, -Wno-command disables
command expansions.


Version 2.1, 2021-07-28

* File inclusion

The constructs

  $$source FILE
  $$include FILE
  $$sinclude FILE
  
cause the FILE to be included.  Unless FILE is an absolute file name,
it will be looked up in the include file path.

If the file cannot be found, $$source and $$include will fail with an
error, whereas $$sinclude will silently ignore that.

* New option -I DIR

Appends DIR to the include file path.


Version 2.0, 2021-06-23

* Command substitution

The $(...) construct has changed its meaning.  Starting from this
version it denotes command substitution, as in Bourne shell.  The
material between the parentheses is collected and executed via
'$SHELL -c'.  The construct is then replaced by output produced by
the command, with any trailing newlines removed.

Before being passed to the shell, the collected material undergoes
sigil replacement, if the current sigil has been changed (either by
the -S option or by the $$sigil statement).  During this process,
each occurrence of the current sigil character is replaced by '$'.
This process is disabled for portions of text enclosed in single
quotes.

Except for this, the command is passed to the shell verbatim.  This
means, in particular, that variable references in the command are
expanded by shell, rather than by xenv and, as a consequence,
that the options '-r' and '-u' have no effect on command substitutions.

Nested command substitutions are processed by the shell.

* The -t option

The '-t N' option sets the maximum time a command substitution is
allowed to run.  If the command runs longer than N seconds, it
is terminated and a diagnostic message to that effect is printed on
the standard error.

* Inline verbatim text syntax changed

Starting from this version, inline verbatim text is indicated by
$[...].

This change breaks backward compatibility.

* Expansions in $$set statement

In the statement

   $$set NAME "STRING"

STRING argument can occupy multple lines and is subject to variable
expansion and command substitution.

The form with single quotes:

   $$set NAME 'STRING'

suppresses both variable expansion and command substitution.  However,
STRING can still occupy multiple lines.

* Use of single and double quotes in variable expansions

The 'word' part in the following constructs can contain quoted parts,
which are treated exactly as in Bourne shell:

  ${variable:=word}
  ${variable:?word}
  ${variable:+word}
  ${variable:|word|word}

That is: the quotes are removed, the text enclosed in double quotes is
subject to both variable substitution and command expansion, whereas
the text between single quotes is reproduced as is.  Within double
quoted part, a backslash can be used to escape the sigil character,
double or single quote character and itself.


Version 1.2, 2021-06-05

* Changed semantics of the \ escape.

Backslash serves as escape character only in two cases: when it
is followed by a sigil or by another backslash character.

In particular, \ loses its special meaning in a $(...) verbatim block.

* New preprocessor statement: $$sigil

The $$sigil statement changes the sigil to the character supplied as
its argument.

* New preprocessor statement: $$verbatim

Any block of text between $$verbatim and $$end is reproduced
unchanged.  This is a block analogue of the inline $(...) statement.

* Preprocessor statements are affected by the -S option.

Changing sigil with the -S option affects preprocessor statements
as well.  E.g. if you run `xenv -S@', then `$$set' becomes `@@set',
and so on.

* New option: -v

Prints program version and license terms.


Version 1.1, 2021-02-11

New option -S allows the user to select the sigil (indirection
operator) to use instead of '$'.

Version 1.0, 2021-02-02

Initial release.

=========================================================================
Copyright information:

Copyright (C) 2021-2025 Sergey Poznyakoff

   Permission is granted to anyone to make or distribute verbatim copies
   of this document as received, in any medium, provided that the
   copyright notice and this permission notice are preserved,
   thus giving the recipient permission to redistribute in turn.

   Permission is granted to distribute modified versions
   of this document, or of portions of it,
   under the above conditions, provided also that they
   carry prominent notices stating who last changed them.

Local variables:
mode: outline
paragraph-separate: "[  ]*$"
eval: (add-hook 'write-file-hooks 'time-stamp)
time-stamp-start: "changes. "
time-stamp-format: "%:y-%02m-%02d"
time-stamp-end: "\n"
end:


