The Meta-Environment

[Feb 26, 2007] Release Candidate 2.0-RC2

We have put the next release candidate of The Meta-Environment online. This includes:

• A full source release of The ASF+SDF Meta-Environment RC2
• A binary installer of The ASF+SDF Meta-Environment RC2 for Linux i386
• A full source release of SDF 2.4
• A binary installer for SDF 2.4

This release candidate is the next step towards a full release of Meta-Environment 2.0. It is still a release candidate, because there are a number of missing features (as compared to release 1.5.3), and development of ASF+SDF specifications in the studio is too slow (not optimized).

Note that if you are upgrading from the 1.5.x series, you should read the release notes of RC1 too. RC2 includes the following improvements with respect to RC1:

Documentation

• The online documentation at www.meta-environment.org now features the following introductory texts:
  • The Language Specification Formalism ASF+SDF
  • The Syntax Definition Formalism
  • Syntax Analysis
• And, we have the first flash based online guided tour:
  • Guided tour: Playing with Booleans
• Specific how-to documents have also been written:
  • ASF+SDF by Example
  • The SDF Disambiguation MedKit for Programming Languages
• Technical documents that are under construction:
  • The architecture of The Meta-Environment
  • The extension points of The Meta-Environment
• All source code API's are now online at www.meta-environment.org, and are documented using javadoc and doxygen incrementally.
• C code generated by ApiGen is now automatically documented with doxygen mark-up.

Portability and stability

• Many GCC 4 portability issues have been resolved. The Meta-Environment should compile easily on the newest distributions.
• There are still sometimes issues with newer version of flex, which generates unused code. If you run in to this problem, simply comment out the yyunparse function from lex.yy.c in the ToolBus package.

Meta-Environment

• Synchronization between several parallel usage scenarios is now implemented. This stabilizes the interface enough to work long hours with it.
• Graph drawing and graph updating coordination works smoothly now.
• New feature: module names are now used to automatically bind the user-interface with user-defined specifications. See the Pico definition in languages/pico as an example. Currently there are already a number of predefined categories: syntax, check, format, test, run, debug, extract. Each category comes with its own functionality, and has its predefined effect in the user-interface. Menu options are added automatically to terms over a language after each specification is loaded.
  • syntax: contains the SDF definition of a language only, used for parsing.
  • check: contains a rewriter from the language to a summary, intended for static (type) checking (see basic/Summary), updates the clickable error window at the bottom of the studio.
  • format: contains a (partial) rewriter from the language to Box for pretty printing (see languages/box/Box), activates the pretty printer and does an in-place replacement of the old text by the formatted text.
  • test: contains a rewriter from the language to a summary, intended to run unit tests, updates the clickable error window at the bottom of the studio.
  • run: contains an interpreter for the language, simply pops up an editor with the normal form.
  • debug: contains an interpreter for the language that connects with TIDE, starts tide and pops up an editor with the normal form when done.
  • extract: contains a fact extractor for the language, that rewrites the language to an RStore (see basic/RStores), currently does nothing, but will load the data into the fact browser in the studio for selecting visualizations.
• Undo works better in the editors
• Save-a-copy added to the file menu for editors (handy for reduct.out windows)
• Tooltips explain the meaning of 90% of the menu options in the Studio.
• The tree views in the studio (navigator, errors) try to keep the expansion state of the nodes even if the trees are updated with fresh information.
• Editors now have pop-up menu's that may depend on the current sort of the focus.

SDF

• New feature: non-transitive priorities. Normally the priority relation is transitively closed. For some languages, this is not the desired behavior. The . left of the > prevents a priority to be used in the transitive closure computation: A "+" A -> A .> A "*" A -> A.
• Associativity was transitively closed by accident in RC1. This bug has been fixed.
• Fixed several issues related to the `...` lifting symbol operator.
• The filterPT tool was resurrected from the 1.5.3 branch to the 2.0 branch.
• The syntax definition of C in the library was renamed to ansi-c.
• The ASF+SDF Meta-Environment contains a standard comment preserving formatter for SDF now. It may loose comments in unforseen places, but most locations where people usually write their comments are covered.

ASF

• Traversal function intensive applications will be 5% faster due to an optimization targeted at traversal functions. In some specific cases an enormous difference can be measured (if your traversal functions always have variables as first arguments on the left-hand side of its defining equations).
• More static checks are done on the proper use of lexical constructors and lexical variables
• New feature: "start constructors" can be used to rewrite the top node of a parse forest. If a symbol X is declared a start-symbol in SDF, you may rewrite the top now by matching: "start(X, ...)", where X is the literal symbol name, and "..." contains a matching pattern for something of sort X. This may also be used to acquire access to the whitespace and comments surrounding the start sort, for example: "start(CompilationUnit? , L1 Decl* L2)", where L1 and L2 are variables ranging over LAYOUT, and Decl* is a variable ranging over a start-symbol.
• The ASF compiler more aggressively optimizes the use of constant terms in equations. Depending on your specifications the gain may very from 0 to 10%.
• The old "=" syntax for positive conditions has dissappeared completely after having been deprecated for a while. Use ":=" for matching, and "==" for equality!
• The basic/Integers module was rewritten for efficiency, and more operators. It now includes division, max, min, gcd and some more.
• The "Natural" sort has dissappeared from basic/Integers. You are advised to let your old Naturals range over Integer now. NatCon? will not remove leading zeroes, Integer will.

ATerm

• ATerm Java now includes a new type: ATermLong. This is not yet portable to C.
• Serious work has been done cleaning up the ATerm library for the new GCC compilers.
• Peep into the future: we are working on good 64-bit support and improved/faster/streamable binary serialization of ATerms

BOX

• Pandora supports 'layout-to-box' productions, next to the standard 'to-box' productions. These can be used to write comment preserving pretty printers in ASF. See languages/pico/format/Pico for an example.
• I and WD boxes now accept more than one argument
• Added the G box. This box will divide its children into groups of fixed length, and wrap each group by a parameterized Box operator. This operator is specifically designed to make creation of tables easy, and also facilitates the pretty printing of separated lists.
• Added SL box, which is an abbrevation for G that can be applied to separated lists easily.
• Debugged the table operators A and R. They are much more stable now, and more robust against user errors.

ToolBus

• Bugfix: "true" and "false" are now typed with "bool" instead of "term". This makes equals on booleans work, and tools may send "true" or "false" in events and results.
• Several issues with dynamic process creation in combination with notes subscription and reception have been solved
• Fixed the viewer!
• Added right-biased choice +>

[May 16, 2006] Release Candidate 2.0-RC1

We have decided to put a Release Candidate of the full ASF+SDF Meta-Environment online: version 2.0-RC1

It has been a while since the previous release, and a lot of work has been done:

• many requests for enhancement and bugfixes have been implemented.
• a number of experimental features have been implemented
• some features have been optimized for speed or memory consumption
• and some legacy implementations have been replaced

This is a Release Candidate, because we think a lot more has to be done to finish version 2.0 as a product:

• some features are only half implemented (e.g. only by the ASF interpreter, not by the ASF compiler)
• some features are not optimized (too slow or too big)
• the documentation is out of date

Below you will find the only available documentation on what changed and what you need to do to get your specifications working. We ARE catching up on the documentation front.

BASIC INFORMATION

Website

• http://www.meta-environment.org

Help

• Problems, bugs and enhancements:
  • mailto:meta-support-list@cwi.nl
• General help (also by other users):
  • mailto:meta-users-list@cwi.nl
• Submitting issues directly:
  • bugzilla.sen.cwi.nl:8080/index.cgi

Required software

• The following software is necessary to compile and run 2.0RC1:
  • GNU gcc 3.4.x (the 4.x series contains bugs that we can not work around)
  • GNU make
  • Graphviz dot >= 1.12
  • Java JRE 1.5 (will not work with 1.4.x)

CHANGES and TIPS

Commandline

• The ASF+SDF Meta-Environment starts up with the command "asfsdf-meta", not with "meta" anymore.
• The SDF Meta-Environment (without ASF) can be started with: "sdf-meta".
• The dump scripts, pt-dump, eqs-dump, def-dump, etc. do not have options to save the "term-store" anymore. This feature has dissappeared and we are thinking about better ways to cache intermediate results.

User interface

• The third-party editor interface is gone. So no more emacs or vi.
• The editors are now implemented using a standard Swing implementation.
• All Meta-Environment GUI tools are implemented in Java/Swing and hosted in a dockable window environment, implemented by InfoNode? .com (GPL)
• Note that many menu options have keyboard shortcuts, and all menus have changed.
• Actions->Parse does not exist anymore, after saving an editor (CTRL+S), all necessary updates are done to the state of the system automatically. Any errors that may be the result of your changes are listed in the tab of the error viewer. (This is one of the experimental features that are not fast enough yet, sometimes after a save it takes a long time for the system to become idle again)
• The full state of the system is viewable from the 'Progress' viewer tab.
• All editors apply a generic syntax highlighting scheme, which you can influence in a number of ways. Contact meta-support-list@cwi.nl for questions.
• We use "prefuse", an open source graph visualization and animation package: www.prefuse.org. The graphviz layout algorithm is used, but the actual displaying is done by prefuse:
  • left click and drag: move a node
  • right click and drag up/down: zoom out/in
  • right click on canvas: zoom to fit
  • left click on node: select node
  • right click on node: open context popup menu
  • the "Graph" menu contains some other options
• Switch workspace: The "Workspace" is the directory where new modules will appear and existing modules are found. The File menu contains a switch workspace button that can be used at anytime.
• Open module: the file browser that is opened gives access to:
  • the Workspace
  • the SDF library
  • the ASF+SDF library You should not open files outside of these domains, the interface tries to limit this, but it is not hacker proof.
• Note that syntax definitions are now in the SDF library, while ASF+SDF data-structures and utilities are still in the ASF+SDF library.
• Library modules are read-only, and coloured grey in the import graph.
• The save termstore feature has dissappeared. We are thinking about a replacement for caching intermediate results between runs of the Meta-Environment.

Syntax Definition Formalism and SGLR

• The SDF library now comes with some new grammars:
  • C grammar
  • Parameterized C preprocessor grammar
• The syntax of SDF has been redesigned to use the same syntax for string literals everywhere. This also implies that unquoted string literal symbols are not literals anymore. You will get parse errors in your SDF modules. Before, the SDF checker would warn you about deprecated use of unquoted literals.
• The AsFix2ME? format has changed. It does not flatten lexical syntax anymore. Fully structured lexical trees are generated from now on.
• AsFix2ME? also does some subtle rearrangements of LAYOUT nodes. For details read the source code or contact meta-support-list@cwi.nl
• The SDF checker generates less false warnings on old issues, and some more false warnings for new issues.
• CASE-INSENSITIVE LITERALS are added as a feature to SDF2. The syntax is as follows:

  'begin' STAT* 'end' -> PROGRAM

  Defines a program with two case-insensitive literals begin and end. (use of single quotes instead of double quotes). The SDF normalizer generates for you:

     [bB][eE][gG][iI][nN] -> 'begin' 
     [eE][nN][dD]         -> 'end'

remember the case-sensitive notation is still:

"begin" STAT* "end" -> PROGRAM

which generates:

     [b][e][g][i][n] -> "begin" 
     [e][n][d]       -> "end"

• ci-lit() was added as a symbol constructor to AsFix2? [ME]
• CYCLIC GRAMMARS: SGLR now accepts cyclic grammars. A parse forest with a representation of a cycle will be generated instead of a parse error. The representation is NOT minimal. Contact meta-support-list@cwi.nl for details.
• AsFix2? [ME] is extended with a new Tree constructor, next to appl, amb and char we now have: cycle.
• The backend of SGLR was optimized for highly ambiguous languages. The use of memoization in several algorithms has improved the worst-case complexity from exponential to polynominal.
• SGLR does not provide a detailed ambiguity report anymore, use the renewed implementation of the tool "ambtracker".
• NOTE: we are currently reimplementing the API's around SGLR and remodularizing its implementation.
• Syntax productions are generated from priority sections like was done several versions back. BEWARE: we removed this then because the attributes of productions are merged if the rest of the production is the same. It might lead to unexpected shapes of productions in an AsFix2? parse forest. The benefit in grammars for C, Java and C++ is so big, that we decided to put it back in (several dozens of lines less in SDF modules with large expression grammars)
• PRIORITY FOR SELECTED ARGUMENTS. A priority can now be limited to be applicable in a certain argument only. This works for single productions as well as groups, and priorities are still transitively closed. Example:

     context-free priorities
     {  E "[" E "]" -> E
        E "bla"     -> E
     } <0> >
        E "*" E     -> E {left}

The first group has a higher priority than the second group, but only in the first (0) argument. Between the <..> brackets comma separated lists of argument numbers are accepted. We start counting at zero, and count EVERY MEMBER of the left hand side including literals, counting nested symbols as 1 member, and ignoring the implicit LAYOUT? non-terminal.

Algebraic Specification Formalism

• STRUCTURED LEXICALS: this is the new feature that breaks old ASF+SDF programs that made use of lexical constructor functions. This is a long story, here is the summary:
  • The CHAR sort is not to be used by you anymore. From now on you use 'lexical variables' as such defined in SDF2:

        lexical syntax
          [0-9]+ -> Number
          Number "." Number -> Real
        lexical variables
          "digit"[0-9]*  -> [0-9]
          "digits"[0-9]* -> [0-9]+
          "num"[0-9]*    -> Number
          "real"[0-9]*   -> Real

• • In other words, for lexical non-terminals and character classes do not define normal variables, but lexical variables.
  • Variables that range over SUBCLASSES may be used. If a variable does not range over a subclass of the expected character class, an error will be generated. Example:

        lexical variables
          "half"   -> [0-4]
          "more"   -> [0-9A-F]

The variable 'half' is acceptable everywhere [0-9] is also acceptible. The variable 'more' is not accepted at [0-9] locations.

• • Lexical constructors are now fully NESTED, because lexical syntax is not flattened to a list representation anymore, we now also need STRUCTURED LEXICAL CONSTRUCTORS.
  • For each production in lexical syntax a lexical constructor function is generated:
    • the prefix function name is the sort name in lowercase
    • each function has as many arguments as the production had members, including literals.
    • literals are to be typed literally
    • characters are escaped in the same manner as characters are escaped in the definitions of SDF2 character classes.
    • lexical variables are only accepted in the context of these generated constructor functions
    • example (see syntax and variables above):

                  equations
                  [zeros] number(0 digits)            = number(digits) 
                  [trunk] real(num . number(digits2)) = real(num . 0)
         

    • another example:

                  lexical syntax
                   [\"] ~[\"]* [\"] -> String
                  lexical variables
                   "char"[0-9]*  -> ~[\"]
                   "chars"[0-9]* -> ~[\"]* 
                  equations
                  [remove-nl] string(\" chars1 \n chars2 \") = 
                              string(\" chars1 chars2 \")
         

      * You can not construct any lexical that does not conform to the SDF definition of the lexical syntax anymore. Not even as an intermediate step. This sometimes requires some extra thinking. See the library modules basic/Identifiers and basic/Strings and basic/Bytes for examples on how to analyse or transform lexical syntax.
• (Traversal) Functions that returns lists, there used to be many problems with functions that returns lists, including traversal functions that visit list nodes: e.g. f(A) -> {B ","}* SDF, AsFix2? and ASF have been changed to fix these issues.
• REWRITING LAYOUT. Layout nodes are now first class citizens in ASF specifications. Each layout node in a syntactic pattern (i.e. left-hand sides or right-hand sides of conditions and equations) can now be in either one of these two modes:
  • IGNORED: the layout is ignored by a match, and put in literally by a constructing pattern. The syntax for this is just as it used to be: simply type the layout as it is defined and it will be ignored.
  • NOT IGNORED: the layout is matched and constructed according to your specification. The mode is activated when:
    • You use a lexical constructor function for layout
    • You use 'layout brackets'
  • Examples:

            lexical syntax
              [\ \t\n] -> LAYOUT
              "%%" ~[\n]* [\n] -> LAYOUT
            context-free syntax
              "_" "_" -> DUMMY {layout-bracket}
            lexical variables
              "L"[0-9]* -> LAYOUT
            equations
            [] if !E then S*1 else S*2 fi = if  E then S*2 else S*1 fi
            [] if layout(\n) E then S* fi = if layout(\ ) E then S* fi
            [] if L1 layout(\n) L2 E then S* fi = if L1 layout(\ ) L2 E then S* fi
            [] _ layout(\t) _ = _ layout(\ ) layout(\ ) _
       

  • The first equation ignores layout completely, like usual.
  • The second equation only matches an if statement if there is EXACTLY one newline between the if and the E.
  • The third equation matches an if statement with a newline SOMEWHERE between the if and the E.
  • The fourth equation matches ALL single tabs and translates them to two spaces.
• Condition syntax
  • The deprecated '=' condition operator has dissappeared
• Integrated unit tests * There are still unit tests in ASF specifications, example:

        equations
        [] myfunction(...) = myanswer(...)
        tests
        [] myfunction(xyz) == myanswer(abc)
     

• Rewriting with pos-info annotations and others
  • You can store and retrieve parse tree annotations using the modules utilities/Annotations, utilities/ATermAnnotations and utilities/PosInfo.
  • The ASF interpreter supports this by default
  • The ASF compiler has an extra commandline option to activate support for annotations in the generated code.
• New builtin functions:
  • See utilities/Parsing for calling the parser on files and strings
  • See utilities/Files for reading and writing files from ASF
  • The commandline versions of the interpreter and the compiler have a new option to pass a parse table in, for use with the parsing builtin
• Interface of compiled specifications
  • A compiled specification can now read source code directly, without using sglr first.
  • If you compiled a specification from the Meta-Environment directly, this is the default behavior: the binary will rewrite source-to-source.
• Wild and strict variables (not experimental anymore)
  • The experimental feature of wild and strict variables is consolidated.
  • example:

            variables
              "_x"[0-9]* -> Integer {wild}
              "x"[0-9]* -> Integer  {strict}
            

  • wild variables may only be used as 'wildcards'. No further use of a variable is allowed after using it in a match.
  • strict variables are used for information propagation. A strict variable MUST be used after it's introduction.
  • NOTE: the use of strict variables leads to less errors, it prohibits unintended loss of information.
  • NOTE: the use of wild variables also leads to less errors, it prohibits unintended failure of matches by use of the same variable names twice.
• Complete functions
  • An experimental feature: complete functions. You can annotate a function to be complete, which means that it may not appear in a normal form. Use this if you intend to implement a function that deals with all input correctly.
  • You will get a run-time error (interpreter only) if a complete function does not reduce.
  • Example:

            context-free syntax
               myfunction(Statement) -> Booleans {complete}
       

• Rewriting ambiguities
  • The rewriting of ambiguities implementation has changed recently, but not tested thoroughly yet. Use with caution.

[January 18, 2006] sub-release of Pandora

• A minor bugfix release of all components below Pandora was released to support a new version of the Risla flattener.
• Users of the 1.5.3 releases are NOT advised to update.

[December 17, 2004] Release 1.5.3:

A small release with some bugfixes and some polishing:

• Fixed a bug in obtaining a parse table when restoring brackets with a grammar without start symbols.
• Fixed implosion of alternative literals. "-" | "+" becomes alt(1,[]) or alt(2,[]) because a literal does not have a representation in an imploded tree. This fixes a core dump.
• Added ts spaceoption to Pandora and added generic modules for list prettyprinting in the library.
• Several small issues (DailyChangeLogs? )

[December 2, 2004] Release 1.5.2:

This is mainly a maintenance release, fixing a number of bugs. However, a number of important features have been added:

• A requirement changed: graphviz version 1.12 is needed (was 1.5 before).
• One requirement dissappeared: the libltdl library is now distributed with the MetaEnvironment? .
• MetaStudio? contains a new debugging tab. It contains the generic debugging framework TIDE? . Currently, TIDE? can be used to debug ASF? specifications. Each term editor contains a Debug menu to activate the debugger during rewriting.
• ASF+SDF variables declarations can be annotated with {wild} and {strict} to convey more of the intention of the programmer to the asf checker. This information is used to produce informative error messages and warnings.
  • A {wild} variable will only be used as a wildcard in a matching pattern, and not used anymore afterwards.
  • A {strict} variable is used to retrieve a value during a match, an is intended to be used afterwards.
  • A variable without annotations does not trigger any new warnings or errors.
• The efficiency of the syntax checker was improved drastically. This reduces parsetable generation times.
• Pandora has support for alignments.
• A new commandline tool: filterPT. It filters parse forests based on some command line arguments. Currently, we have one filter to find trees with a maximum or minumum number of occurrences of a certain non-terminal. It is used to implement filters for island grammars.
• A new small package: tide-support. Contains a library for developing new TIDE? debugging adapters.
• GVIM editor support has been improved. Mouse clicks are handled just like in the emacs editor, and the focus does not use the selection mode anymore.

[July 2, 2004] Release 1.5.1:

A small release with some bugfixes and some polishing:

• Traversal Functions are now fully implemented in the ASF compiler
• In parse trees the representation of variables was improved
• Several small issues (DailyChangeLogs? )

[May 10, 2004] Release 1.5:

We are pleased to announce the following release: ASF+SDF Meta-Environment 1.5

This release contains many radical improvements of many aspects of ASF, SDF and the Meta-Environment. There are new features, some old features have been replaced by new features, and some features have become deprecated without replacement.

Old ASF+SDF specifications can be upgraded automatically using the Upgrade menus in the environment. Detailed comments on the important changes can be found below at NEWS. You are advised to read them.

We would like to thank you for your valuable feedback on previous versions, especially the Beta testers of release 1.5.

Have fun!

The Meta-Environment development team.

http://www.cwi.nl/projects/MetaEnv

Contents of this announcement

• REQUIRENTS
• DOWNLOAD AND INSTALLATION
• PROVIDING FEEDBACK
• NEWS
  • Changes in the Meta-Environment
  • Changes in SDF
  • Changes in ASF

REQUIREMENTS

Installing the system requires the following software pre-installed on your system, and available in your PATH:

• GNU make
• GCC
• Java 1.4.2 SDK
• GNU Emacs (compiled for X11) or GVIM (with server-support)
• AT&T graphviz (version 1.5 or higher)

See www.gnu.org, www.vim.org, www.sun.com and www.graphviz.org.

NOTE: XEmacs is not supported anymore

DOWNLOAD AND INSTALLATION

An integrated source bundle can be downloaded from MetaEnvironmentDownload?

First make sure the required software is installed and available in your PATH.

The following commands will compile the system and install it in /opt/asfsdf-meta15:

$ gunzip -c asfsdf-meta-distribution-1.5.tar.gz | tar xvf -
$ cd asfsdf-meta-distribution-1.5
$ ./configure --prefix=/opt/asfsdf-meta15
$ make

PROVIDING FEEDBACK

Feedback can be submitted directly using Bugzilla: http://www.cwi.nl/projects/MetaEnv/bugs

Or send us an email: mailto:meta-feedback@cwi.nl

Please address questions to: mailto:meta-support@cwi.nl

NEWS

Changes in the Environment

• The memory footprint of the environment has decreased significantly!

• IMPORTANT: in the term editors the following disambiguation heuristics have been TURNED OFF:
  • injection counting
  • prefer/avoid counting

• The rationale is that these disambiguations silently disambiguate and therefore hide important problems of a syntax definition. They might also disambiguate in a 'context-sensitive' manner; choosing one alternative in some context, and another in a different context. Several examples of these effects have lead to the decision of turning them off. Notice that these heuristics are still online while parsing ASF or SDF. This will be remedied in future releases. Please contact us if you are stuck with this decision, we can help you. The new behavior in the editors is mimicked on the commandline by providing the -fi and -fe options to sglr:

  $ sglr -fi -fe -i <input> -o <output>

  So, the standard behaviour of sglr on the commandline has NOT changed.

• The XEmacs editors have been replaced by GNU Emacs for X11 editors, or GVIM editors. GNU Emacs is standard, GVIM can be activated using a commandline option of the 'meta' command. See 'meta -h' for help.

• Editors are faster and more stable. Suddenly killing an editor does not crash the environment anymore.
• There are three types of feedback:
  • Warnings and errors in the "Error" tab. Example: "undeclared variable"
  • System information and errors in the "Info" tab. Example: "file not found".
  • Status log indicating what the environment has done in the "Log" tab. These are copies of all message to the status bar. Example: "Opening module X" and "Done"

• Messages of type 1. dissappear automatically when the source of the error has been removed AND only after the source is rechecked for errors. Messages of type 2. and 3. message do not dissappear automatically. The "Clear History" button clears all messages in both tabs.

• All errors and warnings in the "Error" tab are clickable. An editor will be opened and the cursor will jump to the source of the error or the warning.

• TIP: if many warnings are produced about the use of deprecated functionality, consider using the automatic Upgrade functionality in ASF and SDF editors. The Upgrade tools should be used as an editing assistent, blindly applying them without reviewing the changes is possible (the changes are all semantics preserving) but not necessarily the best plan. You might find undiscovered bugs in your specifications. Do not simply ignore warnings, they indicate serious problems in your definitions in 90% of the cases.

• The parse tree viewer has been extended. Next to the graph display a "nested boxes" visualization has been added. "View Tree" shows the parse tree without the characters at the leafs and without layout nodes. "View Full Tree" shows all nodes and leafs.

• The removal of redundant static checking has lead to a significant speed-up for generating parse-tables and parsing equations for big specifications.

• Bugs related to loading big specifications are solved. Several tools contained minor but nasty pointer dereference bugs that break the ATerm garbage collector.

• A heuristics based pretty printer has been added called "Pandora". Use the Pretty Print option of the Actions menu in an editor. NOTE: the UNDO in an editor does not bring you back to before running the pretty printer. User-defined pretty-printing is not supported fully yet, but the basic foundations are there.

• The layout of the GUI has changed, many menus have been added or removed or changed. The goal was to improve modularity of the Java/Swing source, but several user-interface improvements have been made to.

• For each choice in the Export menu a file browser is started for selecting the target file.

• You can safe the state of the environment in a file (termstore) and later return to that state by loading the same file. Exception: all files on disk must not change w.r.t. to the saved state. If files are changed on disk, and old 'termstore' will be updated with the new files automatically. This updating with respect to the filesystem is obligatory. Use the caching feature to quickly start up where you left off earlier.

• Modules with circular imports can be closed now (if no other modules depend on them than the modules in the import cycle).

Changes in the SDF formalism and its implementation

• Declaring sorts is obliged for all simple sorts now. The sort declarations are used for (and only for) checking typos in sort names.

• Declaring sorts is forbidden for complex symbols (such as tuples, lists and optionals).

• A hidden declaration is now really hidden, just like the productions in a hidden section. Before the sdf checker would assume a sort that was declared hidden was still declared outside of the module.

• Sort declarations do not make a sort a 'start symbol' anymore. Before, all symbols in the sort declaration were automatically promoted to start symbols. This lead to unexpected ambiguities in a number of cases and to vague warnings and error messages by the SDF checker. A sort declaration is now a separate concept from a start symbol. Sort declaration declare which sort names are going to be used in a grammar, start symbols declare the top non-terminals of a grammar.

• A new grammar type: 'context-free start-symbols' can be used to declare a start symbol.

    Example:
      context-free start-symbols
        Compilation-unit 

• The Upgrade menu automatically promotes all sorts to context-free start-symbols. Usually you can remove a number of start-symbols after that. NOTE: a parse error at the beginning of a file probably means that you forgot to introduce a context-free start-symbol. Use the Upgrade menu in a syntax editor to promote all sorts to start-symbols (a transformation that mimicks the old semantics of the module, all sorts are start-symbols.)

• Several more static checks of SDF specifications have been added.

• Unquoted literals are deprecated. A warning will be printed. Use the Upgrade menu to automatically quote all unquoted literals.

• Tuple syntax A#B is replaced by <A,B> (Upgrade menu)

• Set and Permutation symbols have been removed.

• Restore-brackets tool has been improved.

• {reject} on a language bigger than a literal triggered a bug sometimes. This was fixed.

• Cycles are reported better by sglr and a bug concerning cycles has been solved.

Changes in the ASF formalism and its implementation

• Conditions: The "=" sign in conditions is deprecated. The new conditions can be:
  1. "==" for equality tests,
  2. "!=" for inequality tests,
  3. ":=" for matching a term on the right onto a pattern on the left,
  4. "!:=" for matching, failing if the match succeeds and vice versa.

• The new condition operators replace the "=" sign, allowing more static checking of the conditions for uninitialized variables etc. Static checks are now the following:
  1. For "==" and "!=" both sides may not introduce new variables.
  2. For ":=" and "!:=" the right-hand side may not introduce variables. The left-hand side MUST introduce at least one variable.
  3. Variables introduced by "!:=" can not be used by following condition or the right-hand side of a rule.
• NOTICE: the Upgrade menu of an ASF editor automatically translates all "=" conditions to ":=" or "==" conditions.

• New unit testing feature : Integrated ASF unit tests are a tool for:
  • Documenting the behaviour of a specification.
  • Debugging specifications
  • Regression testing specifications

• An ASF file is now a list of sections. A section is either an old-style equations section (with the "equations" keyword"), or a tests section (with the "tests" keyword). See for example basic/Integers in the library. Tests can be conditional, or unconditional and have almost the same syntax as equations, except that the "=" sign is replaced by a "==" or a "!=":

  [tag] lhs == rhs   or   [tag] lhs != rhs or

  [tag] conditions   or   [tag] conditions
        ==========              ==========
        lhs == rhs              lhs != rhs

• A tests is run by first evaluating the conditions, then reducing the lhs and the rhs and then checking equality. All failed tests are reported in the environment.

• Static checks of the tests ensure that:
  • The lhs and rhs do not introduce new variables
  • The conditions are checked in the same manner as the conditions in the equations.

• Rewriting with annotations is supported by both the interpreter as the compiler. This means that you can extract position information about a term during rewriting. See the utilities/PosInfo library module.

• All builtins are now supported by compiled specifications.

• Several builtins have been added for parsing and file IO. See utilities/Parsing and utilities/Files.

• The result of a builtin call is always normalized before continuing.

• Memoization in compiled specification using the {memo} attribute works.

• The ASF library has been restructured and several modules have been added extended or fixed.

• labels and cons attributes are not interpreted by ASF anymore (this was hidden functionality). Instead a specification generator generates several onvenience functions from an SDF definition using labels and cons attributes. he tool is called ASFSDFApigen and is available from the Tools menu in the module popup. Try to generate an API for basic/Booleans for example. This will generate a new module with all kinds of ready-to-use functions.

• The order in which rules are tried by the interpreter and a compiled specification is equal now.

• Compiled specification can deal with deeper and more complex nesting patterns. A bug in the generated code sometimes lead to problems for these kinds of patterns.

• Unused variables in the generated C code that the ASF compiler produces have been removed.

[August 6, 2003] Release 1.4.1:

Release 1.4 contained a number of inconsistencies that have been quickly remedied. Some of the features announced in 1.4 were not released properly.

Other changes since 1.4:

• SGLR only considers real user-defined injections in the "priorities over injections" disambiguation filter.
• A bug in the SGLR cycle detection phase caused in some subtle cases a loss of parse trees.
• A memory aligmnent bug in the ATerm library was fixed.
• Warnings are printed in blue, errors in red in MetaStudio? .
• Deleting and editing modules with parse errors improved.
• Speed of SDF2 parser improved.
• The build process of the JJTraveler subpackage was improved such that JUnit is really not needed anymore at build time.

[July 31, 2003] Release 1.4:

The Meta-Environment version 1.4 is a maintainance release. The openness and flexibility have been improved again, at the cost of some of the run-time performance, between 20% and 30% performance loss. Some practical features have been added on the user-interface level, making editing specifications and finding bugs in ASF+SDF specification much easier. The editors have become much more stable.

Important detail:

• In order to be able to compile the ASF+SDF Meta-Environment user interface (MetaStudio? ), you will need version 1.4 of SDK or higher, see http://java.sun.com/.

A summary of changes:

• The module list in MetaStudio? is now displayed in a tree view, visualizing compound module names as a hierarchy.
• The ATerm Java library now supports development with JJTraveler.
• In order to reduce the memory foot print of the ToolBus when shipping of (very) big terms explicit packing and unpacking is performed. Besides a memory reduction this allows maintaining maximal sharing of subterms.
• The ToolBus viewer has been improved, the delay time can be adjusted and the handling of breakpoints has been improved.
• The calculation of the layout of graphs drawn by the user interface is performed by a separate component, which increases reusability. A consequence is that we are able to visualize the parse trees in the user interface as well, via the "ViewTree" button. This gives more flexibility with respect to finding ambiguities. Furthermore, some of the nodes in the displayed parse trees are clickable and moves the focus in the corresponding editor.
• Fixed a serious bug SGLR with respect to cycle detection and filtering over injections.
• New warnings are added to the (asf)sdf*checker with respect to: * literals in the right hand side of production rules. * the traversal attributes.
• The ASF interpreter (asfe) and the ASF compiler (asfc) have the same behaviour with respect to specificity ordering of equations. Various minor bug fixes in both the interpreter and compiler.
• A major internal refactoring of architecture of the Meta*Environment has been performed. The internal data base in which all information of modules was stored has been replaced by a more flexible term store. The "hard-wired" functionality coded in the old data base has been lifted to the ToolBus level. On one hand this has lead to a tremendous gain in flexibility, on the other hand the Meta*Environment as a whole has become slower. More communication of the ToolBus in needed to calculate for instance import relations etc. However, a consequence of this refactoring has been that users are now able to store explicitly this term store and re*start the Meta-Environment with a saved term store. This leads to a considerable speed*up due to the fact that it saves the re-generation of parse tables, etc.
• Another major internal refactoring is with respect to functionality of buttons, etc. This functionality was hard*wired in the user interface, ToolBus scripts, etc. and is now defined by means of an internal action language. This increase the openness and genericity of the Meta-Environment in the future. All user interface actions are now defined via this internal action language. Furthermore, the distinction between the generic "Meta-Environment" and the "ASF+SDF" part has been strengthened.
• "Renamed" the functionality of some of the icons, added entries to the file menu in order to save and read term stores. Introduced a "Clear History" button in order to clear explicitly the message pane.
• The functionality to "save" a single module has been removed given the introduction of the term store, instead it is now possible to save explicitly the parse table for parsing the equations for debugging purposes (See the "Export" menu).
• Saved .baf (files with this extension have completely disappeared) and .tbl files are no longer read in when loading a specifcation.
• A visual distinction with respect to errors (printed in red), warnings (printed in blue), and messages (printed in black) has been introduced.
• Fixed quite a number of bugs related to crashing of editors, etc. One of the most important ones is that it is now able to edit erroneous Sdf modules.

Summarizing, version 1.4 of the ASF+SDF Meta-Environment is completely different from version 1.3 with respect to its internal structure and also to its behaviour of buttons, etc. We gained quite some flexibility and the maintenance of the entire system has become much simpler.

However, the price we paid for it was a loss of efficiency. Depending on the application it can be between 20% and 30%. This is specially noticable when starting the system and loading in modules. We will try to fix a number of these inefficiency bugs in the near future.

[February 11, 2003] Release 1.3:

• Global improvements:
  • The stability of editors has been improved!
  • Various small bug fixes in parser and parse tree tooling.
  • Openness of Meta-Environment has been improved in order to allow the plugging in other rewriting languages.
• Meta-Environment:
  • Meta-Environment editors can deal with ambiguous terms.
  • Interpreter rewrites ambiguous terms in the Meta-Environment.
  • AsFix syntax added to the ASF-library.
  • The Java/Swing GUI is now the default interface, the old Tcl/Tk interface can only be reinstantiated using manual changes in the source code.
  • New menu options in the GUI:
  • Close module: Removes a module from the environment, but not from disk. Optionally recursive closing of all imported modules that are not imported by other modules.
  • Delete module: Deletes a module from disk!
  • Copy module: Copies an entire module with only the name changed.
  • Import module: Adds an import to a module.
  • UnImport module: Removes an import from a module.
  • Viewing trees is temporarily disabled, this functionality will return in the next release.
  • Tide support improved (for debugging ASF specifications).
• Implementation details:
  • Some compatibility issues with Mac OS-X and CygWin removed.
  • The C ATerm library has a completely new generational garbage collector! (Thanks to Pierre-Etienne Moreau and Olivier Zendra at LORIA (Nancy)).
  • The new SharedObjectsFactory offers an interface to share any type of objects in Java.
  • "asfc -t" generates a ToolBus tool from an ASF specification.
  • New tool "generic-meta" start a Meta-Environment without an implementation of a rewriting language. Can also dump-equations and dump parse tables.
  • Many hard-wired details of the Meta-Environment scripts are now configurable.
  • The tools eqs-dump and pt-dump are now wrappers based on "generic-meta".
  • The new tool parsePT takes an AsFix parse tree and returns a parse tree of this parse tree over the AsFix syntax that can be found in the ASF-library. It can also parse any ATerm and return a parse tree of this ATerm over the ATerm library from the asf library.
  • Prefer/avoid counting mechanism changed slightly: Removed the check that a preferred/avoided subtree occurring in both trees are not counted as preferred/avoided when counting nested preferred/avoided subtrees.
  • ToolBus prints a list of deadlocked messages when CTRL+\ is pressed.
• Packages:
  • The "asf-library" package is removed, library is put into the "asf" package.
  • The "meta" package made independent of "asf" package.
  • A new package "asfsdf-meta" implements connection between meta and asf tools.
  • The asf package restructered in syntax, semantics and library part.
  • The ATerm library split into ATerm and ATerm-java
  • The ATerm-java library split into shared-objects and aterm-java.

[July 25, 2002] Release 1.2:

• The new user interface based on JAVA/Swing has been thoroughly tested over the last few months. In order to use this new user interface it is essential to configure the Meta-Environment in the following way:

 
 ./configure --prefix=/path/to/prefix
  --with-java=/path/to/jdk1.3
  --with-graphviz=/path/to/graphviz
  --enable-metastudio

• In case you use the old user interface the configuration should be performed as usual:

 
./configure --prefix=/path/to/prefix
  --with-graphviz=/path/to/graphviz 

• Major improvements/extensions of the Meta-Environment:
  • An online version of the user manual of the ASF+SDF Meta-Environment is available via: MetaEnvironmentUserManual?
  • An alfa version of a new button definition language was added to the Meta-Environment. One can add buttons with user-defined functionality to term editors, based on a collection of builtin Meta-Environment procedures and the ability to call external commandline tools. This extension of the Meta-Environment is still very experimental. There is no documentation of the button commands available.
  • An ASF checker was added that tests for uninstantiated variables.
  • Traversal functions support two new primitives: break and continue. This functionality is only available in the interpreter!
  • An ASF+SDF library is added. Currently the library only contains Booleans, Integers, Strings, the syntax of ASF, the syntax of SDF2 and the normalizer for SDF2 to Kernel SDF. There is also ALFA support for reading and writing files to disk and calling a shell command. The user manual of the Meta-Environment does not yet contain a description of this ASF+SDF library, but its use is quite straight-forward.
  • The ASF interpreter supports built-ins. The compiler does not yet support built-ins.
  • All Meta-Environment tools can work with 'compound module names' now. New module names are automatically constructed from a name and the shortest path from the meta.conf paths to the location of the new module.
  • The MetaStudio? is now fully compatible with the GraphBrowser? interface.
  • The JDK 1.4 requirement is relaxed to JDK 1.3 for MetaStudio? .
  • A new operator was added to SDF2: "`" Symbol "`" -> Symbol. It is used for making SDF2 symbol syntax available in the language you are defining.
  • Another new operator was added to SDF2: "(" Symbol "->" Symbol ")" -> Symbol. It is used for type definitions in ELAN strategies.
• Less important modifications:
  • Compiled specification have a '-t' switch for dumping textual ATerms
  • Implementation of ASF interpreter simplified
  • Restructuring of Meta-Environment ToolBus scripts to make button language work and to simplify the scripts.
  • All specification based on SDF2 now point to the SDF2 syntax of SDF2 in the asf-library.
  • Apply-function can be used as a ToolBus tool.
  • Tree2dot can handle more different kinds of input (partial parse trees and lists of partial parse trees)
  • AddPosInfo can now ignore literals and layout during annotation.
  • AddPosInfo is can be used as a ToolBus tool.
  • SGLR eagerness counting filter was changed to deal with some more subtleties.
  • restore-brackets tool was relocated from the asf to the sglr package.
• Bug fixes:
  • Concat-asf can deal with empty equation files
  • Functions that return lists still caused some trouble in the interpreter.
  • Some small bugs with module parameters were fixed. Both in the flattening from AsFix2? to AsFix2ME? and in the renaming of symbols in existing parse trees of equations.
  • An important bug in the toolbus interpreter concerning the use of many small proces definitions was fixed.
  • Many small bug fixes in the MetaStudio? made it more stable.

[April 30, 2002] Release 1.1.1

Release 1.1.1 contains bugfixes and some experimental features, including a new interface based on Swing:

• Features:
  • SDF checker extended with new warnings
  • Parameterized sorts added to SDF2
  • Experimental new user-interface (./configure --enable-metastudio)
• Bugfixes:
  • Bugfix in compiled traversal functions.
  • Bugfix in ASF predicate functions.
  • Several small incompatibility issues with MacOS X resolved.
  • Limit of data sizes going over the toolbus increased to 12 digits.
  • sglr produces an error message if there are more than 10000 ambiguities.
• Notes:
  • Due to the new data size limit in the ToolBus every ToolBus client needs to be recompiled against toolbuslib-0.2
  • The new user-interface is still in alpha status.

[February 25, 2002] Release 1.1:

• Due to bugs in the latest release of the gcc optimizer, we were forced to remove all '-O' optimization flags in the ATerm library. This will be reactivated as soon as the gcc compiler is fixed with respect to optimizations.
• The functionality of the tool "implodePT" is improved.
• The efficiency of the conversion to AsFix2ME is improved. This means that parser returns its results a faster.
• Functionality to restore brackets in case of priority or associativity conflicts in a normal form has been added to the Meta-Environment. This tool can also be used from the commandline as 'restorebrackets'
• The editor for a normal form is initialized with a parse tree now. This means that the structure of the normal form can be inspected without parsing first.
• The "non-assoc" attribute is no longer ignored by SGLR? .
• Ambiguity reporting has been slightly improved in SGLR? .
• The asfc compiler has been extended with full support for traversal functions.
• The Sdf to KernelSdf normalization phase contained a serious bug with respect to applying renamings to lexical/context-free restrictions.
• Added functionality to generate "getters/setters" given labels and contructor information in the context-free grammar rules of the SDF definition.
• Added an Sdf-Checker. Warnings are printed on stderr.
• Made the entire Meta-Environment less "ASF" specific, to make to Meta-Environment more open on the user-interface level.

[November 21, 2001] Release 1.0.1:

Release 1.0.1 of the Meta-Environment quickly follows 1.0 with some bugfixes:

• The gvim experiment has been disabled due to portability issues with libtool.
• Saving and restoring parse tables has been fixed.
• The fine-grained communication between xemacs and the structure editor was removed. From now on, the focus is only valid after parsing and before changing anything in the text. This solves the 'locking' problem in xemacs.
• The big core dumps of xemacs after closing a window have been solved.

[November 15, 2001] Release 1.0:

• IMPORTANT! The "Delete Module" functionality in the GUI now actually REMOVES the correspond ing SDF and ASF files from disk!
• Parametrized modules and symbol renaming have been added to ASF+SDF.
• Module name renaming functionality has been added to the user interface. Note that this also renames the corresponding SDF and ASF files on disk.
• Full SDF2 syntax supported by ASF interpreter and compiler. For example, alternatives and optionals can be rewritten.
• Traversal functions supported by the ASF interpreter.
• An experimental implementation of traversal functions implemented in the ASF compiler.
• An experimental version of gvim support has been added. Use 'meta -g' to try it out. Note that this requires vim-6.0 + GUI to work.
• Heavy maintainance on the SGLR parser has been done. In many cases, the parser produces less ambiguities.
• Commandline interfaces of asfc, asfe, apply-function, sglr and other less frequently used batch tools have been improved.
• Asfe has a new switch '-l' to indicate that all layout in the output term should be replaced by a single space.
• The tools eqs-dump and pt-dump have '-o' switches to designate output file names.
• Some optimizations in asfe have made the interpreter twice as fast.
• Compiled specifications have apply-function functionality incorporated.
• The parse tree format AsFix2? [ME] has changed a bit. The syntax of attributes has changed and the 'id' attribute was removed from exported productions.
• The pt-support package has a new tool called implodePT which maps a parse tree to an abstract syntax tree according to the cons attributes of the productions.
• The apply-function tool was moved from the asf package to the pt-support package. The '-m' switch was removed from its interface.
• The collaboration between the parse table generator and sglr has been improved. Sglr checks the version number of a parse table for compatibility.

[August 20, 2001] Release 0.8.1:

Release 0.8.1 of the ASF+SDF Meta-Environment involves a number of bug fixes and small changes with respect to previous release.

• tree2dot and ambtracker functionality has moved from SGLR to pt-support library.
• In SGLR quite a number of changes are performed:
  • The filters to deal with ambiguities in the parse forests are simplified and improved:
    • The MultiSet calculation is completely removed and replaced by a simpler Avoid/Prefer counting mechanism.
    • Improved the DirectEagerPrefer code (looks through injections).
    • Improved the FilterReject code, rejects are inspected before the actual filtering kicks in.
• Adapted for new parsetable format (marginally more compact).
• The parsetable generator is completely restructured in order to increase its efficiency:
  • Adapted for new parsetable format (marginally more compact).
  • Fixed a number of serious memory leaks.
• In the ASF+SDF compiler various optimizations are implemented:
  • Automatic constructor detection.
  • Optimization of recursive (list) functions.
• Added functionality to dump parsetables (pt-dump) and equations (eqs-dump) via the command line.

[July 18, 2001] Release 0.8:

Release 0.8 of the ASF+SDF Meta-Environment involves some small and some major changes with respect to previous releases:

• A serious circular dependency between the ATerm and ToolBus distributions has been resolved. As a result, the ATerm library no longer contains references to the ToolBus at all. The correct order of building is: FIRST the ATerm library, THEN the ToolBus.
• AsFix1? support has been removed entirely from all components of the Meta-Environment. Most notably, the libraries pt-support (to manipulate general parse trees), sdf-support (to manipulate SDF2 parse trees) and asf-support (to manipulate parse trees of ASF equations) have been stripped of AsFix1? functionality. The new tree format is called AsFix2ME? and it is still based on ATerms. The only effect visible to end users should be that the Meta-Environment takes less time to compile and install.
• The performance of the parser generator has been improved: speed gains of upto 60% have been reported. Filtering in sglr has been fine-tuned (thanks to remarks and bugs reported by Peter Mosses). With the disappearance of AsFix1? , the commandline option "-1" was removed from sglr. The new default behaviour (which can be enforced by passing sglr the "-m" flag) is to produce AsFix2ME? . Output can also be produced in plain AsFix2? by passing the "-2" option to sglr (see "sglr -h").
• The ASF+SDF compiler and the corresponding runtime library (asc-support) are more or less fully SDF2 compatible, exceptions are:
  • separators in separated lists can only be simple literals;
  • renaming is supported, but only for symbols.
• The compiler generates a single C file instead of C files per reshuffled function.
• Parse tables for parsing a term can now be dumped via GUI (click on a module and select "Dump Parse Table"). The GUI now has a "Print" button, which generates a single text file containing both syntax (.sdf) and equations (.asf) for a selected module. The Buttons "Save" and "Clear" have been renamed to "Save All" and "Clear All" respectively, to better explain their functionality.

Older releases

• [June 6, 2001] Release 0.7.3 contains major improvements in stability over 0.7, it can be completely bootstrapped without to old Meta-Environment.
• [March 31, 2001] The completa bundle is renamed to asfsdf-meta (release 0.7) and bundling is done using autobundle software
• [...] releases 0.2, 0.3, 0.3.1, 0.4, 0.5, 0.6 of completa slowly add more functionality to the Meta-Environment
• [April 4, 2000] First bundle of all tools in the new Meta-Environment released: completa-0.1