Extensions may have different goals (from less to more ambitious):

Some of these goals can be achieved by adding some configuration information and some scripting, others require full-fledged software development.

There are various types of extension points to achieve these goals. For some of these goals, more than one type of extension point should be considered. We distinguish:

Firstly, the source code for a particular extension needs to be organized in a new package. Secondly, this new package needs a configuration script. Then, we need new ToolBus scripts, new tools and new MetaStudio GUI plugins that implement the extension and coordinate its behavior with the rest of the system. Finally, if the extension represents an executable language, a debugging adapter may be included.

To make the above a bit more explicit, we describe the steps that are needed to build an IDE for a domain specific language on top of the Meta-Environment:

For some extensions not all of the above steps are necessary. The IDE construction goal is the most demanding extension, while the other goals will require less effort.

The Meta-Environment has many extension points, and some of them are still in flux. Still, this document provides an overview of the more stable extension points, and a brief explanation of how to use them. It is a first attempt at making these APIs more widely usable (meaning outside of the Meta-Environment development team).

When this document does not provide you with enough detailed information, please refer to the following ToolBus scripts:

These files implement or directly control all extension points of the Meta-Environment, since they control all features of The Meta-Environment.

Note that all ToolBus scripts (.tb and .idef.src) files can be found online via the online Meta-Environment API reference, althought the syntax highlighting and linking does not work correctly for these files.

For each new package you have to set-up the build environment. This is done as follows in the ./reconf file:

#! /bin/sh

meta-build || ("Please make sure meta-build is in your PATH" && false)

This file bootstraps the build environment of a package. Make reconf executable and run the command ./reconf, and the meta-build command runs the appropriate tools like automake, libtool, aclocal, and autoconf to set up the package's configure script and makefiles. These tools generate files, and create several soft links in the working directory for later use.

A major issue in software development is how to configure the software to compile and run on as many computing platforms as possible. Typical issues are: Where is the C compiler? Which command line options does it accept? How do I run the Java compiler? How do I build libraries? Where should the binaries or documentation be installed? And so on and so forth. We use an extended version of the autoconf system to achieve this. Consider the following example of configure.ac:

AC_INIT
META_SETUP
META_C_SETUP
META_JAVA_SETUP
AC_PROG_LIBTOOL
AC_OUTPUT

This file is used by autoconf to generate a configure script. The example file here prepares the package for C and Java code. The C code will use dynamically linked libraries. Configure.ac must at least contain (in this order): AC_INIT, META_SETUP, and AC_OUTPUT. It may contain META_C_SETUP, or META_JAVA_SETUP, or both. Furthermore, any autoconf macros are allowed if your package needs more detailed configuration. Please read the autoconf manual for details.

Note that the macros with the prefix META_ take care of inter-package dependencies, see the <packagename>.pc.in file.

Having set-up the build environment, it is now time to define the properties of the new package such as its name, version, required packages and the like. Here is a skeleton that also contains some example information:

prefix=@prefix@
Maintainers=jurgenv@cwi.nl,economop@cwi.nl
Name: <packagename>
Version: <packageversion>
Description: <packagedescription>
Requires: <dependency1>,<dependency2>,<...>
Cflags: -I${prefix}/include
Libs: -L${prefix}/lib -l<libraryname1> -l<libraryname2> <...>
#uninstalled Libs: -L@abs_top_builddir@/lib -l<libraryname1> \
 -l<libraryname2> <...>
#uninstalled Cflags: -I@abs_top_builddir@/lib 
ToolBusFlags=-I${prefix}/share/<packagename>
JarFile=<packagename>.jar
Jars=${prefix}/share/${JarFile} 
MainClass=<mainclassname>
#uninstalled UninstalledJars=@abs_top_srcdir@/${JarFile}
Packages=<javapackagenames>
TestClass=<toptestclassname>

For a package with name <packagename> the package description file has the name <packagename>.pc.in after, of course, replacing <packagename> by the actual name, e.g., sglr.pc.in or meta.pc.in. This file declares all there is to know about a package. This information is typically used by the reconf script, by continuous integration toolkits, by package managers, and other deployment and product composition tools.

Before using the package definition file, it is instantiated by substituting all variables like @prefix@, @abs_top_builddir@ by their actual, installation-dependent, values. The first line regarding the prefix variable is an obligatory implementation detail. The #uninstalled variables are used for compiling compositions of packages before they are installed (packages may refer to the source locations instead of the installed locations using this information).

The Requires field is very important. It declares the dependencies on other packages. The package definition file given above supports three different programming languages (C, Java and ToolBus scripts). If a package contains any of these, the associated variables are needed, otherwise not.

C packages that export libraries use Cflags, Libs and their uninstalled variants. This information is used by the makefiles of other packages. Namely, for each package they depend on those variables will be made available. Example: If your package depends on the aterm package, then you will have ATERM_CFLAGS, and ATERM_LIBS at your disposal in the makefiles.

Java packages export jar files, and have a test interface. From this information a full ant build configuration is created. This assumes that the source code is located in the subdirectory src. The other variables explain where the jar file should be installed and which packages should be included. The MainClass variable is used to properly instantiate a jar file manifest. This is also relevant for constructing correctly working plugins of the MetaStudio.

Packages containing ToolBus scripts explain where the scripts are installed such that dependent packages may set their search paths automatically.

Note that the implementation of the semantics of the .pc.in file can be found in the package meta-build-env. This package contains extension to automake and autoconf, written as shell scripts and m4 macros. Understanding, writing or changing meta-build-env is considered to be a black art, but it does allow us to remove a lot of code duplication, and improve consistency, in the infra-structure of packages.

The following naming conventions are used for source code subdirectories:

Each of these directories, except for src, has a Makefile.am file. Most (but not all) Meta-Environment packages follow this convention.

Now we have reached the level of actual source files and their build instructions. We use automatically generated Makefiles that have been tailored with installation-dependent information. The recipe for generating each Makefile is given in an Makefile.am file. The suffix .am refers to the tool automake that is used to do the actual Makefile generation. The entire tool chain of automake, autoconf, aclocal, configure, etc. starts with the Makefile.am file. Here is an example:

include $(top_srcdir)/Makefile.top.meta

SUBDIRS = <subdirs>

ACLOCAL_AMFLAGS = -I .

Makefiles are notoriously sensitive to spaces. For instance, the exact number of spaces as given on the line ACLOCAL_AMFLAGS should be used (this is also due to tools such as autoreconf that scan the contents of Makefile.am files using rather brittle regular expressions).

The example above is relevant the top level directory of each package. For C packages and ToolBus packages there are additional Makefile.am files. For these kinds of packages, each subdirectory should contain its own Makefile.am. Please read the automake manual on how to write Makefile.am files for compiling C libraries and C programs. Each Makefile.am may include Makefile.meta, such that make rules and other utilities become available and can be reused. You are advised to look at the Makefile.am files of existing packages for inspiration. Inclusion of the Makefile.meta file is not obligatory however. What do we illustrate with the following:

include $(top_srcdir)/Makefile.meta

There is an exception, in case of a Java package. The toplevel Makefile.am should look like this:

include $(top_srcdir)/Makefile.java.meta

ACLOCAL_AMFLAGS = -I .

Java packages should have all their code in a subdirectory called src. This subdirectory may contain packages, but the corresponding directories do not need copies of a Makefile.am file. Note that by including Makefile.java.meta the package will use ant to compile your source code using the information from <package>.pc.in.

It helps other developers to describe the changes you have made to source files. The file ChangeLog is the place for collecting this information. Usually, it contains verbatim copies of the entries in the version management system. Here is an example:

Fri Jun 30 21:43:04 CEST 2006 <jurgenv@cwi.nl>
 * libmept/MEPT-utils.[ch]: reorganized this long file into a number 
   of smaller files that each deal with a specific topic.
 * libmept/*.c: added much documentation.
 * libmept/*.h: fixed bug #774
 * pt-support.pc.in: Bumped version to 0.1231

ChangeLog contains remarks on all changes to the source code of a package, in reverse order of date. Maintaining this information in this file makes it independent of a version management system. It should contain references to bug numbers when bugs are fixed. The ChangeLog file has a fixed format. Editors such as gvim and emacs have plugins for its syntax. ChangeLog files are maintained very well in Meta-Environment packages.

The COPYING file contains the license for the source code located in this package. This is usually the LGPL license for Meta-Environment packages. The AUTHORS file lists the authors email addresses. The README file should contain a brief description of the package. The NEWS file should list remarks on the evolution of the package on a high level of abstraction. Note that README and NEWS files are usually not maintained too well in Meta-Environment packages.

Recall that the configuration manager provides a ToolBus API for the configuration parameters that have been loaded from the configuration scripts. This information is used by a number of generic tools in the Meta-Environment:

Let's focus on 'action' configuration parameters. Each action identifies for which component it is relevent, which menu option or other user-interface component it should be associated with, and which ToolBus process should be called when the action is activated (see configuration scripts).

The above named tools use the configuration information to build up menu's and register listeners to GUI events. They translate the events to ToolBus events, and then inside the ToolBus script such an event is used to call the specific ToolBus process. Each component may pass different parameters to the configured process name.

For example, the editor-plugin provides an EditorId. Suppose there is a menu option called CompileDSL configured for a certain editor:

action([
    description(
      term-editor, 
      menu-shortcut([label("DSL"),label("Compile")], 
                    shortcut([M_ALT,M_CTRL],VK_C)))], 
      "CompileDSL"),

Which means that the process CompileDSL will be called when somebody clicks DSL->Compile, or types ALT+CTRL+C. Then, we need to implement the following ToolBus script:

process CompileDSL(EditorId : term) is
let
  ...
in
  ...
endlet

The EditorId term can be used to look up information about the editor, such as filename (see the standard ToolBus actions ).

These are currently the ways processes are called from each component:

The above information is enough for anybody implementing actions for existing components.

When you implement a completely new component, you should instantiate a connection between defined actions in a configuration script, and the way a ToolBus process is called. The basic design pattern for this is as follows. We provide it here because the referenced source code is more complex. The real source code introduces one more layer of abstraction, by not letting a tool interface communicate directly with the configuration manager. This is also advised for any new components you develop, but we have left this out in the following example for clarity:

tool exampleTool is { ... }
process ExampleToolInterface is
let 
  T : exampleTool,
  Id: term,
  Event : term,
  Events : list,
  Action : str
in
  rec-connect(T?)                                                               /* a tool initiates a connection, for example a MetaStudio plugin */
  . snd-msg(cm-get-events(example-tool-identifier))                             /* retrieve the appropriate menus from the configuration manager, */
  . rec-msg(cm-events(Events?))                                                 /* the 'example-tool-identifier' will occur in configuration scripts! */
  . snd-do(T, register-events(Events))                                          /* Use the menus to initialize the tool, the tool should store the Events information somewhere */
  .
  (                                                                             /* After initialization, the tool goes into a processing loop */
    rec-event(T, event(Event?))                                                 /* An event is send back by the tool, (this is one of elements of the original Events list) */
    . snd-msg(cm-get-action(example-tool-identifier, Event))                    /* Lookup the related process name from the configuration manager */
    . rec-msg(cm-action(Action?))
    .
    (
      printf("Warning: process not found: %s\n", Action)
    +>
      Action()                                                                  /* The process is called dynamically, if it is not found, the printf above is executed */
    )
  +
    ...                                                                         /* The process will have rec-msg's and snd-do's for implementing the rest of the tool API */
  )*
  rec-disconnect(T) /* This process terminates when the tool disconnects */
endlet

The above skeleton can be extended by calling Action with different parameters, by adding more kinds of events, by adding different 'example-tool-identifiers' in the same tool, etc. Of course, any tool interface will probably have more snd-do's and snd-evals than is shown in the above skeleton.

The ToolBus scripts that implement The Meta-Environment already provide a wealth of functionality that you can reuse in configuration scripts. This functionality is separated in three layers: meta, sdf-meta and asfsdf-meta. The meta layer contains only language independent processes. The sdf-meta layer introduces SDF specific scripts. The asfsdf-meta layer introduces ASF specific scripts. Note that the sdf-meta layer depends on the meta layer, and the asfsdf-meta layer depends on the sdf-meta layer. To load them use any or all of the following include statements:

#include <meta.tb>
#include <sdf-meta.tb>
#include <asfsdf-meta.tb>

Including these files makes a number of things readily available:

So, for implementing a certain Action, you should start looking for functionality in -utils.tb files, if not found you should fall back to .tb files, if still not found, there is a chance of finding things in the .idef files of specific tools. Note that the -actions.tb files are usually too specific for reuse, but in some very simple cases they may come in handy.

The best resource for finding out about tools and processes is reading the source code of .idef and .tb files at the Meta-Environment online API reference.

Todo

Todo

Todo