Neverlang

Questions and issues to Walter Cazzola.

This page is about the first version of Neverlang, this is outdated and unmaintained. This page is maintained for historical reasons. You probably are interested in Neverlang 2.

DSLs are used to solve several problems, such as typesetting documents and code (TEX/LATEX, lout, ...), to express and verify constraints in several domains (OCL, iLOG CP, C4J, ...) and to coordinate the computation and/or to data query (Linda, SQL, ...). In some cases, these are simply a bunch of programming features useless standalone embedded in a general purpose programming language or provided as external libraries (e.g., Linda and SQL). In these case performances and flexibility are often compromised especially when the DSL is realized as program transformation towards another high-level programming language.

A DSL integrating features from different programming languages and paradigms would be the best choice to express a concise and clean solution to a problem avoiding a cumbersome one due to the absence of a specific feature or to a farraginous general-purpose programming language. Unfortunately, to develop an ad hoc programming language implies a considerable effort.

To simplify and speed up the development of problem-tailored programming languages we developed Neverlang. The Neverlang tool basically reflects the fact that programming languages have a sectional definition and each language feature could be easily plugged and unplugged. A complete compiler/interpreter built up with Neverlang is the result of a compositional process involving several building blocks.

In this scenario, to design a domain specific language consists of implementing a set of slices (each of them coding a single programming feature and the necessary support code, such as type checking and code generation) and composing them together. The whole structure of the compiler/interpreter is the result of the composition of such a slices, in particular of the code necessary to compile/interpret each single feature.

How does it work?

The framework basically provides: a language for writing the building blocks and a mechanism for composing the blocks together and for generating the compiler/interpreter.

The basic units composing a programming language developed by using Neverlang are modules. Each module encapsulates a specific feature of the language, e.g., a module can encapsulate the syntactically aspect of a loop, the type checking code of a comparison, or the code generation for a method call. Roles define how modules composing together forming the compiler/interpreter. syntax, type-checking and evaluation are examples of roles. Finally, modules regarding the same language structure but with different roles are grouped together in slices.

The example below shows how Neverlang can be used to define a if-else conditional construct. Three roles (syntax, typechecking and evaluation) are involved; each role is defined in a separate module and combined together in the if slice.

if.nl

The first role (syntax) contains two productions that define the syntax of the if-else structure with or without the else branch. Each production is composed by: terminals (surrounded by ') and nonterminals (e.g., StatementL and ExprB). These productions are bound to the nonterminal Statement, other productions bound to such a nonterminal can be defined in the syntax module of other slices.

All the other kind of modules (in our example those with role type-checking and evaluation) add some semantics action to the grammar rules defined in the module with role syntax in the slice. Each of these modules associates to the nonterminals the semantic actions necessary to carry out the corresponding compilation/interpretation phase.

The nonterminals are identified through their position in the productions numbering with 0 the top leftmost nonterminal and incrementing by one left-to-right and up-to-down all the nonterminals independently of repetitions and for the whole set of productions defined in the slice. The heading of the action just figures out where the semantic action is anchored in the productions.

The first enriches the head of the first production (position 0) and simply tests the evaluation of the boolean expression associated to the nonterminal in position 1 (ExprB) and accordingly to that respectively evaluates the nonterminal in position 2 or 3 through their eval attribute. The second action similarly behaves but refers to the case without the else branch and it is associated to the head of the second production (position 4).

The associated semantic action that composes the type-checking role just checks that the test expression has a Boolean type otherwise it prints an error message.

The three modules are composed in a slice, the user has to define which role is implemented by each imported module.

All the slices are finally composed together to define the new programming language and to generate its compiler/interpreter. The example below shows the composition of 7 slices in a basic DSL able to perform some basic tests and print results. if slice apart, the other slices implement the compare operators the string and int types and the print command.

if_dsl.nl

This is an example of a source code written in the produced language:

if_else_01.dsl

To support the computation of the various interpretation phases, the developer may need some ancillary structures or services that concerns the whole compilation process affecting all the other modules crosswise. Simple examples are the symbol table --- a static table used (among the others) to check which types are comparable ---, and the code to deal with the memory management. To support this kind of behavior, we introduced a slightly different form of slice called endemic.

The example below shows the implementation of a simple endemic slice that implements a simple Var Table to map the identifiers with their values.

Fields and methods defined in an endemic slice are accessible by all the modules in the language independently of the compiling/interpreting phase. To add/replace an endemic slice permits to easily redefine the whole behavior of the interpreter.

The following is an example of how the value of a variable is retrieved using the Identifier and the VarTable.

Installation

REQUIREMENTS

In order to run the package in the system must be present:

• Java SDK 6.0
• AspectJ 1.6
• some common bash commands: dirname, which.

Optionally

• astyle (http://astyle.sourceforge.net/astyle.html) in order to format the generated source code

INSTALLATION

1. Download the version v0.9ß of the framework (TGZ).
2. Just unpack the tgz file and optionally add to the PATH environment variable «unpack dir»/Neverlang/bin/ where «unpack dir» is the directory where the package has been unpacked.

USAGE

To use the tool we provide two bash scripts: nlgc and nlg. The former reads the modules and generates the interpreter; the latter uses the generated interpreter given as first argument to execute the program files specified by the remaining arguments.

Launching the command below Neverlang produced the compiler as a package of class in the If_DSL directory.

Now with the nlg command is possible to use the compiler created to interpreter the simple source code contained in if_less_01.txt

Other options can be specified in order to use external classes.

nlgc -d «destDir» [-j «jar packages»] [-a «suppPackDir»] [-k] «file»+

where:

«file» is the file composing the grammar.

-d «destDir» specifies where the interpreter will be created; the directory will be overwritten if it already exists.

-a «suppPackDir» specifies where are the classes that should be compiled with the generated source code to create the interpreter.

-j «jar packages» specifies the necessary Java packages not in the classpath; multiple jar files are separated by a ':' symbol.

-k keeps the source code generated, the code will be pretty-formatted if the astyle package is available.

nlg [-j jars] «dir» «file»+ [-a «args»*]

where:

«dir» is the directory created by nlgc containing the interpreter to be executed.

«file» is the file containing the source to be executed by the interpreter.

-a «args» permits to specify the arguments to pass to program to execute.

-j «jar packages» specifies the needed Java packages not in the classpath; multiple jar files are separated by a ':' symbol.

Getting started with Neverlang

The package L+P contains the Neverlang modules to support a DSL that mixes the Linda coordination language and the functional part of Python. Two versions are provided; they differ on the implementation of the tuple space: one thread-based and the other distributed.

In order to run the examples:

• unpack the neverlang package.

• add to the PATH environment variable «unpack dir»/Neverlang/bin/ where «unpack dir» is where the package has been unpacked.

• add the ./ directory to the CLASSPATH environment variable if not already there.
• unpack the examples package and go to the test directory of the first example.

Note that, to run the second example contained in LindaPyComRmi directory you have to run one instance of the rmiregistry.

A plugin for Eclipse and few language extensions are under development, as well.

Neverlang Staff

The Neverlang project is led by Walter Cazzola.

• Ivan Speziale, defined the framework's back-end;
• Davide Poletti, developed the front-end of the framework and integrated it with the existing back-end;
• Edoardo Vacchi, developed DEXTER a on-the-fly parser generator that has replaced RATS! in Neverlang.

1. Walter Cazzola and Ivan Speziale, Sectional Domain Specific Languages, in Proceedings of the 4th Domain Specific Aspect-Oriented Languages (DSAL'09), Charlottesville, Virginia, USA, March 2009, pp. 11-14, ACM. [PDF1].
2. Walter Cazzola and Davide Poletti, DSL Evolution through Composition, in Proceedings of the 7th ECOOP Workshop on Reflection, AOP and Meta-Data for Software Evolution (RAM-SE'10), Maribor, Slovenia, June 2010, ACM. [PDF2].
3. Walter Cazzola, Domain-Specific Languages in Few Steps: The Neverlang Approach, in Proceedings of the 11th International Conference on Software Composition (SC'12), Thomas Gschwind, Flavio De Paoli, Volker Gruhn, and Matthias Book, Eds., Prague, Czech Republic, May-June 2012, Lecture Notes in Computer Science 7306, pp. 162-177, Springer. [PDF3].
4. Walter Cazzola and Edoardo Vacchi, DEXTER and Neverlang: A Union Towards Dynamicity, in Proceedings of the 7th Workshop on the Implementation, Compilation, Optimization of Object-Oriented Languages, Programs and Systems (ICOOOLPS'12), Eric Jul, Ian Rogers, and Olivier Zendra, Eds., Beijing, China, June 2012, ACM. [PDF4].