Access Keys:
Skip to content (Access Key - 0)
Truffle FAQ and Guidelines
Attachments: • Added by Michael Haupt, last edited by Michael Haupt on Feb 04, 2014  (view change)
Labels
  • None

To get started with Truffle, read the self-documenting test programs in com.oracle.truffle.api.test. Each test program both describes and demonstrates a basic Truffle feature. Start with TruffleRuntimeTest.java, and follow the Javadoc links through the thread. Consider using a IDE/debugger to walk through each example.

The rest of this page contains answers to questions you may have after you complete this mini tutorial and start using Truffle to implement your guest language.

From Guest Languages to Truffle

How can I generate Truffle ASTs?

You need a parser to create a Truffle AST for each guest language program you want to run. There are several approaches:

  • Reuse an existing parser, as is done for the (internal) Javascript and Ruby implementations. Either modify the parser to create Truffle ASTs directly or build a "translator" that walks parser output and creates Truffle ASTs.
  • Use a parser generator, as is done for example with SimpleLanguage (using Coco from Linz), to create a parser and configure it build a Truffle AST directly.
  • Write a parser by hand to do the same.

How should I map language operations to Truffle nodes?

Provide one node class per operation, and specialize it for different operand types: each node class should do one simple thing—in many different ways, by virtue of specialization.

How do I determine mappings that are well optimizable?

Provide microbenchmarks that challenge all variants of single operations of the language you’re implementing. These operations should of course map to single nodes. From the analysis of these benchmarks, the need for new specializations may arise (see below).

What about operations with implicit parallelism, as found in array languages?

Implement the language operations that can be implicitly parallel as dedicated nodes. In their execute() methods, implement the parallelism as simple for loops. The compiler will apply loop unrolling and vectorization to optimize. It is also possible to specialize for special cases, such as single- or two-element vectors. Note that the loops should avoid allocation, and be side-effect free.

How much of a language’s standard library should be “Truffleized”?

If there is a large number of language operations that map to simple basic library functions, it can be very beneficial to implement those library bits in dedicated nodes.

How should I name methods pertaining to AST node execution?

Parent nodes evaluate child nodes by calling methods referred to informally as execute() methods, but Truffle imposes no restriction on those methods' names.  However, having "execute" in those method names is a useful convention.

To help partial evaluation, should I declare as many fields as possible as final?

Yes; make fields and parameters final whenever possible, except for fields marked @Child, which must not be final. (This hampers AST rewriting.) Fields annotated with @Children, which hold arrays or other structures, must be final.

How large should my Java methods in Truffle nodes be?

Keep them small, and do not shy away from factoring functionality out into helper methods. Method calls are not inherently bad: Graal is smart about inlining.

How can I implement caching at polymorphic call sites?

Assuming you have a CallNode that performs late-bound calls. In its uninitialized variant, the execute() method looks up the receiver and rewrites the CallNode to one that caches the receiver. In this CachedCallNode, execution checks, using a guard, whether the receiver class is still the same and, if the guard fails, takes further action. This further action can consist in caching additional receivers, or in rewriting to a GenericCallNode that performs resolution on each call. When implementing a language supporting reflection, you might consider guarding receiver methods with an Assumption (see below) to react accordingly if they are changed.

Guards and the Fast Path

How can I model global assumptions in my language implementation that hold for a while but may eventually break?

Use the Assumption abstraction. Assumptions allow you to check efficiently that a boolean flag hasn't been changed. In the interpreter it is implemented by reading a field, but in the compiler it is completely elided and instead if the flag changes the machine code is deoptimized. This can, for instance, be used to guard against modification of methods and classes by reflection in guest language programs. You can create an Assumption via Truffle.getRuntime().createAssumption(), and use Assumption.check() to dynamically check the assumption, and Assumption.invalidate() when it is no longer correct.

How do I implement local guards?

Any boolean condition in Java can serve as a guard causing a fallback from compiled to interpreted mode. Whenever you write something like if (!condition) { CompilerDirectives.transferToInterpreter(); ... }, the condition is interpreted as a guard that causes deoptimization if false. There is one constraint: the code following the call to transferToInterpreter() should make sure that subsequent execution does not periodically hit the if branch. One possible way to do that is via replacing the node with a different node that no longer relies on the guard and instead performs a more generic version of the operation.

If I replace a node when a guard fails, do I have to transfer to the interpreter explicitly?

No. Node replacement always implies deoptimization. When your logic following guard failure only performs node replacement, it is enough to write if (!condition) { this.replace(createMoreGenericNode(this)); }.

Specialization

What is specialization good for?

Any language operation may have a set of cases for execution. These cases can cover different operand types, but also special values (e.g., null or constant vectors). Specialization addresses such cases by providing dedicated implementations of an operation for given input characteristics. Specialized nodes allow for type propagation up the AST, and for better partial evaluation results.

What is the rationale for specialization?

Provide specializations for operations that are likely to asymptotically stabilize for certain types and values: if the latter do not change at one particular application of a language operation when it is executed infinitely often, that operation is a good candidate for specialization for the given types and values.

How should I design specialization for asymptotic stability?

Think of the specializations that are possible at one operation as a finite state machine. Each of the machine’s states is a specialization (starting with “uninitialized”), and each of the machine’s transitions is a node rewrite, replacing the node’s current specialization with another. If the state machine does not have any cycles, asymptotic stability is an implicit characteristic.

What can I do to analyse my implementation for specialization opportunities?

Use the -G:+TraceTruffleCompilationDetails command line flag. This will display a histogram of nodes in compiled code. If a node appears very often in the histogram, it is a good candidate for additional specialization.

In what state should nodes be that my parser emits?

Create nodes in "uninitialized" state. Let the nodes specialize, rather than emitting optimistic nodes. Partly this is just clean design: the parser has no business guessing anyone's types when it emits nodes. If you correctly speculate then you might have one extra execution that's the correct type, but who cares about just one execution in the interpreter? Also remember that generally we want to move from uninitialized, to specialized, to generic, and if you start off in the middle of that at specialized, it's easier to become generic (slow) than go back up to uninitialized which isn't such a natural process in Truffle.

To avoid node explosion, can I apply in-place specialization at method level?

Yes, you can use a member variable in your node class to signify the condition(s) for which the node should be specialized. This member should be annotated with @CompilerDirectives.CompilationFinal to let it look like a constant to the compiler. Then, you can use a pattern as follows:

@CompilerDirectives.CompilationFinal private boolean everSeenSpecializationCondition = false;
public void operation(…) {
  …
  if (specializationConditionMet()) {
    if (!everSeenSpecializationCondition) {
      CompilerDirectives.transferToInterpreter();
      everSeenSpecializationCondition = true;
    }
    // handleSpecialization
  }
}

If the method gets partially evaluated and compiled while the specialization condition has never been met, the compiler will not generate machine code for the specialization handling branch, as it regards the boolean flag as a constant with the value false. Instead, it will generate a deoptimization at the transferToInterpreter() call and keep the complete specialization handling branch on the slow path. The (now true) flag is still considered compilation-final, so if the method gets partially evaluated and compiled in the specialized state, the specialization handling branch will now be compiled into the fast path.

Truffle Compilation

How do I know what is getting compiled?

When you run with --vm server, Truffle is fairly talkative and will tell you when methods are (Truffle) compiled and when they are deoptimized. If it doesn't tell you it's compiling anything, then nothing is getting compiled.

How can I get information about which nodes are being compiled?

Use the -G:+TraceTruffleCompilationDetails argument. You'll see a histogram of which nodes have been compiled.

How do I make Truffle compile my methods?

There is a default threshold of 1,000 runs before a method is compiled. You can modify this threshold with the -G:TruffleCompilationThreshold=1 argument.

How can I effect a warm-up phase for performance measurements?

To let the compiled code warm up (get into the instruction cache and warm up branch prediction etc.), run the main method 10,000 times before you start timing, which both provokes the compiler and lets the generated code warm up.

How can I be sure some code is not being compiled?

Insert calls to the pseudo-method CompilerAsserts.neverPartOfCompilation() into the respective code. This will lead to error messages if the code ends up on the compilation path.

How can I be sure some code is being compiled?

There is no inverse of CompilerAsserts.neverPartOfCompilation(), but to do a quick check, put that assertion in. If it fails, then you know the compiler tried to compile it.

How can I make my Truffle program faster?

Look at what is getting compiled; look at the node histogram. Make those nodes simpler and more specialized, starting with the most numerous.

Why do my methods keep getting deoptimized?

You are probably causing this by calling CompilerDirectives.transferToInterpreter() somewhere, or invalidating an Assumption. Try a simple println before places where you call that. Is your node rewriting running in a loop? Your tree should stabilize over time. Perhaps you need to move nodes towards a more general case rather than eternally switching between specialized nodes.

If that's not it, occasionally Graal gets into an unintentional deoptimize loop. It appears to be some corrupt memory somewhere.

There is a tool to see why methods are invalidated, -XX:+TraceDeoptimization, but this can be a little cryptic.

Can I be notified when Graal optimizations cannot be applied?

You can use the -G:+TraceTrufflePerformanceWarnings option. If it is given, the system will issue a warning when a Graal IR graph exceeds the size defined by the TruffleOperationCacheMaxNodes constant and Graal stops optimizing.

Does it make sense to analyze compilation results at the level of Graal IR graphs?

Going to Graal IR level via IGV should not be necessary when writing Truffle language interpreters. Even passionate compiler hackers could get a bit lost there. Instead, use the -G:+TraceTruffleCompilationDetails flag. If a node appears a lot in the histogram, try to make it simpler by specializing it more, and then the program goes faster.

Things to Avoid

Does it make sense to cache large objects for reuse?

Do not try to be clever caching values. It makes sense to allocate even large values in execute() methods. If the allocation and all uses of the allocated object are visible to the compiler after partial evaluation, it can apply good optimizations.

Couldn’t I generate ASTs with pre-initialized (specialized) nodes right away?

In general, this is not advisable. It complicates your parser and node generator. Nodes will specialize upon first execution in the interpreter, which is a bearably small overhead. The design is also cleaner if you emit nodes that begin in the initial state of the finite state machines that describe their specialization lattices.

Can I store VirtualFrame instances in fields?

No. The frame represents an execution scope and provides access to local variables in the guest language. Truffle can in most situations "virtualize" the frame, meaning that there is no Frame object allocated and variable access is compiled into very efficient direct access. Don't interfere by storing the frame in a field or by casting the frame to Object, which would cause the frame be "materialized" and an object allocated. If you really need to store a frame, turn it into a MaterializedFrame by calling materialize().

Can I have both @SlowPath and @Specialization annotations on the same method?

Yes. This forces the specialization to never be compiled: it will always be run in interpreted mode. Be aware, though, that all methods accepting a VirtualFrame parameter are force-inlined - the @SlowPath annotation has no effect on them.

Running Truffle

Can I explicitly run a program in interpreted or compiled mode?

The --vm argument chooses which JVM compiler to use:

  • --vm server-nograal: This is the server compiler (HotSpot) running your Truffle program as a normal Java program. We talk about this as being interpreted, but the JVM is applying JIT compilation as normal. This is the system we normally use for development and testing.
  • --vm server: This is the server compiler running Graal.
  • --vm graal: This is Graal, running Graal, via a lengthy bootstrapping phase. This is currently less stable, and there is no speedup for your Truffle program, as it's still the same Graal compiler, just running on top of something else. This shouldn't be needed for most guest language implementations.
  • There are also --vm client-nograal and --vm client, which use the JVM client compiler instead of the server compiler.

So, for normal development, use --vm server-nograal, and when you want to check out the speedup Graal gives us, use --vm server.

How do I pass arguments to Graal?

It goes after the command you're running in mx, before the arguments that get passed to it, and you need an "@" before them. For example:

     ./mx.sh --vm server ruby @-G:TruffleCompilationThreshold=1 <my ruby arguments>

Error Handling

How can I debug a Truffle-based application in Eclipse?

Use the -d flag to the mx script to start the VM in debug mode; it will wait on port 8080 for an external debugger to attach. In Eclipse, use the predefined debug launcher configuration (it is generated when running ./mx.sh eclipseinit to attach to the running VM.

See also: Debugging.

Why am I getting errors about escaping frames?

Truffle is currently unable to determine that private/final methods for getting "node fields" (e.g., @Child fields) in parent classes should be treated as final. To fix:

  • Short term: always reference node fields directly, even if in parent classes.
  • Medium term: we might introduce a @Fold annotation for methods.
  • Long term: we need to design our compiler optimizations in a way of "do the minimum amount of work necessary to make this value constant"—that's a hard problem though.

Why am I getting errors about materializing frames?

Graal is not able to escape analyze the VirtualFrame objects like you think it should be able to. You are probably using slightly too much abstraction. It's hard to give a definitive solution to this, because Graal will cope with some legal Java code, but will fail with different but semantically equivalent legal Java code. Abstract less until it goes away, then try not to let it creep back in.

More Information

License

As with the rest of Graal, Truffle is published under GPLv2. In addition, the Truffle API (i.e., the classes in the com.oracle.truffle.api and com.oracle.truffle.api.dsl projects) comes with the "Classpath Exception". This allows language implementations to link with the Truffle API without any license restrictions.