Inlining closure

I did some manual experiments of inlining the body of closure from the previous post to make it fast. So far the inlining technique would work only for anonymous closures as it won't be referred by other codes.

For example, if a$a is an anonymous closure:

DefaultMethods.times(100, a$a);

can be inlined as the private method inlined_DefaultMethods_times:

inlined_DefaultMethods_times(100);

Then the body of DefaultMethods#times will be copied to construct the above private method. After that the callsite of Closure a$a will be replaced with its body, the method a$a__0.

The original DefaultMethods#times:

public static void times(int num, Closure closure) {
for(int i=0;i<num;i++) {
closure.call(i);
}
}

The inlined method inlined_DefaultMethod_times:

private void inlined_DefaultMethod_times(int num) {
for(int i=0;i<num;i++) {
a$a__0(i);
}
}

The result is reallly impressive, 22 sec, which is comparable to original Java code.

Steps toward encoding fast closures, reflection-based closure

With the reflection-based closure approach from last post, I've got some codes to compare performance.

The following is a Groovy code,

class TestGen_002 {
def a = {
1000000.times {
println it
}
}
}

And this is the manually translated code. Class TestGen_002 contains reflection-based closures, a and an anonymous closure for times.

package org.codehaus.qdg;

import org.codehaus.groovy.qdg.Closure;
import org.codehaus.qdg.runtime.DefaultMethods;

public class TestGen_002 {

public Closure a = new Closure(this, "a__0");

private Object a__0(Object... args) {
Closure a$a = new Closure(this, "a$a__0");
DefaultMethods.times(1000000, a$a);
return null;
}

private Object a$a__0(Object... args) {
System.out.println(args[0]);
return null;
}

}

Although it's fast, 28 secs, compared with traditional Java for loop (22 secs). I've been trying to get its speed closer to Java.

Implementing Closure without using inner classes

This implementation is inspired from Charles Nutters' discussion about his experiences implementing JRuby. I didn't have much time to look into it, until now. So here's my first implementation of it.

I have my Closure field a, and its body a_body.
In the Closure class, the ctor will reflectively get the declared method "a_body" by name.
When you'd like to invoke the closure, just call a.call(). Then it will normally invoke the "a_body" method with reflection.

package org.codehaus.qdg;

import org.codehaus.groovy.qdg.Closure;

public class TestGen_001 {

public Closure a = new Closure(this, "a_body");

private Object a_body(Object... args) {
System.out.println(args[0]);
System.out.println("a_body");
return null;
}

}

And here's a simple implementation of the class Closure:

package org.codehaus.groovy.qdg;

import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.security.AccessController;
import java.security.PrivilegedAction;

public class Closure {

private Object target;
private Method method;

public Closure(Object target, String methodName) {
this.target = target;
try {
method = target.getClass().getDeclaredMethod(
methodName, new Class[]{Object[].class});
AccessController.doPrivileged(
new PrivilegedAction(){
@Override
public Object run() {
method.setAccessible(true);
return null;
}
});
} catch (SecurityException e) {
e.printStackTrace();
} catch (NoSuchMethodException e) {
e.printStackTrace();
}
}

public Object call(Object... args) {
try {
return method.invoke(target, (Object)args);
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (IllegalAccessException e) {
e.printStackTrace();
} catch (InvocationTargetException e) {
e.printStackTrace();
}
return null;
}
}

A closure body, which is a private method, is accessible by setting Method#setAccessible(true).

Quasi Dynamic Groovy

I'm trying to summarise what I want from Groovy.
One of my goal is to use Groovy for numerical computing. I love Groovy syntax because it's concise, compact, and elegance when writing an algorithm. So I've started writing code generator from Groovy parser. It's relatively easy to start, if you have .jar distribution of Groovy.

Here's a list of my requirements:
- Partially static Groovy
- AOP support by design, at the level of the runtime system.

Challenges so far are:
1. Is it possible to make a Groovy class partially static? IMO, I think it would. The support reason is very obvious. Now, we can mix Groovy and Java classes in the same system.
So what if we resolve method references at compile-time. If the method reference is found we make it static, if not we just make it dynamic.
2. Is it possible to use the envelope technique to wrap around method calls, getters, setters, etc.
This technique can be used to have a dynamic AOP support.
3. How to keep the class semantic to be as similar as possible to the classic Groovy.

Groovy AOP - Part #4, Getter and Setter PCDs

I haven't implemented pointcut designators (PCDs) for getter and setter because I actually forgot about them.

I recall them this morning, and really see the advantage of using them for Grails domain classes. I think using it can definitely solve the problem's been caused by Hibernate setter issue. Here's the situation.

I have the domain class, User containing the password field passwd:

class User {
...
String passwd
}

What I wanted to do is to store MD5ed password by just calling new User(password:'abc').save(). But this won't work. It's the Hibernate limitation that does not allow me to have the setter for passwd, say setPasswd, because the setter will be called, at least, twice when Hibernate 1. initialises the object, 2. sets the value into the property. Yes, I got my password hashed twice !

So here can be a solution (I have to try it first to confirm this anyway).

Groovy AOP can provide a shortcut for getter and setter like:

class User {
static aspect = {
before(set:'passwd'){ value -> value = MD5(value) }
}
...
String passwd

}

Why this's going to work? I think it works because Groovy AOP works at meta-layer, while Hibernate works at bytecode (proxied) layer. It's possible for Hibernate to set the property directly (as it does not invoke the call through Groovy invoker). Good use case?

Groovy AOP - Redesign #3, Advise By Convention

This idea came solely from seeing what Grails's using for interception.
The idea is simple, like ActiveRecord pattern used in Grails (e.g., find*)

In Grails, users'd like to have before, after interceptions for their controllers, domain classes, etc.

Like this:

class MyController {
static beforeInterceptor = ['*', this.&auth]
}

and this:

class MyDomain {
String field1

static beforeUpdate = {
}

}

So Groovy AOP would support these kind of functionalities with conventions in its aspect language like the following for controllers:

MyController {
static aspect {
beforeAll { println 'test' } // = before(call:'*') { }
beforeUpdate { println 'test' } // = before(call:'update') {}
}
}

and following for domain class dynamic methods:

MyDomain {
static aspect = {
beforeSave { }
beforeUpdate { }
afterUpdate { }
afterSave { }
beforeNew {}
}
}

To sum up here, what Groovy AOP should support in its aspect language are before*, after*, and around* to simplify the use of AOP in practice.

Groovy AOP - Redesigned #2

OK, I'm now continuing to show that what've I made changes to the syntax of Groovy AOP.
Last post, I've showed the new syntax, and there's a few more tweaks here.

In the trunk, now we can have aspect like this:

class MyClass {
static aspect = {
before(call:'update1') { println 'before update 1 ..'}
after(call:'update2') { println 'after update 2 ..' }
}
}

and this:

class MyClass {
static aspect = {
before('update1') { println 'before update 1 ..'}
after('update2') { println 'after update 2 ..' }
}
}

Both have the same semantic. They read, before the call of (join point of) MyClass#update1, do ..., and after the call of (join point of) MyClass#update2 do ...
I think these shortcuts will be comfortable enough for people to use without AOP knowledge.

The implementation's also changed internally to override MetaClassImpl#invokeMissingMethod rather than MetaClassImpl#invokeMethod.

Groovy AOP - Aspect Builder Redesign #1

I've just got Groovy AOP running first time with Groovy 1.1, and seriously I'm looking to integrate it with Grails when its 1.0 debuts. My problem now is still the classic AOP problem, how could we have an expressive and easy pointcut language for people?

I know that advantages of Groovy AOP to Grails are quite clear, but if and only if developers uses it. So I'm trying to re-design the pointcut language before getting 0.3 out.

class MyController {
static aspect = {
before(call:'*') { println 'something' }
before(withIn:'') { }
around(call:'update*') { inv ->
println 'before'
inv.proceed(inv.args)
println 'after'
}
def pc = call('update*') & within('main')
around(pc) { inv ->
println 'before'
inv.proceed(inv.args)
println 'after'
}
}

def update() {
}

def update2() {
}
}

Above is how we can embed "aspect" into Grails controller. I intend to leave the "static aspect" block to say that "hey, you're using AOP here". Don't know if people will have objection to this.

The shortest code we can use to have "before interceptor" for every method in the class is:

static aspect {
before(call:'*') { println 'before' }
}

Tomorrow, I'll continue with next thought about this.

Groovy equivalence to Chapel's code

Ability to have our own DSL in Groovy makes it possible to have a high-level language for parallel computing. I'm trying to clone all features of Chapel to Groovy and then optimise the generated codes to make it run (I hope) fast.

int n = 1000
def A = new Vector(size: n)
def B = new Vector(size: n)
forall(2..n-1) { i ->
B[i] = ((A[i-1] + A[i+1])) / 2
}

Hello Android, Are you wanting to sit on Grails?

Google's released Android SDK yesterday. After quick skimming on packages supported by the SDK, here what I can summarise,
- android packages, UI + 2D + 3D, SAX, DB, device APIs, OS calls
- java packages, java.nio, and java.concurrent (including atomic)
- httpcli
- JSON support
- cryptography, SSL
- Bluetooth support
- Google Map APIs
- XMPP

It seems to be not Java ME packages, but likely the striped down version of Java SE. So I think this could be possible to make quite complicate applications.
My first try here is not about making an app, but I'll see if I can have some templates to generate Grails scaffolding for it. If this can be done in a month, a full-blow application with Grails as a back-end will be easily developed. And it's going to be a big boosting up.

Does Grails need a real compiler?

We all know what a DSL is, so what's about a domain specific compiler (DSC)?

I've just accidentally seen Grails 1.0-RC1's been invoking its compiler during my work. So I think it's possible to have a DSC to support Grails in someway. It could be definitely based on the Groovy compiler with addition Grails knowledge. For example, in a Grails controller, we can use "render" method to render things like XML, JSON. What if we can compile this render block into a real XML, or JSON chunk embedded in the controller, rather than invoking the dynamic method.

In the development time, we've got Grails running fine and it offers us a great rapid environment. So we just keep using the original Groovy here. And if we have DSC, and it works just before grails war, then our Grails apps performance will be really great during its execution.

Convention over Configuration for Map-Ruduce

I'd just like to have a quick look to Hadoop's word-count example, if it can be written in Groovy.

Disclaimer: this code doesn't work !

class WordCountMapReduce {
def map = {key, value, output, reporter ->
def line = value.toString()
def itr = new StringTokenizer(line)
while(itr.hasMoreTokens()) {
word.set(itr.nextToken())
output.collect(word, one)
}
}

def reduce = {key, values, output, reporter ->
int sum = 0;
values.each {
sum += it.get()
}
output.collect(key, sum)
}

}

It's time for Groovy to go fore massive computation?

Profiling Groovy Meta-layer

I'm planning to profile the meta-layer of Groovy. What I'm looking for in this profiling is a number of arguments needed by methods of general applications.

Currently, ScriptByteCodeAdapter#invoke* alway use Object[] as a wrapper of a target method's arguments when dispatching it, and it's difficult for my JIT compiler to replace it with a new call, as suggested by Charles Nutter that my dataflow analysis may be time-consuming. So I came up with the idea of profiling.

There are invoke*0 and invoke*N in ScriptByteCodeAdapter class, and I think there would be invoke*1, invoke*2 ... invoke*K, where K is a threshold number found by the profiler. Then the class ASMClassGen would be patched to generate these newly added methods.

How does Groovy AOP relate to GIT?

GIT is the name of Groovy just-in-time compiler, which is being developed. I'm using it as an underlying layer for Groovy AOP.

MetaClassImpl, the default meta-class of Groovy, picks a call for each invocation, which is a joinpoint in AOP theory.

GIT is trying to optimise this invocation process by checking the change of a MetaClass of each class. It replaces the meta-call with the real call, which is returned by the MetaClass. Weaving process in Groovy AOP will take the similar approach. But when MetaClass picks a real call, it would return "WovenMetaMethod" that contains also "before", "after", and "around" information. The runtime weaver then uses this information to generate an equivalent list of bytecode instructions and use them to replace the meta-call.

This is how GIT relates to Groovy AOP.

An ERP built with Grails

I've setup the project called Geogia at Google Code for developing an ERP system using Grails. All domain classes are being re-written based on UML diagrams of project Neogia. Gentleware AG kindly sponsored Poseidon UML CE edition for this project. Thank you :).

This project is being developed using version 0.6 of Grails.

Xfire plugin for Grails 0.6 released

Nothing's fancy. I've just released my latest Xfire plugin for Grails 0.6.
It's now available from Grails plugins repository.

This command:

$ grails install-plugin xfire

is enough to give it a go.

A few note here, it uses Xerces 2.8.1, where Grails ships with 2.6.2. Just in case for any incompatibility.

Dynamic Recompiling a Groovy Class

After we identified an invariant dynamic call, the call replacement technique could be applied to the caller method. The following describes an early version of the algorithm to do so:

1. find an invocation statement to ScriptBytecodeAdapter
2. determine a number of parameters that the statement needs.
2.1 if the number of parameters = 3, this invocation has no argument.
2.2 if the number of parameters = 4, this invocation has a set of arguments.
3. statically unwrap the last parameter passed to this statement
(note that Groovy's using Object[]
to wrap a set of method arguments,
and we need to unwrap and pass them
to the new invocation statement)
3.1 in this step, we can also clean up unneeded the Object[] variable
and its assignment statements.
4. construct the new invocation statement,
using information gathered from the profiler.
(we can not infer the target class using
any information provided by the statement,
because the real call will be chosen by Groovy meta-class).
5. fix object casting, this can be done by traversing backwardly
to the statement prior to the invocation.
Then change casting type from GroovyObject
to the correct type (also get this value from the profiler)

The result so far:

$r19_2 = $r19;
goto label7;
label6:
$r19_2 = org.codehaus.groovy.aop.tests.Target.class$groovy$lang$GroovyObject;
label7:
$r20 = ScriptBytecodeAdapter.castToType(r0, $r19_2);
$r21 = (org.codehaus.groovy.aop.tests.Target) $r20;
$r23 = $r21.method2("test");
$r18[$i0] = $r23;
$r24 = ScriptBytecodeAdapter.invokeMethodOnCurrentN(r2, $r16, $r17, $r18);
return $r24;

Bytecode incompatibility

Last post, I've blogged about using Soot to optimise class files. Soot is alright for me when I used it as a generator. But in the dynamic context, I've got some compiled classes from the different generator (say Groovyc), but I'd like to get them optimised with Soot and then re-define then with the Intromentation. This approach completely failed as Soot-compiled classes have different attributes compared with the original classes.

After this problem's kept me busy for a couple of days, I've came with a solution to use Javassist, another bytecode manipulation to help me re-mix two different versions of class.

I got the original class from Javassist class pool, removed the old target method. Then I loaded the Soot-optimised class from disk, using Javassist's ClassFile, and took only the optimised target method from it. After that I created a new MethodInfo for that method with constant pool of the original class and then I added it to the original class. I finally obtained its bytecode and give it to Intrumentation for class re-definition.

soot.Main.main( new String[]{
"-d",
"mydir",
"-via-shimple",
className
});
try {
String fname = "mydir/"+ className.replace('.', '/') +".class";
ClassPool cp = ClassPool.getDefault();
CtClass target = cp.get(className);
target.defrost();
String mName = "main";
ClassFile srcClassFile =
new ClassFile(new DataInputStream(new FileInputStream(fname)));
MethodInfo srcMethodInfo =
getMethod(srcClassFile, mName, "([Ljava/lang/String;)V");
MethodInfo newMethodInfo =
new MethodInfo(target.getClassFile2().getConstPool(),
mName, srcMethodInfo, null);
target.removeMethod(target.getMethod(mName, "([Ljava/lang/String;)V"));
CtMethod newMethod = CtMethod.make(newMethodInfo, target);
target.addMethod(newMethod);
Instrumentation i = Agent.getInstrumentation();
i.redefineClasses(
new ClassDefinition[]{
new ClassDefinition(Class.forName(className), target.toBytecode())
});
} catch (Exception e) {
e.printStackTrace();
}

Tell me, if you have a better idea !!

Optimising Groovy bytecodes with Soot

I'm playing around Soot framework and get some optimised bytecodes after applying its default optimisation to Groovy-generated classes.
This does not help much for speed improvement, but I think it could help reduce the usage of PermGen space.

I've tested with a trivial class, and its file size after optimisation is 8.5% shrinker than the original. I'm expecting that this figure would be large if we optimise a whole application.

Possible JIT for Groovy with JVMTI

Nothing involves Groovy bytecode analysis at the moment, but I show you here the possible way to re-define a Java class with a Java 6 JVMTI agent. Yes, I say Java 6 as Java 5 does not support class re-definition through java.lang.instrument package.

As you might know, several AOP systems, that enable load-time weaving, use class transformation at load-time to weave aspects to the existing programs. They use a java JVMTI agent to achieve this.

Now in Java 6, you can do a bit more further as it offers method

Instrumentation.redefineClasses(ClassDefinition[] defs);

I'm experimenting this technique with a normal class, here:

package org.groovy.git.test;

public class TestClass {

public void myMethod(){
System.out.println("a");
}
}

The re-definition process is performed just to change a TestClass object from printing "a" to printing "b".

package org.groovy.git.test;

import java.lang.instrument.ClassDefinition;
import java.lang.instrument.Instrumentation;

import javassist.ClassPool;
import javassist.CtClass;
import javassist.CtMethod;

import org.groovy.git.Agent;

public class AgentTest {

public static void main(String[] args) throws Exception {
// get an instrumentation from the agent
Instrumentation i = Agent.getInstrumentation();

// here, print "a"
TestClass t1 = new TestClass();
t1.myMethod();

// doing bytecode manipulation,
// replace a new method body for "myMethod"
CtClass ct = ClassPool.getDefault().get(TestClass.class.getName());
CtMethod m = ct.getDeclaredMethod("myMethod");
m.setBody("{ System.out.println(\"b\");}"); // make it printing "b"
byte[] bytes = ct.toBytecode();

// prepare a new class definition
ClassDefinition[] def=new ClassDefinition[1];
def[0] = new ClassDefinition(TestClass.class, bytes);
i.redefineClasses(def);

// this shows "b"
TestClass t2 = new TestClass();
t2.myMethod();
}

}

You might see that the above code get instrumentation from the Agent class, which is in a jar loaded by -javaagent:jarfile. In this experiment, I use Javassist for bytecode manipulation.

As you expected, the above code will show:

a
b

as the TestClass has been re-defined and its method, myMethod, body's been replaced.

Groovy AOP coming soon on the desktop near you.

I've just committed the early Groovy AOP's grammar, its compilation unit, and ant task. Actually, everything is getting to work now, but I'd like to add some advice caching mechanism to have, at least, some performance impression.

I'm trying to have it released by the end of this month (July 2007).

If you'd like to see what's going on in the mean time, check out its SVN.

XFire plugin for Grails 0.5 released

I've released the XFire plugin for Grails 0.5 recently. Now it's using XFire 1.2.6 and
Its forked aegis engine has been updated as well. In Grails 0.5, the Artefact APIs for plugin development has been introduced. All Grails core artefacts (domain, controller, etc.) are moved to use these APIs.

This week seems to be a 0.5-plugin-release week. Beside my XFire, we now have Tsuyoshi's Acegi, Sigi's converters, and Peter's remoting plugins running for 0.5. Check them out on Grails Plugins Page.

JIT dynamic weaving for Groovy AOP

From the previous post, I was talking about selecting metaclass using pattern will give some benefits for AOP in Groovy. Now I come up with a cool idea of what is that benefit. The point here is about MOP infrastructure of Groovy is flexible enough by letting us to change the metaclass of any class at runtime, and this allow JIT (Just-in-Time) dynamic weaving of aspects for Groovy.

What does it mean by JIT dynamic weaving?

In my current implementation of Groovy AOP, there's a big overhead of matching pointcuts and invoking advices in the AOP metaclass, but this process can be improved by assigning a new, optimised, metaclass to the woven class. This new metaclass will be generated at runtime (using any bytecode manipulation framework) and it will contain a minimum set of instructions for invoking advices (no more matching process). However, there're still some important open questions like conflictions, etc. that need to be further investigated.

I'm pretty sure that this technique can improve some performance for Groovy AOP. Although, it's obviously slower than AspectJ, but it's worth comparing.

Groovy running fine on .NET

It's just a crazy idea to test Groovy on .NET, after I read this post on IKVM weblog that the author test IKVM using JRuby. If JRuby runs fine, probably Groovy does so.

Simple steps here,

1. Download .NET 2.0 Runtime.
2. Download SharpDevelop IDE 2.1, just in case that you're lazy enough to code.
3. Download IKVM for .NET 0.34 rc2 binary
4. compile your Groovy jar with the following command:

ikvmc -target:library groovy-all-1.1-SNAPSHOT.jar

Some errors will appear, just ignore them.
You'll get groovy-all-1.1-SNAPSHOT.dll.

5. Open SharpDevelop, create a new .NET console application.
6. Add references, all IKVM DLLs and your groovy DLL.
7. Open your main class, type the following code (in C# for example)

using System;
using System.Collections.Generic;

namespace GroovyDotNet {
  class MainClass {
    public static void Main(string[] args) {
      groovy.ui.InteractiveShell.main(args);
    }
  }
}

8. Run the program, and you'll also have an executable file as well.

Alright, if you don't want to get your hand dirty, do step 1 and wait for the binary. I'll upload it soon.

Update:
Here's the screen shot.


and here's the binary.

Patterns for selecting metaclass will allow better support AOP in Groovy

I'm still being with quantification thoughts for dynamic AOP.

There're some diffirences between dynamic and static weaving implementation. I'm saying about what AspectJ does, and what I'm going to do with Groovy AOP.

In AspectJ, it's focusing on static weaving for better performance. The idea behind is that every dynamic joinpoint has the corresponding static shadow, which is projected on source or bytecode of the target program. All quantifying will be done by AspectJ compiler to match every defined pointcuts, of course, at compile-time or load-time. Easy kinds of quantifiable joinpint are, for example, constructors and methods.

But nothing like this exists when we're talking about dynamic AOP. Dynamic AOP has nothing to do about 'static' shadows. This kind of weaving allows re-definition of aspect at run-time. Dynamic joinpoint in Groovy is powerful and a bit more complex to implement because of its MOP mechanism. Every class in Groovy has its own Metaclass to control the behaviour. ACtually, from AOP perspective, the metaclass can be though of as a basic per-target aspect. But when we're saying about AOP weaving, it allows multiple aspects to weave into a single target class. Comparing with MOP in Groovy, it's clear that AOP is somehow more powerful. In case of my AOP implementation, I have the AOP-metaclass to help dispatching advices.

So what's more ?

To fully support AOP quantification in Groovy, I came up with pattern matching idea to assign a metaclass for the target class. I'm feeling strange with this. I'm thinking of it as a good way to support pointcut-advice AOP model for Groovy. But there's something interesting about it that's beyond my head.

I hope I can figure it out soon.

Ubuntu 7.04

Ubuntu Linux 7.04 came out, I did upgrade and everything goes fine.
Many guys in Java community get excited that JDK 1.6 is already in it's repository. My OpenOffice got updated to 2.2 as well, and that's great.

I've also tried it in VMWare player and, of course, there's no correct driver for 7.04 at the moment. The problem I have is just about the mouse. But it's not a big deal, I tweaked XOrg.conf and set the driver back to the original one ("mouse" instead of "vmmouse").

My new Ubuntu experience has just started !

AOP for Groovy is here !! Code moved to be Grails-AOP

AOP for Groovy is here !

I've just refactored and uploaded my AOP subsystem into the Grails-Plugins repo. Anyway, it cannot say that this AOP implementation is Grails-Plugins. It's likely to be a drop-in module because it contains ObjectMetaClass that will hook into system-wide metaclass instantiation process.

This is an AspectJ-like, the pointcut-advice model implementation of AOP for Groovy. Both 2 important properties of AOP, Quantification and Obliviousness, has been implemented by design.

Browse the repo: http://svn.codehaus.org/grails-plugins/grails-aop/

The ways to express pointcuts, the choice is yours

From the non-AOP programmers' perspective, the pointcut mini-language in AspectJ seems to be a bit difficult to learn. Before they can start writing some aspects, they need to know background concepts about join points, the way how to quantify them with a combination of pointcut designators, and the way how to express advice codes.

I cannot argue this because making pointcut languages simple yet powerful is quite an active research in the AOP area.

Some workaround to solve this problem could be having different kind of way to express pointcuts in one framework. So this means we're going to have several mini-languages inside the host language for describing aspects. They are what I'm going to implement for my AOP subsystem for Groovy.

For example, call() PCD is a quite common one you often use, so having:

Test.around['test*'] = {} instead of around(): call(Test.test*(..)) { }

can reduce the learning curve.

But when, the power of the basic pointcut language is not enough, you can use the full version of pointcut language, like this:

static aspect = {
def p = pointcut{
pcall('public *':'set*') & pcall( ... ) &
within(MyClass:'some*')
}
around(p) { context ->
proceed(context.args)
}
}

The choice of expressing pointcuts is just yours.

Aspect Groovy: Per class and Per instance

From last post, I'm going to the next step by re-implementing my AOPSupportMetaClass again in Java to have some precise controls. Now the AOP metaclass got updated and support weaving both per class and per instance. However, the pointcut expression style is still based on John McClean's implementation.

This is for per instance weaving

def t = new TestObject()
t.around['save'] = { inv ->
inv.args[0] = inv.args[0] + '.'
inv.proceed(inv.args)
}
t.save('test')

And you can guess, this is per class weaving

TestObject.around['save'] = { inv ->
inv.args[0] = inv.args[0] + '.'
inv.proceed(inv.args)
}
def t = new TestObject()
t.save('test')

Last test code can be obtained from here.

Groovy and AOP Metaclass

I've recently found John McClean's last year article about AspectJ-like AOP model implemented in Groovy. In its comments, his readers posted that he should have a custom MetaClass to support his approach. I'm sure he probably did it because he's going to talk about AOP in Groovy at Grails Exchange conference this May. However based on his implementation, I've done the similar MetaClass as well, as I need some kind of AOP facility to support my research.

Probably after the Grails Exchange event, I can see his AOP code somewhere.

Here's my implementation:

package org.plaop;

import groovy.lang.DelegatingMetaClass

public class AOPSupportMetaClass extends DelegatingMetaClass {

def after = [:]
def before = [:]
def around = [:]

public AOPSupportMetaClass(MetaClass delegate) {
super(delegate);
initialize();
}

public Object getProperty(Object object, String prop) {
if(prop == 'around') return this.around else
if(prop == 'before') return this.before else
if(prop == 'after') return this.after else
return super.getProperty(object, prop)
}

public Object invokeMethod(Object arg0, String name, Object[] args) {
if(before[name]!=null) {
before[name](inv)
}

def inv = new InvocationContext(methodName: name, args: args, called: arg0)
inv.proceed = { arglist ->
super.invokeMethod(arg0, name, arglist)
}

def result = null
if(around[name]!=null) {
result = around[name](inv)
} else {
result = super.invokeMethod(arg0, name, args)
}

inv.proceed = null

if(after[name]!=null) {
after[name](inv)
}
return result
}
}

This is how to use it:

void testBasicAround() {
def mcr = InvokerHelper.getInstance().metaRegistry
AOPSupportMetaClass amc = new AOPSupportMetaClass(
mcr.getMetaClass(TestObject.class))
mcr.setMetaClass(TestObject.class, amc)

def t = new TestObject()
t.around['save'] = { inv ->
inv.args[0] = inv.args[0] + '.'
inv.proceed(inv.args)
}
t.save('test')
}

The big bug here is that this MetaClass hasn't yet supported per class weaving model. But it's not that hard to implement. John's article really realised me the power of Groovy MOP from AOP perspective.

Invariant Dynamic Calls

Basically, dynamically typed languages, Groovy in this context, dispatch calls by firstly choosing appropriate methods, then invoking them. The choosing process is obviously expensive. Several techniques, such as method caching, are currently implemented in Groovy to speed things up. However, choosing methods through reflection won't allow JIT to optimise method bodies for callers. With caching, the shortest steps for Groovy are still:

calling an invokeX method -> looking up cache -> calling the cached chosen method,

and you can not further optimise the invocation beyond this point.

The only way to do this is to simply replace that call by the chosen method call.
It's basic and everyone on the groovy-dev list knows about this (I supposed). Anyway, there're a lot arguments in the list that we should preserve the "dynamicity" of Groovy. I agree with those arguments, so we should leave Groovy core as it is, but we can really do somethings at the application level, in production phase.

During unit testing, developers may find sets of "invariant dynamic calls" of their applications. I roughly define an invariant dynamic call as a call that has only one chosen method. Trivial examples for this kind of call in Grails framework are Domain class' .save() and .delete(). If we assume that our test suites cover the application enough, so we can basically conclude that each dynamic call identified by the test suites is an invariant dynamic call, if there is only one chosen method for it. Then, we just simply replace these calls with static signatures, in class files or even by on-the-fly transformation, to gain performance back.

My current technique to analyse calls at bytecode-level is, firstly scanning for ScriptBytecodeAdapter.invoke*, then reverse looking for LDC "methodName" and other arguments that will be stored onto the stack.

Beyond Java 6
I dont exactly know when INVOKEDYNAMIC will arrive, but with this instruction replacing invariant dynamic calls will be much easier by
1. changing INVOKEDYNAMIC to appropriate kind of invoke, say INVOKEVIRTUAL, INVOKESPACIAL, INVOKESTATIC or INVOKEINTERFACE.
2. changing type descriptors to the correct classes.

I haven't followed the spec of INVOKEDYNAMIC at the moment, it's time to have a quick read again.

OpenLaszlo 4.0.0

OpenLaszlo 4 released a few days ago. I was impressed with its CSS support, although it's a pull (not push - like HTML's CSS) style. Anyway I've found some bugs that prevented me from compiling an LZX program for the DHTML runtime. Hopefully, these bugs will be get fixed soon.

Another thing is that styles, which are working well in Flash runtimes, doesn't work for the DHTML runtime as well. However, these bugs don't stop me from using this great software !!!

Get OpenJDK Hotspot Compiled !

I'm very newbie on hacking JVM and I'm very happy to get OpenJDK Hotspot B10 compiled on Ubuntu 6.10. Compilation time is 1 hour and 5 mins. Actually, what I'm wanting to do from now on is to add some modification into its interpreter mode. Everything's just started.

Some configuration
- OpenJDK Hotspot VM 7-EA-B10 (21 Mar 07)
- Importing JDK 1.6_01
- Ubuntu 6.10
- GCC 4.1.2 (Ubuntu 4.1.1-13ubuntu5)
- Linux 2.6.17-11

Flattening Xml to Map using depthFirst

It's very wonderful and extremely easy to use groovy.xml.XmlParser to flatten an XML document into a Grails compatible Map:

def Map xmlToMap(String xml) {
def root = new XmlParser().parseText(xml)
def map = [:]
root.depthFirst().each { n ->
if(n.value.size == 1) {
def key = n.name
def p = n.parent
while(p.parent != null) {
key = "${p.name}.${key}"
p = p.parent
}
map[key] = n.value[0]
}
}
return map
}

Multiple Assignment soon in Groovy 1.1

Beside other cool features, the last entry in GDC 3 report about what's going to be added into Groovy 1.1 is multiple assignment. I'm definitely looking forward to use it !!!

GORM and Data Grid

Graeme Rocher revealed the possibility for GORM to support data grid on top of Terracotta or Tangasol Coherence.

This's going to be a big impact, at least, from my research perspective.

Groovy is the most interesting JVM-based non-Java language

By Java.net poll, Groovy gets 27.3% and becomes the most interesting non-Java language running on JVM. However, JRuby ranks no. 2 with 20.9%.

This survey gives a good picture of Groovy, and of course Grails, momentum.

Configuring Grails datasource with a .properties file

A Grails user needs to configure Grails data sources with an external .properties file (posted here), but having Spring beans inside "resources.xml" for it won't work for some reasons. I was trying to investigate what's going on, and found that there're steps done by Grails Hibernate plugin to retrieve references of datasouce before some beans defined in "resources.xml" are processed.

Anyway, I like the concept of using .properties file as well. So I modified *some*DataSource.groovy to see how can I use .properties to fill the data source properties, and here's the code:

import java.util.Properties
import org.springframework.core.io.FileSystemResource

class DevelopmentDataSource {

def static propFile = "web-app/WEB-INF/conf.properties";

boolean pooling = true
String dbCreate = "create-drop"

String url
String driverClassName
String username
String password

public DevelopmentDataSource() {
def props = new Properties()
try {
props.load(new FileSystemResource(propFile).inputStream)
} catch(Exception e){
e.printStackTrace();
}
this.driverClassName = props.getProperty("jdbc.driver","org.hsqldb.jdbcDriver")
this.url = props.getProperty("jdbc.url","jdbc:hsqldb:mem:inDB")
this.username = props.getProperty("jdbc.username","sa")
this.password = props.getProperty("jdbc.password","")
}
}

I know that it's not an elegant solution for the moment, but it works :).
With this modified datasource, you use the "propFile" to point to some file outside .WAR, and have some better level of abstraction.

XFire plugin changed convention and upgraded to XFire 1.2.4

I've patched the XFire plugin to 1.2.4 and changed some convention to be more compatible with Grails.
Now you can just tweak your Grails services to be Web services by adding the static property 'expose'.
Here's an example:

class TestService {

static expose=['xfire']

def String doSomething(DomainClass dc) {
return "done"
}

def String getData() {
return "data"
}

def void setSomething(String something) {
// set some values here
}
}

The current build is going to skip all properties in a service, including transactional.
However, it's smart enough to expose some methods like 'getData' or void 'setSomething' correctly.
The policy used to expose methods in this build will detect existence of both "getter" and "setter" for a property.
Because in Groovy and Grails, we usually have property by simply defining a property with the 'def' keyword.

BTW, I was trying to compile it using JDK 1.4 but nothing works.
Some type casting exception has been occurred inside XFireSpringServlet and I cannot fix it yet.
So please stay tuned if you're going to use this plugin with Grails on 1.4 VM.

I've finally done XFire plugin for Grails

Last post, I mentioned about Grails plugin architecture in 0.4. This great feature allows us to develop our own plugins. I firstly thought that it might be difficult to develop my own one. But after having a look into some plugin codes in Grails itself, and another one - OpenLaszlo plugin. I finally got some ideas about how the plugin works, and what to do.

However, I was suffered a bit by bugs from Spring Bean Builder because it doesn't provide complete APIs to cover methods I need, such as "bean.initMethod". Anyway, it's not too hard to figure it out, and I have some workaround to avoid these bugs. I'm planning to fix them soon after my XFire plugin is stable for Grails 0.4.

So, what's about the XFire plugin for Grails ?
It's a plugin to expose a Grils service as a Web service. Only requirement is that the service must have "XFireService" suffix. The plugin scans pulbic methods of the service, and generates complex type definition for parameter and return types using a modified XFire's Aegis engine.
For a Grails domain class, this plugin will generate type information for every properties, including "id", but it will ignore "version" and other Groovy specific properties.
If you want to exclude some methods, you can define "static excludes = []" in the service.

Here's some example:

class MyXFireService {

def static excludes = ["notMe"]

def String myMethod(MyDomain d) {
d.save()
return "test"
}

def String notMe() {
return "nothing"
}
}

I've added a page for this plugin at Grails space here:
http://grails.org/XFire+plugin

XFire for Grails Applications - the pre-plugin Era

I've updated my Grails from 0.3.1 to 0.4 snapshot and its new coming plug-in architecture looks very promising. I'm hoping that someone will implement an XFire plugin for Grails soon (mentioned some where in Grails' wiki). Anyway, I'm going to talk about how to integrate XFire into an Grails application manually.

Here, I briefly show you steps to integrate annotation-based XFire Web services into your Grails Applications.

1. download XFire distro (I'm using 1.2.3).
2. put everything (.jar files) of XFire into ${YOUR_GRAILS_APP}\lib
3. create your Web service class in src/java
4. modify resources.xml
5. modify web.template.xml
6. grails run-app

It's still configuration-based approach, not conventional yet.
Anyway, this might be useful to someone, who currently need to use XFire, when the XFire plugin still be an imaginary thing.

Note that, this covers only HTTP-based transportation of XFire, since Grails is mainly for Web.

Let's go into details:
Before starting, you have to put all XFire .jar files into your /lib folder first.

So, you can start by put your annotation-based classes in "src/java" folder.

@WebService
public class MyService {

@WebMethod
public String doSomething() {
return "hello";
}

@WebMethod(exclude=true)
public void hideMePlease() {
return;
}

}

what you needs are modification of "resources.xml" for spring, and "web.template.xml".

OK, then add some XML chunks into your spring/resources.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN"
"http://www.springframework.org/dtd/spring-beans.dtd">

<beans>

<!-- You need this line to import "xfire" and
"xfire.transportManager" references in -->
<import resource="classpath:org/codehaus/xfire/spring/xfire.xml" />

<!-- Your service goes here -->
<bean name="myService" class="org.codehaus.xfire.spring.ServiceBean">
<property name="xfire" ref="xfire"/>
<property name="serviceBean" ref="myService"/>
<property name="serviceClass" value="MyService"/>
<property name="serviceFactory" ref="jsr181"/>
</bean>

<!-- Here's initialising an "jsr181" instance to be
a factory object for your services -->
<bean id="jsr181"
class="org.codehaus.xfire.spring.config.ServiceFactoryBean"
init-method="initialize">
<constructor-arg index="0" value="jsr181" />
<property name="transportManager" ref="xfire.transportManager" />
</bean>

</beans>

Finally, put some servlet configuration into your "web.template.xml", like this

<servlet>
<servlet-name>XFireServlet</servlet-name>
<display-name>XFire Servlet</display-name>
<servlet-class>
org.codehaus.xfire.spring.XFireSpringServlet
</servlet-class>
<load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
<servlet-name>XFireServlet</servlet-name>
<url-pattern>/servlet/XFireServlet/*</url-pattern>
</servlet-mapping>

<servlet-mapping>
<servlet-name>XFireServlet</servlet-name>
<url-pattern>/services/*</url-pattern>
</servlet-mapping>

OK, let's type "grails run-app" and browse to "http://localhost:8080/${your_grails_app}/services/myService/" to see its WSDL link.
And that's all, your XFire service is now ready.