Syntax of Type Sheet

It's beautiful, isn't it?

class(C) => { field(f) => f: String; method(m) && args(a) => { // bound a local(v) => v: int; // order of binding -> is left to right local(n) && call(Integer.minus(_)) && this(n) => return: int; call(print(n)) && match(n) { case call(Integer.plus(_)) => n: int; default => n: String; } } }

Online Twitter App written in Grails

It is very exciting to tell you that we has now successfully used Grails to clone Twitter's Election. It seems to be a very first online Twitter application written in Grails. The most important thing is that the prototype has been done in just 2 days and we can have 2 Grails applications (a crawler with Quartz plugin to collect tweets, and a front to serve users) working together. Now it serves for the coming Bangkok Governor Election.

It is currently hosted by our lovely cloud, Morph Appspace.

Try it here: http://bangkok51.morphexchange.com

A non-English BDD framework

I am a big fan of easyb, a BDD framework for Groovy and, of course you can use it to validate spec of your Java programs as it's running on JVM.

But what I really want to have is a support of my native, Thai, language. Recently found out that Groovy can compile UTF-8 code, and I have no problem to make it invoke some method with Thai characters. So, I have been porting the idea from easyb into my own, non-English BDD framework, TSpec.

Now you can tell a story of software specification in Thai, wanna learn it ?

Here's at Github.

Hangman in ZKGrails

Actually, it's not for your eyes, but Robert Lee has found my hangman source code via Pastbin. Then he posted it on DZone, but what he mentioned is not 100% correct. This hangman app is built on ZKGrails, which is a ZK plugin for Grails (not only Groovy - check the language tag there in the source code it's 'GroovyGrails', not 'Groovy').

Hangman is the first challenge among us, a group of PAW66 developer sites in Thailand.(rails66, grails66, django66, seam66 etc.) We set this program up to show the power of each Web framework we love. I have blogged about this hangman in Thai on Grails66.com, if you can read. That's why I said it's not intended for your eyes.

One more thing I'd like to mention is that I host this app with Morph AppSpace. It's really cool that not only you can host Rails apps, all kind of my Java apps including Grails and even ZK just work there. You did the great job, guys !

Trust me, Grails will be a platform

Recently, SAP has released its Composition on Grails 1.0, the world first Grails distribution outside the project itself.
Yes, I said it is "the world first Grails distribution". This is really important to me as the release sent out a special signal that "Grails" will be "something" more that a normal web framework on JVM.

Thanks to Grails plugin architecture, you are able to get its core (thinking of it as a Linux kernel), put some plugins and scripts (think of them as programs around Linux), pack them together and that's it ! You got a web framework distribution that contains special features !

For example, imagine that you can have a Blog-ready Grails distro, a distro that contains some preset plugins, which help you create a blog like Wordpress in minutes. Or you can have a CMS-ready Grails distro, which enable you to 'grails create-cms' and you've got a Drupal-like system.

This will be a big step to take Grails into the next level ! Trust me, it's going to be a platform !

Hey Guillaume Laforge, Thai Geeks are going to interview you

After Groovy has been got the big attention in Thai community this moment, I've been co-operating with Isriya of Blognone.com (a leading technology news site in Thailand) to arrange an interview for Guillaume Laforge. This interview will be a community-based. All question will come from news readers, which of course are Thai geeks. After collecting questions there, I'll forward them to Guillaume via email.

This will be the first time for them to get in touch with him.

See the post (if you can read Thai): http://www.blognone.com/node/8770

Do anything at BarCamp Bangkok 2

To help promote BarCamp Bangkok 2, here's my screencast:

http://screencast.com/t/JRZprfG6

Enjoy !

A Java-near-speed Groovy

Almost 2 months that I have not posted here. I'm still in the final phase of Google Summer of Code. The world is going to be wonderful when you have got a really good result from your hard work, is that right ?

I'm currently be able to make Groovy speed step closer to Java. It's a high aim, and it's clear to be possible with JVMIT now. I was working on optimising callsites, but could not find a way to make it faster than Alex T.'s implementation because he did really good job of implementing it. So I went back to look at my old GJIT code. What I found is that something had been fooling my eyes for almost a year. There is a way to avoid JVM 6 specific code to implement an agent for JVM 5. Then I re-read AspectJ source again, and found something useful. I has started to re-write a GJIT agent for JVM 5 from then.

My prototype of the second generation of GJIT was done with Soot framework, where I really learned a lot of useful optimisation tricks (e.g., SourceValue). Later, I found that Soot is not suite for implementing JIT, because of 1. it's somewhat big structure 2. it's class loading problem. I then have had to implement a whole thing using ASM. It really was a nightmare because there is no Jimple to help your analysis. I re-wrote at least 3-4 version of ASM optimisers, and finally got the current one, which has a good enough structure to patch.

Now my goal is to beat all shootout benchmarks. Here's the result from partial sums benchmark compared with Java on my machine.

$ export JAVA_OPTS="-server"
$ time `java -cp bin alioth.PartialSumsJ 5000000`
real 0m10.246s
user 0m0.030s
sys 0m0.015s

$ time `groovy.bat test/partial_sums.groovy 5000000`
real 0m27.110s
user 0m0.030s
sys 0m0.016s

$ time `groovy.bat test/alioth/PartialSums.groovy 5000000`
real 0m28.409s
user 0m0.030s
sys 0m0.015s

$ export JAVA_OPTS="-server
-javaagent:C:\\groovy-ck1\\healthy\\target\\install\\lib\\gjit-0.1.jar"
$ time `groovy.bat test/alioth/PartialSums.groovy 5000000`
real 0m13.434s
user 0m0.030s
sys 0m0.030s

It's 200% faster than current Groovy, and ~20-30% slower than Java. There's still room to go. I hope to take some more steps closer to Java's speed.

Expression-Wide Optimisation (Primitive Callsite #2)

This is a technique I proposed for optimizing Groovy primitive expressions.

Let's say, we have:

public double a(int i, int j) {
return 1 / ((i+j)*(i+j+1)/2.0D +i+1);
}

This expression consists of 7 callsites. If we can make sure that the following 3 conditions
1. All variables are primitive.
2. The metaclass of each callsite is default
3. The meta method of each callsite is default
are true, the we can replace the entire expression with primitive and native Java bytecodes. Thus, we can by-pass any overhead here.

When some meta-method has been replaced, then the above condition is invalid. The original code will be performed instead of the fast one.

This technique is reasonable to implement because:
1. Callsite makes this checking cheap. We can define "isDefault" flag for a "default" callsite.
2. Instead of checking the default flag one-by-one, checking these flags for a whole expression makes this even cheaper. This also results in easier code generation.

However, we have to generate code twice, say fast and slow paths, for each expression-to-be-optimised.

final boolean exprCallSet = $getExprCallSet(0);
if(exprCallSet) {
return 1 / ((i+j)*(i+j+1)/2.0D +i+1);
} else {
$reevaluateExprCallSet.get()[0] = true;
return DefaultTypeTransformation
.doubleUnbox(ScriptBytecodeAdapter.castToType(
acallsite[0].call($const$0, acallsite[1].call(
acallsite[2].call(acallsite[3].call(
acallsite[4].call(acallsite[5].callOII(i, j),
acallsite[6].call(acallsite[7].callOII(i,j),$const$0)),
$const$1),
DefaultTypeTransformation.box(i)),
$const$0)),
$get$$class$java$lang$Double()));
}

The above snippet is a mock code I'm working on. The next step is to generalise this and modify the Groovy class generator. With this technique, we can unleash all power of Java primitives which will be > 800% faster than the normal callsite caching in 1.6-beta1.

Primitive Callsite Again (Part 1)

Current Groovy implementation always do boxing, unboxing regardless types of primitive parameters. This is the reason why it's slow down for arithmetic operations. I've come up with some experiments to optimise primitive for Groovy.

The idea is that I'll generate primitive callsite, or primitive calls when:

1. typing is obvious. For example,

int a = 5
int b = a * (a + 5)

in the above example, the second expression will have 1 box/unbox call and 1 primitive call because of type of (a+5).

With (a +5), this can be generated as Object call((int)a, (int)5) because type of a is obvious, also '5'.
But we won't know type of the result. So that a * (...) will be generated using Object call ((int)a, Object) rather than Object call(int, int).

One may argue that this might be not worth, as for a very long and complex expression this optimisation will be useless. For example,

private double a(int i, int j) {
return 1 / ((i+j)*(i+j+1)/2.0D +i+1)
}

The above optimisation can do it job only just for 2 inner most expressions. But you will be surprised that it's actually ~23% faster ! Thus IMHO, it's worth to do that.

2. This second part is to trying to optimise that long expression. The entire expression can be replaced with all primitive calls if only if their meta-class and meta-methods are "DEFAULT". I hope I could detect this via a property of CallSite, then generate 1. fast path, 2. original path. This second idea has not been fully experimented yet. So wait and see what can be done.

Proof-of-Concept Backport Invokedynamic in Groovy

It is my first day of Google Summer of Code, and here is what I've done today ;-)

I have been reading hard and trying to understand Invokedynamic EDR since last week. It is quite difficult for me, who has little background of this kind of things. However, I have got some good sources for studying how invokedynamic work from John Rose's blog, Alex Tkachman's Method Handle code, Jeroen Frijters's blog, and Rémi Forax's blog.

Although, there is no something complex like Jeroen's sample, here is a working version of Hello World that uses my backport of invokedynamic APIs.

package groovy.dyn.subject.vm5;

import groovy.dyn.CallSite;
import groovy.dyn.Linkage;
import groovy.dyn.MethodHandle;
import groovy.dyn.MethodHandles;
import groovy.dyn.MethodType;
import groovy.dyn.StaticContext;
import groovy.dyn.internal.CallSiteImpl;
import groovy.dyn.internal.StaticContextImpl;

import java.io.PrintStream;
import java.lang.reflect.Method;

public class InvokeDynamicSubject {

private static CallSite[] callsites;
private static final int NUM_OF_CALLSITES = 1;

static {
initCallSites();
try {
Method method = InvokeDynamicSubject.class
.getDeclaredMethod("bootstrapInvokeDynamic",
CallSite.class, Object.class, Object[].class);
MethodHandle mh = MethodHandles.unreflect(method);
Linkage.registerBootstrapMethod(InvokeDynamicSubject.class, mh);
} catch (Throwable e) {
// do nothing now
}
}

private static void initCallSites() {
callsites = new CallSite[NUM_OF_CALLSITES];
callsites[0] = new CallSiteImpl(0,
new StaticContextImpl(
PrintStream.class,
"println",
MethodType.make(void.class, String.class)
)
);
}

private static Object bootstrapInvokeDynamic(CallSite cs,
Object receiver, Object[] args) throws Throwable {
Method method=null;
CallSiteImpl csi = (CallSiteImpl)cs; // down cast: language specific
switch(csi.index()) {
case 0: {
StaticContext sc = cs.getStaticContext();
Class<?> c = sc.getCallingClass();
method = c.getDeclaredMethod(
sc.getName(),
sc.getType().parameterType(1));
MethodHandle mh = MethodHandles.unreflect(method);
cs.setTarget(mh);
return mh.invoke(receiver, args);
}
}
return null;
}

public void test() throws Throwable {
// System.out.println("Hello World");
MethodHandle mh = callsites[0].getTarget();
if(mh == null) {
bootstrapInvokeDynamic(callsites[0], System.out, new Object[]{"Hello World"});
} else {
mh.invoke(System.out, "Hello World");
}
}
}

Groovy, Bazaar and bzr-svn

In the past, I was wondering how could I manage my Groovy fork (SVN is not good to do that because it's centralized nature). I was thinking to use SVK, but failed to set it up.

Today, Russel posted to groovy-dev that he set up a branch of Groovy on Launchpad. I have heard about Launchpad, but never tried to be its user. More that 2 hours that I have tried to set everything up.

I registered to be a Launchpad user, then forked my branch from Russel's one. OK, then what's next?
I'm still on Windows XP, so I need a set of putty programs for SSH. Later, I use PUTTYGEN to generate my public key and import it into Launchpad.

The next step is to download Python 2.5, Bazaar for Windows. Then, I installed bazaar and was able to branch Russel's trunk locally.

bzr branch http://bazaar.launchpad.net/~russel/groovy/trunk

After that I pushed it back to my branch.

bzr push bzr+ssh://chanwit@bazaar.launchpad.net/~chanwit/groovy/ck1

Then, something went wrong. I did not setup SSH properly. bazaar blamed me that I forgot to set BZR_SSH.

set BZR_SSH=plink

OK, here we go. Now everything went smoothly. I can push my local codes back to Launchpad. So I did clean and re-build Groovy to see if everything was fine. But hey !, the branch of Russel is out-of-date. I need the latest patch Alex just committed. So what to do?

I checked some docs on bazaar wiki and they said it is able to merge code from SVN using bzr-svn plugin. That sounds great to me. So I downloaded bzr-svn, but thing did not go smooth again. I need Python-Subversion binding with the special patches. Come on, where on the earth it is? Alright ! It seems I got everything to run bzr-svn.

I ran this command:

bzr merge http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/

but it was not successful. The document said it should be svn+http. How silly I am. I tried merging again:

bzr merge svn+http://svn.codehaus.org/groovy/trunk/groovy/groovy-core/

And it worked ! After that I did commit.

bzr commit -m "message goes here"

and push all committed code back to Launchpad again:

bzr push

Wow, it was fun and cool!. Now I can have my experiment branch without missing any new patch from the main trunk! That's great, isn't it?

My SPLAT 2008 Homework

Using Groovy AOP to express business logics in Grails framework

This document describes a number of problems found in the scenario of applying Groovy AOP to Grails framework that reflects Comprehensibility.

From http://grails.org,

"Grails aims to bring the "coding by convention" paradigm to Groovy. It's an open-source web application framework that leverages the Groovy language and complements Java Web development. You can use Grails as a standalone development environment that hides all configuration details or integrate your Java business logic. Grails aims to make development as simple as possible and hence should appeal to a wide range of developers not just those from the Java community."

There are high-level artefacts e.g., Controllers, or Domain classes in Grails framework. An AOP language needs a good design to cover high-level pointcuts for expressing such software artefacts in Grails. Moreover, Grails is extensible via its plug-in system. A developer can define their own artefacts in a newly developed plug-in. For Grails, the AOP language needs to be extensible to capture join points introduced by these artefacts.

For example, Groovy AOP comes with a set of primitive pointcuts like:
- pcall(pattern)
- etc.

But these do not reflect the semantics of Grails artefacts. Their users need some PCDs like:
- controller(pattern)
- action(pattern)
- etc.

Here is an example of a web application written using Grails.

class BookController {
def list = {
def books = Book.list()
render(view:'book', model:[books: books])
}

def show = {
...
}

def update = {
...
}
}

An aspect written in Groovy AOP:

aspect SecurityAspect {
def pc = pcall('BookController.*')
around(pc) {
// check access
}
}

An aspect written in Groovy AOP, with a higher level of PCDs:

aspect SecurityAspect {
def pc = controller('book') & action('*')
around(pc) {
// check access
}
}

An aspect written using a variant syntax Groovy AOP, with a higher level of PCD (shortcut form):
This syntax is probably preferred by the developers, according to idioms they are familiar with.

aspect SecurityAspect {
around(controller:'book', action:'*') {
// check access
}
}

Groovy AOP also needs to support a good way to be able to extend and cover a new software artefact added through Grails plugin system as well.

Conclusion

As a general purpose AOP system for Groovy, Groovy AOP can be used with Grails framework. It may help expressing business logics as application-level aspects. However, the implementation of Grails framework is dynamic because a developer can add a new software artefact by creating a plug-in. To make an AOP system meets this requirement, it should support the ability to define new PCDs through the same plug-in architecture.

Ability to understand by developers ?
- A system should provide idioms that developer are familiar with in the AOP language
Ability to evolve & customize ?
- A system should provide the way to define new PCDs
How well does AOSD support/go along with Comprehensibility?
- Depends on its implementation, and its base language (e.g. Java / AspectJ, Groovy / Groovy AOP). Currently, Groovy AOP has not support this approach well yet.

Composite Id and FK as PK in GORM

I've been discussing with my colleage about how to map 2 classes in GORM. It's a bit complicated, where we need to use the foreign key from one, Process, class to be the primary key for another class, State.

Code listing as follows:

class Process implements Serializable {

String name
static hasMany = [states: State]

}

class State {

String stateId
Process process

static mapping = {
version false
id composite:['stateId','process'], generator:'assigned'
}

}

Don't forget to use obj.save(insert:true), when you have a GORM mapping with version=false and generation='assigned'.

You see, life is much more easier with GORM.

G2One's released Grails 1.0.2

The team has just released the simple but powerful Web framework on JVM, Grails 1.0.2. I've also uploaded its Installer for Windows. Check them out.

G2One Inc (http://www.g2one.com) and the Grails development team are
pleased to announce the release of Grails 1.0.2, which includes 84 bug
fixes and improvements. The release is available to download from http://grails.org/Download

Fixes in this release seem to focus on GORM component.

From: Grails.org

OpenLaszlo 4.0.10 released

OpenLaszlo 4.0.10 is released today. It seems to contain a lot of bug fixed (compared with 4.0.9). This version is getting closer to 4.1, which will be the production version that supports DHTML runtime.

I'm really looking forward to use 4.1, but now it's time to upgrade one of my production application to use 4.0.10.

Source: OpenLaszlo.org

Grails Popularity Indicator?

Just got this nabble link from Graeme's blog. It's sure to show how much popularity of Grails to date.

Powered by ScribeFire.

Matrix computation environment over Hadoop/HBase

I've been forming up the Hama project with Edward Yoon.
He's been doing the really great job of getting matrix computation work over Map/Reduce pattern. Now we are planning to have a MATLAB-like environment for them.
Here's a number of issues:

- The language needs to support lazy evaluation,
for re-arragement of a numerical expression to properly pick the right algorithm before computation.

I love Groovy, so it's time to develop the Groovy-derived langauge that suites numerical computing.

- Support BigDecimal, BigInteger by default.
Yep, we're again looking on large-scale computation, and sometime it's beyond the integer's boundary.
So we need the built-in support for these kinds of data type.

- Matrix and Vector to be the first class elements. For example,

def a = [1 2 3] // this is going to be a Vector
def b = [1 2 3; 4 5 6; 7 8 9] // this is going to be a matrix

while,

def a = [1, 2, 3] // this is going to be an ArrayList
def a = [a:"a"] // this is going to be a HashMap

- Java integration in the same spirit as Groovy
So it should compile to bytecode.
Compile code will be exactly Java class, not like e.g., JRuby compile .class, in which contains a lot of specific magic

- Support Closure
An anonymous closure will be inlined, whilst a normal closure will be encoded as an inner class.

More to come ...

Dataflow Analysis with ASM 3.1 and StackMap

Very surprised that I've got a huge benefit from stack map feature of Java 6.

I'm trying to do some dataflow analysis over compiled Groovy classes, and what I've found in ASM 3.1 is that it comes with method visitor for doing this for me. I've just added some partial evaluation for constants (for almost kinds of JVM instructions with a constant as their operand) and when the visitor visits method instruction, like INVOKESTATIC, I've got type information of each wrapped arguments (Groovy uses Object[] to hold them for dispatching the call with ScriptBytecodeAdapter).

Now my new unwrapping meta-method algorithm is blazing fast. Thanks ASM guys !!.

Application-Specific Performance Tuning for Groovy

You all know that if we bypass meta-layer of Groovy, the performance will be boosted. I have 1 class, 1 closure (as an inner class) and 3 method calls in the following codes (System#currentTimeMillis, times, and println):

package org.codehaus.groovy.aop

class Test {
static void main(args) {
def s1 = System.currentTimeMillis()
1000000.times {
int i = 10
println "test ${i}"
}
def s2 = System.currentTimeMillis()
println "time: ${s2-s1}"
}
}

I have 100% confidence that those calls won't be dynamic. Then I can have an aspect (written in AspectJ) applied to them for the performance reason. Although, it's quite complicated piece of codes here, but at least your application will be faster.

public aspect TestBooster {

pointcut inv_println(String methodName, Object[] params):
call(static * ScriptBytecodeAdapter.invoke*Method*N(..)) &&
(
withincode(public static void org.codehaus.groovy.aop.Test.main(..))
|| withincode(public Object org.codehaus.groovy.aop.Test$_main_closure1.doCall(..))
)
&& args(..,methodName,params)
&& if("println".equals(methodName));

Object around(String methodName, Object[] params):
inv_println(methodName, params) {
DefaultGroovyMethods.println((Object)null, params[0]);
return null;
}

pointcut inv_times(Object self, String methodName, Object[] params):
call(static * ScriptBytecodeAdapter.invokeMethodN(..)) &&
(
withincode(public static void org.codehaus.groovy.aop.Test.main(..))
)
&& args(Class,self,methodName,params)
&& if("times".equals(methodName));

Object around(Object self, String methodName, Object[] params):
inv_times(self, methodName, params) {
DefaultGroovyMethods.times((Number)self, (Closure)params[0]);
return null;
}

}

The above aspect is for by-passing calls to "times" and "println" with the replacement of the real targets. The woven code is averagely 15% faster (in both client and server VM) because the call distance is shorten from 36 levels to 17.
With this technique, you can apply the optimisation in the way you want just before you deploying the app (without touching your source). Anyway, don't forget to do the regression tests after the optimisation ;-).