Tuesday, February 05, 2008

Matrix computation environment over Hadoop/HBase

I've been forming up the Hama project with Edward Yoon.
He's been doing the really great job of getting matrix computation work over Map/Reduce pattern. Now we are planning to have a MATLAB-like environment for them.
Here's a number of issues:

- The language needs to support lazy evaluation,
for re-arragement of a numerical expression to properly pick the right algorithm before computation.

I love Groovy, so it's time to develop the Groovy-derived langauge that suites numerical computing.

- Support BigDecimal, BigInteger by default.
Yep, we're again looking on large-scale computation, and sometime it's beyond the integer's boundary.
So we need the built-in support for these kinds of data type.

- Matrix and Vector to be the first class elements. For example,


def a = [1 2 3] // this is going to be a Vector
def b = [1 2 3; 4 5 6; 7 8 9] // this is going to be a matrix


while,


def a = [1, 2, 3] // this is going to be an ArrayList
def a = [a:"a"] // this is going to be a HashMap


- Java integration in the same spirit as Groovy
So it should compile to bytecode.
Compile code will be exactly Java class, not like e.g., JRuby compile .class, in which contains a lot of specific magic

- Support Closure
An anonymous closure will be inlined, whilst a normal closure will be encoded as an inner class.

More to come ...

1 comment:

rchatsiri said...

Wow!last time,I read array list on the text book but not have example about matrix,hash- -''.