Tuesday, February 05, 2008

Matrix computation environment over Hadoop/HBase

I've been forming up the Hama project with Edward Yoon.
He's been doing the really great job of getting matrix computation work over Map/Reduce pattern. Now we are planning to have a MATLAB-like environment for them.
Here's a number of issues:

- The language needs to support lazy evaluation,
for re-arragement of a numerical expression to properly pick the right algorithm before computation.

I love Groovy, so it's time to develop the Groovy-derived langauge that suites numerical computing.

- Support BigDecimal, BigInteger by default.
Yep, we're again looking on large-scale computation, and sometime it's beyond the integer's boundary.
So we need the built-in support for these kinds of data type.

- Matrix and Vector to be the first class elements. For example,

def a = [1 2 3] // this is going to be a Vector
def b = [1 2 3; 4 5 6; 7 8 9] // this is going to be a matrix


def a = [1, 2, 3] // this is going to be an ArrayList
def a = [a:"a"] // this is going to be a HashMap

- Java integration in the same spirit as Groovy
So it should compile to bytecode.
Compile code will be exactly Java class, not like e.g., JRuby compile .class, in which contains a lot of specific magic

- Support Closure
An anonymous closure will be inlined, whilst a normal closure will be encoded as an inner class.

More to come ...


~inSiderboy said...

Wow!last time,I read array list on the text book but not have example about matrix,hash- -''.

wow power leveling said...

Youth is warcraft leveling not a time of life;warcraft leveling it is a wow lvl state of mind; wow power level it is not power leveling amatter of World of warcraft Power Leveling rosy cheeks, red wrath of the lich king power leveling lips and supple knees;WOTLK Power Leveling it is a matter of thewill,wlk Power Leveling a quality of buy aoc gold the imagination,aoc gold a vigor of the emotions; it is thefreshness of the deep springs of life. Youth means a tempera-mental maplestory mesos predominance of courage over timidity, of the appetite formaple story mesos adventure over the love of ease. This often existsin a man of 60 more than a boy of 20. Nobody grows old merely by anumber of years.