Would be Glassfish v3 a better Darkstar?

I reviewed last week the Sun Gaming Server, also called Darkstar. The architecture is interesting. The server supports "ManagedObjects" which are controlled by the server. A ManagedObject is thread safe (from the developer perspective it looks like a monothreaded instance) and has access to resources like Channels (for Pub/Sub communication), and almost POJO-based persistence. The Darkstar Server is also transactional, and provides support for user-sessions. Actually from the conceptual point of view Darkstar is very similar to an Java EE Server, even the programming model and the best practices are almost identical. It seems like Darkstar was developed from scratch, so it does not reuse the Grizzly framework from Glassfish and it doesn't come with monitoring capabilities. Darkstar supports  the  client-server, as well as a publish-subscribe programming model. In both cases asynchronous communication takes place between the client and the server. Basically a byte-array can be send with invocation of the method "send". Both communication style are very similar to JMS. The point to point style can be easily realized with javax.jms.Queues, the publish-subscribe with javax.jms.Topics. For sending byte arrays the javax.jms.BytesMessage could be used. One benefit of darkstar ist the ordering of messages - this cannot be easily implemented with Java EE.
The Client-Session is a classic sample for Stateful Session Bean - it's exaclty the same 1:1 communication model. The next "added" value of Darkstar are singletons, which are not a part of the EJB 3.0 specification, but will probably come with EJB 3.1. Singletons little bit problematic in a Java EE environment, because the semantic of this pattern just does not work in multiple VMs.

The Darkstar's persistence is neither so simple, nor powerful and cannot be compared with JPA. It seems like Darkstar uses serialization for the persisting the object. The developer has to use special reference objects to connect persistent objects together. From my perspective the JPA framework, perhaps with own "darkstar-provider",  would do  a much better job, than a home-grown solution.

Also darkstar's TimedTask facility could be easily replaced with EJB's timers.

Darkstar is very interesting projects which already works well. The best proof of concept is the Wonderland project (a private second life :-)), which uses Darkstar as back-end.
Although Darkstar seems to scale, it would be better to use the synergies with Glassfish and rely on a common infrastructure. From my perspective it would be better to provide a lean, domain specific, layer on top of Glassfish (perhaps with real time extensions), than building a whole server from scratch. Glassfish already comes with some facilities like Shoal (clustering solution), Grizzly and HK2.

Comments:

One major difference between SGS and an App server is that one of SGS's main goals is low latency, while with an App server it's high throughput. I think it's this difference in goals that makes these to projects quite different.

Posted by Hoyle on June 07, 2007 at 08:32 PM CEST #

Hoyle,

you are right, but I don't really think, that the Glassfish's latency is much higher, than SGS. From my perspective glassfish could be easily configured to provide lower latency.
Some comparisons would be interesting - are there some load-tests results available?

I would be also interested in the architectural differences or assumptions between glassfish and sgs. I think glassfish with some configuration or even RTS extensions should provide similar, if not better latency and perhaps throughput.

thank you for your comment,

adam

Posted by Adam Bien on June 07, 2007 at 08:41 PM CEST #

re: low latency vs throughput -- A real-time AS emphasizes predictable and relatively low latency vs throughput. It is quite possible to build a real-time AS by using a real-time JVM and some care in the AS and applications... - eduard/o

Posted by eduardo pelegri-llopart on June 07, 2007 at 08:45 PM CEST #

So Hoyle is right.

As anyone who has coded such a system knows, optimizing for over-all average throughput v. worst case latencies ultimately pull in opposite directions.

Darkstar is designed from ground up for *very* low latency apps. Every piece of every level of its framework is optimized for this. This has not been the case with the design of system desigend to supprto J2EE.

Some other major differences:
(1) Darkstar presents an apparently mono-threaded event driven model. The coder cannot create races and deadlocks are all resolved behind the scenes by the scheduler. This is very important for the target market of Darkstar-- game developers are not multi-threading aware and competent nor should they be.

(2) Darkstar does indeed present a simpler data storage model. This is to allow for raw speed. Part of designing for low latencies was designing a data system that could sale massivley in a 50% read/write situation and still deliver object access times in *tenths* of miliseconds. Its also a hidden model because, again, game developers do not know about things like lock-levels or SQL and shouldn't have to learn.

(3) Darkstar is desigend to scale *massively* as mentioned before. We are targeting million-participant worlds with out explicit zones or shards.

Finally, your comment on no monitoring is somewhat misplaced. The SDK stack already has tools for monitoring internal state. The full multi-node production stack will have full admin interfaces as well as doing load balancing and fail-over all on its own.

Glassfish is a fine piece of software, but its not the answer to every problem. If you believe it can answer this one though then the best proof is an existance proof-- go build a competing system. So far every attempt that I am aware of to build any game faster then a turn based on EJBs has failed miserably.

(P.S. "real time" in the real time VM and low lantency are not equivalent. Real time is about predictable latency, not minimal latency.)
Jeff Kesselman

Chief Darkstar Architect

Posted by JK on June 07, 2007 at 09:03 PM CEST #

link to parallel discussion:
http://www.projectdarkstar.com/index.php?option=com_smf&Itemid=44&topic=67.0

Posted by Michael Bien on June 07, 2007 at 09:27 PM CEST #

Eduardo,

thank you. I was already confused at the JavaOne about the latency vs. throughput optimization and begun to review both platforms :-).
Do you also think, that in the near future most of the appservers will be real time capable?

regards,

adam

Posted by Adam Bien on June 07, 2007 at 10:02 PM CEST #

Hi Jeff,

>As anyone who has coded such a system knows, optimizing for over-all average
>throughput v. worst case latencies ultimately pull in opposite directions.

Yes, you are right, but I still do not see why e.g. HK2 should have higher latency than SGS

>Darkstar is designed from ground up for *very* low latency apps. Every piece
>of every level of its framework is optimized for this. This has not been the
>case with the design of system desigend to supprto J2EE.

I cannot get this point. In Java EE spec there is nothing, which causes higher latency.
A full blown server implementation can have high latency, but this is not the case by glassfish (or e.g. JBoss)

Some other major differences:
(1) Darkstar presents an apparently mono-threaded event driven model. The
coder cannot create races and deadlocks are all resolved behind the scenes by
the scheduler. This is very important for the target market of Darkstar-- game
developers are not multi-threading aware and competent nor should they be.

> This is also very true for EJBs. You are even not allowed to start and synchronize threads.

(2) Darkstar does indeed present a simpler data storage model. This is to
allow for raw speed. Part of designing for low latencies was designing a data
system that could sale massivley in a 50% read/write situation and still
deliver object access times in *tenths* of miliseconds. Its also a hidden
model because, again, game developers do not know about things like lock-levels
or SQL and shouldn't have to learn.

> You could use an own "darkstar-JPA-persistence" provider which is as fast as your realization (actually event faster, serialization
> does not always scale very well). There is nothing in JPA which makes it slow. It shouldn't be a problem to receive objects in millisecods. In facts

>(3) Darkstar is desigend to scale *massively* as mentioned before. We are
>targeting million-participant worlds with out explicit zones or shards.

I think Glassfish was also built for massive scaleability. And I still do not see huge architectural differences between both.

>Finally, your comment on no monitoring is somewhat misplaced. The SDK stack
>already has tools for monitoring internal state. The full multi-node
>production stack will have full admin interfaces as well as doing load
>balancing and fail-over all on its own.

Why misplaced? Glassfish has already a good-looking admin UI, why not reuse it?

>Glassfish is a fine piece of software, but its not the answer to every problem.

Of course not. But I think the reuse of basic infrastructure (e.g. Grizzly, HK2) and concentrating in the specific domain problems
can save effort and money.

>If you believe it can answer this one though then the best proof is an
>existance proof-- go build a competing system. So far every attempt that I am
>aware of to build any game faster then a turn based on EJBs has failed
>miserably.

I would use Stateful Session Beans for session management and JMS for communication. JMS is only a spec and says nothing about the wire protocol, so even the native darkstar protocol
could be reused...

>(P.S. "real time" in the real time VM and low lantency are not
>equivalent. Real time is about predictable latency, not minimal latency.)

Of course. But hopefully the predictable latency is short enough.

>Jeff Kesselman

>Chief Darkstar Architect

I actually like the idea of gaming server and darkstar. Wonderland is just awesome. But I do not like to reinvent the wheel...

Posted by Adam Bien on June 07, 2007 at 10:25 PM CEST #

JMS has been tried for game messaging and failed specifically for latency reasons.

This is semi-pointless. Words are just words and can be argued forever.

Darkstar has my 15 years of game industry experience and some of the brightest minds in distributed computing behind it. We examined all the options available and chose our technologies based on our knowledge and experience.

Your thesis is you can do better. The only possible proof of that thesis is an existence one. So go make one.

I'll be the first to applaud if you succeed.

Regards

Jeff Kessleman

Posted by Jeff Kesselman on June 07, 2007 at 10:44 PM CEST #

P.S. We should quantify some of these terms...

Darkstar latency budget: A TOTAL system budget of low 10s of MS from message arrival at system edge til response back out. That includes ALL messaging, ALL scheduling and ALL data access.

Data access: 50% read/50% write. This makes all simple replication based schemes for scaling unworkable.

Load Balancing: Dynamic and even across a 1 million user user base.

Reliability: Failure of any node should not effect system execution. Failure of the entire back end should preserve a historically correct and referentially integrous data state within a small number of seconds of failure.

Oh and while your at it, create a programming model that looks just like regular Java code and requires no management of locks or explicit programmer control.

Those are your targets for your implementation because those are our targets.

Now go build it.

Posted by Addendum on June 07, 2007 at 10:57 PM CEST #

I do not have the capacity to write neither glassfish nor darkstar "from scratch". It isn't also my intention. I only pointed out, that your interfaces are very similar to JMS and Stateful Session Beans.
As I mentioned before, JMS is only a spec, which could be easily implemented by any provider. So if I take JMS and use your implementation behind, I will succeed and you applaud :-). Even in both cases the name of the method is: send(byte[]). What I like is to reuse as much as possible (especially the interfaces).

I like the idea, your project, but I still think that SGS could implement parts of JMS and EJBs with the same performance, latency and scaleability and have another benefit - standardized interfaces. With glassfish you would gain a slick monitoring and command line interface, autodeployment etc.

Btw. I am an independent consultant and have nothing to do with glassfish!

Thank you for your comment,

(I'm thinking now heavily about a proof of concept),

adam

Posted by Adam Bien on June 07, 2007 at 10:59 PM CEST #

I understand that you don't see the difference.

Maybe it will become more obvious as you go about your implementation.

For that implementation we should quantify some of these terms.

Darkstar latency budget: A TOTAL system budget of low 10s of MS from message arrival at system edge til response back out. That includes ALL messaging, ALL scheduling and ALL data access.

Data access: 50% read/50% write. This makes all simple replication based schemes for scaling unworkable.

Load Balancing: Dynamic and even across a 1 million user user base.

Reliability: Failure of any node should not effect system execution. Failure of the entire back end should preserve a historically correct and referentially integrous data state within a small number of seconds of failure.

Those are our design goals and what any implementation has to live up to. In our opinion J2EE did not meet our design goals. If you can actually meet them using it, more power to you.

P.S. I also dislike reinventing the wheel. But wheels work very badly on snow ice. For that you want skis.

Tools are built for their problem domain. When you exceed that there are often better or different tools that need to be used.

Posted by JK on June 07, 2007 at 11:49 PM CEST #

*sigh* One again your server is not letting me post.

That spam filter needs serious work.

Posted by JK on June 07, 2007 at 11:50 PM CEST #

Jeff,

1. I have to disable the spam filter, even my comments are marked as spam :-)
2. Regarding non-functional qualities: perhaps not millions, but thousands users is a common scenario in the Java EE space.
3. I have to admit: I have no experience with game servers and also very little with gaming. But I would like to understand the optimization and main difference between glassfish and darkstar. The APIs are very similar, the implementation is of course different. Are there some architectural whitepapers etc. available? In Java EE space there is a lot of innovation in clustering objects with different approaches. It would be interesting to see the difference between something like Shoal, JGroups, Terracota and darkstar. Perhaps Java EE could also learn something from darkstar. However synergies are important.

thanks for your comments and sorry for the spam filter ;.).

regards,

adam

Posted by Adam Bien on June 08, 2007 at 12:11 AM CEST #

It may be worth to point out that GlassFish v3 is NOT intended to be just a JavaEE AppServer. The nucleus is roughly HK2 (module and service/container support) + Grizzly (NIO framework @ TCP/IP level) + basic services). On top of this you can just run a basic web server, or a scripting framework, or web tier, or web services, or a full fledged AppServer, like SJS AS 9.x.next. We have several customers designing very large configurations around the nucleus.

There are multiple projects being designed around the GFv3 nucleus. I'd interpret Adam's question as whether it is possible for Darkstar to take advantage of that nucleus. Seems a reasonable question to me.

Posted by eduardo pelegri-llopart on June 08, 2007 at 10:19 PM CEST #

Viagra is a term that is probably no longer new to anybody. It has come
to be recognized with the who’s who of the male sexual disease called
erectile dysfunction. Everybody is aware of the fact that Viagra has
something to do with the male sex organ. These tit bit informations are
alright for the layman, but for those who are prescribed the drug, it is
very important that they have complete knowledge about it, for this
they can refer any <a
href="http://www.besthealthmed.com/viagra_guide.html">viagra guide</a>
available on the net.

Posted by Viagra guide on October 24, 2007 at 09:44 AM CEST #

Post a Comment:
  • HTML Syntax: NOT allowed
...the last 150 posts
...the last 10 comments
License