A blog about software – researching it, developing it, and contemplating its future.

Archive for June 2007

Pluggable types, objects as services, and Google Gears

leave a comment »

Warning: rambling language research post ahead!

I admit it, I am a static typing weenie. Having gotten hooked on autocompletion, real-time code error highlighting, and advanced refactoring support — IntelliJ is my king — I find that it’s hard to imagine doing without.

But that’s not to say that type systems aren’t sometimes a real pain. Particularly ornery is all the machinery around making changes to the type system. Not all of us can be Scott Blum (GWT compiler ninja), and compiler hacking is not a well-known art. But without access to compiler internals, type-system hacking is very hard to come by.

Gilad Bracha, while he was at Sun, had some interesting things to say about pluggable type systems. Now, that paper is somewhat of a gripe fest, in that while it may be clear what he’s complaining about, it’s not entirely clear what to do about it. What does a pluggable type system actually look like?

I occasionally scour the recent proceedings of the ACM for interesting papers. (Any language lawyer who’s not a member of the ACM’s Digital Library is seriously missing out. Also, I’ve got to catch up on Lambda the Ultimate.) Recently in OOPSLA 2006, Andreae, Noble, Markstrum, and Millstein gave some real body to the theory with their paper on JavaCOP, a framework for building pluggable type systems in Java.

One of their examples is a @NotNull annotation to mark variables that can’t have null assigned. Check out their syntax for verifying that no bad assignments happen:

rule checkNonNull (node <<: sym) { 
where(requiresNonNull(sym)) {
error(node, "Possibly null expression "+ node+" considered @NonNull");

Working through this, this defines a constraint on all symbols that have the requiresNonNull property (which they implement with the @NotNull annotation on the symbol). All such symbols further require that the AST node assigned to them definitely has a non-null value. Their system implements all this as a layered extension to javac, that is able to check eight or so extended type properties in no more than double (at worst) the compile time.

Very interesting stuff, which points (to me, anyway) in the direction of generalizing the research to re-express Java's whole type system in rules like this. In other words, they're starting from Java and layering more type rules on top, but they could also work downwards and re-engineer Java's existing rules in their framework.

What would be the ultimate goal here? Well, giving more flexibility to juggle the nominal versus structural issues that Gilad talks about would be one reason. Another reason would be found in a more recent Gilad paper (apparently presented at Google last fall), about implementing offline objects that provide local support for centralized software services.

Now, if you read this paper after the recent shipment of Google Gears, it has all kinds of interesting ramifications. Gilad proposes a generic synchronization system implemented by reflective logging of local object modifications, supporting reconciliation on the server by change log replay. (This is actually very similar to the Dojo Offline model of synchronization, as presented at GDD07 last week. I wish there were a text post describing the model, because all these video posts and podcasts give me a pain. ) I'm not entirely convinced that model is as generic as he would like, and I will need to bang my head against data-based synchronization some more before I go to operation-based synchronization, but it's nonetheless thought-provoking.

Pluggable type systems come in when he discusses upgrading local code in a modular way. It looks to him like a fully static type system such as Java's has inadequate flexibility for changing shared type definitions at runtime. This is getting into the arena of major magic -- I'm personally very queasy about the whole concept of upgrading code modules that are in mid-execution. I'd prefer to treat that problem via an Erlang-like model of strict isolation through message passing -- the model that the E programming language implements. (And by the way, if you haven't bought into capability security yet, read one of Mark Miller's recent papers already and get with the program!)

At GDD07, I ran into Doug Crockford, who mentioned that he and Mark had discussed taking Google Gears' support for isolated worker threads and using it to implement securely encapsulated downloadable code. That squarely jibes with Gilad's recommendations, and is something that could be prototyped Real Soon Now. Of course, Gilad's whole pluggable-type machinery is definitely not on that agenda, but we must walk before we can run.

So where does this leave us? Seems to me there are several directions to pursue:

  • Gears provides a reasonably deployable framework for experimenting, not only with offline applications in general, but with an encapsulated framework for mobile code, and with the overall synchronization / update model Gilad discusses.
  • Pluggable type systems in general, along the lines of the JavaCOP system, provide a rich variety of new data points on the traditionally impoverished "static versus dynamic" axis in programming language design.
  • The right kind of type system to implement extensible, upgradeable local software objects -- particularly local objects with the ability to have their message formats, persistence structure, and inter-module interfaces -- is a huge open question, and pluggable type systems give us a much more fine-grained and flexible toolset for experimentation.

I find myself wondering what a JavaCOP framework built into the GWT compiler would look like....

(Does anyone have any links to work on JIT compilation for Javascript? If Javascript is moving in the direction of being a generic intermediate language, it'd be good to know how fast it can be made.)

Written by robjellinghaus

2007/06/05 at 06:13

Why Blogger Needs Offline Editing Support, and How To Build It With Google Gears

leave a comment »

I’m writing this in Notepad while riding the BART train on my normal morning commute into San Francisco. This commute takes about forty minutes each way for me. Since there’s no Internet, I can’t connect to work at all. Hence this is dedicated time for reading, or writing, or hacking on things that don’t need a full development environment (my laptop is a bit too old to cope with our entire code base from work anymore).

So of course blogging would be a natural. And sure, writing in Notepad is not torture. But it’s also not the right thing. What I want is to be able to write blog posts, edit them, save drafts, etc., etc., and then the minute I get back online, upload ‘em all to Blogger and worry no more about it.

To do that requires some kind of offline application with local storage. As my last post mentioned, I originally wanted to build such an offline app as a standalone Tomcat-based Seam webapp that would run in a local webserver. But Google Gears offers a much more direct route to the same destination.

So this post is a breakdown of how to construct such a client, based on conversations with various Googlers last week at Google’s Mountain View developer day.

The Requirements

Must-have requirements for an offline Blogger editor:

  • be a web application running in the browser (duh!)
  • support rich text editing at least at the level of Blogger’s own online editor
  • support authentication to Blogger
  • support local storage of some or all of the user’s blog posts
  • support creating, viewing, drafting, and editing of posts whether online or offline
  • allow synchronizing any offline edits to Blogger once back online
  • allow synchronizing of any posts edited on Blogger down to local storage
  • allow very minimal reconciliation of conflicting edits to posts on Blogger
  • require minimal installation of plugins or other application-specific software

Nice-to-have requirements for an offline Blogger editor:

  • support archiving / backing up of a user’s entire blog
  • support re-importing some set of content to a new blog that supports the Blogger API
  • support migrating content between blog services
  • support deleting postings ahd handling synchronization of offline deletes
  • provide a nicer browsing/reading UI (arguably Google Reader has this job sewn up already)

(Needless to say, the nice-to-haves don’t get done until all the must-haves are working!)

Explicit NON-requirements:

  • No need for this application to render any static HTML (the blog can presumably still be viewed and searched through Blogger itself; this app can be *just* an editor)

The API Implementation

Clearly this is exactly the kind of application Google Gears was created to support. So let’s assume we’re using Gears. Further, since I like GWT a whole lot, let’s also assume we’re using GWT.

I originally thought that this app could be built as a GWT/Gears webapp connecting directly to the Blogger API. Dan Morrill of the GWT team set me straight on that. The Blogger API is very pure in its REST-fulness, meaning specifically that it uses all the HTTP verbs. However, XMLHTTPRequest, the entire foundation of AJAX, can’t do anything other than GET and POST. So whatever we do is going to have to involve some kind of servlet bridge to the Blogger API.

(Possible workarounds for this: 1) get the Blogger and GData API teams to make GET/POST-only variants of their PUT/DELETE API methods; 2) get the browser vendors to implement PUT/DELETE support in XMLHTTPRequest. Clearly #1 is a lot more doable than #2 given the dreaded words, “legacy support”. You listening, GData team???)

Once we’ve accepted the need for some kind of bridge servlet, we’d better make sure that bridge servlet is fairly scalable and generic. Whoever is hosting this bridge servlet is going to have potentially a lot of traffic. So the bridge servlet needs to be stateless, re-entrant, and as close to a simple pass-through as possible. It shouldn’t have per-session state; it should be just a message relay.

We’ll also need some GWT modules for communicating with the bridge servlet, including beans that map to the data objects defined in the Blogger API, and service interfaces that map to the Blogger API’s methods. And in fact, this suggests that the servlet itself should simply use the Java implementation of the Blogger client API, exporting a GWT-friendly version of it (via GWT RPC) to the GWT client application.

The persistence implementation

Gears has this lovely local SQL database. But how do we get things in and out of it? We could write to the JDBC-like Gears SQL API. But I have tasted Hibernate. And if I’m programming in GWT, then writing JDBC-like code by hand would be like going back to the pre-Hibernate days. And I can’t do it. I CAN’T DO IT, I TELL YOU! I can’t go back. I WON’T go back!

Now when I first started brainstorming this with Ray from at GDD07, we got wild-eyed and started contemplating porting Hibernate to GWT. Saner heads rapidly prevailed. It turns out that Bob Vawter of the GWT team is doing work on a GWT-generator framework for generating a variety of useful artifacts (starting with Javascript-library method exports) from GWT beans.

So let’s use his framework, only instead of exporting to Javascript libs, we’ll make it generate dead simple CRUD operations for our Blogger beans to the Gears data store.

It looks like some other folks on the Gears list have had similar thoughts. Check out this Gears ORM implementation. But since that’s not based on GWT, it’s not directly applicable. Still cool though! But let’s assume we’ll proceed with Bob’s stuff, just for fun. This is all still early days and all experiments still deserve love.

The todo list

So, what do we have?

  1. GWT beans mapping to Blogger API data objects.
  2. GWT service interface mapping to Blogger API methods.
  3. Bridge servlet implementing GWT service interface and using Blogger Java client API implementation to relay messages to Blogger.
  4. Simple persistence code generator (using a GWT compile-time generator implementation) to take the GWT beans and store them in the Gears local database.
  5. Business logic to implement the client-side synchronization to the Blogger server. (That’ll be another post, or maybe lots of them!)
  6. GWT UI code to let the user use all of this infrastructural goodness to meet the must-have requirements!

If we get it right, it hopefully lands in the Google GWT API project. And once Blogger does this, Calendar is next, and then some.

Sounds like a fun project, huh? Interested? Chime in in the comments. Or join this GWT-Contributors thread on the topic.


Written by robjellinghaus

2007/06/05 at 05:33

Why start this blog?

leave a comment »

[ObNewBlogIntentions] Will try to update often. Will try not to spend so much time farting around with templates that I never get around to posting. [/ObNewBlogIntentions]

So, with that out of the way, hello. I’ve been planning a blog for a year and a half now. I realized in early 2006 that I’d spent my whole life programming for companies who owned everything I did, and I wanted to do something open source on my own time. So I thought, what about a blog system? Clearly the world needs more of those :-)

(The nice thing about programming on your own time is you don’t have to justify anything to anybody! Except your wife. But if you cut back on the computer games, so you’re not blowing your personal time budget away, then it’s all good.)

The thing is, I really wanted a blogging system that would support offline editing and offline backup of my blog content. I ride the commuter train, and I don’t entirely trust any single system (even one run by Google) — I like lots of redundant backups that I personally maintain.

So I started on a project to implement a distributed blog application. Which, of course, since I’m a Java weenie, meant that I really needed a Hibernate-based peer-to-peer distributed object version control system.

I built most of that over the bulk of 2006 (or prototyped it, anyway), using a Seam / EJB3 / /Tomcat / JSF stack. Then I thought, how do I build the UI? I started looking into GWT and liked it. I even found a JSF / GWT integration library. But it didn’t handle GWT RPC properly.

Then GWT got open sourced. I started working on fixing the RPC issues in GWT in early 2007, and that led to my submitting what I think was the first major external patch to GWT. I also put together a demo of Seam + JSF + GWT — adding a GWT blog-reading interface to the Seam blog example.

Michael Yuan from JBoss mentioned this at his JavaOne BOF this year, and I got to demo it at the GWT hack session at JavaOne, where I had the pleasure of meeting Bruce, Joel, Kelly, Bob, and the Dans (Peterson and Morrill) from the GWT team. Then I took a couple of weeks off to recuperate — my wife is expecting our second child in late July (our first child is two and a half), so there’s plenty of other distractions keeping me busy!

So that brings us to yesterday. Google Developer Day in Mountain View. And the announcement of Google Gears and GWT for Gears. Now, remember that my original use case was to build a blogging app that could work offline. I was using a technology stack that was Seam + EJB3 + Tomcat + JSF + GWT + Postgres. So to use this app I was building, you’d have to be running a local Postgres install and a local Tomcat install, and you’d have to do a bunch of Java deployment. I might have wound up with a few dozen users, or maybe only myself. (The code could have wider applicability, but direct product usage would’ve been low.)

Along comes Gears. Suddenly there’s a whole new stack option: Gears + GWT + Blogger. Offline functionality usable with any Blogger blog, and a much, much, much easier deployment, with a possibly much larger user base as a result.

So yesterday at GDD07 I brainstormed about this a bunch with Dan Morrill, Miguel Mendez, and Bob Vawter from the GWT team. It looks like a go. My next post will be about the detailed plan for that.

Meanwhile I still have to finish landing the JSF+GWT code, and documenting and submitting the Seam example. This whole project has been one surprise after another, and it’s only getting more fun so far :-)


Written by robjellinghaus

2007/06/02 at 03:32

Posted in GDD07, GWT, Java, Seam


leave a comment »

Hello folks. This blog has been too long in coming, due to many complicated reasons of my own self-inflicted devising. Better late than never. Enjoy!

Written by robjellinghaus

2007/06/01 at 08:10

Posted in Uncategorized


Get every new post delivered to your Inbox.