Zobby, a collaborative editor

I thought that it would be a good idea to start writing an AJAX heavy web application to get a better idea of the various deficiencies that currently exist. Doing this will also help me learn how to use jsonserver and jquery, which will be vital in improving the deficiencies.

With that, I would like to introduce Zobby, which will be a Zope based clone of Gobby, a cross platform collaborative editor. Last night I spent several hours getting to know jsonserver and jquery and produced code that allows any number of people to chat, irc style, through the web. Currently I haven't used any auto generated forms, but just one straight up Page template and lots of javascript. The repository is publicly available at http://bzr.carduner.net/zobby.

Preliminary Discussion of Goals

I've begun to write up a set of things I'd like to be able to do with an AJAX enhanced form framework. This is by no means complete, so check back for updates.

Last updated: Wednesday 25th of April.


Use Cases for AJAX form framework
=================================

General Goal
------------

The general goal is to speed up the users ability to work with a web application. The smaller the amount of data transfered between server and client, the faster the user can perform tasks. Since lots of data transfered between client and server is duplicated, there is a lot of room for improvements.

Validation
----------

Some form validation can be done with javascript before the data is ever submitted to the server. Some examples would include data input for:

- numerical fields
- date fields
- choice fields (assuming they aren't using select elements).

One goal we should consider is implementing form validation in javascript for the simpler field types (those listed above). We will also want the ability to hook widgets up to custom javascript validators. So before a form gets submitted to the server, all widgets that have javascript validators, call the validators first.
The javascript validators should not replace the validators on the server side, to avoid problems with clients that do not support javascript. The overhead of validating input data twice will still be much less than the overhead of sending a whole error page to the client.

note: some widgets might do validation directly rather than through some other function that gets called on data submission. For example, a numerical input field might by a text input where the text turns red when the data in the field is not a number. This is the sort of thing where the "validator" would be called as the user types in the input field.

Unification of Display and Edit fields
--------------------------------------

When a user wishes to only change one attribute of an object, it can be cumbersome to load the entire edit form for the object. Since edit forms are generally accessed by first navigating to a display form, it would be beneficial if display forms could act as edit forms. Specifically, I see this working with widget switching, where a
display widget is switched to an edit widget. In this case there are two possibilities.

1. In the case that simple fields are being used, such as text
fields, modifying a simple display widget into a simple edit
widget is trivial and can be done entirely with javascript.

2. In the case of more complicated widget fields, for which there
are no javascript based renderers, a javascript function would
request the edit widget from the server via AJAX.

Wrapping Sub Forms in AJAX
--------------------------

When working with complex data input systems, it is sometimes the case that there are sub forms within a larger form. For example, we might have some kind of nested tree data structure where we wish to edit individual nodes within the tree. In a case like this, forms are recursively generated for each node and its children in the tree. Here the distinction between a widget and a sub form becomes vague. We may choose to define a widget that displays/edits a node or we may choose to define a sub form that displays/edits the node. Either method would work but the implementations would be slightly different.

I would like the ability to wrap a sub form in a widget. This way it would be possible to take existing sub forms and modify their data submission actions through javascript to funnel them into an ajax based data submission function. Here is how it might work.

1. Form object iterates through all widgets
2. One of the widgets links to a SubForm object.
- SubForm fields are not wrapped in form tags (i think that is the
only distinction between sub forms and regular forms)
3. Widget renders subform
4. js associated with widget parses subform html looking for
<input type="submit"> tags.
5. js modifies DOM to replace submit input elements with button elements
and hooks buttons up to a special handler
6. user clicks on button, handler is called
7. handler gathers all the data from the subforms fields and packages them
into a json-rpc request
8. Request is sent to json-rpc view on server that corresponds to the
wrapper widget
9. wrapper widget creates subform view, passing it input data.
10. subform view performs data conversion and validation, generating a
new html form (which might have error messages)
11. json-rpc view for widget sends back json message containing
subforms newly generated html
12. js handler replaces old widget with new widget that contains
updated subform.

In this scenario, the subform machinery is totally unaware that anything is different. But on the user side, the subform is submitted and regenerated without a page reload.


A look at jsonserver

I have just taken a brief look at jsonserver. I managed to get it installed without too much trouble except for missing the jsolait installer script. Jsolait is the javascript library that comes with jsonserver to perform the requests. I also installed the kwdemo app to test it out. Now I'm trying to figure out what the role of jsolait will be in the overall implementation of the ajax form framework. Should I go ahead and use jsolait since it already comes with jsonserver? Or should I try to replace the functionality it provides with jQuery?

I went ahead and reimplemented the javascript functions in kwdemo using jQuery. One problem I came across is that jsonserver is setup to accept a json format string in the post (i.e. '{"param1":"Some data"}'), while jQuery is set up to send regular http query strings (i.e. "?param1=Some Data"). Although jQuery can parse a json string into a javascript object, I haven't found the method (if there is one) to do the reverse.

Just for comparison, here is the code using jsolait:

function test1() {
       // injecting plaintext as html
       var result = aServer.my_output(test1callback);
}

function test1callback(resp,err){
   var node = document.getElementById("appendChild");
   if (!err){
       node.innerHTML = resp;
       }
   else
       {node.innerHTML = 'Something went wrong!'
       }
}

And here is the jQuery version:

function jquerytest1() {
   var result = null;
   $.ajax({url: '.',
           type: 'post',
           dataType: 'json',
           data: '{"id":1,"method":"my_output", "params":[]}',
           contentType: 'application/json-rpc',
           success: function(json) {$("#appendChild").append(json.result)},
           error: function() {$("#appendChild").append("Something went wrong!")}});
}

Although the jQuery version seems a bit more obfuscated, a lot of it can be simplified by setting presets for the $.ajax() method. But nevertheless you can see how that data string is quite ugly. On the other hand, the response handling is a lot nicer without the if (!err) part. I guess I would like to use the jsolait methods but use jquery response handling syntax, rather than having a straight callback.

Javascript Libraries?

So, before I get down to the nitty gritty of implementing this functionality, I have to decide what JS libraries I'm going to use. So far I'm looking in to

• MochiKit
  I originally wanted to use MochiKit for the sole reason that it was the javascript framework I am most familiar with and have used before. On top of that, it claims to add a lot of functional programming features to javascript that fix javascript. Furthermore it integrates a bunch of cool visual effects that I think it stole from scriptaculous that are just awesome. Although visual effects are mostly irrelevant for the functionality we need, they nevertheless sway the right side of my brain :). But just to make sure I covered my bases, I thought it would be best to look into some other frameworks as well before coming to a final decision.
  
  
• JQuery
  Upon first glance, jquery really impressed me. With a deeper look into it though, it also really impressed me :). The CSS3 selectors are extremely powerful, and so is the concept of chaining. I really like the chaining. At first I was unsure about it's capabilities to do AJAX, which don't seem as developed as MochiKit's, which offers a cool query string generator among other things. However, I came accross Simplifying Ajax development with jQuery, which really makes it clear how easy it is to do even some advanced Ajax stuff - primarily with the $.ajax() function. I also like what appears to be the ability to just automatically pass javascript objects as JSON strings, which is a huge plus. At this point, JQuery is my top choice.
  
  
• Prototype
  After spending 5 minutes on the Prototype web site, and not finding anything as cool as chaining, I decided to ditch Prototype in favor of JQuery. Who says you shouldn't judge a book by its cover? Ok. I looked a bit more at Prototype and it does seem to have cool stuff like chaining as well. The AJAX capabilities look pretty cool too, especially things like the periodical updater with decay. Now I just wonder whether Prototype can use the same CSS selectors as jQuery. I'll do some more research into this when I get back from class.
  
  It seems that Prototype can also use CSS selectors.
  

Conclusion: It will be a difficult choice between jQuery and Prototype. They have many similar features and I think both have everything I'd need. At this point I think I'm leaning more towards jQuery because I've been able to find a bit more documentation/tutorials on their site. Fortunately, I won't have to make an actual decision until I start working on the implementation. There is still a lot more things to think about before I get to that point.

Preliminary Introduction

Here is some preliminary information about the project with links.

The abstract:

AJAX is one of the not so new buzzword people have been throwing around when talking about what their web application can do. For a while now, I have wanted to be one of those people. Unfortunately, Zope does not make it brain-dead simple to use javascript to enhance your web application - at least when it comes to the kind of asynchronous and sometimes complex data transmission you get when you work with AJAX style technologies. I would like to add support for javascript enhanced forms to Zope, via formlib, widgets, and a json namespace for data transfer.