New Blog

I have a blog running on my own personal webserver using wordpress set up at
http://blog.carduner.net. Given the ample customizability of wordpress, I'll be using that blog from now on.

New Blog Coming Soon

So I got tired of the blogger thing and decided that since I was upgrading my current web hosting solution, I might as well setup wordpress on my own box and use that instead. So check it out at

blog.carduner.net

z3c.formjs 0.1 coming soon

After a few weeks of fruitful and not so fruitful work sessions, I am slowly approaching the benchmark for a 0.1 release of z3c.formjs. So what will be provided in the 0.1 release?

• Easy to use JavaScript button widget built right into z3c.form
• Easy JavaScript event handler attachment to form widgets
• A demo of the above two features.

Currently the JavaScript buttons work and the generic JavaScript event handlers for any widget are almost ready - it can be done, it's just not easy. I also now have a demo of button usage. In fact, here is a code snippet from that demo:

import zope.interface
from z3c.form import form, button
from z3c.formui import layout
from z3c.formjs import jsbutton

class IButtons(zope.interface.Interface):
  show = jsbutton.JSButton(title=u'Show Code')
  hide = jsbutton.JSButton(title=u'Hide Code')

class ButtonForm(layout.FormLayoutSupport, form.EditForm):

  buttons = button.Buttons(IButtons)

  @jsbutton.handler(IButtons['show'])
  def handleShow(self, id):
      return '$("#code").slideDown()'

  @jsbutton.handler(IButtons['hide'])
  def handleHide(self, id):
      return '$("#code").slideUp()'

As you can see, creating JavaScript buttons is as simple as creating regular buttons. Under the hood there is some rather complicated code that makes heavy use of Zope's component architecture to allow for maximum customization and (drum roll) js library agnosticism. That's right - you will be able to customize z3c.formjs to work with any JavaScript library you want. By default though, JavaScript renderers for jQuery are provided.

What about future releases?

I'm looking at some pretty nifty possible features for doing common - yet tedious JavaScript tasks. Here is a scenario I plan to make simple to do. Let's say you have an address form which includes fields for city and state. If the user enters in San Francisco for the city and New York for the state, that shouldn't validate. The trick is connecting the city and state fields so that a modification to the state updates the validation of the city. In a form this might look like:

@jsevent.handler(fields['state'], event=jsevent.CHANGED)
def handleStateChange(self, id):
  return jsevent.updateWidgetFor(self.fields['city']).render()

So this would generate JavaScript code (using whatever js framework renderers have been written for) such that when the state field iss changed there would be an HTTP Post sent to the server with the data for the state. The post will probably be sent to a url like context/@@form.html/widgets/city/get?state=WA. Then the city widget would be replaced with the HTTP Post returns - presumably a widget with a modified js validator.

But these are just thoughts at this point and the API is very likely to change as problems and their solutions are discovered.

Using z3c pagelet/template/layout paradigms

(This post is liable to change in the near future as I fix mistakes)
I began work on trying to use Stephan Richter's forthcoming form library, z3c.form (not yet public) with my existing yet unstable zobby code. z3c.form is built on the idea that you are already using other z3c components for your web application - which apparently is now the standard among Zope experts for advanced application development. This was difficult for me as I have never used any z3c components before. The following are steps I had to take to move what I already had in zobby to be ready for z3c.form stuff.

1. Make sure you have your own skin and layer.
   
   If you use Zope's default skin/layer then you will not have access to various templates and resources used by z3c.form, as they are registered on their own layer (z3c.form.interfaces.IFormLayer). Your custom layer must inherit from z3c.form.interfaces.IFormLayer. Specifically, widgets are looked up by adapting a view class, a request and some context. But the request must implement IFormLayer, which means you must be browsing to your pages through your custom skin that extends IFormLayer.
   
   For Zobby I have
   

   
   from z3c.form.interfaces import IFormLayer
   from z3c.formui.interfaces import IDivFormLayer
   from z3c.layer.pagelet import IPageletBrowserLayer
   
   class IZobbyBrowserLayer(IDivFormLayer, IFormLayer, IPageletBrowserLayer):
    """Like IMinimalBrowserLayer including widget layer."""
   

   
   This is also inheriting from some other layers. IPageletBrowserLayer provides some simple error handling pages for your skin and IDivFormLayer provides templates for doing form layouts with divs (versus tables or something).
   
   Then of course you have to register the layer:
   

   
   <interface
      interface="zobby.layer.IZobbyBrowserLayer"
      type="zope.publisher.interfaces.browser.IBrowserSkinType"
      />
   

   
   but there is nothing special about that.
   
   Now of course you have to go through your application and make sure all the resources and pages are registered on this layer and not the default zope layer (which will no longer be accessible through the skin)
   
   Just to be complete, here is the code for the skin
   

   
   import zobby.layer
   
   class IZobbyBrowserSkin(zobby.layer.IZobbyBrowserLayer):
      """The ``Zobby`` browser skin."""
   

   
   and the registration
   

   
    <zope:interface
        interface="zobby.skin.IZobbyBrowserSkin"
        type="zope.publisher.interfaces.browser.IBrowserSkinType"
        name="ZobbySkin"
        />
   

   
   


2. Register a layout template for your skin.
   
   In standard zope, skins always have macros that define where things are supposed to go on the page. Then each of templates for various views uses the macro for things like headers and footers and general layout. However, with z3c.form you will want to use z3c.template for this functionality. So rather than having this complicated thing with standard macros and overriding certain other macros, you just have
   

   
    <z3c:layout
        for="*"
        layer="zobby.layer.IZobbyBrowserLayer"
        template="template.pt"
        />
   

   
   which somehow magically does everything for you. I'm sure it wouldn't be magic if I just took the time to read the README.txt files in z3c.template. Which is here if you have the time: http://svn.zope.org/z3c.template/trunk/src/z3c/template/README.txt?rev=72085&view=markup.
   
   The one thing that differs from the older standard macro definitions, is that instead of having a big fat metal:define-macro attribute on some tag near the top of your macro, you just include a tag somewhere in the middle of your document (where you want the content to go) like so:
   

   <tal:block replace="structure provider:pagelet" />

   
   This references a content provider called pagelet, which is actually registered somewhere in z3c.pagelet. This will probably make a lot of sense if you read the README.txt file in z3c.pagelet which is here: http://svn.zope.org/z3c.pagelet/trunk/src/z3c/pagelet/README.txt?rev=72087&view=auto.
   
   This of course suggests that you can no longer use your typical page templates from standard zope as the pagelet content provider won't pick them up (I think). Which brings us to the next step.
   


3. Change all your page registrations to z3c:pagelet registrations, and change the templates accordingly.
   
   z3c.pagelet and z3c.template serve the purpose of splitting up the registration for templates and views. Where as in standard zope it is the practice to register a template for a view with the page directive. However, this is not entirely pluggable so some folks came up with z3c.pagelet and z3c.template. From what I can tell, a pagelet is just like a regular page, except it does not have a template associated with it, and must have a view class set for it. The registration for a pagelet looks like this:
   

   
    <z3c:pagelet
        name="index.html"
        for="zobby.interfaces.IZobbyApplication"
        class=".browser.ZobbyApplicationDisplayForm"
        permission="zope.Public"
        layer="zobby.layer.IZobbyBrowserLayer"
        />
   

   
   Then you have to register a template separately, that gets registered for the view class - not the content component (which the pagelet is registered for). This registration looks like this:
   

   
    <z3c:template
        template="client.pt"
        for=".browser.ZobbyApplicationDisplayForm"
        layer="zobby.layer.IZobbyBrowserLayer"
        />
   

   
   This seems a bit superfluous and you might ask the question, why not just put template=whatever in the pagelet registration since the template registration clearly has the same layer as the pagelet and is registered for the same class used by the pagelet. Well, yes this is true, but now you can register multiple templates for different views, that each live on a different layer/skin.
   
   Now I think I can explain how the provider:pagelet thing from the previous step works. It actually figure out what to display by doing an multi adapter lookup, which tries to adapt (view, request, context) to something providing ITemplate. The z3c:template directive essentially registers this adapter. So that is pretty cool.
   
   So what does client.pt look like? Well rather than having a big fat metal:use-macro at the top of the page template, it is just a plain old template with macro funny business. (If you want to use macro funny business, then you should use z3c.macro instead.) In fact, client.pt could just be
   

   
   Hi! I am the zobby client
   

   
   and when the page gets rendered, Hi! I am the zobby client will appear surrounded by everything defined with the z3c:layout directive (i.e. whatever is in template.pt).
   


4. Pay Attention to Layers
   
   When I was first trying this stuff out, I forgot to register my first the template that went with my first pagelet in my custom layer. Thus when it did the adapter lookup for ITemplate, it got back template.pt, which of course makes it do another adapter lookup for ITemplate (trying to find the pagelet's template rather than the layout template) and in a tenth of a second you have a difficult to debug maximum recursion depth exceeded error in your face.
   

And that is the end of my brief introduction to using z3c stuff.

Zobby, a collaborative editor

I thought that it would be a good idea to start writing an AJAX heavy web application to get a better idea of the various deficiencies that currently exist. Doing this will also help me learn how to use jsonserver and jquery, which will be vital in improving the deficiencies.

With that, I would like to introduce Zobby, which will be a Zope based clone of Gobby, a cross platform collaborative editor. Last night I spent several hours getting to know jsonserver and jquery and produced code that allows any number of people to chat, irc style, through the web. Currently I haven't used any auto generated forms, but just one straight up Page template and lots of javascript. The repository is publicly available at http://bzr.carduner.net/zobby.

Preliminary Discussion of Goals

I've begun to write up a set of things I'd like to be able to do with an AJAX enhanced form framework. This is by no means complete, so check back for updates.

Last updated: Wednesday 25th of April.


Use Cases for AJAX form framework
=================================

General Goal
------------

The general goal is to speed up the users ability to work with a web application. The smaller the amount of data transfered between server and client, the faster the user can perform tasks. Since lots of data transfered between client and server is duplicated, there is a lot of room for improvements.

Validation
----------

Some form validation can be done with javascript before the data is ever submitted to the server. Some examples would include data input for:

- numerical fields
- date fields
- choice fields (assuming they aren't using select elements).

One goal we should consider is implementing form validation in javascript for the simpler field types (those listed above). We will also want the ability to hook widgets up to custom javascript validators. So before a form gets submitted to the server, all widgets that have javascript validators, call the validators first.
The javascript validators should not replace the validators on the server side, to avoid problems with clients that do not support javascript. The overhead of validating input data twice will still be much less than the overhead of sending a whole error page to the client.

note: some widgets might do validation directly rather than through some other function that gets called on data submission. For example, a numerical input field might by a text input where the text turns red when the data in the field is not a number. This is the sort of thing where the "validator" would be called as the user types in the input field.

Unification of Display and Edit fields
--------------------------------------

When a user wishes to only change one attribute of an object, it can be cumbersome to load the entire edit form for the object. Since edit forms are generally accessed by first navigating to a display form, it would be beneficial if display forms could act as edit forms. Specifically, I see this working with widget switching, where a
display widget is switched to an edit widget. In this case there are two possibilities.

1. In the case that simple fields are being used, such as text
fields, modifying a simple display widget into a simple edit
widget is trivial and can be done entirely with javascript.

2. In the case of more complicated widget fields, for which there
are no javascript based renderers, a javascript function would
request the edit widget from the server via AJAX.

Wrapping Sub Forms in AJAX
--------------------------

When working with complex data input systems, it is sometimes the case that there are sub forms within a larger form. For example, we might have some kind of nested tree data structure where we wish to edit individual nodes within the tree. In a case like this, forms are recursively generated for each node and its children in the tree. Here the distinction between a widget and a sub form becomes vague. We may choose to define a widget that displays/edits a node or we may choose to define a sub form that displays/edits the node. Either method would work but the implementations would be slightly different.

I would like the ability to wrap a sub form in a widget. This way it would be possible to take existing sub forms and modify their data submission actions through javascript to funnel them into an ajax based data submission function. Here is how it might work.

1. Form object iterates through all widgets
2. One of the widgets links to a SubForm object.
- SubForm fields are not wrapped in form tags (i think that is the
only distinction between sub forms and regular forms)
3. Widget renders subform
4. js associated with widget parses subform html looking for
<input type="submit"> tags.
5. js modifies DOM to replace submit input elements with button elements
and hooks buttons up to a special handler
6. user clicks on button, handler is called
7. handler gathers all the data from the subforms fields and packages them
into a json-rpc request
8. Request is sent to json-rpc view on server that corresponds to the
wrapper widget
9. wrapper widget creates subform view, passing it input data.
10. subform view performs data conversion and validation, generating a
new html form (which might have error messages)
11. json-rpc view for widget sends back json message containing
subforms newly generated html
12. js handler replaces old widget with new widget that contains
updated subform.

In this scenario, the subform machinery is totally unaware that anything is different. But on the user side, the subform is submitted and regenerated without a page reload.


A look at jsonserver

I have just taken a brief look at jsonserver. I managed to get it installed without too much trouble except for missing the jsolait installer script. Jsolait is the javascript library that comes with jsonserver to perform the requests. I also installed the kwdemo app to test it out. Now I'm trying to figure out what the role of jsolait will be in the overall implementation of the ajax form framework. Should I go ahead and use jsolait since it already comes with jsonserver? Or should I try to replace the functionality it provides with jQuery?

I went ahead and reimplemented the javascript functions in kwdemo using jQuery. One problem I came across is that jsonserver is setup to accept a json format string in the post (i.e. '{"param1":"Some data"}'), while jQuery is set up to send regular http query strings (i.e. "?param1=Some Data"). Although jQuery can parse a json string into a javascript object, I haven't found the method (if there is one) to do the reverse.

Just for comparison, here is the code using jsolait:

function test1() {
       // injecting plaintext as html
       var result = aServer.my_output(test1callback);
}

function test1callback(resp,err){
   var node = document.getElementById("appendChild");
   if (!err){
       node.innerHTML = resp;
       }
   else
       {node.innerHTML = 'Something went wrong!'
       }
}

And here is the jQuery version:

function jquerytest1() {
   var result = null;
   $.ajax({url: '.',
           type: 'post',
           dataType: 'json',
           data: '{"id":1,"method":"my_output", "params":[]}',
           contentType: 'application/json-rpc',
           success: function(json) {$("#appendChild").append(json.result)},
           error: function() {$("#appendChild").append("Something went wrong!")}});
}

Although the jQuery version seems a bit more obfuscated, a lot of it can be simplified by setting presets for the $.ajax() method. But nevertheless you can see how that data string is quite ugly. On the other hand, the response handling is a lot nicer without the if (!err) part. I guess I would like to use the jsolait methods but use jquery response handling syntax, rather than having a straight callback.

Javascript Libraries?

So, before I get down to the nitty gritty of implementing this functionality, I have to decide what JS libraries I'm going to use. So far I'm looking in to

• MochiKit
  I originally wanted to use MochiKit for the sole reason that it was the javascript framework I am most familiar with and have used before. On top of that, it claims to add a lot of functional programming features to javascript that fix javascript. Furthermore it integrates a bunch of cool visual effects that I think it stole from scriptaculous that are just awesome. Although visual effects are mostly irrelevant for the functionality we need, they nevertheless sway the right side of my brain :). But just to make sure I covered my bases, I thought it would be best to look into some other frameworks as well before coming to a final decision.
  
  
• JQuery
  Upon first glance, jquery really impressed me. With a deeper look into it though, it also really impressed me :). The CSS3 selectors are extremely powerful, and so is the concept of chaining. I really like the chaining. At first I was unsure about it's capabilities to do AJAX, which don't seem as developed as MochiKit's, which offers a cool query string generator among other things. However, I came accross Simplifying Ajax development with jQuery, which really makes it clear how easy it is to do even some advanced Ajax stuff - primarily with the $.ajax() function. I also like what appears to be the ability to just automatically pass javascript objects as JSON strings, which is a huge plus. At this point, JQuery is my top choice.
  
  
• Prototype
  After spending 5 minutes on the Prototype web site, and not finding anything as cool as chaining, I decided to ditch Prototype in favor of JQuery. Who says you shouldn't judge a book by its cover? Ok. I looked a bit more at Prototype and it does seem to have cool stuff like chaining as well. The AJAX capabilities look pretty cool too, especially things like the periodical updater with decay. Now I just wonder whether Prototype can use the same CSS selectors as jQuery. I'll do some more research into this when I get back from class.
  
  It seems that Prototype can also use CSS selectors.
  

Conclusion: It will be a difficult choice between jQuery and Prototype. They have many similar features and I think both have everything I'd need. At this point I think I'm leaning more towards jQuery because I've been able to find a bit more documentation/tutorials on their site. Fortunately, I won't have to make an actual decision until I start working on the implementation. There is still a lot more things to think about before I get to that point.

Preliminary Introduction

Here is some preliminary information about the project with links.

The abstract:

AJAX is one of the not so new buzzword people have been throwing around when talking about what their web application can do. For a while now, I have wanted to be one of those people. Unfortunately, Zope does not make it brain-dead simple to use javascript to enhance your web application - at least when it comes to the kind of asynchronous and sometimes complex data transmission you get when you work with AJAX style technologies. I would like to add support for javascript enhanced forms to Zope, via formlib, widgets, and a json namespace for data transfer.