Small gaps in the grails docs

Just for reference, if you come across one of the following problems:

Validation only datasource

Looking at the options of dbCreate in Datasource.groovy I only found 3 values: create-drop, create or update. But there is a fourth one: validate!
This one helps a lot when you use schema generation with Autobase or doing your schema updates external.

Redirect

Controller.redirect has two options for passing an id to the action id and params, but if you specify both which one will be used?

controller.redirect(id:1, params:[id:2])

Trying this out I found the id supersedes the params.id.

Update:
Thanks to Burt and Alvaro for their hints. I submitted a JIRA issue

Enable Capture/Replay for Selenium Flex

One of the missing features of the SeleniumFlexAPI was capture/replay. So I looked into different ways to enable it:

  • Approach 1: Dispatch a DOM event and listen for it in ide-extensions.js

    Problem: Where do I include the name/id of the Flex control?
  • Approach 2: Custom events

    Problem: How do I listen to them in ide-extensions.js?

Solution: Additions to the Selenium IDE code: window.record

The solution is to add a new method to the Selenium IDE code: window.record which delegates to recorder.record. So that the Flex code can call this method directly through the ExternalInterface. The clear advantage of this technique is that there is no code pollution in your production code. But you have to change the SeleniumIDE code. There is an issue in the SeleniumIDE Jira which describes the additions, so go and vote for it!
Addtional code is also needed in the SeleniumFlexAPI.as in the applicationCompleteHandler:

private function applicationCompleteHandler(event:FlexEvent):void {
  ...
  registerListeners(appTreeParser.thisApp.parent);
  ...
}
private function registerListeners(subject:*):void {
  subject.addEventListener(MouseEvent.CLICK, recordClick);
  subject.addEventListener(MouseEvent.DOUBLE_CLICK, recordDoubleClick);
  subject.addEventListener(Event.ADDED, childAdded);
  addListenerRecursive(subject);
}

Bubbling events like MouseEvent.CLICK can be added here but for the non-bubbling ones you have to recursively walk the displayobject hierarchy:

public function addListenerRecursive(root:*):void {
  for(var i:int = 0; i < root.numChildren; i++) {
    try {
      var child:Object = root.getChildAt(i);
      if (isMenuBar(child)) {
        child.removeEventListener(MenuEvent.ITEM_CLICK, recordMenuItemClick);
        child.addEventListener(MenuEvent.ITEM_CLICK, recordMenuItemClick);
      }
      if (isTextControl(child)) {
        child.removeEventListener(Event.CHANGE, recordTextChange);
        child.addEventListener(Event.CHANGE, recordTextChange);
      }
      if (isDataGrid(child)) {
        child.removeEventListener(DataGridEvent.ITEM_FOCUS_IN, recordItemClick);
        child.addEventListener(DataGridEvent.ITEM_FOCUS_IN, recordItemClick);
      }
      addListenerRecursive(child);
    } catch(e:Error) {}
  }
}

In the event handling functions you just call the record function with the appropiate SeleniumFlex command:

private function recordClick(event:MouseEvent):void {
  ExternalInterface.call("record", "flexClick", "name=" + event.target.name, "");
}

Since Flex Sprites have no ids I use the name here for identifying the clicked target.
Another pitfall is when components are added dynamically (like when a Date opens, it adds a calendar view):

private function childAdded(event:Event):void {
  if (isDate(event.target.parent.parent)) {
    event.target.parent.parent.removeEventListener(CalendarLayoutChangeEvent.CHANGE, recordDate);
    event.target.parent.parent.addEventListener(CalendarLayoutChangeEvent.CHANGE, recordDate);
  }
}

Conclusion

So finally capture/replay in SeleniumFlex becomes a reality! Nonetheless there is some work to do to support the different kinds of flex controls and Selenium commands.

Followup: Selenium Flex API with Firefox 3 and Selenium IDE 1.0 Beta 2 now working

Because of a bug in the Selenium Flex API, it didn’t work with Firefox 3 and the Selenium IDE 1.0 beta 2.
To fix this bug add the following line:

    if (flashObj.wrappedJSObject) {
        flashObj = flashObj.wrappedJSObject;
    }

to ide-extensions.js in Selenium.prototype.callFlexMethod after

var flashObj = selenium.browserbot.findElement(this.flashObjectLocator);

The state of functional testing in Flex

Functional testing of UIs is an important and often neglected way of ensuring quality and prevent regression. The Flex world of functional tests seems at the very beginning. We evaluated some of the tools available and used the following criterias:

  • OS independence
    the tool and the created test scripts should run on at least every platform the Flex SDK and the Flash platform are available
  • Tool changes
    how much we need to change or adapt the tool to suit our needs
  • Code pollution
    how much the actual code needs to be polluted to support this testing tool
  • Capture/Replay
    the tool needs at least an option to capture and replay test scripts
  • Additional License Costs
    if we need to pay additional (besides the tool) license costs for things like the FlexBuilder Pro
OS ind. Tool changes Code pollution Capture / Replay Add. costs
Automation based tools + 0 +
SeleniumFlex + 0 + 0 +
FunFx 0 + 0
Fluorida + + +

Automation based tools (like FlexMonkey, QTP and RIATest) use the Flex automation API and have additional costs for FlexBuilderPro (700$ per license). For custom components you have to add automation code to them (pollution) and introduce them and their events in FlexMonkeys event map (tool changes).
SeleniumFlex uses the JavascriptBridge (ExternalInterface) of the FlashPlayer and needs you to add the custom components and events to this external interface which resides in the tool/test code (therefore a 0 at tool changes). You can use the Selenium plugin for spy (the ids)/replay but the capture option isn’t working yet (0 for capture/replay).
FunFx also uses the ExternalInterface and is written in Ruby but runs only on Windows (- for OS independence) because it connects to the Flex application via Win32OLE. I found no capture/replay (-) and the website says you need FlexBuilder (I don’t know why therefore a 0 for license costs, we use IntelliJ IDEA for Flex development)
Fluorida seems to be at the beginning and there is very little documentation so it looks like to need an investment (- for tool changes). It has no capture/replay (-).

Conclusion

So our tool of choice is SeleniumFlex and we hope to get capture/replay working in the near future.
What experience have you made with functional testing in Flex? Which one do you use?

Visualizations with Flare/Prefuse

Recently we were in need of visualizing lots of data in networks. We started with a Javascript based version using RaphaelJS. RaphaelJS uses SVG/VML as the underlying graphics API. Drawing lots of nodes and edges in different shapes and colors worked like a breeze. But we quickly got performance problems when animating the transitions between different states of the network. So we had to look to an alternative technology which is performant enough and pragmatic to use. Reading a lot about Flex and its promising animation capabilities we gave it a try.
Shortly after we stumbled upon Flare/Prefuse and the stunning demos convinced us. It is very easy to use and gives remarkable results in a short time. We integrated it into our web app but suddenly the visualizations weren’t drawn upon startup. Everything went fine when the visualization was ready before the Flex app was fully completed but when the data was visualized after that the edges and nodes weren’t shown. Debugging our app and reading the Flex Sprite API wasn’t much helpful. But one comment in a superclass of all the Flare sprites solved out problem:

Internally, the DirtySprite class maintains a static list of all "dirty" sprites, and redraws each sprite in the list when a Event.RENDER event is issued. Typically, this process is performed automatically. In a few cases, erratic behavior has been observed due to a Flash Player bug that results in RENDER events not being properly issued. As a fallback, the static renderDirty() method can be invoked to manually force each dirty sprite to be redrawn.

These few cases were the default behaviour of our app, so invoking renderDirty solved all our drawing issues. I found no blog entry or hint in the docs and the demos run perfectly since all data is there before the app is shown. So the lesson here is:

  • retest your assumptions (we thought the Flare sprites were all DataSprites and the next superclass is the Flex API Sprite and forgot about the DirtySprite)
  • besides the demos and tutorials/docs read the API docs carefully, often there are implementation/technology specific hints

Drawing your background with Javascript

Having complex backgrounds (like radial gradients) or even dynamic backgrounds in a web page is not an easy task using only images. Javascript helps here and the SVG/VML support in modern browsers makes drawing a reality. Looking around in the Javascript library world there are some which have APIs for crossbrowser drawing or charting. But if you are looking for a small and easy to learn on you have to search. We found raphaeljs, a project which started as a 20%er at Atlassian. It has an API which is close to SVG but also supports non SVG browsers like IE. How to make animations and complex drawings is easy to learn. The examples give a hint of what is possible.
We needed a radial gradient to have a smooth background, looking at the docs and the SVG spec, it was a four liner:

var Paper = Raphael("canvas", width, height);
var b = Paper.circle(width / 2, height / 2, Math.max(width, height));
var gradient = {type: "radial", dots: [{color: "#FFFFFF"}, {color: "#D5D5D5"}], vector: [0, 0, 0, "100%"]};
b.attr({"gradient": gradient});

A DSL for deploying grails apps

Everytime I deploy my grails app I do the same steps over and over again:

  • get the latest build from our Hudson CI
  • extract the war file from the CI archive
  • scp the war to a gateway server
  • scp the war to the target server
  • run stop.sh to shutdown the jetty
  • run update.sh to update the web app in the jetty webapps dir
  • run start.sh to start the jetty

Reading the Productive Programmer I thought: “This should be automated”. Looking at the Rails world I found a tool named Capistrano which looked like a script library for deploying Rails apps. Using builders in groovy and JSch for SSH/scp I wrote a small script to do the tedious work using a self defined DSL for deploying grails apps:

Grapes grapes = new Grapes()
def script = grapes.script {
    set gateway: "gateway-server"
    set username: "schneide"
    set password: "************"
    set project: "my_ci_project"
    set ciType: "hudson"
    set target: "deploy_target.com"
    set ci_server: "hudson-schneide"
    set files: ["webapp.war"]

    task("deploy") {
        grab from: "ci"
        scp to: "target"
        ssh "stop.sh"
        ssh "update.sh"
        ssh "start.sh"
    }
}

script.tasks.deploy.execute()

This is far from being finished but a starting point and I think about open sourcing it. What do you think: may it help you? What are your experiences with deploying grails apps?

JTable index madness

A coworker of mine recently stumbled upon a strange looking JTable:
A broken down JTable

This reminded me of an effect I have seen several times. Digging through the source code of the JTable we found an unusual handling of TableEvents:

    public void tableChanged(TableModelEvent e) {
        if (e == null || e.getFirstRow() == TableModelEvent.HEADER_ROW) {
            // The whole thing changed
            clearSelectionAndLeadAnchor();

            rowModel = null;

            if (getAutoCreateColumnsFromModel()) {
		// This will effect invalidation of the JTable and JTableHeader.
                createDefaultColumnsFromModel();
		return;
	    }

	    resizeAndRepaint();
            return;
        }
...

The hidden problem here is that the value of TableModelEvent.HEADER_ROW is -1. So sending a TableEvent to the table with a obviously wrong index causes the table to reset discarding all renderers, column sizes, etc. And this is regardless of the type of the event (INSERT, UPDATE and DELETE). Yes, it is a bug in our implementation of the table model but instead of throwing an exception like IndexOutOfBounds it causes another event which resets the table. Not an easy bug to hunt down…

Deploying a Grails app on an Oracle DB

Running our new grails app on HSQL and a Postgresql everything went fine. But the production DB was decided to be an Oracle. And suddenly the app crashed several times. Here’s a list of what problems we encountered:

  • ORA-00972: identifier is too long
  • want to store a null value in a non null column

Oracle identifiers are limited to 30 characters. So we thought using a mapping for the table should do the trick. But grails uses the table names to construct the n:m relations and their id column names between the domain classes. Looking at the grails docs we found a joinTable mapping:

static mapping = {
    table 'PROP'
    tablePerHierarchy false
    instrumentInfos joinTable: [name:'PROP_INS', key:'id', column:'instrumentInfos_id']
}

This worked most of the time but in some cases grails just didn’t want to take our definitions. The problem was a bug in grails. The workaround we took was to shorten the domain classes names.
The second problem arose as we tried to store empty strings into the database. Oracle stores empty strings as null values which causes a constraint violation exception. The solution was to declare the string columns nullable or not nullable and not blank but you cannot use a not nullable and blank string with an Oracle DB.

Global error pages with Jetty and grails

We wanted to configure global error pages for our grails app. Using your favorite search engine you quickly find the following info.

At the bottom it says that in order to support global error pages you have to forward to a context because error pages can only be handled within contexts/webapps.
So I started adding a context to the contexts dir which looked like this:

<Configure class="org.mortbay.jetty.webapp.WebAppContext">
      <Set name="contextPath">/</Set>
      <Set name="war"><SystemProperty name="jetty.home" default="."/>/webapps/mywebapp.war</Set>
      <Set name="extractWAR">false</Set>
...

This caused an “IllegalArgumentException: name” at startup. The solution was to set extractWAR to true. JSPs or other resources (like GSPs) cannot be used inside a war when extractWAR is set to false. But this way got another pitfall: using localhost:8080/mywebapp won’t work. So why not just forward all requests from / to /mywebapp. Said and done:

<Set name="handler">
  <New id="Handlers" class="org.mortbay.jetty.handler.RewriteHandler">
    <Set name="rewriteRequestURI">false</Set>
    <Set name="rewritePathInfo">false</Set>
    <Set name="originalPathAttribute">requestedPath</Set>
    <Call name="addRewriteRule"><Arg>/mywebapp/*</Arg><Arg></Arg></Call>
    <Call name="addRewriteRule"><Arg>/*</Arg><Arg>/mywebapp</Arg></Call>
    <Set name="handler">
here the old handlers are inserted...

Now /mywebapp points to my webapp. / gives a 500 and other invalid urls give a 404.
To use your custom error pages inside a grails app just add the error codes you want to map inside the UrlMappings.groovy file:

class UrlMappings {
  ...
  static mappings = {
    "500"(view:'/error')
    "404"(view:'/error404')
  }
}