SpringOne 2GX

Geographically Distributed Agile Teams Have Choices for Their Lifecycles

Johanna Rothman — Mon, 06 Feb 2012 13:00:19 CST

I hope that by now you see that you have any number of choices for your lifecycle if you are geographically distributed team and you are transitioning to agile. I do recommend a servant leader agile project manager, for coordination and risk management. With people all over the world, it’s difficult to coordinate the project, which leads to more risk.

Scrum is often not the best approach to geographically distributed agile. That does not mean the distributed team should not go agile. It just means they should not use Scrum. If the distributed team can all travel to one place so they can get trained by the same Scrum trainer together, and if they can take the opportunity to talk together to discuss what they need from the Scrum Master then maybe they can use Scrum, especially if they use their retrospectives well. It’s a lot of responsibility for the team new to agile and new to Scrum. If you’re a team new to agile and new to Scrum, and you try this, do yourself a favor and use a coach.

Electronic tools do not always make seeing and managing the risk easier when you first start–tools can prevent transparency until you understand what you are doing. I like starting with stickies or cards on a wall or board, and as the team members learn how to work, and what they, as a team need, then choosing a tool. I don’t see how to choose a tool before you know what tool you need. One tip: do not allow your management to select a tool for you. You, as a team, can select a tool for yourself. You might start with digital pictures of your task board on a project wiki before you decide which tool to use.

Let me recap:

Agile Lifecycles for Geographically Distributed Teams, Part 1 discussed iterations and silo’d teams.

Agile Lifecycles for Geographically Distributed Teams, Part 2 discussed kanban and silo’d teams.

I got all hot under the collar and discussed Why an Agile Project Manager is Not a Scrum Master.

Agile Lifecycles for Geographically Distributed Teams, Part 3 discussed iterations and kanban and silo’d teams.

You should also read What Lifecycle? Selecting the Right Model for Your Project.

If you have several teams all working towards one business goal, you have a program, and that is a different problem. Look for more blog posts on that, later.

These are precisely the problems no one teaches you in project management certification courses. If you have read Manage It! Your Guide to Modern, Pragmatic Project Management, you know I consider knowledge of different lifecycles necessary for project managers.

Project managers who try to control their teams better have a darn good reason–these people are adults. They manage the rest of their lives. Are you trying to tell me they can’t manage their work? And, these are the problems that can cause geographically distributed projects to fail, splat.

If you want to learn to work more effectively on your geographically distributed team, please join Shane Hastie and me in a workshop April 17-18, 2012. We would love to have you.

Dependency Management with .NET – Doing it Right

Baruch Sadogursky — Mon, 06 Feb 2012 08:00:18 CST

This post was originally posted on ‘From the Frog’s Mouth‘.
You’re welcome to comment here or there.

The problem of dependency management is neither new nor original, it exists in all development platforms, and .NET is no different.

Let’s go through different solutions and see how they perform. I’ll list them here in no particular order.

Keeping dependencies in your source control

That’s a very popular solution, and for a reason. The benefits are obvious. Here are some of them:

No setup. You already have your source control in place (hm, I hope you do!). Add \bin directory, and you are fine.
No learning curve. Developers are used to work with source control.
Shared. The whole team gets changes and updates from the server as they occur.
Enterprisy (in a good way). The software is proven, backed up, DRP is done.

Sounds good, doesn’t it? So, what’s wrong with it? Only one thing – source control systems are designed to control, well, sources. As such they aren’t so great in controlling binaries. These are the shortcomings we all encountered during the years of Version Control System usage for dependencies:

It isn’t a proxy. VCS can’t download the dependency you need from a central repository when you need one. You need to manually download it and add it to VCS. The history starts from there – you just lost the link to the original file. So, you work hard, and on top of it lost information; this repeat itself for each new dependency.
Versioning mismatch. Source files are versioned by their content. VCSs know how to diff them and understand what changed. Binaries, on the other side, usually versioned by their name. From VCS point of view they are different entries, each one without any version history.
Some very popular VCSs (like Subversion) can’t obliterate files. That means – once a file was added, it stay in the repository forever. That’s not a big issue for small source files, but can become quite a pain when it comes to obsolete large binaries.
Source control knows how to search sources. And, of course, the most important type of search is by content. Searching for binaries is different: what matters there is the location, structure of the file name and, in case of archived artifact, the contents of archive.
The permissions scheme of VCSs is tailored for versioning sources (again!). For example, there is no override permission. That’s because overriding sources is something we do all the time (that’s what diff is for in VCS) – it’s the same security level as, let’s say, adding a new source file. With binaries the situation is very different. While adding new binaries is fine, overriding released binary is something that shouldn’t be done, one should have a special permission for it.
Distributed VCSs, awesome by themselves, are particularly unsuited for handling big binary files. When cloning a remote repository to your machine you are bringing all the history of all the files in it. Now just think about all the huge binaries sitting there…

As you see, the conclusion is simple – we can do better. Let’s try something specialized for binaries.

GAC and WebGAC

Global Assembly Cache is, on contradictory to VCS ,tailored for storing binaries. It understands versions, prevents conflicts, and generally does a good job being your local dependencies storage. The main problem with GAC is being local, which means – each and every developer should take the binaries from somewhere and install them in their local GAC. You see the troubles coming in that setup, don’t you? WebGAC to the rescue here. It’s essentially GAC shared by WebDAV and enables clients to fetch dependencies from the server, simplifying dependencies management for a team. Let’s do our pros/cons math. The benefits:

GAC is good, standard and proven solution for binaries management. It deals well with versions.
WebGAC is a central binaries repository for a team. Every team member synchronizes with it.
WebDAV is popular well-known HTTP extension with locking, security management, etc. Working with the Apache WebDAV module is generally straightforward.

Let’s see what won’t work so great with that solution:

It isn’t a proxy. WebGAC can’t download the dependency you need from a central repository when you need one. You need to manually download it, add it to WebGAC and only then it becomes available to the team. Not only must you work for every version of every dependency needed, the link to the original file is lost.
No notion of packages. GAC contains single dlls. You install them one by one. But think about NUnit, as an example. It contains about dozen of dlls along with various xml and configuration files. How can you install it to GAC?
Security is cumbersome. You’ll need to configure Apache Server’s security, and even then it won’t be flexible enough to determine between deployer (a user that can publish private dependencies) and promoter (a user that can move dependencies from a private repository to a public one).
Search is basic. WebDAV by itself only knows about files. It doesn’t care about the structure of the filename, or about the presence of Strong Names.

Looks like we still didn’t find what we are looking for, and then…

Here comes NuGet

This is something else. NuGet designed to be “a developer focused package management system for the .NET platform intent on simplifying the process of incorporating third-party libraries into a .NET application during development”. That’s exactly what we need. Let’s look how great it is:

Manages packages, not dlls.
Provides NuGet Gallery – almost 4.5K (at the time of writing) packages are at your disposal for all your development needs.
Supports binary versioning.
Integrates with Visual Studio.
Integrates with your build.
Integrates with your build server (only TeamCity at the moment of writing).

This tool’s like a dream come true. So, what can I mention as downsides? Most of them are downsides of the NuGet Gallery, not NuGet itself. The Gallery is a young and relatively small project (just for the sake of comparison, Maven Central is 6 years old and contains more than 290K artifacts) and, as such, it has its downsides:

The content of submissions to the Gallery is (almost) unverified. Everyone can register, get the API key and start uploading whatever they like. Scary, isn’t it? (yes, very scary).
Being public, NuGet Gallery can’t be used for inter-team packages exchange. Private Remote Feeds are the recommended solution. Next we’ll see if it is good enough.

Working with NuGet Remote Feeds

Remote Feeds, introduced in NuGet 1.4 are crucial need for any development team. It serves a dual purpose: it allows sharing 3rd party packages that aren’t available on Gallery (or even replaces the Gallery for those who can’t trust it) and it serves as a target for internal deployments – both for team collaboration and for other usages, as making packages available to QA, or even serving them to the customers from the outside world (by using Chocolatey, for example). If that’s so right, what’s wrong? Here’s what:

You saw it coming: It isn’t a proxy. You know the score by now.
It can’t aggregate. The NuGet Remote Feed exposing one monolithic repository: the one you have on your machine. It can’t aggregate NuGet packages from remote repositories, or expose number of local repositories (separated for security reasons, for example).
You can’t attach your own metadata. Let’s say you want to annotate some package with compatibility information (e.g. works with certain browsers). No, can’t do.
The repository is very simplistic. It doesn’t provide any web interface; it browsable and searchable only from a client – be it Visual Studio or the command line interface (pretty basic by itself).
Even the VS search interface is very basic (all you have is arbitrary sorting and free text search). It should be enough for starters but lack of searching inside the packages or by properties (from the previous bullet) will bite you eventually.
The security scheme is even less than simplistic. All that’s required to authenticate a deployment/delete of all the users at once is an API key. What about separation of duties? Some users should only be able to read, others only to annotate with metadata (QA team that tests compatibility in my previous example), and only small subgroup – to deploy. The all-or-nothing scheme is definitely insufficient.
Storage format is suboptimal:
- The packages are stored on the filesystem in a naive simple format. That fine for small repository,but as you grow, you’d expect storage which more optimized for binaries.
- The metadata is not indexed. Again, fine for small repo, troubles are foreseen when it comes to scaling.

So, is there a good alternative to NuGet Remote Feed to be your in-house Gallery for NuGet packages? We, the proud makers of Artifactory, believe there is.

Meet Artifactory

Artifactory is an enterprise-grade Binary Repository that centralizes all aspects of managing software binaries. That means that we tackle all the problems mentioned above. We are developing Artifactory since 2006. Being used by millions of users for storing, sharing and managing binaries, we have gathered great feedback from our users.

That’s what we’ve learned:

Binary packages are different from sources (by being big and binary) and deserve smart storage.
Binary packages are usually archives (be it jars, zips, rpms or nupkgs). They should be browsable and searchable without the need to download them locally to developer’s machine.
Big public repositories exist on the net, they need to be proxied smartly (variations, auditing, managing)
Users come in different flavors. Their permissions should match possible responsibilities (and in the case of binary packages they are different from other cases).
A binary repository holds critical information, it should be rock-solid, backed up, and DRP ready.
Your software ends up being a package. We know how to help you…
- build it in a reproducible manner, integrating with your build tools and your build server.
- stage it to ensure the best quality.
- distribute it to your customers.

You get the gist behind Artifactory by watching this 2.5 minutes video:
If you’re not in the mood for movies (or ran out of popcorn), here’s quick recap:
As you see, instead of working with a number of NuGet Feeds (NuGet Gallery, Orchard Gallery, Remote Feeds from co-workers and from different teams) developers work with exactly one repository. It simplifies setup and daily work and centralizes management and maintenance.
The work is bi-directional, the users resolve their 3rd party dependencies from Artifactory and deploy their created packaged into it.

Now let’s add a build server to the picture (literally):
Yup, with numbers this time. So, here we go:

Developers find and fetch new 3rd party packages from Artifactory in Visual Studio. The packages are downloaded from Artifactory to the developer’s machine. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds. On developer machines packages.config is updated with the list of used packages.
Developers commit their code and packages.config (but not the binaries) to VCS.
The Build server (as I already mentioned, TeamCity now supports NuGet) takes the changes from the VCS.
It builds the solution and packs the produced artifacts as NuGet packages.
During the build it fetches the needed packages from Artifactory. If the packages aren’t present in Artifactory it will look for them in remote galleries/feeds.
Once the packages are built they are deployed to Artifactory.

Built packages in Artifactory can be used by other teams (as their 3rd party dependencies), by QA for running tests and even by the end users (Chocolatey FTW), all this with fine-grained permissions and robust promotion procedures (moving a package between repositories with different visibility rules).
You know what? It deserves dedicated how-to blog post. I’ll link it here once published.

Assuming you’ve read up to this point, you’ve gathered that starting from Artifactory version 2.5.0 we are proud to serve the .NET world with full NuGet support. We can proxy any remote NuGet feed (starting with NuGet Gallery, of course), we can host the packages that aren’t found on any remote NuGet feed, we can host the packages you produce and we can aggregate any number of repositories of any kind under single a URL. We provide you with an awesome UI for configuring your repositories, browsing and searching for your packages. We also feature smart storage that enables attaching searchable metadata on top of your binaries. We can do it all on the cloud with our SAAS version.

Hopefully , you’re convinced by now and probably looking for the download link on our site (here’s it, BTW, click on “Evalution”). If not, give it a try by playing with our live demo. Look at the nuget-gallery cache: that’s how we proxy the NuGet Gallery. You’ll find some of the packages saved locally; once you’ve selected a package, you’ll see all kinds of information about it: its name and size, who deployed it to Artifactory, where it came from (from NuGet Gallery, naturally for this is the NuGet Gallery cache) and the operations you can perform on this package (as anonymous the selection is naturally limited). Clicking on the triangle in the tree will open the package and let you dive into its content, including downloading specific files from the archive (click for full-size image):

We, at JFrog, believe that Artifactory is the missing piece of the puzzle for a robust, agile .NET dependency management, which can make the development process easier compared to other alternatives.
We’ll be happy to receive any insights, thoughts and comments on the ideas presented in this blog and/or your experience using Artifactory together with NuGet.

Integrating, integrating, integrating. That’s what we do in Java enterprise development. Persisting objects with Hibernate wrapped by JPA using C3Po (or JTA?) (or MongoDB over Morphia?), processed with JBMP, created by JAXB (jackson-json?) from JAX-RS scheduled by Quartz … (a few dozen frameworks later) … all this glued with Spring (or Guice?) deployed on Jetty (or Tomcat, JBoss, Resin?) into cluster by Terracotta (or Hadoop, GigaSpaces, JBoss cache, Infinispam?). Ah, and all this built using Maven Gradle with Artifactory on Jenkins. I sure forgot ½ of the frameworks we constantly use.

Generally we don’t mind much about the internals of the frameworks we use (as long as they are good) – the whole encapsulation stuff is the last undoubted good thing. But except for the API (part of which is the configuration) frameworks have another user-facing end – the logging. When we build a system we want it to behave as one system – single configuration from one end, and single log from another (break it to different files, if you wish, but it should still be a unified logging system).

The reality is that there is no standard de-facto for logging. The standard de-jure – JUL, is not very popular because of its lack of functionality (compared to alternatives) and its suboptimal performance. And then there is Log4J, which almost became standard, but did not. And there is logback, which is a Log4J trashover, and there are facades (JCL and SLF4J), which try to unite all this zoo, and some others, which you have probably never heard of, like syslog4j*, logging framework by the Object Guy, jLo, MonoLog, Lumberjack, Houston, JTraceDump, qflog, LN2, TracingClassLoader, SMTPHandler, Log4Ant, Simple Log, Log Bridge, Craftsman Spy, Pencil, JDLabAgent, Trace Log, JDBC Logger, LimpidLog and Microlog.

Let it be, you’d say – why not have many logging tools, which are good and diverse! Well, the problem, as I’ve already mentioned, is that they leak out of the frameworks. Their diverse configuration leaks from one end, while their diverse output from another. Spring uses Log4J over JCL. So does Hibernate. Jetty uses Logback over SLF4J. Some (like Terracotta modules) use plain Log4J, Jersey uses JUL. This means we end up with 5 separate configurations (Log4J, SLF4J, Logback, JCL and JUL) and 3 different types of log files (Log4j, Logback and JUL). What a system!

To make the long story short – How can we achieve the desired consolidation? Clearly, we need a facade. There are two most commonly used – SLF4J and JCL. JCL is known for its classloader hell, SLF4J is newer, better performing, smarter, simplier to use and generally provides better quality for the same buck (well, no buck – both are open source, of course), so we’ll stick to it. SLF4J is an adapter – thin layer of API to and from different logging implementations. Yap, both ways. It means with SLF4J we can use JUL API on top and log using Log4J in the bottom!

First we need to pick an actual logger. Log4j was considered the best choice up until recently (2006) when Ceki Gülcü decided he needed a fresh start and rewrote from scratch a new Java logging framework, just better than log4j, called Logback. We can give it a try as our underlying logging implementation (we can switch in a moment, as we are using good facade, remember?).

So, here’s what we have to do:

Establish our own good logging:
Done here. Now our own brand new code will use top-notch logging.
Now for the tricky part. Let’s make the example stack I listed above taking configuration from one source (our config files) and writing to one target (files, listed in our configuration)
1. All the tools using SLF4J will just work. That includes dozen of Apache projects, inc. Camel and Mina, some SpringSource projects and many others.
2. Now let’s start rolling with all the rest. This is how you do it (click to enlarge):
  1. Jakarta Commons Logging:
    1. Remove commons-logging.jar from your classpath. Usually, it is transitive dependency from the framework, so you need to instruct your build tool on how to do it. What a lucky coincidence, I just wrote short and instructive blog post about how to do it!
    2. Add jcl-over-slf4j.jar instead. It contains alternative commons-logging API implementation, so the code will run just fine.
  2. Log4J:
    1. Same goes here! Remove log4j.jar from your classpath (Again, it would usually be a transitive dependency from the framework, look here).
    2. Add log4j-over-slf4j.jar instead. It contains alternative log4j API implementation, so the code will run just fine.
  3. JUL:
    1. Well, you can’t remove JUL from classpath (it’s a part of the JRE, dude). For the same reason SLF4J can’t reimplement JUL’s API.
    2. Add jul-to-slf4j.jar. It will translate java.util.logging.LogRecord objects into their SLF4J equivalent.
    3. Install SLF4JBridgeHandler and LevelChangePropagator.
    4. Expect 20% decrease in performance (so use it wisely).

All done. Now both our code and all the 3rd paries configured from single source and write to single target. Hooray!

* syslog4j claims it is cross-platform. Well, I’ll just quote: “Is Syslog4j cross-platform? Yes! Syslog4j UDP/IP and TCP/IP clients should work in any typical Java JRE environment.”

Written by jbaruch Edit

22/06/2011 at 08:40

Posted in Frameworks

Tagged with jcl, jul, log, log4j, logback, slf4j

Filed under: Artifactory, JFrog Tagged: artifactory, jfrog, nuget, teamcity

HTML5 is the new DHTML

Aaron Gustafson — Fri, 03 Feb 2012 13:00:44 CST

For all intents and purposes, “HTML5” has become a meaningless catch-all marketing phrase defining a platform rather than a specification. It’s “DHTML” all over again.

Agile Lifecycles for Geographically Distributed Teams, Part 3

Johanna Rothman — Fri, 03 Feb 2012 11:01:37 CST

Example 3: Using a Project Manager with Iterations and Kanban and Silo’d Teams

Here, the developers were in Cambridge, MA, the product owners were in San Francisco, the testers were in Bangalore, and the project manager was always flying somewhere, because the project manager was shared among several projects. The developers knew about timeboxed iterations, so they used timeboxes. Senior management had made the decision to fire all the local testers and buy cheaper tester time over the developers’ objections and move the testing to Bangalore. The Indian testers were very smart, and unfamiliar with the product, so the developers suggested the testers test feature by feature inside the iteration.

The project manager suggested they use cumulative flow diagrams and cycle time measurements to make sure the developers were not developing “too fast” for the testers. The developers, still smarting over the loss of “their testers” were at first, peeved about this. They then realized the truth of this statement, and developed this kanban board.

You can see in this board, that four items are waiting to go into system test. Uh oh. The developers are out-producing what the testers can take. This is precisely what a kanban board can show you.

The testers aren’t stupid or slow. They are new. They cannot keep up with the developers. It’s a fact of life, not a mystery of life. The developers have to act in some way to help the testers or the entire project will fail. The reason they are working in timeboxes as well as using kanban is that they have several contractual deliverables, that management, bless their tiny little hearts, committed to. The timebox allows the team or the product owners to meet with their customers and show them their progress. (They were deciding who would meet when I last worked with the team.) The kanban board help make the progress even more transparent.

Iteration planning: The product owner and the project manager jointly work on the agile feature roadmap, and the product owner owns the roadmap responsibility for it. The product owner owns and generates the backlog. The product owner and the agile project manager present a strawman iteration backlog to the team at the start of the iteration. They have had difficulty finding iteration planning time that allows everyone to be awake and functioning, bless the senior managers’ little hearts.

Daily commitment: They do a handoff, asking each other what they completed that day and what the impediments are. If you have read Manage It!, you know I modified the three questions to “What did you complete, what are you planning to complete, what is in your way?”

Measurements: cumulative flow, average time to release a feature into the product. They are experimenting with burnup charts and impediment charts. They are still having trouble bringing the testers up to speed fast enough.

Yes, they do retrospectives at the end of each iteration. Yes, the product owners own the backlogs.

I’ll summarize in the final part, the next entry.

(Want to learn to work more effectively on your geographically distributed team? Join Shane Hastie and me in a workshop April 17-18, 2012.)

Tutorial: jQuery Mobile, HTML5 Mobile App Connected to Parse Mobile Backend

Max Katz — Thu, 02 Feb 2012 08:00:20 CST

A few weeks ago I promised to publish a tutorial on how to build a mobile app and connect it to Parse backend.

You can get the full tutorial here.

You really have to try it yourself building a mobile app in Tiggzi app builder and connecting it to Parse mobile backend is amazingly easy. You can literally build a real mobile app in about 30 minutes. Creating the actual backend is fast as well, takes about 2 minutes. Any backend object is instantly available via REST API and then used in the mobile app.

Here is a general outline of the tutorial:

Sign up for Parse and create the mobile backend. The mobile backend object (class) is a list of text messages
Build the mobile UI with jQuery Mobile and HTML5 (see screen shot below)
Create and use REST service to load text messages (REST: https://api.parse.com/1/classes/Words, GET request)
Create and use REST service to create a new texg message, (REST: https://api.parse.com/1/classes/Words, POST request)
Update the mobile UI to include a Delete button (REST: https://api.parse.com/1/classes/Words/objectId, DELETE request)
Create and use REST service to delete a text message
At any step in the tutorial, you can test the actual app in Web browser
Optionally, update the look and feel of the app

The final app looks like this:

Why an Agile Project Manager is Not a Scrum Master

Johanna Rothman — Wed, 01 Feb 2012 13:00:26 CST

A reader asked why the lifecycle in Agile Lifecycles for Geographically Distributed Teams, Part 1 is not Scrum. It’s not Scrum for these reasons:

The project manager and product owner start the release planning and ask the team if the release planning is ok. The team does not generate the initial draft of release planning itself. In Scrum, the team is supposed to generate all of the planning itself.
The checkin is different from the Scrum standup and the objectives of the checkin are different. I did suggest to the teams that if you want to create a cross-functional team where the functions are separated, if you ask people how they are working together, you might help them work together. Sometimes those questions work, and sometimes they don’t. It depends on the team and whether the people want to work together.

I didn’t mention retrospectives or backlogs in my examples so far, because I took them for granted. Yes, both examples of these teams do perform retrospectives and have product backlogs. They also have agile feature roadmaps, which are on my list to blog about.

The real difference is the difference between a Scrum Master and an Agile Project Manager. A Scrum Master is not a project manager. A scrum master does not manage risk by him or herself. A project manager will take on the risk management responsibility without asking the team.

A Scrum Master has only allegiance to the team. A project manager has responsibility to the team and to the organization. That means that the project manager might feel torn when the organization pressures the project manager to do something stupid. (Although, I just downloaded the Scrum Guide, and the Scrum Master’s responsibilities have grown considerably since I took my CSM with Jeff way back in 2006.)

But agile provides transparency when the organization asks the agile project manager to do something stupid, so it’s easier to retain your integrity as a project manager.

Want to move a feature higher in the backlog? Change the feature roadmap with the product owner and then change the backlog with the product owner. I expect the agile project manager to collaborate on the feature roadmap and the backlog with the product owner.

Want to change the velocity of the team to please some crazed manager? Both the Scrum Master or the agile project manager protects the team in these ways:

Explain that velocity is not a productivity metric
Say No and explain why
Play the Double Your Velocity schedule game
Or choose some other way to remove this management obstacle.

Agile makes it easy to protect the team. The question is this: does the Scrum Master have other responsibilities in addition to protecting the team or is the Scrum Master full time? An agile project manager tends to be full time on a geographically distributed team. Even on a geographically distributed team, a Scrum Master is not seen as a full time position. Bless their tiny little hearts, managers don’t seem to understand that transitioning to agile, especially for silo’d distributed teams with different cultural norms is non-trivial. They will make room for a project manager, but a Scrum Master? Oh no. Makes me nuts.

Cut corners on quality? I don’t see how. The team doesn’t meet the acceptance criteria on the stories and doesn’t meet their criteria of done for an iteration, and can’t show a demo. How does that serve anyone?

Help a team go faster? This is the one place where a project manager may have an edge over a Scrum Master, and that’s only because of education. An agile project manager is a project manager. That means he or she is actively studying project management, which means he or she is studying lean also, looking into work in progress. (I realize many project managers do not actively study project management.) I have high expectations of an agile project manager, and that is to limit WIP, work in progress, to measure cumulative flow. But, Johanna, that’s a lean project manager. Yes, that’s correct. Why not use all of the tools available to us at all times? This is not to help a team actually go faster, but to provide feedback to the team about their WIP. If everyone takes a story at the start of the iteration and everyone always works on their own story, it’s likely the team is at the slowest possible velocity. It’s worth knowing that, or at least retrospecting about the data. A project manager will gather the data. A Scrum Master, especially one who was not a trained project manager, may not know to gather the data.

I have nothing against Scrum Masters. Some of my good friends are CSTs (Certified Scrum Trainers). However, they are not all project managers, and have not been project managers, and have not studied the field of project management. Some have been. And, the real issue is this: In a two or three day workshop, they cannot convey to a person who may or may not have been a practicing project manager all of their project knowledge.

Organizations do not always pick project managers to be Scrum Masters. And, with good reason. Some project managers are command-and-control project managers. I suspect back in my long-ago past, I was. I gave it up long ago because it didn’t work. Some people never gave up command-and-control project management. Those people are not good project managers for agile projects. They are terrible project managers for geographically distributed projects, where you must work through influence.

You can have self-managing teams that are geographically distributed. You can have self-directed teams that are geographically distributed. But, they don’t start that way. They evolve into self-directed and self-managing teams. They start as management-led teams.

And, especially when they are silo’d teams, they need the coordination of a project manager, someone who will manage the risk between the silos, and someone who has the organizational backing, and yes, someone who has the allegiance to the organization to say, “We need to do this project” to write the project charter.

In a geographically distributed team, the agile project manager writes the project charter either with the team, or as a strawman for the people to edit and approve. Shane and I recommend that the people get together to write it together. We like it if people get together in person. We know how rarely that happens. (Penny wise, pound foolish.) So we teach people how to write a project charter when they are divided in space.

Because until there is a project charter, there is no organizing principle for the silo’d teams. Those developers in France, testers in Belarus, product managers and project manager in San Francisco, they all need something to coalesce around. The charter, which includes the project vision provides that. The iterations provide the project heartbeat.

So, that’s why I don’t think Agile Lifecycles for Geographically Distributed Teams, Part 1 is Scrum. It’s close, but no cigar. I respect Ken and Jeff’s work too much to call it Scrum when it’s not.

Now that I’m mostly recovered from my cold, I can continue the series about lifecycles.

(Want to learn to work more effectively on your geographically distributed team? Join Shane Hastie and me in a workshop April 17-18, 2012.)

HTML5 Cloud-based Mobile App Builder Now With jQuery Mobile version 1.0

Max Katz — Tue, 31 Jan 2012 13:00:19 CST

In mid January we have released an update to Tiggzi mobile app builder and now every app (new or old) is based on jQuery Mobile version 1.0.

Now that there is a Free plan, nothing should stop you from building your mobile app.

Static type checking and compilation in Groovy 2

Guillaume LaForge — Tue, 31 Jan 2012 11:00:16 CST

This Week in Grails (2012-04)

Burt Beckwith — Tue, 31 Jan 2012 08:00:26 CST

Peter Ledbrook started a HOWTO series this week for Grails; the first one is on upgrading to Grails 2.0 and there’s also a HOWTO on writing HOWTOs.

We could use your help finding invalid JIRA issues (e.g. already fixed or no longer an issue). Check out the section “Reviewing issues” in the wiki page on contributing to Grails for how to use the new “Last Reviewed” and “Flagged” fields.

Tomas Lin wrote up his experiences deploying Grails applications to AppFog. It’s great to know that there’s another viable cloud provider for Grails developers.

STS 2.9.0.M2 was released this week. Check out the new and noteworthy PDF for what’s been updated in the Groovy, Grails, and Gradle support.

I released a new plugin this week, to support using Ratpack in a Grails application. Thanks to James Williams for his post that got me started and inspired the plugin. And within a day there was already a blog post by Matthias Hryniszak on using the plugin.

If you want to keep up with these “This Week in Grails” posts you can access them directly via their category link or in an RSS reader with the feed for just these posts.

Translations of this post:

Miscellaneous Items
Plugins
Interesting Tweets
Jobs
User groups and Conferences

Miscellaneous Items

The Wizard Design Pattern
- Not Grails-specific, but an interesting approach, especially for DSLs and builders
Groovy DSL – A Simple Example
Apache Tomcat 7.0.25 released
Grails on Jelastic
Have you upgraded a #grails application to version 2 yet? Let us know how it went!
Custom Grails Test Types/Phases
- Older but cool
C.R.A.P. metrics for Grails
Static compilation for Groovy poll results
Groovy annotations for ToString and EqualsAndHashCode
- A good reminder, and useful tip about adding to the domain class template
Grails Database Migration Gotchas
Upgrading your app to Grails 2.0.0? Better wait for 2.0.1
Using the Tomcat 7 JDBC Connection Pool in Production
Grails: Bootstrapping data with DomainBuilder
Groovy Goodness: Solve Naming Conflicts with Builders
Static type checking talk from Paris Groovy/Grails User Group
Java Groovy / Grails Developer in Toronto
Grails, Weceem and Apache2
- Getting the GR8Conf sites running on new hardware
CacheManager’s diskStorePath
- Interesting stuff on configuring Ehcache
http://observatoriodegrails.com/hoy/?date=2012-01-23
http://observatoriodegrails.com/hoy/?date=2012-01-24
http://observatoriodegrails.com/hoy/?date=2012-01-25
http://observatoriodegrails.com/hoy/?date=2012-01-26
http://observatoriodegrails.com/hoy/?date=2012-01-27
This Week in Gradle (2012-4)
This Week in Spring, January 24th, 2012

Plugins

There were 3 new plugins released:

jquery-ui-timepicker version 0.9.8.1. Provides resources for http://trentrichardson.com/examples/timepicker/
eco-resources version 0.1. Compiles Eco templates to Javascript
ratpack version 1.0.1. Lets you use Ratpack inside Grails

and 17 updated plugins:

atmosphere version 0.4.2.1. Provides integration with the Atmosphere project, a portable AjaxPush/Comet and WebSocket framework
ckeditor version 3.6.2.1. Implements the integration layer between Grails and the CKEditor web rich text editor.
date-formatting version 0.2.5. Adds functions to the Date object to convert into various string formats
foundation version 2.1.4.3. Provides the Foundation CSS framework resources
google-visualization version 0.5. Provides a taglib for the interactive charts of the Google Visualization API
grails-melody version 1.11. Integrates the JavaMelody system monitoring tool
gsp-taglib version 0.3.2. Makes it possible to declare tags in a gsp in grails-app/taglib
jasper version 1.5.2. Enables use of JasperReports
rich-domain version 1.0.6. Provides dependency injection for POGOs that are not Grails domain classes
spring-security-mock version 1.0.1. Mock authentication support for Spring Security
spring-security-saml version 1.0.0.M11. SAML 2.x support for the Spring Security Plugin
spring-security-shibboleth-native-sp version 1.0.2. Shibboleth Naitive SP support for the Spring Security grails plugin
syntax-highlighter version 3.0.83. Adds a Syntax Highlighter for displaying code samples in GSP pages
tiny-mce version 3.4.7. Integrates the TinyMce editor javascript and tags to embed the editor in your GSP pages
twitter4j version 0.3.2. Send and receive Twitter messages using the twitter4j library
xwiki-rendering version 0.4. Convert texts using XWiki Rendering Framework
zkui version 0.4.1. Seamlessly integrates ZK with Grails’ infrastructures; uses the Grails’ infrastructures such as GSP, controllers rather than zk’s zul as in ZKGrails plugin

Interesting Tweets

@CedricChampeau: More details about my talk, static type checking at the Paris #groovy #grails user group. http://t.co/mUwPQcJD
@LondonGGUG: Anyone free and willing to talk about their #grails deployment setup at next GGUG? 20th Feb. Length 10-30 mins.
@joshareed: ‘git clean -df’ is great for cleaning up all the images, css, js that a grails upgrade re-creates
@wangjammer5: Great discussions with #Grails team this morning yielded this idea to replace install-plugin: http://t.co/fQcHbXeJ
@CodersUniverse: SpringPeople launches open house training in India: … Tomcat, Apache, Groovy & Grails, Testing, Databases, Ope… http://t.co/DfnHxmk7
@fifthposition: Grails 2.0 and emacs shell-mode: http://t.co/ruPXN0i2. (Thanks to @ataylor284 for this.) #grails #emacs
@aalmiray: hacking groovy/javafx by day; griffon/grails by night. 2012 is certainly looking interesting as days pass by
@gr8conf: We’re happy to announce, that SpringSource is the premium sponsor of #gr8conf Europe 2012. #groovy #grails #gradle #griffon #vmware
@nhoussos: New release of Free/Open source applicant tracking (e-Recruitment) system by EKT (developed in Grails!): http://t.co/n7qaC2Fy
@jatin_Shephertz: I created group Grails 4 Newbie on Linkedin.: http://t.co/FNjG7dEq
@bmuschko: Released #grails Google Visualization plugin v0.5: http://t.co/LL6aqysm. New visualizations, minor bugfixes, JS API taglib, updated docs.

Jobs

Groovy/Grails Developer (contract) in Slovakia
Sr. Developer Analyst (Grails) – Columbus, Ohio
Freelance Grails/Java udviklere – København
@SaraJaneJones2: @lakah Hey John, I’m looking for #groovy/grails developer for a client in #nyc. Would you have any referrals? Plz rt!
Groovy / Grails Developer in NYC
Java Groovy/Grails Developer – Toronto, ON, Canada
Java/Grails Developer in Phoenix, AZ
Groovy Grails Developer in NYC
Sr. Consultant (Java/Grails/iOS) at Object Partners in Minneapolis, MN
Remote Java / JVM / Grails Developer
@coryhodnett: Looking for a Web Developer for Java Script, Grails, & AJAX contract. Call me! 6022242486
@mobiquityinc: Still looking for #mobile #developers. Especially with #Cocos2D and #Grails experience. #Jobs #Boston #PVD.
Looking for Java Developers, Java / Grails developers
@gsuhm: Are you a #grails developer in Atlanta? Drop me a note for a great opportunity…
@jvandersande: 2 groovy/grails development roles for a new startup. Great company w/ a great story. Willing to look at java, python, ruby, etc engineers
@errr_: Are you a bad ass Java developer? Do you know groovy and grails? Do you also know vmware? We NEED you!! Shoot me your resume!
Java Dev – Grails in Nashville
Java, Grails/Groovy, Restful Web Services
Java Developer in Bristol, UK
Java udviklere søges (Grails, Groovy, Spring) – Sjælland & Hovedstad
Contract Java Developer in Vancouver
Senior Java Grails/Groovy Developer (contract) in NYC
@DevelopmenGC: Recherche Analyste programmeur Groovy (Grails) (Montréal)
Contactez : cv@developmen.ca
Lead Java Developer / Architect – Groovy, Grails in Northampton, UK

User groups and Conferences

Spring I/O 2012
- February 16^th and 17^th in Madrid

Thursday, February 9, 2012
- Seattle Groovy/Grails Users Group
March 8, 2012
- Seattle Groovy/Grails Users Group

Running Ratpack inside Grails

James Williams — Sun, 29 Jan 2012 13:00:15 CST

When I would present on Ratpack at conferences, one of the main questions I'd get would be around the migration path from Ratpack to Grails. I'd usually answer noting that you could run Ratpack along side Grails. It turns out it is incredibly simple to run a Ratpack app inside Grails. I don't think it's a long term solution and best as an intermediary step to migrate from one side to the other.

1. Grab and build Ratpack.

Provided you already have Gradle installed, run gradle buildDistro. The only file you need is the main Ratpack-x.x file. Grails will provide the Groovy and servlet container run time for you. Add this to the lib directory of your Grails app.

2. Create your Ratpack app class.

Putting the script into a proper class, as shown below, makes it easier to reference it from our Servlet.

import java.text.SimpleDateFormat

import com.bleedingwolf.ratpack.*

class SampleApp {

def app = Ratpack.app {

set 'port', 4999

get("/") {

def ua = headers['user-agent']

"Your user-agent: ${ua} from Ratpack"

}

get("/foo/:name") {

"Hello, ${urlparams.name}"

}

get("/person/:id") {

"Person #${urlparams.id}"

}

public static void main(String[] args) {

SampleServlet.serve(new SampleApp().app)

}

3. Subclass *RatpackServlet*.

If you don't mind getting an logging error, you can use something as simple as the following file:

import com.bleedingwolf.ratpack.*

class SampleServlet extends RatpackServlet {

			

            void init() {

		app = new SampleApp().app

}

4. Add the servlet information to your application.

There is a really confusing way to do it using the resources.groovy file but I prefer just adding it to the web.xml by hand. Run

grails install-templates

and navigate to src/templates/war/web.xml and add the following:

<servlet>

<servlet-name>SampleServlet</servlet-name>

	

        <servlet-class>SampleServlet</servlet-class>

	

       </servlet>

	<servlet-mapping>

<servlet-name>SampleServlet</servlet-name>

		

                <url-pattern>/ratpack/*</url-pattern>

</servlet-mapping>

The servlet needs to be put on a url-pattern that isn't root (/*) so I used (/ratpack/*). Run grails run-app and you're all set.

Tea leaf [Flickr]

Aaron Gustafson — Sun, 29 Jan 2012 11:00:16 CST

Aaron Gustafson posted a photo:

Grails plugin for Ratpack

Burt Beckwith — Sun, 29 Jan 2012 08:00:15 CST

I saw James Williams’ post on Running Ratpack inside Grails earlier this week and thought that it should be implemented as a Grails plugin, so I started playing with it. There isn’t much in the way of documentation yet, but what’s there is at the plugin portal page. The source for the plugin is here if you have any fixes or suggestions.

Server side action methods on JSF ValueChange events using AJAX listeners

Lincoln Baxter III — Sat, 28 Jan 2012 16:00:18 CST

How many hours have you wasted trying to get AJAX events working “just right in your fancy JSF 2 application, only to find that something just doesn’t quite work right? .. If you’re like me, many. But I’m about to show you a pattern that will make your heart sing. I have to thank Brian Leathem for his original idea to use CDI events in the ValueChangeListener, but when combined with a little <f:ajax> magic, there’s almost no limit to what you can do without writing a single line of JavaScript.

The following example code was taken from our latest Project: SocialPM, a budding Agile Project Management tool. How many times have you tried something like this in a loop, only to find that you have no way of accessing the loop variable (in this case, ‘s’) in your ValueChangeListener?

<ui:repeat value="#{stories.list}" var="s">
   ...
   <h:inputText id="priority" value="#{s.priority}" size="2" style="width: 20px;">
       <f:valueChangeListener binding="#{storyPriorityChanged}"/>
       <f:ajax execute="@this save" render="@form" />
   </h:inputText>
   ...
   <h:commandLink id="save" value="Save" action="#{stories.saveAjax(s)}" style="display:none;"/>	
</ui:repeat>

@RequestScoped
@Named("storyPriorityChanged")
public class StoryPriorityChanged implements ValueChangeListener
{
   @Override
   public void processValueChange(final ValueChangeEvent event) throws AbortProcessingException
   {
      if(...)
      {
         // do something with our event, but, we have no reference to '#{s]'
         // only to the old and new values of '#{s.priority}'
      }
   }
}

There are a variety of reasons why something like this might not work, none of which are your fault – most issues arise from subtleties in composite components, or looping constructs. But fortunately there is a solution, and it is called CDI.

A ValueChangeListener will execute before data is populated in the Model

And the ValueChange event is not an ActionEvent, so we cannot attach an <f:setPropertyActionListener> to set the value in our backing bean. Yet another limitation – there really should be an <f:setPropertyOnValueChangedListener>, and while we could write one ourselves, that would be fairly inconvenient.

Fortunately, there’s a better way to do both of these things, and it all hinges on our friends CDI and <f:ajax>. There are better ways to both make sure our loop instance variable is accessible, and also execute the command action.

A viable, reliable solution

Our code looks something like this, and before you start worrying about the fact that you need to write “more” or “more complicated” Java code, I’ll explain why this method actually loosens the coupling between your UI and your Model/Controller:

Notice here, that we are using the same ValueChange listener, but we are still not passing the value of ‘s’, our Story object, into the listener. We have also eliminated the need for invoking our <h:commandLink> by instead using the listener="" attribute on our <f:ajax> tag. This allows us to invoke any server-side EL method when the Ajax event occurs. Using EL 2.0, we can pass our Story ‘s’ into the method as a parameter.

<ui:repeat value="#{stories.list}" var="s">
   ...
   <h:inputText id="priority" value="#{stories.current.priority}" class="span1">
      <f:valueChangeListener binding="#{storyPriorityChanged}"/>
      <f:ajax execute="@form" render="pr" listener="#{storyCommandListener.save(stories.current)}" />
   </h:inputText>
   ...	
</ui:repeat>

But we still don’t have access to ‘s’ in our ValueChangeListener, so what?

This is where we need to pull out a few tricks from our CDI and Java Patterns playbook, so the first thing we need to do is re-think our strategy.

We know that the ValueChangeListener will be executed before our AJAX listener, so we need a way of performing our action without the Story ‘s’. Wait, how are we going to do that? The answer is our good friend the Command pattern.

The command pattern “is a design pattern in which an object is used to represent and encapsulate all the information needed to call a method at a later time. This information includes the method name, the object that owns the method and values for the method parameters.”

We will combine the command pattern with CDI Events in order to pass this event to a listener which will later invoke all queued commands when we trigger it with our AJAX listener method.

Our updated ValueChangeListener looks like this. Notice how we are not actually performing the operation yet, we are simply storing all information that we will need to perform this operation at a later time, when we do have access to our Story ‘s’. Once we have prepared the StoryCommand instance, we will fire it as a CDI event, using the the built in injected Event changeEvent mechanism.

@RequestScoped
@Named("storyPriorityChanged")
public class StoryPriorityChanged implements ValueChangeListener
{
   @Inject
   private Event<StoryCommand> changeEvent;
 
   @Override
   public void processValueChange(final ValueChangeEvent event) throws AbortProcessingException
   {
      StoryCommand command = new StoryCommand() {
 
         private final Integer oldVal = (Integer) event.getOldValue();
         private final Integer newVal = (Integer) event.getNewValue();
 
         @Override
         public void perform(Story story)
         {
            if(...)
            {
               // perform some action with the values from our event
            }
         }
      };
 
      changeEvent.fire(command);
   }
}

The events fired by our ValueChanceListener are captured by a method which @Observes StoryCommand command events, and stores them in a list until we are ready to use them.

The same class which observes these events is going to perform all queued commands when triggered by our AJAX listener: listener="#{storyCommandListener.save(stories.current)}". Now that we have access to our Story ‘s’, we can run each command passing in the Story on which it will operate.

@Named
@Stateless
@RequestScoped
public class StoryCommandListener
{
   @PersistenceContext
   private EntityManager ss;
 
   private final List<StoryCommand> commands = new ArrayList<StoryCommand>();
 
   protected StoryCommandListener()
   {}
 
   public void capture(@Observes StoryCommand command)
   {
      this.commands.add(command);
   }
 
   @TransactionAttribute
   public void save(Story s)
   {
      for (StoryCommand c : commands) {
         c.perform(s);
      }
      em.save(s);
   }
}

Just for Reference, this is what our StoryCommand class looks like:

public abstract class StoryCommand
{
   public abstract void perform(Story story);
}

Problem solved

We now have access to our Story object in the logic that needs to be executed, but why is this better than simply updating the database in our AJAX listener method itself? We could just do this, after all:

<ui:repeat value="#{stories.list}" var="s">
   ...
   <h:inputText id="priority" value="#{stories.current.priority}" class="span1">
      <f:ajax execute="@form" render="pr" listener="#{storyCommandListener.save(stories.current)}" />
   </h:inputText>
   ...
</ui:repeat>

My answer is, it depends on if you want to aggregate more than one change command before you save events to the database. You can also do things with CDI events that would clutter up your other code, like for instance, when you want to log change events into a notification feed as we are doing in SocialPM.

It is easier, cleaner, and a more decoupled system to have a second @Observes method that creates a status update when users perform operations, as opposed to integrating the notification service with every method call. In the end, it is a trade-off, but the real benefit is that this approach, using AJAX listeners is a quick, reliable way to build in complex AJAX functionality to your JSF2 website.

Cheers! If you want to check out more of this type of code in action, grab the SocialPM Source Code on GitHub, and give us a shout. We’d love to have some more hands in this project!

Review: Gradle Class with Luke Daley

Howard Lewis Ship — Sat, 28 Jan 2012 13:00:07 CST

Last week, Luke Daly arrived in Portland to teach a three day Gradle class; the folks at Gradleware were nice enough let me audit the class (so it only cost me a couple of thousand dollars of lost billing revenue to attend). My goals for the class was to gain a deeper understanding of how Gradle works, so that I could write more efficient builds, diagnose problems, and write my own plugins. The class scored very high on all of those counts!

Much of the first day was spent on basics, including a very useful introduction to the Groovy programming language (which is the basis of the Gradle DSL). Even though I have used Groovy pretty extensively for testing purposes over the last couple of years, there were features I've glossed over in the past that turned out to be very useful.

The class is taught largely bottom up: from the basics of declaring tasks, then actions on tasks, and gradually working up towards defining tasks inputs and outputs. Compiling Java came pretty late, which seems curious since that's the primary job of Gradle, but this makes sense from a bottom-up approach as SourceSets can then be described correctly.

The class alternates between lecture and discussion, and short focused labs. At the end of the third day, we worked on tuning up our own builds, passing the video projector cable around ... we had a good time making the Tapestry build more efficient and readable, as well as adding new features to it.

I'm feeling very good about my Gradle skills after attending the course; I've already been able to make further improvements to the Tapestry build subsequently, and I have plans for more involving things going forward.

The return on investment for this class is a bit tricky; Gradle simply does so much straight out of the box, and even without a strong understanding of its structure, you can kind of get it to do what you want just by guesswork and Googling. If you've been using Gradle for a while, I give this a cautious recommendation: getting back the three days of invested time may take a while to pay off. On the other hand, if you are currently dependent on Ant or Maven ... sign up for the next class and get yourself switched over, today!

Reply To: Cross-Platform Mobile Visual Development – a Tool Comparison

Max Katz — Sat, 28 Jan 2012 11:00:07 CST

My reply to this Cross-Platform Mobile Visual Development – a Tool Comparison. I tried posting this in the comments but that didn’t work. Tiggzi: the cloud-based mobile app builder should definitely be there.

Home page: http://tiggzi.com
Genesis: The cloud-based HTML5 and hybrid mobile app builder.
Version: Cloud (and white label)
Licensing: Free and premium — http://tiggzi.com/pricing
Download: Nothing to download, cloud-based
Documentation: http://help.gotiggr.com/videos
Sample application: http://tiggzi.com/common/showcase/browse.seam
Wikipedia: http://en.wikipedia.org/wiki/Mobile_application_development
Additional: The app is built with jQuery Mobile/HTML5. Connected to any REST API, tested, and exported as mobile Web, Android or iOS. PhoneGap is used for access to native device features.
Article on how to use: http://help.tiggzi.com/getting-started, Tiggzi and PhoneGap.

Tapestry 5.4: Focus on JavaScript

Howard Lewis Ship — Sat, 28 Jan 2012 08:00:16 CST

Tapestry 5.3.1 is out in the wild ... and if Tapestry is to stay relevant, Tapestry 5.4 is going to need to be something quite (r)evolutionary.

There was some confusion on the Tapestry developer mailing list in advance of this blog post; I'd alluded that it was coming, and some objected to such pronouncements coming out fully formed, without discussion. In reality, this is just a distillation of ideas, a starting point, and not a complete, finalized solution. If it's more detailed than some discussions of Tapestry's evolution in the past, that just means that the mailing list discussion and eventual implementation will be that much better informed.

In posts and other conversations, I've alluded to my vision for Tapestry 5.4. As always, the point of Tapestry is to allow developers to code less, deliver more, and that has been the focus of Tapestry on the server side: everything drives that point: terseness of code and templates, live r2eloading, and excellent feedback are critical factors there. Much of what went into Tapestry 5.3 strengthened those points ... enhancements to Tapestry's meta-programming capabilities, improvements to the IoC container, and reducing Tapestry's memory footprint in a number of ways. I have one client reporting a 30% reduction in memory utilization, and another reporting a 30 - 40% improvement in execution speed.

Interestingly, I think that for Tapestry to truly stay relevant, it needs to shift much, much, more of the emphasis to the client side. For some time, Tapestry has been walking a fine line with regards to the critical question of where does the application execute? Pre-Ajax, that was an easy question: the application runs on the server, with at most minor JavaScript tricks and validations on the client. As the use of Ajax has matured, and customer expectations for application behavior in the browser have expanded, it is no longer acceptable to say that Tapestry is page based, with limited Ajax enhancements. Increasingly, application flow and business logic need to execute in the browser, and the server-side's role is to orchestrate and facilitate the client-side application, as well as to act as a source and sink of data ultimately stored in a database.

As Tapestry's server-side has matured, the client side has not kept sufficient pace. Tapestry does include some excellent features, such as how it allows the server-side to drive client-side JavaScript in a modular and efficient way. However, that is increasingly insufficient ... and the tension caused by give-and-take between client-side and server-side logic has grown with each release.

Nowhere is this more evident than in how Tapestry addresses HTML forms. This has always been a tricky issue in Tapestry, because the dynamic rendering that can occur needs to be matched by dynamic form submission processing. In Tapestry, the approach is to serialize into the form instructions that will be used when the form is submitted (see the store() method of the FormSupport API). These instructions are used during the processing of the form submission request to re-configure the necessary components, and direct them to read their query parameters, perform validations, and push updated values back into server-side objects properties. If you've ever wondered what the t:formdata hidden input field inside every Tapestry forms is about ... well, now you know: it's a serialized stream of Java objects, GZipped and MIME encoded.

However, relative to many other things in Tapestry, this is a bit clumsy and limited. You start to notice this when you see the tepid response to questions on the mailing list such as "how to do cross-field validation?" Doing more complicated things, such as highly dynamic form layouts, or forms with even marginal relationships between fields, can be problematic (though still generally possible) ... but it requires a bit too much internal knowledge of Tapestry, and the in-browser results feel a bit kludgy, a bit clumsy. Tapestry starts to feel like it is getting in the way, and that's never acceptible.

Simply put, Tapestry's abstractions on forms and fields is both leaky and insufficient. Tapestry is trying to do too much, and simply can't keep up with modern, reasonable demands in terms of responsiveness and useability inside the client. We've become used to pages rebuilding and reformatting themselves even while we're typing. For Tapestry to understand how to process the form submission, it needs a model of what the form looks like on the client-side, and it simply doesn't have it. There isn't an effective way to do so without significantly restricting what is possible on the client side, or requiring much more data to be passed in requests, or stored server-side in the session.

The primary issue here is that overall form submission cycle, especially combined with Tapestry's need to serialize commands into the form (as the hidden t:formdata field). Once you add Ajax to this mix, where new fields and rules are created dynamically (on the server side) and installed into the client-side DOM ... well, it gets harder and harder to manage. Add in a few more complications (such as a mix of transient and persistent Hibernate entities, or dynamic creation of sub-entities and relationships) into a form, it can be a brain burner getting Tapestry to do the right thing when the form is submitted: you need to understand exactly how Tapestry processes that t:formdata information, and how to add your own callbacks into the callback stream to accomplish just exactly the right thing at just exactly the right time. Again, this is not the Tapestry way, where things are expected to just work.

Further, there is some doubt about even the desirability of the overall model. In many cases, it makes sense to batch together a series of changes to individual properties ... but in many more, it is just as desirable for individual changes to filter back to the server (and the database) as the user navigates. Form-submit-and-re-render is a green screen style of user interaction. Direct interaction is the expectation now, and that's something Tapestry should embrace.

What's the solution, then? Well, it's still very much a moving target. The goal is to make creating client-side JavaScript libraries easier, to make it easier to integrate with libraries such as jQuery (and its vast library of extensions), make things simpler and more efficient on the client side, and not sacrifice the features that make Tapestry fun and productive in the first place.

Overall Vision

The overall vision breaks down into a number of steps:

Reduce or remove outside dependencies
Modularize JavaScript
Change page initializations to use modules
Embrace client-side controller logic

Of course, all of these steps depend on the others, so there isn't a good order to discuss them.

Reducing and removing outside dependencies

Tapestry's client-side strength has always been lots of "out of the box" functionality: client-side validation, Zones and other Ajax-oriented behaviors, and a well-integrated system for performing page-level initializations.

However, this strength is also a weakness, since that out of the box behavior is too tightly tied to the Prototype and Scriptaculous libraries ... reasonable choices in 2006, but out-of-step with the industry today. Not just in terms of the momentum behind jQuery, but also in terms of very different approaches, such as Sencha/ExtJS and others.

It was a conscious decision in 2006 to not attempt to create an abstraction layer before I understood all the abstractions. I've had the intermediate time to embrace those abstractions. Now the big problem is momentum and backwards compatibility.

Be removing unnecessary behaviors, such as animations, we can reduce Tapestry's client-side needs. Tapestry needs to be able to attach event handlers to elements. It needs to be able to easily locate elements via unique ids, or via CSS selectors. It needs to be able to run Ajax requests and handle the responses, including dynamic updates to elements.

All of these things are reasonable to abstract, and by making it even easier to execute JavaScript as part of a page render or page update (something already present in Tapestry 5.3), currently built-in features (such as animations) can be delegated to the application, which is likely a better choice in any case.

Modularizing JavaScript

Tapestry has always been careful about avoiding client-side namespace polution. Through release 5.2, most of Tapestry's JavaScript was encapulated in the Tapestry object. In Tapestry 5.3, a second object, T5 was introduced with the intention that it gradually replace the original Tapestry object (but this post represents a change in direction).

However, that's not enough. Too often, users have created in-line JavaScript, or JavaScript libraries that defined "bare" variables and functions (that are ultimately added to the browser's window object). This causes problems, including collisions between components (that provide competing definitions of objects and functions), or behavior that varies depending on whether the JavaScript was added to the page as part of a full-page render, or via an Ajax partial page render.

The right approach is to encourage and embrace some form of JavaScript module architecture, where there are no explicit global variables or functions, and that all JavaScript is evaluated inside a function, allowing for private variables and functions.

Currently, I'm thinking in terms of RequireJS as the way to organize the JavaScript. Tapestry would faciliate organizing its own code into modules, as well as application-specific (or even page-specific) JavaScript modules. This would mean that de-referencing the T5 object would no longer occur (outside of some kind of temporary compatibility mode).

For example, clicking a button inside some container element might, under 5.3, publish an event using Tapestry's client-side publish/subscribe system. In the following example, the click events bubble up from the buttons (with the button CSS class name) to a container element, and are then published under the topic name button-clicked.

Consider this an abbreviated example, as it doesn't explain where the element variable is defined or initialized; the important part is the interaction with Tapestry's client-side library: the reference to the T5.pubsub.publish function.

Under 5.4, using the RequireJS require function, this might be coded instead as:

Here, the t5/pubsub module will be loaded by RequireJS and passed as a parameter into the function, which is automatically executed. So, this supports JavaScript modularization, and leverages RequireJS's ability to load modules on-the-fly, as needed.

Notice the difference between the two examples; in the first example, coding as a module was optional (but recommended), since the necessary publish() function was accessible either way. In the 5.4 example, coding using JavaScript modules is virtually required: the anonymous function passed to require() is effectively a module, but its only through the use of require() (or RequireJS's define()) that the publish() function can be accessed.

This is both the carrot and the stick; the carrot is how easy it is to declare dependencies and have them passed in to your function-as-a-module. The stick is that (eventually) the only way to access those dependencies is by providing a module and declaring dependencies.

Change page initializations to use modules

Tapestry has a reasonably sophisticated system for allowing components to describe their JavaScript requirements as they render, in the form of the JavaScriptSupport environmental (an environmental is a kind of per-thread/per-request service object). Methods on JavaScriptSupport allow a component to request that a JavaScript library be imported in the page (though this is most commonly accomplished using the Import annotation), and to request the initialization functions get executed.

Part of Tapestry's Ajax support is that in an Ajax request, the JavaScriptSupport methods can still be invoked, but a completely different implementation is responsible for integrating those requests into the overall reply (which in an Ajax request is a JSON object, rather than a simple stream of HTML).

Here's an example component from the TapX library:

The @Import annotation directs that a stack (a set of related JavaScript libraries, defined elsewhere) be imported into the page; alternately, the component could import any number of specific JavaScript files, located either in the web application context folder, or on the classpath.

Inside the afterRender() method, the code constructs a JSONObject of data needed on the client side to perform the operation. The call to addInitializerCall references a function by name: this function must be added to the T5.Initializers namespace object. Notice the naming: tapxExpando: a prefix to identify the library, and to prevent collisions with any other application or library that also added its own functions to the T5.initializers object.

The JavaScript library includes the function that will be invoked:

Under 5.4, this would largely be the same except:

There will be a specific Java package for each library (or the application) to store library modules.
The JavaScriptSupport environmental will have new methods to reference a function, inside a module, to invoke.
Stacks will consist not just of individual libraries, but also modules, following the naming and packaging convention.

Embrace client-side controller logic

The changes discussed so far only smooth out a few rough edges; they still position Tapestry code, running on the server, as driving the entire show.

As alluded to earlier; for any sophisticated user interface, the challenge is to coordinate the client-side user interface (in terms of form fields, DOM elements, and query parameters) with the server-side components; this is encoded into the hidden t:formdata field. However, it is my opinion that for any dynamic form, Tapestry is or near the end of the road for this approach.

Instead, it's time to embrace client-logic, written in JavaScript, in the browser. Specifically, break away from HTML forms, and embrace a more dynamic structure, one where "submitting" a form always works through an Ajax update ... and what is sent is not a simple set of query parameters and values, but a JSON representation of what was updated, changed, or created.

My specific vision is to integrate Backbone.js (or something quite similar), to move this logic solidly to the client side. This is a fundamental change: one where the client-side is free to change and reconfigure the UI in any way it likes, and is ultimately responsible for packaging up the completed data and sending it to the server.

When you are used to the BeanEditForm component, this might feel like a step backwards, as you end up responsible for writing a bit more code (in JavaScript) to implement the user interface, input validations, and relationships between fields. However, as fun as BeanEditForm is, the declarative approach to validation on the client and the server has proven to be limited and limiting, especially in the face of cross-field relationships. We could attempt to extend the declarative nature, introducing rules or even scripting languages to establish the relationships ... or we could move in a situation that puts the developer back in the driver's seat.

Further, there are some that will be concerned that this is a violation of the DRY pricipal; however I subscribe to different philosophy that client-side and server-side validation are fundamentally different in any case; this is discussed in an excellent blog post by Ian Bickling.

Certainly there will be components and services to assist with this process, in term of extracting data into JSON format, and converting JSON data into a set of updates to the server-side objects. There's also a number of security concerns that necessitate careful validation of what comes up from the client in the Ajax request. Further, there will be new bundled libraries to make it easier to build these dynamic user interfaces.

Conclusion

In this vision of Tapestry's future, the server-side framework starts to shift from the focus of all behavior to the facilitator: it paints the broad stokes on the server, but the key interactions end up working exclusively on the client.

I'm sure this view will be controversial: after all, on the surface, what the community really wants is just "jQuery instead of Prototype". However, all of the factors described in the above sections are, I feel, critical to keeping Tapestry relevant by embracing the client-side in the way that the client-side demands.

I think this change in focus is a big deal; I think it is also necessary for Tapestry to stay relevant in the medium to long term. I've heard from many individual developers (not necessarily Tapestry users) that what they really want is "just jQuery and a restful API"; I think Tapestry can be that restful API, but by leveraging many of Tapestry's other strengths, it can be a lot more. Building something right on the metal feels empowering ... until you hit all the infrastructure that Tapestry provides, including best-of-class exception reporting, on-the-fly JavaScript aggregation and minimization, and (of course) live class reloading during development. java

I'm eager to bring Tapestry to the forfront of web application development ... and to deliver it fast! Monitor the Tapestry developer mailing list to see how this all plays out.

The apparent results of meeting room boredom [Flickr]

Aaron Gustafson — Fri, 27 Jan 2012 16:00:37 CST

Aaron Gustafson posted a photo:

LinkedIn Etiquette

Howard Lewis Ship — Fri, 27 Jan 2012 13:01:06 CST

I've used LinkedIn for many years now, long before I joined Facebook ... I liked the concept of never losing contact information with business contacts and technologist. It just seemed like a good idea (though I do sometimes wonder if LinkedIn has any particular purpose).

I tend to only connect with people I've met in person, or at least talked to on the phone.

One thing that drives me crazy about LinkedIn is that you aren't forced to customize the message. As far as I'm concerned, the default message is like no message at all, and its a sign that you are just trolling for contacts. Just like you should always write a cover letter for a resume.

So the first rule of LinkedIn is: always customize the message. You chances of a connection go up significantly (at least with me) ... and with the default message, your chances drop down to near zero.

Batch Update All Your Vendor Repositories

Mike Girouard — Fri, 27 Jan 2012 11:01:20 CST

I’ve got just shy of 100 vendor repositories that I pull from for my side projects and client work. Updating them all has become quite a bother so I wrote this little guy to save me some time.

It’s certainly not perfect, and sometimes falls over, but this does the trick. This handles both Git and SVN repositories, but I’m sure with a few extra lines you can make it work for whatever SCM you prefer to use.

Tiggzi – The Cloud-based HTML5 and Hybrid Mobile App Builder, Now Free!

Max Katz — Fri, 27 Jan 2012 08:03:03 CST

Last week we have we have renamed our cloud-based mobile app builder to Tiggzi (from Tiggr). Tiggzi is the most powerful and easy to use cloud-based mobile app builder.

Free Plan!

But, even more interesting news (for you) is that now we have a Free plan. No credit card nothing, just sign up and start building your mobile app.

All the gotiggr.com links work and will continue working and we will be updating all the social media links. For now, use the links below to get help.

Email
Forum
Docs and Getting Started
Blog
Twitter
Facebook