All too often, software developers take ( or are forced to take ) the kitchen sink approach when adding features. Over the lifetime of a program or site, features are rarely, if ever, removed, only added. This despite the fact that every added feature increases the complexity and maintenance requirements exponentially. People should simply Write Less Code.

In that spirit, I found this post by Manton Reece very insightful. He talks about surveying his users to find out what they use. However, for me, here's the money quote:

I eventually did remove a feature, and the survey to customers served as a nice sanity check that the feature wasn't heavily used. The interesting part, to me, is that the feature I removed was the entire 1.0 product for Wii Transfer. Literally everything that 1.0 did is now gone.

Quite sobering. How many features of YOUR app are actually used?

One of my colleagues told me the other day: “Szczepan, last year, when I started working in the X team someone warned me not to speak too loud about unit testing” Apparently, there were feisty TDDers in the X team and they witch-hunted devs who not necessarily had written test-first. I guess it’s easy to become a [...]

Build failed today: ERROR: Failed to update http://svn.dev.sabre.com/svn/apd_centiva2/trunk org.tmatesoft.svn.core.SVNException: svn: blah blah blah ‘Bummer, I need to log in to the ci box and figure out what’s wrong with svn.’ – I thought. Being in the middle of something I didn’t get to it immediately. Then I got another notification from Hudson: Updated failed due to local files. Getting [...]

Not sure I like the punchline but the story is real. Once upon a time there was a project, pretty bad one actually. The were big performance and stability issues. The were no docs about functionality whatsoever. The code base quality was disastrous. There was even a local framework implemented that dealt with mapping java classes [...]

QuickSilver is an application launcher for Mac OS that is very popular productivity enhancer. It's extensible through a ton of plugins. But it also allows people to launch custom scripts, which allow people to quickly script any action they commonly perform. I first saw the possibilities when I read this article. Some of those scripts were quite useless to me, but there are a few I end up using all the time. Like shortcuts to turn my wireless radio off when Im not online and wanna save power. So this is about 3 QuickSilver scripts I recently wrote



Initially I wrote a few of them in applescript, but when I wanted to write the script to "automatically pastie my code", I decided it would be easier to use a language I'm more comfortable with. I found a ruby-applescript bridge, so I decided to go with ruby. The bridge is obtained by installing the rb-appscript gem. You can find on these scripts on Github at http://github.com/pathsny/QuickSilver



I also attempted a "maximise" script based on this article. Sadly it just doesn't work :/. It's always incredibly slow the first time you launch it.

http://citconf.com/raleigh-durham2010

I wanted to let everyone know about CITCON North America in Raleigh Durham on April 16 & 17 because I co-chair the conference and the more of you that come, the better the conference becomes. This is the 5th year for the conference. We started the conference back when I was at ThoughtWorks doing CruiseControl work and JTF was at Agitar. It’s great to see new and old faces alike each year. Maceij from Urbancode described it as “an annual reunion of sorts”.

I love the open spaces format, largely because I have no idea what I will learn until I get there. I learn whatever everyone else is interested in. That’s an amazing way to take in new information. It’s humbling. In fact it reminds me of one of the first concepts I had to accept when I studying the philosophy of Socrates in university, “Before we can learn anything, we must first admit we know nothing.”

Can’t make it to CITCON RD? Then make it to one of the other ones – Wellington, NZ – Prague, CZ – Singapore.

CITCON, the Continuous Integration and Testing Conference, is an open spaces conference in its 5th year. The topics aren’t decided until the first night of the conference, but past events have proven extraordinarily useful to anyone interested in Continuous Integration and the type of testing that goes with it. Developer-testers, tester-developers, tool creators, and Agile advocates have all gained a lot from attending.

The conference is free to attend! You have to pay $20 to register, but you get that back when you attend the conference. (We are using long time conference supporter Obtiva’s EventWax for registrations. Just follow the registration link below.)

This year, the North American event will be in Raleigh-Durham at the Sheraton Chapel Hill Hotel. Registration is open, but limited to the first 150 registrants. Sign up now to reserve your spot at this innovative, exciting, education, fun event.

http://citconf.com/raleigh-durham2010/register.php

The conference operates as a non-profit, so our advertising budget is basically zero. We can use your help to get the word out. If you know someone that might be interested, please pass this announcement along to them.

Also, if your company or a company you know would be interested in sponsoring CITCON, there are a wide range of sponsorship options.

http://citconf.com/raleigh-durham2010/sponsor.php

If you have any questions, please don’t hesitate to contact me.

I’ve wanted to write this post for a while, and reading “Metaphors we live by” has given me some language and ideas to express it in. So here goes.

Requirements come from above

In a straw-man Waterfall project, requirements are delivered to senior developers to design. Senior developers deliver the designs to junior developers to implement. Managers instruct teams in how to progress. We have, in our language and hierarchical organisation, a metaphor which maps “up” to the origin of the project, and “down” to the implementing details. We also think in terms of seniority and power, with the originators of the vision having that seniority and power, and the more junior developers and testers being at the bottom of the pile. We even talk about the team members “on the ground”.

Think of every organisational chart with the managers at the top, or the V-model in which the requirements are split into increasing detail towards the bottom.

In life, things naturally flow downwards.

Orders come from above

The other hierarchy with which we’re familiar is the military. We can map our employment and communication hierarchies to those of the military. We even talk about companies fighting for market share, defending their reputation, a hostile trading environment, captains of industry, command-and-control management, etc. It’s hardly surprising that we have, in our subconsciousness, another pattern: that the more junior members of a company should obey orders. (This isn’t necessarily true even in the military, but it fits our perception of it.)

Turning the world upside-down

I once heard of a business analyst who got tired of explaining the requirements to the developers. “I’ve told you three times already!” she snapped. “Everything’s clear. Just do it.” The BA sees the developers as working to fulfil her requirements. They serve her needs, rather than the other way around.

When we write and deliver software to a user on an Agile project, we ask them for their feedback.

• Is this useful to you?
• Is this easy to use?
• Is anything difficult to use?
• Does it help you to do your job more effectively?
• Can you think of any ways we could make this more intuitive?
• What would you like next?

Because we think of communication in terms of orders, we also think of junior staff delivering value to senior staff. We don’t necessarily think in terms of the communication itself being a form of delivery. If we did, we might ask for feedback from the users of our communication.

• Is this communication useful to you?
• How easy was our communication to use?
• Was anything difficult to understand?
• Did it help you to do your job more effectively?
• Can you think of any format you’d find more intuitive than this?
• Any questions?

If the BA above was a piece of software, her users would be filing bug reports, working around her, and using her competitors instead. I imagine instead that she’ll get a poor review and teams will prefer to work with her colleagues. If they only have the one BA to work with, the project will probably fail – the developers won’t be able to use her to get their job done.

Stakeholders aren’t users

I’ve written about this before, and it takes on a new importance in the context of users, and stakeholders, of communication. When we get a management report, we often think, “So what?” We hit the delete key. Instead, we could try to think, “Who is it that cares about us understanding this? Why does he care?” It’s often the case that a user is meant to do something, as part of his job, which is for the benefit of someone else. Similarly, we may be asked to understand or act on something for someone else’s benefit – and it won’t be the person delivering the message either.

The stakeholders of communication on a project are often stakeholders of the project itself – the security expert, the chief architect, the facilities manager, etc.

Project experience

Eric gave me the concept of a “Project experience”. In the same way that we can think of communication as a form of delivery, we can think of the experience that our stakeholders and customers have when they ask us, as a project team, to deliver their code. We can ask usability questions about the team.

• How easy is it to use the team?
• Is it easy to see what’s going on and get information about the progress of the team?
• Is it easy to undo a mistake?
• Is it easy to input a new idea?

We often hold retrospectives amongst ourselves to work out how to change our processes. I’d also like to see us actively getting feedback from the people who use the project. And next time someone gives you some instructions which are unclear or don’t help you to do your job, perhaps this metaphor will help.

• akoha / django-digest / wiki / Home — bitbucket.org

I’ve been in the IT industry for about 20 years now, and for nearly the last 8 years I’ve been a consultant with the rather excellent ThoughtWorks. Being anywhere for that long in our industry is quite uncommon, and to spend that long in a consulting role, with all the travel and disruption that implies, is even moreso. I’ve had a fantastic time with ThoughtWorks and I feel I’ve grown tremendously during the time I was there. Eventually something had to give though, so at the end of last year I decided it was time to try something else.

In particular I found I had moved away from the things I really enjoyed – writing software that matters and building high-performing software teams – more towards big organisational change, which, while it arguably has a bigger impact on an organisation, isn’t really where I wanted to be. So my criteria for what to do next came down to: writing business-critical software in a small, high-performing team, in an organisation that trusts its people and encourages them to excel. Having a great relationship with the consumers of that software and having them closely engaged with its delivery would be a huge plus.

Luckily for me such organisations do exist. One of them is a proprietary trading firm (basically a bunch of partners trading their own money) called DRW Trading, and it works a bit like this: The partners want to trade, so they have traders. In order to trade well the traders need to work with good financial models, so they have analysts who build those models. In order to trade effectively with these models they need good software, so they have programmers who build the software. That’s about it. The software development value stream is something like:

- I’ve had an idea for something that will make money. - Ok, here’s some software. - Thanks.

The procurement process seems equally onerous:

- I need a Thing. - You should get it then.

So there you have it. I’m taking a break from consulting, at least for a while, in order to rediscover good old-fashioned software delivery. I’m going to be doing a lot less on the conference circuit, although I don’t intend to vanish altogether, and hopefully I’ll be blogging and writing more now I’m doing less travelling and feeling less exhausted all the time.

Obviously I no longer have a ThoughtWorks email address, but you can contact me at dan@dannorth.net or I’m occasionally on Twitter as @tastapod. We now return you to your regular scheduled programme.

Since I am prancing through the playground that is Objective-C at the moment, I'm having to face concepts that have been pushed to the back of my C#'nd mind. Of course, I'm talking pre-C# 4.0 so assume the CLR and compiler are not collaborating to allow any form of duck typing, which is primarily what I'm interested in right now. I also don't count the C# libraries out there that work around the language and sealed classes because IMHO if the language doesn't let me do it then it's all just too much work on top of what I'm already writing. Yes, this even applies to assisted automated testing. Bring on C# 4.0 but until then, let's look at a language that complies... Objective-C.



This came up while looking at mocking in Objective-C and how the language makes it easy, whether you choose to use a mocking framework or implement something quickly yourself (as for how to do that, you'll have to wait until another post). Duck typing makes testing delightful in dynamically typed languages and I want to make sure that the tricks learned while testing in Ruby are not lost to me.



Take a look at some clear and concise code examples of Duck Typing on Wikipedia.



Over a conversation on IM with an ex-colleague, I had to spend quite a lot of time making it clear that a dynamic language does not mean the same thing as dynamic typing in a language. This is an error often made and voiced as if the terms are interchangeable. They are not.



Dynamic typing is when type checking is done at run-time, meaning while the program is executing and not at the time of compilation. The implication is that types are associated with values rather than variables. Run-time dynamism is different to dynamic typing, in that the language does a lot of things at run-time that other languages might do at compile time like adding code, optimising decision paths, extending objects and type checking (dynamic type checking). A dynamic language can have dynamic types but it does not have to.



Now one of the bonuses of this is that the type of an object can be used to make a decision while the program is running, to decide what code should be executed next. This is Duck Typing. The term comes from the idea that if something looks like a duck and quacks like a duck then it is probably a duck. That assumption allows for an instance of an object with the correct accessors or methods to be used without checking first if the object actually is of the right type. Of course, if the object does not have the right characteristics then is will throw a run-time error.



Run-time errors seem to scare people a lot more than I first realised. In a static world, a run-time error usually means that something very bad has happened in your compiled code with respect to resource management or the compiler corrupting the intent of your code. That is scary. Thing is that testing is a great way to check that your code isn't going to blow up unexpectedly, whether your typing is checked statically or not.



If you don't understand duck typing and are from a statically type checked world then you can end up hamstringing yourself by adding meta data to an object that is set and checked to indicate the type of an object. Explicit type checking in this case means you have missed the point and the advantages of the type system you are using. You must unlearn these habits and understand your language to use it correctly. Changing languages is not as simple as changing development environments. It's a mindset change. You as a developer have a responsibility to use your tools in the right way. You can always hammer a screw in to place but it's not necessarily the right way to do it. Maybe use a screwdriver and learn how it works. Many times, you will see a language that is written with the accent of another language and it works and looks almost right but simply isn't. Engineers who go from C# to Java and back will see themselves doing this and smack themselves on the hand when they realise. This is a habit to break.



While playing with Objective-C, I have seen compiler warnings given that won't stop you from running your code but will highlight that the compiler isn't sure that you are sure what you are doing. Explicit type casting will remove these warnings if you have the kind of OCD that doesn't allow you any warnings at all in what will ultimately become production code. For me, the majority of this is in my testing code so I'm more 'laxed than I might usually be.



C# has what is called nominative typing. In other words, you use the name of the object's type to determine what it is. To handle the type you use explicit naming to deal with what is expected. Duck typing allows part of an object's structure to be accessed at run-time to check compatibility. Before C# 4.0, any attempt to duck type sealed classes was shut down by the CLR. There are ways around that involve inheriting from your own interfaces that look similar to other interfaces but this is not sincere enough for me. If you must add these tricks to your code then maybe you are at the point of considering using another language that is more suitable for the job. Luckily for us .NET'ers, we can use a lovely language like C# for our glue and integrate a lot of other funky technologies to do cool stuff. I can't wait to see the next version of the language. Hopefully, the changes are real and not just work-arounds.



In summary, Duck Typing allows a method to use an object as long as the object it expects supports the methods or properties that the method is looking to call. The given object doesn't even have to inherit from the same base or interface. The object does not have to support all methods and properties to be passed into a method.



There is no actual point to this post. It's more a bunch of realisations as I journey from one language to another.



There is much space for misuse and making stuff go bang but that can be said about a lot of dynamic features of languages. You can't take all the power to create beauty away from developers simply because some may shoot themselves in the foot with it.



Dynamism doesn't kill people, people kill people.

I noticed the below snippet in Martin Fowler in his article on the subject of Mock based testing (Mocks Aren’t Stubs) “I’ve always been an old fashioned classic TDDer and thus far I don’t see any reason to change. I don’t see any compelling benefits for mockist TDD, and am concerned about the consequences of coupling [...]

• Setting up Django on SliceHost with Ubuntu Hardy, Postgres, Apache, and Nginx
  
  Punteney.com Topics  |  Projects Setting up Django on SliceHost with Ubuntu Hardy, Postgres, Apache, and Nginx Friday, May 2nd, 2008 Step by step process of setting up Django on an Ubuntu Hardy slice at Slicehost. This week I setup Django on a slicehost slice using Ubuntu Hardy for the OS, Postgresql for the db, Apache for serving the django app, and Nginx for serving the media and proxying to apache. For my future reference and hopefully to help out others here is the process I went through. Slicehost has some easy to follow and straight forward articles to start I just followed along with the articles. Setting up Ubuntu Start with the Ubuntu Hardy setup - page 1. You can follow this guide straight through. The only complicated part is when you do the “visudo” command to add yourself to the sudoers it puts you in the “vi” editor which isn’t very intuitive if you’re not use to it. To add your user to the list go to the end of the root ALL=(ALL) ALL line and type “a” to start appe
• Dynamic Dummy Image Generator - DummyImage.com
• CheddarGetter for Python and Django - FeedMagnet
• jaylett's django_session_stashable at master - GitHub
  
  Interesting mixin
• Startup Software Development – Do Your Homework Before You Develop Anything
• Email marketing software for web designers - Campaign Monitor

I am really starting to dislike the term "User Experience", but I'll get back to that.



In the mid-90s I was a bass player in the acoustic music scene in the South, living in Atlanta. If you happen to know Atlanta, to give you some perspective, my band opened New Years Eve at Eddie's Attic in 1994, and headlined New Years Eve in 1995 and 1996.



Eddie's Attic was and still is one of the most important and influential clubs on the acoustic music circuit in the South. Also on that circuit is a club in Nashville called The Bluebird Cafe.



The Bluebird is interesting because it enforces a strict no-talking policy. If you talk to your companions at all during a performance at the Bluebird, you are asked to leave the room.



At one time back in the 90s there was an intense discussion among people on the acoustic music scene as to whether Eddie's should implement a no-talking policy like the Bluebird's. As far as I could tell, the musicians who advocated the most for such a policy were the mediocre performers (you really couldn't get onstage at Eddie's if you were outright bad). The more that the performers had inattentive audiences, the less able they were to command a stage and hold the attention of the audience, the more likely they were to support a no-talking policy.



At the time there was a low-circulation newspaper devoted to the Atlanta acoustic music scene that interviewed me on the subject. In that interview I said three things: first, that if you intend to get on stage in the first place, it is your job to command that stage and compel the audience to listen solely by means of your own talent; second, that if you consistently have talkative audiences that don't pay attention, then you should either work on improving your performance, or else stop performing at all; third, that a no-talking policy robs performers of valuable feedback during the course of the performance.



I dislike the term "User". I think the word "user" has bad connotations and associations. I think it is too easy to turn "users" into "lusers" in our own minds. I far prefer the term "audience" to describe those who consume our software. It is only inanimate objects that have users. Performers have an audience.



Given that preference, I think the use of personas as proxies for types of users may not be a very good practice. It seems far too easy to exclude valuable segments of the potential audience and to miss valuable feedback by blinding oneself by the limits of the particular personas being considered as users.



I think a far more interesting and valid approach to UX work is to do things like instrument servers so that we can track in real time the activity of the largest groups of people using our software that we can muster, and to have that activity influence our development and delivery work as nearly instantaneously as possible. And of course, doing that also lets us know when the audience is not paying attention. This is exactly the feedback loop that audience applause provides performers on stage.

• Django Forgot Password | blog.montylounge
• simple links
• » How to do minimalism in steps | mnmlist.com

This presentation is so delightful, I weep that I was not present. And when I thought it could not get better, I learned the term duck-punching. Where have I been?

• Design Analyzer for SEO

I'm notoriously bad at getting my kids to school on time. It's an ongoing gag that you're only late to school if you're after me. My eldest daughter had suggested a checklist to help get get organised; so this morning we made one. So here it is. Most of the bad handwriting is mine. The kids added an item to the bottom of the list; and they scrupulously ticked each item off.

What or who is a Hub Director?

It’s my personal definition of anyone organizing a group, community, conference, unconference, barcamp, open space, event, meeting, or gathering revolving around one or more topics of interest. In fact I chose Hub because my idea and actual experience of a conference fits nicely with its etymology:

perhaps from hubbe, originally ‘lump’, the source of hob of a fireplace and hobnail, as in boots. A wheelwright’s word, not generally known or used until c.1828; it reached wider currency in connection with bicycles. Meaning ‘center of interest or activity or importance’first recorded 1858 in writings of Oliver W. Holmes.

I was originally leaning towards something like Hub Facilitator because facilitation is, again, more in line with my idea of the role but I wasn’t completely satisfied with it, something was missing. And then I thought about an orchestra Director and that was it 

No related posts.

Related posts brought to you by Yet Another Related Posts Plugin.

Andrew Beacock posts exactly the advice I needed: how to get the scala prompt working on Cygwin. Thanks!

Are you reading Jeremy Manson? Why not? (Hint: yet another reason finalizers are wrong even if Gosling won't say it. Alternatives here.)

Rob Bowley has set up a petition to ask the government to review the way in which IT projects are done.

I don’t know about you, but I’m fed up of hearing tales of Government IT failure. I even heard one story of a company asked to triple its bid so that a council department could spend, and therefore retain, its budget.

Go sign it already.

Reading Simon Baker’s post on metrics made me smile. I rant about similar misuse of metrics quite a lot. A common reason that I see targets and metrics fail is because they’re aimed at a perceived circle of responsibility – for instance, developers who are measured on how few bugs they produce. This ignores the possibility that something outside of the development team might be causing the bugs – poor analysis, a difficult technical environment, lack of a decent production-like staging platform, etc.

Simon’s post calls out another common local optimisation. He says, “…the true purpose of the people doing the work, which is to improve service and quality and satisfy users.”

One of the ways in which we can satisfy users is through usability and user experience. At one of the Agile 2009 keynotes, Jared M. Spool gave a beautiful example of why this could be a bad idea. He talked about a company that had reduced a 5-stage process to 2. The users loved it. “Unfortunately, they forgot that the company was paid per page impression,” he said. The company went bankrupt shortly after putting the beautiful new website in place.

In Feature Injection, Chris Matts calls out the people with the original idea, or the people who are investing in the idea, as the core stakeholders. These are the people who need to be satisfied. Everyone else is incidental, and the vision is usually one that either makes money, saves money or protects money (stops another company spoiling your idea, or stops them from stealing your market share). Everyone else whose input is needed to make the vision a success is an incidental stakeholder. This can include the users.

Here’s an example of how focusing on the true stakeholder can help. I was talking to a friend at XtC about a user-centric story.

As a user

I want register to be emailed when the game is released

So that I can buy it as quickly as possible.

I was discussing alternative ways of implementing this story which didn’t involve email. “Who really wants the users to be notified? Who’s actually paying to notify them?”

“Marketing, I guess.”

“Why do Marketing want the user to be emailed?”

“So that they can make a big hype about the game when it’s released.”

So, rephrase this from Marketing’s point of view.

In order to make a big hype about the game when it's released

As Fred, the Marketing guy

I want users to register for an email about the release.

Now we can see that perhaps, users may not want to register for the game. We might have to entice them with something else. We also see that our goal – the big hype – could be met by other methods; with advertising, for instance, or by getting favourable reviews in magazines.

By allowing the developers to focus on the goal, we may come up with different or better solutions. We also come up with different tests. Will the user actually want to navigate through the 5 screens we make them fill in to register? Will we get the big hype we’re looking for?

I find when using the Feature Injection template that I frequently put the real name of the stakeholder on the card. This means that if we run into trouble, we can go and talk to the stakeholder about alternate ways of achieving their goal.

They’re not user stories. They’re stakeholder stories.

• django-calais: a semantic tool for Django — django-calais v0.2 documentation

The writing-about-testing mail list began in September 2009, and has already played a part in a remarkable number of achievements:



The following writers appeared in print for the first time or contracted to be published in print for the first time:



Abby Fichtner

Dawn Cannan

Catherine Powell

Lanette Creamer

Parimala Shankaraiah



Lanette Creamer and Matt Heusser collaborated on an article about test automation in waterfall and agile projects.



Alan Page, Matt Heusser, and Marlena Compton published the "Code Coverage Cage Match" collaboration. http://www.stpcollaborative.com/knowledge/538-heusser-v-page-code-coverage-cage-match



Matt Heusser started a book project "Testers at Work".



Adam Goucher signed a contract for a book on Selenium.



Fiona Charles edited the Women of Influence issue of Software Test and Performance, with contributions from many list members. http://www.stpcollaborative.com/magazine/year/2010



Yvette Francino landed an editorial position at SearchSoftwareQuality.com



Dawn Cannan and Lisa Crispin collaborated on a piece for Agile Record #1. http://www.agilerecord.com Dawn's first conference presentation ever was in Second Life. http://www.passionatetester.com/2010/01/welcome-to-virtual-agile-world.html



If you would like to join the list, contact me or any other member for information. If we don't know you or your writing, please point out links to some public examples like blog posts, public documentation, conference papers, or similar work when you do so.



The deadline to apply for the Writing About Testing conference is now past. The conference is completely full with 15 attendees. Any future applications will go onto a waiting list. The attendees will be:



Elisabeth Hendrickson (http://testobsessed.com/)

Lisa Crispin (http://lisacrispin.com/wordpress/)

Geordie Keitt (http://tester.geordiekeitt.com/)

Marlena Compton (http://marlenacompton.com/)

Rich Hand (http://www.stpcollaborative.com/)

Joey McAllister (http://www.stickyminds.com)

Marisa Seal (http://thetestingblog.com/)

Jon Hagar (http://www.swtesting.com/)

Ben Simo (http://www.questioningsoftware.com/)

Marisa Burt (http://www.burtconsultinginc.com)

Lanette Creamer (http://blog.testyredhead.com/)

Yvette Francino (http://www.yvettefrancino.com)

Rick Scott (http://rickscott.posterous.com/)

Dawn Cannan (http://www.passionatetester.com/)

Chris McMahon (http://chrismcmahonsblog.blogspot.com/)

A wonderful post against magic by Stephan Schmidt.

Remember: If you don't understand it, don't use it.

• fíam » Building a website API with Django. Part 3: OAuth in Django
• django-voice - Project Hosting on Google Code

I’ve been thinking about high availability websites lately. In particular, I want sites that can be upgraded (including database migrations or even infrastructure changes) without downtime.

I’ve also been playing with node.js lately, and I decided to spike out a web proxy that would sit between users and the actual website (eg, a rails app). When performing upgrades, the proxy would hold users connections and wait. Once the upgrade was done, the proxy would forward requests as usual. Users would see an extra long request, but as long as the upgrade was short (eg, less than a minute), the user should not know the site was down.

This type of proxy server seems like a good fit with node. Node’s event model means that there will be very little overhead when holding connections. There are no threads stacking up and waiting. Since everything is non-blocking, this server should scale well.

Here is a very simple version of the code:

var posix = require('posix'),
      sys = require('sys'),
     http = require('http');

http.createServer(function (req, res) {
  checkBalanceFile(req, res);
}).listen(8000);

function checkBalanceFile(req, res) {
  var promise = posix.stat("balance");
  promise.addCallback(function () {
    passThroughOriginalRequest(req, res);
  });
  promise.addErrback(function () {
    setTimeout(function() {checkBalanceFile(req, res)}, 1000);
  });
}

function passThroughOriginalRequest(req, res) {
  var request = http.createClient(2000, "localhost").request("GET", req.url, {});
  request.finish(function (response) {
    res.sendHeader(response.statusCode, response.headers);
    response.addListener("body", function (chunk) {
      res.sendBody(chunk);
    });
    response.addListener("complete", function () {
      res.finish();
    });
  });
}

sys.puts('Server running at http://127.0.0.1:8000/');

Here is a gist if anyone would like to fork.

Basically, I use http.createServer to create a web server on port 8000. On incoming requests, I call checkBalanceFile. This method will try to stat a local file called balance. If it finds it, it will call passThroughOriginalRequest, which forwards the request to another web server on port 2000. If the balance file does not exist, I use setTimeout to call checkBalanceFile again in one second.

With a proxy server like this, the main application can be upgraded by removing the balance file. While the file is missing, the node web server will hold all of the connections and check every second for the reappearance of the balance file. Once it comes back, all requests will be forwarded along and then streamed back to the user.

Currently, this spike only works with GET requests and does not pass any headers through, since I wanted to keep the code simple.

At the first Software Craftsmanship Conference in London I met Steve Counsell. He's an academic at Brunel University with an interest in Object Orientation, Metrics and Refactoring.



A while ago Steve invited me to give a short presentation at one of a series of workshops he's running. These 'Reftest' workshops are aimed at bringing academia and industry together to share our insights and experiences with both Test Driven Development and Refactoring.



There were too many interesting presentations at last week's event for me to describe them all. But they should all eventually end up on the Reftest website so you can read them there. That site doesn't have a feed yet so I've set up this: http://www.google.com/notificationservice/webchanges/webfeeds/11897478376600010732 which basically notifies you of any changes to the website.



Anyway, there was a lovely presentation by Charles Tolman from Quantel about some of the issues you run into when you have a 12 year old codebase containing 17 million lines of C++ and Python. This lead into an interesting discussion about detecting duplicate code. I maintain Same but nowadays I find that Eric Raymond's Comparator is a better solution for most people so I recommended that.



One of my favourite asides from Charles's 30 years in software (20 of them spent at Quantel) was "tools should enable the intelligence of the programmer rather than attempt to encapsulate the intelligence of the programmer."



The folks from the University of Kent showed some impressive results for their tools which provide:

• model checking
• property-based testing
• interactive clone elimination

Best of all many of the principles behind them aren't restricted to Erlang. So these kinds of techniques and tools can be applied to mainstream languages. They just chose Erlang because the tools were built as part of the ProTest project they're doing with Ericsson.



My own presentation looked at some of the problems with TDD and Refactoring.



I discussed problems with teaching the cluster of tacit knowledge that we call TDD and covered issues like the convenient fictions we use, the mis-leading tutorials and the overly simplistic examples that involve starting with a blank slate. I also threw in a brief digression about the Norvig-Jeffries kerfuffle.



The section of my presentation on refactoring mostly involved going back to Opdyke's thesis and looking at some of the consequences of his ideas. One thing that struck me was how insightful Opdyke's work was when it came to the issues like preserving class invariants and the subtle side-effects of most refactorings. Somewhere along the line though as we automated refactoring we seem to have lost the heart of his ideas.



I'd expected the last section with it's references to model checking, property-based testing tools like ScalaCheck, mutation testing and fuzz testing to be controversial but the discussion ended up being a very productive look at ways in which we could fix the problems. The whole day made me very hopeful about the prospects for significant progress in the tools and techniques for TDD and refactoring.

---

---

Updated 2010/02/06: Ben Stopford has written up his perspective on the workshop and uploaded his PDF slides.

Make It Easy is a tiny framework to simplify writing Test Data Builders in Java, cutting down on duplicate and boilerplate code. There is also a .NET port by Graeme Foster.

I've used Test Data Builders on a few projects now and found that they really pay off when the team share a common system of names for syntactic sugar, such as static factory functions for creating new builders. I've often written a new test as I'd like to read it and found that the IDE prompts me to import all the syntactic sugar I need: other team members have already written it with exactly the same names as I would have used. This really speeds up the writing of tests and helps everyone write tests that are readable and flexible.

However, I've never been satisfied with the way of implementing Test Data Builders that I described previously on this site and in our book. There is far too much repetitive boilerplate code and duplication. All that code has to be written and then maintained as the system's domain classes evolve. Overall, I think Test Data Builders have a positive effect, but a lot can be done to reduce the overheads of using them.

For example, consider writing builders for the following class hierarchy:

public abstract class Fruit {
    private double ripeness = 0.0;

    public void ripen(double amount) {
        ripeness = Math.min(1.0, ripeness+amount);
    }
    ...
}

public class Apple extends Fruit {
    private int leaves;

    public Apple(int leaves) {
        this.leaves = leaves;
    }
    ...
}

public class Banana extends Fruit {
    public final double curve;

    public Banana(double curve) {
        this.curve = curve;
    }
    ...
}

To write a builder for, say, Bananas, I have to: declare the properties of a Banana; specify the default values of those properties; write a method to build a Banana with the property values collected by the builder; define a static factory function to act as syntactic sugar; make the constructor private to enforce use of the factory function; write chained "with..." methods that capture properties explicitly defined by tests; and write a "but" method so that tests can define builders relative to the properties of other builders. That requires a lot of code. For the Banana class, which has just two properties, it involves all of this:

public class BananaBuilder implements Builder<Banana> {
    private double ripeness = 0.0;
    private double curve = 0.5;

    private BananaBuilder() {}

    public static BananaBuilder aBanana() {
        return new BananaBuilder();
    }

    public Banana build() {
        Banana banana = new Banana(curve);
        banana.ripen(ripeness);
        return banana;
    }

    public BananaBuilder withRipeness(double ripeness){
        this.ripeness = ripeness;
        return this;
    }

    public BananaBuilder withCurve(double curve) {
        this.curve = curve;
        return this;
    }
    
    public BananaBuilder but() {
        return new BananaBuilder()
                .withRipeness(ripeness)
                .withCurve(curve);
    }
}

And I'd have to do the same for Apples.

Really, the only thing that I care about a builder of Bananas is: the properties of Banana objects; the default values used for each property when a test doesn't explicitly supply a value; how a Banana is constructed with those property values. The rest is boilerplate code.

Make It Easy is a framework that lets me create Test Data Builders by defining only those details. Make It Easy takes care of the boilerplate: gathering property values and defining the syntactic sugar that makes tests easy to read.

For example, to define a builder for Bananas, I only have to specify the following (and the ripeness property can be used for Apple builders as well).

public static final Property<Fruit,Double> ripeness = newProperty();
public static final Property<Banana,Double> curve = newProperty();

public static final Instantiator<Banana> Banana = new Instantiator<Banana>() {
    @Override
    public Banana instantiate(PropertyLookup<Banana> lookup) {
        Banana banana = new Banana(lookup.valueOf(curve, 0.5));
        banana.ripen(lookup.valueOf(ripeness, 0.0));
        return banana;
    }
};

The code to create Bananas with Make It Easy looks like this:

Banana defaultBanana = make(a(Banana));
Banana squishyBanana = make(a(Banana, with(ripeness, 1.0)));
Banana straightBanana = make(a(Banana, with(curve, 0.0)));

To create a builder (or "maker" in the framework's lingo) that can be used multiple times:

Maker<Banana> anIllegallyCurvedBananaWithinTheEU = 
    a(Banana, with(curve, 45.0));

Banana naughtyBanana = make(anIllegallyCurvedBananaWithinTheEU);

To define makers in terms of other makers:

Maker<Banana> aBananaThatCanBeUsedInTheManufactureOfSmoothies = 
    anIllegallyCurvedBananaWithinTheEU.but(with(ripeness,1.0));

There is also syntactic sugar for making lists and sets of objects. See the example code for more details.

Make It Easy 1.1.1 is available for download now.

Update 02/02/2010: Graeme Foster has ported Make It Easy to C#.

Image by Leonid Mamchenkov, used under the Creative Commons Attribution 2.0 Generic license.

My New Year's resolution was to write more, so here my thoughts on how to actually make that happen. In a sense it's a plan for myself to be more productive, but hopefully the ideas also work for other folks. Ironically, my work on EIP II has been stalled for a good while, so you're getting advice on how to be a prolific writer from someone who has not written much in 2 years. Read on at your own risk.

The beginning of a New Year is the time to reflect on the past and make resolutions for the future. It's become my tradition to kick off the year with some reflection on EIP, so here we go.

I joined JavaOne this year as a panelist on Cloud Computing. Here my belated impressions on this year's JavaOne.

By now everyone must have heard about Google Wave, the communication and collaboration platform announced at Google I/O. Google just announced that they are ramping up towards 20,000 developers on the current sandbox and will extend to 100,000 users by end of September. This will give many more people to opportunity to experiment with Wave.

I guess I am the last person to blog about Google I/O. By now, everyone should have read about Wave and the free Android phone (with 30 day SIM!). I am just now getting to write this up on my flight back to Japan, so I’ll give you my personal impressions and viewpoints, if you are still interested.

I have been speaking more frequently about cloud computing in recent days. As SOA is becoming a daily reality, I needed to advance to new, still slightly nebulous topic. What could be a better fit than cloud computing? All jokes aside, cloud computing provides for some interesting discussions and one should safely assume that Google has a word or two to say about it.

Blogging about design patterns seems about as original as blogging about the Java (TM) Programming Language, except for the missing (TM). However, as I just attended a workshop on software service engineering, I realized once again that people from different fields have very different notions about the concept and usage of patterns. Many workshop attendees raised questions about formalizing patterns, tooling, etc. I touched on this subject a tiny bit in a post from long time ago, but I figured that the topic really deserves more attention, especially since to me a pattern is so much more than the "proven solution to a recurring problem within a specific context."

The end of the year is always the time to reflect on the past happenings. My friends in Japan often send a New Year's card with 12 pictures, each showing the significant event during the month. I am not sure something significant related to Enterprise Integration Patterns happened each month, but I think it's still nice to reflect a little bit on the history and current state of the patterns. After all, the patterns have matured from a draft paper to being adopted as the lingua franca for a number of open source messaging projects.

Since Google's Developer Day, I have been promising to speak more regularly about Google products. This is a bit of a balancing act for me, as I want to avoid the slippery slope of becoming a corporate sock puppet. I also don't want to neglect my interest in connected systems, SOA, and asynchronous messaging. With the Internet turning into the largest connected system, the gap between Web-based technologies and connected systems design has become significantly smaller, allowing me to combine interests and Google developer products more naturally. Of course, the fact that Google is rolling out new developer products and APIs around the clock, does not hurt either.

I am just returning from a trip to Montreal and Keystone, CO for the OOPSLA and Colorado Software Summit conferences. I spoke on SOA Patterns, workshopped my conversation pattern paper, and gave six talks on event-driven architectures and building mashups using Yahoo! Pipes and Google Mashup Editor. Phew!

What’s wrong with this picture? I crash a stylish party at a pricey venue (SF Moma). I have no business being there, nor do I have the required invitation. I nonchalantly talk my way in, and hit the crowd with two fellow geeks. Within a few seconds an attractive blonde with a low cut dress approaches us and engages us in a conversation. We grab a few free drinks, indulge on the tirelessly circulating horse d’oeuvres, and talk to the blonde.

I attended the Enterprise Mashup Summit last Friday. It was a small-ish event, with about 50 people attending. We saw about 10 presentations, mostly by vendors plus an open forum. None of the presentations were too sales-ey, which gave the event a nice workshop feel. Given the limited size, it would have been nice to have more time for interaction. Here is a quick rundown of my impressions.

The Internet is cruel. So you finally made the leap from EAI stone age to become a hip mashup developer. Now it turns out you are once again behind the curve, Unless you've written your won Facebook app and scored a million or so installs you are just a worm in the dust of the global data superhighway. Join the club.

As I reported from Mashup Camp, an increasing number of vendors play in the mashup space. I am obviously not the only one who noticed. Dion Hincliffe recently discussed 17 different mashup tools. Maybe I have an MBA hidden inside me (or 10 years of consulting left some marks), but I was temped to conduct my own market segmentation. Gartner would be proud of me. Just be aware that my analysis is not meant to be comprehensive and all statements carry a 0.5 probability of being right :-)

Mashups pull data from different sources, aggregate and transform the data to be used in different contexts. EAI solutions pull data from different sources, aggregate and transform the data to be used in different contexts. Huh?

I attended the first day of Mashup Camp today. The event took place at the Computer History Museum, which is actually walking distance from my office. Ironically, the only day in the year when something actually happens within walking distance in Silicon Valley it had to rain. Oh well.

Last time I claimed that users like events. This time I want to show how I fulfilled my personal desire for events off the Web. I built two solutions to alert me to new book reviews on Amazon, one using a Python script, the other using Yahoo! Pipes.

Two weeks ago I presented a keynote at the DEBS (Distributed Event-based Systems) conference. The emphasis of the conference was on academic work in the field but event-based systems are rapidly gaining traction in commercial applications as CEP (Complex Event Processing) is becoming a household name like SOA (equally difficult to pin down). All this talk about events reminded me that applications on the Web are becoming increasingly event driven.

When graphical user interfaces became popular thrifty people had the idea that they could generate them automatically. Well, that bubble burst pretty quickly. When people say nothing is new with Web services, they are right in this case. Once again people want to generate the interface automatically.

Last week Google held a developer day in 10 cities around the world. Here are my take-aways.

You can only get away with joking about transactions for so long before the gods of transactions bring you to justice. So is was only a matter of time before Eric Newcomer would catch me waxing about coffee and transactional integrity...

Beginning of this year I submitted an article to a renowned software publication who is running a special issue on software patterns. Sadly, my article did not make it into the magazine but I decided to publish an expanded version on my site. Hey, I got to make it worth your time following my ramblings! Am I bitter that my article was rejected? Not at all, but if I ever get a hold of these #^%#$& -bleep- of a -bleep- I'll...

JavaOne is one of the few geek conferences that take place in San Francisco. The content can be hit or miss but the proximity and the good parties always make it worth stopping by. I also had to fulfil my duty to recruit the last smart software engineer in the Bay Area to Google.

I have been pretty quiet recently, but I finally have some results to show. The current issue of IEEE Internet Computing (May/June 2007) contains my guest column for "Toward Integration". Also, I authored one chapter in the upcoming book "SOA Expertenwissen" (in German).

Even though I have been working for the (self-declared) non-evil empire for a while now, I still have a lot of ties to the Microsoft community, even beyond drinking Weissbier with Christian Weyer. I am particularly proud that I have been able to maintain my MVP status, Microsoft's Most Valuable Professional award. Here my (brief) report from the MVP Summit in Seattle & Redmond.

Erik Doernenburg and I have been presenting our talk on Software Validation a few times now and have received very positive feedback. In the talk we mention that beyond visualization you can use similar techniques to perform checks and validations against a system. However, none of our demos actually included an example. So I went out to correct that and added validations to my messaging visualizer.

I am just riding the train back from Aarhus (the second largest city in Denmark in case your geography skills are as weak as mine were before my first trip here) to Copenhagen. Many speakers consistently rave about JAOO. The speaker roster, the logistics, the attendees – everything just seems to be very high quality and run very smoothly. This year I was nominated to be track host for the SOA track entitled "SOA – What's Left to Say?"