• Joe Duffy's Weblog
  
  Building a custom thread pool

Then no software would get written, but at least we wouldn't have any crap software...

My friend Jay posted two blog entries yesterday on the subject of raising the standard for programmers in our industry. I left a comment about it on his blog and talked with him about it on IM, but I figured I'd expand a bit on my thoughts on it here.

First things first, I completely disagree with him.

I don't think that 50% of programmers should leave the profession. I've worked on a lot of projects with a lot of different teams and I can't ever remember thinking "man, this project would be a lot easier if I got rid of half of this team."

I can also only think of few Net Negative Producing Programmers that I've worked with and most of them usually leave pretty quickly.

Most of the time, I've found that when a person isn't performing up to par, it's because of a management failure. I think that it's up to the development leads to analyze their teams and figure out how everyone can best contribute and make sure that people are performing at the top of their game.

Too often I've seen people fall into the trap of always being assigned to, or always choosing to work on, the easiest features in the project. They then do a half-assed job because they're always working on the boring part of the project and don't feel like a valued part of the team.

Maybe, for a change, we should assign those people the hardest part of the project for a few weeks. Sure, they might not so the best job, but they will work harder because, for once, what they're working on is challenging and exciting. Plus they know that the entire team is depending on them and is there to help them. That's how you bring people back into the fold and make them a productive, happy member of the team.

The point that's neglected in Jay's arguments is that experienced programmers shouldn't just be responsible for writing code. It should also be the job of experienced programmers to take those who are less experienced or interested and pull them up to our level.

Forcing them out of the profession is just as much of a failure on our part as it is on theirs.

Update: I've since updated and moved this script. See my new blog post.

I had a chance to pair with Rob today. On his console window, I noticed something I wanted: his bash prompt had an indication that he was in a Git repository along with the branch he was on. Later, after he'd left, I found myself wondering how that worked and that I simply had to have it, and I wasn't willing to wait a whole day to figure out how he'd done it. Patience is not a virtue. (In that spirit, click here for the code if you're too impatient to read the rest of this blog entry which I spent countless hours writing. Uphill both ways. In the snow.)

A few minutes of googling turned up a blog entry with relevant bash fu. The code there was for changing the prompt such that it indicated whether you were using svn, git, svk or Mercurial along with some repository metadata. A couple of minutes later (it took a while to copy and paste the code since it was interweaved with comments), I found out it didn't work on Leopard. readlink turned out to be the culprit. Instead of figuring out why it wasn't working, I replaced it with something simpler (specifically I used: base_dir=`cd $base_dir; pwd`). Voila, things worked: I now had a hella cool prompt that showed me if I was in a Subversion or a Git repository - I never used Mercurial, and left SVK for Git a couple of months ago. Yay.

Being who I am I had to do something more with this new ability to distinguish between Subversion and Git. And I knew exactly which itch to scratch: 90% of the time I use the same scm commands. I even had shell aliases for them so I wouldn't have to type them. A few minutes later I'd converted my previously Subversion specific aliases to generic ones that worked with Git too. Screenshot says it all:

If you want the same, download the script as ~/.bash_vcs and add source ~/.bash_vcs at the end of your ~/.bash_profile.

Things sure have been quiet around here...

As some of you may already know, I've left the relative safety of my job at Collaborative Drug Discovery to have another go at starting a company.

I'm currently working with a friend to once again create something out of nothing and bring yet another crazy idea into the world. Therefore, if you see me and I look like I just got finished working for 18 hours straight, please buy me a beer and put me to bed.

As for the new company, I'm not ready to talk about what we're doing yet, but I'm sure that I'll be writing about it here as it comes together.

Stuart Halloway posts a clever take on a successor to Java: all of the above. This is part one of a series based on his No Fluff, Just Stuff presentation.

His post also gave me a quick look at side-by-side syntax for common features. I was surprised that out of clojure, jruby, groovy and scala, I preferred groovy.

I think I blew Steve Conover’s mind last week.

He, like me, and every other sane rational person I’ve ever met, thinks that when you say “line up” or “line down” in a text editor, it should take the line(s) you are currently on and move them up, in their entirety. Furthermore, that if you press “line up” 5 times, the line(s) you are currently on should move up 5 lines.

Textmate, as much as I love it, does not do this, and this has long been a source of contention and strife among my colleagues at Pivotal.

This, however, is a story with a happy ending. Last week, Steve and I were pairing on Textmate, and he was complaining that line up / down wasn’t working properly. A couple hours later, I had a brilliant insight. This is how you fix Textmate’s line up / down :

Pull up textmate and type :

command-option-M
command-shift-L
command-control-up arrow
command-option-M
control-option-m

This will start recording a macro, select the current line, move the current line up, stop recording the macro, and then save the macro.

Next, name your macro “line up” in the dialog that just popped up, and give it a key equivalent of command-control-up arrow.

Congratulations, you just fixed line up!

Now let’s do the same with line down :

command-option-M
command-shift-L
command-control-down arrow
command-option-M
control-option-m

This time, name it “line down”, and give it a key equivalent of command-control-down arrow.

You of course don’t have to do all this with keyboard shortcuts, but it’s really impressive if you do :)

So we’re working with Amazon’s SimpleDB. It’s pretty sweet, though the ruby libraries for it are still a bit primitive. One of the problems you run up against when you’re writing integration tests against it is eventual consistency.

1
2
3
4
5

it "should save" do
  customer = Customer.create!(:name => 'bob', :email => 'bob@example.com')
  customer = Customer.find(customer.key)
  customer.name.should == 'bob'
end

The way SimpleDB works, you’re assured that Customer.find will work…eventually, but not right away.

For a couple days we contented ourselves to just run the integration tests a couple times until they didn’t error out. But that got old.

Enter “eventually” :

1
2
3
4
5

it "should save" do
  customer = Customer.create!(:name => 'bob', :email => 'bob@example.com')
  customer = eventually { Customer.find(customer.key) }
  customer.name.should == 'bob'
end

It’s a very simple method (below) that just retries the passed in block until it succeeds, timing out after 10 tries. Super simple, works like a charm. Thank you ruby.

Here’s the source :

1
2
3
4
5
6
7

def eventually(tries = 0, &block)
  yield
rescue
  raise if tries >= 10
  sleep 0.5
  eventually(tries + 1, &block)
end

I will say, I can’t help but smile every time I write “eventually” in a test… :)

• Talis Platform - Home
  
  Interesting semantic platform.
• Ehcache ehcache
  
  Distributed cache for java.
• Ryo Heihase Homepage
  
  Very interesting / disturbing new artist.
• Five Amazing Color Palette Generators - ReadWriteWeb
• .NET, LINQ, and RDF based Linked Data (Update)

An architectural pattern that I have observed a few investment banks implement is a distributed memory cache accessed via a RESTful front-end over HTTP for providing access to market data (e.g.. stock prices, interest rate curves, or derived values like volatility surfaces & correlations) and static data (e.g. counterparty details, settlement defaults). The distributed cache can be ‘easily’ scaled to hold massive data sets and the front-end allows the data to be accessed in a technology agnostic fashion, as long as the client can speak HTTP. These caches tend to use a commercial product such as Oracle (nee Tangosol) Coherence, Gigaspaces Enterprise Data Grid or GemStone GemFire. The REST layer is written in-house however. In the open-source world it appears that Ehcache now has a ‘RESTful, resource-oriented’ layer. Given the growing popularity of REST versus WS-DeathStar its possible that Coherence et al will provide some out-of-the-box support for REST as well. Microsoft might consider pairing Velocity with ASP.NET MVC?

On Friday I came across a nasty bug in selenium (but I still like it).

Selenium wouldn't start

The build was failing - and the reason was the last thing I expected (of course). The usually reliable selenium server just wouldn't start. Looking at the stack trace, then searching for likely looking parts of it (btw, for the search I tried, google, yahoo and altavista find the answer, but msn, cuil and askjeeves don't. Just thought I'd try them out because of the launch of cuil).

I found the answer (I think you can turn the popup blocker back on again too - I think the problem is because the registry entry is missing rather than because of its value).

The stack trace is like this (in case you are searching too):

java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(Unknown Source)
at org.openqa.selenium.server.browserlaunchers.WindowsUtils$RegKeyValue.<init>(WindowsUtils.java:622)
at org.openqa.selenium.server.browserlaunchers.WindowsUtils.deleteRegistryValue(WindowsUtils.java:577)
at org.openqa.selenium.server.browserlaunchers.WindowsProxyManager.handleEvilPopupMgrBackup(WindowsProxyManager.java:106)
at org.openqa.selenium.server.browserlaunchers.WindowsProxyManager.init(WindowsProxyManager.java:81)
at org.openqa.selenium.server.browserlaunchers.WindowsProxyManager.<init>(WindowsProxyManager.java:65)
at org.openqa.selenium.server.browserlaunchers.InternetExplorerCustomProxyLauncher.<init>(InternetExplorerCustomProxyLauncher.java:48)
at org.openqa.selenium.server.browserlaunchers.InternetExplorerCustomProxyLauncher.<init>(InternetExplorerCustomProxyLauncher.java:41)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.openqa.selenium.server.browserlaunchers.BrowserLauncherFactory.createBrowserLauncher(BrowserLauncherFactory.java:124)
at org.openqa.selenium.server.browserlaunchers.BrowserLauncherFactory.getBrowserLauncher(BrowserLauncherFactory.java:81)
at org.openqa.selenium.server.htmlrunner.HTMLLauncher.runHTMLSuite(HTMLLauncher.java:87)
at org.openqa.selenium.server.htmlrunner.HTMLLauncher.runHTMLSuite(HTMLLauncher.java:141)
at org.openqa.selenium.server.SeleniumServer.runHtmlSuite(SeleniumServer.java:1138)
at org.openqa.selenium.server.SeleniumServer.main(SeleniumServer.java:386)

Now that wasn't so difficult was it?

The problem was caused by something I didn't do (but rather by some remote machine update thingie) which made it difficult, and the error message didn't give much of a clue (although looking at it carefully it does look like the registry value for the EvilPopupMgr can't be deleted because it doesn't currently exist).

Having fixed the problem the CI build still won't work

Having thought I'd fixed the problem - I fix up the CI build farm but the build still fails. Running the build manually on the CI farm machines is fine - running through the CI server isn't. The problem is that the CI server doesn't run the build as me - doh - so the "HKEY_CURRENT_USER" I've just fixed isn't the right one!

Time for a cup of tea

From the searches I've done it looks like I'm not the only person who has come across this problem - I hope by writing it up it'll save someone a bit of time. What would save even more time would be if the bug gets fixed - there is a suitable looking issue open for this.

I have just released ehcache-server-0.3, which includes a fully functional RESTful, resource-oriented implementation. The standalone-server has also been updated to 0.3.

Cache API

CacheManager Resource Operations

OPTIONS /

Lists the methods supported by the CacheManager resource

GET /

Lists the Caches in the CacheManager.

Cache Resource Operations

OPTIONS /{cache}/

Lists the methods supported by the Cache resource

GET /{cache}

Lists the elements in the cache.

PUT /{cache}

Creates a Cache using the defaultCache configuration.

DELETE / {cache}

Deletes the Cache.

Element Resource Operations

OPTIONS /{cache}/{element}

Lists the methods supported by the Element resource

HEAD /{cache}/{element}

Retrieves the same metadata a GET would receive returned as HTTP headers. There is no body returned.

GET /{cache}/{element}

Gets the element.

HEAD /{cache}/{element}

Gets the element's metadata.

PUT /{cache}/{element}

Puts and element into the Cache.

DELETE / {cache}/{element}

Deletes the element from the cache.

Simple Client Example

Put XML into cache

String xmlDocument = "\n" +

"\n" +

"Say goodnight,\n" +

"Gracie.\n" +

"Goodnight, \n" +

"Gracie.\n" +

"\n" +

"";

HttpUtil.put("http://localhost:8080/ehcache/rest/sampleCache2/2", "application/xml",

new ByteArrayInputStream(xmlDocument.getBytes()));

The server will respond with 201 for success.

Get XML example

HttpURLConnection urlConnection = HttpUtil.get("http://localhost:8080/ehcache/rest/sampleCache2/2");

assertEquals(200, urlConnection.getResponseCode());

assertTrue(urlConnection.getContentType().matches("application/xml"));

DocumentBuilder documentBuilder = DocumentBuilderFactory.newInstance().newDocumentBuilder();

Document document = documentBuilder.parse(urlConnection.getInputStream());

XPath xpath = XPathFactory.newInstance().newXPath();

String expression = "/oldjoke/burns";

Node node = (Node) xpath.evaluate(expression, document, XPathConstants.NODE);

assertEquals("burns", node.getNodeName());

The server will respond with 200 for success and provide the data along with Content Type of "application/xml"

Get with wget example

This one shows how to do a get with the Unix utiiity wget:

wget -S http://localhost:8080/ehcache/rest/sampleCache2/1

--15:11:54-- http://localhost:8080/ehcache/rest/sampleCache2/1

=> `1'

Resolving localhost... 127.0.0.1, ::1

Connecting to localhost|127.0.0.1|:8080... connected.

HTTP request sent, awaiting response...

HTTP/1.1 200 OK

X-Powered-By: Servlet/2.5

Server: GlassFish/v3

Expires: Sun Aug 17 17:12:55 EST 292278994

Last-Modified: Sun, 03 Aug 2008 05:07:50 GMT

ETag: "1217740070067"

Content-Type: application/octet-stream

Content-Length: 512

Date: Sun, 03 Aug 2008 05:11:54 GMT

Connection: Keep-Alive

Length: 512 [*/*]

100%[=============================================================================================================>] 512 --.--K/s

15:11:54 (8.28 MB/s) - `1' saved [512/512]

Ehcache server fully supports wget, including the site mirroring capabilities.

Getting Started

Read the manual here: http://ehcache.sourceforge.net/documentation/cache_server.html

Check out the project and work your way through the tests. These show how to put and get different content with different mime types and how to use HEAD, PUT, DELETE, OPTIONS and GET. Ehcache server supports conditional GET and follows normal HTTP caching conventions.

I hope to follow up with some articles on how to use ehcache-server with different languages.

Spnego is an SPNEGO and Kerberos plugin for Glassfish. SPNEGO stands for Simple and Protected GSSAPI Negotiation Mechanism. SPNEGO is a standard GSSAPI pseudo-mechanism for peers to determine which GSSAPI mechanisms are shared, select one and then establish a security context with it. Kerberos is a computer network authentication protocol, which allows individuals communicating over an insecure network to prove their identity to one another in a secure manner. Spnego is available under a CDDL v1.0 open source license and is actively developed, maintained and supported.

Version 1.1 has just been released.

Building from source was problematic for the 1.0 release due to some dependencies on snaphot versions. This has been fixed in 1.1.

Version 1.1 has been released as tarballs and also into java.net's Maven repository.

The spnego web site also has a new home: http://spnego.dev.java.net. Previously we were redirecting to another site which would work with the maven site plugin. Those issues have been overcome.

The documentation has been extensively revised, based on issues that were causing new users to trip up.

Spnego has been also been tested with Glassfish V2 UR1 and Java 6 update 7.

See the project at http://spnego.dev.java.net.

We’re very pleased to announce our keynote speakers for this year’s London XpDay.

Chris Ambler, European QA Director for games company Electronic Arts, tests some of the most complex software in the world

Daniel Jones and Marc Baker, Lean Enterprise Academy, are two of the most established Lean consultants in the world. Jones wrote, with James Womack, wrote “The Machine That Changed the World”, the standard book on the Toyota Production System.

There’s still time to propose your session.

• swoop - Google Code
  
  Ontology editor
• Pellet: The Open Source OWL DL Reasoner
  
  OWL reasoner
• D2RQ - Treating Non-RDF Databases as Virtual RDF Graphs - Chris Bizer

I have been happy lately to see the progress in Functional Java. A project truly arrives when it becomes the inspiration for other clever ideas. Witness Lazy Error Handling in Java, Part 1: The Thrower Functor.

Every few days comes another fun bit of functional programming creeping its way in to my workaday language, Java. Current wish list item: the fj team deploys to a public Maven repository so I can just say:

<dependency>
    <groupId>fj</groupId>
    <artifactId>functionaljava</artifactId>
    <version>2.8</version>
</dependency>

UPDATE: This just gets more interesting: Lazy Error Handling in Java, Part 2: Thrower is a Monad and Lazy Error Handling in Java, Part 3: Throwing Away Throws.

I've been meaning to start doing a series of blog entries covering interesting WebDriver checkins. Quite what "interesting" means is open to interpretation --- I doubt that my Mum will ever find them as fascinating as I do --- but covers any checkin solving difficult problems, adding new capabilities or that were just plain fun to write.

The first of these is adding the ability to pass arguments to Javascript being executed from within WebDriver. The basic idea is that it would "teh new hotness" to be able to do something like this:

WebDriver driver = new FirefoxDriver();
WebElement element = driver.findElement(By.id("someid"));
String innerHtml = (String) ((JavascriptExecutor) driver).executeScript("return arguments[0].innerHTML;", element);

The magic is that last line. Conceptually, the element retrieved in the Java code represents an underlying element in the DOM. It should be possible to use this. In the HEAD versions of WebDriver, it already is.

The original checkin created a magic variable ("parameters") do hold the arguments. The version that's now in WebDriver just uses the normal "arguments" magic variable, as if the script we're executing has been wrapped in an anonymous function and then had "call" (umm...) called on it.

Let's walk through this original patch :)

The first thing that jumps out is that this only covers Firefox. Personally, I find that Firefox is far easier to experiment and hack around with than IE, so this is where I do most of my prototyping. The firefox driver is composed of two pieces: a Java layer, which contains as few smarts as possible and delegates as quickly as possible to a Javascript Firefox extension (an XPI)

Being a TDD nut, I started by writing the tests first, gradually making each of them more complex. The goal was to finally be able to pass in a WebElement, as shown in the last test. Having done that, the next step was to modify the signature of the JavascriptExector's "executeScript" method so that the code could compile. This in turn forced me to update the FirefoxDriver itself.

It was pretty late at night when I wrote this first version of the code so there's some dumb things in there, but the gist of the newly modified method is to loop through each of the arguments, converting them into a simple DTO containing both the type and the value. The reasoning behind adding the type, even though JS isn't a strongly typed language, was to allow us to be able to communicate over HTTP through intelligent proxies, such as the RemoteWebDriver (another interesting checkin) In addition, a WebElement for Firefox is represented by an opaque integer which holds the ID of the element within the Firefox extension. Because the ID is a number, it would be indistinguishable from an actual number argument without some sort of typing information.

So, we've converted the arguments into a form capable of being transmitted across the wire. What happens when we land in the Firefox extension itself? Actually, this is pretty easy: we just iterate over the arguments that have been passed in and add them to a "parameters" array. The only thing we need to be smart about is extracting elements where necessary, because the JSON encoding has already Done The Right Thing already. Hurrah!

One interesting thing that does stand out in the firefox driver code is that on versions of Firefox after 2.0.0.14 the security model changed sufficiently that just using "eval" doesn't do what we want to do. As a result, we borrowed a leaf out of the GreaseMonkey book (the "injectScripts" method), and create a sandbox that can be used for executing the Javascript. It came as a bit of surprise when the security change happened, and was something that I missed completely, because my workstation had Firefox 2.0.0.14 on it. As a result, I now regularly run the firefox tests against several versions of Firefox 2 and 3 in an effort to avoid running into the same problem again.

It's taken me a little googling to find this out, but when a "git rebase" halts because of a merge conflict, the way to deal with it is by manually editing the file, correcting the merge and then doing either a "git add" or "git rm" of the affected file, followed by "git rebase --continue"

It's all so obvious now.

A ‘fat finger error’ occurred at the ASX on Wednesday. A broker submitted sell orders for QBE within a range of 0.1-0.2c per share, when the prevailing market price was in fact $22.85. This was an error of several orders of magnitude. Needless to say, day traders and other market participants jumped on the opportunity immediately. Fortunately the ASX quickly recognised the error, suspended trading in the shares and cancelled the original and all subsequent trades.

As I am currently reading Michael Simmon’s Securities Operations, which whilst primarily focusing on post-trade reconciliation and settlement, has been reinforcing to me the need for validation at all steps of the process. I am also starting to work in the area of trade capture at the moment. So I’m wondering why more trade ticket entry screens don’t perform simple validations to prevent errors of this sort, given that this does seem to happen somewhat regularly. I may be wrong but even in a crashing market would you really want an order to sell at less than 1% of the current market price without your screen quickly asking you to confirm? If this QBE trade was issued as an agency order at the broker’s discretion it also raises issues of best execution standards; it would be quite an embarrassing conversation to have with your client about the quality of your service and why they should continue to keep it.

I know I have some friends and former colleagues now working at trading systems vendors, if they are reading perhaps they can shed some light on the issue?

• amCharts: flash charts, stock charting software
  
  VERY nice charting lib
• Open Flash Chart - Home
  
  free charting lib
• Semantic Python Scripting
  
  Just what the title says.
• Scripting for the Semantic Web
  
  Workshop homepage.
• Web Standards and Search Engine Optimization (SEO) -- Does Google care about the quality of your markup?
  
  Interesting post about SEO and web standards.
• BARCODE PLANTAGE
  
  Cool
• The American Scholar - The Disadvantages of an Elite Education - By William Deresiewicz
  
  fantastic essay on education in general and specifically elite education.
• College Sports on TV
• Resources
  
  Great collection of CFB resources.
• 2008 College Football Schedule
  
  Interesting schedule creator.

I am pleased to announce that Dylan Stamat has just released Ehcache for JRuby.JRuby on Rails.

Quick Start

Following is a quickstart reproduced from his blog. HIs company is migrating from Ruby to JRuby and wanted to try in-process caching.

== INSTALL:

jruby -S gem install ehcache

== BASIC USAGE:

manager = CacheManager.new cache = manager.cache cache.put("key", "value", {:ttl => 120}) cache.get("key") manager.shutdown

== RAILS:

An EhcacheStore is available for use within Rails, so all the native Rails caching methods are supported. Make sure your config/environments/*, are setup to support caching, eg: config.actioncontroller.performcaching = true

1) From your RAILSROOT, run this command: - ehcache rails ## just copies ehcachestore.rb into lib/ at the moment

2) In your environment.rb, specify: - config.cachestore = :ehcachestore

3) Cache stuff

Q and A - Greg Luck and Dylan Stamat

Q: What are the reasons you are migrating from Ruby MRI (C Ruby) to JRuby?

A: There were quite a few reasons we decided to adopt JRuby, and probably like most adopters, the deployment and applications options that became available were extremely appealing. Our current stack is deployed on MRI, and runs pretty heavy. In some crude testing of our application deployed to Glassfish, we had a significantly smaller footprint... and an easier to maintain one at that.

Q: Ok, so why the move from memcached to ehcache? You can still use memcached from JRuby and that is the default cache most people use.

A: In the MRI world, we use memcached, which is a Ruby interface to the libmemcached C client. It is one of the only Ruby libraries that currently supports consistent hashing, which is a requirement for our production environment. When preparing for JRuby, we had a different set of options. One was writing our own hooks into libmemcached via JNA, JRuby FFI, etc. The other was to look for another cache solution in Java, and Ehcache was definitely on top of the list. We happen to have a very read heavy application, and are not bound to using a distributed cache... so Ehcache seemed like a very logical fit.

Q: How is JRuby packaged and what plans are there to keep the JRuby interface in sync with ehcache?

A: Anyone running JRuby, and needing a caching solution, should look at the Ehcache JRuby gem. It's a very simple interface to what Ehcache has to offer (which is a lot), and I would love to see it become an up-to-date mirror of the Ehcache project at some point. (Greg: Ehcache 1.5.0 got released just as Dylan was finalising the JRuby adapter)

Q; What are the production benefits of JRuby/Ehcache over Ruby MRI/Memcached?

A: I'm running the gem on a very low volume, internal system, and it definitely works like a charm... and am pretty confident that our current application is testing the limits of JRuby / MRI compatibility ;) We currently have two bottlenecks on our JRuby initiative, one is getting our entire test suite JRuby compatible, which is proving to be a bit more involved than we thought. The other one was tracking down some intermittent marshalling issues we're running into when using the Ehcache JRuby gem which we have just resolved today (Greg: 28 July 2008).

I'll definitely try and get some sign off on using it (the intended production web sites) in the future so you can add them to your "who uses Ehcache" list in the future. (Greg: Suffice to say that the JRuby/Ehcache/Glassfish solution is intended to run some large public facing websites).

Project Details

Visit the project:

Code: http://github.com/dylanz/ehcache/tree/master Google Group: http://groups.google.com/group/ehcache-jruby Bug Tracker: http://dylanz.lighthouseapp.com/projects/14518-ehcache-jruby/overview

And, a ticket system for the project here: http://dylanz.lighthouseapp.com/projects/14518-ehcache-jruby/overview Will definitely get a ticket in there for updating it to use ehcache-1.5.0 ;)

• Build Your Own Speakers – Home Speaker Parts - Popular Mechanics
• What is your RDF browser’s Accept header?
• Tabulator does N3
• Tabulator: Generic data browser
• The Linking Open Data dataset cloud
  
  Wow, tons of dataset links
• Proceedings of Extreme Markup Languages®
• Pubs in Southampton
  
  Another, more festive dataset.
• tommorris's rena at master — GitHub
  
  Rena RDF Ruby library
• OpenLink RDF Browser
  
  Interesting. Not sure what data it's browsing
• Information Arbitrage: Monitor110: A Post Mortem
  
  Informative post mortem

For those of you who do not know, the ehcache project has released a SOAP based server. It comes as a WAR which can be deployed to any web container. It also comes as a standalone server which uses Glassfish V3 embedded and can be started up with a simple script.

Why am I doing this? There are lots of theories that have made their way on to the ehcache mailing list. The prosaic truth is that a large US corporate using ehcache for their Java apps on 200+ servers also wants to use it for their C++ apps. And they are prepared to sponsor development. The Web Services API lets them do it. That's it.

As to the larger question of how interesting this is to the world at large, my view is not very. However having to jump through all the hoops to get a server infrastructure done, I thought that the world at large may be interested in a RESTful, resource oriented ehcache server.

So, I am implementing the RESTful piece now. To me, this is pure exploration. I am on the journey and interested to see where it leads. I am using Jersey for this. Jersey and JSR311 are yet to be finalised. I have been logging bugs and interacting with the team as I go. They are releasing monthly, so I can release with a version of Jersey with all my bugs fixed.

I have a copy of Leonard Richardson's and Sam Ruby's book open in front of me. My aim to ensure that the ehcache server truly follows the resource oriented ideas in this book. Also that it honours ALL of the vaguaries of HTTP/1.1.

I expect to release the first complete RESTful implementation this weekend.

The conclusions I have right now:

1. Any language can use ehcache server. This includes C, C++, C#, VB, Perl, Python, Ruby, Scala, D ...

2. There is no need to have a client per language. Because we just use HTTP, and almost all languages can do HTTP, we're done.

3. Glassfish V3 uses NIO. The server performance should be awesome. Having said that, it will, as is memcached, two orders of magnitude slower than in-process caching, which uses references or pointers rather than network operations.

4. Ehcache server adds a few tricks due to HTTP. HEAD requests, conditional gets and expires headers. What's faster than hitting a cache? Not needing to hit the cache at all. I am testing with curl, wget, various Java libs and browsers to make sure that this is done right. The combination of all these could massively outperform implementations like memcached.

5. Ehcache servers can be placed behind industry standard HTTP load balancers. You can have 2 or more of these in an ehcache cluster behind a single URL, offering load balancing and redundancy. So, if a single ehcache server dies, no problem.

The primary interest I have is providing a caching service which can act as a component in a RESTful resource oriented architecture. How far we take this will depend on who uses it and what they want.

Check out this creative video from The Mighty Underdogs. Even if you don't like hip hop that much, you have to admit that the video is creative and a welcome break from the flow of usualy bling-bling crappy rap videos out there.

So the book review I did for Distributed Systems Online finally made its way online over the weekend. It's a book review of Parallel Metaheuristics. It's a pretty, let's say, esoteric topic, but I had fun reading the book. I learned a lot. You can check it out here:

Parallel Metaheuristics

Feels good to finally have something published again. My last published work was my Dr. Dobbs Article about DSL Workbenches and that was over two years ago.

As always, tell me if I suck.

• ADD JQUERY INTELLISENSE TO YOUR VISUAL STUDIO | Dev102.com

We went to see Wall-e last week, the tale of a humble garbage-collecting robot that finds love and a bigger purpose. It’s very good: good story, great jokes and animation, and the most amazing CG-rendered images—with some minor subliminal promotion for Apple. I especially recommend seeing it if you’re near a cinema with digital projection.

So, am I only the person who is disoriented that a film against consumerist waste and dumbed-down resort culture was produced by the Disney corporation? And that I’m watching it surrounded by empty popcorn buckets?

This sounds like what the Situationist International called “Recuperation”

To survive, the spectacle must have social control. It can recuperate a potentially threatening situation by shifting ground, creating dazzling alternatives–or by embracing the threat, making it safe and then selling it back to us. — Larry Law, The Spectacle—The Skeleton Keys

Still worth seeing, though

For the lastest release of FEST-Swing, we replaced the Maven Surefire reports with ReportNG. ReportNG is a reporting plug-in for TestNG.

Surefire is not bad at all, but it doesn't take advantage of some of TestNG's nice features. For example, it doesn't show the method arguments passed to parameterized test methods, or how tests are organized (in suites and groups) either. It seems to me that Surefire only supports features that are common to both JUnit and TestNG, which are, IMHO, pretty basic: successful tests, failures, errors and execution times.)

On the other hand, ReportNG was designed for TestNG. The reports are not only good-looking, they also provide useful information like:

• Method arguments (which I found extremely useful)
• Information about test categorization (suites and groups)
• Java version and operating system (this is particularly useful for publishing FEST-Swing tests results for Windows, Linux and Mac OS)

Setting up ReportNG was effortless. It took me less than 5 minutes to get up and running! On top of that, documentation is clear and easy to understand!

You can see FEST-Swing's test results (using ReportNG) here.

Many thanks to Dan Dyer for this cool and useful project

If you’re like me, you value traveling light. If you’re like me, anything that can live in digital form does. I haven’t seen most of the actual DVD’s I own for quite some time now. But the movies on them all live on a hard drive that’s hooked up to my television. I love the instant gratification. I love that I can download a movie to my laptop and watch it on the plane.

Delicious Library makes this even cooler. As much as I love having everything digital, I miss being able to take a quick glance at 100 covers and recognize the one I want. I miss the association between images and memories. Delicious gives this back to me.

However, it’d be really cool if I could go browsing in my delicious “library”, find the movie I want, and double click it to play. Turns out you can easily extend the program with applescript, and a couple hours after I decided such a script didn’t already exist on the interweb, it now does.

This applescript will open vlc with the file associated to a media object in delicious. If there isn’t one already, it will prompt you for this file and then remember it for next time. It’s really simple, and happy :)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

-- this will play the currently selected file in vlc using either 
-- the associated url or prompting you to find the file on disk

tell first document of application "Delicious Library 2"
    set selectedMedia to selected media
    repeat with selectedMedium in selectedMedia
        set movieUrl to associated URL of selectedMedium
        
        if movieUrl starts with "file://" then
            set movieFile to rich text 8 through end of movieUrl
        else
            set movieFile to (choose file with prompt "Find the file for " \
                                            & name of selectedMedium & ":")
            
            set associated URL of selectedMedium to "file://" & movieFile
        end if
        
        tell application "VLC"
            activate
            open movieFile
            fullscreen
        end tell
    end repeat
end tell

- thanks Wes for pointing me to such a cool program!

Steve and I have been working on a book for the last few months entitled Growing Object-Oriented Software, Guided by Tests. We're now at the stage where we can put content online to garner feedback. We'll post a chapter every week or so. Eventually, the book will appear as a Rough Cut on Safari.

If you'd like to read and comment, please join the Yahoo group we've set up for discussion about the content. We'd love to hear what you think.

Here's the current table of contents:

Two of my favourite conferences are looking for submissions.

	XP Day is looking for experience reports, tutorials and structured workshops to run in a track alongside the more freeform lightning talks and open space that will make up the rest of the conference. This year the focus of the conference will advancing the state of the art, rather than introducing Agile or Scrum or XP or TDD.
	SPA 2009 is looking for session proposals which are interactive and leading edge, possibly even experimental. They can be about technology or teams, practice or process - in fact anything to do with improving software development.

I think both conferences are excellent places for first-time presenters to run a session. New presenters will get a lot of help from experienced presenters as they go through the shepherding process. And this year, SPA is offering a free place to the best proposal from a new presenter.

We are proud to announce the release of FEST-Swing 1.0-Beta1.

FEST-Swing is a Java library that provides a fluent interface for functional Swing GUI testing. This library provides an easy-to-use API that makes creation and maintenance of GUI tests easy.

Changelog:

1. Added support for testing Java applets. Fixes issue 11.
2. Added support for killing running GUI tests using 'CTRL ALT A'. The key combination can be configured. Fixes issue 174. Many thanks to Simeon Fitch.
3. Added common-use component matchers. Fixes 158. Many thanks to vnguyen.sqa.
4. Fixed issue 14: Test 'shouldApproveFileSelection' in JFileChooserFixtureTest fails on Mac OS X.
5. Fixed issue 146: JPopupMenuFixture#menuItemWithPath. Many thanks to Juhos Csaba-Zsolt.
6. Fixed issue 165: JMenuItemFixture.click() not working. Many thanks to Simeon Fitch.
7. Fixed issue 166: [PATCH] Add GenericFixture from Simeon Fitch. Many thanks to Simeon Fitch.
8. Fixed issue 167: JListFixture#cellReader and similar methods should be modified to return the fixture. Many thanks to Juhos Csaba-Zsolt.
9. Fixed issue 173: ContainerFixture.comboBox().select() fails to close comboBox *sometimes*. Many thanks to fred76131.
10. Fixed issue 177: JTableCellFixture#enterValue(String) with JComboBox as editor. Many thanks to Juhos Csaba-Zsolt.
11. Fixed issue 179: JTreeFixture#separator(String) and similar methods should be modified to return the fixture. Many thanks to Juhos Csaba-Zsolt.
12. Moved abstract methods in org.fest.swing.fixture.ComponentFixture to the interfaces FocusableComponentFixture, KeyboardInputSimulationFixture, MouseInputSimulationFixture and StateVerificationFixture.
13. Moved classes KeyPressInfo and MouseClickInfo from package org.fest.swing.fixture to org.fest.swing.core.
14. Replaced method pressAndReleaseKey(int keyCode, int modifiers) with pressAndReleaseKey(int keyCode, int... modifiers) in org.fest.swing.core.RobotFixture.
15. Removed all methods related to keyboard input simulation from JListItemFixture and JTableCellFixture.
16. Replaced IllegalArgumentException with NullPointerException when a method argument has null as its value.
17. Replaced ActionFailedException and LocationUnavailableException with IndexOutOfBoundsException when an index is out of bounds (e.g. the row index in a JTable.)

FEST-Swing can be downloaded here (file fest-swing-1.0b1.zip.) FEST requires Java SE 5.0 or later.

Here are some useful links:

• Documentation
• Change report
• Javadocs
• Getting Started
• Issues
• Users' Group

Feedback is always appreciated

This is surprisingly well done for an internet video:

SPA 2009 is now open for session proposals.

I'm honored to be programme co-chair (with Mike Hill) for SPA 2009, having attended this great conference for many years. Although there are several conferences that I enjoy - SPA is unique thanks to being residential and extremely interactive. The high proportion of attendees who are presenting, and the level of interactivity of the sessions, means that everyone is involved in the sessions - there aren't any powerpoint "sit and listen" sort of sessions. There's also a lot of interesting discussions outside of the sessions and in the evenings.

Please consider submitting a session proposal particularly if you've never presented before - it's a great conference to be a first time presenter at because of the shepherding and the interactivity of the sessions - good sessions don't have to start with all the answers - they are often ones that start with a good question. This year we are offering a free place to the best proposal from a new presenter. Of course, I'm also hoping for lots of proposals from experienced presenters too!

The submission deadline is Monday 15th September 2008.

Who would have thought I’d be doing php?

Anyway, yesterday I found myself doing php and really wanting it to hurt less. I downloaded a textmate bundle for phpunit from github. It wasn’t quite what I wanted, so I forked it and hacked it, and made it work a bit more like the rspec bundle.

It now has:

• run tests (in current file)
• run single test

• test

And much to my surprise, it was way easy to do. All in ruby, all with specs testing it.

This article was going to be telling all y’all to come over to my cool fork of phpunit, but I sent the original author a pull request yesterday, and I just looked, and he’s pulled all my changes. Rock!

So go forth and enjoy the goodness that an extensible editor like textmate, oss code like the phpunit bundle, and github can bring about.

PHPUnit.bundle for textmate

Entities

The Universe Web needs Entities. (I'm going to be capitalising Significant Things a lot here to help the Web comparison - hope you don't mind). There's no need to make the Entity list too complicated since we're not doing online games yet - just a world like Second Life.

Here's a list of different kinds of Entity:

• Places: buildings, streets, hills, lakes
• Things: trees, books, birds, clocks
• People

State, Links and animation

Entities aren't exactly the Objects of Object-Oriented programming: they have lots of public State. In a Virtual World, you can directly experience their colour, position, orientation, physical relation to each other, etc., thanks to the local render engine.

The basic structure of the Universe Web is made up of Linked Entities. A tree is Linked to its ground which is Linked to a building which is Linked to its occupants which are Linked to their clothing. Kneebones may even be Linked to thighbones.

Entities are often animated. Actually, Places are generally static, and People generally animate themselves. So, when talking in general, we should look at the animation of Things, like trees, books, birds and clocks.

There are two ways an Entity's animation can be guided: by Laws and by Rules.

Laws

Springiness is a handy Law. For example, tree branches, pages in a book and birds' wings and legs can all have springiness. We can declare a mid-air location in a bird's State and a springiness in its legs, then if the bird lands on a branch, the springiness can trigger the bird and branch waving gently to a halt.

We could even allow a single bird to become part of a flock Entity, where the flock only knows how many birds it has and the coordinates of its centre. Each bird will then be animated by global flock Laws.

The book can have a State of open or closed. If bookness was elevated to a global Law, then a book deciding that its State was now simply closed could trigger the elaborate, springy closing of its pages with a nice 'shlump' sound at the end.

The clock's current time may be set by the clock Entity itself, then the Law of time take over in between the clock updating its own time directly.

A Law can be invoked over an Entity's animation by simply adding the Law's name and parameters to its State. Such Laws can be handled by the physics engine in the same way surface texture is handled by the render engine: States such as 'red' need global meaning, Law applications such as 'springy' need global meaning.

Rules

When a clock reaches its alarm time, it rings. When its wizard master approaches, a magic book may glow and open. Birds fly away from danger (if a fox is too close, fly away from it). In between danger, birds and foxes look for food (if a bird is close, run towards it).

These are Rules. Rules are driven by the Observation of one Entity by another. The fox Observes the bird, the book Observes the wizard. Rules apply to Entity types: fox Rules, bird Rules, book Rules, etc. Rules can guide the animation of an Entity by evolving its State.

State evolution depends on the current State plus the States of the Observed Entities: Rules make an Entity change from one State to the next by taking the Entity's own State, Observing the States of the Entities around that it cares about, then deciding what the next, new State should be. If an Entity changes State, those Entities Observing it may themselves change State according to their own Rules, and so-on.

Links are part of an Entity's State, and Links may therefore change on the application of Rules. In other words, the virtual world may restructure when a Rule is run.

An Entity may either Observe another Entity, or Link to it, or both. An Entity that Links to another needn't Observe it. Trees don't bother to Observe the ground they grow from. Conversely, an Entity that Observes another needn't Link to it. A fox would only Link to a bird if she was lucky enough to catch it. Meanwhile, it is true that there's probably a chain of Links to everything that's Observed.

Unlike Laws, Rules can't or shouldn't be second-guessed, so have to be run 'at source' or 'inside the Entity', then the new State distributed to both the Observing Entities and to all the render engines.

But render engines are actually Observing People Entities..

People are First Class

In Virtual Worlds, we have People avatars: People are first class Entities. People can Observe and can change State, but it's a real user doing the animation, not a set of Rules. Observation and State change by a Person is implemented through the render engine.

Although self-animated, People can still make use of Laws. For example Laws can be invoked to allow gestures to be easier to make and to look better, and to allow People to 'run forward' rather than putting their 'left foot forward, ..'. Avatar skills become global Laws which can be triggered by an avatar's State animated by a user.

The Universe Wide Web

The Universe Web I've just described is a lot like the World Wide Web:

• An Entity is like a Web Resource
• The State of an Entity is like an HTML Web Representation of a Web Resource
• A Link in an Entity's State is like a Web URL inside an HTML Web Representation
• Laws are like Javascript, Flash, link hover, blink tags and animated GIFs
• Rule sets are like PHP scripts or Java servers
• A State Observation is like a Web Representation Transfer

A crucial difference between the Resource Web and the Entity Web is that, here it's another Entity doing the Observation. One Entity transfers its State to another.

It's as if the GET or POST has another Entity 'at the client end'. GET is like an Entity subscribing or Observing; POST is like an Entity publishing or notifying its State to another (an HTML form has a content type, too).

People are just examples of that Entity 'at the client end'. Instead of a user hidden behind a client browser, we have first-class People Entities doing 'GET's and 'POST's: Observing the Virtual World and acting (i.e., changing State) within it. Other examples are foxes and clocks.

Notation and Protocol

There are two open standards we need to have in progress while implementing the Universe Web: the notation and the protocol (our HTML/URI and HTTP).

Firstly, we have to define the notation for Entities and their Rules. Both the State of Entities and the Rules that animate them will be written in the same notation: a Rule talks directly about State. To me, basing that on JSON (rather than XML) is a clear choice. Add inter-JSON Entity Links (essentially UUIDs), then work out a simple JSON Rule language. We will then need to define JSON 'schemas' for the kinds of Entity we would model. These can be written in the Rule language itself.

Secondly, to define the protocol for Entities to Observe or subscribe to one another. It will be a 'symmetric, asynchronous HTTP' - a Peer-to-Peer, Publish-Subscribe protocol. UDP and Multicast stand out for this kind of interaction. A packet will subscribe, a packet will return current State, a subsequent packet will multicast State updates. We can re-use all the HTTP cache headers, only now we can push updates into caches and into render programs. Packets will inevitably end up tunnelled through TCP and HTTP, mind...

This lighter notation, which also breaks the data up into smaller pieces, and this multicast pushing of State change into caches, along with P2P-style operation, will mean the Universe Web scales even better than its parent.

Also, programming declaratively using Rules is both highly productive and further cleanly scalable through parallel processing.

Of course, it will be pretty trivial to allow inter-Web cross-over such as referring to PNG texture images via URL and pulling Universe Web Entities into a Javascript application that subscribes to them via Comet.

Building the Universe Web

So that's my Universe Web. It's basically the same as the World Wide Web and thus as scalable, linkable and interoperable. It comes with added symmetry: Entities, including People as avatars, exchange State in each direction.

It's a two-way data Web, not a one-way document Web.

I've started work on the Universe Web: the notation, the protocol and the implementation (in C, for portability across platforms, including mobile). Contact me if you want to help!