My current project integrates with the Microsoft Dynamics CRM system for managing customer records. We already spiked out authenticating and our next step was attempting to update a record via its “REST Endpoint“.

On retrieving a record, you follow the OData style for finding something of relevant. Updating a record is interesting as we only want to send the fields that need to change remotely. Looking at their sample code, you need to:

• Create an authenticated POST request
• Set the X-HTTP-Method to MERGE
• Set the appropriate content type for the content you intend on sending. In our case, set the header Accept to application/json and the Content-Type to application/json; charset=utf-8

Our end point for a contact looked something like:

https://crm.thekua.com/xrmservices/2011/organizationdata.svc/ContactSet(guid'867dc3f2-909e-e111-9912-0050569c2d72')

.

Updating simple fields off a contact is easy. We post something like:

{
  "EMailAddress1":"spike.jones4@gmail.com",
  "MobilePhone":"33335"
}

We receive a HTTP 204 (No Content) on a success. Posting an invalid attribute (i.e. one that does not exist on the ContactSet such as “EMailAddreXXXs1″) results in a HTTP 400 (Bad request) and a nice description about what’s wrong. You will also get a HTTP 400 if you post the wrong datatype such as sending a string where they expect a number. If you pass invalid values (but of the correct datatype) for a field, your response is a HTTP 500 with a message like The value of 'gendercode' on record of type 'contact' is outside the valid range."

Updating Option Set Values

Updating simple datatypes is easy and obvious from a JSON point of view. You have a couple of complex datatypes, such as the standard GenderCode. When you query for a record, you get back something that looks like

{"GenderCode"=>{"__metadata"=>{"type"=>"Microsoft.Crm.Sdk.Data.Services.OptionSetValue"}, "Value"=>1}}

To update something like this, you need to send a nested JSON object. The example follows

{
  "EMailAddress1":"spike.jones4@gmail.com",
  "MobilePhone":"33335",
  "GenderCode": {
    "Value" : "1"
  }
}

A HTTP 204 (No Content) response indicates a success. Viola!

In the comments on my post about generating random numbers to test a function David Turner suggested that this was exactly the use case for which QuickCheck was intended for so I’ve been learning a bit more about that this week.

I started with a simple property to check that the brute force (bf) and divide and conquer (dc) versions of the algorithm returned the same result, assuming that there were enough values in the list to have a closest pair:

prop_closest_pairs xs = length xs >= 2 ==> dcClosest xs == (fromJust $ bfClosest xs)

I could then run that as follows:

> import QuickCheck.Test
> quickCheck(prop_closest_pairs :: [(Double, Double)] -> Property)

It failed pretty quickly because the bf and dc versions of the algorithm sometimes return the pairs in a different order.

e.g. bf may say the closest pair is (2.0, 0.0), (2.1, 1.1) while dc says it’s (2.1, 1.1), (2.0, 0.0) which will lead to the quick check property failing because those values aren’t equal:

> ((2.0, 0.0), (2.1, 1.1))  == ((2.1, 1.1), (2.0, 0.0))
False

The best way I could think of to get around this problem was to create a type to represent a pair of points and then write a custom equality operator.

I initially ended up with the following:

type Point a = (a, a)
data Pair a = P (Point a) (Point a)

instance Eq (Pair a) where
	P a b == P c d = a == c && b == d || a == d && b == c

Which didn’t actually compile:

qc_test.hs:41:58:
    No instance for (Eq a)
      arising from a use of `=='
    In the second argument of `(&&)', namely `b == c'
    In the second argument of `(||)', namely `a == d && b == c'
    In the expression: a == c && b == d || a == d && b == c

The problem is that while we’ve made Pair an instance of the Equality type class there’s no guarantee that the value contained inside it is an instance of the Equality type class which means we might not be able to compare its values.

We need to add a class constraint to make sure that the value inside P is a part of Eq:

instance (Eq a) => Eq (Pair a) where
	P a b == P c d = a == c && b == d || a == d && b == c

Now we’re saying that we want to make Pair an instance of the Equality type class but only when the value that Pair contains is already an instance of the Equality type class.

In this case we’re just storing pairs of doubles inside the Pair so it will work fine.

Now if we compare the two points from above we’ll see that they’re equal:

> P (2.0, 0.0) (2.1, 1.1)  == P (2.1, 1.1) (2.0, 0.0)
True

I had to go and change the existing code to make use of this new but it didn’t take more than 5-10 minutes to do that.

Today, I feel like a massive RANT.

Although I’d rather book myself in for a root canal than get involved in creating proposals or responding to RFPs, I do find myself in this position frequently these days and I’ve yet to develop a suitably sustainable perspective on these things to make them enjoyable.  Hell, forget enjoyable as a goal – I’d settle for a beige sort of feeling rather than the darker colours I tend to find myself wallowing in during those times.

And here’s why – proposals suck.  The act of building them… sucks.  The process they are a part of… sucks.

Why?  Well, let me give my completely objective and not-at-all bitter take on how this stuff works.

Step 1: Company A wants some stuff done.  They decide to go to market for whatever reason.  Their governance mechanisms demand some form of RFP process to ensure fairness and transparency, etc, etc (hint: it doesn’t and it isn’t, but that’s another story entirely)

Step 2: Company A taps the shoulder of some poor schmuck(s) to write the documentation describing the work to be done.  This document is usually 40% boilerplate with the rest being the gristle of the problem to be solved.  Generally the authors of these documents are domain experts so understand the problem domain quite well.  However using domain experts to explain complex problems to people who don’t share that expertise also sucks because:

• They have sublimated all the obvious questions naive people would initially ask and present the information at a level best consumed by other domain experts
• They are forced to filter their knowledge through the ultimate comprehension mangler that is… written sequential documentation
• They are often not naturally good technical writers, so I’ll leave you to your own conclusions about what impact this has
• No one ever thinks to “test” the quality of the documentation against typical end users

Step 3: Documentation is distributed to interested parties for response by a certain time.  Some RFP processes include open Q&A sessions with all the potential respondees which inevitably turn into variations of pissing contents (i.e., who can ask the questions that intimidates the other parties in terms of specificity, detail, prior knowledge, etc) or exercises in studied banality (i.e., don’t ask anything publicly which might reveal how we’re thinking about the solution).

Step 4: Respondees digest documentation and subsequently devise solution, estimates, timelines, costs, staffing models, risks, etc., etc.  Now this is where things get really interesting because I’m often part of the  ”devise solution plus estimates”  side of things and a whole new circle of Hell must be navigated to survive this phase.  The natural tensions are:

• You know that at least two stakeholders have unrealistic expectations about the accuracy of any estimates produced; Company A and your own “pursuit team”.  Often people talk about +-30% as though this is a wide enough variance at this point in the project.  All empirical studies I’m aware of around estimation accuracy suggest otherwise.  How many times have I come across a line item in the RFP document that could range anywhere between a NOOP and a 6-month piece of work in itself depending on semantics or unstated context?  How many times do literally dozens of similar statements occur in these documents.
• By definition, the accuracy of the estimate can never exceed the accuracy of the statement of the requirements.  See my previous comments about the likelihood of these requirements being accurate and/or unambiguous.
• There are three main paths you can agree to take to resolve these tensions, although none of them satisfy all the groups involved.  You can Optimise for Initial Success (i.e., winning the work) which seems to involve making the quoted numbers low and tightly bound to give the impression of thrift and confidence through an illusion of accuracy.  This process is made considerably easier if you know what the budget for the work is.  You can Optimise to Inspire Confidence by just giving an unreasonably small range for your quotes, implying we have sufficient control over all the variables required to predict with this level of accuracy (hint: we don’t).  Alternatively, you can Optimise for Prolonged Trust (between you and Company A) by responding with the numbers that initially estimation gives you and with a suitably large variance to reflect the reality of project physics.  IMO, this last type of response doesn’t lead to a lot of success (this also sucks).

Step 5: Respondees send response to Company A who now have the unfortunate task of reading, understanding, comparing and ranking n proposals.  Sometimes there is a shortlist process and you get a second bite at the cherry and refine your response.  Rarely is there any formal feedback process from Company A back to the respondees apart from the tersest of communications saying you have been successful or otherwise.

If you’re unsuccessful, I believe the required behaviour is to either justify the response by saying you didn’t want the work in the first place or rationalise the decision by saying that whoever won did so by lowballing the price with a strategy to change manage their way to profitability.

If you’re successful, then the once the project gets started, the vast majority of the assumptions you made to justify your estimates will be immediately or eventually invalidated rendering the basis of your estimates and costings null and void… this part doesn’t suck so much

Next proposal – BRING IT ON!

Last week I completely re-worked the short presentation I share with nonprofit clients who want to get others in their teams excited about what agile practices can do for them.

If you’re a staff member or volunteer on a team, this stuff can be so very helpful. Check out the presentation: Agile – What is it and why should I care?. And feel free to ask questions. Or steal it!

One of the big problems with successfully executing projects is that while we know projects are very different from each other, we often manage them and measure their success in the same way. Think for a moment about the oft quoted Standish reports in which project success is measured on the traditional iron triangle basis of meeting scope, schedule, and cost plans. In their scheme, all projects, of any type, are successful or not based on the same criteria.

A friend of mine worked on a project recently where the client said, “I have a fuzzy vision of what we want. I don’t have any idea what the detail requirements should be. I need results fast.” The client even refused to participate in a story identification process, leaving that up to the development team to experiment. Managing this project against traditional measures of scope, schedule, and cost would damage any chance of success. This is a different kind of project. The team evolved the product from a vision and evolutionary learning and the client was very pleased with the results, because he understood it was a different type of project.

In Agile Project Management I wrote about assigning an Exploration Factor (EF) to each project (or release of a product). The EF attempts to identify a level of uncertainty and risk for a project by looking at both technology and requirements. Technology can run the gamut from “well-known” to “bleeding edge” and requirements from “stable” to “erratic.”  Combining the two factors yields EFs (see figure) of from 1 (very low uncertainty) to 10 (very high uncertainty and risk). EF 1 and EF 10 projects are very different—they need to be managed differently and they need to be assessed differently. For example, if the requirements are uncertain, measuring progress against a scope plan is ridiculous. This doesn’t mean the project is uncontrollable, just that we have to control it differently.

So, before we think about how to measure success on projects with high EFs, we need to understand what makes these projects successful. The pressures on high EF projects are usually uncertainty (about either requirements, technology, or both) and speed. Typically high EF projects are also strategic, customer facing, web or mobile, etc.—all aspects that lead to uncertainty. Sponsors want them done quickly. So how are projects like this managed successfully? By using a combination of a well-articulated vision, quick iterations, evolving functionality to a minimal viable product and beyond, being adaptable to learning from customers as the product evolves, focusing on value delivery, and time boxing.

One important aspect of controlling high EF projects is to view scope, schedule, and cost as constraints—not plans. Because the project requirements are not known it is often difficult to estimate time. However, given the lack of specificity the team needs constraints, for example a time box of 3 months that limits exposure. At the end of this timeframe the project sponsor can evaluate the value delivered, the questions answered, and then decide to invest in the next increment or not. The question to be asked at every iteration and release is, “What is keeping us from deploying this product now?” This is very different from the normal progress question of “Have we developed all the planned scope yet?”

Conversely, a very low EF project, say a 1 or 2, could be managed with scope questions because the requirements are knowable in the beginning (or at least people think they are even though we know they are often wrong). I might add that software development projects are rarely low EF projects.

The bottom line is:

• All projects are not the same.
• Projects with different Exploration Factors should be managed differently.
• Performance measures should be different for high EF projects than for lower ones.

TLDR Version: User Stories Applied is an excellent resource for anyone who wants to gain a better understanding of the ideas behind user stories. The book has very good examples and guidelines on writing better user stories and integrating them into your development process.

User Stories Applied is a book that has often been mentioned to me in the six months I’ve been at ThoughtWorks and over the last two weeks I finally managed to get my hands on a copy. It’s not a very long book (230 pages) and it took me about 3 plane trips to finish it.

The Good

This book provides a very good outline of how user stories work and the intent behind them. Even though I work with user stories every day I found some of the details quite refreshing – especially how user stories are not meant to contain the details behind features, but rather to be a medium for us to start discussions around features.

I particularly liked the discussion on how to write good user stories and the guidance on when stories are too small or too large. This is an area where I think experience really helps, but the pointers in this book are a great starting point.

The Bad

Every chapter in the book ends with a summary and an outline on how the chapter influences the different roles on a development team. I’m not a big fan of this kind of writing – it reminds me of the textbooks I had at school and it doesn’t seem to really add any value.

The second half of the book seems to focus on comparing user stories to other approaches. I felt like the author was trying to defend user stories, but the arguments presented were more focused on the shortcomings of the other approaches rather than the advantages of user stories. Having said that, this can be very helpful if you’re trying to convince a client to use this practice on a project.

Interesting

I was rather intrigued by the difference between how the author describes user stories and what I’m used to. The author advocates that user stories should contain very little detail and the details should be discussed when the story is developed. I’m used to user stories containing most of the detail around a story – in fact stories will usually be pushed back if the details are left out.

I think using user stories as a placeholder for conversations around the details can work really well, but on projects with a large amount of developers (I’m guessing more than 8 developers) the interaction with the user will quickly become a bottleneck. User stories can therefore also work very well when they contain all the necessary details which are analysed and discussed in full before development is started – it’s not a massive difference, but on large teams it can certainly have a massive effect.

User Stories Applied as an excellent resource for anyone who wants to gain a better understanding of the ideas behind user stories. The book has very good examples and guidelines on writing better user stories and integrating them into your development process.

I have come across too many people who are either not willing to blog, or do not see the point. This blog post is for them.

(Recently, Elena Yatzeck posted on this subject as well.)

I recently read a one-liner “I see I scorn, I do I regret, I blog to not forget”. (Usually I am turned off by one liners intending to pack everything including the reason for life in minimum words, but this one I like because it is funny enough.)

This above quote is enough motivation to keep me going on my blogging. In the day and age of #fb and #RT it is difficult to get people excited about the value of details and being able to express in elaboration. Blogs come to the rescue.

I experience this loss of details quite a bit. Recently on my M$ platform based assignment I forgot to blog what we did to make our .NET application development painless. Now when I try to recollect some lessons I learnt I have to depend on my memory which is sloppy about details. I blog to ensure that I have less of these situations and that I can use my experience to my advantage – also maybe readers of my blog will benefit as well.

I would like to encourage more blogging so that I can benefit from your knowledge and learnings.

Here are a few reasons you should consider blogging:

1. Twitter is too short to explain anything in detail.
2. Facebook will not understand when you say a lot without being emotional about it.
3. Your long term memory is not that good – also it is not google searchable.
4. All the short messages are easily lost in your tweetdeck. Google search will yield the desired message but will not provide you the context it appeared in.
5. Blogs are on the web and searchable via google.
6. Blogs can be as elaborate as you like – can be tagged, followed, copied and pasted from, tweeted about, and can turn into articles for publication.
7. Blogs can also be collected and turned into books – for free.
8. Self promotion – A good post by Jay Fields
9. Blogs are permanent records(mostly) and they can be used to jog your own memory – or reminisce.

How and where to blog?

1. Wordpress
2. Tumblr
3. blogger – beware of their weird copyright requirements.
4. github – octopress
5. build your own blog engine and push it to heroku

What are you waiting for? Go write something.

Does your team feel safe expressing every idea that’s relevant to your work?

Imagine your team sitting around a table. People either express what they’re thinking (putting their ideas, like cards, on the table) or they don’t (protecting their ideas by hiding them under the table.

When we work in teams, we most likely only talk about what’s on the table – that is, the issues, feelings and ideas that people have expressed and can talk about. It works great, as long as our team feels comfortable enough to get their thoughts out. When people fear the outcome of saying something, though, they start to push their thoughts under the table – this keeps them and their ideas safe from criticism, ridicule, or embarrassment.

The things we keep under the table have just as much bearing on our ability to participate in our team’s work as they stuff we do talk about, though. Imagine a team conversation when…

• One member has been made fun of for always agreeing with the presenter, who’s a lot like them
• One member just had a close relative die, and hasn’t mentioned this to the team
• Another member learned before the conversation that their biggest work project is way behind & customers are angry
• The member presenting has shared ideas before and had them shot down by the group; they’re worried about presenting again

How well is this group really going to be able to pay attention, listen and give opinions openly?

You’ve probably been at this team meeting before. It’s easy to spot – the group goes silent or listless… or, on the other end of the spectrum, gets confrontational with no real purpose. What do you do?

Lead by example, if you can – bring up something you’re keeping under the table that stops you from participating fully. Talk about how it’s holding you back, and ask others if they have similar thoughts to discuss. Ask your team members some powerful questions.

What hopes do you have for the team that you hesitate to express?
What needs to change so you feel safe expressing those hopes?
What issues, ideas or feelings are we avoiding as a group?
What could we gain by talking about these things?
What is holding us back in this conversation?

You can do this formally or informally. You probably know what makes sense for your own team members. What else has worked for your teams when you find yourselves stuck like this?

The circumstances of our lives may matter less than how we see them, says Rory Sutherland. At TEDxAthens, he makes a compelling case for how reframing is the key to happiness.

The stupid white background on the blog, just made me averse to posting or blogging. Now I have gone back to a colour that suits me better. Hopefully this will lead to more posts
Okay, I've watched loads of movies since the last time and I am not in the zone to write about them all. But here are some(in no particular order) that you must try and watch - 

Drive

Ryan Gosling plays a getaway driver for hire. He will drive you away from the scene of the crime for a fee. His character is socially inept and given to long brooding silences. The movie does not pick pace until he falls for his neighbour (who is married with a kid and a husband in jail).When the movie does pick up pace, its still slowish :)
The action becomes very violent in the second half and the very interesting soundtrack almost contradicts what you see on screen. The soundtrack is super-awesome and the fight scenes reminded me of Mel Gibson's "Payback". The direction is brilliant,Its a movie that you will either like or hate. But its one that you must watch.

Avengers

Enough has been written about this. It starts all slow and talkative but once Bruce Banner turns into the Hulk we have a good movie on our hands.
Yes, Tony Stark again gets all the witty one-liners, but its the Hulk who gets the best cheers. Captain America and Hawkeye are mostly, also-rans but the Black Widow gets some decent screen time.And there's Robin! No not the Batman one, the 'How I met your mother' one :DThe good: The Hulk and Tony Stark.The bad: What? all these super heroes assemble for a sissy villain like Loki? Doesn't seem right. Now Red Skull would have definitely made a more menacing villain. That's the next of the series I suppose.

Tinker, Tailor, Soldier, Spy

The real reason I watched this movie was for 'Benedict Cumberbatch'. Who? you ask.
Well, Cumberbatch plays Sherlock Holmes in the super impressive series "Sherlock". I was so impressed that I looked if I had any movie in my collection that had him in it. 'Tinker, Tailor, Soldier, Spy' met the requirements.Gary Oldman plays 'George Smiley' a popular le carré character.A mission goes bad and the presence of a mole in the British intelligence is evident. Smiley's handed the job to ferret out the mole.I loved this movie.

The girl with the dragon tattoo

Watched the English version. It's really good. Rooney Maara is fantastic!  Haven't read the books yet, but I have the next book in the series on my bedside.
I also plan to watch the Swedish version's of the films since something tells me they will be great.Must watch this one.

I was out shooting today, putting the x-pro through its paces. No real purpose – meandering around the Portobello Road, shooting stall keepers and tourists alike. I turn for home, walking alongside the westway, when a cyclist zips past. I see him, heading towards me, bring the camera up, track him and *click*. Enough time for one shot, and it’s perfect. He is captured on the top right, pin sharp, looking straight into the lens. The background is beautifully blurred. Perfect shot.

I walk on, and get a tap on my shoulder. I turn around to see the cyclist.

“Can you delete that, please?”.

And I do.

Walking back, I think idly “I could probably recover that…”. Legally, I’ve done nothing wrong. And it is a great shot – one of the best I think I’ve taken. But I sigh, and know I won’t. At least this is one shot I can’t claim to have lost due to the Fuji x-pro’s AF.

Agile software development is getting more and more attention now a days. One of the variations of it is called ‘Distributed Agile Development’. And a key ingredient for distributed teams to work effectively is “communication”. Quote from Kent Back on communication "Problems with projects can invariably be traced back to somebody not talking to somebody else about something important."

Currently we are doing an inception with a client and their past experience with distributed team has left them with some bitterness. In addition, the client has just started adopting agile practices in their software development cycle, so along with delivery we are also doing enablement for them. Also making sure business and new team member understand the different buzzwords of agile software development is very important. 
In my experience every team has variations in mechanics of implementing agile, keeping all the principles intact. For example, story prioritization for some teams is Iteration Planning Meeting (aka Iteration Kick Off) and  some teams go for  more lean  like method of story prioritization and adopts an iteration-less Kanban style pull mechanism. For  distributed teams, the variations are important to know and everyone should understand the why’s of it. Daily standup need to be little different when working in distributed model, Iteration showcase and iteration length varies too from team to team. In my opinion all such mechanics are decided and derived based different parameters. However, the important part is to discuss, decide and communicate  the mechanics & process of project’s agile development with all team members including business.

And its very important to plan for some session during inception to address need around all of the above. 
Taking “Story Lifecycle” and “Anatomy of Iteration” our team (Sunit Parekh, Birinder Singh, Sarbashrestha Panda & Amit Dhakad) did the following session in little different settings and format. This blog is all about sharing the mechanics of running the session. I am calling it “Communication plan using anatomy of iteration lifecycle” and some of following is for the distributed team structure, where the business, SMEs, Product Owner and Product Sponsor are in UK and the development team is back in India.  Hence few of the mechanics differ from co-located team structure.
Topics to broadly cover during session are,

• What is a story? &  Story lifecycle
• What is an iteration? &  Anatomy of iteration
• Definition of Done (Story)
• Testing approach (automated and manual)

We created a timeline on the board to start the session with first talking about what is story and story lifecycle.

As we kept discussing about the iteration, we started moving story stages onto iteration lifecycle. Also started building the events as shown in the picture below. Session took an hour to run.


Keep most of the stickies written before the session. This helps to remember all points to cover and running the session quickly. Have discussions during the session and make it as interactive as possible. As a team decides most of the stuff (e.g. definition of story done, iteration length, meetings/calls, Skype or Google Hangout) rather than dictating to others, this is very important as that makes it team's plan.

Here are a few things, that is different for a distributed agile team: 

• Standup, can be little different format. We use mingle card wall to just talk about cards updates if there are any queries or updates, achievements of the team during day, along with any blockers and show-stoppers for team. Keep it short. 15 min.
• BA Catchup, business is busy most of the day with their job and so it is important to block SMEs calendar for certain time of the day so that can be utilized for any questions, queries and demo’s for feedback. Daily 1 hr. We do it just followed by distributed team standup.
• Story prioritization, continuos process like Kanban. Regularly and BA Catchup call is used for story prioritization. 
• Iteration Kickoff and Iteration Showcase, we like Kanban style prioritization however it is good to get everyone together and share the project status, updates and showcase key features build since last meet and what’s the plan ahead for team. We do this as one meeting scheduled at the end of iteration as iteration showcase.
• Tech Huddles, during initial stage of the project couple of times a week and later it’s kind of need basis once or twice a week.

Please share your comments, suggestions & experiences about this kind of sessions. What more we should cover, what we should watch out for...

Special thanks to Sarbashrestha Panda for reviewing the post and recommending valuable amendments.

自从迈出了ThoughtWorks校园行的第一步，西电的校园活动就一次次进行下来，我们讲了如何写代码，讲了完整的开发过程，后续的一系列校园技术讲座也已经排上了日程。

其实，单以效果而言，技术讲座所带来的影响是有限的，对于参加讲座的学生来说，除了知道很多新名词、新做法，多半也就是看个热闹。直接动手实践，才是一个更好的做法。

感谢我们优秀的市场MM，她为我们打开了一片新天地。感谢西安交通大学软件学院的领导具备的卓越眼光，为他们的学生提供了一个接触外面世界的机会。

于是，我们有了一个新的机会，在西安交通大学开设一门软件开发的课程。

这门课，我们称之为“现代软件开发”，其目的就是为了告诉同学们，不同于传统方式的开发方法。其实，我们本可以叫敏捷软件开发方法，但我着实不喜欢这个名字，因为这已不是我追求的东西，我只想把一些好的东西告诉同学们，所以，选了一个不容易过时的名字。

我们是这样设计的这门课，采用上下半场的方式运作。上半场，我们会介绍一些基本的开发方法，比如整洁代码，比如重构，比如自动化等等。而下半场，则完全是实践。我们选取了一个项目，按照我们的方式运作了一个项目，我们的同事会与同学们结对，让他们直接体会最原汁原味的ThoughtWorks开发方式。

就在这个周末，这个系列课程终于迈出了第一步。

第一次课基本上算是一个课程介绍，让为有兴趣选修这门课的同学了解这门课，以及我们的上课方式。我们讲了公司里如何做软件，还演示了结对开发和TDD。

第二步就这样迈了出来，在接下来的一段时间里，虽然要牺牲一些业余时间，但这也是一个很好的尝试，让我们的课程更加系统化，也给了我们的同事一个很好的锻炼机会。

我们的同事在课程的结尾送给同学们一句话，与其周末逛街打DOTA，还不如来写写代码。嗯，就是这样。

When I was looking over my solution to the closest pairs algorithm which I wrote last week I realised there there were quite a few if statements, something I haven’t seen in other Haskell code I’ve read.

This is the initial version that I wrote:

dcClosest :: (Ord a, Floating a) => [Point a] -> (Point a, Point a)
dcClosest pairs
  if length pairs <= 3 then = fromJust $ bfClosest pairs    
  else 
    foldl (\closest (p1:p2:_) -> if distance (p1, p2) < distance closest then (p1, p2) else closest) 
          closestPair 
          (windowed 2 pairsWithinMinimumDelta)
  where sortedByX = sortBy compare pairs	      
        (leftByX:rightByX:_) = chunk (length sortedByX `div` 2) sortedByX
        closestPair = if distance closestLeftPair < distance closestRightPair then closestLeftPair else closestRightPair  
          where closestLeftPair =  dcClosest leftByX
                closestRightPair = dcClosest rightByX
        pairsWithinMinimumDelta = sortBy (compare `on` snd) $ filter withinMinimumDelta sortedByX
          where withinMinimumDelta (x, _) = abs (xMidPoint - x) <= distance closestPair   
                  where (xMidPoint, _) = last leftByX

We can remove the first if statement which checks the length of the list and replace it with pattern matching code like so:

dcClosest :: (Ord a, Floating a) => [Point a] -> (Point a, Point a)
dcClosest pairs
  | length pairs <= 3 = fromJust $ bfClosest pairs    
  | otherwise = foldl (\closest (p1:p2:_) -> if distance (p1, p2) < distance closest then (p1, p2) else closest)
                      closestPair 
                      (windowed 2 pairsWithinMinimumDelta)
    ...

We can also get rid of the if statement inside the first argument passed to ‘foldl’ and replace it with a call to ‘minimumBy’:

dcClosest :: (Ord a, Floating a) => [Point a] -> (Point a, Point a)
dcClosest pairs
  | length pairs <= 3 = fromJust $ bfClosest pairs    
  | otherwise = foldl (\closest (p1:p2:_) -> minimumBy (compare `on` distance) [closest, (p1, p2)])
                      closestPair 
                      (windowed 2 pairsWithinMinimumDelta)
    ...

We can do the same to replace the if statement where we work out the closestPair which results in this final version of the code:

dcClosest :: (Ord a, Floating a) => [Point a] -> (Point a, Point a)
dcClosest pairs
  | length pairs <= 3 = fromJust $ bfClosest pairs    
  | otherwise = foldl (\closest (p1:p2:_) -> minimumBy (compare `on` distance) [closest, (p1, p2)])
                      closestPair 
                      (windowed 2 pairsWithinMinimumDelta)
  where sortedByX = sortBy compare pairs	      
        (leftByX:rightByX:_) = chunk (length sortedByX `div` 2) sortedByX    
        closestPair = minimumBy (compare `on` distance) [closestLeftPair, closestRightPair]
          where closestLeftPair =  dcClosest leftByX
                closestRightPair = dcClosest rightByX     
        pairsWithinMinimumDelta = sortBy (compare `on` snd) $ filter withinMinimumDelta sortedByX
          where withinMinimumDelta (x, _) = abs (xMidPoint - x) <= distance closestPair
                  where (xMidPoint, _) = last leftByX

It takes up marginally less space and I think the change to use pattern matching on the length of ‘pairs’ makes the biggest difference as the code is now lined up at the same level of indentation.

The other changes would have more of an impact if there were more than 2 things being compared – right now I think either of the versions of the code are equally readable.

As I mentioned in a blog post about a week ago I decided to restructure the ThoughtWorks graph I’ve modelled in neo4j so that I could explicitly model projects and clients.

As a result I had to update a traversal I’d written for finding the shortest path between two people in the graph.

The original traversal query I had was really simple because I had a direct connection between the people nodes:

neo = Neography::Rest.new
paths = neo.get_paths(start_node,
                      destination_node,
                      { "type" => "colleagues" },
                      depth = 3,
                      algorithm = "shortestPath")
           .map { |x| x["nodes"] }
           .uniq
paths.map { |p| p.map { |node| neo.get_node_properties(node, "name")["name"] } }

I changed the way the graph was modelled so that you needed to follow a ‘worked_on’ relationship to a project in order to go between people:

In the first version I’d written some pre processing code in Ruby to check whether or not people worked on the project at the same time before creating the relationship between the nodes.

It wasn’t possible to do that with the new structure since I was working out if there was a colleagues relationship dynamically.

I therefore added a ‘start_date’ and ‘end_date’ property to the ‘worked_on’ relationship between a person node and project node so that I’d be able to take it into account when traversing the graph.]

I initially thought it would be possible to do this using cypher and wrote the following query:

start_node_id = neo.send(:get_id, start_node)
destination_node_id = neo.send(:get_id, destination_node)
 
query =  " START a=node(#{start_node_id}), x=node(#{destination_node_id})" 
query << " MATCH p = allShortestPaths( a-[:worked_on*]-x )" 
query << " RETURN p, extract(person in nodes(p) : person.name)"
paths = neo.execute_query(query)

I wasn’t sure how to do the filtering on ‘start_date’ and ‘end_date’ and Andres pointed out that it’s not actually currently possible to take relationship properties into account when traversing a graph with cypher so we need to do the filtering on ‘start_date’ and ‘end_date’ in code.

My first attempt to do that looked like this:

paths = neo.get_paths(node1, node2, { "type" => "worked_on", "direction" => "all" }, 
                      depth = 5, algorithm = "shortestPath").uniq
matching = paths.select do |row|
  relationshipPairs = row["relationships"].each_slice(2).to_a
  relationshipPairs.all? do |pair|
    r1 = neo.get_relationship(pair[0])["data"]
    r2 = neo.get_relationship(pair[1])["data"]
    r1["start_date"] <= r2["end_date"] && r1["end_date"] >= r2["start_date"]
  end
end

The problem with this approach is that it’s really slow due to the fact that I’m pulling back every relationship back in order to check the start and end dates.

Michael Hunger suggested an alternative approach where I still used a cypher query but returned the relationships instead of just the nodes:

start_node_id = neo.send(:get_id, start_node)
destination_node_id = neo.send(:get_id, destination_node)
 
query =  " START a=node(#{start_node_id}), x=node(#{destination_node_id})" 
query << " MATCH p = allShortestPaths( a-[:worked_on*]-x )" 
query << " RETURN p, rels(p), extract(person in nodes(p) : person.name)"
paths = neo.execute_query(query)
matching = paths["data"].select do |row|
  relationship_pairs = row[1].each_slice(2).to_a
  relationship_pairs.all? do |pair|
    r1 = pair[0]["data"]
    r2 = pair[1]["data"]
    r1["start_date"] <= r2["end_date"] && r1["end_date"] >= r2["start_date"]
  end
end
 
matching.map { |x| x[2].each_with_index.select { |x,idx| idx.even? }.map(&:first) }.uniq

This approach mostly works although it runs into problems in the scenario where two people have worked on the same project but not at the same time.

In that scenario the above code will return the relationship between them but it will then be filtered out by the start date/end date logic which means we won’t see a shortest path between those nodes.

I’m not sure how to solve that problem so for the moment I’m going to take the Josh Adell’s suggestion to keep the ‘colleagues’ relationship between nodes and use that relationship for the shortest path traversal.

The ‘worked_on’ relationship is still useful for other things that I want to do but not this particular one.

You're a Rails programmer. You've developed your site and have a good set of features that you're delivering to your users. You use RSpec to test your application and have Cucumbers pushing WebDriver tests. You may even be doing some cool process things like continuous deployment. But, do you write Ruby or do you write Rails?

Writing Rails

Rails is a great web framework that has helped many teams deliver business value in a very short amount of time. It helped to create the mantra of Opinionated Software that exists in the Ruby community and has even started to trickle down to users (for the benefit of all, in my humble opinion). Rails has also been a great leader in the area of testing and craftsmanship in the greater software practice.

That being said, Rails has become a crutch for many people (as all frameworks tend to do). It encourages a very flat world of MVC where the models can go get themselves out of a persistent store and controllers and views can access everything.

How much code do you have that is not MVC?

Name the number of classes in your code that do not inherit (directly or indirectly) from ActiveRecord::Base (or, in my case, Mongoid::Document), ActionView::Base, or ActiveController::Base. Is it more than zero? Is it a significant percentage of your application? Is it most of your application?

At a certain point your web application will become more than a CRUD web server where users manipulate objects in a database. Let us take for example GitHub. When you first log in to GitHub, you are sent to your dashboard. I'm going to write some bad code:

class DashboardController < ApplicationController
  def index
    # we're using devise and the user is logged in (checked in ApplicationController)
  end
end

.dashboard
  .news
    - current_user.alerts.limit(20).each do |alert|
      .alert
        .type= alert.type
        .header= alert.commit? ? commit_header(alert) : pull_request_header(alert)
        .message
          .committer_image= alert.commit.committer.gravatar
          .commit_mesasge= alert.commit.message

Excuse the incompleteness of the example but this is enough to illustrate. This is a Bad Thing™ because we are doing many N+1 queries against our SQL database. Easy enough to fix. We should pre-fetch all of our dependencies. I am going to use a more explicit way instead of trying to let the ORM do my work for me, again for illustration.

class DashboardController < ApplicationController
  def index
    @alerts = current_user.alerts.limit(20)
    @commits = Commit.where(:_id.in => @alerts.map(&:commit_id)).to_a # using Mongoid here
    @committers = User.where(:_id.in => @commits.map(&:committer_id)).to_a # using Mongoid here
  end
end

.dashboard
  .news
    - @alerts.each do |alert|
      .alert
        .type= alert.type
        .header= alert.commit? ? commit_header(alert) : pull_request_header(alert)
        .message
          .committer_image= alert.commit.committer.gravatar
          .commit_mesasge= alert.commit.message

Alright, so now we've taken care of the N+1 query issue. However, I'm loading a ton of unnecessary data from the database, building it into objects in member and then garbage collecting it later down the road. We're GitHub and we cannot afford this luxury, so we cut the fat (always test that this sort of optimization is necessary though!!).

class DashboardController < ApplicationController
  def index
    @alerts = current_user.alerts.only(:type, :commit_id).limit(20)
    @commits = Commit.where(:_id => @alerts.map(&:commit_id)).only(:message, :committer_id).to_a # using Mongoid here
    @committers = User.where(:_id => @commits.map(&:committer_id)).only(:gravatar).to_a # using Mongoid here
  end
end

.dashboard
  .news
    - @alerts.each do |alert|
      .alert
        .type= alert.type
        .header= alert.commit? ? commit_header(alert) : pull_request_header(alert)
        .message
          .committer_image= alert.commit.committer.gravatar
          .commit_mesasge= alert.commit.message

And, we're starting to notice that our .header= HAML line is really big. We could pull it into another helper method. Or, we could pull it up into the model itself. Which is it? View logic or model logic? For sake of my example, let's put it in the model.

class Alert
  include Mongoid::Document
  
  def header
    commit? ? commit_header : pull_request_header
  end
end

And we notice that our controller method is getting big-ish. We should move that code to the model, as well, right?

class Alert
  def self.alerts_for_dashboard(current_user)
     alerts = current_user.alerts.only(:type, :commit_id).limit(20)
    commits = Commit.where(:_id => @alerts.map(&:commit_id)).only(:message, :committer_id).to_a # using Mongoid here
    committers = User.where(:_id => @commits.map(&:committer_id)).only(:gravatar).to_a # using Mongoid here
    alerts
  end
end

And this is the state in which we push the code to production and call it a day.

What about Ruby?

We forgot about using Ruby itself at some point in learning how to develop Rails applications. I would contend that the model is the place for neither of these peices of code. The header code is not part of the model but is instead presentation logic.

On helpers: helpers are the default way to encapsulate presentation logic in Rails. There is nothing wrong with helpers themselves but they start to get out of hand when you start to pull in many different objects, cascade helper calls and use helpers from many different helper files. When that starts to happen, we need to move away from helpers to a more robust method of handling presentation logic.

This dashboard view should probably have something called an AlertPresenter (for another pattern, look at Avdi's Exhibit Pattern). The AlertPresenter would look something like this

class AlertPresenter # look ma, I'm an Object
  def initialize(alert) # look dad, I'm using explicit constructor arguments!
    @alert = alert
  end

  def header
    @alert.commit? ? commit_header : pull_request_header
  end
end

This would encapsulate all of the logic needed to write the "header" of the GitHub dashboard alert in one place. If I ever needed to change what that looks like, I know exactly where to look. Another side benefit of this class is that I do not need to load Rails to run this test. That means the test can run fast. Very fast.

The other peice of code is not presentation logic, but persistance logic. Because of ActiveRecord, we have started to conflate persistance and domain logic in our codebases. This is a bad trend. It creates confusion and a dangerous amount of coupling in our code. While I could go on about the benefits of a DataMapper pattern over the ActiveRecord pattern, I will just show you what we could have done with our code to not bind ourselves so tightly with AR.

Instead of defining a class method on Alert, we could create an AlertService that accesses these alerts for us. At this point, it really is just putting the code into another file but it does have benefits. First, its not Rails and doesn't actually require Rails to test. It makes the finding of alerts for the dashboard logic all easy to find and located in a different location. And, it provides a way for us to change how we access it. If, for example, we were to implement CQRS in our application, the Service would be the place to do it. Or, if we decide that we need to implement an HTTP service on the backend that serves us the objects instead of direct access to the database, the service layer can be a point of abstraction for persistance logic. There are more benefits if we take the service/repository pattern further.

class AlertService
  def self.alerts_for_dashboard(current_user)
    alerts = current_user.alerts.only(:type, :commit_id).limit(20)
    commits = Commit.where(:_id => @alerts.map(&:commit_id)).only(:message, :committer_id).to_a # using Mongoid here
    committers = User.where(:_id => @commits.map(&:committer_id)).only(:gravatar).to_a # using Mongoid here
    alerts
  end
end

So, in this very small example, we have decided to create a couple of Ruby objects that make our application more flexible, testable and cleaner.

Cleanliness

I challenge Rails developers to develop a sense of cleanliness of their code. We have a huge commitment to tests and working software. We have a commitment to opinionated software that relies on convention over configuration and to give back to the open source community. It think us Rails developers can learn to appreciate the cleanliness of code. I would define cleanliness as follows:

• Thin controllers (the same thing we have been doing for a long time)
• Logicless views (use Presenters/Exhibits to wrap any presentation logic)
• Domain models (avoid persistance and presentation logic)
• Repositories [they can be called Services] (for persistence logic)
• Services (for manipulation of Domain Models -- puppet masters if you will -- this is another area for domain logic)
• Fast tests that do not rely on Rails (see Corey Haine's Fast Tests talk)

I've only talked about a couple of the things that make clean Rails code possible (Presenter and [Persistance]Service). I hope to talk about other things in the near future.

自从开始做产品之后，我就成为了产品运营人员的一部分。每天，我与行政收到同样多的邮件。出差请求，感谢，抱怨，我都一一看在眼里，记在心里。得益于之前出了无数的差，因此对于出差人员的感受非常直接，然而在行政这边，估计许多人如我一般，对她/他们了解甚少。与程序猿不同，她们（多数是女性）的时间被扯得稀烂，最新来的事情优先级最高。她们善良体贴，乐于帮助他人，一点点的工作失误自己会难过得不行，而来自同事的称赞则能让她们高兴好久。

这种真实的体验让我对产品与用户之间的距离有了更深刻的理解。产品还在试运营。一天行政苦笑着跑来找我：网页出错了，500 Error。我立刻几乎可以想象，她正在使用着软件，点着界面，突然之间这个恐怖的、白色的、带着几行红色粗体的错误信息页面出现在面前，对于一个没有任何编程经验的用户，是多么恐怖的体验。一种无法解释的内疚油然而生，我看着她说：对不起，理解你的感受，我马上修复。

她楞了一下，迅速说：没事，不着急。

几分钟之后修复好了。她在IM上回复说：太感谢了！

这些让我对软件的价值重新考虑。我们需要真正的使用这些软件，才能够得出真正的体验，并能够在使用不便、出现问题时真正感同身受的说：说：对不起，我理解你的痛苦。我们正在修复。

很多软件忽视这个过程。软件的设计者、制造者、销售者、客服与用户之间，通过各种部门、KPI隔离开来，然而又妄图通过各种制度来标准化服务流程。可惜，销售者不用自己的软件，客服也没用过自己的软件，制造者根本见不到用户使用的样子，又如何能够真正体会用户的感受，制造出贴心的软件，提供贴心的服务呢？

Paulo: Zip RogerRoger: Zip MarcoMarco: Zoom Mathias...
Zip Zap Zoom is a great energy boost ice breaker. I made a small addition to it for helping the team memorizing each other names.After the Z__ command (Zip, Zap, or Zoom), the person passing the imaginary relay has to say the target person name.This keeps the energizer fun while helping the team memorize each other names.

Winterthur, DE

Retread of post orginally made on 06 Sep 2004

I've heard a couple of questions recently about coming up with a standard story point mechanism for multiple teams using extreme programming's planning approach. The hope is have several teams all using equivalent story points, so that three story points of effort on one team is the same as on another.

I think trying to come up with this at best of limited value, and at worst dangerous.

The estimating system of extreme programming is based on XpVelocity and YesterdaysWeather. Inherent in this is the idea that when you make estimates, the actual units you estimate aren't important - what's important is you estimate by rough comparative value and use YesterdaysWeather for calibration.

In this situation the story points act as an anchor for the feedback loop that Yesterday's Weather provides - nothing more. Baked into them are all sorts of assumptions about the nature of the team's task, the capability of the team, and whether the team are optimistic or pessimistic estimators. Once you try to come up with a standard across teams you are trying to normalize all of these factors. Trying to do this sounds very hard to me, and I'm not aware of anyone who has done this effectively. This doesn't mean its impossible, it just means it's hard.

The dangerous aspect comes from once you have a standard unit for measurement across teams, someone is inevitably going to use it to compare the performance of teams. Even if everyone swears till they are blue in the face that they won't use it for cross team measurement, there will always be the suspicion that this will happen eventually. This will cause teams to distort their measurements so that it seems that they get more story points done. My fear is that this will break the feedback loop of yesterday's weather and knock the planning process off kilter. I'm always suspicious about these things because while it would be incredibly valuable to have a way to measure productivity I think the nature of software is such that we CannotMeasureProductivity.

So to be worth trying, this has to yield some valuable benefits - but I don't see any. One reason that I've heard is to help people move onto teams and estimate more quickly. But you can't estimate on a new team until you get reasonably familiar with the problem and the current code base. As you do that you'll also get a feel for the relative sizes of story points on that team. We move people around between projects at ThoughtWorks and I've never heard anyone complain about difficulty of estimating when coming onto a new team due to differences in story points.

reposted on 10 May 2012