Writing a Web App: The Idea

The best ideas for web applications are usually ones where you identify a need of your own – something that you personally want.

Something I’ve done ever since I first owned a car is to record my gas mileage – how many miles I get on each tankful and hence how many MPG (or l/100km) I’m getting. It’s interesting to watch how the numbers vary according to different factors:

  • how much of the tank has been used for city driving and how much has been used on the highway
  • summer driving vs winter driving
  • times when you’ve been consciously driving economically vs times when maybe your right foot’s been a little heavier

When I bought my first car, I typed these figures into a spreadsheet. The spreadsheet grew to be quite sophisticated with different stats, averages and graphs. It gave me everything I wanted. In 2000 when I bought my beautiful Palm Vx I switched to an MPG tracking app on there which meant I always had the numbers with me, but at the cost of using a tool which wasn’t entirely to my liking, didn’t give me all the features I wanted and locked my data away from me.

These days I still record the numbers but I haven’t been tracking them with anything for years. I have a couple of envelopes packed with old gas receipts but I have no statistics based on those figures.

So, what I want is a website where I can enter my gas purchases and track my MPG. Simple.

With the idea in mind, the next stage is research

This post is part of a series – read them all

Writing a Web Application

I’m going to write a new web application from scratch. Just for fun & with no intention of making my fortune from it (but it wouldn’t hurt if I did).

I’ll be writing it in PHP – possibly plain PHP or possibly using a framework. I’ve done some work in the past with the Zend Framework, CakePHP and Symphony and not felt entirely satisfied with any of them so, if I do use a framework, I’ll be using Code Igniter. I’ve heard good things about CI, talked to a couple of people who love it and worked my way through the tutorials without being put off yet… it seems to fill a nice middle-ground between CakePHP and Symphony.

To make life more fun, I’ll be documenting my progress here, talking about what I’m doing and sharing some of the code.

This post is part of a series – read them all

Where have I been?

Hmmm – where HAVE I been?

Well, the answer to that question is “right here!”. I guess the more accurate question is “what have I been doing?”.

One of the things I’ve been doing is restarting my homebrewing. It’s been going very well indeed – incredibly enjoyable work producing 4 batches of beer, 1 of wine and a batch of apfelwein (a dry German cider) so far. Made and enjoyed with (touch-wood) no failures or anything that wasn’t absolutely top-notch – I’ve been really impressed with the outcome (and, I guess, impressed with myself for producing it!). I’ve been photographing the hell out of the process and you can see the photos on flickr over here and maybe we’ll see about another form of documentation in the near future.

I’ve also left my Toastmasters club. I felt I’d gone as far as I could with them. Three years membership, half of it as VP Education taking us to two successive Presidents Distinguished Club awards. I definitely don’t feel I’ve finished with Toastmasters, but it was time for some new challenges for me, so I’m currently looking at new clubs to join.

Professionally, I’ve been doing some part-time consultancy work but I think it’s time to find a full-time development job. To help with that goal I’m going to be brushing off a couple of demo sites and projects over the next month. Always good to have something to show people.

Toastmasters Contest Rules

Twice a year, Toastmasters turn their thoughts to contests. In September each club stages a Humorous Speech Contest and a Table Topics Contest with the club winners going on to compete in Area, Division and District competitions.

The organization required for these competitions can be quite daunting to newer members but it’s very important that everything happens in accordance with the official rules. To help ensure that people know how to run a contest successfully there are training sessions held to help get them up to speed.

In the past, I’ve competed, judged, chaired and mentored speech contests so I’m fairly confident that I know my way around one, but I know that you can never know EVERYTHING, so last weekend I attended a speech contest training session run by District 21 Division B in Vancouver.

Despite having experienced many contests in the past and despite being very familiar with the contents of the official rulebook, I learnt LOTS. I took lots of notes and decided to pass them on in the hope that they might help someone else too.

This information isn’t a definitive guide on how to run a contest but it expands on the information in the Toastmasters Speech Contest Rulebook. You should probably be familiar with the rulebook in order to get the most out of these notes…

The main goals of competing in a contest are:
– to experience a new audience
– to challenge ourselves

Why is it important to run a good contest? It’s important because the competitors have put time and effort into taking part, you owe it to them to ensure that your contest is FAIR and abides by the rules.

Contest chair:
  • Be organized
  • Make sure you have plenty of copies, plus spares, of all the papers & forms
  • Confirm who’s bringing the timing lights & ensure at least one backup
  • Have a copy of the contest rulebook
  • Keep in contact with Area/Division/District Governor
  • Hand out and collect eligibility and bio forms IN ADVANCE of the contest – that way you’re not scrambling to get them all signed and collected at the beginning of the meeting
  • Brief contestants/judges before the contest starts (ie don’t announce the meeting start, welcome the audience and THEN break for briefings) that probably means asking contestants & functionaries to arrive 30 mins early or advertising the meeting start as 30 minutes later
  • Get the chief judge to pick an experienced member as tie-breaker judge, known only to them
  • Keep tie-breaker & counter forms after contest in case of complications/enquiries or if top 2 competitors can’t compete at the next level & the club needs to send 3rd place
  • Double-check name spelling & pronunciation
  • Rehearse your briefing, don’t wing it – the contest briefing at District level takes 30 minutes!
  • Check the District 21 website… there are example briefing scripts on there
  • Eligibility form must be filled out for ALL contests
  • Originality section is only required for appropriate contests
  • Contestants can reuse a speech that they’ve already given
  • Contestants must attribute ALL quotes
  • Contestants arriving late can compete only if they arrive before the contest chair has been announced but they have to waive their right to the briefing if they’ve missed it
  • Sit the judges scattered around the room so they can check for voice & eye contact
  • Judges should be anonymous – don’t put their names on the agenda
  • Draw for speaking order on the night – not in advance
  • For Area contest, each club should provide 2 judges
  • For District contest, no clubs with members still competing can provide judges
  • No photos are permitted
  • Video is a grey area – if ALL contestants agree then you can do it but DON’T put on YouTube until the winner has been knocked out (so there’s no chance a judge from a higher contest level might see the speech in advance)
  • Sgt-at-arms is responsible for dealing with talkers in the audience or removing hecklers
  • For a serious interruption, eg fire alarm, speaker gets 30 seconds grace on timing
  • Chair stays at podium for the minute’s silence… they’re still in charge of the meeting
  • BEFORE contest starts, brief visiting dignitaries & ask them to help present awards
  • Ensure the results are passed to Area/Division/District Governor immediately after contest so they can plan the next level contest
Counting:
  • When collecting ballots, don’t hover over the last judge completing their ballot. Totting up the scores is a stressful time for judges and hovering doesn’t help them
  • Double-check the count: one counter reads scores from ballots, another writes them down, another adds them up. Then switch roles & repeat AT LEAST once
  • Prefill the counting sheet with the judges’ and contestants names before the count to help speed things up
  • Each ballot MUST be signed, names must be legible, can’t have 2 names in one space
  • Must have each competitors’ full names on ballot
  • If a name on the ballot is illegible, you can call the judge out for clarification
  • If one ballot space contains 2 names eg “Smith/Jones” or ballot not signed, ENTIRE ballot is spoilt and must be ignored
Timer:
  • Ensure you have backup lights (and coloured cards), use two stopwatches for timing AND have a spare
  • When speaker gets close to time limit, don’t keep looking up at speaker & looking down at watch. That constitutes a visual cue to the speaker & is prohibited
  • The speaker always gets the benefit of doubt if two stopwatches show different times eg if one watch says 7:32 & the other says 7:29, the contestant gets 7:29 & isn’t disqualified
  • Lights should be only visible to speaker so audience doesn’t follow along with the timing
Eligibility:
  • Contestant’s dues must have been paid and club must have paid them on to TI
  • Club must be in good standing with TI
  • Only speakers, chair & judges can protest
  • Contest chair is responsible for ensuring contestant eligibility (for club contest they could delegate this eg to President or VP Ed)
  • Easiest way to check eligibility is go to TI website before contest & print out club member list – if they’re on it, they’re eligible, if they’re not then they’re not
  • Doesn’t matter WHO the contestant has given their dues to, TI MUST have their money or they’re ineligible
Judging:
  • For Table Topics, it’s important to ANSWER the question
  • For evaluation contest, it’s important to give tips and give a summation
  • If you have a tie on your ballot for 1st, 2nd or 3rd pick the person you think did the best overall job TONIGHT
  • Only write one name in each space on the ballot
  • Poor word choice and/or clothing may be part of the speech so don’t judge on those until you’ve heard the full speech
Contestant interviewing:
  • Gives audience a light break from the contest
  • Gives counters time to count & recount ballots
  • For timing, base it on how long you estimate the counters will take. Typically 2 minutes per contestant
  • Take question material from the bio sheet
  • Don’t be afraid to ask for extra information before the contest or even ask contestant in advance “is there something you’d like me to ask you?”
  • Simplest questions come directly from info on the bio sheet
  • Can also ask (eg) “How has Toastmasters helped you with {item from your bio sheet}”
  • Can take question material from the speech itself
  • Interviewer isn’t the centre of attention – don’t upstage the interviewee
  • Respect the speaker, don’t ask awkward questions & don’t make them difficult – this isn’t Table Topics!
  • Interview should be inclusive… direct the questions at the audience as well as the speaker and use body language so that the audience doesn’t feel excluded
  • Ask open-ended and short questions
  • Compliment & congratulate the speaker
  • Choose questions appropriately to get answers of suitable length
  • Be prepared to cut off interviewee if over-running… step forward, offer handshake

I was guest judge at a neighbouring club’s contest this week and noticed several things which didn’t go smoothly so I guess I can add a supplemental set of tips:

  • Remember that if a contestant in the FIRST contest is also taking part in the SECOND contest then you shouldn’t interview them after the first contest. This is to prevent the judges for the second contest hearing anything that might sway their opinion
  • Make sure that no competitors wear any badges or other indication of rank or awards
  • If you’re using borrowed lights or anything else that isn’t your club’s usual equipment then make sure that everybody who will use this or be affected by it is given a full demonstration beforehand to ensure there are no slip-ups or misunderstandings
  • Prepare your questions before the contestant interview – don’t be reading the bio on stage
  • If you’re handing out participation certificates then do it when you interview the competitors. This saves you having to call them all up to the stage again later on

There were also plenty of things that went RIGHT, and I picked up some tips myself:

  • If you’ve finished interviewing the contestants and the counters haven’t finished calculating the result yet then call up one of your visiting dignitaries to talk to the audience. If you’ve got an Area/Division/District Governor in attendance then everybody would love to hear from them and this way you don’t have to tag on extra time for that at the end of the contest
  • I think it’s reasonably well understood that you only announce third place if you have five or more competitors in a contest. The reason for this is to ensure that there are at least two contestants whose names are not announced. This makes sure that nobody knows who came last. However what do you do if you only have three contestants? You need a second place because you need to have a reserve in case the winner can’t represent your club at the Area contest. But if you announce second place then everybody knows who came last. The answer is that you only announce first place and then you discretely present the second place contestant with their certificate after the meeting has ended. That way you have your second place backup but the audience don’t know who came last. (That tip came direct from our current District Governor, Tom Jones, who was in attendance)

I mentioned earlier “be sure you have plenty of copies of all the forms”. Here’s a quick checklist of the forms I think you need for a contest:

  • contestant eligibility & originality form
  • contestant bio form
  • a sheet to record the speaking order for both contests
  • humorous speech judging form
  • table topics speech judging form
  • tie-breaker judging forms for both contests
  • timing record form
  • counters record form
  • notification of winner form

I’ve got one final tip that I put into our Club Contest agenda when I last ran a contest and has now spread down the road to our neighbours. When you’re giving a prepared speech in a contest and you’re NOT speaking first, you gain an advantage over the first speaker as you have one minute of silence before you take the stage in which to gather your thoughts. This is the one minute that the judges are using to write their scores for the preceding speaker. With nobody speaking before them, the first speaker doesn’t get this time. In order to balance things out and make everything as fair as possible I added one minute of silence to the agenda before the first speaker speaks – I think it’s a nice little touch.

Facebook Places – how to turn it off

Earlier this week, Facebook announced their location sharing service “Facebook Places“. If you’re familiar with services like Foursquare and Gowalla then you’ll know what to expect here… when you’re out-and-about you ‘check-in’ at your destination – an action which lets your friends (and possibly other people) know where you are. Like a lot of social media broadcast services there’s bound to be a certain amount of ego associated with location sharing – “I’m out at an exclusive/expensive/exciting place and you’re not” but the services talk about the benefits to users such as discovering your friends are in the pub next door so you can meet-up and share a beer.

However there are differences between Foursquare/Gowalla and Facebook. Facebook has, time and time again, played fast and loose with its users’ information and privacy. The general vibe I get from Facebook is that they’re only providing a service in order to get data from you that they can exploit and/or sell. Over the last year I’ve gradually reduced the amount of information I have on there, the amount of information I add and the level to which I share it. I just don’t feel comfortable with the site – yet, with any social media service, you have to be where the people are in order to make the connections so I keep my account active. I tend to use it as a back-up service to Twitter… somewhere for conversations that can’t be constrained into 140 characters (which is kinda weird of me because, as someone pointed out the other day, my Twitter stream is unlocked so EVERYBODY can see that and do what the hell they want with it).

Facebook Places isn’t available in Canada yet but they HAVE enabled the privacy settings so you can go in there and preemptively set your privacy before it comes North of the border. This is important even if you don’t intend to use the service.


You need to click Account->Privacy Settings->Customize Settings. Then there are three different settings to adjust:

  • “Things I Share->Places I check in” determines how widely your whereabouts will be broadcast. Do you want everybody to see where you are? Just your friends? Just a subset of your friends? Nobody at all? The most private you can get with this setting is to click on “Custom” and select “Only Me” from the list box

  • “Things I Share->Include me in ‘People Here Now’ after I check in” will cause your presence to appear in lists of people at an establishment/event. Depending on Facebook’s implementation, this might override your “Places I check in” setting and let people outside your friends list see where you are. If you’re interested in your privacy you probably want to disable this setting

  • “Things Others Share->Friends can check me in to Places” is super-important. It’s the geo equivalent of having other people tag you in photos. Worse in fact. If someone checks you in as being down the pub in the middle of the afternoon there’s nothing to show that you WEREN’T actually there. At least with a photo other people can look at the photo and SEE that you aren’t in it. If this option isn’t disabled then your whereabouts and your privacy are totally out of your control. I suspect all but the most cavalier of users will want to set this option to Disabled. Even if you never intend to use Facebook Places yourself, you should go and turn this setting off

Good luck and be careful out there.

Vacuuming – Your Mum Was Right

Yes, it IS important to vacuum once in a while… who’d have guessed?

The first computers I ever built (and then disassembled and then rebuilt) never seemed to gather any dust. But over the last 5 years I’ve seen an increasing amount of dust gathering inside my computers’ cases. I have no idea why… it seems to have corresponded roughly to when I moved from the UK to Canada so maybe Canada is a dustier country 😉

Over the last year I’ve seen the internal temperatures in my current desktop PC steadily rising:

Computer internal temperatures before cleaning

Before cleaning

These are the Core0 – Core3 temps and two copies of the hard drive temp.

So last week I opened up the case and vacuumed it. I cleaned all the vents, sucked the dust off the fans, sucked all the dust bunnies out and cleaned the processor heatsink.

The heatsink was problematic. The PC has a Core2Quad processor (big heatsink) in an Apevia X-QPack mATX case (small case). As a result there’s not much space and the narrow nozzle of the vacuum cleaner was never going to get anywhere near the heatsink without me disassembling the case. Rather than do that, I used the poor man’s compressed air duster (ie I blew through a straw) and blew the dust out of the heatsink vanes then vacuumed everything up.

With everything reassembled and allowed to run for 24 hours, the temperatures dropped to:

Computer temperatures after vacuuming the case and motherboard

Temperatures after cleaning

Those temperatures are Core0 – Core3, “CPU temp”, “motherboard temp” and the hard drive. [Actually, at time of writing, they’re 5C below those numbers – but today’s a much colder day]

Not sure where the “CPU temp” and “motherboard temp” sensors are being read from. The case has a front panel temperature LCD and two free-floating sensors which I’ve attached to the top of the hard drive and the chipset heatsink… but those are currently reading 26 and 47 degrees… so apparently not related.

To complicate things slightly, between taking the two sets of readings I also upgraded from Ubuntu 9.10 to Ubuntu 10.04 (more about that later). I’ve seen some reports on the internet from people who noticed the reported sensor temperatures dropping as a result of the upgrade. I did the upgrade a day before I cleaned and didn’t see any fall in reported temperatures afterwards so I don’t believe that that’s a factor in the improvement that I’m seeing.

So there you have it: a year’s worth of dust had clogged up my CPU heatsink and fans to elevate temperatures by approximately 15C. If it’s been over a year since you opened up YOUR computer’s case then maybe you should give it a spring cleaning too?

Protecting Your Users’ Passwords – Part 2

Last week I showed you how NOT to store your users’ passwords in your database: the biggest sin of all is storing them as plaintext and the ‘false sense of security’ solution is to apply a hashing algorithm to them.

We saw that we can use a common hashing algorithm (the algorithm I used is called MD5: http://en.wikipedia.org/wiki/MD5) to turn “donkey” into “9443b0fceb8c03b6a514a706ea69df0b” and I told you that there’s no programmatic way to turn that back into “donkey” – the hashing algorithm is one-way. However, if you did last week’s homework and pasted that ciphertext into a search engine you’ll have found you got many returns. Why?

A little history: when the commonly used hashing algorithms were created, they were designed to be computationally “expensive”. That means they take a lot of processor power (and hence time) to calculate. This was deliberate – a user only has to login occasionally so it didn’t matter if it took 2 or 3 seconds to check their password. The excellent side effect of this delay was that it prevented a hacker from trying to guess your password by brute-force. Even assuming you’d been silly and used a dictionary word as your password, a hacker couldn’t break into your account by trying every word in the dictionary as he’d be there for a very long time. A quick calculation with my machine’s dictionary says, taking 3 seconds per attempt, it would take 3.4 days to attempt every dictionary word. Unfortunately for hashing algorithms, computers have got very much faster in the last 20 years – even my little laptop can generate a hash in 0.04 seconds. Suddenly the time to run through the entire dictionary has shrunk to one hour and our apparent security has vanished.

Things get even worse though. If you have a dictionary word as your password and I have access to a hash of it, I can tell you your password in just 5 seconds. I paste the hash into a search engine – one click on “search” and I have your password. What’s happened is that hackers have done all the hard work up front – they’re already run entire dictionaries through the common hashing algorithms and they’ve posted the lists of words and hashes on the internet where search engines have found them and indexed them. So although it’s technically true that we can’t take a hash value and “unhash” it, hackers do have access to functionality that can perform a similar job – for single words.

“OK”, I hear you say, “but I’d never be stupid enough to just use a plain dictionary word as my password – I’ll put a number on the end of it”. Right then… that might help, but it might not… 8339e38c61175dbd07846ad70dc226b2 and 2484b2d1aec71de2ca87f88af401a6af are hashes of dictionary words with numbers on the end and both are indexed by Google (vote1234 and password99 in case you can’t be bothered checking). Although if your password is “aardvark50” then you’re safe as its hash 0913c211b2eaa2a8b3b11fe53bdf9b4f doesn’t appear on the internet (until now of course because Google will index this blog post and your secret will soon be out!).

So how should we, as programmers, prevent our users’ passwords being cracked like this? The answer is surprisingly simple. We concatenate the password with some other information before we hash it.

The best approach is two-pronged. Firstly we concatenate with a fixed nonsense string eg “78g^&FB%V^&I” – this ensures that, however simple a password the user has entered, we’ve created something that’s pretty much guaranteed to never have existed as a string before in the history of the Internet. Secondly we also concatenate it with a piece of information that’s specific to that user on our site eg their username. This is just icing on the cake to make sure that the hashing is different for each user – so if two users use the same password then their hashes will be different. The procedure is the same as before: we apply this “super-hash” to the password that the user initially sets before we store it in our database and we apply the same “super-hash” to the user’s password attempt before we check it against the database entry.

So now, if user “smith” sets their password as “donkey”, the hash that we’re storing is the hash of “smithdonkey78g^&FB%V^&I”. Good luck finding an online hash dictionary that contains THAT!

Incidentally, my previous post is currently the second return on Google for “9443b0fceb8c03b6a514a706ea69df0b” (the hash of “donkey”) and I’ve actually had incoming traffic from that as a search term, so we KNOW that people are actually using search engines to crack hashed passwords like this. Consider yourself warned and make your code secure.

Protecting Your Users’ Passwords

I’m currently working on a PHP-based web site that stores member details – username, address, password etc in a database. Nothing unusual here… literally millions of web sites on the internet are doing exactly the same thing.

But password storage is a dangerous area. Every month we hear about a high-profile web site being hacked into and all the user accounts made public together with their passwords. This is not good – especially as it’s fairly common for users to keep the same password across all of the sites they access.

Good web site security is about defence in depth. Yes, you set up security so that hackers hopefully can’t get access to your site’s files and databases. But you shouldn’t stop there. Any site whose password list has been published has made one other simple and easily avoidable mistake: they’ve made the mistake of storing their users’ passwords in plain text.

Storing passwords in plaintext is a dangerous mistake that’s easily avoided – there’s a much smarter way to do it and it involves something called a hashing algorithm. A hashing algorithm is a form of encryption which is ONE WAY i.e. you can convert the plaintext to the encrypted form (know as ciphertext) but you can’t convert it back again. For example, if you start with “donkey” and run it through a well-known hashing algorithm you end up with “9443b0fceb8c03b6a514a706ea69df0b”. In theory, there’s no easy way to go the other way and turn that back into “donkey”.

But how the heck does this help us with passwords? Surely we’re going to have to turn the encrypted password back to plaintext in order to check it? Nope – there’s a neater way of doing this.

When the user initially sets their password, we run the hashing algorithm on the plaintext password and generate a hashed version of it. We store that hashed version in our database. When a user attempts to login, we hash their password attempt and compare THAT to the ciphertext of the previously hashed password that we’ve stored. Because the hashing algorithm is repeatable, if the password they attempted to login with matches the password they setup originally, then the two hashed ciphertexts will match too and we’ll successfully validate their login.

There’s no excuse for not knowing about this design pattern – Unix & Linux systems have been handling user passwords in this way for the last 30 years.

When you implement a system like this, there’s one thing you CAN’T do. And that’s recover a password that a user’s forgotten. Remember that the hashing algorithm is one way. You can’t turn “9443b0fceb8c03b6a514a706ea69df0b” back into “donkey” when the user can’t remember their password. As a result of this you should, as a user, be very wary of any websites which offer to email you your password when you’ve forgotten it. If they can email your password to you then they’re not using a hashing algorithm to store it in their database and therefore their database is not secure should a hacker get access to it. As a programmer, if you use a hashing algorithm to safeguard your users’ passwords then all you can do if a user has forgotten their password is to generate a new one for them (or let them set a new one themselves). This is generally done via an email that you send them – either containing a new random password that you generated for them or (better still) a one-off link that gives them access to a special page on the website where they can set a new password.

OK then, we’re sorted are we? Everything’s secure and protected from the hackers? Unfortunately not.

DO NOT IMPLEMENT WHAT I’VE JUST DESCRIBED.

There’s a flaw and I’ll tell you in a few days what that flaw is. In the meantime, you might like to paste that ciphertext into your favorite search engine and wonder about what just happened.

Agile Vancouver: Trust and Team-Building

I went to a very interesting talk at Agile Vancouver last night and thought I’d share my notes. I love human psychology experiments and this was packed full of them. Combine that with software development and you’ve got a winner.

The talk was given by Linda Rising and titled “Who Do You Trust?”

Trust is the most important factor in implementation of the Agile software development methodology because social dynamics & interaction are the biggest cost driver in software development – bigger than better tools and methods.

A previous talk of Linda’s has covered the Agile placebo effect… does Agile software development work because we THINK it’s going to work? We go into an Agile project expecting things to work.

She started with a disclaimer that, amongst other things, this is a “Presentation of a disturbing nature”. A nice introduction that got everybody’s attention.

In Computer Science, we never do experiments… proper scientific experiments with a hypothesis, observation etc – seeing results from a change in development methodology doesn’t prove anything.

She introduced the Robber’s Cave Experiment – conducted at a campground in Oklahoma in 1954. Two teams of 12 year old boys, same backgrounds, balanced teams. Both teams were transported to the campground separately, didn’t know the other team existed.

The experiment was conducted in three phases:

Phase I: The 1st week was spent in isolation… each team quickly “became an us”: gave their teams names, designed a flag, places became “our” swimming hole, “our” firepit etc

Study team gradually let them know the other group existed – but not yet seen. This resulted in immediate division, saying things like “I hope THEY don’t use OUR swimming hole” – even though the makeup of the other group was identical, there was nothing different between the two sets of children.

Phase II: The two groups were introduced to each other and staff organized competitions & events with a winner & loser, prize trophies and money. Deliberately designed to create friction between the teams.
The trophies were displayed in the common mess hall where both teams could see them and talk about their victories.

As phase II progressed, one team burnt the other team’s flag – then the other team retaliated and burnt their flag… eventually staff had to intervene.
Then they started raiding each other’s cabins at night. More retaliation led to rocks being stockpiled for use defending themselves against attacks. Again staff intervened and had to bring phase II to an end.

Phase III was about having fun together. They scheduled non-competitive activities eg watching movies, eating together.

(At this point Linda compared this action to corporate team building exercises – you bring together different parts of the business, different teams, and just expect them to enjoy mingling)

Phase III failed – the teams were still far apart – there were food fights and yelling. If you’ve ever been on a corporate team building exercise, this probably isn’t a surprise to you.

Linda pointed out that you might expect behaviour like this if the team divisions had been based on things like religion & politics.
But we also see it in development teams and other relatively trivial settings.
For example, during software project recaps, you hear people talking about “us”, “them”, “the others”.
eg something trivial like maybe some of the developers got a T-shirt, others didn’t. Maybe this was caused by a distributed team. The T-shirt is not the actual problem but is a symptom of thinking about the distributed team differently. – “oh, I forgot to order enough for the guys in Victoria”.

We see the “others” as the enemy. And yet we think of ourselves as unbiased & rational when it comes to decision making.

Psychologists see this as a hard-wired reaction – so it must have an evolutionary benefit. Our ancestors had to answer lot of classification questions quickly: is this food edible or not edible? Is this person a friend or a foe? The decisions are made very rapidly.

Another experiment: conducted by Jane Elliott, a 3rd grade teacher in Iowa in 1968. She was trying to give her class of white kids a feeling of what it might be like living in the US as a black person. She separated the class into blue eyed and brown eyed children and told them that it had been scientifically proven that blue eyed children were better. There’s a TV documentary you can watch about this experiment here: http://pbs.org/wgbh/pages/frontline/shows/divided/etc/view.html

The part of the outcome that’s interesting to us right now is that the kids were all as good as each other and yet the kids who were set aside “knew” that they weren’t as good as the others – they bought into the labelling.

Research says managers sort employees into winners/losers as early as 3 weeks after starting to work with them. At an Agile conference Linda asked a table of managers if this was true. They said “well yes, but we are always right”

If you get two groups of people who disagree on an issue and give them the same paper on the subject to read about it, both groups will say that it supports their point of view. We see what we want to see. The managers alter whatever facts they see about the employee to suit the decision that they’ve already made.

Everybody makes mistakes. However we forgive our own behaviour but not others. “MY intentions were good even if I missed the deadline” – people’s judgment of their own performance is “contest sensitive”, but their judgment of other people’s behaviour is absolute.

So people get stereotyped. But people are complex. When we label people we lose appreciation of their other talents. BUT interestingly we also do this to ourselves.

Eg: take a group of maths students and give them a hard test. When the test starts with a male/female tick box, men outperform women – as this is the gender stereotype. Remove the box and give the same test and both genders perform the same. Similar research playing video games has shown that both genders will give up easier if they’re playing as a female avatar.

Another example is the Solomon Asch experiment:

A group consisting of one test subject and a number of actors are asked simple questions related to line lengths on cards. The test subject is always asked last.
For the first couple of rounds, everyone agrees then the 1st actor gives a clearly wrong answer & the others agree with them.
A significant number of test subjects go along with the incorrect answer too.

Recent research shows that if you put the test subject in an MRI machine you can see that there’s no debate going on in their head – they’re not thinking “should I say this – I don’t want to look out of place”. They actually see the wrong line as correct. The actors set the filter for the line lengths & our brain believes it.

So if other people can lead us to believe that about line lengths, how about more significant things?

So what have we got now…

  • We’ve shown that stereotypes change our behaviour.
  • And we’ve shown that our behaviour affects other people’s behaviour.

And so we have a self-fulfilling prophecy: we judge our employee as not being very good and treat him accordingly and so that’s how he behaves. Hence the managers at the Agile conference ARE always right about their judgments.

So how do we change this – how do we use these things to our advantage.

Rule #1 of good management: catch your team members doing something right and praise them for it.

Linda mentioned she’d been to project retrospectives where they always started by repeating Norm Kerth‘s Prime Directive: “Everyone is doing the best job they can”. It seems cheesy but after a while the team members start to believe it’s true & eventually they create that in the team – everyone IS doing their best.

Back to the boys in the campground: the study team created problems that the groups must work together to solve.

They cut off the water to the campground and set the boys to search for the “leak” along the mile long pipeline. Required all the boys to work together. They discovered a clogged valve… they said “we” found it & celebrated together.

Other tasks followed that required everybody to work together to accomplish, culminating in a ‘last night of camp’ where they sat around the same camp fire and alternated singing songs for each other. Both groups insisted on going home on the same bus and the team who’d won the most prize money bought everybody milk-shakes.

Linda suggested maybe this experiment wasn’t surprising… after all the boys were all very similar. The experiment was repeated in 1963 in Beirut with a mix of Christians & Muslims. During phase II there was serious fighting and three group members threatened a member of the other group with knives that they’d stolen from the camp kitchen. The study team intervened and cancelled the experiment – there was no Phase III. To be expected?

However the groups were NOT divided by religion. The three group members who had the knives were all Christians – and the opposing group member that they threatened was also a Christian.

In this case, group membership trumped religion even though the religious divisions were hundreds of years old and the groups had only been together for a week.

So there are two responses here that are hard-wired. One is that we quickly judge people into groups as “us” and “them”. The other is that we like to work in small teams and we like to collaborate.

So to resolve conflict it’s necessary to cooperate on shared goals.

Similarly to the campground, this must combine the entire organization – it can’t just be development.

Agile practices help with this:
– face-to-face communication increase cooperation. Strongest effect of any variable
– Stand-up meetings
– pairing – produces better result than either individuals could achieve individually
– short iteration timeboxes means everybody gets frequent goals to work together on
– project retrospectives

Liking someone is not required for these activities to pay-off
Liking someone is different from “respect”

Social interdependence requires:
– common goals
– outcomes affected by actions of others
– individuals only reach goals if others reach goals

In a collaboration, nobody succeeds unless everyone does. Therefore efforts must be coordinated.
The coordination produces respect.
We all like being trusted and respected.
It follows then that Agile teams get trust and respect all day and are therefore happy 🙂

This has a positive impact and results in effort improvement in both individuals and the group.

Linda has a book out called ‘Fearless Change’ that documents design patterns for introducing change into organizations. People tell me I’m lucky at draws but last night was one of my unlucky nights… in the door prize raffle the guy immediately to my right won a copy and the guy immediately to my left won a copy and I left empty handed. Although I did leave with a head full of interesting information!

Vancouver Cloudcamp

Saturday just passed was Vancouver Cloudcamp. An unconference dedicated to all things cloud related. Errr, that’s aka internet-based computing… not lying in a field saying “hey, that one looks just like a rabbit’s head”.

I’m taking advanced PHP at BCIT on Saturday mornings at the moment so couldn’t get there for the morning sessions but raced over to Discovery Parks for the afternoon. Very very glad I did – some interesting sessions, great discussion and met a lot of interesting people.

I photographed all the flip charts I could find and took notes in the sessions I attended. I’ll post the notes I took once I’ve edited them so they make a least a little bit of sense. In the meantime, here’s the flip charts: