Category

general development

Loss Aversion and Tech Debt

0 Comment general development management

Humans are loss-adverse. We place an irrationally high value on losing something over gaining an identical item. So for example, I’d be more upset about losing $10 than the happiness I’d feel by gaining $10. If I buy a meal and hate it, I’ll likely finish it anyway.

In general, people would rather gamble on a 50% chance of losing $1000 rather than giving up $500. Down $50 at the blackjack table? Odds are most people will want to try to win that back rather than take the loss. Curiously, most people would rather accept a guaranteed $500 rather than accept a 50% chance of making $1000. Irrational? Yup, but extremely predictable.

Loss Aversion is the fancy name for the phenomenon. People prefer avoiding losses to acquiring gains, which drives them to become more-risk tolerant in the face of loss. I think it can help explain how we build up and continue to live with technical debt in software development.

Tech debt is a useful metaphor describing the long term consequences inflicted upon a code base by deferring work. Similar to financial debt, we often choose to avoid doing the “right thing” now in favor of a faster path to the end result. The interest accrued by the debt adds up over time and can cause major problems.

There are lots of reasons that software engineers knowingly take on tech debt – deadlines, lack of knowledge or skills, too much work in progress – the list goes on. Sometimes it is unavoidable, sometimes not. Every project has some level of debt, though.

Paying off accumulated technical debt is where I see the ties into loss aversion. The time spent fixing a hastily implemented data access strategy, for example, is time not spent implementing a cool new feature. There is rarely any directly visible customer value delivered by paying off technical debt. In most people’s eyes, this is a loss of time, opportunity, and resources.

We are irrationally risk-tolerant in the face of this loss. Instead of spending $500 to pay off the debt, we’ll flip the coin, let the problems grow, and take the risk of losing $1000. Who knows, maybe the problems won’t surface for a long time. Maybe never. Maybe tomorrow, though.

So how do we fix this if the human mind is hardwired to avoid losses?

Shift the mindset of technical debt. Knowingly taking on technical debt is a loss, not a gain. We are losing the ability to easily respond to future requirements; we are not gaining a new feature in a shorter time frame. And existing tech debt should be seen as a sunk cost – it’s lost, and it’s better to forget the past.

If we accept the current state rather than treating tech debt as an incurred loss we will be less likely to gamble with the possibility of future losses. And hopefully our minds will start to blast warning sirens as we consider taking on new technical debt in the future.

TDD: Back to Hand Rolled Stubs

0 Comment agile general development

I’m unashamedly an Agile practitioner and self-proclaimed enthusiast. It’s not a perfect way to build software, but I haven’t found anything better. And until I do, this is the approach I’m taking.

Building Quality In

One of the core principles of Agile is the focus of building quality in from the start, not asserting quality after the product is built. At first this is counter-intuitive, right? How can you ensure the quality of something which is in the process of being built? This is where TDD saves the day. Write your tests first – set the expectations of what the code is supposed to do – then write the production code.

When I rewrote Booked for the 2.0 release I did it all test-first. In fact, there are thousands of assertions covering the majority of the code base. This is super important for a dynamic language like PHP where it is very easy to shoot yourself in the foot. Lots of tests = high confidence that my change didn’t break something. Of course, unit tests in a statically typed language are also critical.

The Unit Testing Approach

I use PHPUnit in Booked and it works great. Aside from providing your typical unit testing functionality, PHPUnit also includes test doubles. Mocks and stubs are great for substituting behavior and isolating components in your tests. It’s a technique I rely on heavily and it leads to loosely coupled code and more focused tests.

Martin Fowler has a great writeup comparing mocks, stubs and other substitution patterns.

Death By Over-Specification

When I first starting practicing TDD there weren’t a lot of mocking frameworks or libraries available. We wrote our stubs and mocks by hand and used a plain assertion framework (nUnit in the .NET space). It takes some time to write this code – you’re building new objects to stand in for existing objects. The interfaces need to match, the return types need to match, and in some cases the return object needs to be set up in a certain way. It can be a decent amount of work. My laziness is what led me to mocking frameworks.

Mocking libraries gained a lot of traction around 2006/2007. Mocks, and mocking frameworks in particular, work by setting up expectations and then returning appropriate responses when you exercise the code. This awesome because you can quickly substitute behavior of specific methods without building up a whole object.

Great! Look how fast I’m going!

So Super Fast

One problem with mocks is that they want to know everything about your code. For example, given you pass parameters x, y, z to this function then return this specific result. This is great when you care about how you execute a function – and there are valid reasons to care about that. This is not great when all you want to do is return a canned response and move on. One of the most frequent arguments I hear against unit testing and TDD is that the tests are fragile and difficult to maintain. Over-specification is generally the real cause of these problems.

An Example

Here’s a test pulled straight from the Booked test suite.

public function testBindsDefaultScheduleByMonthWhenNothingSelected()
{
	$this->page = $this->getMock('ICalendarPage');
	$this->repository = $this->getMock('IReservationViewRepository');
	$this->scheduleRepository = $this->getMock('IScheduleRepository');
	$this->calendarFactory = $this->getMock('ICalendarFactory');
	$this->resourceService = $this->getMock('IResourceService');
	$this->subscriptionService = $this->getMock('ICalendarSubscriptionService');
	$this->privacyFilter = new FakePrivacyFilter();

	$this->presenter = new CalendarPresenter(
		$this->page,
		$this->calendarFactory,
		$this->repository,
		$this->scheduleRepository,
		$this->resourceService,
		$this->subscriptionService,
		$this->privacyFilter);

	$showInaccessible = true;
	$this->fakeConfig->SetSectionKey(ConfigSection::SCHEDULE, ConfigKeys::SCHEDULE_SHOW_INACCESSIBLE_RESOURCES, 'true');

	$userId = $this->fakeUser->UserId;
	$defaultScheduleId = 10;
	$userTimezone = "America/New_York";

	$calendarType = CalendarTypes::Month;

	$requestedDay = 4;
	$requestedMonth = 3;
	$requestedYear = 2011;

	$month = new CalendarMonth($requestedMonth, $requestedYear, $userTimezone);

	$startDate = Date::Parse('2011-01-01', 'UTC');
	$endDate = Date::Parse('2011-01-02', 'UTC');
	$summary = 'foo summary';
	$resourceId = 3;
	$fname = 'fname';
	$lname = 'lname';
	$referenceNumber = 'refnum';
	$resourceName = 'resource name';

	$res = new ReservationItemView($referenceNumber, $startDate, $endDate, 'resource name', $resourceId, 1, null, null, $summary, null, $fname, $lname, $userId);

	$r1 = new FakeBookableResource(1, 'dude1');
	$r2 = new FakeBookableResource($resourceId, $resourceName);

	$reservations = array($res);
	$resources = array($r1, $r2);
	/** @var Schedule[] $schedules */
	$schedules = array(new Schedule(1, null, false, 2, null), new Schedule($defaultScheduleId, null, true, 3, null),);

	$this->scheduleRepository
			->expects($this->atLeastOnce())
			->method('GetAll')
			->will($this->returnValue($schedules));

	$this->resourceService
			->expects($this->atLeastOnce())
			->method('GetAllResources')
			->with($this->equalTo($showInaccessible), $this->equalTo($this->fakeUser))
			->will($this->returnValue($resources));

	$this->resourceService
			->expects($this->atLeastOnce())
			->method('GetResourceGroups')
			->with($this->equalTo(null), $this->equalTo($this->fakeUser))
			->will($this->returnValue(new ResourceGroupTree()));

	$this->page
			->expects($this->atLeastOnce())
			->method('GetScheduleId')
			->will($this->returnValue(null));

	$this->page
			->expects($this->atLeastOnce())
			->method('GetResourceId')
			->will($this->returnValue(null));

	$this->repository
			->expects($this->atLeastOnce())
			->method('GetReservationList')
			->with($this->equalTo($month->FirstDay()),
				   $this->equalTo($month->LastDay()->AddDays(1)),
				   $this->equalTo(null), $this->equalTo(null),
				   $this->equalTo(null), $this->equalTo(null))
			->will($this->returnValue($reservations));

	$this->page
			->expects($this->atLeastOnce())
			->method('GetCalendarType')
			->will($this->returnValue($calendarType));

	$this->page
			->expects($this->atLeastOnce())
			->method('GetDay')
			->will($this->returnValue($requestedDay));

	$this->page
			->expects($this->atLeastOnce())
			->method('GetMonth')
			->will($this->returnValue($requestedMonth));

	$this->page
			->expects($this->atLeastOnce())
			->method('GetYear')
			->will($this->returnValue($requestedYear));

	$this->page
			->expects($this->atLeastOnce())
			->method('SetFirstDay')
			->with($this->equalTo($schedules[1]->GetWeekdayStart()));

	$this->calendarFactory
			->expects($this->atLeastOnce())
			->method('Create')
			->with($this->equalTo($calendarType),
				   $this->equalTo($requestedYear), $this->equalTo($requestedMonth), $this->equalTo($requestedDay),
				   $this->equalTo($userTimezone))
			->will($this->returnValue($month));

	$this->page->expects($this->atLeastOnce())->method('BindCalendar')->with($this->equalTo($month));

	$details = new CalendarSubscriptionDetails(true);
	$this->subscriptionService->expects($this->once())->method('ForSchedule')->with($this->equalTo($defaultScheduleId))->will($this->returnValue($details));

	$this->page->expects($this->atLeastOnce())->method('BindSubscription')->with($this->equalTo($details));

	$calendarFilters = new CalendarFilters($schedules, $resources, null, null, new ResourceGroupTree());
	$this->page->expects($this->atLeastOnce())->method('BindFilters')->with($this->equalTo($calendarFilters));

	$this->presenter->PageLoad($this->fakeUser, $userTimezone);

	$actualReservations = $month->Reservations();

	$expectedReservations = CalendarReservation::FromScheduleReservationList($reservations,
																			 $resources,
																			 $this->fakeUser,
																			 $this->privacyFilter);

	$this->assertEquals($expectedReservations, $actualReservations);
}

This is 75 lines of mock setup code! We’re expecting specific parameters and have mock objects being returned all over. Let’s take a look at this same test with stubs.

public function testBindsDefaultScheduleByMonthWhenNothingSelected()
{
	$this->page = new StubCalendarPage();
	$this->repository = new StubReservationViewRepository();
	$this->scheduleRepository = new StubScheduleRepository();
	$this->calendarFactory = new StubCalendarFactory();
	$this->resourceService = new StubResourceService();
	$this->subscriptionService = new StubCalendarSubscriptionService();
	$this->privacyFilter = new FakePrivacyFilter();

	$this->presenter = new CalendarPresenter(
		$this->page,
		$this->calendarFactory,
		$this->repository,
		$this->scheduleRepository,
		$this->resourceService,
		$this->subscriptionService,
		$this->privacyFilter);

	$showInaccessible = true;
	$this->fakeConfig->SetSectionKey(ConfigSection::SCHEDULE, ConfigKeys::SCHEDULE_SHOW_INACCESSIBLE_RESOURCES, 'true');

	$userId = $this->fakeUser->UserId;
	$defaultScheduleId = 10;
	$userTimezone = "America/New_York";

	$calendarType = CalendarTypes::Month;

	$requestedDay = 4;
	$requestedMonth = 3;
	$requestedYear = 2011;

	$month = new CalendarMonth($requestedMonth, $requestedYear, $userTimezone);

	$startDate = Date::Parse('2011-01-01', 'UTC');
	$endDate = Date::Parse('2011-01-02', 'UTC');
	$summary = 'foo summary';
	$resourceId = 3;
	$fname = 'fname';
	$lname = 'lname';
	$referenceNumber = 'refnum';
	$resourceName = 'resource name';

	$res = new ReservationItemView($referenceNumber, $startDate, $endDate, 'resource name', $resourceId, 1, null, null, $summary, null, $fname, $lname, $userId);

	$r1 = new FakeBookableResource(1, 'dude1');
	$r2 = new FakeBookableResource($resourceId, $resourceName);

	$reservations = array($res);
	$resources = array($r1, $r2);
	/** @var Schedule[] $schedules */
	$schedules = array(new Schedule(1, null, false, 2, null), new Schedule($defaultScheduleId, null, true, 3, null),);

	$this->scheduleRepository->SetSchedules($schedules);

	$this->resourceService->SetResources($resources);

	$this->page->SetScheduleId(null);

	$this->page->SetResourceId(null);

	$this->repository->SetReservaions($reservations);

	$this->page->SetCalendarType($calendarType);

	$this->page->SetDate($requestedDay, $requestedMonth, $requestedYear);

	$this->page->SetFirstDayStart($schedules[1]->GetWeekdayStart());

	$this->calendarFactory->SetCalendarMonth($month);

	$details = new CalendarSubscriptionDetails(true);
	$this->subscriptionService->SetDetails($details);

	$this->presenter->PageLoad($this->fakeUser, $userTimezone);

	$actualReservations = $month->Reservations();

	$expectedReservations = CalendarReservation::FromScheduleReservationList($reservations,
																			 $resources,
																			 $this->fakeUser,
																			 $this->privacyFilter);

	$this->assertEquals($expectedReservations, $actualReservations);
	$this->assertEquals($month->FirstDay(), $this->repository->_SearchStart);
	$this->assertEquals($month->LastDay()->AddDays(1), $this->repository->_SearchEnd);
	$this->assertEquals($calendarType, $this->calendarFactory->_CalendarType);
	$this->assertEquals($requestedYear, $this->calendarFactory->_CreateYear);
	$this->assertEquals($requestedMonth, $this->calendarFactory->_CreateMonth);
	$this->assertEquals($requestedDay, $this->calendarFactory->_CreateDay);
	$this->assertEquals($userTimezone, $this->calendarFactory->_CreateTimezone);
	$this->assertEquals($month, $this->page->_BoundMonth);
	$this->assertEquals($details, $this->page->_BoundSubscription);
	$calendarFilters = new CalendarFilters($schedules, $resources, null, null, new ResourceGroupTree());
	$this->assertEquals($calendarFilters, $this->page->_BoundFlters);
	$this->assertEquals($this->subscriptionService, $this->subscriptionService->_ScheduleId);
}

This is still a huge test (that’s a problem for another post). Here the stub setup code gets cut down to about 15 lines. What I also love is that I don’t have expectations set in the mock specification. Any assertions around parameters have been moved down with the rest of the assertions. This is wonderful for a few reasons.

1 – This follows the Arrange-Act-Assert model that helps us in writing clean tests.
2 – Most of the time I don’t care about what parameters I pass to a method.
3 – Mocks will typically not return anything if you don’t pass the right value to a mocked-out function. When you pass the wrong value null is returned, causing a null reference exception in your code. This is very frustrating and difficult to trace, especially when you changed the parameters intentionally.

I don’t care which method is called (most of the time). Often times I don’t care about the data that we’re feeding to a collaborating object, either. I just want to test this specific object’s functionality and assume that everything else is tested elsewhere and works as it should.

The Stub Library

One of the biggest ancillary benefits I’ve found to hand writing my stubs is that I’ve built up a pretty solid library of reusable objects. I have stub objects ready to go for different pages, for reservations, data access and so on. I can pull in an existing stub and continue on with my test. I don’t need to set up the same stub expectation or response over and over. I don’t even have to think about how this collaborating object works. And if I apply an automated refactor to a method signature I don’t have to worry about breaking a ton of tests – the refactoring would apply to the stub object, as well.

Looking Forward

I’ve come full circle on stubs since my early days of TDD. I used to view them as a productivity killer, but my opinion now is that they actually save me time. Sure, there’s a little upfront investment, but the return on that investment is high. Who knows, maybe I’ll venture back into the land of mocks again one day. Until then, I’m happy with my well-tested, easy to maintain production code base.

Software Development is Expensive

4 Comments general development

This is a bit of a rant, so I apologize ahead of time.

Software development is a crazy blend of art, science and luck. I’ve compared it to painting in the past and I still think that’s one of the better comparisons out there. It is difficult to build a software application that solves a problem, addresses technical challenges, and is intuitive and simple to use.

It shouldn’t be a surprise that software development is expensive. And the higher the skill level of the developer, the more well-known the developer, or the more specialized the skill set, the more expensive it will be. Art follows a similar cost pattern. A Picasso is going to cost more than something by a local art student. Fair or not, that’s the world we live in.

Booked is free and open source and will continue to be. I choose to release it free of charge because I enjoy giving back to a community that has given me so much. I also do a fair amount of custom software development for Booked. This development is not free, but I make every attempt to charge a reasonable fee for my expertise and time.

In some ways open source has spoiled us. It can give the perception that software development is easy and cheap. Most software developers – open source or not – are highly skilled experts in their field.

If you have a custom table built by and expert carpenter it will cost more than the particle board table that you pull off the shelf at the local Ikea. If you reach out a developer for custom software development be prepared for it to be costly. After all, you’re getting a solution expertly crafted to solve your problem. I know I’ll pay a premium for that.

Are You Mentoring or Lecturing?

0 Comment general development

Great team leaders raise the skill level of people around them. There are so many ways to do this – team presentations, sharing articles, demonstrations, etc. Mentoring is one of the most personal and powerful ways of developing individuals on your teams. Unfortunately, this term gets tossed around freely without really understanding it.

To the Wikipedia!

Here’s what Wikipedia says:

Mentoring is a process for the informal transmission of knowledge, social capital, and the psychosocial support perceived by the recipient as relevant to work, career, or professional development; mentoring entails informal communication, usually face-to-face and during a sustained period of time, between a person who is perceived to have greater relevant knowledge, wisdom, or experience (the mentor) and a person who is perceived to have less (the protégé)”

Knowledge. Social Capital. Psychosocial Support. Mentoring is not just teaching. It’s not lecturing or demonstrating.

Let’s just be frank

You are not mentoring if you’re: criticizing, doing, judging, scolding, arguing. The goal is not to claim your throne as supreme master of knowledge and make others feel bad. The goal is not to make others do things “your way.”

As a mentor you should be asking questions and guiding others to answers. You’re building skills as well as the inherent desire to apply those skills. You should want this person to become a mentor themselves – to grow others on your team.

Ask yourself this

Are the individuals that you are mentoring only doing what you want because they are scared of you or want to avoid confrontation? If you left today, would the process or behavior cease? If you answered yes to either of these then you’re not mentoring.

Empower others

People want to be great. If you’re in a leadership position – and let’s be honest, we’re all leaders – make the time and emotional investments to help others be great. I guarantee you’ll also grow as part of the experience.

There’s a place for mentoring and there’s a place for lecturing or instructing. Be aware of which role you’re playing.

Update Jan 28 2015
Harvard Business Review has a great article with advice on how to avoid some common mentoring mistakes

Does your manager know how to code?

0 Comment general development

Who cares? Can you manage people?

I used to think that a good manager could come from any background. Hell, I’ve had some incredible managers that have never written a single line of code.

But as a full time manager of software developers I’m starting to think that a background in development does matter. I can honestly empathize with the highs and lows of being a software developer.

I understand my team’s frustration when they hit roadblocks. I share in their excitement when the business loves a new feature that they just spent a ton of time on. I feel the energy when they’re absolutely killing it during a sprint. I’ve been there. I’ve written a lot of software and have experienced the full range of emotions that come along with it.

Wait. Emotion and software development in the same sentence? Are we taking crazy pills? Software development is just a form of engineering, right? Feelings don’t produce a more accurate design or get us to production faster, you know? Obviously, I disagree, but that’s a topic for another post. Let’s just leave it at ‘software developers are humans, too’.

Why does it matter?

Not all developers are cut out to be managers. You know who I’m talking about. We’ve all had that team member who is too brilliant for their own good. The one who can solve problems before you even realize they exist. But they’d rather talk tech than talk about about you as a person. You may get some interesting insights into a new framework from these folks, but you’re not getting a deeper discussion on your career objectives. Teams need to be mentored – they don’t need to be lectured.

It’s incredibly unfortunate that some organizations only provide career advancement through the managerial path. There are a ton of talented people out there who may not fit a ‘management’ profile but can contribute from a leadership perspective. And honestly, I want a team of leaders.

OK, so here’s the big drawback.

Programming is addictive. You get into it and it’s really hard to stop. So when my team is facing a technical challenge – or when they’re exploring some new design or technology – it’s really hard to stay out of it. My instinct is to jump in. To solve problems. But that’s not the role of a manager.

Good managers help teams solve problems. They build their teams skill set. Managers don’t grow great teams by solving problems. They build the problem solving skills within the team. “Teach a man to fish”, right?

The jury’s out.

This isn’t to say that the folks who don’t have a background in software development are bad development managers. These people bring an entirely different – and generally very positive – dynamic to the team.

I know that I still have a lot to learn, but if I was back into full time development and you asked me, I’d tell you that I want a manager that has been ‘in the trenches’. I’d want someone who loves technology. I’d want someone who gets technology. Someone who knows the challenges, the highs, and the lows. Someone who knows that even the seemingly simplest problem may turn out to be an event that turns your hair gray. And by that line of reasoning, I’m becoming more qualified by the day 🙂

Update: November 13, 2014

JD Meier has a great post about building digital skills. He talks about how “hybrid leaders” – those who are skilled in technical and leadership aspects – are the leaders of tomorrow.

Love Your Code

Comments Off general development

Me and Code, we had something special. Like all new relationships, this one started out with the best of intentions. Although, I suppose we both knew it couldn’t last forever.

Looking back I realize that I didn’t support Code the way I should have. The kind of support that could only be fulfilled by unit tests. I got so busy and felt like there just wasn’t enough time. It didn’t take long for things got messy, as things with source code tend to do. Code would be telling me one thing, but doing another. Randomly, Code would throw a NullReferenceException and I’d have no idea why. The API was fragmented, making it difficult to communicate.

I had a feeling this it was my fault. Red, Green, Refactor – I knew the rules but by the end I only saw red. I wanted to help change Code but I couldn’t. A change here caused a problem there. Each fix just made Code angrier with me. Eventually we broke up and I started over. What else could we do?

Now I give my source code all the tests that it can handle. This relationship I’m in now was test-driven, so it’s exactly what we both need. There’s an occasional spat, but the core is strong and it’s easy to get things back on track. It’s been three years since I started up with Code 2 was and things are still going strong.

Happy Valentine’s Day, everyone! Be sure to give your code some love.