How Much UML Modeling Is Right For Your Team Or Project?

This is a question I see asked a number of times in various forums. It is one of those “It Depends” sort of topics, but is one that I think is worth discussing here. In this post, I’ll give you my perspective on how I’ve used it in my projects.

Probably the most important thing you have to understand is, “Why are you using UML at all?” – assuming you are in the first place, or are considering using it in the near term. As the old saying goes, a picture is worth a thousand words. A UML diagram is simply a picture of part of a (potentially) complex system. And sometimes the best way to break down the complexity is through pictures.UML diagrams convey structure or behavior through pictures of classes and their interactions. It supports numerous diagram types to break down the system into various views, which can be combined in order to represent the system from different angles in order to help bring perspective and clarity.

  

Organized in a logical way, the UML diagrams you create help to tell the story of the system you are developing. The $64,000 question is, “Who is the story’s primary audience?”. Generally, the answer to that is the development team that is building the application. However, with the success of Agile, many teams feel that UML is unnecessary, or at best, it’s used in ad hoc ways with very lightweight and high level diagrams – maybe even temporarily on whiteboards during meetings. It’s simply viewed as a means to convey a concept or outline. I certainly agree with the spirit of that. UML can help to speed up the team because it helps to bring understanding and consensus through a shared view of what they’re building. How lightweight or detailed your diagrams are, or even if they are persisted, should be a decision you make based on the team’s appetite and ultimately the overall value add to the team and project. Flexibility is the key.

And no matter what the long term goals are, the short term benefit of an increase in velocity is probably the most valuable takeaway you will realize by using UML.

In order to gauge the right amount of diagraming to provide to the team, I have used the sprint retrospectives to reflect on this with the team and arrive at the right answer. The teams that I’ve worked on have all been comprised of a mixture of experience levels. Typically, the less experienced team members or newer team members gain the most benefit from the diagrams. The more experienced team members still benefit from them, though, as they almost always help to disambiguate the design details. The net effect is that it helps to level the playing field across the team and improves communication and ultimately productivity. I have almost always chosen to persist the diagrams in a modeling tool – even if for no one else but myself for future reference.

In a highly Agile team where practices like TDD are used, the use of UML may likely be perceived as an impediment. You can make a pretty well supported argument that this viewpoint is right. The design should emerge from the creation and iterative refactoring of code and unit tests to flesh out the details. Up front design using UML is the opposite of that approach. Unless you consider it from the perspective of general high-level architectural point of view that is. The UML diagrams you use can describe many higher level aspects of the system like: patterns; guidelines on organizing system components; describing the layers of abstraction;  deployment details; or other high-level details. Let TDD do what it’s best at – creating flexible and resilient designs at a low level of detail.

Most development teams have a certain amount of turnover throughout their lifetime. The reasons for this are myriad, but the bottom line is that you will need to bring new team members up to speed at various points in time throughout the life of your project. If the system that they are working on is reasonably complex, the UML diagrams that you created to convey your design concepts to the original team members can be an excellent way to help the new team members understand what they are working on. I am a firm believer that every team member should get the big picture. They shouldn’t be relegated to some dark corner of the application with little or no visibility into how it all fits together. Use of tools like UML helps to bring everyone up to speed, which is a good thing.

A common argument that I have heard against using UML is that like most documentation it will always be stale when compared to the code. I would say that this is generally true. There are many ways to deal with this, but probably the best way is to just accept that fact and understand that the main use of the tool is to help people understand the system. If the diagrams are too far out of alignment with reality, update them. Most tools support reverse engineering. Use it to refresh the details and update the diagrams. Your future team members will thank you. Just don’t get bogged down in the minutia of sync’ing code with models all the time. That’s one of the quickest killers of usefulness.

In the end, I will also yield with an “It Depends” answer for the original question. Since no two teams and no two projects are exactly alike, I’d say you will have to gauge the right answer based on your current circumstances. For me, I’ve always relied on UML as a means to reducing complexity and organize thought around the structure and design of complex systems. Let your team help you to decide the right answer.

Happy Modeling

Productivity Tip–Add Your FxCop Project to the Solution

I’m very interested in streamlining the workflow that our team uses when developing. That means I need to make the process get out of our way as much as possible. Sometimes, small changes can make a big difference in your overall productivity.

To some, code analysis is a waste of time, but our team sees it as a vital step in ensuring high quality code in the application. Obviously, I don’t want to forego code analysis, but I would like it to be as unobtrusive as possible.

As Martin Fowler says, “if it hurts, do it more often”. Committing changes that break the continuous build for something simple like code analysis violations is a waste of valuable cycles. So we do it earlier and more often. We’ve simply added the .fxcop file to our solution and added all of the project outputs to the project. Now, before committing our changes, we simply run the analysis prior to our commit and eliminate the breaks before they break the build.

This keeps our CruiseControl projects green longer and reduces the churn of having to rebuild to fix something we shouldn’t have committed in the first place.

I suggest that in your team retrospectives you reflect on your daily practices and look for small, quick wins like this. You’d be amazed at how much time throughout your day you can reclaim by eliminating friction points like this.

Using Architecture to Improve Team Velocity

I’ve been following a number of discussions lately on more than one LinkedIn group that I’m a member of that have been focused on an important concern – namely the role of architecture and design in an Agile project. Many people that I’ve spoken with over the last few years are unclear how much or how little time should be spent on this activity, its relative importance to a team, and ultimately the benefit that it brings to a project. The participants in these recent discussions are asking the same questions. In  this post, I will share my perspective on architecture’s role in an Agile team and how it has benefitted us on our current project. First, I’ll focus on three important things that can impact the answer.

Team Composition Impact on Architecture

A recurring theme amongst many of the discussion participants was team composition. More than once it was mentioned that a team of seasoned software developers could forego up front architectural work and “just start coding”. The architecture would then evolve sprint by sprint. Interestingly, no one discussed the scenario of a team composed of a significant percentage of more junior members.

Project Requirements’ Impact on Architecture

What I found most interesting in these discussions was the absence of emphasis on project requirements and the bearing that they may have on the need for architecture. Perhaps it was an oversight or perhaps it’s because there is little or no difference in need based on project requirements. I’m surprised because in my opinion the more complex a system is, the higher the need for a guiding architecture.

Delivery Timeline’s Impact on Architecture

Without question, nearly every team feels an intense level of pressure to deliver more in a shorter period of time with fewer bugs. It’s the universal facet of what we do – bringing business value to a company in the shortest time and in the most cost effective way. A number of participants in these discussions felt that the demand to drive out a product as quickly as possible made the concept of architecture impossible to consider. This feeling was often supported by arguments that the time spent on architecture would negatively impact the ability of the team to focus on the creation of functioning production code.

What’s the Right Answer?

One thing that I’ve found to be universally true is that no two teams and no two projects are exactly alike. And probably the greatest truth I’ve found in software development is that there is no silver bullet that will solve all of your challenges. What works extremely well on one team or for one project may be completely useless in another setting.

With that in mind, however, in my experience there are some recurring themes and enough similarities that I can say with some confidence that architectural activities in a project have proven to be more beneficial than harmful. Perhaps not to the same degree, but certainly the exercise is not a fruitless endeavor. So what about from the perspectives of the three concerns above?

While team composition can certainly affect how a team perceives the degree of need for architecture, my experience has been that even the most seasoned team can benefit from defining architectural guidance for their project. I like to think of it in terms of an orchestra. The most talented musicians need the notes of the musical score to play to. It’s simply a way for them to all stay in tune with one another. While the action of defining architecture is done by the role of “architect”, the person fulfilling that role may alternate within a team of pros. The net result is a team that operates at a higher level of coordination yielding a higher overall velocity.

The need for architecture on a team with junior members is even more pronounced. Without a clear picture of what these team members need to do, the project can and will degrade into a mire of spaghetti and unmaintainable code. Architectural guidance gives these junior team members development goals and a clearer picture of what they are trying to achieve. It makes clearer the constraints that they need to operate within. This has been invaluable as a communication tool for our team, which has a number of junior team members. It helps to frame conversations and eliminates ambiguity. Rework is down substantially since going with a process that involves an architect role.

The solutions that my company develops are not your average run of the mill line of business applications. They are sophisticated custom solutions that bring predictive analytical models to different vertical markets. They challenge us from many angles including data acquisition, cleansing, modeling, and delivery. Scalability, security, and flexibility all contribute to additional complexity. As a result, delivery of these kinds of products requires management of the technical details and the best way to do that is through architecture and design. However, don’t be fooled into thinking that only sophisticated projects require this level of attention. Even basic LOB applications would benefit from architecture and design in order to reduce duplication, improve reuse, and generally improve maintainability.

Our company is no stranger to tight delivery timelines. Our clients need results quickly to improve their bottom line and our solutions are intended to help with that. In the end, this becomes a value proposition where architecture and design is pitted against time to market. A product that doesn’t meet its objectives because it can’t perform well, can’t interoperate well, or is overly expensive to maintain isn’t a solution – it’s an impediment. Many projects forego the effort of defining architecture and design and instead jump right to the implementation phase. While it can get you over the hump in a hurry to deliver a product, the long-term costs of this can be quite extreme. How many projects have had to be rewritten simply because of decisions like this?

Wrapping Up

This has been a long post and if you made it this far, thanks! In conclusion, I argue in favor of applying sound application architecture and design to the projects I work on because I see their benefit. That benefit comes to teams of all types, projects of varying degrees of complexity, and timelines that are either reasonable or tightly compressed. My advice to you? Take the time to define the architecture of your solution, apply sound design principles to the product, and reap the rewards. Those rewards will be overall improved team velocity and a lower defect and rework rate. These will also have the effect of improved morale on the team because they’ll come away with an improved sense of accomplishment and pride in their workmanship.

Reflecting on a Successful Software Release

What goes into a successful software release? I think to really answer that, it would boil down to a really abstract answer – “It depends.” In this blog post, I’ll give you a quick look at what went into making our most recent release a success.

First a little background …

The project that was just delivered was several years in the making. From concept to delivery took a LONG time. The reasons behind this were numerous; some were technical, but the majority were business-related.

The team members that worked on the project in the early days were not the same members that ended up delivering the solution. A couple of hangers-on were there for the whole duration, but largely it was a new team. Even the product owner from the client side was a new person. The team was also a geographically-dispersed multi-national team with language and time zone challenges.

The project’s development methodology changed over time. The process that persisted the longest and the one that was in place at the time of its delivery was a Scrum-hybrid that was chiefly centered on daily stand-ups. It lacked some of the important best practices that successful Scrum practitioners rely on. For example, it wasn’t until the last few weeks that the team had a well-defined and prioritized backlog.

Key Drivers for Success? I’ll give you the top 3 reasons – at least from my point of view, which was as a latecomer to the project.

The background of the project paints a pretty challenging picture for any team that finds itself facing the pressure to deliver a product. And this is certainly true for the team that delivered our product. So what made it possible to pull this off?

#1 Reason – Commitment

The team that delivered the project to the satisfaction of the client was determined to make it happen. Everyone involved knew the stakes and knew that success depended on each of their individual contributions to pull it off. I’ve been involved with projects in the past and almost without exception those that failed to deliver, or at least to deliver on time, were a direct result of a lack of personal commitment from the entire team.

#2 Reason – A Spirited Team Leader

I’m convinced that what brought the team’s commitment into focus and pulled things together at the end was the efforts of a new Scrum master. The team’s newest business analyst was a success catalyst. Her efforts to wrestle the last set of features and defects into a manageable backlog and focus the team’s efforts onto the right areas was a critical piece of the puzzle that helped the team cross the finish line.

#3 Reason – A Little Bit of Luck

Let’s face it, the project background that I gave at the beginning set the stage for another disastrous failure. However, as luck would have it, a number of events came about near the end that made success possible. For example, a new product owner became more engaged near the end and helped to focus the attention on must-haves. The team gained some new members that had experience with delivering results. They pulled together with the original team members and raised the bar for everyone. And lastly, the work of the team was largely usable and had relatively few “big ticket” issues. Had there been some time bombs that were unrealized by the new team, it could have been a very different result.

What was missing that would have made things better?

My top two things on this list are pretty solid in my mind: First – A well-groomed product backlog with priorities set by the product owner. Having that view gives the team a picture of what defines success. Anything short of that and it’s like shooting at a moving target. Second – Team continuity. Churn on a team can be a killer. When people leave they take with them something that can’t be underestimated – knowledge. If that person is a pivotal person on the team it can spell doom. Sometimes that knowledge can be rebuilt – other times not. Reduce your churn to improve your team’s velocity.

So what now? Well our team held a retrospective where a lot of great feedback was gathered and plans were made to put things into place to fix things. We’ll be relying on Scrum best practices to help guide our new process and hopefully will help reduce or eliminate some of the challenges earlier in the cycle.

Good luck on your next project!

On the Path to Continuous Delivery–Part 1

In this post, I’m going to begin describing the way that I’ve been addressing our pain points associated with delivery of our software. This will be the first of a number of posts that delve into the specific ways that we are altering our methodology to be able to build and deploy our software faster and better each iteration.

First, a bit of background…

When you look at the process that was in place when I joined the company, it was somewhat disjointed. Some parts of it were working pretty well, while other parts were a complete mess (like the deployment process). That’s probably not all that different from what the situation is at many companies (assuming they have a process at all). But considering our company uses Scrum and considers itself Agile, it seemed the process wasn’t allowing us to realize the full benefits of our methodology.

The phrase “Continuous Delivery” was new to me until I attended Agile2010 in Orlando last year; however, “continuous integration” was old news. I’d been using CI for quite some time and found it to be an invaluable part of the software development process. I attended Jez Humble’s session on Continuous Delivery. This practice focuses on making your entire product continuously ready for deployment at any time, all the time. That means all parts of your application – application code, database schema, and any data needed by the app to function. A quick poll of the session’s attendees showed a wide variance in their delivery iteration lengths. The least frequent was once a year and the most frequent was once every six hours. Both were extreme, but it made me realize that if you can deliver your working software every six hours, you could certainly do it once a sprint, once a week, or even daily if needed.

Jez’s talk started me thinking more deeply about how continuous integration represents only the first step in the overall process. What we needed was to apply some of the same principles from CI into how we package and deploy our software as well. What we needed was a way to go from requirements to delivery as continuously and quickly as possible. What we needed was more automation and more repeatability.

Martin Fowler also attended Jez’s session and added some commentary. One of the tenets that he spoke of was about bringing pain points forward in your process and doing them often. The idea being that it would force you to address them and smooth them out. For us, that significant pain was in our deployments.So I began from the backend of the process (deployments) and worked my way backward into the development build/test/stage process. This allowed me to tackle our biggest pain first, look for ways to improve it, and from that learn what our earlier parts of the process needed to produce in order to make deployments easier.

We are faced with a number of complications that make our transition difficult. What fun would there be if it was simple? In particular, we have six development teams (not counting the database development group) and two sets of development tools/environments (Visual Studio 2008/2010 and TFS 2008/2010). Up until this point, the database developer group did not use version control and had their own set of tools, so you could really say we had three sets.

Each team has a slightly different methodology, only one using CI, with the others having some similarities in their TFS build project structure. These six teams are building numerous applications on differing sprint schedules with varying degrees of cross-team integration. They are also responsible for maintenance of their applications as well, which the business expects to be delivered intra-sprint, particularly for more critical bug fixes.

As each team completes their various updates, including testing, they funnel all of their deployment requests to a single team (of two people right now) that review, accept, and perform the request to push the update into the production environment. Today, those steps look very different from team to team and from request to request. And to make it worse, a significant portion of it is manual – from file copies to individual SQL script executions one-by-one. This team is on the verge of burn-out as you can imagine.

That pretty well sums up the current state of affairs. In my next post, I’ll describe the automation pieces that have been built and are now beginning to be put in place by the teams to ease the deployment burden.

Just Read “Making Too Much of TDD”

As I read Michael Feather’s points in this blog post, I found myself agreeing so many times that I felt I had to link to this.

http://www.typepad.com/services/trackback/6a00d8341d798c53ef0147e1235b4c970b

I work in a company where the “scientist” in me is constantly challenged and where most in the developer group fall into the engineer group. I think it’s an excellent example of a polarity that exists (strongly) in our company. The business puts extraordinary pressure on the developers to deliver a working solution in the shortest time possible. The natural response is to forego practices that many in the Agile community consider “best practice” in order to just keep up with the pace of change and new features.

I personally get the same sense of satisfaction when patterns just emerge from the iterative process I follow, which is not strictly TDD, but is more or less a hybrid of it. It bothers me that refactoring is not given much attention in our daily development activities, but I am finding myself now questioning many of the beliefs which I held to be incontrovertible (am I being dogmatic?).

I think my most important takeaway from Michael’s post is the unmistakable position that we MUST question our approaches regularly to improve and to seek new approaches that make us better. Out with the old, in with the new. Constantly focus on pragmatism. If doing something doesn’t improve your product or your time to market, drop it.

I also want to better understand his concept “language granularity”. I believe it has an important impact on how we do development here. It touches on a very important business pattern here. Namely, the constant churn our teams find themselves in and the costs associated with the changes that are required in order to satisfy the latest/greatest requirements.

Great read Michael …

When Team Velocity is King

Our team met yesterday to do a walk-through on a project that was developed by a group of contractors for a high-priority project. A significant portion of the project architecture relied on patterns like dependency injection, factories, and the like to gain a high degree of loose coupling. This was motivated by the ever changing requirements for the product owner and the demands that the system be easily extensible.

At various points during the review, some of the team members that had the longest time on the team made comments that the design was too abstract, that “we’d never do a system this way.” When we came across a set of tests that had been commented out, the response was, “Good, we didn’t want to maintain tests anyway.”

Being the new guy on the team, I wanted to understand why they felt that way. The general opinion of the team was that abstraction and unit tests were simply too time consuming to implement and yielded too little value to consider for their applications. This position intrigued me – considering how much adoption and support Agile practices for software engineering have across the industry.

I believe this opinion is rooted in the business’s belief that “better is the enemy of ‘good enough’”. They are more interested in getting applications out quickly, with as few bugs as possible, but when bugs do occur, they are VERY tolerant of them. The cost associated with fixing the bugs, even if they are found in the field by end users, is not seen as a significant reason for being more strict in the development methodologies to prevent them.

Instead, they rely heavily on business analysts acting as QA and end users to ferret out the defects that are most critical and fix them then and there. More esoteric bugs that don’t dramatically affect the usability of the applications are glossed over and may be fixed in the future when time allows (or may not).

All of this is motivated by the belief that “going fast is the single most important requirement for our development teams.”

Velocity is King here – and it’s good to be the King.

What is TDD and BDD and How Do They Relate to One Another?

In this post I will attempt to define TDD and BDD and show how they are similar. I’ll also describe the way that each apply to software development and where I’ve seen shortcomings with how TDD is often applied.

TDD - Test Driven Development/Design

It depends on the age of the information you’re looking at which of the two terms you’ll see used – Development vs. Design. Newer literature will refer to it as Design more often than not. The reason for the change is to emphasize the important software engineering practice that is primarily benefited from its practice.

Can you describe TDD in a sentence?

TDD is the practice of using automated unit tests to drive the creation of classes and methods that satisfy a set of requirements for a software application.

On a team, who does TDD?

TDD is performed by the software developers that are responsible for the delivery of the code that satisfies the requirements of the application. It is not performed by QA or others responsible for quality control of an application.

Why are developers writing these tests? Shouldn’t testers write tests?

Because the primary purpose for TDD isn’t testing per se. It’s strange that it’s called Test Driven Design when its primary purpose isn’t testing. Perhaps it’s a matter of semantics, but in my opinion, the simple unit tests that are created as a part of this process are a byproduct of the effort. The real output from this process is a design that embodies two very important traits: cohesiveness and loose coupling. For information on SOLID design principles, please see: http://www.butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod. This book has an excellent coverage of these principles.

Cohesiveness refers to the degree to which a class’s interface is well organized into single set of related responsibilities. This refers to the ‘S’ and the ‘I’ in the SOLID principles that are widely accepted as good Object-Oriented design principles.

Loosely coupled refers to the degree to which classes that work together to solve a problem are concretely knowledgeable of each other. In other words, how abstract are the relationships. More loosely coupled is desirable. This refers to the ‘L’ and ‘D’ and a bit of ‘O’ of the SOLID principles.

How does TDD fit into a SDLC?

In my opinion, before writing the first TDD test and subsequently any code, developers come together and through a general understanding of the high-level functional and non-functional requirements make a determination of the high-level architecture. They should be able to do this based on past experience with applications that are similar in nature. This decision process includes deciding what technologies will be used (WinForms vs. WPF vs. Mobile; SQL Server; etc.) and what high-level layers exist in the application (UI, Business Logic, Data Access). They should also generally have some ideas about how these layers will interact. For instance, you might determine that because you must support desktop clients, web clients, and automated processing that you wish to separate business logic into a separate layer and make it accessible via a web service. The latter might be skipped to begin with, but I think it’s helpful to form these mental maps early on.

Once the developers have this high-level roadmap, it becomes easier to proceed with work in specific areas, but the next part can be somewhat tricky. It is at this point that the team needs to figure out where to start working. Most TDD examples are very simplistic and in my opinion don’t help much in making this decision. Often, a developer will decide, “I know how I want the database to look so I’ll do the data model and procs and then code up my data access objects.” If that’s where they wish to start, they would create a test fixture for the DAO and then create tests that would drive the new methods that the DAO must provide to satisfy the test. As each test is written, the DAO has logic added to it to satisfy the new test. These tests are often very, very small and simple (that’s a good thing). The developer may try to “guess” all the corner cases that a consumer would hit against the DAO and would create a lot of tests to validate those and subsequently put code into the DAO to handle the cases. As you can imagine, the number of these small unit tests will be many. That’s OK since unit tests are supposed to be fast, atomic, and repeatable. It’s not painful to rerun them many, many times. The end result is that you have a DAO that has a well-defined public API for getting your data and you have a unit test fixture that exercises that logic based on “rules” that the developer established that the DAO was supposed to adhere to.

The above pattern would then repeat itself at the web service layer, the business logic layer, and potentially up into the UI layer. All during this time, you make numerous “assumptions” about how the code will be used by a consumer. As you move higher and higher through the layers of your application, you may be required to tweak classes in lower layers, but that’s OK. You have the benefit of unit tests to help you if your assumptions are wrong and you have to go back and refactor. Each change is supported by a compilation and execution of tests to ensure you didn’t break something along the way. This entire process should be iterative, fast, and focused on small sets of functionality.

However, the above scenario is where I’ve seen TDD fail. Why? Because the view is that the reiteration of changes as you move up the layers is seen to be expensive. It can be costly to go back and touch several layers lower down in the design. You often have to: 1) Identify a missing or incorrect assumption; 2) Identify where to make the change; 3) Change the test(s); 4) Change the code; 5) Recompile; 6) Rerun the tests. If your requirements are constantly in flux or you make lots of assumptions that are wrong, this cycle can be expensive.

Enter BDD…

BDD

Behavior Driven Development/Design

Similar to TDD, the emphasis in this practice is on Design not development. In contrast to TDD, where you often see examples or real-world projects that begin at the bottom of the logical application stack, BDD focuses on “Behavior” aka what the user experiences and sees from the application at the logical “top” of the stack. That’s why I refer to it as a top-down approach while I refer to TDD as a bottom-up approach.

Not surprisingly, it shares a TDD principles: 1) use tests to drive design; 2) use small, fast iterations to identify a requirement, codify that in a test, implement code to make the test pass, repeat. Why is it similar? Because BDD evolved from TDD. I believe many practitioners of TDD realized that the most important aspect of the apps we write is what the user experiences. As a result, the emphasis shifted from the bottom of the stack to the top and BDD was born.

So rather than starting with writing tests for a DAO, you would start with writing tests for a UI-related class that defines the behavior the user wishes to have. Depending on some of the prevalent design patterns for the UI technology you chose, you may have in mind various design patterns that are popular to use with that technology. For instance, Model-View-Controller (MVC) is very popular with web technologies, Model-View-View-Model (MVVM) is popular with WPF, and Model-View-Presenter is often applied for WinForms. Strictly speaking, when you first begin writing your UI tests and supporting code, it’s too early to choose one design pattern over another, but I believe it’s useful to have them in mind as you
code to see where the design is leading you. But most importantly, let the tests and the resulting behavior drive which pattern is the winner.

So why is BDD better than TDD?

Well, I believe that they are very closely related and in reality BDD is nothing more than a refinement in how to approach TDD. In my opinion, BDD is superior because it focuses on the most important aspects of the system you’re building – the thing the user experiences. Secondly, while I don’t have empirical evidence at hand to prove this, I believe it reduces the churn that is often experienced when you build systems from the bottom up. As long as you design and code in small chunks and focus on YAGNI (you ain’t gonna need it) and let the top-level classes drive the requirements for the lower-level classes then the amount of churn is less. Most importantly, you don’t need to make ASSUMPTIONS about what is needed. Your top-level classes as consumers TELL the lower-level classes what is needed and you code that and nothing else. YAGNI wins.

Where do Business Analysts fit into this process?

The emergence of DSL (Domain Specific Language) and tooling to support them have opened a number of possibilities. It allows the Business Analyst (or subject matter expert SME) to define the specifications for how an application should behave in a language they are familiar with. The tooling then “translates” that to executable code and with the help of the developer the required BDD unit tests are born thus driving the development of the app. This is often referred to as “executable specifications”.

One example of a tool like this is SpecFlow. This tool is a .NET tool that supports creating executable specs by integrating with numerous supported unit test frameworks like NUnit, xUnit, etc. The SME can write specifications in English sentences that conform to a predetermined structure that outlines pre- and post-conditions for each “test” and the tool converts those into actual unit tests, which the developer then implements.

@bradwilson of Microsoft did a demo of this tool at #agile2010 and showed how he was able to integrate the resulting test fixtures with a web-automation tool to drive the web UI of a project. The end result was a set of specifications written in English backed by executable tests written in the developers language that drove the creation of the web UI the end user wanted.

I hope this has helped to clarify the differences between TDD, BDD, and how it might apply to your development initiatives.

/imapcgeek

Agile 2010 - Day 1 - Evolutionary Web Development with ASP.NET MVC (Part 1)

Presented by Brad Wilson, Senior Developer Microsoft ASP.NET MVC Team

Part 1 – TDD

The first session for me of the Agile 2010 Conference in Orlando was the “Evolutionary Web Development with ASP.NET MVC” given by Brad Wilson (@bradwilson). The first 90 minutes of the three-hour session was dedicated to Test Driven Development (TDD) basics. Brad and his sidekick Scott not only demonstrated the fundamentals of how testing drives design, but gave a good example of how paired programming works. For me, the jury is still out on the effectiveness of pairing, but then I’ve never been in a team that it was encouraged or embraced so that it could sway me.

Although TDD isn’t practiced at my current company, I do use TDD in my personal programming projects and it was a good reminder for me of the fundamentals. Repetition is what helps to keep our skills fresh

A few key points from the first half that resonated with me were:

1) When developing, think in terms of component development that facilitates isolation making it possible to test your units in isolation

2) Use your tests to drive public API’s – tests should also only exercise publicly visible behavior

3) Your tests should be small, simple, and fast – makes it easier to run them over and over again. If anything, I’d add that the tests should also be repeatable though that wasn’t discussed (maybe it was just that obvious).

4) The basic workflow in TDD is – Green –> Red –> Green. Although, I’ve often seen the workflow start with Red where my tests initially fail until I provide the requisite implementation to make them pass. In fact, re-reading James Newkirk’s book on TDD starts off with the Red->Green->Refactor workflow.

5) Refactor, refactor, refactor – if you aren’t regularly refactoring, your code probably (almost surely) has a lot of smells. The beauty of the unit tests is that it provides the safety net and gives confidence that while refactoring you aren’t breaking the contract that you promised to provide in your implementation.

6) Replace your dependencies with “test doubles” (mock objects) – Brad’s demo touched lightly on Moq (http://code.google.com/p/moq/), which I hadn’t used before. It looks like a good framework and I’ll have to experiment with it myself to see, but having used NMock, and others, it appears to address some of their shortcomings (like mocking classes not just interfaces).

7) TDD is more about design than it is about testing. The conscious act of creating tests that define the specifications for the public API is a radical departure for anyone that has never tried TDD. It can represent quite a leap in style and thought process, but one that can yield excellent results. Most people that aren’t practicing TDD say they believe that the process would slow them down. I believe (though I have no hard evidence to support my argument) that it’s quite the opposite. If nothing else, it certainly yields a cleaner design that makes maintainability higher reducing time over the long haul.

8) A side effect of #7 is that the unit tests aren’t the goal; they’re an artifact of the process. This can be a distraction for some – myself included. Until you realize that the unit tests are a means to an end, you may lose focus on what they are supposed to be doing for you: a.) driving your design and b.) providing a safety net for when you refactor (you want to refactor don’t you?).

9) Refactoring should focus on small, incremental changes. In my second session of the day (which I’ll cover in another post), Joshua Kerievsky (pretty sure it was him) spoke up about the importance of refactoring to patterns to improve overall design.

I came away with a few little catchy tidbits as well.

1) xUnit – uses the word “fact” to avoid the stigma that developers have against testing. I began experimenting with xUnit after the conference and find it equally usable to NUnit, but it also emphasizes some best practices that NUnit missed (as James Newkirk’s blog pointed out, xUnit was developed after some experience with “programmer testing”).

2) “Do the simplest thing that can make it work.”

3) TDD drives a very small incremental development style. Add test, watch it fail, add code, watch it pass.

4) In the real world you would write the acceptance tests first – BDD then TDD. The acceptance tests will fail for quite some time.

5) TDD tests are the documentation for other developers. BDD tests are the documentation for the business.

6) @bradwilson, “TDD is like the scientific method.” You form a hypothesis, write a test to represent it, and then see if your assumption is correct.

7) The basic test structure should be similar to this:

// Arrange
Set up the conditions for the test

// Act
DO whatever work is required

// Assert
Validate that the results are what you expected

See (http://c2.com/cgi/wiki?ArrangeActAssert)

That pretty much sums up the first part of that session. In closing, the important takeaways were using TDD and refactoring to drive your designs and improve code readability and overall health. Though this session talked about using mock objects as a means to loose coupling, a later session with Arlo Belshee discussed other design patterns that can also help to achieve loose coupling, but that don’t necessarily have the same overhead that mocks bring. I’ll cover more on that in a future post.

Where Scrum Falls Short and Software Engineering Has to Pick Up

For any company that adopts Scrum, there will be an initial period where productivity dips a bit while everyone understands the process, but then within a very short time their productivity will spike higher. Ayende Rahien in a recent post gets into a some excellent detail into where Scrum as a “Product” development process falls short when developing software.

The reasons for this are pretty obvious when you look at Scrum side-by-side with what we know to be best practices for software engineering. It isn’t until you understand what each bring to the table that you can see their shortcomings and then blend the two together to create a process that brings the strengths of both together to bear on the problem.

Scrum’s short iterations, frequent feedback, and quick turn-around are clear winners. It shortens the feedback cycle and makes staying focused on the highest priorities for the Product Owner much easier. It also allows for changing direction when requirements change. However, as Ayende points out, there is nothing in this that is software-specific. That’s where software engineering practices like Test Driven Development, Continuous Integration, and Refactoring come into play. These work together to ensure that the “product” is built using best practices that focus on the actual product and the idiosyncrasies of its development.

It is critical that these best practices be interwoven into your software development process – especially an Agile process like Scrum. Because without them, you are missing out on the aspects that emphasize quality. Over time, as quality declines, so too will your velocity. Ayende describes this:

“You hit the Scrum wall when you adopt Scrum and everything goes well, then, after a few Sprints things don’t work any more - to use an English expression, they go pear shaped. You can’t keep your commitments, you can’t release software, your customers get annoyed and angry, it looks like Scrum is broken.”

How many times have you gotten a few sprints into your project, and then had to put on the brakes to introduce a “go back and fix stuff” sprint? For our team, it happened a lot early on before we adopted those best practices. After adopting them, we rarely had to go back to code due to quality issues; rather, we were more likely to have to revisit code due to changes in requirements or additional features being added. Clearly this is much better – we’re now focusing on business drivers and less on technical debt.

Bottom line – don’t forget to roll into your process the details for following the software engineering best practices. Let Scrum manage what it does well – requirements and deliverables. When you include the best of each of these your product will benefit greatly.

del.icio.us Tags: Agile,Scrum,TDD,ContinuousIntegration,Refactor