Archive | SW process RSS feed for this section

Apathetic feedback

3 May

As I’ve been developing Zed9, and reflecting on what we’re doing at Replicate, I’ve spent a lot of time trying to make sure I’m building the “right” thing.  So what’s right?  One theory is “Build it for yourself”.  As recently discussed by Tim Bray, John Gruber, and 37 Signals, the basic idea is: unless you’re a very strange individual, make a product you use all the time and chances are others will too.  You can focus on what you know, and make something great.  Build for yourself suffers from survivorship bias - it’s a necessary and helpful precondition, but far from sufficient.  There are countless products built for their creator that never made it anywhere.  The very fact that you have the skill set to create a product to solve your needs means that you ARE NOT LIKE the vast majority of your potential user base.  Using yourself as the prototypical customer can lead to dangerous assumptions.  What’s easy for you isn’t for others.  What’s obvious to your user could be totally obscure for you.

In creating any product, there’s always a disconnect.  You, as the product manager, startup founder or whatever, are either trying to solve a product you’ve personally experienced, or one you’ve seen.  Either way, it’s based on your experience.  And as much as we all like to scratch our own itches, selling to yourself is a hard way to make a living.  Many times you’re not creating this product to make a living, it’s an artifact of some other focus.  That’s certainly what happened with Rails, and I believe for Gruber with Markdown as well.  And when that happens, GREAT!  We all love the serendipitous success.  But what happens when you’re setting out from the start to actually make money on your product?  

That’s where the MVP comes in.  MVP stands for “Minimum Viable Product“.  Closely related to agile practices, the idea is put together the real minimum product to get customer feedback, and use that to validate and move forward.  Sometimes, your MVP can be as simple as a slide deck or even just an adword.  If people click and sign up for a waiting list, it’s probably a good sign that they’re interested, and it’s worth pursuing.  Don’t build a 2 month alpha if a prototype will suffice.  Don’t build a prototype if a marketing landing page will suffice.  The key here, of course, is the word “suffice”.  The MVP is all about getting customer feedback.  And apathy is the death of any feedback process.  We know what to do with negative results (try something new!), with positive (do more!), but what about no results?  

Apathy is really the scariest thing that a product owner can experience.  The lack of feedback gives us nothing to hold on to.  We start to breath our own fumes, going in circles.  What are the main causes for apathy? Three stand out: 

  • Not a large enough sample size. We’re aiming for early adopters with any MVP based feedback.  If we’re lucky, they make up 5% of our target population.  Depending on your marketing and engagement practices, this means you might need to kiss 1000 frogs just to get 10 qualified responses.  Let’s take the ad-words example.  Your ads have a 2% CTR, and you expect 10% of the visits (a high number) to translate into actual feedback.  To get 10 actions, that’s 100 CTR, and 5000 impressions. Want 100 user feedback base?  Now you’re talking 50,000 impressions!  Dealing with contacts?  Assume you can get a 15% conversion rate and you’ll still need to talk to 66 people just to get 10.  This is probably one of the biggest issues many startups face.  From day 1, you need to be VERY aggressive about talking to as many people as you can to increase the population base.
  • Didn’t actually make a minimum product. This is probably the first place any engineer will go.  ”Clearly, if we just add feature X and Y, THEN they’ll understand what we’re doing, and give us feedback”.  This is the most dangerous path to go down.  It’s the one where you eventually throw away the MVP, because you’ve never satisfied with the minimum.  It’s one I’m personally struggling with on Zed9 right now in fact.  We’ve launched a minimal product, that at the least gives the user some interesting rudimentary comparative analytics.  It’s different from other offerings, but not earth shatteringly so.  It hints at where I’m taking it.  I’ve had some interest, but not droves beating down my doors.  I keep thinking “if I just add this feature, then I’ll get 100x more people interested”.  This way madness lies.  You need to set clear metrics, and hear from prospects.  Go back to step 1 – not a large enough sample size.
  • Not solving a broad/interesting problem. At some point, you need to call it.  Remember, the point of an MVP is specifically to find out IF IT’S GOING TO WORK.  It MAY NOT.  Hell, it probably won’t.  If you’ve talked to enough people, and still not getting a decent response, it just might be time to move on to the next idea.  If you keep going, you’re not building a product, you’re satisfying a hobby.  A hobby that may well turn into something interesting, but for now a hobby.

For me, I’m redoubling my efforts on increasing the sample size.  Before I spend a few more long nights and weekends adding all these cool features, I’m going to go out into the field, and talk to people.  I’m heading out to some bike and running stores during the weekday, to talk to the sales guys and see what HW is moving, what tools they recommend, and their thoughts on the viability of the product.  I’ll be joining some group rides, checking out the local meetup groups, and just talking to people about their problems today.  I’ll let you know how the next few weeks go!

Sphere: Related Content

Webinar: Using Jira & Greenhopper for Agile Development

4 Nov

After all these posts on agile process and tools, if you’re interested in hearing live how I’m using Jira and Greenhopper, then sign up for this webinar Wednesday.

After introductions from Atlassian and Pyxis, I’ll be talking for 15 min on how we’re using them here, on our live dataset.  I’m talking about why we choose Jira/Greenhopper, how the entire team uses it, and what some of my favorite features are.

Sphere: Related Content

Freedom by unrealistic expectations

8 Oct

We’re at that stage right now where the whole team is clicking.  Every 24 hours I’m seeing improvements that blow my mind away.  The more we get done, the fewer items left for us to do, the more I’m looking back and realize how TOTALLY INSANE my expectations were when we started.

When I first joined Replicate, our founders had already spent a bunch of time on the core databae and inference engine that drives everything we do.  Their up-front investment is what has enabled us to get to where we are today.  The rest beyond the core engine was good, but not what I was hoping for.  After a month of getting to know what’s going on, we emabarked on a massive push to completly overhaul our UI, the workflows, and everything about the product experience.

User experience is incredibly important to me, and I wasn’t happy with our state.  I confidently laid a plan for ripping the entire thing up, changing everything to the very core.  I committed to my boss that we could get this, “no problem”.  I signed up for all sorts of improvements, from new visualizations, new interface paradigms, new toolkits, and more.   Now, as we’re just two weeks away from release, I know I was totally nuts.  Not because we didn’t deliver everything I hoped we would – we did/have/are!  No, I was nuts because looking back, if each and every little thing hadn’t gone just right, we would never have made it here.  If we hadn’t hired an AWESOME UI engineer.  If our lead engineer wasn’t a rock star.  If we hadn’t managed to get an automation engineer and harness to remove the friction.  If the entire team wasn’t flexible enough to deal with massive change daily.  But we did, he is, we did and they are.  And so now, on time and on schedule,  we’re two weeks away from launching a product that I’m really excited about.

We’re a startup.  Odds say, we’re going to fail.  It’s only by having unrealistic expectations in the first place that any of us even can get into this job.  If we don’t keep having unrealistic expectations every day, we’ll get trapped by what we can’t do, and deliver just another piece of software.  IF we can keep being unrealistic, signing up for insane deliveribles every day, then I think we just might make it.

Sphere: Related Content

Living the ultimate online agile workflow

6 Oct

Living the ultimate online agile workflow

Over at my job, we’re just two weeks away from our first release!  I thought this would be a great time to look back a little at our tools and process.

Process

From the day I joined, we’ve been developing via a scrum process.  For those not familiar, the highlights are:

  • Everything you want to do goes onto a prioritized list called the backlog.
  • Write requirements in the form of user stories.  (As a <USER> I can <FUNCTION> so that <REASON>)
  • Anyone can add, only product owner can prioritize.
  • Team agrees on what backlog items they are going to work on during a planning meeting.
  • Create todo list from user stories.
  • Track progress through iteration.
  • Repeat!
We started out with 4 week iterations, with some serious pros and cons.  This gave us plenty of time to get large features in, but we were also finding that EVERYTHING was getting checked in on the last day. Despite the claims to mimize dependencies, I’ve found that there are ALWAYS going to be some there.  We didn’t do a great job identifying and planning for these, so often we had a huge crush at the end as everything was unblocked.  Everyone was busy the entire time, but we were losing sight of the iteration goal and occasionally coding to the todo list, vs the actual goal of the user story.
As part of our sprint review, we talked about some fixes, and identified a T-2 meeting.  Basically, we decided to meet halfway through the iteration, discuss how things are going, and plan out the rest of the iteration.  Make sure dependencies are met, etc.  These were taking 4 hours, almost as long as the sprint planning meeting.  That’s when Steve Ross had the brilliant idea – we’re basically just doing 2 week iterations at that point.
2 week iterations have worked well for us so far.  It’s way more focused.  Our review and planning sessions are shorter, and easier to handle.  Our velocity is still increasing, and no one has started making weird sounds and muttering in a corner… yet.

Tools

Say what you want about pen and paper, but I’m a geek, and love the toys. We use a bunch of tools here:

User stories are input into Jira, either directly or through the Greenhopper plugin.  Once a week, I go through and prioritize the user stories in the backlog with the great drag-and-drop interface Greenhopper provides.  As needed, we have planning sessions in which we break up large user stories into smaller ones, and play planning poker to assign story points.  We use paper cards for planning poker still. :)  Results are all put into greenhopper.
As we go out and talk to customers, we create a confluence wiki page for each customer visit.  We then go through the notes, and link to relevant user stories with a simple user macro (GALAXY-562@jira) that expands out to the full URL of jira.  Trackbacks are turned on, so when an engineer wants to know more about a user story, they can find the notes easily directly linked from the Jira issue.
We do NOT bother creating a Confluence page for each iteration.  Haven’t seen the need so far.
As part of our planning process, each user story is decomposed into it’s tasks.  During planning, we walk around with big post-it paper in a conference room, writing down all the tasks, then marking the hours and time each of us is going to spend. At the end of the planning, one person then types it all into Jira/Greenhopper.  Yes, this is tedious, but it only takes ~1 hour.  We’ve tried having people type their own items in, distributing the problem, but nothing shows up.  We’ve tried skipping the paper, but for brainstorming, nothing beats paper.
As we start development, users drag-and-drop their tasks in the task board view of Greenhopper.  Once a day, they indicate how much time they’ve spent and how much remaining.  Usually takes <5 min per day.  For each user story, we create a branch in subversion.  All development is done in the branch, until the user story is complete.  This does sometimes cause some trunk merging issues, but it’s not that bad, and the benefit of keeping this all separate is well worth it!.
Our product is UI driven.  When we need to figure out what to display to the user, we use this AWESOME Confluence plugin called Balsamiq.  It’s plain and simple the best way to do mockups that I’ve seen yet. All done through a web browser, you drag and drop super quick.  It’s perfect for conveying the thought, without getting bogged down in the stupid details that don’t matter anyway.
Our build is a little strange.  We’re creating an appliance, made up of postgres SQL, python and ruby code. Technically, there’s nothing to build, just pull together.  We created a customer shell script that pulls all this stuff together into a nice tar.gz file automatically.  We have bamboo call this script once a night for trunk, and poll the subversion server every 10 min for each user story.  The build script has a test-harness call out available, for long and short testing.  Our trunk builds get the long testing, user stories the short.  The short test just does a quick selenium test to make sure that everything loads as expected, and there is no crasher bugs.  The long test actually goes through and exercises all the UI components, timing how long everything takes.  All testing outputs junit XML, which bamboo then slurps up automatically.
At the end of the iteration, we delete the user story build plans from bamboo, label the build we’re keeping, and create a branch for our iteration release, just in case.
Overall, this set of tools has been fantastic.  We’ve spent right around $5,000 for all the software.  That’s a significant chunk for a small company like us, especially when there are such great free tools all around, and I’d spend it again in a heartbeat.  The quality of these products is just outstanding.  The support is great from everyone, and Atlassian in particular needs to called out.  I’ve had 4 issues worth submitting as tickets, 3 of which they handled in < 2 hours, and 1 was my fault.
Sphere: Related Content

Accept_360.version++; clean_up_interface();

24 Apr

Accept_360.version++; clean_up_interface();

For the past two years, at Sun and Montavista, I used Accept to manage our requirement process.  Although we’re not using it right now at the new job, I still think that Accept 360 is one of the better requirements management tools I’ve run into out there. Over the past few days, Accept has been rolling out a new update to version 4.6.  I had the chance to sit down with Nils Davis and John Talbott from Accept, who walked me through some of the new features.  

They’ve only bumped the version by .1, this is a fairly major release.  In fact, I kept asking why they didn’t just go to 5.0.  Apparently, they’ve got some great stuff in the hopper for 5.0.  Me, I’d just have called this 5.0, call the next 6, and move along.  But then, I am into version inflation.  Accept has always excelled at that “phase 1” task set.  The ability to cleanly trace requirements from customer input through to implementation is fantastic.  For 4.6, they’ve addressed some of my key issues:

  • A logout button!  this is just downright embarrassing that it took them this long.  Sorry guys, but it is 2008. So now, you don’t need to quit your browser to logout.
  • A whole new L&F skin.  It’s way cleaner.
  • Some real agile support.

The above screenshot shows off the nice new login page.  Notice that logout link in the upper right!  Hooray, finally!  Also notice that they’re starting to bring in some useful information, instead of that lame “click here to launch product” page.  It’s clear that this is just the first step, but one in the right direction.  

Once in the product itself, there’s all sorts of small stuff that just makes a HUGE difference.  For example, they’ve moved the search box out of the tree view on the left and into the header in the upper right.  This box was ALWAYS getting obscured due to size/rendering issues.  There are a number of small and thoughtful changes throughout that on their own aren’t much, but in aggregate make this a much easier product to work through.  Add in the more modern look and feel (again, 2008 guys), and it’s feeling like a real, modern product. 

They’ve also added in some iteration support.  No screenshots of this one.  The summary is you can now create iterations under a release (i1, i2 and i3 are all part of the GA release for example).  you can then assign tasks, requirements, etc to iterations through a nice drag and drop interface.  Perhaps most importantly, you can also stack rank your requirements through drag and drop.  That’s so exciting, it bears repeating: you can stack rank requirements via drag and drop.  I had a project back at Montavista that didn’t use Accept specifically because it lacked this feature.  

Hopefully they’ll get a screencast or two up soon showing off the new product.  In the meantime, give em a call and see if they can show you a demo.  

Lest you think it’s all roses, I did take the time to harp on my favorite issue – one source of truth. I believe the fundamental problem with any requirements system remains how you keep it and your bug tracking system synchronized.  What is the difference between a bug and a requirement anyway? Nothing.  We talked in depth about what the requirements for such an integration might be.  Here’s my take:

 

  • Don’t bother trying to force people to use one system.  Eng will use bug, PM will use accept.  so be it.
  • Don’t bother getting all info into everything.  Links are fine.
  • Make sure there’s some basic sync.  Creating a requirement should auto-create a bug.  Creating a bug should be seen in accept.
  • Just focus on managing the lifecycle.  Don’t worry about the details of the bug.  
I actually think that Rally has done just about a perfect job on this one.  It’s lightweight & useful.  Accept tells me they’re looking into some solutions.  For O’s sake, I hope they come up with them soon. :)
Sphere: Related Content

Using Jira, Confluence & Greenhopper for Agile

22 Apr

Using Jira, Confluence & Greenhopper for Agile

Although it took some time, I’ve decided on the tools.  As the title says, I’m going with Atlassian’s suite, plus a really nifty plugin I’ve come across called Greenhopper.  Here’s the overview:

 

  • Create user stories in Jira.
  • Create wiki page for each release (not iteration, but release).  Link up to Jira dashboard.  Provide details, necessary information, any supporting stuff available.
  • Planning poker to estimate user story size.
  • Prioritize user stories with drag and drop via Greenhopper.
  • Drag user stories into iteration releases via Greenhopper.
  • Create sub-tasks for each user story to track hours.
  • Drag tasks on planning board for open, in progress and done via Greenhopper.
  • Use Greenhopper to generate burn down, velocity, and other fun graphs.
The beauty of greenhopper is it provides a trivial and simple interface into a very powerful bug tool.  You get all the benefits of a shared collaboration tool, without the pain.  Greenhopper lets you edit directly in place, use drag and drop, and basically make it all so nice and easy.

Now, all that said, there are times when paper is just the way to go.  We currently have ~85 user stories, and trying to figure out the ranking really is easier when you can have paper in front of you.  Luckily, it’s easy to do both!  By exporting the issues into excel, and then using mail merge from word, finally printing to Avery postcards, I’ve got a great way to create cards.  You can use my attached user-story-cards word file if you want to create your own.

 

Sphere: Related Content

SW Development tools – Wiki & Bugs

22 Mar

I’ve concluded the tools analysis that I kicked off a few weeks ago. What initially began as a look into requirements management tools quickly expanded into the entire product development lifecycle. Tools can not make up for broken process, so that’s the place to start. As I looked back at what’s worked and not in the past, I realized that the MOST critical success factor has always been alignment. If engineering, product management, sales and the customer aren’t all aligned on what, when, how, and why, we create massive headaches for ourselves. To ensure alignment, we need one source of truth, one place to track everything that’s going on. If we need to keep two system in sync, we lose. If everyone in the company can’t see everything, we lose. If everyone isn’t bought into the process, we lose.

In order to get everyone on the same page, we need to look at the toughest participant. I’m sure this won’t come as a surprise to anyone, but usually that’s engineering. Not only are they the largest team, they also have the strongest opinions. Starting from that vantage, I realized the plan of record has to be kept in a bug database. It’s the only thing engineering will update. In my past lives I’ve many times tried to keep a separate list, one focused on features, and engineering’s DB focused on bugs. But what happens when you classify a bug as a feature enhancement? It’s such a grey area, do you start replicating and moving data? Duplicating it? Either way, it’s a disaster. I’ve gotten into massive fights over this before, and even if I was “right”, is it worth fighting that over and over?Yes, tools like Rally and Accept can interop with Bugzilla or Jira. But if I need to have the bug DB anyway, is the extra effort and cost worth while? For such a small shop as ours, I’m not sure the answer is yes. I’ve used Accept for a few years, and have loved the tool. For traceability and analysis, it’s the perfect tool. I think I may have let those aspects get in the way of focusing on what the requirements themselves are, and how we get alignment in the groups. Frankly, all the tools I looked at in the previous post just seem like total overkill for any <50 (and maybe even <300) person company.

Now clearly, the bug database by itself isn’t sufficient. You need a place for more unstructured data, for planning, notes, etc. For marketing data, sales plans, and just the rest of the company. Obviously, a wiki is the best place to put this. This leaves us needing a great bug DB, and a great Wiki. In a stroke, I’ve thrown out almost all the tools I was looking at before: Accept, Rally, Feature Plan, VersionOne. They all are… complex. I’m sure for a large company, they’d work great, and have their advantages, but for a startup looking at Scrum and frankly trying to save some money, I just couldn’t see the value. So that leaves us with only four options that I could see out there:

  • Fogbugz (with or without external wiki)
  • Twiki + Bugzilla
  • Trac
  • Confluence + Jira

Fogbugz I quickly ruled out. Although it’s got a great UI, and I hear good things about it, it’s also just not the right fit. Some of the additional features were lackluster (the wiki is atrocious, the forum useless compared to phpBB), plus it’s very general (good), and needs to be shoehorned to fit Scrum or other agile processes (bad).

The next three was a much tougher decision. Trac looks great. For an internal focused project, or an open source project, it’s probably the one I’d use. However, we have need for both private and external content, tighter security controls, and some more advanced commercial features that they don’t seem to cover. I REALLY liked Trac, but I didn’t see it fitting into our commercial environment.

Now it’s really hard. I LOVE twiki. Better than Confluence. I like Jira better than Bugzilla. Ultimately, the support & integration that Atlassian is providing, plus the larger suite of products such as Fisheye, Bamboo, etc all pushed me over the edge. Maybe it’s just a dream, but the thought of having a view into CI data, test status, check-ins, all tied and traceable, makes me a tingle.

From a cost perspective, for a small team like ours we’re looking at a $3,600 investment for Jira and Confluence, plus $1,800 for maintenance. We’ll put them in VMs on existing HW, running Fedora. The incremental admin costs for two new machines should be minimal. I’m figuring if we buy into their whole stack, we may wind up spending 10K with them. On the one hand, that ain’t chump change, especially when we could be getting it for “free” with Twiki/Bugzilla or even Trac for internal + Twiki for external wiki needs. I’m counting on the support, the features, and the overall new development will all make this pay off.

I know many other people out there on the internets have gone through similar evaluations. Which way did you go? 6 months or a year later, are you happy about it?

BTW, the photos are from a recent drive I did. They are there just for decoration.

Sphere: Related Content

SW Development Lifecycle Tools Review

19 Feb

What should I build? And who says so? When do they need it? Will we be able to deliver?I’m working with my team and engineers every day to answer these. Although tools will never make up for a terrible process, a good tool can also make things go much much smoother. For 2 years, I’ve been using Accept 360 for requirement tracking. My wife is leading up an evaluation at her company right now, which sparked some more investigation on my part.To look at all the players out there, you need to break the SW development process down. I’d say there are three phases, irrespective of development methodolgy. Regardless if you’re a Scrumer, a XP developer, or a plain old waterfall, it seems like you always have three tasks:

  1. CRM/Customer interface. Collect data from customer, define and justify your requirements with data. This is almost always driven by email, personal interaction, meetings, notes. The tool is here to help the product manager collect their thoughts, organize the data, collaborate with other marketing people, and feed the status back to the field and customer. Very focused on traceability – show me who is asking for what, when and how. Very CRM like.
  2. Project Management. Manage the development, what is in what release. Know when and what you’ll have. A very blury line often between product management, engineering, and gobs of other people. Focus on the deliverables, the specifications, and getting releases out. Heavily used during development by all involved.
  3. Release management. Bugs on released products, feedback into step 1. Close the loop. Collect the feedback from users, community, post-release QA. Feed back in to Phase 1 and continue ahead. I see this as a combination of bug database and Wiki.

I looked at bunch of tools. All except featureplan I’ve looked at in the past month – featureplan I last looked at over a year ago, so much may have changed.

  • Accept 360
    • Phase 1 Focused
    • Web UI, Slowest interaction, though still acceptable
    • Very focused on the requirements gathering phase. Very strong relationship definition.
    • Recreating the CRM – if you use anything else gets frustrating to enter in data duplication.
    • Methodology agnostic. Neither helps nor hurts with Scrum.
    • Weak Phase 2. Not worth using IMO. Too complex, can’t actual inteagrate. Very heavy project management overhead.
    • Seems big-company centric. Doesn’t feel “cool”
    • Doesn’t publish pricing, but it’s >$1000/user/year.
  • Rally Software
    • Phase 2 focused. Plugins with others to satisfy phase 1 and 3.
    • Medium-complex UI. Very usable, fast.
    • All about Agile/Scrum. Uses terminology, flows that force you to use agile.
    • Focused on what they can do – doesn’t try to solve it all
    • Best integration – subversion, eclipse, Visual studio, jira, salesforce, etc.
    • Super cool wiki/community integration with end-to-end salesforce tracking as well.
    • $35/user/month or $420/user/year. Salesforce integration extra.
  • FogBugz
    • Phase 2 & 3 focus
    • No methodology implied, do whatever you want.
    • Integrated wiki appears weak – separate from reset of product. Would personally use mediawiki instead.
    • Very slick UI, very fast, VERY easy to use.
    • $25/user/month
  • VersionOne
    • Waiting to get a demo account still. Demerits to them. Who requires talking on the phone in this day and age to get a demo account?
    • $30/user/month
  • Feature Plan
    • Phase 1 only, but crushes it.
    • Deep requirements planning. Integrates with Rally.
    • Development methodology agnostic.
    • Only windows only client (they’ve got a web UI, didn’t exist when I looked last)
    • All about pragmatic marketing (but then, I am too. :)
    • Darn expensive as I recall.
  • Trac
    • Phase 2 and 3.
    • The dark horse in this group – open source, PITA to install
    • Scrum plugins to make project management in Scrum very reasonable.
    • Good integrated Wiki features
    • Possibly better as a project tracking than a product – need investigation but initial feeling is this works GREAT for open source projects, but may run into issues with wiki and others for real product release lifecycle (i.e. permissions on documentation).
    • “free”

Rally really impressed me. I used it 18 months ago, and was underwhelmed, now it seems like a great tool. I especially like the integration – Salesforce for your CRM (and bugs even), take continuous integration data, and have it all useful from one tool. It’s probably a high investment strategy, but I could see starting small, just trying it out, and growing to make it a core part of the platform.Alternatively, fogbugz looks like a great choice. It’s less of the uber-tool, and more focused on the development. For Scrum management, I’d suggest using good old paper (burn down, planning, etc).The data geek in me says go for Rally. KISS says Fogbugz and paper.

Sphere: Related Content