The art of unshipping

The hallmark of a successful company is shipping fast. I would argue there is a causal relationship. If you are shipping fast, that means you pretty much have to do the following:

• Make small bets: you’re not putting all your eggs in one huge feature that will take you months to build and may fall flat once you ship it.
• Build efficiently: to ship fast means you have to build fast. That means your engineering team isn’t bogged down with unnecessary kludges like multiple rounds of code review and QA.
• Avoid overthinking: you can’t waste time in the realm of analysis paralysis.
• Generating feedback: there is no substitute for friction with reality. Getting customers using (or not using, as we will soon discuss) your product is the best way to learn what is working (so you can double down) and what isn’t (which is the subject of this post).

A consequence of shipping fast is that you will ship lemons. Lemons are features that don’t get used. Lemons mostly fall into two categories:

• Bruised lemons: these were features with some fatal flaw that prevented them from being adopted. That flaw could be a result of engineering (the feature could have had a bad bug), design (it was hard to use or find), or marketing (didn’t make sense to users). Or a combination. The key principle defining this category is that there is some well-defined reason you can point to that explains why it didn’t work. Think Juicero. A $400, Wi-Fi-enabled juice press that was no more effective at making juice than squeezing the pre-packaged fruit with your hands.
• Sour lemons: features that don’t resonate. They probably didn’t address a real customer pain or were so hard to use that the effort to do so didn’t match up with the magnitude of the pain it was intended to solve. Take Google Video, for example. Before Google acquired YouTube, it built its own competitor, Google Video. It focused on “TV quality” content with superior video streaming, but that wasn’t what people wanted. YouTube won by making it quicker and simpler to record and upload videos.

What is unshipping?

Unshipping just means removing or sunsetting a feature from your product. The point of this post is to encourage you to regularly scan for features that are lemons in your product so that you can unship them.

What happens when you ignore bad features?

What do you do with lemons? If you don’t unship poor features and continue to ship fast (which you should), bad things happen.

Firstly, you accumulate tech debt: every lemon has to be considered when shipping new things. If you make a change to your settings, you have to make sure that change is compatible with all of your existing features, including the lemons. When you want to rearrange your menu, you have to account for your lemons. This will slow you down going forward. This can be especially bad when an engineer working on a project that comes into contact with a lemon doesn’t know it’s a lemon. They might think it’s really important and make design decisions that sacrifice simplicity or some other features for the sake of preserving the lemon.

Secondly, you accumulate abstraction debt. Essentially, the more features you have, the harder your product is to describe simply. You will have to account for the lemon somewhere in your docs. Its presence will probably confuse some of your users. By definition, a lemon isn’t adopted. When a new user is learning about a product, learning about a lemon could make them think, “this product isn’t for me because this feature doesn’t seem useful.” Imagine walking into a luggage store to buy luggage and seeing some very weirdly designed handbags. You might think, “ok, this is a store for handbag fashionistas, not people who need an extra carry-on”.

Thirdly, you risk the “broken windows” problem. Essentially, your product will feel busy, chaotic, and cluttered. Just like a person walking down the street would feel less safe in a street with broken windows, your users won’t feel happy in a product with lots of lemons. That means they’re less likely to commit the energy to become a user in the first place.

How do you unship features?

Regularly unshipping is the solution to all of these problems. At Command AI, we unship all the time but make a point to do a formal unship exercise every quarter.

Here’s how we do it.

• First, we list out all lemons. How do you figure out what features are lemons? You need a way to assess usage. A product analytics tool like June or Amplitude helps here. You need to decide how you define “used.” My advice is to define used based on the number of customers using the feature. If you’re B2B, don’t count multiple users in the same company. The goal is to understand whether the feature seems to resonate with your current customer set and future customer set — an unweighted percentage is the best measure of that.
• Classify lemons into the two categories above — bruised lemons and sour lemons. Customer anecdotes are an important input here. Maybe only a few customers use it, but those that do, LOVE it. Why do they love it? What is different about those customers? Answering these questions could lead you to pinpoint a reason why other customers aren’t adopting a feature. If you can fix that reason easily, then the feature is salvageable. If you can’t find a reason or you can’t fix the problem, then the feature is too poorly executed to be saved.
• You now have two lists. Each bruised lemon should have one clear reason associated with it.
• For the bruised lemons, rank the reasons by how easy they are to fix. You might be able to fix them all.
• For the sour lemons, you should probably unship them all. If you need to prioritize, rank them by (1) tech debt and (2) abstraction debt and start with the ones with high scores in both columns.
• Premortem the unship. Which customers will it impact? What needs to be updated? Would it allow something to be refactored? This will help you surface reasons you might not want to unship in the first place.
• Tell your customers you are going to unship the feature (especially any that use it). Important: don’t tell them, “we’re thinking about unshipping the feature.” If you ask a customer that, they will almost always tell you to keep it. Why? The costs of the lemon aren’t salient to them. They probably don’t understand the tech debt impact, and if they use the feature, they don’t care if others don’t. Instead, tell them you will unship it by a certain date. Anyone who truly loves the feature will complain. This might change your decision. And people who don’t use the feature today might make cogent arguments for not unshipping it (”we were just about to use that!”).

Bad reasons to not unship

Warning: at this point in the exercise, you will hear (and maybe you’ll be the one to voice) some reasonable-sounding but bad reasons to delay unshipping a lemon. Here are some particularly common ones:

But we might ship it again later: In that case, you can probably quickly re-add it. If it's not being used in the near term, you can reap the benefits of reducing product debt in the near term by removing it now.

Let's just comment it out: This creates unnecessary clutter. You can always go back and look at the old PR where you introduced the feature.

It will hurt the feelings of the people who built it: This very well could happen. The key to avoiding this is creating a culture where every feature is treated as an experiment. That way, the expectation is that anything you ship could be unshipped, and that unshipping isn’t a failure, but an opportunity to learn.

Someone uses it: This isn't the bar you should accept for your product.

My competitor has it: You shouldn't ignore this. Your competitor may have figured out how to build the feature in a way that drives adoption. Most likely, though, they don't have a culture of unshipping, and their version of the feature isn't used either.

An example of unshipping in Command AI

Let’s walk through an example of a feature we unshipped at Command AI: the keyboard shortcut pane.

Command AI lets companies set up keyboard shortcuts for everyday actions. These keyboard shortcuts can be seen in the Bar widget on the right side of the commands they correspond to. We also used to support showing them another way, in a separate pane.

We did this because (1) we thought it looked slick and (2) it would lead to higher keyboard shortcuts usage by being easier to parse. However, during a recent unship ritual, we saw that usage rates were relatively low. What's more, where the feature was being used, it wasn't having its intended purpose: end users in apps using the pane weren't using keyboard shortcuts any more than apps that weren't. We couldn't understand why this was happening. Our guess was that shortcut pane usage was just cannibalizing the same browse behavior in Command AI.

Then we looked at debt.

• Tech debt: the feature was annoying to maintain because customers wanted fine-grain control over how the pane looked. This meant we constantly added new CSS hooks so they could style it.
• Abstraction debt: the shortcut pane made more sense when Command AI’s main value proposition was enabling power users. But since shipping the shortcut pane, we'd focused more on enabling new and confused users. We still serve power users, but when customers saw this feature, they often said, “oh, my users would never use this.” It supported the idea that our product was primarily for power users, which didn't help conversion.

So we decided to unship it. Since then, we have had several requests to add it back. But we don't see any reason to, because we don't think either of the above issues will have gone away.

Every successful company will ship lemons. It’s inevitable. Not everything you release will be a winner, and that’s ok. What isn’t ok is leaving those lemons to go bad. So make sure to set aside regular time to review all features, identify those not delivering what they were supposed to, and unship them. Turn those lemons into lemonade, and, in the long run, your customers will thank you for it.