Jason was on WPwatercooler again. Summary.

The WPwatercooler is a great show, and I was on it again this past Monday with my BWAwWP co-author Brian Messenlehner. I may make it a habit to get on that show. And I may make it a habit to follow up each show with a summary of what went down.

The show topic was How do I make sure I don’t have to re-do my website just months after it’s done?

The consensus was that you can’t make a website that won’t require updates. But you can do a few things to prepare for future updates and minimize their impact:

  • Make sure you are using a good developer. Good developers will do things “the WordPress way” (e.g. using hooks and filters instead of hacking plugins), which will be less likely to be broken by plugin and core updates.
  • Make sure you are using a good designer. Good designers will simplify things and use proven methods that are less likely to be broken by software and browser updates.
  • Make sure your website has a purpose other than looking pretty. One year after your website goes live, it will probably be out of style… at least a little bit. But if you built a website that encourages people to call your sales phone number, it will still do that no matter what the design trends are.
  • Make sure you have a maintenance plan. In addition to having someone on call to perform WordPress upgrades, have a developer on retainer or on call to fix things if anything goes wrong.

We need to do a better job of communicating with our clients about what role maintenance will play in their website. There were a few analogies thrown around on the show (websites are like babies?). My favorite was to think about your website like a car bought on lease (get regular oil changes and be ready to trade it in every 2-3 years) instead of a used car you buy and run into the ground.

Thoughts on GPL WordPress Plugins and Themes

This was originally posted as a comment on a Reddit thread of all things, but kind of too good to stay there.

WordPress is distributed under the GPLv2 license. The basic gist of that license is that you can do whatever you want with the code, but if you distribute any altered code, that code must be distributed with the GPL license as well. Meaning anyone you sell your code to can do whatever they want to do with it. This makes things like “one site licenses” and other restrictions unbinding or outright illegal.

If you do a consulting project for someone, the GPL license doesn’t come into play. Only if you want to sell or otherwise distribute your plugin on your website, etc, will your code need to carry the GPL or a GPL-compatible license.

First, what everyone agrees with is that if you copy and paste someone’s GPL code, even if you tweak it a bit, you are inheriting that code and the GPL will apply to it.

But let’s say you built your plugin from the ground up and it doesn’t borrow any code from GPL code. (Which is actually hard to do.) Most would say that the GPL will apply to your theme or plugin anyway since it relies on WordPress to run. The most cogent argument is by Mark Jaquith here[1] . Part of Mark’s argument:

Note that WordPress theme PHP files are not “templates” or “documents” in the way that most people think of those words (though the word “template” is sometimes used, it is not strictly accurate). They are PHP script files that are parsed and run on the same exact level and by the same PHP process as all the core WordPress files.

The particulars of the GPL would make exceptions if the theme was actually being loaded and parsed by the WordPress code, but it’s not. It’s being executed in tandem. Your plugin is really like a patch for WordPress in a nice little package vs. a separate application run by WordPress.

Some people might disagree with Mark and the rest of us, but for the most part their argument is “but I don’t wanna use the GPL, wah!” (Anyone is free to reply with a real argument if you are in this camp.)

There is another thing called a “split license” where you license the .php (and maybe some .js code) under the GPL and license static assets like CSS, images, .html files, and some .js files under a different (more restrictive) license. This way people can’t just redistribute your theme as a whole without replacing the non-GPL pieces first… kind of an extra little hindrance.

Most will agree that this is legal for you to do with your plugin or theme under the GPL. However, some organizations will have stricter rules, requiring a plugin or theme to be “100% GPL” (meaning all files included in the package) in order to be listed/etc. Most notably themes and plugins hosted at WordPress.org must be 100% GPL. Also, WordPress won’t recognize or work with designers and developers who don’t release stuff that is 100% GPL. So, for example, as soon as Woo Themes (a big theme developer) made all of their themes 100% GPL instead of split license, they got a big ol’ link on the WordPress.org homepage. And recently a designer who sold a split license theme was banned from speaking at a WordCamp.

The idea behind requiring that themes and plugins be 100% GPL is that making them split license betrays the intent of the GPL, which is to make it easier for people to do other things with your code, improve it, and generally make the world a better place.

The intent of the GPL is NOT to keep you from making money. Even though there are companies (including mine) selling themes and plugins on their site, this doesn’t mean those plugins are not GPL. You can still charge for a GPL plugin even though someone could buy it and turn around and give it away for free.

There are several reasons people will pay for a plugin or theme even though they might be able to find it for free elsewhere: * They want to get it from “the source” * They are worried about free versions containing malware or other nasties * They want support from who they are buying from (Paid Memberships Pro) * They want to receive future updates automatically (Gravity Forms) * The plugin relies on a 3rd party service (VaultPress)

Now there are basically two camps of people selling GPL themes on their sites.

Camp 1 embraces the GPL fully and will have GPL stickers in their footer and mention it whenever they get a chance. Camp 1 is more likely to give away versions of their code for free (maybe a lite version, or one minus addons, or even the full code) and use other business models to make money.

Camp 2 will admit that the plugin is GPL, but you’ll have to dig around to figure it out. They won’t mention it on their website at all. They might not even include a license.txt or similar comment in their code, which they are supposed to. They are basically reluctantly GPL and hope they can hide that fact to avoid people trying to get their stuff for free. But if you confront them about it, they will admit that their code is GPL to avoid the legal and community ramifications.

I’m in camp 1. I’m still not sure if it’s best for my bottom line. However, I’ll leave with this:

If you are going to work with WordPress and try to make money off of it. It makes sense to be in alignment with the people running the show, namely Matt Mullenweg. WordPress is a community project, but Matt and the WordPress foundation are all about 100% GPL and definitely against non-GPL themes and plugins. You probably should be too. If you don’t want to be, you should probably look into other code bases with different licenses and different people running them.

Update: The best argument I’ve found for the position that WP Plugins and Themes are NOT “derivative” works is by Chip Bennett here. Give it a read. He cites copyright cases and applies their decisions to the question of WP Themes and Plugins. If he sways you, it’s possible that Mark Jaquith’s comment will convince you back to the other side of the argument. 🙂 It’s a very interesting legal question.

Continuing the WPWatercooler Discussion on Rapid WP Core Development

In his 2013 State of the Word Speech, Matt Mullenweg shared his plans to release two major versions of WordPress later this year in addition to 3.6 coming out this week and to generally increase the rate of WordPress updates. We discussed this a bit on the WPWatercooler today, but there wasn’t really time to get into things.

The discussion (argument?) went something like this: (paraphrasing here)

Scott Bolinger: The WP survey reported back that the two worst things about WordPress were updates and plugins. And Matt’s State of the Word Speech was basically a call for more updates and plugins.

Chris Lema: It’s really annoying and costs a lot of money to retrain people when WordPress is updated, especially in the enterprise.

Jason Coleman: You’re whining.

Chris Lema: Of course updates are easy for you, you bleed blue WordPress blood. You need to be able to relate to end users who are confused by UI updates, etc.

And that’s about where we left it before moving on to the next topic. If I had time, I would have continued on like this…

First, I do understand the end user’s perspective. I have to do these updates for clients, answer the support calls, and sometimes retrain them. I used to be in the Chris Lema camp and thought WordPress was needlessly updating their UI too fast. I mean, I thought the change in version 3.3 to hide submenus in the dashboard behind flyouts was a big big deal. But that’s nothing compared to the potential change between the current version of the WP dashboard and this mockup of how things might look under the new “MP6” dashboard UI.

mp6_screenshot mp6_screenshot2

I’ve since changed my mind. Here are a few arguments for faster WordPress updates.

  1. What’s the alternative? Slow down? Have fewer updates? I don’t think Chris’ enterprise clients would like that option. WordPress needs to iterate to stay relevant. If WordPress gets stale, we’ll stop using it to build cool stuff.
  2. Things aren’t even changing that fast. That flyout menu feature was added in version 3.3 back in December 2011. Since then, the media library update was the only big UI change which happened earlier this year. An MP6-like update to the WordPress UI possibly (possibly) happening by the end of this year will be a big change, but then so is the jump from Microsoft Office 2010 to Office 2013. Enterprise should be ready for these kinds of things.
  3. What about new users? Sure existing users are disrupted when the UI updates. But if the changes are meant to make WordPress easier to use, it will make training new clients easier. I use a similar argument all the time with clients: sometimes you can’t worry about the few existing users you have and need to instead focus on the many future users you want to have.
  4. Finally, faster updates should encourage some changes that will make things easier for enterprise users. Things like automatic minor version updates or a better decoupling of the admin UI to the underlying WordPress framework are a bigger focus of WP development right now. I think the updates in these areas will make up for the inconvenience of a larger number of core updates.

I really mean for this to be a continuation of the discussion we started on the show. So I look forward to feedback and counterarguments in the comments below. Why am I wrong?

Update: Chris Lema has posted a reply on his blog.

Why We Changed The Domain of Our Book’s Site to BWAwWP.com

(Originally posted to my Google+ account.)

Just a quick post here RE why we decided (without hearing explicitly from the WordPress foundation) to change the domain of our book’s site frombuildingwebappswithwordpress.com to bwawwp.com.

So first, the original domain we chose is the full book title. It’s a pain in the butt to type all that, but we liked that the domain matched the book title exactly so readers wouldn’t get confused since there are and will be many similarly worded WordPress and app development books.

John Parris (@john_parris) and others quickly pointed out that the WordPress foundation, on their website, asks “Please do not use WordPress or WordCamp as part of a domain name.” http://wordpressfoundation.org/trademark-policy/

Many companies and orgs protecting trademarks ask the same thing and actively work to keep people from using their trademarked name in a domain name.

We were aware that the Foundation had this policy. However, it is my understanding that we have a case under “nominative fair use” to use the full WordPress term in our domain name.

The Wikipedia page explains this pretty well.

Another good description and overview of a test case where a website was able to use the LEXUS brand in their domain name can be found here: http://thompsonhall.com/nominative-fair-use-allows-trademark-used-on-website-domain-name/#nominative-fair-use

So we originally went with the full domain name. After launching (very quickly) we realized that even if we weren’t going to ask permission, we should at least inform the WordPress Foundation of the domain. So I submitted a note through their contact form.

We’re all busy at WordCamp San Francisco, so I never heard back from them, but I am willing to bet that they don’t want us to use the mark in our domain name even if we have a case for it. (Maybe they’ll get back and say otherwise.) My thinking is that their thinking would be that having our site in the wild makes it a bit harder for them to ask other sites to switch domains (but Jason can do it!), etc. And they might even think that nominative use doesn’t apply. And their lawyers are probably better than our lawyers. (Hint: we don’t have lawyers.)

So expecting a no, and potentially some friction with the foundation and software community that we are trying to work with and promote, we decided to just change the name anyway. The full book title will redirect to bwawwp.comanyway, and Bwawwp! is pretty fun to say.

So that’s why we originally used WordPress in our domain name and quickly changed course without hearing directly (yet) from the WordPress Foundation.

Mistake #3: Starting with Design

I’ll focus on WordPress sites and themes here, but this advice applies to any website built using an existing theme as a framework or just for inspiration.

I’m a big fan of using existing premium themes for WordPress sites. There are so many quality themes available now that you should be able to find something that fulfills 80% of the design and layout needs for your site. Tweak the colors and layout a bit, and only web designers will be able to tell that you didn’t design your website from scratch. (And only web designers will care.)

However, there is a mistake that nearly every client we use a premium theme for makes. They fall in love with the demo site and THEN think of things to put in each little widget area of the site.

That is, they start with design, then they supply the copy for the site.

It really should be the other way around: Write your Copy → Find a design for your copy.

Doing something like “mobile first website design” helps with this. Maybe we need “email first website design” where instead of a website, you write an email to explain your company (keep in mind your goal).

Start with words. Then build a website around those words.

The thing is all of those widgets may look nice, but they’ll detract from your #1 goal.

The best example of filling a design rather than using a design is with sliders. Sliders are cool. They work for a lot of site. But for most sites, they are just an excuse to put off figuring out your #1 goal and enter EVERY thing you want to say.

If slide 1 is more important than slide 2… then you shouldn’t have a slider. Just show slide 1 and keep it there.

Mistake #2: Playing Designer

Unless your business is website design or something else in the arts, you don’t need a beautiful website. A nice looking website is a bonus, but make sure you’re working towards a functioning website instead of something that will look great printed out and framed on your wall.

Oftentimes when going over design mockups or newly updated websites, you’ll find yourself leaning back in your chair and staring at your homepage for a minute or two taking it all in.

Stop it! No one browses the web this way.

If you think of your website as a work of art, you surely will find little things here and there that might be smaller or larger or a little bit to the right. Resist the urge to do this.

If you know the primary goal of your website (see post #1), make sure the design focuses on that goal. Focus your design feedback on how well the design enables sales, mailing list sign ups, contact requests, etc.

As for website design, hire an experienced web designer at a decent rate and trust their instincts for what looks good.

If you hassle your designer with a lot of feedback on what “looks good”, they are going to shut down and move into “code monkey” mode where they just code up whatever requests you have. Unless you are paying bottom dollar (in which case you get what you pay for) you are wasting money by paying designer rates for code monkey work.

More importantly, micro managing a code monkey will not get you as good of a website as one where you control the vision via a strong primary goal, and an experienced designer controls the particulars of the site’s look and feel.

View all posts in this series here.

Mistake #1: Lack of Goals

One of the biggest mistakes made by website owners is to have a website that is totally detached from any business activity.

Maybe your website is beautiful. Maybe it has tons of information about your business. But what is the goal of your website?

What do you want people who visit your website to do?

If you haven’t thought about this question yet, do it. Some good answers are:

  • Buy my product.
  • Sign up for my mailing list.
  • Call me for an appointment.
  • Follow me on social media.
  • Fill out a contact form.

Mistake #1 has a corollary: Too Many Goals.

When asked what the goal of your website is, you might have answered 2-5 things. Great! Throw away items 2 through 5. What was your first answer? Do that. On your homepage.

View all posts in this series here.

Making QR Codes with Google URL Shortener and Google Charts

One way to make a QR codes to use the Google URL shortener.
  1. Goto http://goo.gl/
  2. Enter a URL
  3. Click shorten
  4. Click “details”
  5. Note/save the QR code

If you dig deeper, you find that you can click on the QR code there, which will take you to a Google Charts link. For example, here is the Google Charts link for a link I made Paid Membership Pro’s page in the WordPress repository:


Hmmm, what if we just change that URL at the end of the query? Will it generate a new QR code for me? For a non-goo.gl URL? Yes!

This link works too, and doesn’t go through Goo.gl:

Some people will want to use the shortened code since Google adds some value by showing link throughs, etc. Some people may like that their own URL shows up in the QR reading for a moment and choose to roll their own code using the Google Charts API there.

Have fun making those QR codes!


Seeking Programming, Design, and Marketing Interns for 2012

A true internship benefits the intern more than the employer. We’re looking for motivated people to take part in development, design, and marketing activities at Stranger Studios. Interns will be involved in real projects and given the education, resources, and tools to do productive work for a cutting edge web development company. Interns will be groomed to take on a full time position at a company like Stranger Studios or to do freelance work in their field.

The three positions we are hiring interns for are WordPress Developer, WordPress Designer, and Internet Marketing Specialist.

All positions are full time, 40 hours per week, for a 3-month period beginning February 2012.

All positions are unpaid. Although some payment may be possible.

All work will be done remotely. Interns within driving distance of Reading, PA will meet with us onsite or offsite on a biweekly basis.

We are interested in recent college graduates or candidates between jobs looking to gain experience and build their resumes. Applicants who are still in school may also be considered.

WordPress Developer

Help with the development of our various open source and for-pay WordPress plugins and themes. Assist with customizations for website clients. Learn how to bend WordPress to do anything. Learn how to find, sell, deliver, and profit from website development projects.

Candidates should either be strong programmers with an interest in learning how to develop with WordPress or be familiar with WordPress setup and management with a desire to learn development.

WordPress Designer

Setup and customize WordPress installs for real clients. Tweak WordPress theme designs and develop new themes from scratch. Learn UI and UX best practices for website designs. Learn how to work with clients to deliver excellent websites that demand premium compensation.

Candidates should have some HTML/CSS experience or a strong desire to learn. Javascript or PHP experience a plus. Should have strong design skills. Experience with Photoshop and Illustrator a plus.

Internet Marketing Specialist

Generate and execute a marketing strategy for our WordPress membership plugin Paid Memberships Pro. Help design the email marketing integration and affiliate features for Paid Memberships Pro. Manage a real advertising budget. Learn how to make money online.

Candidates should be comfortable internet citizens with an interest in social media and business. Experience with WordPress a plus.

How to Apply

Please email me at jason@strangerstudios.com or via our contact form. Note the position you are applying for and provide a way to get in contact with you. Briefly state why you believe you are qualified and provide any information you think would be useful.

We will do our best to respond to all candidates and will follow up with qualified candidates to schedule a phone interview and further screening.

Here are some of the exciting things going on at Stranger Studios that you can take part in.

  • Our memberships software Paid Memberships Pro launched last fall and has been downloaded over 5000 times.
  • We have over 40 paying customers for our Paid Memberships Pro support package.
  • Last year we launched over 30 websites for our clients.
  • Our site WineLog.net has grown to over 33k users and an email list 27k strong.

Don’t Call Your Customer a Dick

Yesterday Adriaan Pienaar, a.k.a. “Adii Rockstar” of Woo Themes posted on his blog about a tough client asking, Am I being irrational? He later followed up on Twitter, calling this customer a “dick client”.


Now I’ve definitely had similar conversations among friendly company in private, but I would never post something like this to my blog or Twitter. I feel bad even propagating the story more, but I’ve seen some similarly toned tweets go out by developers and consultants and I want to (1) talk about why this might be a bad idea and (2) find out why most people seem to be okay with these kinds of expressions.

Judging by the comments on the blog and Twitter, the WordPress community has mostly got Adii’s back on this one. However I think in this situation, while Adii has some good points and understandable “beef” with this client, he is being a bit irrational.

The back story.

You should read his blog post for the proper context and to read Adii’s and others’ comments. I’m going to paste what Adii wrote of the exchange here, even though it’s about 90% of the entire post, because it’s short and I want to be clear about what I’m referencing. Adii writes (emphasis mine):

  1. Customer notes to us that he is struggling to achieve something with our product.
  2. We explain that this is currently a limitation, but immediately update & release a new version of the product to help the customer achieve their goal.
  3. Customer isn’t happy, e-mails us for refund.
  4. [This is where I come in.] I ask the client whether the fix worked in an attempt to determine how I can help the customer.
  5. Customer says they didn’t try it and won’t try it, because they don’t want to be a guinea pig. Insists on refund, threatens with chargeback.
  6. I explain that we released a fix for the problem and hence it’s not about being a guinea pig; we’re just doing our job & helping them out.
  7. Customer ignores last e-mail, rudely threatens to publicize this and again threatens to go the way of a chargeback.
  8. I issue refund and at least attempt to explain our actions in this regard & how we actually tried to help. Still awaiting response (if any is going to be forthcoming).

Now, to be fair, Woo Themes has a “no refunds” policy (see below). So Adii is in his right to refuse a refund initially (though he eventually gave one). And I think the client is lame for threatening to do a charge back through his credit card (which would refund the money and add another fee to Woo Theme’s account). However, despite the no refunds policy, Adii does end up giving one and seems open to giving them with justification. It’s clear that Adii doesn’t think “fixing a bug”, as he called it, is a good reason for a refund. And it’s possible Adii only gave the refund to avoid the bad publicity and bad affects of a charge back. Adii writes in the comment how he is frustrated at the power customers wield in these transactions.

Is this customer irrational to want a refund?

I think the customer here wasn’t being irrational (in his refund request at least, though he may have had an irrational tone). It sounds like this customer bought the theme thinking it already had a feature he needed. It didn’t, and although Woo Themes fixed the issue right away, I think it is okay for the customer to not be satisfied with this. The customer expected one thing and got another. Despite the quick fix, the customer has reason to question the robustness of the “fix”, which is grounds for a refund in my book.

10. [Woo Themes] Refund Policy

Since WooThemes is offering non-tangible irrevocable, digital goods we do not issue refunds after the subscription or individual theme purchase is made, which you are responsible for understanding upon registering at our site.

By the way, just because the customer may have some justification for asking for a refund doesn’t change the fact that it sucks to lose a customer and to have possibly done some work on a feature just for them. It sucks.

What are we even talking about?

Adii seems to want to bring up two points for discussion. One is about the power customers wield with the threat of a chargeback. The other related point is about the desirability of unconditional refunds. The points are related because it seems the chargeback threats are effectively forcing Woo Themes to offer unconditional refunds even though they may not want to.

For the record, I think unconditional refunds are a good idea… especially when it comes to GPL software.

For one, the point about digital good being irrevocable (made in the Woo Themes policy) doesn’t hold as much weight when the software could technically be distributed for free elsewhere due to the nature of the license. I think many are trying to offer GPL software for a fee and basically hoping that a free version doesn’t crop up… or using split licenses to try to keep a free version from cropping up. Despite the legality of charging for the software in this way and mantras of pro-GPL folks that it’s “free as in speech, not as in beer”, GPL software WANTS to be free as in beer too. It’s kind of natural. it’s supply and demand. People will want your software for free… and if it’s GPL and distributed there will be those able to redistribute for free.

I don’t think you need to stop charging for software (there are many benefits to getting the software “from the source” which is worth a fee), but you shouldn’t be upset if someone gets your software for free… whether it’s from Bit Torrent, an official flavor variant, or by asking for a refund after downloading the software. I’ve only come around this this opinion recently, but I believe that to do GPL software right, we need to make sure most of the value is in that “other stuff” around the software: support, documentation, packaging, trust.

Anyway, back to unconditional refunds. In my experience (and from what I hear) they cause more good than harm. A few will abuse them, but this should be more than made up for by the increased number of sales you’ll get since customers will feel safer about buying something they haven’t tried out yet. That is the argument.

What I would do have done.

It’s funny that the same day Adii posts about this customer, we have a similar experience with a customer for Paid Memberships Pro (our “premium” WP membership plugin). In our case, the customer thought we integrated with a payment gateway that we don’t integrate with yet and asked for a refund. I made the refund first, and then replied with our unwritten policy that we’ll add integration for any payment gateway a paying member requests… it just may take a moment. In this case, our potential/lost customer says no thanks. And I understand, because the customer expected one thing and got another.

As I said, I’ve only come around to this kind of thinking recently. Over the past two months, our return policy has changed from one requesting a reason and requiring the customer to delete the software and vow to not use it, to just requesting the information we need to process the refund. (BTW, Bed Bath and Beyond is a bricks and mortar store that does no-questions refunds to good success.)

Another part of what people expect out of a “premium” WordPress product is better support. And it’s funny that in these cases good, fast support was mostly ignored. This sucks. Not everyone is going to be a customer.

And that brings up another idea that is floating around this discussion: irritable, bad, (dare I say “dick”) clients should be avoided. (The customer we lost was very nice about everything by the way.)

Avoid bad customers. Just don’t feed the trolls.

I agree with the general idea (promoted recently by one of the 37 Signals guys — Jason?) that you don’t need everyone to be your client, and you should use pricing and marketing to target better customers. I would just add that you shouldn’t air the dirty laundry when you do manage to shrug off a bad client. Calling them out in public (if they’re not under a rock, they’ll know that you’re talking about them) may bring them back for more. It will definitely send a message to potential clients reading the rant that working with you might be difficult.

Maybe you don’t want clients who are afraid of being called a dick. You only want clients that are willing to spend a lot of money and never ask for a refund. Just be careful that you’re not turning off clients you do want. There is a fine line between “keeping it real” and pushing away perspective clients.

In summary…

In summary, I think Adii probably handled things correctly up to the point that he posted about it on his blog and twitter. He was trying to bring up some things that should be discussed, but I think his emotions lead him to post it too quickly and in a way that makes it obvious who he is talking about… and he used that D word there.

He could be scaring away potential customers. (I mean look what Chris Pearson’s attitude towards WordPress did to his credibility in the community. Our attitudes toward our clients are similarly important.)

Adii, and some others in comments/etc, missed the point that the customer paid for something under false pretenses. Whether this is the customer’s fault.. or if in our opinions the customer should have been happy about the quick fix, he or she still has a decent reason to want a refund: it is valid to question whether the “quick fix” will be as robust as the customer expected. Whether or not one should be granted is up to the business and their refund policy.

Customers threatening chargebacks when they aren’t warranted sucks. I feel in general though, it is more important to protect customers by allowing chargebacks than it is to protect businesses from their misuse. We don’t make money by forcing people who don’t want to pay for our products to pay for them. We make money by making the customers that are willing to pay happy.

The digital nature of software works both ways. Someone can get the software and then chargeback to effectively get something for free. But then it didn’t really cost us much to deliver the software either. Especially when working with GPL software, we should be ready to accept situations where people obtain our software for free.

Alright. Let me know what you think. Maybe I’m being irrational here. I don’t mean to pile on Adii. Like I said, I think he did it right up to the point of blogging/tweeting about it. This is all very subtle stuff. We probably agree about more than we disagree, but I don’t think it’s black and white… which is why I wanted to write a million rambling words about it.

Ok, hit publish already, Jason.


Hacking the Long Tail

Making Collaborative Filtering Work for WineLog.net

What is the Long Tail?
Coined (in its current form) by Chris Anderson in his October 2004 Wired magazine article, “long tail” refers to a common pattern seen in graphs depicting product demand or units sold. The graph below is a simple chart of units sold verse the total number of units in a particular domain. This could represent, for example, the number of copies sold of each book in Amazon.com’s catalogue.

The Long Tail.
Image of the long tail, borrowed from the Wikipedia entry of the same name.

Continuing with the Amazon example, the x-axis would represent all books ever printed, and the y-axis would represent the number of units sold for each particular book. On the far left of the graph are large sellers like the the DiVinci Code or the Harry Potter series. On the far right are some books only a few people are interested in, with a relatively low number of sales.

The line in the center of the graph is very important as well. This line represents the cutoff point a traditional retailer might have for stocking items in their domain. In our books example, books to the right of the line have too low a volume to be worth the shelf space they would take up at a bricks-and-mortar store. Everything to the left of the line represents the books which would be readily available.

The term long tail refers to the shape of the graph, but more specifically to the area under the graph to the right of the center line. This represents an opportunity for sales which couldn’t be tapped into until recently. It’s a big opportunity too. A visual scan of some of these graphs shows that the area (representing revenue) of the long tail section can be as large or larger than the area of the so-called “short tail”.
People who haven’t done so should read Chris Anderson’s original article (or better yet, his book, The Long Tail: Why the Future of Business is Selling Less of More). The article explains in more detail why long tails exist at all and discusses the technological and business advances that have lead to a situation where companies can benefit from catering to the long tail.

– – –
Wine Sales Has a Long Tail
Now on to why I’ve been interested in the long tail lately…

Patrick Angeles of InertiaBev published a chart of data from their system, showing that there is a long tail in wine sales. InertiaBev provides “on-demand software that empowers wineries to sell direct and measure the performance of their sales”. Here is a graph that Patrick posted along with his article:

The Long Tail.
A long tail in wines, borrowed from Patrick article at the InertiaBev blog.

Looks familiar doesn’t it. Like with books, a relatively few number of wines account for a large number of sales. Also like books, there is a large group of wines that sell in smaller units but add up in total to a lot of revenue.

Quick note: Patrick originally published a graph based on actually numbers, but took it down to “protect the innocent”. You’ll see the same generic graphic when you visit the link above. However, the observations and numbers in the article come from real data.

– – –

Good and Bad Long Tail Business Models
So we have a long tail. Does that mean that we can build an Amazon or Netflix-like business model on top of wine? Maybe. Maybe not.

The thing to understand is that the long tail is not enough to have a good business model because there is a problem with scaling from offering 5000 books to 5,000,000, or from 500 wines to 10,000. What’s needed is a way to intelligently introduce customers to items in the long tail they might be interested in. While there are a lot of books in the long tail I would be interested in reading, I may never hear of them. Mass media and advertising is going to be focused on those books to the left of the cutoff point, the most profitable ones (on a per-title basis). But you can’t just recommend any book to me; there are at least as many books in the long tail that I would hate as ones I would want to read.

So successful long tail business models have measures to effectively prune or recommend items from their catalogue. Recommendations can be as simple as notifying a user of books written by the same author or wines from the same region. These can work to an extent, but something more is needed to make the recommendations feel personal.

This is where collaborative filtering comes in. It sounds complicated, but it’s really a simple idea. Make recommendations to me based on what I and other users have expressed satisfaction with in the past. At Amazon, we have the now ubiquitous “if you like this item, you might also enjoy…” recommendations. Netflix is another company that has successfully implemented collaborative filtering with their star rating system (and is now offering $1 million to anyone who can improve those recommendations.)

There are other methods of making recommendations (and WineLog uses a bunch of them), but the hacking section of this article focuses specifically on how our data can be tweaked to make it better for collaborative filtering algorithms. The ideas should be applicable to other non-trivial recommendation systems though.
Amazon and Netflix are examples of companies successfully making business in the long tail. How about some business models that wouldn’t work?

Consider the long tail for dating. This graph is extremely flat. The most promiscuous of humans are dating no more than 1000 people per lifetime. One thousand dates may sound like a lot, but then compare that to the 6 billion people in the world. This tail is really long… and so really flat. Even if you limit yourself to just people of one sex in the U.S. (about 150 million people), the curve is still extremely flat.
The problem with a long tail for dating (and flat long tails in general) is that there isn’t enough overlap to make useful recommendations. How often do you find people with more than one past partner in common? Not often.

Side note: collaborative filtering for dating is an interesting problem for a number of reasons as well. Consider the fact that your “ratings” will be skewed to the negative. If you are rating someone 5-stars out of 5, you’re going to try your hardest to take that person off the market. Also, locality affects dating (as it does almost any business). “It’s nice that you’re recommending this nice Asian girl for me, but I’d rather not have to leave Philadelphia to find a soul mate.” I think challenges like this just make the problem more appealing to me. DateLog next?

Another problem for collaborative filtering is with domains where the subjects are too similar. A recommendation engine for drinking water would fall into this category. One the one hand, similarity is a good problem because users are less likely to be upset about a recommendation (all water tastes refreshing served cold). On the other hand, if you want to make good recommendations, you’re not going to have a lot of data to work from. Some long tails may have too little differentiation among their top performers to be able to make good recommendations from.

– – –

What about WineLog.net?
Wine is a domain that seems to suffer from the two problems stated above. The long tail is generally flat, with little overlap in wines drunk by each user (compared to the level of overlap in movie viewing or book reading). It could also be said that the typical wine drinker can’t differentiate between a wine they like and one they don’t like, putting the quality of ratings in question.

Side note: Oddly, the fact that new wine drinkers can’t differentiate between a wine they like and one they don’t like is one of the reasons this domain is so interesting to us. Wine is an acquired taste, and we’d love to help our users on their path to enjoying wine on a new level. Also, many drinkers know they like a wine, but don’t know why. Besides making unit recommendations, our system can teach a user what it is they like about wine even if they don’t know it themselves by showing the user common tags among wines they like and dislike. Making recommendations based on tags is a topic for another article.

Alder at Vinography.com posted what’s become a popular article in our industry about Why Community Tasting Note Sites Will Fail. Here are his major points (paraphrased).

  1. Need a lot of wines.
  2. Users are stupid.
  3. No incentive to use.
  4. Wine “lovers” is too small a niche.

Not all of these deal with the long tail directly, but they hit on the problems that make collaborative filtering difficult for the domain of wine. (For a more direct rebuttal to these points, look for my comment on that blog post. Search for “Jason Coleman” on the article page.)

– – –

Hacking the Long Tail
So how can we hack the long tail to make collaborative filtering work for WineLog.net (or any domain with the same issues)? Visually, what we are trying to do is “build up the short tail” of the graph to give us better data for making recommendations on the long tail.

The Long Tail.
A marked up version of the original long tail image showing how suggesting “key units” can build up the short tail.

One of the great benefits of having a community site focused on wine like WineLog.net is that the community will build up the short tail on their own. Users of WineLog will notice the most popular wines of the site and are more likely to try those wines themselves. In this way, the users create the overlap necessary to make better predictions in the long tail space. Visually, the long tail graph will be growing “fatter” on the left side as these popular wines are tasted by more users.

So our first step in utilizing the long tail to the max is to recommend and encourage the rating of these popular “key” wines to our users. I call them “key” wines because, in a sense, they are used to build a taste profile for the collaborative filtering algorithms.

A good key wine is popular. A GREAT key wine is both popular and DISPUTED. A disputed wine is one with a variety of ratings. A wine which is rated 3-stars by all drinkers won’t tell you very much about a user’s taste. Now a wine with three 5-star ratings and three 1-star ratings is going to tell you a lot. Here’s a wine that is polarizing our users into two distinct camps.

We won’t necessarily be stretching out the short tail any further by recommending key wines with polarized ratings like this, but we will be getting the maximum use out of the short tail that we have.

So now you have our strategy: recommend key wines to our users which are both popular and disputed. This is going to build up the short tail for us, which should lead to better recommendations in the long tail space.

So while it may not be true that most wine drinkers have drunk Mano a Mano Tempranillo 2004 (even though they’ve all seen the movie Titanic), we can encourage our users so that the statement “Most WineLog.net users have 3-5 of our 20 key wines” will be true.
– – –

The Big Picture
I’m a technical guy. The collaborative filtering problem is an interesting one to me, but it’s not what this business hinges on. Effectively suggesting wines from the long tail will help our business, but there are other ways to do this besides CF algorithms. Users will find recommendations from friends, trusted sources, and other simple recommendations (like “try another 2005 Shiraz from Australia). Other data collected, from tags for instance, can also be mined in interesting ways to profile users and suggest wines for them.

I hope this article has explained the long tail well and how it applies to sites trying to sell (or like WineLog, just try to recommend) wines. We learned that a site’s ability to suggest units from the long tail is a large factor in the success of their business model. We learned about two problems some domains have with their long tail: lack of overlap and homogeneous units. And we finished up with a strategy for combating these problems with respect to collaborative filtering strategies: suggest key units which are both popular and polarizing.

The focus was on wine and collaborative filtering, but the ideas brought up should be applicable to other domains and recommendation systems.