In case you have not been following the open source blogosphere, there is an interesting discussion happening around a new term called "open core." Open core describes the strategy of building a business on selling commercially licensed software with open source licensed components or companion products. Open core companies tend to use third party open source components and distribute some of their own components (not the complete solution) under an open source license (typically the GPL because that is the most protective). Matthew Aslett has a nice breakdown of what is and what isn't open core; although I disagree that the subscription model is open core when the vendor requires a subscription fee to use the software.
This strategy is in contrast to software companies that primarily offer a free open source version of their product and make their revenue entirely from support services (such as technical support, maintenance, training, implementation). eZ Publish and Hippo are pretty good examples of pure open source companies. The term "open core" was coined by Andrew Lampitt and all the chatter around it shows how much it is needed. The market needed something to distinguish the two types of companies: open source and open core. Nicholas Goodman makes the observation that it doesn't make sense for a company that only sells proprietary software to call itself an open source software company. Now each type of company gets its own descriptive classification.
Many of the "household name" open source software companies follow the open core model: Alfresco, SugarCRM, Pentaho, Magnolia... to name a few. The viability of the open source version varies from company to company. Magnolia Community, for example, is full featured enough for most typical uses and is as well maintained as the Enterprise Edition. In other cases, the Community Edition is deliberately hobbled to encourage an up-sell. In other cases, the licensing is just confusing. For example, Alfresco's Enterprise Edition is officially GPL licensed but you need to pay an annual subscription fee to use the copy that was compiled and tested by Alfresco, Inc. — and they won't sell you support or allow their partners help you unless you are using the version they compiled.
From a market perspective, the "open" companies that were established with venture capital tend to be open core. This makes sense because venture capitalists make big investments and are looking for big returns — bigger than what a pure services company can hope for. To get VC money, a start-up would have to position itself as a software company that needed to build product with scalable sales and decent margins downstream. Services companies don't work that way. A services company can make money from day one but shouldn't expect hockey stick growth and low variable costs. A successful service-based software company may also need huge market penetration to create a big enough market to sell services into. It is a matter of multiples. If for every 100 users you get 1 paying customer, you better have millions of users. Many software categories do not have that market potential. For every 1 CRM instance, a large company will probably have 3 CMS instances, 30 database instances, 60 web server instances, and 100 operating system instances. This is why RedHat can be more open to unpaid use than SugarCRM.
So, what does this all mean to the buyer? As I mentioned in the Open Source Web Content Management in Java Report, it is important to understand the vendor's business model to judge the viability of the company and the total cost of the software that you will bear. Differentiating between open source and open core will add some clarity to a confusing marketplace with complicated licensing models.