Evaluating open source and closed source software
Gartner has been saying how the current recession favors open source software. Many people in the open source software business are experiencing the same thing. They are finding themselves included in software selections against commercial players that were once considered out of their class (certainly out of their market). I think this is a great new development. There are some very good open source platforms that have been ignored for the simple reason of being open source and not having the same recognition as their closed source competitors.
That said, using licensing cost as a primary decision factor is a bad idea. Licensing is still a relatively small portion of the overall cost of a software solution. When a company looking to save money on licensing also tries to cut costs in other aspects of the solution (implementation, migration, deployment, and training) project risk balloons. In fact, most of the open source horror stories I have heard can be attributed to this same flawed logic. An open source software implementation project will probably cost the same to execute (not less, not more) than an equally suitable commercial product. I predict that 2009 will see many failures from unrealistic technology investments as companies try to unscrupulously cut costs and suspend their disbelief of a free lunch. But there will be an equal number of successes as companies consider leaner (easier to use) technologies and more agile (and pragmatic approaches) to implement them. This is where a trusted and experienced implementation partner can be invaluable by setting realistic budgetary expectations and efficiently expending resources. If you go with an open source solution, be sure not to skimp on the resources that will make the execution of the project a success.
The primary challenge in evaluating open and closed source software at the same time is that they tend to be represented unevenly. Even commercial open source products (which behave very similarly to traditional commercial software companies) usually cannot afford to field a sales team like a commercially licensed product company can. Without the upside of an all-profit license sale, the risk simply isn't worth it. Even traditional commercial software companies balk at spending money on sales when the potential license deal is too small. Somebody else needs to represent the solution to level the playing field.
The open source product advocate could be your own internal I.T. or it could be a systems integrator that presents a complete solution. Both options come with complications. If it is your internal I.T. organization, things can get emotional and political. They will get attached to their solution. In fact if they don't feel a personal connection to what they are selling (like a commission-dependent external salesperson is), they won't be a good advocate. If their solution is not selected, there are bound to be hurt feelings and the last thing you want is someone on the team hoping for an "I told you so" moment when their victorious competitor runs into problems. Another potential issue is that I.T. may not have the resources to support their solution if it is selected. Presumably they have other responsibilities that won't go away if they "get this deal." Of course, this is less of a risk with commercial open source software that comes with a support contract that is similar to the other commercial solutions you are considering.
The primary problem with using an external systems integrator is knowing which one to use. Because every open source project is offered by many consultancies, the systems integrator market is even more chaotic and confusing than the software market; and there are fewer resources available to help you figure it out. Furthermore these companies are in constant flux as their staffs turn-over. The secondary issue is that there are big stylistic differences between a consulting sale and a product sale. A consultant sells by consulting. He wants to learn about the problem and work with you to solve it. He is less comfortable just quoting specifications and prices of a pre-defined like a software salesman may. His quote will be appear high because he is unable to ignore or trivialize the implementation effort like commercial software salespeople tend to do. And, because consulting carries a lower margin than software licensing, he is probably less willing to invest as much in a customized demo. The last thing he wants to do is build a big part of the system for you and then have you turn around and have someone else finish it.
Still, I do think that using external systems integrators is the way to go. In fact, I spend a lot of time talking to consultancies to keep track of who is good at what so I can refer my clients to the right integration partner. The best way to normalize the software vs. consulting stylistic differences when you are considering an open solution is to have all the solutions presented by integration partners. This will give you a more realistic comparison of the true cost and scope of the solution. A consultant is less likely to exaggerate "out of the box" capabilities than the software vendor because he will be around when the box is opened. This will also level the playing field on the investment that each candidate is able to make. If you take this approach, you will come out of your selection with a trusted partner that knows your business and is personally invested in your success. With that, how can you go wrong?