Advice for vendors dealing with independent analysts
Alan Pelz-Sharpe, from CMS Watch, has written some great advice for Analyst Relations professionals in an article called "Advice for vendors dealing with independent analysts." The only thing I would add to his list of dos and don'ts is to use the information from the evaluation to make the product better.
One of the nice things about writing about open source software is that many of the products that I cover do not have analyst relations people. Instead, I talk to developers, committers, and CTOs. The big difference is this... AR people's job is to make the product look good, a developers job is to make the product be good. Nearly all of the products that I reviewed in Open Source Web Content Management in Java were extremely gracious with the criticism. Part of it is that they are grateful for the coverage. But a bigger part of it is that they are used to interacting directly with the community and getting direct feedback. Most open source developers know that software doesn't get better by convincing yourself that it is great. It gets better through continuous improvement that uses criticism as a catalyst for creative solutions.
It doesn't make sense for software vendors to reject (essentially) free feedback that can be used to make their product better and pay "tier one" analyst firms to try to delude themselves and the market with their own hype. In the end, the truth (as experienced by the user) always comes out. In a Web 2.0 world where everyone has a voice, it comes out even faster.