I have been loving the book Web Operations: Keeping the Data On Time
. It is indispensable reading for anyone responsible for hosting a web application or any important website. As the editors say, web operations is an emerging field with little in the way of training programs or officially sanctioned best practices. Nobody is classically trained. The best web ops people grew up in the field and learned from other greats and being faced with tough challenges.
One of the book's chapters is a reprint of an academic article called How Complex Systems Fail by Richard I. Cook. Interestingly, this article wasn't written about Web Ops at all. Richard Cook is an MD and he was writing about hospitals and healthcare. But the similarities with web ops are striking. It's a quick read and it's free. The points that were especially salient to me were:
-
"Complex systems contain changing mixtures of failures latent within them."
The more fault tolerance you add, the more unnoticeable little failures become. And even when you notice them, issues can be fleeting and too difficult and/or not important enough to fix. You just live with them. Even if you have a very simple set up, the cloud hosting service that you are running it on is incredibly complex — so complex that it can suffer from living organism diseases like auto-immune vulnerabilities. When adding yet another layer of complexity to make your system more resilient, sometimes you introduce more potential for failure that may reduce the health of the overall system. Our hosting infrastructure has lots of redundancy built into it. But sometimes I yearn for simpler times when I had one physical server that I just rebooted when things got ugly.
-
"All practitioner actions are gambles."
There are a few points in the article that deal with post accident analysis. As frantic as it feels to have a site outage, the author is talking about much higher stake failures (he's a doctor, remember). But his analysis is right on. Whenever we do anything, we are making a gamble — no matter how much we test. Sometimes the risk of the gamble is much less than the risk or opportunity cost of doing nothing. But you can't always tell. It is easy to get down on yourself or your team when a mistake is made or a failure occurs. But as the article says: "That practitioner actions are gambles appears clear after accidents; in general, post hoc analysis regards these gambles as poor ones. But the converse: that successful outcomes are also the result of gambles; is not widely appreciated." In other words, system improvement depends on gambles.
-
"People continuously create safety."
The point here is that you can't make anything totally idiot-proof. In the world of web ops, you can script things but you always need experienced people thinking about what they are doing and keeping an eye on things.
-
"Failure free operations require experience with failure."
This point really extends the previous point. The only way people get experience is by facing failure. In the early days of my career (amazingly, nearly 20 years ago), I worked in a tiny I.T. department serving a large and rapidly growing company. Our team was brilliant but inexperienced and we were constantly faced with new problems that we had to solve under pressure (my favorite was our "total network crash" that was actually the folding card table that held all of our servers collapsing). We all learned so much and everyone on the team went on to a successful career. But we wouldn't have learned anything if everything went smoothly. As a side note, I experienced more appreciation from the user community during those years than since. People saw that we were creative, dedicated, humble, and fearless. We didn't have that dysfunctional I.T. v.s. business dynamic. We were all in it together.
I know that the message I have been sending is that marketing I.T. is complex and messy and you just need to deal with it. To an extent, this is true. But it also means that whenever there is an opportunity to simplify something, you need to take it. I see every piece of unjustified complexity as a risk with unknown cost. It is easy to relate to the added up-front cost of adding a little flourish that makes an application more interesting. It takes experience to understand the long term cost of maintaining something that is more complicated than it needs to be. Even if that clever code is lucky enough not to break when something else changes, it will perpetually need to be circumnavigated by others who barely understand how it works. Before long, you have a culture of people who are afraid to touch things and cruft is inevitable.
If there is a way to do something in a more direct way, take that approach. When thinking of development costs, budget for the cost of maintaining that code. Prioritize refactoring and refinement over adding new features. Make is suck less. Most of all, don't let your complexity outgrow your capacity to manage. it.
James Robertson has a great post called James Robertson's Rule of CMS Usage. The rule is:> Over time, CMS usage will head towards the middle of the road.What it means is that companies typically start out thinking that they have very unique requirements and select elaborate systems to cater to the peculiarities of their process. Natural selection extinguishes individuality that is not a competitive advantage and companies are left with capabilities that are no longer needed. A classic example of this kind of unstable situation is where you have a web site that looks very average but is managed by exceptionally complex systems and processes. In James's words:> The gap between back-end complexity and front-end simplicity quickly became apparent. This has increased the amount of effort needed to create new sites, and to manage the CMS on an ongoing basis. In summary, a greater than average amount of work to publish very average sites.
This kind of suboptimal solution often happens when a software selection is done by a generalist project manager or functional analyst that does not have the perspective necessary to challenge the status quo. It also happens when the selection is dominated by an expensive software vendor that tries to win the opportunity on flexibility. In order to make the case for flexibility you need to convince the customer that they are an exceptional case that cannot be supported by a simple solution. This is why I tend to focus on leading requirements and keep the progress grounded in the practical by pushing back on unconventional requirements that do not add to extraordinary value. That is not to say that all companies are the same; I love working with a client and discovering something innovative that gives them an edge. It is just that companies with similar goals are less different than they tend to think they are and can benefit from converging to the "middle of the road."
CMSWire has a nice little article on Google Sites and MOSS. The general gist is that, while not as capable as MOSS, Google Sites may be just good enough for basic websites and collaboration. The absence of some advanced features may be compensated for by the simplicity of the platform. Sometimes just good enough is exactly what you want.
Over on the Sharepoint side, I just read Michael Sampson's excellent report The 7 Pillars of IT-Enabled Team Productivity: The Microsoft Shareport 2007 Analysis. The report uses Michael's 7 Pillars framework to evaluate Sharepoint. Not to be a spoiler but MOSS doesn't do so well.
I am not convinced that you want one platform to try do do everything but Michael assumes that this is Microsoft's goal (it certainly isn't building a robust WCM system). So, it seems that Sharepoint is caught in the middle. It is not the silver bullet, one-stop-shop for enterprise collaboration, and it is not the simple, cuddly tool that everyone loves to use. That is not a very fun place to be.
While catching up on my Blog reading, I noticed this post on CMS Watch which links to an InfoWorld case study of a project that was almost ruined by lack of user testing. I very much agree with Tony Byrne's point that usability is one of the biggest issues in the CMS industry and I am happy that Tony is being so vocal (one article, another article) about defining the problem and challenging the industry to fix it. James Robertson, of Step Two Designs, has also added his respected opinion to the discussion with a fine article on the importance of simplicity in large CMS deployments.
A CMS is one of those applications like a word processor where many users, often non-technical users, need to become proficient with no little or no training to get their job done. Frequently CMS users are under time pressure and, if it is quicker to misuse the tool than to use it properly, the tool will be misused - or, if possible, not used at all. Content Management is a very broad term but I think this applies to everything from this blogging software that I am using right now, to a Web CMS, to a document management application. A CMS user wants to publish what needs to be published with as little overhead as possible. As a matter of personal pride, author's want their content to read well and look good. But other than that, the less time spent with a CMS, the better.
Producing content is also a creative process and content creators don't like to feel like data entry people. CMS users don't like to waste time tagging their content with metadata. They frequently rebel when forced to fill in structured forms and give up the artistic license to determine how their content will be displayed.
But on the other side, are the consumers of the content. They are important users too. They want to be able to find the information they need (which requires good metadata) and get in a format that is accessible to them. The latter requires separation of content and layout so that content can be re-purposed to different formats. As Yair Dembinsky eloquently put it in a post on the CM Professionals mailing list:
Our main focus should be the target audience - how they want to consume the content: In some cases - they want a well written article, and in others they want to be able to retrieve small but relevant chunks of information. Writers and editors must adapt to supply the specific need of their target audience.
So, the CMS is stuck in the middle between the content contributors and managers and the content consumers. Both sides want their jobs to be easier and simpler. Some of the answer lies in automation adding information (metadata) and structuring a content asset for better reuse. But that only takes you so far. We don't want to rely on the system too much to interpret our content and make false assumptions. Workflow helps too by creating an opportunity to divide responsibility between those who create content and those who organize and classify content. However, companies typically under-invest in this area and do not dedicate editors to this task
At the end of the day, the CMS will always be a compromise between the efficiency of creating content and the utility and re-usability of the content. Content Management is a challenging problem. So what can we do to make it better?
One initiative that is just starting a new project on Open Usability called CMS User Interface Guidelines. I hope that this project gets some traction because I think there is a lot of potential here. Unfortunately the cards are somewhat stacked against it because usability is such a nebulous thing and people are more likely to complain about usability than to do something about it. I think success will lie in the ability to set down some common metaphors and idioms that users are familiar with so that they can quickly get their bearings in the application.
Like Microsoft Windows or hate it, there are some conventions that make all Windows applications familiar enough so that a new user does not get totally lost. For example, the File Menu is always on the left, Help is on the right. Ctrl-C is Copy, Ctrl-V is paste (except when you are using Intuit Quicken where they are "Categorize" and "Void". Grrr. How can you people call yourself Intuit!?). Microsoft also achieved high level of adoption for its Office applications Word and Excel by copying the UI functionality of then market leading applications WordPerfect and Lotus 1-2-3. Consistency is the key. An application does not have to be intuitive if all the users are familiar with it. Remember going to a travel agent not so long ago and watching him or her zip around the SAABRE terminal with two character codes?
Another area where things can be improved is the standardization of terminology. Different CMS products have different names for things. For example, in response to another CMS list request for different names for sub-components of content, Tony Byrne of CMS Watch listed the following:
-
Elements
-
Snippets
-
Objects
-
Placeholders
-
Pagelets
-
Styles
-
Chunks
-
Briicks
-
Parts / Fragments
-
Nodes / Fields
-
Components
Standardization in naming, design, and behavior will not only help CMS become easier to learn; it will also create a community that crosses organizational boundaries and includes both developers and users that can work together to solve this very difficult problem.
While standardization will help improve "out-of-the-box" functionality and the underlying framework of the application, in reality, CMS are usually heavily customized to work within the business context (perhaps more than they need to be, but that is another story). This is where the CMS is not a build vs. buy, it is a build vs build AND buy. Most CMS projects spend 2 to 3 times the license cost on integration. To get these customizations right, I would say usability testing is an understatement.
The worst thing you can do is spend a lot of time on gathering abstract requirements and creating abstract designs, then select a product, then do detail design with the users, then implement it, then test it and finally run it through usability testing. Any issues found that late in the process will be too expensive to fix - too expensive because you are at the end of the project and you are out of time and money. When you test this late in the game, you are basically just hoping that the users just rubber stamp what you did. And if that is the case, why test at all?
What I would recommend is have usability testing drive the design rather than validate it. Select a product, install it and show it to the users in its out of the box state. Ask them what is wrong with it. Prioritize a bunch of modification requests, implement them, then show the application to the users again. Repeat this cycle until the benefit of the modification is outweighed by the cost of implementing it. This approach, which, for lack of a better name, I call "change driven development" (a nod to Peter Coad's "Feature Driven Development"), gives business users a chance to experience the default behavior which may not be that bad - especially if not modifying the feature leaves more budget for improving another feature. In
addition to helping business users make better choices in design, involving the users at this stage of the project builds their comfort level and ownership of the application and leads to less complaints after it is deployed.
Business users know their job. They know the tools that they use. But one of the things they have the hardest time with is imagining something new and thinking abstractly. They are too immersed in the concrete of their day to day to think outside of their box. They are even worse at understanding the scope implications of their requests. So the result of waterfall approach to CMS customization is that we force users to think blue sky and, just when they get comfortable, we slam them down with admonitions of scope creep. Then we ask them to test the application for usability and we ignore their feedback. This is a recipe for failure and disappointment.
I just read James Robertson's short article "More Users = Simpler CMS.". In it, James puts forth a proposition that if you are deploying a company wide CMS, it should be as simple as possible. This idea runs directly against the CMS buying behavior over the last couple of years where companies rolling out the large CMS implementations were looking for "industrial grade" software that had all the bells and whistles. Cross departmental software selection committees aggregated their feature wish lists to come up with selection criteria that favored products that had (or at least claimed to have) everything. And after all this, companies were surprised to find that the implemented solutions were difficult to use and required more training than was budgeted for.
I think the tide is changing from feature richness to simplicity and usability. A couple of other data points:
-
Salesforce.com like ASP CMS solutions are gaining popularity
-
Oracle's ECM product has been designed for simplicity. In the words of Rich Buchheim, Oracle's senior director of product management: "From the user's perspective it's not different than using a desktop application or file server."
- Leading CMS industry analyst Tony Byrne of CMS Watch sees usability as being one of the most important trends in content management: "In conference rooms around the world, authors are standing up and declaring, 'Our CMS tool sucks!' Many CMS vendors have noted this user backlash and now fall over themselves to tout products as "intuitive," "adaptive," or just plain "easy to use."
In the world of CMS, sometimes less is more. Maybe its the feature that the software does not have that will make it a success.