While moderating a comment on my "10 Django Master Class action items" post, I was inspired to evaluate how I am doing on these action items and whether they are helping. Below is a brief summary of my progress; but first a little background. Recently, I had the rare opportunity to rebuild (from the ground up) an application that I wrote for a client. The context was that the first version of the application was a prototype that I built to help demonstrate an idea to potential investors and customers. The prototype served its purpose excellently. It was able to evolve alongside the idea as my client got feedback and refined the value proposition. We came out of the prototyping phase with a strong vision and an excited group of investors and beta customers. To minimize costs I avoided refactoring the application and cut a lot of corners. By the end of the prototype phase, the idea had changed so much that we were really faking functionality by overloading different features. Still, for a ridiculously small investment, my client was able to develop and market test an idea. And now I get to build the application for real and apply the best practices that I learned about in the Django master class. Here is what I am doing and how it is working out.
Use South for database migrations (adopted). I have grown so attached to South that I find it hard to imagine life without it. This is especially important because I am managing different environments and the object model is changing as I add new features.
Use PostgreSQL rather than MySQL (adopted). I am steadily getting more comfortable with PosgreSQL. pgAdmin has been really helpful as I get up to speed with the syntactical differences from MySQL. So far, the biggest differences have been in user management and permissions.
Use VirtualEnv (adopted). VirtualEnv + VirtualEnv Wrapper has been great. For a little while I was working on both the prototype and the actual application. VirtualEnv made it easy for me to switch back and forth. This will also be helpful when I upgrade to Django 1.2.
Use PIP (adopted). I really like how you can do a "pip freeze" to create a requirements file that you can use to build up an environment.
Break up functionality into lots of small re-usable applications (adopted). The prototype had one app. The production app that I am building has 6. One of the apps contains all the branding for the application and some tag libraries. Templates in other apps load a base template from my "skin" app. The best part of using this strategy is in testing and database migrations because you can test and migrate a project one app at a time. The hardest thing for me to figure out is how to manage inter-dependencies and coupling. One strategy that has worked well for me is to focus dependencies on just a couple of applications. For example, I have profile application which manages user profiles (extended from the base django.contrib.auth.User model.). I have other apps that relate to people but I am careful to create foreign key relationships to the User model rather than my profile model.
Use Fabric for deployments (adopted). One word. AWESOME! I have scripts to set up a server and deploy my project without having to ssh onto the server. The scripts were not that hard to write. I took inspiration from some great posts (here and here). Now I can reliably push code (and media) with one local command. I am managing the development of another site running a PHP CMS and I am strongly considering having the team use Fabric for that as well.
Use Django Fixtures (adopted). Managing fixtures in JSON has turned out to be really easy. I typically have two fixtures for each app: initial_data.json and <app_name>_test_data.json. initial_data.json mainly contains data for lookup tables. It is run automatically when syncdb (the Django command to update the database schema) is run. I typically create these files with the dumpdata command and then edit them manually.
Look into the Python Fixture module (not adopted). I looked into this module but, to be honest, editing the JSON files is pretty easy so I don't see the need for it.
Use django.test.TestCase more for unit testing (adopted). I have been doing a considerable amount of test driven development (TDD). It all started when I wanted to rewrite the core functionality but I needed to wait for someone else to re-build the HTML in the presentation templates. Now I have around 130 unit tests that I run before I commit any code. Focusing on unit testing has made me write code that is more atomic and easier to test. Now I think "how will I test this?" before I write any code.
Use the highest version of Python that you can get away with (adopted). A big motivator for me here was when I upgraded my workstation to Snow Leopard which ships with Python 2.6.3. Getting 2.6.3 on my server was a little more complicated. I wound up using a host that comes with Ubuntu Karmic Koala which also comes with 2.6.3. I am really pleased with Ubuntu and it seems like most of the Django community is going that way.
I feel really lucky for the opportunity to rewrite an application and apply lessons learned. Too often you are stuck managing code that you (or someone else) wrote before you knew what you were doing. That is, before the functionality of the application was fully understood; before a feature of the API was available or known; before a more elegant solution was discovered. I am sure that I will continue to learn new things and want to apply them and I plan to continually refactor as long as I am involved with this project. But this full-reset has been a great experience.
I couldn't help but return to a conclusion that I reached long ago: "knowledge management," as an enterprise practice and class of software, is a false promise. Furthermore, traditional corporate training programs are doomed to failure.
I was first struck by this realization around ten years ago when I was working on a project for a department of the federal government. The premise of the project was to "capture the knowledge" from a generation of experienced staff that were on the cusp of retirement. This department was structured so that knowledge was concentrated in minority of senior employees. Underneath them there was a thin layer of mid-level staff; then a large group of juniors. The strategy was to video pre-retirees reminiscing about their experiences and the department could somehow do something with that "knowledge." The idea was dead on arrival and the prime contractor (that originally pitched it) knew it. I remember suggesting an alternative strategy of setting up an apprentice program where people could learn by doing rather than watching TV; but I was laughed out of the room. They had no interest in "capturing knowledge." Their primary business was hiring retirees and staffing them as consultants at the department. Failure was more profitable than success.
Ever since that experience, I have been keenly interested in the process of learning. As a technologist and a consultant, I am always learning so I have developed tactics that work for me. What surprised me in reading Pragmatic Thinking & Learning is that there is actual scientific theory that supports many of the tactics that I employ. What I like most about the book is that it talks about thinking and learning as a personal process that you have to do yourself. The most a teacher or a computer can do for you is provide information — data. To turn that information into knowledge, you have to internalize it into something that is meaningful to you. You need to put the information into context with other things you know.
Most corporate professional development programs ignore this truth about learning. They practice what the book calls "sheep-dip" training programs where training classes "dip" employees in information that quickly wears off. The only way that you learn from these classes is to apply what you heard right away. The learning happens after the class. This is why I like the idea of "drop-in labs" so much. I think the industry is starting to accept this "learn by doing" philosophy. Knowledge management experts are talking less about repositories and more about communities and workspaces. Emphasis seems to be shifting from the assets to the learning process.
On a personal level, being more conscious of these ideas is helping me be more deliberate about how I learn. The book advises setting SMART (specific, measurable, achievable, and relevant) objectives and using techniques like SQ3R (Survey, Question, Read, Recite (summarize), Review). But most importantly, learning has to be fun because we learn best through play. Yet another reason to work in a field that you love.
When an approver is reviewing a change to a previously approved asset, Hippo CMSdefaults to a version compare view. Adds, changes, and deletes are highlighted by color and strikethrough (see screenshot).
The concept reminds me of source code control system commit emails that highlight changes to code whenever anyone checks anything in (click on screenshot for a larger view). I do a lot of web development team mentorship and I have come to rely commit emails.
They allow me to quickly determine if there are any material changes that I need to dig into and saves me loads of time (in the case of the screenshot example, no important changes have been made). I am wondering if the Hippo implementation adds excerpts of changes in their workflow notification emails. That would be convenient.
I really like the Hippo team's pragmatic approach to enhancing Hippo CMS. Rather than adding lots of a new "checklist" features (that make the product more complicated), they tend to focus on refining and streamlining functionality that customers probably already use. It is that "make it suck less" philosophy that makes software continuously better for the users but tends not to get recognized by most analysts and buyers.
There is a great Reddit thread on The Onion's migration from Drupal to Django.The Onion was one of the companies that I interviewed for the Drupal for Publishers report. One of the things I mention in the report is that The Onion was running on an early version (4.7) of Drupal. The Onion was one of the first high traffic sites to adopt Drupal and the team had to hack the Drupal core to achieve the scalability that they needed. While versions 5 and 6 of Drupal made substantial performance improvements, The Onion's version was too far forked to cleanly upgrade.
Still, The Onion benefited greatly from using Drupal. They were able to minimize up-front costs by leveraging Drupal's native functionality and adapt the solution as their needs changed. Scalability was a challenge but it was a manageable one. Even though forking the code base was not ideal, it was a better alternative than running into a brick wall and having to migrate under duress. The Drupal community also benefited from the exposure and learning that came from The Onion using Drupal. Everybody won &mdash how often can you say that?
I can understand the choice of Django 1.1 (current) over a hacked version of Drupal 4.7. Having built sites in both Drupal and Django, I can also see the appeal of using a Django over Drupal 6.16 (current). Django is a more programming-oriented framework and The Onion has programmers. Django is designed to be as straightforward and "Pythonic" as possible. Drupal tries to make it possible to get things done without writing any code at all; and if you can avoid writing code in Drupal, you should. As a programming framework, Drupal has more indirection and asserts more control over the developer. The Onion's staff of programmers clearly appreciate the programmatic control that Django affords and they are quite happy with their decision.
After hearing all this news about status.net, I took a few moments to connect my Identi.ca account with my Twitter account. I am not at all sure what good that did but I figured it couldn't hurt. While I was there, I got to thinking what is so special about a microblog? And I came to the conclusion — nothing. You can think of a microblog as a title-only blog (entries with no bodies). What is new there? Subscriptions have already been handled quite well by RSS. What makes Twitter (and Facebook) so important is not the ability to post 140 character messages. It's not even really the API or mobile integration. What makes Twitter/Facebook important is that they are used by so many of the people that you want to reach. There are plenty of failed status-oriented services that have had the technology but, either through bad timing or other missteps, failed to build an audience.
I wonder how attempts at internal microblogs are working out. I haven't heard any success stories and I doubt if any survive past the initial novelty phase. If there are urges to microblog within the enterprise, employees could satisfy them with the internal blogging infrastructure that has been idle in most organizations. If there is any internal application for the microblog it is aggregating what employees are tweeting out on Twitter. But, depending on the staff, I don't know how interesting that would be either.
I think that the microblog is going to fail as a category of software. There just isn't a big enough market of buyers that can build a sustainable microblogging community. I can't think of existing sites with large audiences investing to buy or build a microblogging capability. If you were CNN, would you rather have a reader tweet a story link on Twitter (where it could be re-tweeted by millions of users) or on your CNN own microblog community? The only purpose I see for status.net is just in case Twitter kills itself (by running out of money or ruining the service). I seem to remember a lot of people threatening to defect to Identi.ca and FriendFeed when Twitter was going through its problems. But that didn't happen. How long can the status.net's and Identi.ca's of the world sustain the energy and motivation to be an understudy behind a healthy actor?
I know the argument that suggests that distributed communication services will eventually win like how the open, distributed web was destined to eventually beat out then popular online services like AOL, Compuserve, and Prodigy. But I don't think this is the same. If I am missing the point here, please enlighten me. I would love to see competition in this sector if it makes sense. I just don't see it.
The other day I was trying to describe to a client how content management systems are different. My audience was not familiar with some of the core features of a CMS so my examples were too abstract to get my points across. However, they were familiar with Microsoft Office so I used that as an analogy. They found it quite useful so I thought I would repeat it here.
The Microsoft Office Analogy of Web Content Management Systems
Imagine your boss wants you to "put down a few thoughts" on a topic to share electronically with a group. You might choose to create a Word document, a PowerPoint presentation, an Excel work book, or maybe even an Outlook email message and then start typing out a list. Because you just want to capture some ideas and your boss was so vague about the format, it doesn't really matter what you choose. Your boss asked for ideas, not a deliverable. Let's say you choose PowerPoint because you have a nice looking master template and you figure a few bullet points would look nice in big text on one slide. It doesn't hurt that you are really comfortable with using PowerPoint.
So you show your boss the PowerPoint deck and he wants you to elaborate more. You have used terms that need to be defined and he wants you to support your points with some background information. This "few thoughts" is starting to sound like a memo or a report. Now PowerPoint is not looking so good. What do you do? You can start to overload PowerPoint's features to make it work like a word processor; or bail out and copy/paste everything into Microsoft Word. You will probably make this decision based on how familiar you are with PowerPoint vs. Word, how particular your boss is going to be about the format, and how long this document will live. If you make the wrong decision, fighting the technology will start to dominate the time you spend on this project. Every change will risk blowing up the whole document.
Strengths and Weaknesses
Like all analogies, this had its strengths and weaknesses:
Analogy Strengths
Many companies choose a CMS for arbitrary reasons without really understanding their requirements. When requirements are vague (like "we just want a simple website"), all options look equally viable.
Stakeholders often have tacit but very specific requirements. e.g. "I will know it when I see it."
If you really know a technology, you can force it on a problem by overloading features and redefining the problem. For example, while the MS Word user sees a bunch of paragraphs, the PowerPoint user sees a series of vertically arranged left-aligned text boxes. However, eventually these solutions tend to become unstable, especially if they need to get supported by different people.
You need to know when to swap tools. If you do it too late, you waste too much time misusing the old technology and you have more content to move over.
Analogy Weaknesses
My client immediately wanted to know which CMSs were like Word and which CMSs were like PowerPoint. You don't want to go too far with this analogy because a CMS is more than a document editor.
The switching costs from one CMS to another is much much greater that from one document format to another. If this was a CMS implementation, you want to ask your boss to clarify where this project is going before you pick your tool.
While not perfect, I think this analogy did its job of making people think about CMS selection in terms that they can relate to. They can now visualize themselves misusing the wrong tool to get the results they want and living in constant fear that one more hack is going to bring the whole site down. They are now thinking about their website in terms of a development roadmap that will be influenced by different stakeholders over time. They can relate to how people have a difficult time expressing requirements that they know tacitly and can apply familiar skills to get clarification. As long as I can keep them from over identifying the website with a MS Office document, I think I am going to be OK.
is a useful book of how to use your computer more efficiently. One of the several tips that I have adopted is to use one text editor (in my case, TextMate) for all text oriented work. The idea behind this is that when you work in one tool, you get to know it really well and can take advantage of all its nifty time saving features. Most software users, however, only use a tiny fraction of the useful features supported by the software.
I had gradually been moving in this direction for a while. At first, I just used a text editor to program in dynamic languages (Javascript, Python, PHP, Perl, SQL), do HTML/XML markup, and edit large data files. About 6 months ago, I got so fed up with Eclipse's clunkiness that I started to write Java in TextMate. Since reading the chapter in the book, though, I have started to use TextMate as a blogging tool. This was a big step for me because I was quite happy with Red Sweater's MarsEdit software. Yes I know that MarsEdit gives you the option to edit posts in TextMate but I decided to go all in. I have not yet been able to get TextMate hooked up as my email editor. I always thought programmers that did everything in EMACS were silly. But since making the change, I have found a lot of powerful keyboard shortcuts and macros. My one hold-out is that I still use Oxygen for editing my DocBook documents.
My successful experience caused me to question whether there was any merit to the "One CMS to rule them all" ECM (Enterprise Content Management) vision that I have been battling over the past 7 years (a battle that I won, by the way, but I am not gloating). Would there be any benefit of having a knowledge worker getting to be a true expert in one tool? Then I came to senses and realized two key differences:
Web Content Management is about managing semi-structured data, Enterprise Content Management is about managing metadata. A WCMS primarily helps a user edit and and assemble reusable, structured content. In a document-oriented ECM system, most of the documents are binary files that are edited using tools like MS Word. These ECM systems are used primarily for creating metadata, organizing, and managing permissions. Furthermore, most people organize their documents on a file system metaphor. Web content organization tends to be much more fluid and rule based. Your website is not a file system. You will fail at web content management if you think that a website is a bunch of MS Word documents saved as HTML. Because there is so little functional overlap, one tool doesn't make sense.
CMS users don't define them selves as CMS users. Programmers, at least the good ones, care about their craft and take pride in how they work. They read books and blogs to continually hone their skills. They love their tools and treasure knowledge of obscure little tricks. Good designers tend to be the same way. Your average content contributor may be similarly inspired about their profession, but if they are, they don't usually consider using a computer program as part of that quest. For them the computer software is a necessary evil. They are looking for intuitive tools that require no learning. They tend not to invest the time to achieve expertise. If I were to equate using a computer to driving a car, the average computer user drives around in 1st gear or reverse all day long. They discover a way to get the car to move and then leave it at that.
There is a direct relationship between specialization and intuitiveness of software. When the software designer knows exactly what the user will use the software for, he can be very explicit in the user interface. For example, when creating a blogging tool, the software designer can put in a big button that says "CREATE BLOG ENTRY." A designer of a more generalized, multi-functional tool requires more compromise and negotiation with the user. The user needs to learn how to access lots of basic capabilities and string them together to get the result that he wants. Just look at the UNIX command line and piping together commands. TextMate is a little of both, the designer of TextMate knows that the user is going to want to enter text and save files. That is why the program opens with a big area to type in. But the designer doesn't know whether the user will be wanting to post this block of text to a blog or compile the text into executable software or hundreds of other options. This is why those functions need to be buried under cryptic key sequences like "control-command-p" (that's post to blog) or "command-r" (that is compile and run). If a CMS was written for someone that wanted to be a CMS expert, it would probably look something like a command line Sabre terminal. And this is why all purpose tools fail for content managers.
Chief Google Economist, Hal Varian, has an interesting post about online and offline newspaper economics on the Google Public Policy blog. Most of the ideas will be familiar if you read Clay Shirky: cross-subsidization of the news; specialized sites drawing away ad revenue; relative cost of production.
One point that I have been hearing less about was that online news consumption tends to be mostly from work while offline newspapers are read at home on leisure time. In itself, this is not a huge insight; all of my news clients know that they get most of their traffic during the workday. What I had not thought about was that reading time at work is significantly more compressed than at home. The article gives statistics of 70 seconds per day online vs. 25 minutes offline — and advertisers pay for a premium for that longer attention span. The article predicts some good news for newspaper publishers: tablets (like the iPad) and other mobile devices (like the Kindle) will increase the at home consumption of the news and lengthen time spent reading. This should even out the disparity between online and offline advertising revenue. I think the accuracy of this prediction will depend on a) whether advertising formats can effectively adapt to and leverage the strengths of mobile devices and b) the advertisers opinion of the value of online advertising changes. As for the latter, advertisers seem to illogically value the immeasurable benefit of print advertising over the more measurable benefit of online advertising. That is, they probably assume print advertising is more effective than it actually is because there are no statistics to limit the perceived value. There are some great posts about how online advertising is undervalued.. Until this attitude changes, it will be difficult for newspapers to burn their boats.
Recently it occurred to me that video and (to some extent audio) has become a less important requirement for most of my web content management clients these days. If I were to extrapolate the interest trend I was seeing back in 2004, I would expect to see the web resemble billions of tiny television stations. But that hasn't happened. While video and audio remain important for companies in the business of developing and/or distributing multi-media content (primarily for entertainment), multi-media has not achieved ubiquity as a medium for communication. Instead, the focus is still heavily weighted towards text and images. There are some obvious explanations for this trend. Services like YouTube, Vimeo, and BrightCove are taking care of video hosting so my clients don't have to worry about it. Also, companies have realized that, other than the large file sizes, video is just not so special that deserves so much special concern and attention.
But I think that there are larger forces at work. I experience issues with video both as a content consumer and a content management professional. As a content consumer, I don't like how video takes control over my experience of the information. Unlike text, you can't scan video for the little bit information that you want. You have to sit back and wait for the video to get to the point. This is great for entertainment (suspense!) but less so for information exchange. Take, for example, this video (below) where the spokesman just spurts out technical information about a computer. It would be so much better just to have a spec sheet so you could scan right to the part of the specification you want. For a while it seemed like all companies were looking to make their websites more "dynamic" by leveraging multi-media. Customers may have reacted positively at first to the novelty of seeing moving pictures but I think this has played out.
As a content management professional, my issue with video is that it breaks a key principle of content management: the separation between content and format. In video and audio, the information is inseparable from the format. Yes, you can transcode into different encodings and file formats, but reorganizing and editing audio and video is difficult to automate. As a result, video and audio content gets stale and out of date because it is too expensive to change. If you want to change the information, you need to re-produce the whole thing.
There are some things that belong in audio and video, like a recording of some performance or something that you need motion to understand. But in most cases, the production and maintenance effort that multi-media requires introduces an unnecessary barrier between the information source and consumer. This obstacle stands out even more in the social web that tries to blur the line between author and audience. In the social web, video is most successful when it is short, entertaining, and performance-oriented. It is less effective for basic information exchange. The best uses of video I have seen for information exchange is on some news sites where visitors are invited to upload short videos of their eye-witness account. In these cases, the video needs to be authentic (no need to edit!) and shows something that cannot be captured in text and still images. But information about that event is more accessible in plain text.
I am sure that there are people in the video field that have different experiences with the medium. If you have any insights, please leave a comment or send me an email.
The Open Source Way is a great resource for developers who want to start their own open source project. It is a wiki of lessons learned and best practices gathered primarily from experiences working on the RedHat and Fedora projects. One page that I find particularly interesting is "How to tell if a FLOSS project is doomed to FAIL." The format assigns "fail points" for common (and uncommonly stupid) mistakes that project leads make. Making a few of these mistakes doesn't doom your project but making lots of them definitely stacks the odds against you.
While it was written from the perspective of a Linux library developer, the list also contains useful information for other types of open source projects. I was hoping to add some content around project leadership but I was discouraged to see that I was not able to create an account or contribute to the discussion page (FAIL!). Here is what I would have added:
There is no road map showing planned improvements to the software
The project leader(s) is(/are) not transparent about decisions [ +5 points of fail ]
There are no guidelines for contributing to the project [ +20 points of fail ]
The project does not have a way to attract and incorporate new contributors [ +10 points of fail ]
The project does not have a system for recognizing/rewarding contributions [ +5 points of fail ]
