L’Open Source en Entreprise
Par Bernard Golden, Navica Software (en anglais)

Just as Willie Sutton, famous bank robber of the 30's responded, when asked why he robbed banks, "that's where the money is," enterprises represent the vast majority of IT "throw weight" in our economy. While Google, Oracle, and Microsoft garner the lion's share of awareness of technology happenings, enterprises represent the vast majority of technology investment in the world.

Consequently, if one wants to understand the reality of technology, one needs to look at what enterprises are doing. Conversely, a technology cannot really be said to be adopted until enterprises are actively using it.

Which brings up a funny issue. Despite all the attention paid to open source and the enterprise (including my Herculean efforts), the topic remains the province of opinion and anecdote. While some, myself included, believe that enterprises are actively using open source, others vociferously disagree. Here is one comment recently submitted to a relatively innocuous blog posting I made noting that open source use by an enterprise is a positive factor in recruiting, since prospective employees want to use skills that are transferable to other companies, thereby increasing their marketability:

"I work in an IT Group of over 2000 supporting a rather large company. Open Source isn't even on the radar and won't be for many years. We have no trouble recruiting talent world wide.

"Author is expressing an opinion not supported by the market or facts. Open Source is basically a flea speck on the IT world, not something we consider important, let alone critical.

"If you want a job, learn SAP, Oracle or another major technical solution. Open Source is typically low end, small businesses with a few exceptions. Even in the exceptions, proprietary software rides on the open source rendering it a truly closed architecture."

Certainly no doubt in his mind!

Unfortunately, the blog conversation then got off track, with someone from a commercial open source company noting that $115 million of venture capital was invested in open source last quarter, proving that open source is important. The commentary then deteriorated into a back and forth between them, one maintaining that his "really big company" spends billions of dollars each year, none of it on open source, and the other that he was bravely advancing innovation by selling commercial open source.

Frankly, many of the commercial open source companies do no service to the cause. Just because a venture capitalist invests money is no indicator of the validity of a sector (indeed, many might consider it a contra-indicator of validity, viz Webvan, Pets.com, et al). Moreover, too many commercial open source companies persistently and plaintively yammer that the only way an enterprise should use open source is with the comfort of, guess what, a subscription from the vendor. I've written about this a number of times, but commercial open source skims ubiquitous adoption, and the message from commercial open source vendors should be about generating adoption - in fact, adoption of open source of any kind - since that will speed ubiquity. Talking about open source and how great it is and then immediately following up with a "and you should buy from us to be safe" doesn't ease concern -- it increases it.

I understand the plight of the commercial vendors. They're under pressure from their investors to show progress. But their unending mantra that using open source means buying a subscription (not to mention the unstated implication that community open source shouldn't be touched) reminds one of Winston Churchill's definition of a fanatic: someone who can't change his mind and won't change the subject.

Nevertheless, the original comment from the "big company guy" does capture one truth: there aren't hard stats on enterprise open source adoption.

"One truth: there aren't hard stats on enterprise open source adoption"

While IDC does a yearly survey regarding open source adoption, the accuracy of their findings is suspect: what is the sample pool, do they probe on other open source products beyond Linux, do their survey subjects even know about what open source is currently used, given its typical invisibility, etc., etc.

So does it all come down to opinion and taunting?

Fortuitously, just before this verbal fusillade, O'Reilly Media approached me about authoring an "Open Source in the Enterprise" report, with an idea about how we might generate some actual statistics on enterprise open source use. So we agreed to partner on putting the report together, using a combination of experience, research, interviews, and innovative data mining.

The report was released at OSCON two weeks ago. It's generated quite a bit of attention. One of the most complimentary (and perceptive, naturally) reviews is here.

The report is broken into three parts:

Enterprise open source adoption drivers: We identify six drivers for open source adoption: (1) agility and scale; (2) breaking vendor lock-in; (3) quality and security; (4); cost; (5) sovereignty; and (6) innovation. Most of these I've discussed in previous newsletters, so I won't discuss the report's findings here, even though it addresses these topics in greater depth than any of the newsletters have. If you'd like to learn more about what the report has to say on these topics, look in the "Navica News" section of this newsletter for a link to a page offering an extract download as well as a discount code for the entire report if you're interested in purchasing it.

Enterprise open source adoption statistics: We approach this topic from a different direction. Rather than asking enterprises what they're using (fraught with all the issues listed above), we looked at a direct indicator of adoption: hiring. O'Reilly Research has data from SimplyHired, which scrapes job postings throughout the web. We took a subset of that data, representing Fortune 1000 companies, and data mined it for job listings asking for open source skills. I'll discuss this more in a moment.

Enterprise open source action plans: Information without direction is meaningless. Once you understand something, the issue is what do you do with that understanding. Recognizing that, we provide three action plans: (1) early open source adopters - what to do to formally manage open source use; (2) mainstream open source use - how to more effectively leverage open source throughout the enterprise; and (3) innovative open source use - how to create new business offerings and leverage partners and communities for business advantage.

Let's focus on the adoption statistics, since they tell a really interesting tale.

First off, job postings are a really concrete indication of use. You're unlikely to hire specific skills unless you are using or planning to use (in the very near future) products you're recruiting for.

"Job postings are a really concrete indication of use. You're unlikely to hire specific skills unless you are using or planning to use (in the very near future) products you're recruiting for. "

This is especially true in today's economy: nobody hires to build bench strength for some possible far-off need. It just doesn't happen.

So looking at job recruiting is a very accurate indicator of open source use. Furthermore, the skills being recruited for give a very accurate picture of what open source products are being used, since you don't recruit a struts person if you're implementing SugarCRM.

Our methodology was straightforward: we had a list of job postings from F1000 companies. These listings included not only IT jobs, but every other type of job as well, whether sales representative, janitor, or accountant. We created a list of open source products - over one hundred - that would indicate a job was open source-oriented. Then we searched the listings for jobs mentioning those terms (technically speaking, we defined a job as open source only if it included two or more of the terms; that way, we eliminated jobs that might be, say, an Oracle DBA who was expected to run the product on a Linux box).

So what did we find in the research?

There's more open source in the enterprise than most people - including us - expect. To begin with, we had a bit of a false start in which we found that nearly 2% of all F1000 jobs were open source, which seemed way too high, given that IT only represents about 2.3% of all F1000 employees. It turned out that a number of the companies in the sample are high-tech - Google, Yahoo, etc. Some of them had nearly 40% of their jobs being open source, which skewed the overall sample.

We then segregated the sample into two pools: high-tech and, well, non-high-tech, and focused on the latter; remember, that represents the vast majority of IT activity in the economy. When we did that, we found that open source jobs represented about .2% of all jobs, indicating that open source accounts for roughly one in 10 IT jobs. So much for the opinion of the "big company guy"!

Now, that figure is subject to some variability. There might be jobs that list two or more open source skills, yet are really focused on proprietary software; for example, a DBA job might mention MySQL and PostgreSQL but really be a SQLServer position. On the other hand, job recruiting does not address any internal employee job migration from proprietary to open source - a .NET developer becoming a PHP developer, for instance.

Consequently, we concluded that somewhere between 5% and 15% of all IT jobs are open source - significantly more than we expected going in. Certainly, this means that open source is a significant, albeit minority, presence in the IT infrastructures and practices of large enterprises. Definitely, this means that the disdain so many CIOs evince for open source is misplaced - based upon the circumstances of their own organizations. I'll return to this last point at the end of this section of the newsletter.

Just as fascinating as the level of open source employment was the breadth of use. By no means was this all Linux and MySQL employment.

"Just as fascinating as the level of open source employment was the breadth of use. By no means was this all Linux and MySQL employment."

What can we learn from the products represented in the list? First and foremost, open source content management is growing rapidly, representing three of the top four products in terms of growth. Dynamic languages are well-represented in the top 20 list as well, while Java is (perhaps) also strong, given the presence of eclipse and netbeans in the top 20. All of the top 10 products showed greater than 100% growth of job recruitment over the past year, while numbers 11 through 20 averaged somewhere around 60% growth.

Overall, the data mining indicates that open source is alive and thriving in enterprise IT shops today.

It's not all cheese and beer, however. Coincidentally, just before OSCON, Goldman Sachs released a survey of 2009 CIO priorities. Top of the list: server consolidation and virtualization. Bottom of the list: open source.

On the face of it, this latter fact seems contradictory. If something makes up 10% of your workforce isn't it de facto important? Why would it be last on your list of priorities? What accounts for this paradox?

Here are four reasons:

• First, open source is a technology, not a category. When CIOs think about what's important, they think in terms of categories: security, data center consolidation, and so on. To them, open source is a buzzword, not something that helps them solve today's problems.
• Second, there's culture. Open source has zealots. It doesn't come from big companies. You hear about "community." All of this is disquieting and you wish it would go away.
• Third, there's lack of caring. I had lunch with a couple of colleagues today and was discussing this paradox of CIO attention. Their response: what's the big deal about open source? It's all software that's pretty similar to the proprietary equivalent, viz MySQL and Oracle. Open source is just a lot cheaper. This, of course, ignores the evidence that when CIOs are queried about open source, they don't say it's unimportant because it's just like everything else, just cheaper (i.e., they're completely comfortable with it, so of course they don't consider it a priority). In fact, when questioned, CIOs typically respond with a litany of shortcomings of open source (e.g., no one supports it, not everything is free, you know -- you still have to pay people, and so on). On further discussion, my colleagues agreed that maybe open source is a different animal from the perspective of a CIO, so de-emphasizing it isn't because of familiarity and comfort.
• Fourth, CIOs don't know they're using open source. There's a funny anecdote in the report, which comes from the people at Sourceforge. They were meeting with a big company CIO, who asserted that they did not use open source. None, nowhere. "What, not even Apache or Tomcat?" "Sure, those are great products." "Where do you get them?" "Why, from the Apache Company."

Still and all, it seems strange that something accounting for 10% of your employees is so unimportant. I chalk it up to a combination of inattention to what's actually happening in the bowels of the organization and a cultural squeamishness about the topic. Cognitive dissonance is alive and well in the office of the CIO.

Truthfully, I think this is the biggest issue hindering broader open source adoption. Open source is very different than big enterprise software companies, and it's going to stay that way, since it will never have the sales/marketing dollars to invade and conquer CIO minds. Failing to recognize their own inappropriate emotional reaction to the subject of open source, CIOs are absolutely failing in their charter: to apply technology in inventive and cost-effective ways. This isn't an open source issue, it's an irrational behavior issue, but the result is that open source adoption is hampered.