Open Source Software Communities

MIT Sloan School of Management

15.352

Karim R. Lakhani

February 16, 2005

THE BOSTON CONSULTING GROUP

 AGENDA

What is open source and why do people participate

• How does open source work?
• What motivates developers?

How does work get done?

 WHAT IS OPEN SOURCE?
Firm-based software development Open Source software development

Code developed Code developed

by firm by community

Code locked
via binary and Code distributed
sent to to users
customers

Customers Users create

run program binary

Bug Users run

program
complaints &
feature !
request to Users find
company and fix bugs,
and create
(Oops)
!
(Patch)
new features

Users distribute
modifications
-4-
 OPEN SOURCE PRINCIPLES

Intellectual property Development paradigm Resource model

Code should always be open - Extensive involvement of Good ideas come from solving
“Free speech, not free beer” user/developer community a problem or scratching an itch

“Copyleft” “Release early, release often” “The three obligations: to give,

to receive, to reciprocate”

C
C

“Use copyright to Modularize code Peer leadership -

ensure copyleft” vision, engagement, code

-5-
 HOW DOES OPEN SOURCE WORK?

Virtual teams

Provide “home” for

code

Establish team norms

Do Get
e New releases na co
od te de
t c co
de
Ge For-profit
Users Self Coordinate/ companies
Use software organize Influence OEM open source
User developers Leadership
Report bugs “Liberate” code
Scratch coding itch, Initial code base and
Support each Join solve problem vision Provide support
other and consulting
Implement features & Own and modularize
fix bugs kernel

Provide expertise Licensing

Proselytize
movement

Prepackage
open source
-6-
 LINUX DESIGN IS MORE EMERGENT THAN DIRECTED
Excerpts from Postings by Linus Torvalds

Rik van Riel:

“It seems like Linux really isn't going anywhere in particular and seems to make progress
through sheer luck”

Linus (in several emails in a longer thread):

“Hey, that's not a bug, that's a FEATURE! [his emphasis]

“Do I direct some stuff? Yes. But, quite frankly, so do many others. Alan, Al, David, even you.
And a lot of companies are part of the evolution whether they realize it or not. And all the
users end up being part of the ‘fitness testing’....

“A strong vision and a sure hand sound good on paper. It's just that I have never met a
technical person (including me) whom I would trust to know what is really the right thing to
do in the long run....

“Too strong a strong vision can kill you-- you'll walk right over the edge firm in the knowledge
of the path in front of you...

“I'd much rather have ‘brownian motion,’ where a lot of microscopic directed improvements
end up pushing the system slowly in a direction that none of the individual developers really
had the vision to see on their own.”
Source: Linux kernel email list
-7-
 MORE THAN 85,000 MESSAGES A MONTH
COORDINATE THE LINUX ENTERPRISE
User Development Extensions

Corporate redhat- suse-

list linux-e suse-linux
mailing lists suse-security

debian -devel -changes

Community
mailing lists debian -
linux “beer hiking club” devel
linux-newbie linux-raid

linux-kernel
debian -user
alsa-devel

[Link]
linux.
redhat.
install Corporate
bulletin boards
[Link].
mandrake
[Link].
advocacy [Link].
linux.
[Link]. hardware
linux.
networking
[Link].
[Link] [Link]
Community 1,000 Posts/month
bulletin boards

Note: Number of messages posted in June 2000 on 147 relevant bulletin boards and mailing lists (duplicate postings removed)
Source: [Link]
; [Link]; BCG analysis
-8-
 CRITICAL TO UNDERSTAND MOTIVATIONS AND
ORGANIZATIONAL STRUCTURE OF OPEN SOURCE

Motivations
• Are developers working for “free”?
• Why are they participating?
• What kind of effort are they contributing?

Organizational Structure:
• How are projects organized?
• What kind of structure enables dispersed, virtual collaboration?
• What are the lessons for firms?

Is the open source movement a fad or

is it sustainable?

-9-
 AGENDA

What is open source and why do people participate

• How does open source work?
• What motivates developers?

How does work get done?

- 10 -
 SURVEY OF PROJECTS ON [Link] TO
UNDERSTAND MOTIVATIONS IN COMMUNITY

Target Process Response

SourceForge

10% random selection of Sent personal

Received
alpha, beta, production email link to
526 responses;
projects (1); web-based
34% res. rate
1648 developers identified survey
SourceForge II

Mature projects with two or

Sent personal
more developers; Received
email link to
573 developers identified 169 responses;
web-based
30% response rate
survey

Results based on 684 usable responses

(1) Projects had 50% or greater activity level
- 11 -
OVERVIEW OF KEY FINDINGS ON HACKER MOTIVATIONS

Why should
we care?
High creativity

What motivates
hackers? ?
Fun, skill, Increasing knowledge Losing sleep
freedom and need biggest benefit biggest cost
8

Who are 4

these guys? 1

13

16

19

22

25

28

31

34

37

40

43

46

49

52
Volunteer significant time IT professionals Generation Xers

What about the

community?
Strong identification Global effort Peer leadership
preferred - 12 -
 OSS PROJECTS AND PROGRAMMING TURNS ON HACKERS

“Like composing
48.4% poetry or music”
“This project is as (or most)
61.7%
creative as anything I have done”

“When I program,
72.6%
I lose track of time”

“With one more hour in the day,

60.0%
I would spend it programming”

Note: “...like composing poetry...” answer chosen as one of top three attitudes by participants; other answers based on degree of
participant agreement with statement
- 13 -
 OVERALL HACKER MOTIVATIONS
?
Intellectually stimulating 44.9

Improves skill 41.3

Work functionality 33.8

Code should be open 33.1

Non-work functionality 29.7

Obligation from use 28.5

Work with team 20.3

Professional status 17.5

Other 16.3

Open Source reputation 11.0

Beat proprietary software 11.1

License forces me to 0.2

0 10 20 30 40 50

Percent of respondents
Note: Question asked for top three motivators of F/OSS participation, n=684
- 14 -
 VOLUNTEER CONTRIBUTORS
MAKE UP MAJORITY OF RESPONDENTS

Volunteer Paid

Percent of responses 60 40

Selection criteria

“Have you been financially No Yes

compensated in any way for
participating in this project?”

“Is your direct supervisor aware of your No Yes

project participation (during work
time)?”

- 15 -
 MOTIVATIONS DIFFER BETWEEN
?
PAID AND VOLUNTEER CONTRIBUTORS
Intellectually stimulating 46.6
40.8

Improves skill 46.2

30

Code should be open 34.8

29.1

Non-work functionality 34.5

18.7

Work functionality 21.8

62

Obligation from use 28.7

28.1

Work with team 22.5

15.3

Professional status 15.3

22.6
Volunteers
Open Source reputation 11.1
10.8
“Paid” (1)
11.1 to contribute
Beat proprietary software 11.8

0 10 20 30 40 50 60 70
(1) Includes those working on F/OSS project full time, part time, and those sanctioned by supervisors Percent of respondents
(2) Volunteers= 479, paid=205
Note: Question asked for top three motivators of F/OSS participation, n=684
- 16 -
 MOTIVATIONS AND CONTRIBUTION
STATUS SEGMENT HACKERS
“Community
“Hobbyists” (27%)
Believers” (19%)

Motivations

Do it because they feel

obligation and believe
source code should be open

“Professionals” (25%) ? Do it for non-work

“Learning &
Stimulation” (29%)

Do it for Do it for
work need skill improvement and fun

- 17 -
 OPEN SOURCE IS A GENERATION “X” PHENOMENON
Average Age: 30 Years
Percent of 7
respondents Average age: 30
Gen Y Gen X Boomers
13.7% 70.2% 16.1% Volunteers: 29
6
Paid: 32
5

0
14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 55 Age

And 98% male

Note: n = 677 total responses
- 18 -
 OPEN SOURCE IS A GLOBAL ENTERPRISE
Venezuela 1
Argentina 3 Hungary 4
Lithuania Taiwan 1
South
Brazil 9 Austria
Spain 55 1 South Korea 1 Africa
Latvia Singapore 1
Denmark 56 1
1
Vancouver 9 Montreal 2 Spain 7 6
Malaysia 1

Toronto 8 Canada Calgary 1

Switzerland
Belgium 8
Ireland
1
Japan 1
Indonesia 1
Morocco
1
Iceland
Belgium 6
Ottawa 3 39 Quebec City 1
Switzerland
Austria 6 10
1
Estonia
Hong Kong 1
China
Gabon
1 2
1

SF Bay Area14 Norway

Norway 11
9
Croatia
New
Madison 2 1
Zealand
Bulgaria Armenia
Boston 10 Minneapolis 2 Italy
Italy 1
4
1

Denver 10 Nashville 2 15
11
Belarus
1
Angola
Los Angeles 10 Providence 2 Sweden
Sweden Slovak 1
Rep.
Atlanta 6 Sacramento 2 15
12 2

Austin 6 Tampa 2 France Russia

2
Sydney 9
New York 6 Tulsa 2 Canberra 5
Baltimore 5 Ames 1
17
25 Israel
Portugal
Melbourne 5 3
Kansas City 5 Ann Arbor 1 2
Brisbane 2
Netherlands
Portland 5 Bozeman 1 Queensland1
Seattle 5 Charlotte 1
19
25 Poland
2
St. Louis 5 U.S. Cincinnati 1 London
London 16 16
Finland
Washington 5
Columbus 4
267 Cleveland 1
Ft. Lauderdale1
Leeds
Leeds 4 4
U.K. 2

Bristol
Bristol 2 2
Australia
Detroit 4 Gainesville 1 Manchester 2 2
Manchester 45
33 Czech Rep.
2
42
Milwaukee 4 Hartford 1 Edinburgh
Edinburgh1 1
Philadelphia 4 Huntsville 1
Slovenia
San Diego 4 Lansing 1 3 India
8
Dallas 3 Louisville 1
Munich
Munich 7 7 Aachen
Aachen 2 2
Houston 3 New Haven 1
Berlin
Berlin 6 6 Dusseldorf
Dusseldorf2 2
Indianapolis 3 New Orleans 1 Greece
Frankfurt 5
Frankfurt 5 Heidelberg2
Heidelberg 2 3
Pittsburgh 3 Orlando 1
Stuttgart
Stuttgart 5 5
Germany Cologne
Cologne 1 1
Phoenix 3 Richmond 1
Salt Lake City3
Nuremberg
Nuremberg4 4 77
62 Hannover
Hannover 1 1
San Antonio 1 Romania
Hamburg 3
Hamburg 3 Leipzig 1
Leipzig 1 4
Chicago 2 Syracuse 1
Lexington 2

Americas Europe ROW

46.9% 42.4% 10.7%
Note: n = 684 total responses, ROW = Rest of the World
- 19 -
 RESPONDENTS VOLUNTEER A LOT OF TIME

“This” project All projects

Overall mean=7.5 hours/ week Overall mean= 14.09 hours/ week
Volunteers=5.8 hours paid= 11.4 hours Volunteers=13.5 hours paid= 20.9 hours
Percent of 40 35
respondents
35 30
30
25
25
20
20
15
15
10
10

5 5

0 0
0 1 to 5 6 to 13 to 21 to >40 0 1 to 5 6 to 13 to 21 to >40
12 20 40 12 20 40

Hrs/week Hrs/week

Note: n=684
- 20 -
 CONTRIBUTE TO MANY PROJECTS

Current project All projects

Mean = 2.6 Mean = 4.9
Volunteer = 2.4 paid = 3.0 Volunteer = 4.5 paid = 5.8
Percent 35 35
of respondents
30 30

25 25

20 20

15 15

10 10

5 5

0 0
0 1 2 3 4+ 1 2 3 4 5 6+

Number of projects Number of projects

Note: N = 684
- 21 -
 PARTICIPANTS ARE MOSTLY
EXPERIENCED IT PROFESSIONALS
Current occupation
Percent of 50
respondents 45.4
45

40

35

30

25
19.6
20
14.8
15

10 7.0
6.3 6.3
5

0
Programmer Sys. Admin. IT Manager Student Other Academ.

Average 11 years of programming experience

Note: n=678
- 22 -
 PROJECT CREATIVITY LARGEST DRIVER OF EFFORT
Regression on Project Hours/ Week

What is significant? What is not?

+ Creativity on project Age

+ Professional status (1) IT Job
Hacker affiliation
- IT training (1)
Founder of project
Prior social connection
USA based
Work functionality
Non-work functionality
Intellectually stimulating
Improves skill
Work with team
Code should be open
Beat proprietary software
Community reputation
Obligation from use

(1) Volunteers only

- 23 -
 HIGH PROJECT CREATIVITY DRIVES
HOURS CONTRIBUTED

Volunteer Paid

Average hours/ week contributed 5.8 11.4

Impact of unit change in creativity 3.3 6.3

(scale: 1- much less, 2-somewhat less,
3-equally, 4-most creative)

Anticipated hours with one unit 9.1 17.7

increase in creativity

Percent increase in hours 57% 55%

- 24 -
 SUMMARY OF COMMUNITY MOTIVATIONS

No single motivation driving community participation

• Community is a “Big Tent” – participants can contribute for any reason
• Professional needs, community motivation, fun and learning and hobby
are the primary types of motivations for participation

Negativity towards commercial software developers not a prime mover

Feeling creative biggest predictor of incremental effort on projects

- 25 -
 AGENDA

What is open source and why do people participate

• How does open source work?
• What motivates developers?

How does work get done?

- 26 -
OPEN SOURCE COMMUNITIES FACE UNIQUE CHALLENGES

“Voluntary” participation

Dispersed contributors

Part-time and intermittent participation

Lean communication & collaboration technology

Rudimentary management tools

“Missing” project managers

So How Do They Get Work Done?

- 27 -
 Research Questions For My Dissertation

What are the specific practices for distributed development used by

Open Source communities?

How do these practices work and how are they inter-related?

How similar and different are the practices used by Open Source
projects to firm-based development?

- 28 -
 Dissertation Analyzes Practices From Two Cases

Characteristics PostgreSQL Cocoon

Technical
• Area Database XML web dev. environment
• Rate of change in technology Stable – mature technology Fast changing & new
• Lines of code ~500, 000 LOC ~1,000,000 LOC
• End-users Database administrators Web developers
• Competition Oracle, IBM, Microsoft, MySQL ???

“Organization”
UC Berkeley academic project User need from Apache project
• Origins
• Time active (1984-1996)1996 2000

• Steering committee size 5 32

• Commit access 12 60

• Development list size 3,039 1689

• Affiliation None Apache Software Foundation

- 29 -
 Grounding The Model In Actual Data:
Building A Process History Of Each New Feature In PostgreSQL
(7.3 -7.4, November 2002-2003)

Major Changes Detailed Changes CVS Commits E-mail Messages

• 17 major changes • 375 detailed • ~1000 CVS •“Hackers” (16,797

between 7.3 – 7.4 changes commits messages, 3,332
(as seen in release •140 detailed • CVS commit info = threads, 798
notes) changes constitute Committer, Date/Time. authors)
major changes Location, LOC+/-, •“Patches” (3,332
Files& Comments messages, 806
threads, 188
authors)

Follow-up with E-mail, telephone, IRC and face-to-face interviews

- 30 -
 Study Design And Data

Inductive grounded theory building of Open Source development

process
• Virtual ethnography of two communities (PostgreSQL & Apache
Cocoon) – 8 months
• Extensive interviewing of project participants
• Analysis of e-mail and source code change archives
• Analyzing one year’s worth of complete technical activity
(~15,000 e-mails and ~2000 source code changes) for each
project to build innovation history of each new feature
• Have developed a preliminary model of development practices

- 31 -
 User/Developer(s) Creates New Feature

Mario Weilguni (user/developer) outlines a problem

that he faces with database vacuuming (Sept 3, 2002)

9 other individuals participate in the discussion

about this need and possible solutions – general
agreement that it would be good to do – Tom Lane
(steering committee) makes several technical
suggestions (Sept 3, 2002)
Shridhar Dahitankar (new user/developer)
announces that he is going to work on this but
needs some information (Sept 3, 2002)
Matthew O’Connor (new user/developer) also
announces that he is going to work on the same
topic (Sept 3, 2002)

Shridhar Dahitankar – announces that the code is

ready based on prior discussion (Sept 23, 2002)

Other people give feedback and Mario reveals an

attempt at the same feature (Sept 24, 2002)
Matthew O’Connor asks some coding questions
(September 24, 2002)

Matthew O’Connor announces a “new & improved”

version of the Shridhar’s code (November 26, 2002)

Shridar Dahitankar asks questions and makes

suggestions (November 27-28, 2002)

- 32 -
 Preliminary Inductive Model Of Open Source Practices That
Substitute For “Standard Management”
Practices
Minimal
Verifiability
Enabling
Conditions
4
Low Cost
Modularity Action Before Legend
Reversibility
Explicit
Coordination

3
Optimistic Micro
Concurrency Contribution
2
Task Self-
Selection

Community
User need
Need

1
Project 6 7 8
Transparent Repository: Lazy and High
Open Lazy
Knowledge (Code, Distributed Energy
for “all” Consensus
Broadcast Bugs, E- Evaluation Resolution
5 mail, Wiki) Decision Escalation
Continuous
Integration
- 33 -
 Many Thanks To:
Eric von Hippel
Wanda Orlikowski
Leslie Perlow
Carliss Baldwin
Georg von Krogh
Stefano Mazzocchi
Ben Hyde
Brian Behlendorf
Neil Conway
Luis Villa
Bruce Momijian
Guido van Rossum
Anthony Baxter
LMU-MIT User Innovation Workshop
MIT Sloan Doctoral Seminar in Strategy
+ numerous participants in Python, PostgreSQL, Apache, and Freenet projects
- 34 -