One of my major preoccupations is evaluating open source software (OSS) and the projects that develop OSS.  For my Fulbright project, I settled on a rough and ready set of evaluation criteria, but some circumstances demand more rigor.  Picking the wrong development framework or library, for example, could fatally wound an OSS development project.To help me and hopefully the Libraries, Archives, and Museum community as a whole) get a better handle on OSS evaluation methods, I wrote a small grant application to the University of Illinois Library’s Research and Publication Committee.

The project will develop and test a method for evaluating OSS projects (as opposed to the software itself).  In addition, it may yield other useful information, such as demographic and technical information about the user community, which can help in future development and sustainability planning for current projects.

Succinctly, the project will develop (based on existing literature and my past experience) and test a three-part method for describing and assessing the effectiveness of Open Source Software (OSS) projects: 1) an implementation analysis tool, for gathering and analyzing information form publicly-available (e.g. Internet) installations of OSS; 2) A thread analysis tool, for analysis of questions, bug reports, and feature requests on listservs and OSS forums; and 3) an implementers survey.  As a case study, the three tools will be applied to the Archon OSS project, and both the Archon project and the method itself will be assessed.  The method may also be with other projects.

If anyone would like to contribute to the project or receive more information about it, let me know in the comments section or by email.

Tagged with:  
  • Hi Chris,

    I have been working for months on a similar issue. I started by reading all possible open source methodologies and I eventually created my SOS Open Source, a pragmatic methodology supported by automated tools.

  • Chris Prom

    Thanks so much for pointing that out Roberto. We are very early in the project, so perhaps we can use your method, saving ourselves quite a bit of work. I’ll have Emily take a look at it, we’ll consult, and may get back to you with questions. Best Regards!