Common approaches that sometimes end poorly
We list a few common approaches, and some challenges or potential outcomes.
1. Open your existing stack for community development
You’ve heard that by placing your code and development cycle on Github that your software will be adopted by other groups and that you’ll be able to share maintenance costs. You don’t consider your software to be intellectual property so you go ahead and hope for the best.
Positive Outcome: Your existing user-base may appreciate this and some of them may start submitting bugs and even patches.
Negative Outcome: Your software stack is already quite specific to your domain. It’s unlikely that you will see the same response as a general purpose project like Jupyter, Pandas, or Spark.
Additionally, maintaining an open development model is hard. You have to record all of your conversations and decision-making online. When people from other domains come with ideas you will have to choose between supporting them and supporting your own mission, which will frequently come into conflict.
In practice your engineers will start ignoring external users, and as a result your external users will stay away.
2. Start a new general purpose open source framework
Positive: If your project becomes widely adopted then you both get some additional development help, but perhaps more importantly your institution gains reputation as an interesting place to work. This helps tremendously with hiring.
Negative: This is likely well beyond the mandate of your institution.
Additionally, it’s difficult to produce general purpose software that attracts a broad audience. This is for both technical and administrative reasons:
- You need to have feedback from domains outside of your own. Lets say you’re NASA and you want to make a new image processing tool. You need to talk to satellite engineers (which you have), and also microscopists, medical researchers, astronomers, ecologists, machine learning researchers, and so on. It’s unlikely that your institution has enough exposure outside of your domain to design general purpose software successfully. Even very good software usually fails to attract a community.
- If you succeeded in a good general purpose design you would also have to employ people to support users from all of those domains, and this is probably outside of your mandate.
3. Pull open source tools into your organization, but don’t engage externally.
You’re happy to use open source tools but don’t see a need to engage external communities. You’ll pull tools inside of your organization and then manage and distribute them internally as usual.
Positive: You get the best of software today, but also get to manage the experience internally, giving you the control that your organization needs. This is well within the comfort zone of both your legal and IT departments.
Negative: As the outside world changes you will struggle to integrate these changes with your internal processes. Eventually your version of the software will diverge and become an internal fork that you have to maintain. This locks you into current-day functionality and puts you on the hook to integrate critical patches as they arise. Additionally you miss out on opportunities to move the software in directions that your organization would find valuable.
4. Get out of the business of maintaining software entirely.
Your institution may no longer be strictly needed in this role.The open source communities seem to have a decent pipeline to support users, lets just divert our userbase to them and focus on other efforts.
Positive: You reduce operating budget and can allocate your people to other problems.
Negative: This causes an upset to your employees, your user community, and to the maintainers of the open source software projects that are being asked to absorb your userbase. The maintainers are likely to burn-out from having to support so many people and so your users will be a bit lost. Additionally the open source software probably doesn’t do all of the things that the old software does and your existing users aren’t experts at the new software stack.
Everyone needs help getting to know each other, and your institution is uniquely positioned to facilitate this.