Development: Write white paper/review paper about best practices in developing sustainable software

[danielskatz]

No description provided.

[dangunter]

Discussed this in Software Engineering Management Committee in CRD at LBNL, and one thing that came from that discussion was that the white paper might differ depending on whether it is aimed at the practitioners or funders.

[sandragesing]

Abani Patra and I worked on following topics during the first day of WSSSPE3:

Topic 1: Best Practices in Developing Sustainable “Scientific” Software

People: Abani Patra, Sandra Gesing, Neil Chue Hong, Greg Tucker, Birgit
Penzens, Abigail Cabunoc Mayes, Jeff Carver, Frank Loffler, Colin Venter, Lorraine Hwang,
S.C. Choi, Suresh Marru, Don Middleton, Daniel Katz

Post-it Summary:
o Birgit – needs to talk!
o Go beyond scientific computing
o Creation of Software sustainability teams
o Is software death an emergent property of complex ecosystem
o Unpaid labor – github as resume
o Define Sustainability in the context of Software for the sciences
o How to make sure established players do not stop the new ideas and
o Top down – bottom up and long tails
o Embedding scientists with Software team

Six pack
o What did you talk about? – WE NEED MORE PEOPLE WRITING
o What are the Current Barriers? -- Loss of interest AT; Diffuse
tools – “CAREER” program for new tools?
o Is there an obvious next step? – get the co-authors together; agree on
o Are there three or more people that want to work on this? 2 cat 1; 11
o Any splits etc --? None yet!
o Any expertise needed? Lots! Need participation from research software
literature; Focus of the document; Define the target audience? Outlet?
Timeline?
high level specs of questions above in barriers – focus, target, outlet
and timeline;
in cat 2
engineers, scientists (esp. PIs of large science efforts), sustainability
expert, early career scientific software personnel

Essentials of Sustainability – courtesy B.
o Sustainability in context – define on per project, per discipline?
o Sustainable software characteristics – well defined? Need to study
some “sustained software examples” and abstract. – knowledge mgt
(transitions between people), reliability (reproducibility), usability
and part of core workflow of well identified users (stakeholders)
o Requirements eng.
o When should s/w “die”?
o Disruptive development? Environment (new h/w, new methodology,
o Dimensions of sustainability – economic, technical, environmental,
declining interest in primary application area), social

[jedbrown]

Regarding barriers for entry, one partial solution is that if an existing package provides a good user interface and plugin system, then new ideas can be shipped to users without asking the user to invest in significant retooling or workflow modification. Similarly, if an existing non-extensible tool has a well-defined interface, then a new tool can be made to implement that interface.

My perception of the status quo is that most legacy scientific packages have obtuse ill-defined interfaces and poor extensibility, but there are exceptions and the younger generation has been making great strides in this area. We need to emphasize that more.

[jedbrown]

CiSE paper we just discussed:
https://jedbrown.org/files/BrownKnepleySmith-RuntimeExtensibilityAndLibrarizationOfSimulationSoftware-2014.pdf

[sandragesing]

Outline for the white paper:

1. Introduction and Scope of White Paper [Abani/ Sandra, ...]
2. Related Work
3. Case Studies
   3.1 PeTSC
   3.2 NWCHEM
   3.3 CIG
4. Community Related Practices
   4.1 Findings
   4.2 Recommendations
5. Governance and management
   5.1 Findings
   5.2 Recommendations
6. Funding Related
   6.1 Findings
   6.2 Recommendations
7. Metrics for sustainability
   7.1 Findings
   7.2 Recommendations
8. Tools
9. Conclusions

[sandragesing]

Suggestion for the timeline:
[15 Nov] Introduction and scope finished
[15 Nov] Sections assigned
[31 Jan] Analysing funding possibilities for survey
[31 Jan] First versions of section
[15 Feb] Distribution to WSSSPE community
[31 Mar] Final version of white paper
[30 Apr] Submission of peer-reviewed paper?

[kyleniemeyer]

While I didn't participate in this group at WSSSPE, I am interested in being involved in this effort.

[npch]

I'm interested in contributing to this white paper.

[knarrff]

Me too holding my hand up

[JeffCarver]

I am interested in contributing to this white paper

[iliant]

Interesting effort. As a legacy software owner I can say that we have not envisaged that our software could or would be used as a plugin. Although nowadays with multiscale as a buzzword we are contemplating software pieces being bits of researchers' work-flows.

[sandragesing]

Many thanks to all who wants to contribute. I will send an email in the next days.

[dangunter]

pls include me too, thanks.

[olexandr-konovalov]

While I am not at WSSSPE, I'd be interested in contributing too.

[tomcrick]

Ditto -- I'm not at WSSSPE this time, but very keen to contribute.

[olexandr-konovalov]

@tomcrick There may be a chance for some of us to meet at http://www.software.ac.uk/software-credit and http://sustainabilitydesign.org/ssse15/ in UK in October.

[abani1]

Great! Looks like we will have a good set of people ready and willing to go! Some reading that you may or may not have seen https://www.nitrd.gov/csessp/materials/csessp_reading_resources.pdf

[migueldvb]

I'm interested in contributing to the best practices white paper

[bboscoe]

I am interested in this as well! Lovely to meet you in Boulder.
--Bernie

[kreardon]

Please include me in the list of possible contributors.

[ashleysa]

Sorry I was unable to attend WSSSPE, but I am interested in contributing.

[acelster]

I am also interested in contributing.
Anne C Elster
On Oct 1, 2015 12:17 PM, "Ashley E. Sands" [email protected] wrote:

Sorry I was unable to attend WSSSPE, but I am interested in contributing.

—
Reply to this email directly or view it on GitHub
#42 (comment).

[sandragesing]

Great that so many people are interested in contributing! To have all email addresses and names please add your name in the Google doc "Best Practices for Developing Sustainable Scientific Software" under this link as soon as possible. That makes it easier for Abani and me.
https://drive.google.com/drive/folders/0B7KZv1TRi06fbnFkZjQ0ZEJKckk
It would be also great if you would like to add your name to the section you are interested in. Further hints are in the Google doc. There is also a timeline document. Abani and I will work on the introduction and contact you by Sunday.

[migueldvb]

If the final version is going to be written in Latex for submission to a journal, it may be a good idea to start a repository under https://github.com/WSSSPE for the paper.

[sandragesing]

Dear all,
sorry for the long silence and the delay. Let's work on getting this done before WSSSPE4 takes place;-) I have created the repository for the Latex version https://github.com/WSSSPE/meetings/tree/master/WSSSPE3/best_practices. You can add your name as co-author there and indicate here in the discussion whether you would like to lead a section.
I will contact interested people also to assign tasks. Let's aim for having a first version by mid June. Thanks!

[sandragesing]

Anyone who is interested in contributing and hasn't received an email from me may contact under [email protected]. I couldn't find out your email address then;-)

[JeffCarver]

@sandragesing I tried to add my name to the author list for best_practices, but it wouldn't let me push the change back to the repository. Please add me. Thank you.

[tomcrick]

@sandragesing Ditto! (or should we submit a PR?)

[sandragesing]

I prefer that you all get the rights and that you can push to the repository. I will come back to you soon when I know whether you will get the rights (I'm not the owner of the repository) or I will create another repository. Thanks!

[abani1]

Sandra
Might be easier to create a new one.

Sent from my iPhone

On May 19, 2016, at 7:44 AM, sandragesing [email protected] wrote:

I prefer that you all get the rights and that you can push to the repository. I will come back to you soon when I know whether you will get the rights (I'm not the owner of the repository) or I will create another repository. Thanks!

—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub

[sandragesing]

I have created a new repository at https://github.com/WSSSPE/WG-Best-Practices and now I can add you all as contributors. I hope I caught everyone - if you are missing access to the new repository, please contact me!

[sandragesing]

Everyone who is interested in contributing to the paper and has not access to the repository under https://github.com/WSSSPE/WG-Best-Practices yet: Please contact me by 27 May to be added!