The Chan Zuckerberg Initiative’s core values center around people, technology, collaboration, and open science. We adhere to those values in both proposal selection and evaluation of progress.
Applications will be evaluated for their expected impact, the quality of the open source software project(s) involved, the feasibility of the proposal, and their diversity, equity, and inclusion statement—each of which will be assessed through quantitative and qualitative factors. Relevant materials will be provided by the applicants and obtained by CZI from publicly available sources where possible (e.g., GitHub or other public code repositories). Quantitative metrics on software projects will be optional in the application form. Independent expert review will be solicited, and final decisions will be made by CZI staff in consultation with our expert advisors.
Impact will assess the importance of the open source software project(s) involved in the proposal to science and the open source ecosystem, in alignment with our mission to support the science and technology that will make it possible to cure, prevent, or manage all diseases by the end of the century. Reviewers will evaluate:
- Demonstrated scientific impact of the software project,
- The value to the software project provided by the proposed scope of work,
- The role of the software project in the scientific open source ecosystem, and
- Alignment of the software project to areas currently prioritized by CZI Science in accordance with our mission (e.g. imaging, cell biology, genomics).
Alongside qualitative materials, expert evaluation may utilize metrics such as:
- Number of users and recent growth,
- Adoption within relevant communities,
- Number of citations or mentions of the software project in scientific literature,
- Number of potential contributors and diversity of the organizations they represent, and
- Number of past contributions to related software projects in the relevant software stack (pushing changes upstream to dependencies, receiving changes from other nearby software projects).
Quality will assess the maturity of the software project(s), its compliance with best practices in open source development, and the existence of a healthy and diverse contributor community. It will again be assessed qualitatively and quantitatively. Reviewers will evaluate:
- Composition and leadership of team,
- Governance structure of the software project,
- Software project communications and community engagement,
- Existence, clarity, and recency of software project roadmap,
- Clarity of process for external contributions,
- Evidence of external contributions from outside of the core developer team (in the form of code, bugs/issue reports, documentation, etc.),
- Availability of tutorials and examples, and
- Quality and comprehensiveness of documentation.
Alongside qualitative materials, evaluation may leverage metrics such as:
- Frequency and growth trajectory of commits over time,
- Size and diversity of current developer team,
- Frequency of external contributions,
- Number of open issues, and rate of issues both opened and closed, and
- Time between opening and closing of pull requests.
Feasibility will assess the plan of work described in the proposal and whether it can be accomplished given the requested budget and key personnel involved. Reviewers will evaluate the following based on qualitative materials:
- Specificity and clarity of plan of work to be accomplished,
- Proposed use of funds (relative to plan of work),
- Likelihood of the work being accomplished,
- Plan for tracking and validating progress against goals,
- Degree of unmet need given existing resources, and
- Future plans for sustaining or maintaining the work funded by the grant.
In addition to the above, we are asking all applicants to include in their proposal a Diversity, Equity, and Inclusion (DEI) statement, describing any efforts the software project(s) named in this proposal have undertaken to increase diversity, equity, and inclusion with respect to their contributors and audience. This statement will be reviewed alongside the above criteria. Please see examples of DEI statements from successful first cycle applications.
There is no expectation of any specific number of awards, and the Chan Zuckerberg Initiative reserves the sole right to not recommend the funding of any applications. CZI does not provide individual feedback on decisions for unfunded proposals.
Reporting & Progress
Final reports, including a summary of the project progress that will be made publicly available, will be required from successful grantees to ensure that progress toward the deliverables described in the proposal is on track. Measures of progress will also include additional indicators on the growth and uptake of the software project, as obtained from code repositories, issue trackers, package distribution systems, community forums, mailing lists, etc.