Proceedings of Workshop for NLP Open Source Software (NLP-OSS)

Abstract

Open-source software makes sophisticated technologies available to a wide audience.Arguably, most people applying language processing and machine learning techniques rely on popular open source tools targeted at these applications.Users may themselves be incapable of implementing the underlying algorithms.Users may or may not have extensive training to critically conduct experiments with these tools.As maintainers of popular scientific software, we should be aware of our user base, and consider the ways in which our software design and documentation can lead or mislead users with respect to scientific best practices.In this talk, I will present some examples of these risks, primarily drawn from my experience developing Scikit-learn.For example: How can we help users avoid data leakage in crossvalidation?How can we help users report precisely which algorithm or metric was used in an experiment?Volunteer OSS maintainers have limited ability to see and manage these risks, and need the scientific community's assistance to get things right in design, implementation and documentation.

Cite this work

No public copy is stored for this record, so only the metadata is cited.

Stored only when both papers are in the corpus — 19 citations exist upstream.

Papers that cite the same references — the relational fallback while embeddings are still backfilling.