To evaluate software testing and program analysis tools, the research community relies on collections of sample programs (benchmarks) containing realistic code examples with defects. We investigated 23 benchmark projects for test generation in common programming languages and looked at how they can be categorized according to attributes such as programming language, number of programs and defects, license, and size. From our studies, it is evident that the development and especially maintenance of benchmarks are a big challenge. Out of the 23 benchmark projects we investigated, only four are still active as of today, and only nine have been updated after their initial release. With the underlying programming languages and platforms constantly evolving (often without full backward compatibility), this creates a challenge when comparing new tools to older ones. To exacerbate the situation, many benchmarks do not fully track the provenance and license of the code they include. Sustainable benchmark collections share these key factors: Open hosting of complete (actual) data allowing community involvement, systematic maintenance of license and authorship data, and a unified machine-readable format for such data.