Artifact evaluation is an emerging topic in the scientific process. For that, several large conferences started dedicated artifact evaluation committees, and encourage authors to submit their research artifacts for evaluation.

However, in practice, artifact evaluation is often difficult. Authors grew accustom to ‘their own’ environment, posing challenges to reviewers. At the same time, reviewers may not have the resources, like special hardware or already properly configured GPUs, necessary for artifact evaluation. Also, it may not be an option to quickly create environments in which artifacts can be evaluated.

To lessen this burden, we provide an artifact evaluation VM integration for HotCRP, as a set of dedicated patches. You can either use these patches to get your own HotCRP connected to a virtualization backend (at the moment, only Proxmox VE is supported), or we can provide a hosted HotCRP instance with artifact evaluation enabled for you.