For the first time, SPIN 2024 introduces an artifact evaluation (AE). For all accepted SPIN papers, authors are able to submit an artifact to substantiate the claims made in the paper. Participation in the AE is optional, but strongly encouraged. The outcome of the voluntary artifact evaluation will not alter the paper acceptance decision, but a positive outcome increases its credibility.
An artifact is any additional material (software, data sets, machine-checkable proofs, etc.) that substantiates claims made in the paper and ideally renders them fully replicable. For example, an artifact might consist of a tool and its documentation, input files used for tool evaluation in the paper, and configuration files or documents describing parameters used in the experiments.
The Artifact Evaluation Committee (AEC) will read the corresponding paper and evaluate the artifact according to the following criteria:
The evaluation will be based on the EAPLS guidelines, and the AEC will decide which of the badges — among Functional, Reusable, and Available — will be assigned to a given artifact. The corresponding badges can be added to the title page of the paper in case of acceptance.
Artifact submission is handled via EasyChair. Select the SPIN 2024 Artifact Evaluation track and provide the following information:
CertUtil -hashfile <file> SHA256
shasum -a 256 <file>
The artifact should be a self-contained archive that allows the AEC to evaluate the artifact on either the TACAS-23 virtual machine (VM) or as a Docker image. Authors should test their artifact on the VM or Docker prior to the submission and include all relevant instructions. Instructions should be clear and specify every step required to build and run the artifact, assuming no specific knowledge and including steps that the authors might consider trivial.
In particular, the artifact should contain:
License.txt containing the license for the artifact. The license must at least allow the Artifact Evaluation Committee to evaluate the artifact w.r.t. the criteria mentioned above.
We recommend to prepare the artifact based on the TACAS 2023 Artifact Evaluation Virtual Machine for VirtualBox.
The virtual machine is based on an Ubuntu 22.04 LTS image with the following additional packages:
ruby, and a 32-bit
VirtualBox guest additions are installed on the VM; it is therefore possible to connect a shared folder from the host computer.
If necessary, we allow reasonable network access from the VM.
However, we strongly encourage authors to supply a completely self-contained artifact to ensure long-term availability.
The instructions must include all necessary steps for the installation and setup on the “clean” TACAS-23 VM. In the best case, the authors would set up a script which performs all or almost all steps for the setup of their tool in the VM automatically.
In particular, authors should not rely on instructions provided by external tools.
An artifact based on Docker should come with a Dockerfile which automatically sets up the artifact from a base image such as
For long-term availability we recommend to support use of the Docker image without internet access, for instance by also providing the Docker images as exported by
In the following, we list some general suggestions for preparing the artifact:
(all times are 23:59 AoE)
Artifacts can be submitted through the SPIN 2024 Easychair page: https://easychair.org/conferences/?conf=spin2024. Choose the “SPIN 2024 - artifact” track.