REVIEWING GUIDELINES

This document guides those who participate in the ISMAR 2023 reviewing process and is directed towards those who perform full reviews of conference papers (i.e., secondary review coordinator 2AC, committee members, and reviewers) and meta-reviews (primary review coordinator 1AC).

Reviewing a Conference Paper for ISMAR 2023

During the paper review process, review the paper using the following guidelines:

• • The primary aspect for accepting/rejecting a paper is its contribution: We consider a paper of sufficient conference quality if it presents a strong, tangible contribution to a specific aspect of the authors’ research. The authors can present preliminary findings if they support the authors’ assumptions, but more and complete findings are always welcome.
  • Consider the value of the contribution and merits of the paper to the ISMAR community. The paper may be an edge case. However, it discusses a topic, e.g., currently unknown to the community or extremely relevant at the time.
  • The contribution needs to be proportional to the paper length. A small contribution is acceptable if the paper is short (minimum length is 4 full pages NOT including references). Longer papers (e.g., 8 pages NOT including references) with a small contribution may require a major revision, thus, need to be rejected (ISMAR 2023 does not grant major revisions).
  • Papers are acceptable if the authors can easily correct the weaknesses (e.g., missing references, minor spell-checking, fuzzy statements, lack of implementation details, and others).
  • Mind that the ISMAR 2023 review form will ask you to provide a single score (6-point ranking, see below) judging the quality. Your written appraisal must support this score.

In general, as you perform your review, we ask that you reflect on the overall merits and to read papers with care and sympathy. Avoid seeking hidden flaws. Many hours of work — in some cases, years of work — have gone into research and writing this paper. Try to avoid last-minute reviews.

Write a Review

The following guidelines outline the content and key points of a high-quality review for ISMAR 2023. Please adhere to the guidelines and contact the conference program chairs or review coordinators for any questions.

• • A high-quality review should have about 1-page of well-considered commentary (at least 500 words), or even more, if warranted. Short and/or content-free reviews are insufficient and frustrate the authors.
  • State specifically the reason(s) for the score you selected for this paper. Clearly describe on what grounds the paper should be accepted (or rejected).
  • Describe the contributions in the paper and why they are noteworthy or important.
  • Explicitly and clearly discuss the weaknesses and limitations in a positive and constructive manner. Specifically, be positive and not insulting.
  • Your comments should assess the work the authors did and whether their methods are appropriate to support their claims. Avoid judging and explaining what the authors should have done.

Mind that your decisions affect the public appearance of ISMAR 2023. Therefore, the conference program chairs are very serious about ensuring the highest possible reviewing standards for ISMAR 2023. The coordinators and/or program chairs will ask you to improve your review if we think the reason for your judgment is unclear.

Write a Meta-Review

The following guidelines outline the content of a good meta-review. The section is for review coordinators (1AC) and explains the content the chairs believe supports a decision best.

• • Describe the primary contribution of the paper.
  • Summarize the most significant pros and cons of the paper. The most critical are often those the majority of reviewers highlight in their reviews. Abstain from reiterating every single aspect (we have the reviews for that).
  • Explain the decision and the pros and cons that support this decision.
  • In case of conditional acceptance, describe the conditions the authors have to meet before the paper can be accepted.
  • In case the paper is rejected, add suggestions for improvement.
  • Avoid adding discussion details or the score into the meta-review.

Mind that the authors will see the meta-review with the final decision. Be constructive and explain rather more than less, especially in the case the authors receive an unfavorable decision. Very often, the research or paper was not ready at the time of submission. Invite the authors to re-submit next year if feasible.

ISMAR 6-point Ranking

The section explains the ISMAR 6-point ranking and explains when we think one should select a particular score. We are aware that the decision can be subjective in many cases and that selecting between two is often a judgment call. We hope that this explanation removes some fuzziness.

• • Definitely accept: I would argue strongly for accepting this submission.
    Select this option if the paper is acceptable as-is (except for minor edits), with a strong contribution and merits for the ISMAR community.
  • Probably accept: I would argue for accepting this submission.
    Select this option if the paper has a valid contribution and merits for the ISMAR community. Some additional explanations or minor corrections are required.
  • Weak accept: The paper has weaknesses, but the contributions outweigh the weaknesses.
    Select this option if the research is relevant, the topic is of value for the ISMAR community, and the attitude towards this contribution is overall positive despite the identified weaknesses.
  • Weak reject: The paper has contributions, but the weaknesses outweigh the contributions.
    Select this option if the research is relevant, the topic is of value for the ISMAR community, but the attitude is overall negative because of the identified weaknesses.
  • Probably reject: I would argue for rejecting this submission.
    Select this option if the research is relevant and the topic is of value for the ISMAR community, but the research has severe weaknesses.
  • Definitely reject: I would argue strongly for rejecting this submission.
    Select this option if the contribution is not understandable and the paper has no recognizable merits, or it is unclear what information the ISMAR community gains from this submission.

Frequently Asked Questions (FAQs):

Should we reject papers that have been published on ArXiv?

Please read the full “Double-Blind Process and Anonymity Policy” in the Author Guidelines. In a nutshell: No, you should not reject papers that have been published on ArXiv or a similar service as authors may have done it as a way to get a timestamp for their work. However, if the ArXiv submission explicitly states that the submission is under review at ISMAR, or if the authors listed this prepublication on their individual or institutional webpages or generated publicity for it through other forms of media, then yes, it may constitute a violation of ISMAR policies. Please raise any related concerns in your review and/or contact the Program Chairs by email at conference_paper_chairs@ismar2023.org

Should we reject papers that have been presented before in a different format (e.g., poster or demo)?

In some situations, a submission may build upon prior work. As part of the Author Guidelines, conference paper authors were instructed to be proactive about clarifying such cases by uploading additional documents to the submission system that are non-anonymous and can only be seen by the primary coordinator assigned to this submission. If you suspect any issues related to this point, please first contact the primary coordinator of this paper. If no clarification has been uploaded by the authors, please carefully assess how far the publications overlap. Note that ISMAR does not consider a prior non-archival 2-page poster/demo extended abstract a reason for rejection of a conference paper submitted on the same topic.

Thank you for your support and work to ensure the highest-quality ISMAR reviews.

Do not hesitate to contact us for any further information.

ISMAR 2023 Science & Technology Conference Papers Chairs: conference_paper_chairs@ismar23.org

ISMAR 2023 PC Chairs,

Jens Grubert, Coburg University, Germany
Andrew Cunningham, University of South Australia, Australia
Evan Peng, University of Hong Kong, China
Gerd Bruder, University of Central Florida, USA
Anne-Hélène Olivier, University of Rennes 2 – Inria, France
Ian Williams, Birmingham City University, UK

Document History:

This document was updated and extended by the ISMAR 2023 Conference Paper Chairs: Jens Grubert, Andrew Cunningham, Evan Peng, Gerd Bruder, Anne-Hélène Olivier, and Ian Williams. These guidelines were updated for the ISMAR 2022 conference papers review process: Henry Duh, Jens Grubert, Jianmin Zheng, Ian Williams, and Adam Jones. These guidelines are based on the work of the ISMAR 2019 PC Chairs: Shimin Hu, Denis Kalkofen, Joseph L. Gabbard, Jonathan Ventura, Jens Grubert, and Stefanie Zollmann.