Validation of point process forecasts

We introduce a class of proper scoring rules for evaluating spatial point process forecastsbased on summary statistics. These scoring rules rely on Monte-Carlo approximation ofan expectation and can therefore easily be evaluated for any point process model thatcan be simulated. In this regard they are more flexible than the commonly used logar-ithmic score which cannot be evaluated for many point process models, as their densityis only known up to an untractable constant. In simulation studies we demonstrate theusefulness of our scores. Furthermore we consider a scoring rule, the quantile score, thatis commonly used to validate earthquake rate predictions, and show that it lacks propri-ety. As a consequence, several tests that are commonly applied in this context are biasedand systematically favour predictive distributions that are too uniform. We suggest toremedy this issue by replacing the commonly used one-sided by two-sided tests.