Precision, recall, and sensitivity of monitoring partially synchronous distributed programs

Abstract

Distributed programs are often designed with implicit assumptions about the underlying system. We focus on assumptions related to clock synchronization. When a program written with clock synchronization assumptions is monitored to determine if it satisfies its requirements, the monitor should also account for these assumptions precisely. Otherwise, the monitor will either miss potential bugs (false negatives) or find bugs that are inconsistent with these assumptions (false positives). However, if assumptions made by the program are implicit or change over time and are not immediately available to the monitor, such false positives and/or negatives are unavoidable. This paper characterizes precision (probability that the violation identified by the monitor is valid) and recall (probability that the monitor identifies an actual violation) of the monitor based on the gap between clock synchronization assumptions made by the program/application and the clock synchronization assumptions made by the monitor. Our analysis is based on the development of an analytical model for precision, recall and sensitivity of monitors detecting conjunctive predicates. We validate the model via simulations and experiments on the Amazon Web Services platform.