Forward and Inverse Tasks for Inference Rate Estimation in Binary Neural Networks

Abstract

Deploying Binary Neural Networks (BNNs) presents significant challenges, particularly in selecting suitable hardware to achieve desired performance levels and accurately estimating computational costs. To address these issues, a novel metric named "XNOROP" was recently introduced, offering a simplified approach for estimating the computational cost of BNNs and introducing a method for compressing binary filters. This paper leverages " XNOROP" to define and solve two critical tasks in BNNs. The first task, referred to as the "Forward Task", involves estimating the inference rate of a given model M when deployed on a specific target device T. The second task, known as the "Inverse Task", outlines a systematic procedure to identify a set of target devices T capable of achieving a required inference rate M when deploying the model . Additionally, we extend the foundational formula of "XNOROP" and introduce “LXNOROP” which incorporates considerations for memory access time, enhancing its applicability for real-world deployment scenarios.