Neural networks have been widely deployed to achieve state-of-the-art performance in classification and regression tasks within various domains. Inference is typically performed on GPU devices, which are readily available and offer large performance improvements over general-purpose CPUs due to their deeply parallelized architecture.

As cutting-edge models become increasingly complex, GPUs have shown performance limitations due to expensive data copy and synchronization mechanisms. In particular, extreme low-latency applications, such as in high-energy physics or autonomous vehicles, show the need for custom hardware to achieve sub-micro-second predictions. FPGA devices are well capable of meeting these requirements due to their reconfigurable logic fabric and have been shown to achieve up to 10x latency and throughput improvements over GPU counterparts, with orders of magnitude lower power consumption. The added flexibility of FPGAs, stemming from their reconfigurability, enables finer-grained optimizations in network implementation, such as layer-specific quantization schemes. This use of lower precision numerical formats has been shown to reduce memory requirements and computational overhead, as well as power consumption on reconfigurable devices, at a low cost to model accuracy.

In recent times, Graph Neural Networks (GNNs) have attracted great attention due to their classification performance on non-Euclidean data. FPGA acceleration proves particularly beneficial for GNNs given their irregular memory access patterns, resulting from the sparse structure of graphs. These unique compute requirements have been addressed by several FPGA and ASIC accelerators, such as HyGCN and GenGNN.

Despite the relative success of hardware approaches to accelerate GNN inference on FPGA devices, previous works have shown to be limited to small graphs with up to 20k nodes, such as Cora, Citeseer and Pubmed. Since the computational overhead of GNN inference grows with increasing graph size, current accelerators are not well-prepared to handle medium to large-scale graphs, particularly in real-time applications.

This work introduces AGILE (Accelerated Graph Inference Logic Engine), an FPGA accelerator aimed at enabling real-time GNN inference for large graphs by exploring a range of hardware optimisations. A new asynchronous programming model is formulated, which reduces pipeline gaps by addressing the non-uniform distribution in node degrees. Inspired by GNN quantisation analysis from Taylor et al., a multi-precision node dataflow is proposed, improving throughput and device resource usage. Finally, an on-chip Prefetcher unit was implemented to cut down memory access latency. Evaluation on Planetoid graphs shows up to 2.8x speed-up against GPU counterparts.