An MP-capable network stack for DragonFlyBSD with minimal use of locks.

Aggelos Economopoulos

Abstract

Given the modern trend towards multi-core shared memory multiprocessors, it is inconceivable for production OS kernels not to be reentrant. The typical approach for allowing multiple execution contexts to simultaneously execute in kernel mode has been to use fine-grained locking for synchronising access to shared resources. While this technique has been proven efficient, empirical evidence suggests that the resulting locking rules tend to be cumbersome even for the experienced kernel programmer, leading to bugs that are hard to diagnose. Moreover, scaling to more processors requires extensive use of locks, which may impose unnecessary locking overhead for small scale multiprocessor systems. This talk will describe the typical approach and then discuss the alternative approach taken in the DragonFlyBSD network stack. We will give an overview of the various protocol threads employed for network I/O processing and the common-case code paths for packet reception and transmission. Additionally, we'll need to make a passing reference to DragonFlyBSD's message passing model. This should establish a baseline, allowing us to focus on the recent work by the author to eliminate use of the Big Giant Lock in the performance-critical paths for the TCP and UDP protocols. The decision to constrain this work on the two by far most widely-used transport protocols was made in order to (a) limit the amount of work necessary and (b) explore the effectiveness of the approach on the cases that matter at this point in time.