Low-Level Interactions

Low-Level Interactions

Two main objects are used throughout MyCaffe to provide a link to the low-level GPU operations, and provide a mechanism to output status to the user. Those two objects are the CudaDnn and the Log respectively.

CudaDnn

Each CudaDnn object manages the overall state of a single link to CUDA which includes managing all memory, CUDA descriptors, and other internal objects used with that link.

IMPORTANT: Typically only one instance of the CudaDnn object is used per thread to run all low-level GPU interactions taking place on that thread. The memory and other state items created by the CudaDnn instance can only be used by that same instance of the CudaDnn object.

Under the hood, the CudaDnn object (written in C#) uses the CudaControl for all interactions with the underlying CudaDnnDLL to access its high-speed CUDA kernels that implement the low-level AI operations. Each instance of the CudaDnn object maintains the device ID used with the state (within the CudaDnnDLL) that contains all memory and other CUDA states used.

Using this model minimizes data traffic between the C# and C++ portions of code for the CudaDnn merely uses simple handle values to reference each portion of the state maintained within the C++ CudaDnnDLL thus eliminating the need for large data transfers. Once the bulk of the data is transferred to the GPU it remains there as long as possible so as to minimize data traffic (and overall latency).

All MyCaffe objects that interact with one or more GPU’s use the CudaDnn object to do so.

Log

The Log object provides all output, eventually directed to the user. Such output may include the current status of a long-running operation or error information on the latest fault. Similar to the CudaDnn object, most MyCaffe objects hold a reference to the Log object.

Typically a single instance of the Log object is used, thus consolidating all output which can improve overall debugging of the system.

Coding Examples

The coding sample below, demonstrates how to create and use an instance of both the CudaDnn and Log objects.

To download the code for this sample, please see the BlobUsage samples on GitHub.