Caffe

Deep learning framework developed by Yangqing Jia / BVLC

Layers

To create a Caffe model you need to define the model architecture in a protocol buffer definition file (prototxt).

Caffe layers and their parameters are defined in the protocol buffer definitions for the project in caffe.proto. The latest definitions are in the dev caffe.proto.

TODO complete list of layers linking to headings

Vision Layers

Vision layers usually take images as input and produce other images as output. A typical “image” in the real-world may have one color channel (), as in a grayscale image, or three color channels () as in an RGB (red, green, blue) image. But in this context, the distinguishing characteristic of an image is its spatial structure: usually an image has some non-trivial height and width . This 2D geometry naturally lends itself to certain decisions about how to process the input. In particular, most of the vision layers work by applying a particular operation to some region of the input to produce a corresponding region of the output. In contrast, other layers (with few exceptions) ignore the spatial structure of the input, effectively treating it as “one big vector” with dimension .

Convolution

The CONVOLUTION layer convolves the input image with a set of learnable filters, each producing one feature map in the output image.

Pooling

Local Response Normalization (LRN)

The local response normalization layer performs a kind of “lateral inhibition” by normalizing over local input regions. In ACROSS_CHANNELS mode, the local regions extend across nearby channels, but have no spatial extent (i.e., they have shape local_size x 1 x 1). In WITHIN_CHANNEL mode, the local regions extend spatially, but are in separate channels (i.e., they have shape 1 x local_size x local_size). Each input value is divided by , where is the size of each local region, and the sum is taken over the region centered at that value (zero padding is added where necessary).

im2col

IM2COL is a helper for doing the image-to-column transformation that you most likely do not need to know about. This is used in Caffe’s original convolution to do matrix multiplication by laying out all patches into a matrix.

Loss Layers

Loss drives learning by comparing an output to a target and assigning cost to minimize. The loss itself is computed by the forward pass and the gradient w.r.t. to the loss is computed by the backward pass.

Softmax

The softmax loss layer computes the multinomial logistic loss of the softmax of its inputs. It’s conceptually identical to a softmax layer followed by a multinomial logistic loss layer, but provides a more numerically stable gradient.

Sum-of-Squares / Euclidean

The Euclidean loss layer computes the sum of squares of differences of its two inputs, .

Hinge / Margin

The hinge loss layer computes a one-vs-all hinge or squared hinge loss.

Sigmoid Cross-Entropy

SIGMOID_CROSS_ENTROPY_LOSS

Infogain

INFOGAIN_LOSS

Accuracy and Top-k

ACCURACY scores the output as the accuracy of output with respect to target – it is not actually a loss and has no backward step.

Activation / Neuron Layers

In general, activation / Neuron layers are element-wise operators, taking one bottom blob and producing one top blob of the same size. In the layers below, we will ignore the input and out sizes as they are identical:

ReLU / Rectified-Linear and Leaky-ReLU

Given an input value x, The RELU layer computes the output as x if x > 0 and negative_slope * x if x <= 0. When the negative slope parameter is not set, it is equivalent to the standard ReLU function of taking max(x, 0). It also supports in-place computation, meaning that the bottom and the top blob could be the same to preserve memory consumption.

Sigmoid

The SIGMOID layer computes the output as sigmoid(x) for each input element x.

TanH / Hyperbolic Tangent

The TANH layer computes the output as tanh(x) for each input element x.

Absolute Value

The ABSVAL layer computes the output as abs(x) for each input element x.

Power

The POWER layer computes the output as (shift + scale * x) ^ power for each input element x.

BNLL

The BNLL (binomial normal log likelihood) layer computes the output as log(1 + exp(x)) for each input element x.

Data Layers

Data enters Caffe through data layers: they lie at the bottom of nets. Data can come from efficient databases (LevelDB or LMDB), directly from memory, or, when efficiency is not critical, from files on disk in HDF5 or common image formats.

Common input preprocessing (mean subtraction, scaling, random cropping, and mirroring) is available by specifying TransformationParameters.

Database

In-Memory

The memory data layer reads data directly from memory, without copying it. In order to use it, one must call MemoryDataLayer::Reset (from C++) or Net.set_input_arrays (from Python) in order to specify a source of contiguous data (as 4D row major array), which is read one batch-sized chunk at a time.

HDF5 Input

HDF5 Output

The HDF5 output layer performs the opposite function of the other layers in this section: it writes its input blobs to disk.

Images

Windows

WINDOW_DATA

Dummy

DUMMY_DATA is for development and debugging. See DummyDataParameter.

Common Layers

Inner Product

The INNER_PRODUCT layer (also usually referred to as the fully connected layer) treats the input as a simple vector and produces an output in the form of a single vector (with the blob’s height and width set to 1).

Splitting

The SPLIT layer is a utility layer that splits an input blob to multiple output blobs. This is used when a blob is fed into multiple output layers.

Flattening

The FLATTEN layer is a utility layer that flattens an input of shape n * c * h * w to a simple vector output of shape n * (c*h*w) * 1 * 1.

Concatenation

The CONCAT layer is a utility layer that concatenates its multiple input blobs to one single output blob. Currently, the layer supports concatenation along num or channels only.

Slicing

The SLICE layer is a utility layer that slices an input layer to multiple output layers along a given dimension (currently num or channel only) with given slice indices.

Elementwise Operations

ELTWISE

Argmax

ARGMAX

Softmax

SOFTMAX

Mean-Variance Normalization

MVN