We use alexnet as an example

prepare caffe prototxt and model

These files will usually generated when trained with caffe

train.prototxt
deploy.prototxt
snapshot_10000.caffemodel

deploy.prototxt and caffemodel file are enough for TEST phase

alexnet deploy.prototxt can be downloaded here

https://github.com/BVLC/caffe/tree/master/models/bvlc_alexnet

alexnet caffemodel can be downloaded here

http://dl.caffe.berkeleyvision.org/bvlc_alexnet.caffemodel

convert to ncnn model

Convert old caffe prototxt and caffemodel to new ones using tools in caffe

because the ncnn convert tool needs the new format

upgrade_net_proto_text [old prototxt] [new prototxt]
upgrade_net_proto_binary [old caffemodel] [new caffemodel]

Use Input layer as input, set N dim as 1 since only one image can be processed each time

layer {
  name: "data"
  type: "Input"
  top: "data"
  input_param { shape: { dim: 1 dim: 3 dim: 227 dim: 227 } }
}

Use caffe2ncnn tool to convert caffe model to ncnn model

caffe2ncnn deploy.prototxt bvlc_alexnet.caffemodel alexnet.param alexnet.bin

strip visible string

It is already enough for deploying with param and bin file only, but there are visible strings in param file, it may not be suitable to distribute plain neural network information in your APP.

You can use ncnn2mem tool to convert plain model file to binary representation. It will generate alexnet.param.bin and two static array code files.

ncnn2mem alexnet.param alexnet.bin alexnet.id.h alexnet.mem.h

load model

Load param and bin file, the easy way

ncnn::Net net;
net.load_param("alexnet.param");
net.load_model("alexnet.bin");

Load binary param.bin and bin file, no visible strings included, suitable for bundled as APP resource

ncnn::Net net;
net.load_param_bin("alexnet.param.bin");
net.load_model("alexnet.bin");

Load network and model from external memory, no visible strings included, no external resource files bundled, the whole model is hardcoded in your program

You may use this way to load from android asset resource

#include "alexnet.mem.h"
ncnn::Net net;
net.load_param(alexnet_param_bin);
net.load_model(alexnet_bin);

You can choose either way to load model. Loading from external memory is zero-copy, which means you must keep your memory buffer during processing

unload model

net.clear();

input and output

ncnn Mat is the data structure for input and output data

Input image should be converted to Mat, and subtracted mean values and normalized when needed

#include "mat.h"
unsigned char* rgbdata;// data pointer to RGB image pixels
int w;// image width
int h;// image height
ncnn::Mat in = ncnn::Mat::from_pixels(rgbdata, ncnn::Mat::PIXEL_RGB, w, h);

const float mean_vals[3] = {104.f, 117.f, 123.f};
in.substract_mean_normalize(mean_vals, 0);

Execute the network inference and retrieve the result

#include "net.h"
ncnn::Mat in;// input blob as above
ncnn::Mat out;
ncnn::Extractor ex = net.create_extractor();
ex.input("data", in);
ex.extract("prob", out);

If you load model with binary param.bin file, you should use the enum value in alexnet.id.h file instead of the blob name

#include "net.h"
#include "alexnet.id.h"
ncnn::Mat in;// input blob as above
ncnn::Mat out;
ncnn::Extractor ex = net.create_extractor();
ex.input(alexnet_param_id::BLOB_data, in);
ex.extract(alexnet_param_id::BLOB_prob, out);

Read the data in the output Mat. Iterate data to get all classification scores.

ncnn::Mat out_flatterned = out.reshape(out.w * out.h * out.c);
std::vector<float> scores;
scores.resize(out_flatterned.w);
for (int j=0; j<out_flatterned.w; j++)
{
    scores[j] = out_flatterned[j];
}

some tricks

Convert image colorspace and resize image with Mat convenient function, these functions are well optimized

Support RGB2GRAY GRAY2RGB RGB2BGR etc, support scale up and scale down

#include "mat.h"
unsigned char* rgbdata;// data pointer to RGB image pixels
int w;// image width
int h;// image height
int target_width = 227;// target resized width
int target_height = 227;// target resized height
ncnn::Mat in = ncnn::Mat::from_pixels_resize(rgbdata, ncnn::Mat::PIXEL_RGB2GRAY, w, h, target_width, target_height);

You can concat multiple model files into one, and load this single file from FILE* interface.

It should ease the distribution of param and model files.

$ cat alexnet.param.bin alexnet.bin > alexnet-all.bin

#include "net.h"
FILE* fp = fopen("alexnet-all.bin", "rb");
net.load_param_bin(fp);
net.load_model(fp);
fclose(fp);

Source: docs/how-to-use-and-FAQ/use-ncnn-with-alexnet.md