UFLDL Tutorial 学习笔记

卷积层的实现代码。其中使用的matlab中conv2：

matlab中conv2 函数在计算二维卷积应用实例如下：
格式：C=conv2(A,B)
        C=conv2(Hcol,Hrow,A)
        C=conv2(...,'shape')
说明：对于 C=conv2(A,B) ，conv2 的算矩阵 A 和 B 的卷积，若 [Ma,Na]＝size(A), [Mb,Nb]=size(B), 则 size(C)=[Ma+Mb-1,Na+Nb-1];
C=conv2(Hcol,Hrow,A) 中，矩阵 A 分别与 Hcol 向量在列方向和 Hrow 向量在行方向上进行卷积；C=conv2(...,'shape') 用来指定 conv2
返回二维卷积结果部分，参数 shape 可取值如下：
        》full 为缺省值，返回二维卷积的全部结果；
        》same 返回二维卷积结果中与 A 大小相同的中间部分；
        valid 返回在卷积过程中，未使用边缘补 0 部分进行计算的卷积结果部分，当 size(A)>size(B) 时，size(C)=[Ma-Mb+1,Na-Nb+1]

function convolvedFeatures = cnnConvolve(filterDim, numFilters, images, W, b)
%cnnConvolve Returns the convolution of the features given by W and b with
%the given images
%
% Parameters:
%  filterDim - filter (feature) dimension
%  numFilters - number of feature maps
%  images - large images to convolve with, matrix in the form
%           images(r, c, image number)
%  W, b - W, b for features from the sparse autoencoder
%         W is of shape (filterDim,filterDim,numFilters)
%         b is of shape (numFilters,1)
%  W是卷积的稀疏矩阵，b是直流分量是一个偏执。
% Returns:
%  convolvedFeatures - matrix of convolved features in the form
%                      convolvedFeatures(imageRow, imageCol, featureNum, imageNum)numImages = size(images, 3);
imageDim = size(images, 1);
convDim = imageDim - filterDim + 1;%计算维度convolvedFeatures = zeros(convDim, convDim, numFilters, numImages);% Instructions:
%   Convolve every filter with every image here to produce the 
%   (imageDim - filterDim + 1) x (imageDim - filterDim + 1) x numFeatures x numImages
%   matrix convolvedFeatures, such that 
%   convolvedFeatures(imageRow, imageCol, featureNum, imageNum) is the
%   value of the convolved featureNum feature for the imageNum image over
%   the region (imageRow, imageCol) to (imageRow + filterDim - 1, imageCol + filterDim - 1)
%
% Expected running times: 
%   Convolving with 100 images should take less than 30 seconds 
%   Convolving with 5000 images should take around 2 minutes
%   (So to save time when testing, you should convolve with less images, as
%   described earlier)for imageNum = 1:numImagesfor filterNum = 1:numFilters% convolution of image with feature matrixconvolvedImage = zeros(convDim, convDim);% Obtain the feature (filterDim x filterDim) needed during the convolution%%% YOUR CODE HERE %%%filter = W(:,:,filterNum);%取卷积矩阵bc = b(filterNum);% Flip the feature matrix because of the definition of convolution, as explained later% squeeze去除size为1的维度，也就是去除后面filterNum的维度filter = rot90(squeeze(filter),2);% Obtain the imageim = squeeze(images(:, :, imageNum));% Convolve "filter" with "im", adding the result to convolvedImage% be sure to do a 'valid' convolution%%% YOUR CODE HERE %%%% valid 返回在卷积过程中，未使用边缘补 0 部分进行计算的卷积结果部分convolvedImage = conv2(im,filter,'valid');% Add the bias unit% Then, apply the sigmoid function to get the hidden activation%%% YOUR CODE HERE %%%convolvedImage = sigmoid(convolvedImage+bc);convolvedFeatures(:, :, filterNum, imageNum) = convolvedImage;end
endend
function sigmoid = sigmoid(x)sigmoid = 1./(1+exp(-x));
end

池化层，这里池化层步长和卷积核的维度相同

function pooledFeatures = cnnPool(poolDim, convolvedFeatures)
%cnnPool Pools the given convolved features
%
% Parameters:
%  poolDim - dimension of pooling region
%  convolvedFeatures - convolved features to pool (as given by cnnConvolve)
%                      convolvedFeatures(imageRow, imageCol, featureNum, imageNum)
%
% Returns:
%  pooledFeatures - matrix of pooled features in the form
%                   pooledFeatures(poolRow, poolCol, featureNum, imageNum)
%     numImages = size(convolvedFeatures, 4);%图片数量
numFilters = size(convolvedFeatures, 3);%特征数量等于卷积核数量
convolvedDim = size(convolvedFeatures, 1);%图片的行数，这里行数等于列数
%步长是filterdim
pooledFeatures = zeros(convolvedDim / poolDim, ...convolvedDim / poolDim, numFilters, numImages);% Instructions:
%   Now pool the convolved features in regions of poolDim x poolDim,
%   to obtain the 
%   (convolvedDim/poolDim) x (convolvedDim/poolDim) x numFeatures x numImages 
%   matrix pooledFeatures, such that
%   pooledFeatures(poolRow, poolCol, featureNum, imageNum) is the 
%   value of the featureNum feature for the imageNum image pooled over the
%   corresponding (poolRow, poolCol) pooling region. 
%   
%   Use mean pooling here.%%% YOUR CODE HERE %%%
for imageNum = 1:numImagesfor numFilter=1:numFiltersfor poolRow=1:convolvedDim / poolDimfor poolCol=1:convolvedDim / poolDimpatch = convolvedFeatures((poolRow-1)*poolDim+1:poolRow*poolDim,...(poolCol-1)*poolDim+1:poolCol*poolDim,numFilter,imageNum);pooledFeatures(poolRow,poolCol,numFilter,imageNum)=mean(patch(:));endendend
end
end

验证卷积层和池化层的正确性，UFLDL给的代码是：

%% Convolution and Pooling Exercise%  Instructions
%  ------------
% 
%  This file contains code that helps you get started on the
%  convolution and pooling exercise. In this exercise, you will only
%  need to modify cnnConvolve.m and cnnPool.m. You will not need to modify
%  this file.%%======================================================================
%% STEP 0: Initialization and Load Data
%  Here we initialize some parameters used for the exercise.imageDim = 28;         % image dimensionfilterDim = 8;          % filter dimension
numFilters = 100;         % number of feature mapsnumImages = 60000;    % number of imagespoolDim = 3;          % dimension of pooling region% Here we load MNIST training images
addpath ../common/;
images = loadMNISTImages('../common/train-images.idx3-ubyte');
images = reshape(images,imageDim,imageDim,numImages);W = randn(filterDim,filterDim,numFilters);
b = rand(numFilters);%%======================================================================
%% STEP 1: Implement and test convolution
%  In this step, you will implement the convolution and test it on
%  on a small part of the data set to ensure that you have implemented
%  this step correctly.%% STEP 1a: Implement convolution
%  Implement convolution in the function cnnConvolve in cnnConvolve.m%% Use only the first 8 images for testing
convImages = images(:, :, 1:8); % NOTE: Implement cnnConvolve in cnnConvolve.m first!
convolvedFeatures = cnnConvolve(filterDim, numFilters, convImages, W, b);%% STEP 1b: Checking your convolution
%  To ensure that you have convolved the features correctly, we have
%  provided some code to compare the results of your convolution with
%  activations from the sparse autoencoder% For 1000 random points
for i = 1:1000   filterNum = randi([1, numFilters]);imageNum = randi([1, 8]);imageRow = randi([1, imageDim - filterDim + 1]);imageCol = randi([1, imageDim - filterDim + 1]);    patch = convImages(imageRow:imageRow + filterDim - 1, imageCol:imageCol + filterDim - 1, imageNum);feature = sum(sum(patch.*W(:,:,filterNum)))+b(filterNum);feature = 1./(1+exp(-feature));if abs(feature - convolvedFeatures(imageRow, imageCol,filterNum, imageNum)) > 1e-9fprintf('Convolved feature does not match test feature\n');fprintf('Filter Number    : %d\n', filterNum);fprintf('Image Number      : %d\n', imageNum);fprintf('Image Row         : %d\n', imageRow);fprintf('Image Column      : %d\n', imageCol);fprintf('Convolved feature : %0.5f\n', convolvedFeatures(imageRow, imageCol, filterNum, imageNum));fprintf('Test feature : %0.5f\n', feature);       error('Convolved feature does not match test feature');end 
enddisp('Congratulations! Your convolution code passed the test.');%%======================================================================
%% STEP 2: Implement and test pooling
%  Implement pooling in the function cnnPool in cnnPool.m%% STEP 2a: Implement pooling
% NOTE: Implement cnnPool in cnnPool.m first!
pooledFeatures = cnnPool(poolDim, convolvedFeatures);%% STEP 2b: Checking your pooling
%  To ensure that you have implemented pooling, we will use your pooling
%  function to pool over a test matrix and check the results.testMatrix = reshape(1:64, 8, 8);
expectedMatrix = [mean(mean(testMatrix(1:4, 1:4))) mean(mean(testMatrix(1:4, 5:8))); ...mean(mean(testMatrix(5:8, 1:4))) mean(mean(testMatrix(5:8, 5:8))); ];testMatrix = reshape(testMatrix, 8, 8, 1, 1);pooledFeatures = squeeze(cnnPool(4, testMatrix));if ~isequal(pooledFeatures, expectedMatrix)disp('Pooling incorrect');disp('Expected');disp(expectedMatrix);disp('Got');disp(pooledFeatures);
elsedisp('Congratulations! Your pooling code passed the test.');
end

参考网址：

http://ufldl.stanford.edu/tutorial/