Skip to content

Documentation site is now generated by running script from master branch. #155

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 25, 2014
Merged

Documentation site is now generated by running script from master branch. #155

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
a389244
initial gh-pages commit with new build system
sergeyk Feb 25, 2014
f80afc5
5556c62 Moving gh-pages docs to docs/ folder, with script to build them
sergeyk Feb 25, 2014
ae6048e
aea53c7 Safer docs/ build process: into own folder
sergeyk Feb 25, 2014
6c80d2c
3ab8e86 small typo fix in installation.md
sergeyk Feb 25, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions .gitignore
View file
Open in desktop

This file was deleted.

149 changes: 149 additions & 0 deletions imagenet.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
<!doctype html>
<html>
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="chrome=1">
<title>Caffe</title>

<link rel="stylesheet" href="stylesheets/reset.css">
<link rel="stylesheet" href="stylesheets/styles.css">
<link rel="stylesheet" href="stylesheets/pygment_trac.css">
<script src="javascripts/scale.fix.js"></script>
<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no">
<!--[if lt IE 9]>
<script src="//html5shiv.googlecode.com/svn/trunk/html5.js"></script>
<![endif]-->
</head>
<body>
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');

ga('create', 'UA-46255508-1', 'daggerfs.com');
ga('send', 'pageview');
</script>
<div class="wrapper">
<header>
<h1 class="header"><a href="index.html">Caffe</a></h1>
<p class="header">Convolutional Architecture for Fast Feature Embedding</p>

<ul>
<!--<li class="download"><a class="buttons" href="https://github.com/BVLC/caffe/zipball/master">Download ZIP</a></li>
<li class="download"><a class="buttons" href="https://github.com/BVLC/caffe/tarball/master">Download TAR</a></li>-->
<li><a class="buttons github" href="https://github.com/BVLC/caffe">View On GitHub</a></li>
</ul>
<p class="header">Maintained by<br><a class="header name" href="http://ucbvlc.org/">BVLC</a></p>
<p class="header">Created by<br><a class="header name" href="http://daggerfs.com/">Yangqing Jia</a></p>

</header>
<section>

<h1 id="yangqings_recipe_on_brewing_imagenet">Yangqing’s Recipe on Brewing ImageNet</h1>

<pre><code>&quot;All your braincells are belong to us.&quot;
- Starbucks</code></pre>

<p>We are going to describe a reference implementation for the approach first proposed by Krizhevsky, Sutskever, and Hinton in their <a href="http://books.nips.cc/papers/files/nips25/NIPS2012_0534.pdf">NIPS 2012 paper</a>. Since training the whole model takes quite some time and energy, we also provide a model, trained in the same way as we describe here, to help fight global warming. If you would like to simply use the pretrained model, check out the <a href="imagenet_pretrained.html">Pretrained ImageNet</a> page.</p>

<p>To clarify, by ImageNet we actually mean the ILSVRC challenge, but you can easily train on the whole imagenet as well, just more disk space, and a little longer training time.</p>

<p>(If you don’t get the quote, visit <a href="http://yann.lecun.com/ex/fun/">Yann LeCun’s fun page</a>.</p>

<h2 id="data_preparation">Data Preparation</h2>

<p>We assume that you already have downloaded the ImageNet training data and validation data, and they are stored on your disk like:</p>

<pre><code>/path/to/imagenet/train/n01440764/n01440764_10026.JPEG
/path/to/imagenet/val/ILSVRC2012_val_00000001.JPEG</code></pre>

<p>You will first need to create a text file listing all the files as well as their labels. An example could be found in the caffe repo at <code>python/caffe/imagenet/ilsvrc_2012_train.txt</code> and <code>ilsvrc_2012_val.txt</code>. Note that in those two files we used a different indexing from the ILSVRC devkit: we sorted the synset names in their ASCII order, and then labeled them from 0 to 999.</p>

<p>You will also need to resize the images to 256x256: we do not explicitly do this because in a cluster environment, one may benefit from resizing images in a parallel fashion, using mapreduce. For example, Yangqing used his lightedweighted <a href="https://github.com/Yangqing/mincepie">mincepie</a> package to do mapreduce on the Berkeley cluster. If you would things to be rather simple and straightforward, you can also use shell commands, something like:</p>

<pre><code>for name in /path/to/imagenet/val/*.JPEG; do
convert -resize 256x256\! $name $name
done</code></pre>

<p>Now, you can simply create a leveldb using commands as follows:</p>

<pre><code>GLOG_logtostderr=1 examples/convert_imageset.bin \
/path/to/imagenet/train/ \
python/caffe/imagenet/ilsvrc_2012_train.txt \
/path/to/imagenet-train-leveldb 1</code></pre>

<p>Note that <code>/path/to/imagenet-train-leveldb</code> should not exist before this execution. It will be created by the script. <code>GLOG_logtostderr=1</code> simply dumps more information for you to inspect, and you can safely ignore it.</p>

<h2 id="compute_image_mean">Compute Image Mean</h2>

<p>The Model requires us to subtract the image mean from each image, so we have to compute the mean. <code>examples/demo_compute_image_mean.cpp</code> implements that - it is also a good example to familiarize yourself on how to manipulate the multiple components, such as protocol buffers, leveldbs, and logging, if you are not familiar with it. Anyway, the mean computation can be carried out as:</p>

<pre><code>examples/demo_compute_image_mean.bin /path/to/imagenet-train-leveldb /path/to/mean.binaryproto</code></pre>

<p>where <code>/path/to/mean.binaryproto</code> will be created by the program.</p>

<h2 id="network_definition">Network Definition</h2>

<p>The network definition follows strictly the one in Krizhevsky et al. You can find the detailed definition at <code>examples/imagenet.prototxt</code>. Note that to run it, you will most likely need to change the paths in the data layer - change the following lines</p>

<pre><code>source: &quot;/home/jiayq/Data/ILSVRC12/train-leveldb&quot;
meanfile: &quot;/home/jiayq/Data/ILSVRC12/image_mean.binaryproto&quot;</code></pre>

<p>to point to your own leveldb and image mean. Likewise, do the same for <code>examples/imagenet_val.prototxt</code>.</p>

<p>If you look carefully at <code>imagenet.prototxt</code> and <code>imagenet_val.prototxt</code>, you will notice that they are largely the same, with the only difference being the data layer sources, and the last layer: in training, we will be using a <code>softmax_loss</code> layer to compute the loss function and to initialize the backpropagation, while in validation we will be using an <code>accuracy</code> layer to inspect how well we do in terms of accuracy.</p>

<p>We will also lay out a protocol buffer for running the solver. Let’s make a few plans:</p>

<ul>
<li>We will run in batches of 256, and run a total of 4,500,000 iterations (about 90 epochs).</li>

<li>For every 1,000 iterations, we test the learned net on the validation data.</li>

<li>We set the initial learning rate to 0.01, and decrease it every 100,000 iterations (about 20 epochs).</li>

<li>Information will be displayed every 20 epochs.</li>

<li>The network will be trained with momentum 0.9 and a weight decay of 0.0005.</li>

<li>For every 10,000 iterations, we will take a snapshot of the current status.</li>
</ul>

<p>Sounds good? This is implemented in <code>examples/imagenet_solver.prototxt</code>. Again, you will need to change the first two lines:</p>

<pre><code>train_net: &quot;examples/imagenet.prototxt&quot;
test_net: &quot;examples/imagenet_val.prototxt&quot;</code></pre>

<p>to point to the actual path.</p>

<h2 id="training_imagenet">Training ImageNet</h2>

<p>Ready? Let’s train.</p>

<pre><code>GLOG_logtostderr=1 examples/train_net.bin examples/imagenet_solver.prototxt</code></pre>

<p>Sit back and enjoy! On my K20 machine, every 20 iterations take about 36 seconds to run, so effectively about 7 ms per image for the full forward-backward pass. About 2.5 ms of this is on forward, and the rest is backward. If you are interested in dissecting the computation time, you can look at <code>examples/net_speed_benchmark.cpp</code>, but it was written purely for debugging purpose, so you may need to figure a few things out yourself.</p>

<h2 id="resume_training">Resume Training?</h2>

<p>We all experience times when the power goes out, or we feel like rewarding ourself a little by playing Battlefield (does someone still remember Quake?). Since we are snapshotting intermediate results during training, we will be able to resume from snapshots. This can be done as easy as:</p>

<pre><code>GLOG_logtostderr=1 examples/train_net.bin examples/imagenet_solver.prototxt caffe_imagenet_train_10000.solverstate</code></pre>

<p>where <code>caffe_imagenet_train_1000.solverstate</code> is the solver state snapshot that stores all necessary information to recover the exact solver state (including the parameters, momentum history, etc).</p>

<h2 id="parting_words">Parting Words</h2>

<p>Hope you liked this recipe. Many researchers have gone further since the ILSVRC 2012 challenge, changing the network architecture and/or finetuning the various parameters in the network. The recent ILSVRC 2013 challenge suggests that there are quite some room for improvement. <strong>Caffe allows one to explore different network choices more easily, by simply writing different prototxt files</strong> - isn’t that exciting?</p>

<p>And since now you have a trained network, check out how to use it: <a href="imagenet_pretrained.html">Running Pretrained ImageNet</a>. This time we will use Python, but if you have wrappers for other languages, please kindly send me a pull request!</p>

</section>
<footer>
<p><small>Hosted on <a href="http://pages.github.com">GitHub Pages</a>.</small></p>
</footer>
</div>
<!--[if !IE]><script>fixScale(document);</script><![endif]-->
</body>
</html>
101 changes: 0 additions & 101 deletions imagenet.md
View file
Open in desktop

This file was deleted.

Loading