CPU info:
    CPU Model Name: Intel(R) Xeon(R) CPU W3530 @ 2.80GHz
    Hardware threads: 4
    Total Memory: 12580404 kB
-------------------------------------------------------------------
=== Running C:\Program Files\Microsoft MPI\Bin\/mpiexec.exe -n 4 C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu DeviceId=-1 timestamping=true numCPUThreads=1 precision=float SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]] stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16

C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
Changed current directory to C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data
requestnodes [MPIWrapper]: using 4 out of 4 MPI nodes on a single host (4 requested); we (2) are in (participating)
CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16

C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
Changed current directory to C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data
requestnodes [MPIWrapper]: using 4 out of 4 MPI nodes on a single host (4 requested); we (0) are in (participating)
CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16

C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
Changed current directory to C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data
requestnodes [MPIWrapper]: using 4 out of 4 MPI nodes on a single host (4 requested); we (3) are in (participating)
CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16

C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
Changed current directory to C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data
requestnodes [MPIWrapper]: using 4 out of 4 MPI nodes on a single host (4 requested); we (1) are in (participating)
MPI Rank 0: 12/15/2016 08:28:16: Redirecting stderr to file C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr_SimpleMultiGPU.logrank0
MPI Rank 0: CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16
MPI Rank 0: 
MPI Rank 0: C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
MPI Rank 0: 12/15/2016 08:28:16: Using 1 CPU threads.
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: ##############################################################################
MPI Rank 0: 12/15/2016 08:28:16: #                                                                            #
MPI Rank 0: 12/15/2016 08:28:16: # SimpleMultiGPU command (train action)                                      #
MPI Rank 0: 12/15/2016 08:28:16: #                                                                            #
MPI Rank 0: 12/15/2016 08:28:16: ##############################################################################
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: 
MPI Rank 0: Creating virgin network.
MPI Rank 0: SimpleNetworkBuilder Using CPU
MPI Rank 0: 12/15/2016 08:28:16: 
MPI Rank 0: Model has 25 nodes. Using CPU.
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: Training criterion:   CrossEntropyWithSoftmax = CrossEntropyWithSoftmax
MPI Rank 0: 12/15/2016 08:28:16: Evaluation criterion: EvalClassificationError = ClassificationError
MPI Rank 0: 
MPI Rank 0: 
MPI Rank 0: Allocating matrices for forward and/or backward propagation.
MPI Rank 0: 
MPI Rank 0: Memory Sharing: Out of 40 matrices, 19 are shared as 8, and 21 are not shared.
MPI Rank 0: 
MPI Rank 0: 	{ W1 : [50 x 50] (gradient)
MPI Rank 0: 	  W1*H1+B1 : [50 x 1 x *] }
MPI Rank 0: 	{ B1 : [50 x 1] (gradient)
MPI Rank 0: 	  H2 : [50 x 1 x *] (gradient)
MPI Rank 0: 	  HLast : [2 x 1 x *] (gradient) }
MPI Rank 0: 	{ H2 : [50 x 1 x *]
MPI Rank 0: 	  W1*H1 : [50 x 1 x *] (gradient) }
MPI Rank 0: 	{ W0*features+B0 : [50 x 1 x *] (gradient)
MPI Rank 0: 	  W1*H1 : [50 x 1 x *] }
MPI Rank 0: 	{ H1 : [50 x 1 x *]
MPI Rank 0: 	  W0*features : [50 x *] (gradient) }
MPI Rank 0: 	{ B0 : [50 x 1] (gradient)
MPI Rank 0: 	  H1 : [50 x 1 x *] (gradient)
MPI Rank 0: 	  W1*H1+B1 : [50 x 1 x *] (gradient)
MPI Rank 0: 	  W2*H1 : [2 x 1 x *] }
MPI Rank 0: 	{ W0 : [50 x 2] (gradient)
MPI Rank 0: 	  W0*features+B0 : [50 x 1 x *] }
MPI Rank 0: 	{ HLast : [2 x 1 x *]
MPI Rank 0: 	  W2 : [2 x 50] (gradient) }
MPI Rank 0: 
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: Training 2802 parameters in 6 out of 6 parameter tensors and 15 nodes with gradient:
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: 	Node 'B0' (LearnableParameter operation) : [50 x 1]
MPI Rank 0: 12/15/2016 08:28:16: 	Node 'B1' (LearnableParameter operation) : [50 x 1]
MPI Rank 0: 12/15/2016 08:28:16: 	Node 'B2' (LearnableParameter operation) : [2 x 1]
MPI Rank 0: 12/15/2016 08:28:16: 	Node 'W0' (LearnableParameter operation) : [50 x 2]
MPI Rank 0: 12/15/2016 08:28:16: 	Node 'W1' (LearnableParameter operation) : [50 x 50]
MPI Rank 0: 12/15/2016 08:28:16: 	Node 'W2' (LearnableParameter operation) : [2 x 50]
MPI Rank 0: 
MPI Rank 0: Initializing dataParallelSGD with FP32 aggregation.
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: Precomputing --> 3 PreCompute nodes found.
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: 	MeanOfFeatures = Mean()
MPI Rank 0: 12/15/2016 08:28:16: 	InvStdOfFeatures = InvStdDev()
MPI Rank 0: 12/15/2016 08:28:16: 	Prior = Mean()
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:16: Precomputing --> Completed.
MPI Rank 0: 
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:18: Starting Epoch 1: learning rate per sample = 0.020000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:18: Starting minibatch loop, DataParallelSGD training (myRank = 0, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[   1-  10]: CrossEntropyWithSoftmax = 0.69973267 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0202s; samplesPerSecond = 12368.3
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  11-  20]: CrossEntropyWithSoftmax = 0.71436906 * 250; EvalClassificationError = 0.52000000 * 250; time = 0.0208s; samplesPerSecond = 12038.3
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  21-  30]: CrossEntropyWithSoftmax = 0.72871054 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0193s; samplesPerSecond = 12983.6
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  31-  40]: CrossEntropyWithSoftmax = 0.70038993 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0197s; samplesPerSecond = 12689.1
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  41-  50]: CrossEntropyWithSoftmax = 0.70593820 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0214s; samplesPerSecond = 11661.5
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  51-  60]: CrossEntropyWithSoftmax = 0.71604646 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0211s; samplesPerSecond = 11826.5
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  61-  70]: CrossEntropyWithSoftmax = 0.72247950 * 250; EvalClassificationError = 0.48000000 * 250; time = 0.0213s; samplesPerSecond = 11736.5
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  71-  80]: CrossEntropyWithSoftmax = 0.79884413 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0195s; samplesPerSecond = 12793.0
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  81-  90]: CrossEntropyWithSoftmax = 0.69622447 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0195s; samplesPerSecond = 12821.8
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  91- 100]: CrossEntropyWithSoftmax = 0.70749459 * 250; EvalClassificationError = 0.49200000 * 250; time = 0.0197s; samplesPerSecond = 12679.4
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 101- 110]: CrossEntropyWithSoftmax = 0.71485825 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0219s; samplesPerSecond = 11419.7
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 111- 120]: CrossEntropyWithSoftmax = 0.69579152 * 250; EvalClassificationError = 0.43600000 * 250; time = 0.0172s; samplesPerSecond = 14541.6
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 121- 130]: CrossEntropyWithSoftmax = 0.70174138 * 250; EvalClassificationError = 0.44000000 * 250; time = 0.0235s; samplesPerSecond = 10626.1
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 131- 140]: CrossEntropyWithSoftmax = 0.71926585 * 250; EvalClassificationError = 0.54800000 * 250; time = 0.0199s; samplesPerSecond = 12544.5
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 141- 150]: CrossEntropyWithSoftmax = 0.72009918 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0205s; samplesPerSecond = 12180.3
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 151- 160]: CrossEntropyWithSoftmax = 0.71854574 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0218s; samplesPerSecond = 11479.5
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 161- 170]: CrossEntropyWithSoftmax = 0.74083729 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0223s; samplesPerSecond = 11195.2
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 171- 180]: CrossEntropyWithSoftmax = 0.71762852 * 250; EvalClassificationError = 0.51600000 * 250; time = 0.0220s; samplesPerSecond = 11352.3
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 181- 190]: CrossEntropyWithSoftmax = 0.71530686 * 250; EvalClassificationError = 0.48400000 * 250; time = 0.0226s; samplesPerSecond = 11044.4
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 191- 200]: CrossEntropyWithSoftmax = 0.71768617 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0222s; samplesPerSecond = 11261.3
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 201- 210]: CrossEntropyWithSoftmax = 0.71515312 * 250; EvalClassificationError = 0.53600000 * 250; time = 0.0220s; samplesPerSecond = 11358.5
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 211- 220]: CrossEntropyWithSoftmax = 0.72047061 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0223s; samplesPerSecond = 11205.7
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 221- 230]: CrossEntropyWithSoftmax = 0.72033072 * 250; EvalClassificationError = 0.50800000 * 250; time = 0.0176s; samplesPerSecond = 14165.9
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 231- 240]: CrossEntropyWithSoftmax = 0.71295325 * 250; EvalClassificationError = 0.51200000 * 250; time = 0.0199s; samplesPerSecond = 12589.4
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 241- 250]: CrossEntropyWithSoftmax = 0.69737817 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0224s; samplesPerSecond = 11152.3
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 251- 260]: CrossEntropyWithSoftmax = 0.70251892 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0224s; samplesPerSecond = 11160.7
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 261- 270]: CrossEntropyWithSoftmax = 0.70879703 * 250; EvalClassificationError = 0.54400000 * 250; time = 0.0196s; samplesPerSecond = 12737.6
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 271- 280]: CrossEntropyWithSoftmax = 0.69856459 * 250; EvalClassificationError = 0.52800000 * 250; time = 0.0226s; samplesPerSecond = 11052.7
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 281- 290]: CrossEntropyWithSoftmax = 0.69425908 * 250; EvalClassificationError = 0.44800000 * 250; time = 0.0230s; samplesPerSecond = 10890.9
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 291- 300]: CrossEntropyWithSoftmax = 0.69599736 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0229s; samplesPerSecond = 10927.1
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 301- 310]: CrossEntropyWithSoftmax = 0.69591177 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0197s; samplesPerSecond = 12665.9
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 311- 320]: CrossEntropyWithSoftmax = 0.69133098 * 250; EvalClassificationError = 0.40000000 * 250; time = 0.0213s; samplesPerSecond = 11754.2
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 321- 330]: CrossEntropyWithSoftmax = 0.69822649 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0422s; samplesPerSecond = 5931.2
MPI Rank 0: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 331- 340]: CrossEntropyWithSoftmax = 0.71031540 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0243s; samplesPerSecond = 10303.8
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 341- 350]: CrossEntropyWithSoftmax = 0.70097460 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0218s; samplesPerSecond = 11449.5
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 351- 360]: CrossEntropyWithSoftmax = 0.68927869 * 250; EvalClassificationError = 0.45200000 * 250; time = 0.0208s; samplesPerSecond = 11991.0
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 361- 370]: CrossEntropyWithSoftmax = 0.68908391 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0227s; samplesPerSecond = 11000.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 371- 380]: CrossEntropyWithSoftmax = 0.67796903 * 250; EvalClassificationError = 0.45600000 * 250; time = 0.0237s; samplesPerSecond = 10564.6
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 381- 390]: CrossEntropyWithSoftmax = 0.67863597 * 250; EvalClassificationError = 0.38400000 * 250; time = 0.0223s; samplesPerSecond = 11195.7
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 391- 400]: CrossEntropyWithSoftmax = 0.67150942 * 250; EvalClassificationError = 0.42800000 * 250; time = 0.0221s; samplesPerSecond = 11303.5
MPI Rank 0: 12/15/2016 08:28:19: Finished Epoch[ 1 of 4]: [Training] CrossEntropyWithSoftmax = 0.70804124 * 10000; EvalClassificationError = 0.49380000 * 10000; totalSamplesSeen = 10000; learningRatePerSample = 0.02; epochTime=0.910351s
MPI Rank 0: 12/15/2016 08:28:19: SGD: Saving checkpoint model 'C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/models/Simple.dnn.1'
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:19: Starting Epoch 2: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 0, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.69566501 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0207s; samplesPerSecond = 12062.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.64058132 * 250; EvalClassificationError = 0.22400000 * 250; time = 0.0220s; samplesPerSecond = 11377.6
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.62577220 * 250; EvalClassificationError = 0.30400000 * 250; time = 0.0240s; samplesPerSecond = 10430.6
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.62974806 * 250; EvalClassificationError = 0.34000000 * 250; time = 0.0249s; samplesPerSecond = 10039.0
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.60705925 * 250; EvalClassificationError = 0.22800000 * 250; time = 0.0251s; samplesPerSecond = 9970.9
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.59038701 * 250; EvalClassificationError = 0.18000000 * 250; time = 0.0237s; samplesPerSecond = 10535.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.55033237 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0185s; samplesPerSecond = 13515.0
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.53624221 * 250; EvalClassificationError = 0.23200000 * 250; time = 0.0166s; samplesPerSecond = 15025.8
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.48688371 * 250; EvalClassificationError = 0.12000000 * 250; time = 0.0143s; samplesPerSecond = 17497.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.43212992 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0165s; samplesPerSecond = 15194.8
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.38559585 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0199s; samplesPerSecond = 12587.5
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.34249602 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0183s; samplesPerSecond = 13679.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.28670760 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0192s; samplesPerSecond = 12988.4
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.26990444 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0138s; samplesPerSecond = 18057.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.23285544 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0217s; samplesPerSecond = 11537.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.25464216 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0172s; samplesPerSecond = 14506.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.21254012 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0221s; samplesPerSecond = 11311.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.18708229 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0159s; samplesPerSecond = 15755.0
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.21363045 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0165s; samplesPerSecond = 15172.7
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.23505446 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0171s; samplesPerSecond = 14593.4
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.20180383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0188s; samplesPerSecond = 13317.7
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.19780595 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0183s; samplesPerSecond = 13691.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.16131116 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0157s; samplesPerSecond = 15963.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.16479157 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0201s; samplesPerSecond = 12424.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20226367 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0188s; samplesPerSecond = 13326.2
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.14809084 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0184s; samplesPerSecond = 13610.6
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.19001815 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0220s; samplesPerSecond = 11370.4
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19616891 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0150s; samplesPerSecond = 16620.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.17887471 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0185s; samplesPerSecond = 13505.5
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.14040413 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0177s; samplesPerSecond = 14111.5
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17935154 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0199s; samplesPerSecond = 12558.4
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.13249074 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0185s; samplesPerSecond = 13477.8
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.15483359 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0169s; samplesPerSecond = 14791.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19796158 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0184s; samplesPerSecond = 13555.3
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.13179463 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0188s; samplesPerSecond = 13305.7
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.14028323 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0187s; samplesPerSecond = 13340.4
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12849508 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0192s; samplesPerSecond = 13029.0
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16702669 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0164s; samplesPerSecond = 15243.9
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20390303 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0188s; samplesPerSecond = 13324.1
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14594790 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0191s; samplesPerSecond = 13064.4
MPI Rank 0: 12/15/2016 08:28:19: Finished Epoch[ 2 of 4]: [Training] CrossEntropyWithSoftmax = 0.29447327 * 10000; EvalClassificationError = 0.11490000 * 10000; totalSamplesSeen = 20000; learningRatePerSample = 0.0080000004; epochTime=0.788537s
MPI Rank 0: 12/15/2016 08:28:19: SGD: Saving checkpoint model 'C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/models/Simple.dnn.2'
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:19: Starting Epoch 3: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 0, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12813297 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0170s; samplesPerSecond = 14686.0
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.17615627 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0202s; samplesPerSecond = 12355.4
MPI Rank 0: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14587002 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0186s; samplesPerSecond = 13449.5
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15938467 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0173s; samplesPerSecond = 14430.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.17100048 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0174s; samplesPerSecond = 14397.6
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18281055 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0190s; samplesPerSecond = 13171.1
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14781538 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0213s; samplesPerSecond = 11719.5
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18045490 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0177s; samplesPerSecond = 14156.3
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15847198 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0205s; samplesPerSecond = 12203.5
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14513057 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0192s; samplesPerSecond = 13006.6
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13519579 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0200s; samplesPerSecond = 12499.4
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13723645 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0207s; samplesPerSecond = 12061.6
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11692067 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0191s; samplesPerSecond = 13111.7
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16729042 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0194s; samplesPerSecond = 12912.6
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12836481 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0143s; samplesPerSecond = 17507.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17320383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0172s; samplesPerSecond = 14540.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17634558 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0213s; samplesPerSecond = 11748.7
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14124514 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0208s; samplesPerSecond = 12014.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19167717 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0213s; samplesPerSecond = 11747.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20913003 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0198s; samplesPerSecond = 12637.1
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18460750 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0214s; samplesPerSecond = 11671.9
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18188216 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0214s; samplesPerSecond = 11673.5
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14069101 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0145s; samplesPerSecond = 17283.1
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14812247 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0211s; samplesPerSecond = 11843.9
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20274092 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0200s; samplesPerSecond = 12493.8
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12887866 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0179s; samplesPerSecond = 13988.4
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18595255 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0179s; samplesPerSecond = 13988.4
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19565326 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0208s; samplesPerSecond = 12022.1
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16678525 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0148s; samplesPerSecond = 16852.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12552459 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0124s; samplesPerSecond = 20132.1
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17414176 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0229s; samplesPerSecond = 10929.9
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12295855 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0204s; samplesPerSecond = 12259.7
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14757012 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0204s; samplesPerSecond = 12250.7
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19785856 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0186s; samplesPerSecond = 13433.6
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12600285 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0210s; samplesPerSecond = 11901.4
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13742899 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0183s; samplesPerSecond = 13666.4
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12847649 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0220s; samplesPerSecond = 11375.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16652415 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0187s; samplesPerSecond = 13377.6
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20675722 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0240s; samplesPerSecond = 10426.7
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14562268 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0212s; samplesPerSecond = 11790.8
MPI Rank 0: 12/15/2016 08:28:20: Finished Epoch[ 3 of 4]: [Training] CrossEntropyWithSoftmax = 0.15965044 * 10000; EvalClassificationError = 0.07650000 * 10000; totalSamplesSeen = 30000; learningRatePerSample = 0.0080000004; epochTime=0.803128s
MPI Rank 0: 12/15/2016 08:28:20: SGD: Saving checkpoint model 'C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/models/Simple.dnn.3'
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:20: Starting Epoch 4: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:20: Starting minibatch loop, DataParallelSGD training (myRank = 0, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12392293 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0227s; samplesPerSecond = 11011.8
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.18033423 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0179s; samplesPerSecond = 13975.8
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14283999 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0239s; samplesPerSecond = 10469.9
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15662489 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0204s; samplesPerSecond = 12280.2
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.16985800 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0222s; samplesPerSecond = 11278.5
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18190608 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0232s; samplesPerSecond = 10754.5
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14495469 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0230s; samplesPerSecond = 10853.0
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18022153 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0234s; samplesPerSecond = 10677.8
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15852460 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0224s; samplesPerSecond = 11156.7
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14466589 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0208s; samplesPerSecond = 12044.7
MPI Rank 0: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13346404 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0215s; samplesPerSecond = 11607.4
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13683061 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0218s; samplesPerSecond = 11456.3
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11589011 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0239s; samplesPerSecond = 10452.8
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16881193 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0216s; samplesPerSecond = 11585.9
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12736965 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0239s; samplesPerSecond = 10440.2
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17123604 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0243s; samplesPerSecond = 10288.9
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17706403 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0231s; samplesPerSecond = 10814.1
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14104103 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0151s; samplesPerSecond = 16538.8
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19313360 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0156s; samplesPerSecond = 16017.4
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20870745 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0155s; samplesPerSecond = 16152.0
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18510294 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0183s; samplesPerSecond = 13683.6
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18167136 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0160s; samplesPerSecond = 15661.2
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14026275 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0152s; samplesPerSecond = 16484.2
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14811532 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0174s; samplesPerSecond = 14353.0
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20368129 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0168s; samplesPerSecond = 14846.5
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12819271 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0154s; samplesPerSecond = 16209.6
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18632901 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0158s; samplesPerSecond = 15853.9
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19568751 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0191s; samplesPerSecond = 13123.4
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16449544 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0147s; samplesPerSecond = 17008.0
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12454886 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0181s; samplesPerSecond = 13779.4
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17307192 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0152s; samplesPerSecond = 16398.8
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12249522 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0177s; samplesPerSecond = 14103.6
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14709682 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0132s; samplesPerSecond = 18942.3
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19789048 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0172s; samplesPerSecond = 14501.2
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12572171 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0167s; samplesPerSecond = 14951.3
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13732392 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0185s; samplesPerSecond = 13493.1
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12857569 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0148s; samplesPerSecond = 16896.5
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16653116 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0184s; samplesPerSecond = 13583.3
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20715346 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0189s; samplesPerSecond = 13229.6
MPI Rank 0: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14571729 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0191s; samplesPerSecond = 13068.5
MPI Rank 0: 12/15/2016 08:28:21: Finished Epoch[ 4 of 4]: [Training] CrossEntropyWithSoftmax = 0.15917665 * 10000; EvalClassificationError = 0.07660000 * 10000; totalSamplesSeen = 40000; learningRatePerSample = 0.0080000004; epochTime=0.796901s
MPI Rank 0: 12/15/2016 08:28:21: SGD: Saving checkpoint model 'C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/models/Simple.dnn'
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:21: Action "train" complete.
MPI Rank 0: 
MPI Rank 0: 12/15/2016 08:28:21: __COMPLETED__
MPI Rank 1: 12/15/2016 08:28:17: Redirecting stderr to file C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr_SimpleMultiGPU.logrank1
MPI Rank 1: CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16
MPI Rank 1: 
MPI Rank 1: C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
MPI Rank 1: 12/15/2016 08:28:17: Using 1 CPU threads.
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: ##############################################################################
MPI Rank 1: 12/15/2016 08:28:17: #                                                                            #
MPI Rank 1: 12/15/2016 08:28:17: # SimpleMultiGPU command (train action)                                      #
MPI Rank 1: 12/15/2016 08:28:17: #                                                                            #
MPI Rank 1: 12/15/2016 08:28:17: ##############################################################################
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: 
MPI Rank 1: Creating virgin network.
MPI Rank 1: SimpleNetworkBuilder Using CPU
MPI Rank 1: 12/15/2016 08:28:17: 
MPI Rank 1: Model has 25 nodes. Using CPU.
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: Training criterion:   CrossEntropyWithSoftmax = CrossEntropyWithSoftmax
MPI Rank 1: 12/15/2016 08:28:17: Evaluation criterion: EvalClassificationError = ClassificationError
MPI Rank 1: 
MPI Rank 1: 
MPI Rank 1: Allocating matrices for forward and/or backward propagation.
MPI Rank 1: 
MPI Rank 1: Memory Sharing: Out of 40 matrices, 19 are shared as 8, and 21 are not shared.
MPI Rank 1: 
MPI Rank 1: 	{ W1 : [50 x 50] (gradient)
MPI Rank 1: 	  W1*H1+B1 : [50 x 1 x *] }
MPI Rank 1: 	{ B0 : [50 x 1] (gradient)
MPI Rank 1: 	  H1 : [50 x 1 x *] (gradient)
MPI Rank 1: 	  W1*H1+B1 : [50 x 1 x *] (gradient)
MPI Rank 1: 	  W2*H1 : [2 x 1 x *] }
MPI Rank 1: 	{ H2 : [50 x 1 x *]
MPI Rank 1: 	  W1*H1 : [50 x 1 x *] (gradient) }
MPI Rank 1: 	{ H1 : [50 x 1 x *]
MPI Rank 1: 	  W0*features : [50 x *] (gradient) }
MPI Rank 1: 	{ W0 : [50 x 2] (gradient)
MPI Rank 1: 	  W0*features+B0 : [50 x 1 x *] }
MPI Rank 1: 	{ HLast : [2 x 1 x *]
MPI Rank 1: 	  W2 : [2 x 50] (gradient) }
MPI Rank 1: 	{ B1 : [50 x 1] (gradient)
MPI Rank 1: 	  H2 : [50 x 1 x *] (gradient)
MPI Rank 1: 	  HLast : [2 x 1 x *] (gradient) }
MPI Rank 1: 	{ W0*features+B0 : [50 x 1 x *] (gradient)
MPI Rank 1: 	  W1*H1 : [50 x 1 x *] }
MPI Rank 1: 
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: Training 2802 parameters in 6 out of 6 parameter tensors and 15 nodes with gradient:
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: 	Node 'B0' (LearnableParameter operation) : [50 x 1]
MPI Rank 1: 12/15/2016 08:28:17: 	Node 'B1' (LearnableParameter operation) : [50 x 1]
MPI Rank 1: 12/15/2016 08:28:17: 	Node 'B2' (LearnableParameter operation) : [2 x 1]
MPI Rank 1: 12/15/2016 08:28:17: 	Node 'W0' (LearnableParameter operation) : [50 x 2]
MPI Rank 1: 12/15/2016 08:28:17: 	Node 'W1' (LearnableParameter operation) : [50 x 50]
MPI Rank 1: 12/15/2016 08:28:17: 	Node 'W2' (LearnableParameter operation) : [2 x 50]
MPI Rank 1: 
MPI Rank 1: Initializing dataParallelSGD with FP32 aggregation.
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: Precomputing --> 3 PreCompute nodes found.
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: 	MeanOfFeatures = Mean()
MPI Rank 1: 12/15/2016 08:28:17: 	InvStdOfFeatures = InvStdDev()
MPI Rank 1: 12/15/2016 08:28:17: 	Prior = Mean()
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:17: Precomputing --> Completed.
MPI Rank 1: 
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:18: Starting Epoch 1: learning rate per sample = 0.020000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:18: Starting minibatch loop, DataParallelSGD training (myRank = 1, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[   1-  10]: CrossEntropyWithSoftmax = 0.69973267 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0203s; samplesPerSecond = 12316.5
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  11-  20]: CrossEntropyWithSoftmax = 0.71436906 * 250; EvalClassificationError = 0.52000000 * 250; time = 0.0210s; samplesPerSecond = 11929.8
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  21-  30]: CrossEntropyWithSoftmax = 0.72871054 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0194s; samplesPerSecond = 12881.9
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  31-  40]: CrossEntropyWithSoftmax = 0.70038993 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0199s; samplesPerSecond = 12582.4
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  41-  50]: CrossEntropyWithSoftmax = 0.70593820 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0214s; samplesPerSecond = 11660.4
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  51-  60]: CrossEntropyWithSoftmax = 0.71604646 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0211s; samplesPerSecond = 11864.1
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  61-  70]: CrossEntropyWithSoftmax = 0.72247950 * 250; EvalClassificationError = 0.48000000 * 250; time = 0.0214s; samplesPerSecond = 11701.9
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  71-  80]: CrossEntropyWithSoftmax = 0.79884413 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0195s; samplesPerSecond = 12800.8
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  81-  90]: CrossEntropyWithSoftmax = 0.69622447 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0195s; samplesPerSecond = 12817.9
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  91- 100]: CrossEntropyWithSoftmax = 0.70749459 * 250; EvalClassificationError = 0.49200000 * 250; time = 0.0197s; samplesPerSecond = 12669.8
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 101- 110]: CrossEntropyWithSoftmax = 0.71485825 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0220s; samplesPerSecond = 11343.5
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 111- 120]: CrossEntropyWithSoftmax = 0.69579152 * 250; EvalClassificationError = 0.43600000 * 250; time = 0.0173s; samplesPerSecond = 14469.3
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 121- 130]: CrossEntropyWithSoftmax = 0.70174138 * 250; EvalClassificationError = 0.44000000 * 250; time = 0.0239s; samplesPerSecond = 10448.9
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 131- 140]: CrossEntropyWithSoftmax = 0.71926585 * 250; EvalClassificationError = 0.54800000 * 250; time = 0.0199s; samplesPerSecond = 12554.6
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 141- 150]: CrossEntropyWithSoftmax = 0.72009918 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0208s; samplesPerSecond = 12030.8
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 151- 160]: CrossEntropyWithSoftmax = 0.71854574 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0218s; samplesPerSecond = 11484.7
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 161- 170]: CrossEntropyWithSoftmax = 0.74083729 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0222s; samplesPerSecond = 11242.0
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 171- 180]: CrossEntropyWithSoftmax = 0.71762852 * 250; EvalClassificationError = 0.51600000 * 250; time = 0.0221s; samplesPerSecond = 11303.5
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 181- 190]: CrossEntropyWithSoftmax = 0.71530686 * 250; EvalClassificationError = 0.48400000 * 250; time = 0.0226s; samplesPerSecond = 11074.7
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 191- 200]: CrossEntropyWithSoftmax = 0.71768617 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0222s; samplesPerSecond = 11263.3
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 201- 210]: CrossEntropyWithSoftmax = 0.71515312 * 250; EvalClassificationError = 0.53600000 * 250; time = 0.0221s; samplesPerSecond = 11304.0
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 211- 220]: CrossEntropyWithSoftmax = 0.72047061 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0223s; samplesPerSecond = 11198.2
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 221- 230]: CrossEntropyWithSoftmax = 0.72033072 * 250; EvalClassificationError = 0.50800000 * 250; time = 0.0159s; samplesPerSecond = 15764.9
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 231- 240]: CrossEntropyWithSoftmax = 0.71295325 * 250; EvalClassificationError = 0.51200000 * 250; time = 0.0216s; samplesPerSecond = 11554.8
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 241- 250]: CrossEntropyWithSoftmax = 0.69737817 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0226s; samplesPerSecond = 11058.5
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 251- 260]: CrossEntropyWithSoftmax = 0.70251892 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0226s; samplesPerSecond = 11083.0
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 261- 270]: CrossEntropyWithSoftmax = 0.70879703 * 250; EvalClassificationError = 0.54400000 * 250; time = 0.0196s; samplesPerSecond = 12738.9
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 271- 280]: CrossEntropyWithSoftmax = 0.69856459 * 250; EvalClassificationError = 0.52800000 * 250; time = 0.0226s; samplesPerSecond = 11054.1
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 281- 290]: CrossEntropyWithSoftmax = 0.69425908 * 250; EvalClassificationError = 0.44800000 * 250; time = 0.0229s; samplesPerSecond = 10893.7
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 291- 300]: CrossEntropyWithSoftmax = 0.69599736 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0231s; samplesPerSecond = 10839.4
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 301- 310]: CrossEntropyWithSoftmax = 0.69591177 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0198s; samplesPerSecond = 12654.4
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 311- 320]: CrossEntropyWithSoftmax = 0.69133098 * 250; EvalClassificationError = 0.40000000 * 250; time = 0.0213s; samplesPerSecond = 11723.3
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 321- 330]: CrossEntropyWithSoftmax = 0.69822649 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0419s; samplesPerSecond = 5973.1
MPI Rank 1: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 331- 340]: CrossEntropyWithSoftmax = 0.71031540 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0247s; samplesPerSecond = 10103.5
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 341- 350]: CrossEntropyWithSoftmax = 0.70097460 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0218s; samplesPerSecond = 11446.4
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 351- 360]: CrossEntropyWithSoftmax = 0.68927869 * 250; EvalClassificationError = 0.45200000 * 250; time = 0.0209s; samplesPerSecond = 11984.7
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 361- 370]: CrossEntropyWithSoftmax = 0.68908391 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0232s; samplesPerSecond = 10797.3
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 371- 380]: CrossEntropyWithSoftmax = 0.67796903 * 250; EvalClassificationError = 0.45600000 * 250; time = 0.0237s; samplesPerSecond = 10569.0
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 381- 390]: CrossEntropyWithSoftmax = 0.67863597 * 250; EvalClassificationError = 0.38400000 * 250; time = 0.0224s; samplesPerSecond = 11144.3
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 391- 400]: CrossEntropyWithSoftmax = 0.67150942 * 250; EvalClassificationError = 0.42800000 * 250; time = 0.0221s; samplesPerSecond = 11308.1
MPI Rank 1: 12/15/2016 08:28:19: Finished Epoch[ 1 of 4]: [Training] CrossEntropyWithSoftmax = 0.70804124 * 10000; EvalClassificationError = 0.49380000 * 10000; totalSamplesSeen = 10000; learningRatePerSample = 0.02; epochTime=0.910338s
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:19: Starting Epoch 2: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 1, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.69566501 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0206s; samplesPerSecond = 12111.8
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.64058132 * 250; EvalClassificationError = 0.22400000 * 250; time = 0.0222s; samplesPerSecond = 11251.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.62577220 * 250; EvalClassificationError = 0.30400000 * 250; time = 0.0242s; samplesPerSecond = 10322.9
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.62974806 * 250; EvalClassificationError = 0.34000000 * 250; time = 0.0249s; samplesPerSecond = 10041.4
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.60705925 * 250; EvalClassificationError = 0.22800000 * 250; time = 0.0251s; samplesPerSecond = 9972.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.59038701 * 250; EvalClassificationError = 0.18000000 * 250; time = 0.0238s; samplesPerSecond = 10498.5
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.55033237 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0185s; samplesPerSecond = 13542.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.53624221 * 250; EvalClassificationError = 0.23200000 * 250; time = 0.0171s; samplesPerSecond = 14610.5
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.48688371 * 250; EvalClassificationError = 0.12000000 * 250; time = 0.0142s; samplesPerSecond = 17601.9
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.43212992 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0164s; samplesPerSecond = 15198.5
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.38559585 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0200s; samplesPerSecond = 12507.5
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.34249602 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0183s; samplesPerSecond = 13685.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.28670760 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0196s; samplesPerSecond = 12740.2
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.26990444 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0138s; samplesPerSecond = 18058.4
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.23285544 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0217s; samplesPerSecond = 11538.8
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.25464216 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0172s; samplesPerSecond = 14497.8
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.21254012 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0224s; samplesPerSecond = 11181.7
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.18708229 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0159s; samplesPerSecond = 15755.0
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.21363045 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0165s; samplesPerSecond = 15171.7
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.23505446 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0171s; samplesPerSecond = 14586.6
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.20180383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0186s; samplesPerSecond = 13469.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.19780595 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0185s; samplesPerSecond = 13529.6
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.16131116 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0157s; samplesPerSecond = 15970.4
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.16479157 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0201s; samplesPerSecond = 12427.3
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20226367 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0188s; samplesPerSecond = 13327.6
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.14809084 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0184s; samplesPerSecond = 13600.3
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.19001815 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0219s; samplesPerSecond = 11391.6
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19616891 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0149s; samplesPerSecond = 16768.4
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.17887471 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0185s; samplesPerSecond = 13501.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.14040413 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0169s; samplesPerSecond = 14827.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17935154 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0210s; samplesPerSecond = 11922.4
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.13249074 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0190s; samplesPerSecond = 13168.3
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.15483359 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0154s; samplesPerSecond = 16182.3
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19796158 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0199s; samplesPerSecond = 12581.1
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.13179463 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0188s; samplesPerSecond = 13302.8
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.14028323 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0188s; samplesPerSecond = 13269.6
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12849508 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0191s; samplesPerSecond = 13071.9
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16702669 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0165s; samplesPerSecond = 15125.8
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20390303 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0187s; samplesPerSecond = 13336.2
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14594790 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0191s; samplesPerSecond = 13067.8
MPI Rank 1: 12/15/2016 08:28:19: Finished Epoch[ 2 of 4]: [Training] CrossEntropyWithSoftmax = 0.29447327 * 10000; EvalClassificationError = 0.11490000 * 10000; totalSamplesSeen = 20000; learningRatePerSample = 0.0080000004; epochTime=0.788536s
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:19: Starting Epoch 3: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 1, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12813297 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0173s; samplesPerSecond = 14482.7
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.17615627 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0204s; samplesPerSecond = 12248.3
MPI Rank 1: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14587002 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0186s; samplesPerSecond = 13455.3
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15938467 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0177s; samplesPerSecond = 14086.9
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.17100048 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0174s; samplesPerSecond = 14407.6
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18281055 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0194s; samplesPerSecond = 12894.6
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14781538 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0214s; samplesPerSecond = 11687.2
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18045490 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0178s; samplesPerSecond = 14065.5
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15847198 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0209s; samplesPerSecond = 11949.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14513057 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0195s; samplesPerSecond = 12839.6
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13519579 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0190s; samplesPerSecond = 13185.0
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13723645 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0218s; samplesPerSecond = 11493.7
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11692067 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0187s; samplesPerSecond = 13343.3
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16729042 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0197s; samplesPerSecond = 12704.5
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12836481 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0143s; samplesPerSecond = 17491.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17320383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0176s; samplesPerSecond = 14179.6
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17634558 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0210s; samplesPerSecond = 11897.4
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14124514 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0211s; samplesPerSecond = 11849.5
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19167717 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0213s; samplesPerSecond = 11745.4
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20913003 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0197s; samplesPerSecond = 12659.5
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18460750 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0215s; samplesPerSecond = 11650.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18188216 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0214s; samplesPerSecond = 11674.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14069101 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0145s; samplesPerSecond = 17295.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14812247 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0215s; samplesPerSecond = 11611.7
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20274092 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0200s; samplesPerSecond = 12480.7
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12887866 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0181s; samplesPerSecond = 13814.4
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18595255 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0184s; samplesPerSecond = 13618.8
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19565326 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0208s; samplesPerSecond = 12038.9
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16678525 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0148s; samplesPerSecond = 16844.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12552459 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0129s; samplesPerSecond = 19434.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17414176 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0229s; samplesPerSecond = 10924.7
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12295855 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0209s; samplesPerSecond = 11988.7
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14757012 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0182s; samplesPerSecond = 13739.3
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19785856 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0208s; samplesPerSecond = 11999.0
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12600285 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0210s; samplesPerSecond = 11901.4
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13742899 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0183s; samplesPerSecond = 13669.4
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12847649 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0220s; samplesPerSecond = 11374.0
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16652415 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0191s; samplesPerSecond = 13076.7
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20675722 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0240s; samplesPerSecond = 10419.3
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14562268 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0212s; samplesPerSecond = 11768.0
MPI Rank 1: 12/15/2016 08:28:20: Finished Epoch[ 3 of 4]: [Training] CrossEntropyWithSoftmax = 0.15965044 * 10000; EvalClassificationError = 0.07650000 * 10000; totalSamplesSeen = 30000; learningRatePerSample = 0.0080000004; epochTime=0.803127s
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:20: Starting Epoch 4: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:20: Starting minibatch loop, DataParallelSGD training (myRank = 1, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12392293 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0229s; samplesPerSecond = 10904.6
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.18033423 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0183s; samplesPerSecond = 13651.5
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14283999 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0239s; samplesPerSecond = 10468.6
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15662489 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0207s; samplesPerSecond = 12069.7
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.16985800 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0221s; samplesPerSecond = 11301.5
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18190608 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0232s; samplesPerSecond = 10753.6
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14495469 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0230s; samplesPerSecond = 10850.2
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18022153 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0234s; samplesPerSecond = 10685.1
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15852460 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0228s; samplesPerSecond = 10965.4
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14466589 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0208s; samplesPerSecond = 12031.4
MPI Rank 1: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13346404 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0220s; samplesPerSecond = 11370.4
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13683061 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0218s; samplesPerSecond = 11450.0
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11589011 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0239s; samplesPerSecond = 10446.3
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16881193 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0217s; samplesPerSecond = 11541.5
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12736965 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0244s; samplesPerSecond = 10249.3
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17123604 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0244s; samplesPerSecond = 10245.9
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17706403 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0231s; samplesPerSecond = 10840.3
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14104103 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0151s; samplesPerSecond = 16519.1
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19313360 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0156s; samplesPerSecond = 16068.9
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20870745 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0154s; samplesPerSecond = 16185.4
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18510294 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0184s; samplesPerSecond = 13619.5
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18167136 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0159s; samplesPerSecond = 15749.0
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14026275 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0156s; samplesPerSecond = 16032.8
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14811532 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0174s; samplesPerSecond = 14362.9
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20368129 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0166s; samplesPerSecond = 15044.8
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12819271 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0156s; samplesPerSecond = 15989.8
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18632901 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0162s; samplesPerSecond = 15442.6
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19568751 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0190s; samplesPerSecond = 13133.7
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16449544 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0152s; samplesPerSecond = 16486.4
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12454886 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0183s; samplesPerSecond = 13641.8
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17307192 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0150s; samplesPerSecond = 16696.7
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12249522 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0177s; samplesPerSecond = 14092.4
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14709682 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0132s; samplesPerSecond = 18950.9
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19789048 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0177s; samplesPerSecond = 14125.9
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12572171 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0167s; samplesPerSecond = 14970.1
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13732392 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0190s; samplesPerSecond = 13175.9
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12857569 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0153s; samplesPerSecond = 16320.7
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16653116 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0179s; samplesPerSecond = 13968.8
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20715346 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0194s; samplesPerSecond = 12869.4
MPI Rank 1: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14571729 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0192s; samplesPerSecond = 13005.9
MPI Rank 1: 12/15/2016 08:28:21: Finished Epoch[ 4 of 4]: [Training] CrossEntropyWithSoftmax = 0.15917665 * 10000; EvalClassificationError = 0.07660000 * 10000; totalSamplesSeen = 40000; learningRatePerSample = 0.0080000004; epochTime=0.796897s
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:21: Action "train" complete.
MPI Rank 1: 
MPI Rank 1: 12/15/2016 08:28:21: __COMPLETED__
MPI Rank 2: 12/15/2016 08:28:17: Redirecting stderr to file C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr_SimpleMultiGPU.logrank2
MPI Rank 2: CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16
MPI Rank 2: 
MPI Rank 2: C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
MPI Rank 2: 12/15/2016 08:28:17: Using 1 CPU threads.
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: ##############################################################################
MPI Rank 2: 12/15/2016 08:28:17: #                                                                            #
MPI Rank 2: 12/15/2016 08:28:17: # SimpleMultiGPU command (train action)                                      #
MPI Rank 2: 12/15/2016 08:28:17: #                                                                            #
MPI Rank 2: 12/15/2016 08:28:17: ##############################################################################
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: 
MPI Rank 2: Creating virgin network.
MPI Rank 2: SimpleNetworkBuilder Using CPU
MPI Rank 2: 12/15/2016 08:28:17: 
MPI Rank 2: Model has 25 nodes. Using CPU.
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: Training criterion:   CrossEntropyWithSoftmax = CrossEntropyWithSoftmax
MPI Rank 2: 12/15/2016 08:28:17: Evaluation criterion: EvalClassificationError = ClassificationError
MPI Rank 2: 
MPI Rank 2: 
MPI Rank 2: Allocating matrices for forward and/or backward propagation.
MPI Rank 2: 
MPI Rank 2: Memory Sharing: Out of 40 matrices, 19 are shared as 8, and 21 are not shared.
MPI Rank 2: 
MPI Rank 2: 	{ W0 : [50 x 2] (gradient)
MPI Rank 2: 	  W0*features+B0 : [50 x 1 x *] }
MPI Rank 2: 	{ W0*features+B0 : [50 x 1 x *] (gradient)
MPI Rank 2: 	  W1*H1 : [50 x 1 x *] }
MPI Rank 2: 	{ W1 : [50 x 50] (gradient)
MPI Rank 2: 	  W1*H1+B1 : [50 x 1 x *] }
MPI Rank 2: 	{ B1 : [50 x 1] (gradient)
MPI Rank 2: 	  H2 : [50 x 1 x *] (gradient)
MPI Rank 2: 	  HLast : [2 x 1 x *] (gradient) }
MPI Rank 2: 	{ H1 : [50 x 1 x *]
MPI Rank 2: 	  W0*features : [50 x *] (gradient) }
MPI Rank 2: 	{ HLast : [2 x 1 x *]
MPI Rank 2: 	  W2 : [2 x 50] (gradient) }
MPI Rank 2: 	{ B0 : [50 x 1] (gradient)
MPI Rank 2: 	  H1 : [50 x 1 x *] (gradient)
MPI Rank 2: 	  W1*H1+B1 : [50 x 1 x *] (gradient)
MPI Rank 2: 	  W2*H1 : [2 x 1 x *] }
MPI Rank 2: 	{ H2 : [50 x 1 x *]
MPI Rank 2: 	  W1*H1 : [50 x 1 x *] (gradient) }
MPI Rank 2: 
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: Training 2802 parameters in 6 out of 6 parameter tensors and 15 nodes with gradient:
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: 	Node 'B0' (LearnableParameter operation) : [50 x 1]
MPI Rank 2: 12/15/2016 08:28:17: 	Node 'B1' (LearnableParameter operation) : [50 x 1]
MPI Rank 2: 12/15/2016 08:28:17: 	Node 'B2' (LearnableParameter operation) : [2 x 1]
MPI Rank 2: 12/15/2016 08:28:17: 	Node 'W0' (LearnableParameter operation) : [50 x 2]
MPI Rank 2: 12/15/2016 08:28:17: 	Node 'W1' (LearnableParameter operation) : [50 x 50]
MPI Rank 2: 12/15/2016 08:28:17: 	Node 'W2' (LearnableParameter operation) : [2 x 50]
MPI Rank 2: 
MPI Rank 2: Initializing dataParallelSGD with FP32 aggregation.
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: Precomputing --> 3 PreCompute nodes found.
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: 	MeanOfFeatures = Mean()
MPI Rank 2: 12/15/2016 08:28:17: 	InvStdOfFeatures = InvStdDev()
MPI Rank 2: 12/15/2016 08:28:17: 	Prior = Mean()
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:17: Precomputing --> Completed.
MPI Rank 2: 
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:18: Starting Epoch 1: learning rate per sample = 0.020000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:18: Starting minibatch loop, DataParallelSGD training (myRank = 2, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[   1-  10]: CrossEntropyWithSoftmax = 0.69973267 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0200s; samplesPerSecond = 12496.9
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  11-  20]: CrossEntropyWithSoftmax = 0.71436906 * 250; EvalClassificationError = 0.52000000 * 250; time = 0.0208s; samplesPerSecond = 12037.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  21-  30]: CrossEntropyWithSoftmax = 0.72871054 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0194s; samplesPerSecond = 12891.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  31-  40]: CrossEntropyWithSoftmax = 0.70038993 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0198s; samplesPerSecond = 12618.0
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  41-  50]: CrossEntropyWithSoftmax = 0.70593820 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0214s; samplesPerSecond = 11662.1
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  51-  60]: CrossEntropyWithSoftmax = 0.71604646 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0211s; samplesPerSecond = 11823.7
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  61-  70]: CrossEntropyWithSoftmax = 0.72247950 * 250; EvalClassificationError = 0.48000000 * 250; time = 0.0214s; samplesPerSecond = 11703.6
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  71-  80]: CrossEntropyWithSoftmax = 0.79884413 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0196s; samplesPerSecond = 12783.1
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  81-  90]: CrossEntropyWithSoftmax = 0.69622447 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0192s; samplesPerSecond = 12989.0
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  91- 100]: CrossEntropyWithSoftmax = 0.70749459 * 250; EvalClassificationError = 0.49200000 * 250; time = 0.0197s; samplesPerSecond = 12676.8
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 101- 110]: CrossEntropyWithSoftmax = 0.71485825 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0221s; samplesPerSecond = 11332.7
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 111- 120]: CrossEntropyWithSoftmax = 0.69579152 * 250; EvalClassificationError = 0.43600000 * 250; time = 0.0172s; samplesPerSecond = 14566.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 121- 130]: CrossEntropyWithSoftmax = 0.70174138 * 250; EvalClassificationError = 0.44000000 * 250; time = 0.0239s; samplesPerSecond = 10445.0
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 131- 140]: CrossEntropyWithSoftmax = 0.71926585 * 250; EvalClassificationError = 0.54800000 * 250; time = 0.0199s; samplesPerSecond = 12554.6
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 141- 150]: CrossEntropyWithSoftmax = 0.72009918 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0208s; samplesPerSecond = 12034.9
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 151- 160]: CrossEntropyWithSoftmax = 0.71854574 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0215s; samplesPerSecond = 11608.5
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 161- 170]: CrossEntropyWithSoftmax = 0.74083729 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0223s; samplesPerSecond = 11200.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 171- 180]: CrossEntropyWithSoftmax = 0.71762852 * 250; EvalClassificationError = 0.51600000 * 250; time = 0.0220s; samplesPerSecond = 11339.9
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 181- 190]: CrossEntropyWithSoftmax = 0.71530686 * 250; EvalClassificationError = 0.48400000 * 250; time = 0.0227s; samplesPerSecond = 10997.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 191- 200]: CrossEntropyWithSoftmax = 0.71768617 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0222s; samplesPerSecond = 11259.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 201- 210]: CrossEntropyWithSoftmax = 0.71515312 * 250; EvalClassificationError = 0.53600000 * 250; time = 0.0221s; samplesPerSecond = 11297.9
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 211- 220]: CrossEntropyWithSoftmax = 0.72047061 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0221s; samplesPerSecond = 11322.5
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 221- 230]: CrossEntropyWithSoftmax = 0.72033072 * 250; EvalClassificationError = 0.50800000 * 250; time = 0.0175s; samplesPerSecond = 14324.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 231- 240]: CrossEntropyWithSoftmax = 0.71295325 * 250; EvalClassificationError = 0.51200000 * 250; time = 0.0199s; samplesPerSecond = 12586.8
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 241- 250]: CrossEntropyWithSoftmax = 0.69737817 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0226s; samplesPerSecond = 11060.5
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 251- 260]: CrossEntropyWithSoftmax = 0.70251892 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0226s; samplesPerSecond = 11080.6
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 261- 270]: CrossEntropyWithSoftmax = 0.70879703 * 250; EvalClassificationError = 0.54400000 * 250; time = 0.0195s; samplesPerSecond = 12789.7
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 271- 280]: CrossEntropyWithSoftmax = 0.69856459 * 250; EvalClassificationError = 0.52800000 * 250; time = 0.0226s; samplesPerSecond = 11048.3
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 281- 290]: CrossEntropyWithSoftmax = 0.69425908 * 250; EvalClassificationError = 0.44800000 * 250; time = 0.0228s; samplesPerSecond = 10964.4
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 291- 300]: CrossEntropyWithSoftmax = 0.69599736 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0231s; samplesPerSecond = 10842.2
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 301- 310]: CrossEntropyWithSoftmax = 0.69591177 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0196s; samplesPerSecond = 12776.6
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 311- 320]: CrossEntropyWithSoftmax = 0.69133098 * 250; EvalClassificationError = 0.40000000 * 250; time = 0.0213s; samplesPerSecond = 11725.0
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 321- 330]: CrossEntropyWithSoftmax = 0.69822649 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0423s; samplesPerSecond = 5906.7
MPI Rank 2: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 331- 340]: CrossEntropyWithSoftmax = 0.71031540 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0243s; samplesPerSecond = 10306.3
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 341- 350]: CrossEntropyWithSoftmax = 0.70097460 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0216s; samplesPerSecond = 11548.9
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 351- 360]: CrossEntropyWithSoftmax = 0.68927869 * 250; EvalClassificationError = 0.45200000 * 250; time = 0.0207s; samplesPerSecond = 12094.8
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 361- 370]: CrossEntropyWithSoftmax = 0.68908391 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0231s; samplesPerSecond = 10802.9
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 371- 380]: CrossEntropyWithSoftmax = 0.67796903 * 250; EvalClassificationError = 0.45600000 * 250; time = 0.0237s; samplesPerSecond = 10565.9
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 381- 390]: CrossEntropyWithSoftmax = 0.67863597 * 250; EvalClassificationError = 0.38400000 * 250; time = 0.0223s; samplesPerSecond = 11217.8
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 391- 400]: CrossEntropyWithSoftmax = 0.67150942 * 250; EvalClassificationError = 0.42800000 * 250; time = 0.0221s; samplesPerSecond = 11307.6
MPI Rank 2: 12/15/2016 08:28:19: Finished Epoch[ 1 of 4]: [Training] CrossEntropyWithSoftmax = 0.70804124 * 10000; EvalClassificationError = 0.49380000 * 10000; totalSamplesSeen = 10000; learningRatePerSample = 0.02; epochTime=0.910171s
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:19: Starting Epoch 2: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 2, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.69566501 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0205s; samplesPerSecond = 12209.4
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.64058132 * 250; EvalClassificationError = 0.22400000 * 250; time = 0.0222s; samplesPerSecond = 11242.0
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.62577220 * 250; EvalClassificationError = 0.30400000 * 250; time = 0.0242s; samplesPerSecond = 10324.6
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.62974806 * 250; EvalClassificationError = 0.34000000 * 250; time = 0.0247s; samplesPerSecond = 10110.4
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.60705925 * 250; EvalClassificationError = 0.22800000 * 250; time = 0.0248s; samplesPerSecond = 10071.7
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.59038701 * 250; EvalClassificationError = 0.18000000 * 250; time = 0.0238s; samplesPerSecond = 10509.1
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.55033237 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0185s; samplesPerSecond = 13502.6
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.53624221 * 250; EvalClassificationError = 0.23200000 * 250; time = 0.0171s; samplesPerSecond = 14647.3
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.48688371 * 250; EvalClassificationError = 0.12000000 * 250; time = 0.0142s; samplesPerSecond = 17568.5
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.43212992 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0165s; samplesPerSecond = 15181.0
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.38559585 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0198s; samplesPerSecond = 12646.7
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.34249602 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0181s; samplesPerSecond = 13785.5
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.28670760 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0196s; samplesPerSecond = 12736.9
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.26990444 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0135s; samplesPerSecond = 18543.2
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.23285544 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0214s; samplesPerSecond = 11680.6
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.25464216 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0172s; samplesPerSecond = 14495.3
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.21254012 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0223s; samplesPerSecond = 11189.2
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.18708229 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0159s; samplesPerSecond = 15742.1
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.21363045 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0165s; samplesPerSecond = 15157.0
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.23505446 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0167s; samplesPerSecond = 14954.8
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.20180383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0184s; samplesPerSecond = 13618.0
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.19780595 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0182s; samplesPerSecond = 13711.4
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.16131116 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0152s; samplesPerSecond = 16448.5
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.16479157 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0197s; samplesPerSecond = 12683.9
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20226367 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0188s; samplesPerSecond = 13319.8
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.14809084 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0182s; samplesPerSecond = 13741.5
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.19001815 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0220s; samplesPerSecond = 11382.8
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19616891 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0151s; samplesPerSecond = 16590.4
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.17887471 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0181s; samplesPerSecond = 13819.8
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.14040413 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0179s; samplesPerSecond = 13946.2
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17935154 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0199s; samplesPerSecond = 12553.4
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.13249074 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0190s; samplesPerSecond = 13169.0
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.15483359 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0169s; samplesPerSecond = 14803.4
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19796158 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0184s; samplesPerSecond = 13552.3
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.13179463 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0188s; samplesPerSecond = 13300.0
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.14028323 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0187s; samplesPerSecond = 13354.7
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12849508 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0192s; samplesPerSecond = 13022.9
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16702669 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0166s; samplesPerSecond = 15083.9
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20390303 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0189s; samplesPerSecond = 13212.1
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14594790 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0192s; samplesPerSecond = 13053.5
MPI Rank 2: 12/15/2016 08:28:19: Finished Epoch[ 2 of 4]: [Training] CrossEntropyWithSoftmax = 0.29447327 * 10000; EvalClassificationError = 0.11490000 * 10000; totalSamplesSeen = 20000; learningRatePerSample = 0.0080000004; epochTime=0.788532s
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:19: Starting Epoch 3: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 2, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12813297 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0168s; samplesPerSecond = 14853.5
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.17615627 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0204s; samplesPerSecond = 12261.5
MPI Rank 2: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14587002 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0186s; samplesPerSecond = 13445.9
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15938467 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0178s; samplesPerSecond = 14082.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.17100048 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0169s; samplesPerSecond = 14752.7
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18281055 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0194s; samplesPerSecond = 12881.9
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14781538 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0211s; samplesPerSecond = 11874.2
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18045490 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0177s; samplesPerSecond = 14145.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15847198 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0209s; samplesPerSecond = 11958.9
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14513057 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0195s; samplesPerSecond = 12833.0
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13519579 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0200s; samplesPerSecond = 12506.3
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13723645 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0207s; samplesPerSecond = 12058.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11692067 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0191s; samplesPerSecond = 13120.6
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16729042 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0194s; samplesPerSecond = 12917.2
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12836481 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0143s; samplesPerSecond = 17511.9
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17320383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0176s; samplesPerSecond = 14178.0
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17634558 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0209s; samplesPerSecond = 11982.9
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14124514 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0208s; samplesPerSecond = 12008.3
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19167717 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0213s; samplesPerSecond = 11731.6
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20913003 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0195s; samplesPerSecond = 12798.9
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18460750 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0214s; samplesPerSecond = 11670.8
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18188216 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0212s; samplesPerSecond = 11802.5
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14069101 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0145s; samplesPerSecond = 17266.4
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14812247 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0215s; samplesPerSecond = 11605.2
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20274092 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0196s; samplesPerSecond = 12757.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12887866 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0181s; samplesPerSecond = 13795.4
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18595255 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0184s; samplesPerSecond = 13616.6
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19565326 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0208s; samplesPerSecond = 12027.9
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16678525 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0148s; samplesPerSecond = 16846.4
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12552459 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0129s; samplesPerSecond = 19437.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17414176 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0224s; samplesPerSecond = 11140.8
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12295855 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0209s; samplesPerSecond = 11980.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14757012 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0200s; samplesPerSecond = 12506.3
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19785856 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0186s; samplesPerSecond = 13437.2
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12600285 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0210s; samplesPerSecond = 11894.6
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13742899 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0179s; samplesPerSecond = 13983.7
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12847649 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0216s; samplesPerSecond = 11599.3
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16652415 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0191s; samplesPerSecond = 13073.3
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20675722 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0236s; samplesPerSecond = 10611.7
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14562268 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0213s; samplesPerSecond = 11723.3
MPI Rank 2: 12/15/2016 08:28:20: Finished Epoch[ 3 of 4]: [Training] CrossEntropyWithSoftmax = 0.15965044 * 10000; EvalClassificationError = 0.07650000 * 10000; totalSamplesSeen = 30000; learningRatePerSample = 0.0080000004; epochTime=0.803123s
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:20: Starting Epoch 4: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:20: Starting minibatch loop, DataParallelSGD training (myRank = 2, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12392293 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0229s; samplesPerSecond = 10938.0
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.18033423 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0183s; samplesPerSecond = 13644.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14283999 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0235s; samplesPerSecond = 10655.1
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15662489 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0207s; samplesPerSecond = 12075.5
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.16985800 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0220s; samplesPerSecond = 11363.6
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18190608 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0228s; samplesPerSecond = 10970.7
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14495469 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0227s; samplesPerSecond = 11019.5
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18022153 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0234s; samplesPerSecond = 10672.4
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15852460 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0228s; samplesPerSecond = 10957.7
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14466589 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0208s; samplesPerSecond = 12044.7
MPI Rank 2: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13346404 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0220s; samplesPerSecond = 11365.7
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13683061 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0218s; samplesPerSecond = 11456.3
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11589011 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0239s; samplesPerSecond = 10445.0
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16881193 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0217s; samplesPerSecond = 11534.6
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12736965 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0244s; samplesPerSecond = 10253.5
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17123604 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0243s; samplesPerSecond = 10272.8
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17706403 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0231s; samplesPerSecond = 10808.9
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14104103 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0151s; samplesPerSecond = 16536.6
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19313360 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0151s; samplesPerSecond = 16554.1
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20870745 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0155s; samplesPerSecond = 16163.4
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18510294 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0178s; samplesPerSecond = 14055.2
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18167136 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0159s; samplesPerSecond = 15748.0
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14026275 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0156s; samplesPerSecond = 16033.9
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14811532 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0170s; samplesPerSecond = 14713.7
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20368129 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0169s; samplesPerSecond = 14835.0
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12819271 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0154s; samplesPerSecond = 16213.8
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18632901 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0162s; samplesPerSecond = 15433.1
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19568751 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0189s; samplesPerSecond = 13257.7
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16449544 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0152s; samplesPerSecond = 16477.7
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12454886 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0183s; samplesPerSecond = 13693.4
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17307192 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0152s; samplesPerSecond = 16407.4
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12249522 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0173s; samplesPerSecond = 14449.2
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14709682 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0132s; samplesPerSecond = 18933.7
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19789048 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0176s; samplesPerSecond = 14223.9
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12572171 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0167s; samplesPerSecond = 14970.1
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13732392 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0190s; samplesPerSecond = 13173.1
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12857569 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0148s; samplesPerSecond = 16903.3
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16653116 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0184s; samplesPerSecond = 13583.3
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20715346 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0194s; samplesPerSecond = 12897.9
MPI Rank 2: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14571729 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0193s; samplesPerSecond = 12970.2
MPI Rank 2: 12/15/2016 08:28:21: Finished Epoch[ 4 of 4]: [Training] CrossEntropyWithSoftmax = 0.15917665 * 10000; EvalClassificationError = 0.07660000 * 10000; totalSamplesSeen = 40000; learningRatePerSample = 0.0080000004; epochTime=0.796897s
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:21: Action "train" complete.
MPI Rank 2: 
MPI Rank 2: 12/15/2016 08:28:21: __COMPLETED__
MPI Rank 3: 12/15/2016 08:28:18: Redirecting stderr to file C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr_SimpleMultiGPU.logrank3
MPI Rank 3: CNTK 2.0.beta6.0+ (HEAD 5f1fab, Dec 15 2016 06:29:34) on cntk-muc03 at 2016/12/15 08:28:16
MPI Rank 3: 
MPI Rank 3: C:\jenkins\workspace\CNTK-Test-Windows-W1\x64\release\cntk.exe  configFile=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining/SimpleMultiGPU.cntk  currentDirectory=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  RunDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DataDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining\Data  ConfigDir=C:\jenkins\workspace\CNTK-Test-Windows-W1\Tests\EndToEndTests\ParallelTraining  OutputDir=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu  DeviceId=-1  timestamping=true  numCPUThreads=1  precision=float  SimpleMultiGPU=[SGD=[ParallelTrain=[DataParallelSGD=[gradientBits=32]]]]  stderr=C:\Users\svcphil\AppData\Local\Temp\cntk-test-20161215082748.614918\ParallelTraining\NoQuantization_SinglePrecision@release_cpu/stderr
MPI Rank 3: 12/15/2016 08:28:18: Using 1 CPU threads.
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: ##############################################################################
MPI Rank 3: 12/15/2016 08:28:18: #                                                                            #
MPI Rank 3: 12/15/2016 08:28:18: # SimpleMultiGPU command (train action)                                      #
MPI Rank 3: 12/15/2016 08:28:18: #                                                                            #
MPI Rank 3: 12/15/2016 08:28:18: ##############################################################################
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: 
MPI Rank 3: Creating virgin network.
MPI Rank 3: SimpleNetworkBuilder Using CPU
MPI Rank 3: 12/15/2016 08:28:18: 
MPI Rank 3: Model has 25 nodes. Using CPU.
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: Training criterion:   CrossEntropyWithSoftmax = CrossEntropyWithSoftmax
MPI Rank 3: 12/15/2016 08:28:18: Evaluation criterion: EvalClassificationError = ClassificationError
MPI Rank 3: 
MPI Rank 3: 
MPI Rank 3: Allocating matrices for forward and/or backward propagation.
MPI Rank 3: 
MPI Rank 3: Memory Sharing: Out of 40 matrices, 19 are shared as 8, and 21 are not shared.
MPI Rank 3: 
MPI Rank 3: 	{ HLast : [2 x 1 x *]
MPI Rank 3: 	  W2 : [2 x 50] (gradient) }
MPI Rank 3: 	{ W1 : [50 x 50] (gradient)
MPI Rank 3: 	  W1*H1+B1 : [50 x 1 x *] }
MPI Rank 3: 	{ W0 : [50 x 2] (gradient)
MPI Rank 3: 	  W0*features+B0 : [50 x 1 x *] }
MPI Rank 3: 	{ B1 : [50 x 1] (gradient)
MPI Rank 3: 	  H2 : [50 x 1 x *] (gradient)
MPI Rank 3: 	  HLast : [2 x 1 x *] (gradient) }
MPI Rank 3: 	{ B0 : [50 x 1] (gradient)
MPI Rank 3: 	  H1 : [50 x 1 x *] (gradient)
MPI Rank 3: 	  W1*H1+B1 : [50 x 1 x *] (gradient)
MPI Rank 3: 	  W2*H1 : [2 x 1 x *] }
MPI Rank 3: 	{ H1 : [50 x 1 x *]
MPI Rank 3: 	  W0*features : [50 x *] (gradient) }
MPI Rank 3: 	{ H2 : [50 x 1 x *]
MPI Rank 3: 	  W1*H1 : [50 x 1 x *] (gradient) }
MPI Rank 3: 	{ W0*features+B0 : [50 x 1 x *] (gradient)
MPI Rank 3: 	  W1*H1 : [50 x 1 x *] }
MPI Rank 3: 
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: Training 2802 parameters in 6 out of 6 parameter tensors and 15 nodes with gradient:
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: 	Node 'B0' (LearnableParameter operation) : [50 x 1]
MPI Rank 3: 12/15/2016 08:28:18: 	Node 'B1' (LearnableParameter operation) : [50 x 1]
MPI Rank 3: 12/15/2016 08:28:18: 	Node 'B2' (LearnableParameter operation) : [2 x 1]
MPI Rank 3: 12/15/2016 08:28:18: 	Node 'W0' (LearnableParameter operation) : [50 x 2]
MPI Rank 3: 12/15/2016 08:28:18: 	Node 'W1' (LearnableParameter operation) : [50 x 50]
MPI Rank 3: 12/15/2016 08:28:18: 	Node 'W2' (LearnableParameter operation) : [2 x 50]
MPI Rank 3: 
MPI Rank 3: Initializing dataParallelSGD with FP32 aggregation.
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: Precomputing --> 3 PreCompute nodes found.
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: 	MeanOfFeatures = Mean()
MPI Rank 3: 12/15/2016 08:28:18: 	InvStdOfFeatures = InvStdDev()
MPI Rank 3: 12/15/2016 08:28:18: 	Prior = Mean()
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: Precomputing --> Completed.
MPI Rank 3: 
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: Starting Epoch 1: learning rate per sample = 0.020000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:18: Starting minibatch loop, DataParallelSGD training (myRank = 3, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[   1-  10]: CrossEntropyWithSoftmax = 0.69973267 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0203s; samplesPerSecond = 12305.0
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  11-  20]: CrossEntropyWithSoftmax = 0.71436906 * 250; EvalClassificationError = 0.52000000 * 250; time = 0.0209s; samplesPerSecond = 11934.3
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  21-  30]: CrossEntropyWithSoftmax = 0.72871054 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0194s; samplesPerSecond = 12885.9
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  31-  40]: CrossEntropyWithSoftmax = 0.70038993 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0199s; samplesPerSecond = 12572.3
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  41-  50]: CrossEntropyWithSoftmax = 0.70593820 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0214s; samplesPerSecond = 11656.6
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  51-  60]: CrossEntropyWithSoftmax = 0.71604646 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0211s; samplesPerSecond = 11832.6
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  61-  70]: CrossEntropyWithSoftmax = 0.72247950 * 250; EvalClassificationError = 0.48000000 * 250; time = 0.0214s; samplesPerSecond = 11695.9
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  71-  80]: CrossEntropyWithSoftmax = 0.79884413 * 250; EvalClassificationError = 0.47600000 * 250; time = 0.0195s; samplesPerSecond = 12842.2
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  81-  90]: CrossEntropyWithSoftmax = 0.69622447 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0195s; samplesPerSecond = 12812.6
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[  91- 100]: CrossEntropyWithSoftmax = 0.70749459 * 250; EvalClassificationError = 0.49200000 * 250; time = 0.0197s; samplesPerSecond = 12676.2
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 101- 110]: CrossEntropyWithSoftmax = 0.71485825 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0220s; samplesPerSecond = 11338.9
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 111- 120]: CrossEntropyWithSoftmax = 0.69579152 * 250; EvalClassificationError = 0.43600000 * 250; time = 0.0173s; samplesPerSecond = 14465.9
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 121- 130]: CrossEntropyWithSoftmax = 0.70174138 * 250; EvalClassificationError = 0.44000000 * 250; time = 0.0239s; samplesPerSecond = 10443.6
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 131- 140]: CrossEntropyWithSoftmax = 0.71926585 * 250; EvalClassificationError = 0.54800000 * 250; time = 0.0199s; samplesPerSecond = 12555.9
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 141- 150]: CrossEntropyWithSoftmax = 0.72009918 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0208s; samplesPerSecond = 12019.2
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 151- 160]: CrossEntropyWithSoftmax = 0.71854574 * 250; EvalClassificationError = 0.55200000 * 250; time = 0.0218s; samplesPerSecond = 11477.4
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 161- 170]: CrossEntropyWithSoftmax = 0.74083729 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0223s; samplesPerSecond = 11192.2
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 171- 180]: CrossEntropyWithSoftmax = 0.71762852 * 250; EvalClassificationError = 0.51600000 * 250; time = 0.0220s; samplesPerSecond = 11355.4
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 181- 190]: CrossEntropyWithSoftmax = 0.71530686 * 250; EvalClassificationError = 0.48400000 * 250; time = 0.0228s; samplesPerSecond = 10981.3
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 191- 200]: CrossEntropyWithSoftmax = 0.71768617 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0222s; samplesPerSecond = 11264.8
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 201- 210]: CrossEntropyWithSoftmax = 0.71515312 * 250; EvalClassificationError = 0.53600000 * 250; time = 0.0220s; samplesPerSecond = 11380.7
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 211- 220]: CrossEntropyWithSoftmax = 0.72047061 * 250; EvalClassificationError = 0.52400000 * 250; time = 0.0223s; samplesPerSecond = 11201.7
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 221- 230]: CrossEntropyWithSoftmax = 0.72033072 * 250; EvalClassificationError = 0.50800000 * 250; time = 0.0159s; samplesPerSecond = 15741.1
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 231- 240]: CrossEntropyWithSoftmax = 0.71295325 * 250; EvalClassificationError = 0.51200000 * 250; time = 0.0216s; samplesPerSecond = 11555.4
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 241- 250]: CrossEntropyWithSoftmax = 0.69737817 * 250; EvalClassificationError = 0.53200000 * 250; time = 0.0226s; samplesPerSecond = 11055.6
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 251- 260]: CrossEntropyWithSoftmax = 0.70251892 * 250; EvalClassificationError = 0.48800000 * 250; time = 0.0226s; samplesPerSecond = 11083.0
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 261- 270]: CrossEntropyWithSoftmax = 0.70879703 * 250; EvalClassificationError = 0.54400000 * 250; time = 0.0196s; samplesPerSecond = 12729.8
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 271- 280]: CrossEntropyWithSoftmax = 0.69856459 * 250; EvalClassificationError = 0.52800000 * 250; time = 0.0226s; samplesPerSecond = 11052.7
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 281- 290]: CrossEntropyWithSoftmax = 0.69425908 * 250; EvalClassificationError = 0.44800000 * 250; time = 0.0229s; samplesPerSecond = 10896.6
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 291- 300]: CrossEntropyWithSoftmax = 0.69599736 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0231s; samplesPerSecond = 10836.6
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 301- 310]: CrossEntropyWithSoftmax = 0.69591177 * 250; EvalClassificationError = 0.54000000 * 250; time = 0.0197s; samplesPerSecond = 12665.9
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 311- 320]: CrossEntropyWithSoftmax = 0.69133098 * 250; EvalClassificationError = 0.40000000 * 250; time = 0.0213s; samplesPerSecond = 11714.5
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 321- 330]: CrossEntropyWithSoftmax = 0.69822649 * 250; EvalClassificationError = 0.46800000 * 250; time = 0.0418s; samplesPerSecond = 5974.7
MPI Rank 3: 12/15/2016 08:28:18:  Epoch[ 1 of 4]-Minibatch[ 331- 340]: CrossEntropyWithSoftmax = 0.71031540 * 250; EvalClassificationError = 0.50400000 * 250; time = 0.0247s; samplesPerSecond = 10103.9
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 341- 350]: CrossEntropyWithSoftmax = 0.70097460 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0218s; samplesPerSecond = 11464.7
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 351- 360]: CrossEntropyWithSoftmax = 0.68927869 * 250; EvalClassificationError = 0.45200000 * 250; time = 0.0208s; samplesPerSecond = 11995.0
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 361- 370]: CrossEntropyWithSoftmax = 0.68908391 * 250; EvalClassificationError = 0.50000000 * 250; time = 0.0231s; samplesPerSecond = 10800.1
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 371- 380]: CrossEntropyWithSoftmax = 0.67796903 * 250; EvalClassificationError = 0.45600000 * 250; time = 0.0237s; samplesPerSecond = 10570.4
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 381- 390]: CrossEntropyWithSoftmax = 0.67863597 * 250; EvalClassificationError = 0.38400000 * 250; time = 0.0224s; samplesPerSecond = 11144.8
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 1 of 4]-Minibatch[ 391- 400]: CrossEntropyWithSoftmax = 0.67150942 * 250; EvalClassificationError = 0.42800000 * 250; time = 0.0221s; samplesPerSecond = 11307.1
MPI Rank 3: 12/15/2016 08:28:19: Finished Epoch[ 1 of 4]: [Training] CrossEntropyWithSoftmax = 0.70804124 * 10000; EvalClassificationError = 0.49380000 * 10000; totalSamplesSeen = 10000; learningRatePerSample = 0.02; epochTime=0.910335s
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:19: Starting Epoch 2: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 3, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.69566501 * 250; EvalClassificationError = 0.49600000 * 250; time = 0.0207s; samplesPerSecond = 12094.8
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.64058132 * 250; EvalClassificationError = 0.22400000 * 250; time = 0.0222s; samplesPerSecond = 11250.1
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.62577220 * 250; EvalClassificationError = 0.30400000 * 250; time = 0.0242s; samplesPerSecond = 10325.9
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.62974806 * 250; EvalClassificationError = 0.34000000 * 250; time = 0.0249s; samplesPerSecond = 10035.7
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.60705925 * 250; EvalClassificationError = 0.22800000 * 250; time = 0.0251s; samplesPerSecond = 9971.3
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.59038701 * 250; EvalClassificationError = 0.18000000 * 250; time = 0.0238s; samplesPerSecond = 10499.8
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.55033237 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0185s; samplesPerSecond = 13539.1
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.53624221 * 250; EvalClassificationError = 0.23200000 * 250; time = 0.0171s; samplesPerSecond = 14609.6
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.48688371 * 250; EvalClassificationError = 0.12000000 * 250; time = 0.0142s; samplesPerSecond = 17595.7
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.43212992 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0165s; samplesPerSecond = 15196.6
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.38559585 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0197s; samplesPerSecond = 12667.8
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.34249602 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0183s; samplesPerSecond = 13685.1
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.28670760 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0188s; samplesPerSecond = 13328.4
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.26990444 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0138s; samplesPerSecond = 18061.0
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.23285544 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0216s; samplesPerSecond = 11570.3
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.25464216 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0172s; samplesPerSecond = 14527.3
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.21254012 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0223s; samplesPerSecond = 11188.7
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.18708229 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0159s; samplesPerSecond = 15750.0
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.21363045 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0165s; samplesPerSecond = 15169.0
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.23505446 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0171s; samplesPerSecond = 14585.8
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.20180383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0185s; samplesPerSecond = 13477.1
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.19780595 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0185s; samplesPerSecond = 13511.3
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.16131116 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0157s; samplesPerSecond = 15963.2
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.16479157 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0201s; samplesPerSecond = 12424.8
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20226367 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0188s; samplesPerSecond = 13330.5
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.14809084 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0184s; samplesPerSecond = 13613.6
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.19001815 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0218s; samplesPerSecond = 11452.7
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19616891 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0150s; samplesPerSecond = 16632.3
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.17887471 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0185s; samplesPerSecond = 13498.9
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.14040413 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0169s; samplesPerSecond = 14836.8
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17935154 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0210s; samplesPerSecond = 11916.1
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.13249074 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0190s; samplesPerSecond = 13166.2
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.15483359 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0154s; samplesPerSecond = 16185.4
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19796158 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0199s; samplesPerSecond = 12579.2
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.13179463 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0188s; samplesPerSecond = 13283.0
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.14028323 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0188s; samplesPerSecond = 13268.9
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12849508 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0191s; samplesPerSecond = 13081.5
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16702669 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0165s; samplesPerSecond = 15118.5
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20390303 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0189s; samplesPerSecond = 13216.3
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 2 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14594790 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0191s; samplesPerSecond = 13067.1
MPI Rank 3: 12/15/2016 08:28:19: Finished Epoch[ 2 of 4]: [Training] CrossEntropyWithSoftmax = 0.29447327 * 10000; EvalClassificationError = 0.11490000 * 10000; totalSamplesSeen = 20000; learningRatePerSample = 0.0080000004; epochTime=0.788534s
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:19: Starting Epoch 3: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:19: Starting minibatch loop, DataParallelSGD training (myRank = 3, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12813297 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0172s; samplesPerSecond = 14567.9
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.17615627 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0204s; samplesPerSecond = 12257.3
MPI Rank 3: 12/15/2016 08:28:19:  Epoch[ 3 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14587002 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0186s; samplesPerSecond = 13461.8
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15938467 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0177s; samplesPerSecond = 14087.7
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.17100048 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0174s; samplesPerSecond = 14402.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18281055 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0194s; samplesPerSecond = 12886.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14781538 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0214s; samplesPerSecond = 11695.9
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18045490 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0178s; samplesPerSecond = 14078.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15847198 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0209s; samplesPerSecond = 11950.9
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14513057 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0195s; samplesPerSecond = 12835.0
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13519579 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0190s; samplesPerSecond = 13183.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13723645 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0217s; samplesPerSecond = 11494.8
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11692067 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0187s; samplesPerSecond = 13337.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16729042 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0197s; samplesPerSecond = 12707.1
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12836481 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0143s; samplesPerSecond = 17504.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17320383 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0176s; samplesPerSecond = 14180.4
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17634558 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0210s; samplesPerSecond = 11898.0
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14124514 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0211s; samplesPerSecond = 11855.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19167717 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0213s; samplesPerSecond = 11745.9
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20913003 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0198s; samplesPerSecond = 12657.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18460750 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0214s; samplesPerSecond = 11662.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18188216 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0214s; samplesPerSecond = 11675.7
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14069101 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0145s; samplesPerSecond = 17291.5
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14812247 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0215s; samplesPerSecond = 11612.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20274092 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0200s; samplesPerSecond = 12485.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12887866 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0181s; samplesPerSecond = 13809.9
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18595255 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0184s; samplesPerSecond = 13618.0
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19565326 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0206s; samplesPerSecond = 12107.1
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16678525 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0149s; samplesPerSecond = 16828.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12552459 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0129s; samplesPerSecond = 19452.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17414176 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0229s; samplesPerSecond = 10925.1
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12295855 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0209s; samplesPerSecond = 11986.4
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14757012 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0182s; samplesPerSecond = 13750.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19785856 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0208s; samplesPerSecond = 12000.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12600285 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0210s; samplesPerSecond = 11901.9
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13742899 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0183s; samplesPerSecond = 13668.7
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12847649 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0220s; samplesPerSecond = 11378.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16652415 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0191s; samplesPerSecond = 13082.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20675722 * 250; EvalClassificationError = 0.11200000 * 250; time = 0.0240s; samplesPerSecond = 10428.8
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 3 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14562268 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0213s; samplesPerSecond = 11715.6
MPI Rank 3: 12/15/2016 08:28:20: Finished Epoch[ 3 of 4]: [Training] CrossEntropyWithSoftmax = 0.15965044 * 10000; EvalClassificationError = 0.07650000 * 10000; totalSamplesSeen = 30000; learningRatePerSample = 0.0080000004; epochTime=0.803123s
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:20: Starting Epoch 4: learning rate per sample = 0.008000  effective momentum = 0.900000  momentum as time constant = 237.3 samples
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:20: Starting minibatch loop, DataParallelSGD training (myRank = 3, numNodes = 4, numGradientBits = 32), distributed reading is ENABLED.
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[   1-  10, 2.50%]: CrossEntropyWithSoftmax = 0.12392293 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0227s; samplesPerSecond = 11004.0
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  11-  20, 5.00%]: CrossEntropyWithSoftmax = 0.18033423 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0183s; samplesPerSecond = 13652.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  21-  30, 7.50%]: CrossEntropyWithSoftmax = 0.14283999 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0239s; samplesPerSecond = 10471.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  31-  40, 10.00%]: CrossEntropyWithSoftmax = 0.15662489 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0207s; samplesPerSecond = 12070.3
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  41-  50, 12.50%]: CrossEntropyWithSoftmax = 0.16985800 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0222s; samplesPerSecond = 11251.1
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  51-  60, 15.00%]: CrossEntropyWithSoftmax = 0.18190608 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0232s; samplesPerSecond = 10756.9
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  61-  70, 17.50%]: CrossEntropyWithSoftmax = 0.14495469 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0230s; samplesPerSecond = 10851.6
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  71-  80, 20.00%]: CrossEntropyWithSoftmax = 0.18022153 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0234s; samplesPerSecond = 10684.2
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  81-  90, 22.50%]: CrossEntropyWithSoftmax = 0.15852460 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0228s; samplesPerSecond = 10962.0
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[  91- 100, 25.00%]: CrossEntropyWithSoftmax = 0.14466589 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0208s; samplesPerSecond = 12027.3
MPI Rank 3: 12/15/2016 08:28:20:  Epoch[ 4 of 4]-Minibatch[ 101- 110, 27.50%]: CrossEntropyWithSoftmax = 0.13346404 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0220s; samplesPerSecond = 11367.8
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 111- 120, 30.00%]: CrossEntropyWithSoftmax = 0.13683061 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0218s; samplesPerSecond = 11461.1
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 121- 130, 32.50%]: CrossEntropyWithSoftmax = 0.11589011 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0239s; samplesPerSecond = 10441.9
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 131- 140, 35.00%]: CrossEntropyWithSoftmax = 0.16881193 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0216s; samplesPerSecond = 11553.2
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 141- 150, 37.50%]: CrossEntropyWithSoftmax = 0.12736965 * 250; EvalClassificationError = 0.04800000 * 250; time = 0.0244s; samplesPerSecond = 10253.9
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 151- 160, 40.00%]: CrossEntropyWithSoftmax = 0.17123604 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0244s; samplesPerSecond = 10244.2
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 161- 170, 42.50%]: CrossEntropyWithSoftmax = 0.17706403 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0231s; samplesPerSecond = 10837.5
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 171- 180, 45.00%]: CrossEntropyWithSoftmax = 0.14104103 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0151s; samplesPerSecond = 16533.3
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 181- 190, 47.50%]: CrossEntropyWithSoftmax = 0.19313360 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0156s; samplesPerSecond = 16059.6
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 191- 200, 50.00%]: CrossEntropyWithSoftmax = 0.20870745 * 250; EvalClassificationError = 0.10000000 * 250; time = 0.0155s; samplesPerSecond = 16165.5
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 201- 210, 52.50%]: CrossEntropyWithSoftmax = 0.18510294 * 250; EvalClassificationError = 0.08000000 * 250; time = 0.0184s; samplesPerSecond = 13618.0
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 211- 220, 55.00%]: CrossEntropyWithSoftmax = 0.18167136 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0158s; samplesPerSecond = 15814.8
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 221- 230, 57.50%]: CrossEntropyWithSoftmax = 0.14026275 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0156s; samplesPerSecond = 16025.6
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 231- 240, 60.00%]: CrossEntropyWithSoftmax = 0.14811532 * 250; EvalClassificationError = 0.07600000 * 250; time = 0.0174s; samplesPerSecond = 14359.6
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 241- 250, 62.50%]: CrossEntropyWithSoftmax = 0.20368129 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0166s; samplesPerSecond = 15040.3
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 251- 260, 65.00%]: CrossEntropyWithSoftmax = 0.12819271 * 250; EvalClassificationError = 0.07200000 * 250; time = 0.0156s; samplesPerSecond = 15983.6
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 261- 270, 67.50%]: CrossEntropyWithSoftmax = 0.18632901 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0162s; samplesPerSecond = 15443.5
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 271- 280, 70.00%]: CrossEntropyWithSoftmax = 0.19568751 * 250; EvalClassificationError = 0.08800000 * 250; time = 0.0190s; samplesPerSecond = 13132.3
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 281- 290, 72.50%]: CrossEntropyWithSoftmax = 0.16449544 * 250; EvalClassificationError = 0.06800000 * 250; time = 0.0152s; samplesPerSecond = 16482.1
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 291- 300, 75.00%]: CrossEntropyWithSoftmax = 0.12454886 * 250; EvalClassificationError = 0.04400000 * 250; time = 0.0185s; samplesPerSecond = 13480.7
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 301- 310, 77.50%]: CrossEntropyWithSoftmax = 0.17307192 * 250; EvalClassificationError = 0.08400000 * 250; time = 0.0150s; samplesPerSecond = 16710.1
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 311- 320, 80.00%]: CrossEntropyWithSoftmax = 0.12249522 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0178s; samplesPerSecond = 14084.5
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 321- 330, 82.50%]: CrossEntropyWithSoftmax = 0.14709682 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0132s; samplesPerSecond = 18948.0
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 331- 340, 85.00%]: CrossEntropyWithSoftmax = 0.19789048 * 250; EvalClassificationError = 0.09200000 * 250; time = 0.0177s; samplesPerSecond = 14141.1
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 341- 350, 87.50%]: CrossEntropyWithSoftmax = 0.12572171 * 250; EvalClassificationError = 0.05200000 * 250; time = 0.0167s; samplesPerSecond = 14979.0
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 351- 360, 90.00%]: CrossEntropyWithSoftmax = 0.13732392 * 250; EvalClassificationError = 0.05600000 * 250; time = 0.0190s; samplesPerSecond = 13163.4
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 361- 370, 92.50%]: CrossEntropyWithSoftmax = 0.12857569 * 250; EvalClassificationError = 0.06000000 * 250; time = 0.0153s; samplesPerSecond = 16331.3
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 371- 380, 95.00%]: CrossEntropyWithSoftmax = 0.16653116 * 250; EvalClassificationError = 0.09600000 * 250; time = 0.0179s; samplesPerSecond = 13972.7
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 381- 390, 97.50%]: CrossEntropyWithSoftmax = 0.20715346 * 250; EvalClassificationError = 0.11600000 * 250; time = 0.0194s; samplesPerSecond = 12864.7
MPI Rank 3: 12/15/2016 08:28:21:  Epoch[ 4 of 4]-Minibatch[ 391- 400, 100.00%]: CrossEntropyWithSoftmax = 0.14571729 * 250; EvalClassificationError = 0.06400000 * 250; time = 0.0192s; samplesPerSecond = 13012.7
MPI Rank 3: 12/15/2016 08:28:21: Finished Epoch[ 4 of 4]: [Training] CrossEntropyWithSoftmax = 0.15917665 * 10000; EvalClassificationError = 0.07660000 * 10000; totalSamplesSeen = 40000; learningRatePerSample = 0.0080000004; epochTime=0.796895s
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:21: Action "train" complete.
MPI Rank 3: 
MPI Rank 3: 12/15/2016 08:28:21: __COMPLETED__
