Deep Convolutional Neural Networks (CNNs) have re- cently made a remarkable improvement in unconstrained face recognition problem. Researchers are racing in ways to boost the performance using advanced network architec- tures [25, 3, 27, 6, 32] or designing new loss functions to fa- cilitate discriminative feature learning [24, 33, 13, 31, 2, 42, 30]. Some of them even surpass human recognition ability on certain benchmark database [7].