Supplementary material: The following are GIF animations for the experiment of the K-spirals datasets, where the left panel shows the dataset and the decision boundaries learnt by our algorithm, and the right panel shows the latent space representation, i.e., F(X) and the one-versus-all SVMs' separating hyperplanes. - 2spiral.gif - 6spiral.gif: we vary the number of classes K and the latent dimensionality L (shown in the top), in order to investigate the role of L with respect to K. - 2spiral_numBFs.gif: we vary the number of radial basis functions (shown as # centers in the top) for the dimension reduction mapping F, and investigate the ideal nonlinear dimension reduction for classification. - 3spiral_initZ_*.gif: iterations of our algorithm from different initial Z (random, PCA of the inputs, and "ideal", i.e., the corners of a simplex). All these animations may be seen with a web browser or with specialized GIF image viewers.