Author：Shan, Yu ; Xie, Dejun ; Zhang, Rui

Abstract:The study proposes a multi-objective optimization model for bike-sharing industry by monitoring, with high accuracy, the user demand and providing the suitable number of bikes at selected stations. One of the key factors for designing an optimized bike sharing system is to balance the demand of pick-ups (drop-offs) around a given station and the number of available bikes (vacant lockers) in the station throughout the day. The model optimizes the location of bicycle stations and the number of parking slots that each station should have by taking account of the main contributing factors including the total budget of the bike sharing system, the popularity of riding in the city, and the expected proximity of the stations. A case study using the bike-sharing in New York was conducted to test theeffectiveness of themodel. © 2019 Association for Computing Machinery.

Author：Shao, WT;Wu, XH

Abstract:It is well known that the numerical integration process is much less sensitive than numerical differential process when dealing with the differential equations. After integration, accuracy is no longer limited by that of the slowly convergent series for the highest derivative, but only by that of the unknown function itself. In this paper, a Chebyshev tau meshless method based on the highest derivative (CTMMHD) is developed for fourth order equations on irregularly shaped domains with complex boundary conditions. The problem domain is embedded in a domain of regular shape. The integration and multiplication of Chebyshev expansions are given in matrix representations. Several numerical experiments including standard biharmonic problems, problems with variable coefficients and nonlinear problems are implemented to verify the high accuracy and efficiency of our method. (C) 2012 Elsevier Inc. All rights reserved.

Author：Zhang, Shufei ; Qian, Zhuang ; Huang, Kaizhu ; Zhang, Rui ; Hussain, Amir

Abstract:Generative Adversarial Networks (GANs) are powerful generative models, but usually suffer from hard training and poor generation. Due to complex data and generation distributions in high dimensional space, it is difficult to measure the departure of two distributions, which is however vital for training successful GANs. Previous methods try to alleviate this problem by choosing reasonable divergence metrics. Unlike previous methods, in this paper, we propose a novel method called SimpleGAN to tackle this problem transform original complex distributions to simple ones in the low dimensional space while keeping information and then measure the departure of two simple distributions. This novel method offers a new direction to tackle the stability of GANs. Specifically, starting from maximization of the mutual information between variables in the original high dimensional space and low dimensional space, we eventually derive to optimize a much simplified version, i.e. the lower bound of the mutual information. For experiments, we implement our proposed method on different baselines i.e. traditional GAN, WGAN-GP and DCGAN for CIFAR-10 dataset. Our proposed method achieves obvious improvement over these baseline models. © 2019 IEEE.

Author：Ahmed, R;Dashtipour, K;Gogate, M;Raza, A;Zhang, R;Huang, KZ;Hawalah, A;Adeel, A;Hussain, A

Abstract:In pattern recognition, automatic handwriting recognition (AHWR) is an area of research that has developed rapidly in the last few years. It can play a significant role in broad-spectrum of applications rending from, bank cheque processing, application forms processing, postal address processing, to text-to-speech conversion. However, most research efforts are devoted to English-language only. This work focuses on developing Offline Arabic Handwriting Recognition (OAHR). The OAHR is a very challenging task due to some unique characteristics of the Arabic script such as cursive nature, ligatures, overlapping, and diacritical marks. In the recent literature, several effective Deep Learning (DL) approaches have been proposed to develop efficient AHWR systems. In this paper, we commission a survey on emerging AHWR technologies with some insight on OAHR background, challenges, opportunities, and future research trends.

Author：Du, LL;Wu, XH;Kong, WB

Abstract:Sine method is widely used for solving the boundary value problems because of the ease with which it may handle the presence of singularities or unbounded domains. In this paper, the domain decomposition method is used to deal with problems with interior layers and problems posed on irregular domains. With boundary reduction technique, the functional values on internal points can be eliminated. The scheme is based on a direct approach based on the corresponding compatibility matching conditions between sub-domains. Numerical experiments show that rational Sine domain decomposition method based on the interpolation of the highest derivatives (RSIHD-DDM) is effective for treating problems with interior layers or boundary layers on irregular domains. (C) 2012 Elsevier Ltd. All rights reserved.

Author：Zhang, ZJ;Zhang, R;Wang, QF;Huang, KZ

Abstract:Image-to-image translation aims to change attributes or domains of images, where the feature disentanglement based method is widely used recently due to its feasibility and effectiveness. In this method, a feature extractor is usually integrated in the encoder-decoder architecture generative adversarial network (GAN), which extracts features from domains and images, respectively. However, the two types of features are not properly combined, resulting in blurry generated images and indistinguishable translated domains. To alleviate this issue, we propose a new feature fusion approach to leverage the ability of the feature disentanglement. Instead of adding the two extracted features directly, we design a joint block fusion that contains integration, concatenation, and squeeze operations, thus allowing the generator to take full advantage of the two features and generate more photo-realistic images. We evaluate both the classification accuracy and Frechet Inception Distance (FID) of the proposed method on two benchmark datasets of Alps Seasons and CelebA. Extensive experimental results demonstrate that the proposed joint block fusion can improve both the discriminability of domains and the quality of translated image. Specially, the classification accuracies are improved by 1.04%% (FID reduced by 1.22) and 1.87%% (FID reduced by 4.96) on Alps Seasons and CelebA, respectively.

Author：Shao, WT;Wu, XH

Abstract:In this paper, we present a method, which combines the Chebyshev tau meshless method based on the integration-differentiation (CTMMID) with Domain Decomposition Method (DDM), and apply it to solve the fourth order problem on irregular domains. This method, i.e. CTMMID-DDM, is an improvement of our previous job. In early work, it shows that the CTMMID can solve the fourth order problems well in square domain, and leads to the condition number which grows like O(N-4), but it becomes worse when we apply it on irregular domains directly. DDMs extend the applicability of spectral methods to handle complex geometries and large-scale problems. Numerical results show that CTMMID-DDM works well, it circumvents the ill-conditioning problem, further attains a improvement in solution accuracy, and also makes us feasible to solve the problems with boundary layers. (C) 2014 Elsevier Inc. All rights reserved.

Author：Ren, J;Xu, M;Smith, JS;Zhao, HM;Zhang, R

Abstract:To increase the robustness of detection in intelligent video surveillance systems, homography has been widely used to fuse foreground regions projected from multiple camera views to a reference view. However, the intersections of non-corresponding foreground regions can cause phantoms. This paper proposes an algorithm based on geometry and colour cues to cope with this problem, in which the homography between different camera views and the Mahalanobis distance between the colour distributions of every two associated foreground regions are considered. The integration of these two matching algorithms improves the robustness of the pedestrian and phantom classification. Experiments on real-world video sequences have shown the robustness of this algorithm.

Author：Huang, KZ;Zhang, R;Jin, XB;Hussain, A

Author：Shao, WT;Wu, XH;Chen, SQ

Abstract:This paper reports a new method, Chebyshev tau meshless method based on the integration-differentiation (CTMMID) for numerically solving Biharmonic-type equations on irregularly shaped domains with complex boundary conditions. In general, the direct application of Chebyshev spectral method based on differentiation process to the fourth order equations leads to the corresponding differentiation matrix with large condition numbers. From another aspect, the strategy based on the integral formula of a Chebyshev polynomial could not only create sparsity, but also improve the accuracy, however it requires a lot of computational cost for directly solving high order two-dimensional problems. In this paper, the construction of the Chebyshev approximations is to start from the mix partial derivative u(xxyy)(x,y) rather than the unknown function u(x,y). The irregular domain is embedded in a domain of rectangle shape and the curve boundary can be efficiently treated by CTMMID. The numerical results show that compared with the existing results, our method yields spectral accuracy, and the main distinguishing feature is reducing the condition number of fourth order equations on rectangle domain from O(N-8) to O(N-4). It also appears that CTMMID is effective for the problems on irregular domains. (C) 2012 Elsevier Ltd. All rights reserved.

Author：Huang, KZ;Zhang, SF;Zhang, R;Hussain, A

Abstract:Field classification is a new extension of traditional classification frameworks that attempts to utilize consistent information from a group of samples (termed fields). By forgoing the independent identically distributed (i.i.d.) assumption, field classification can achieve remarkably improved accuracy compared to traditional classification methods. Most studies of field classification have been conducted on traditional machine learning methods. In this paper, we propose integration with a Bayesian framework, for the first time, in order to extend field classification to deep learning and propose two novel deep neural network architectures: the Field Deep Perceptron (FDP) and the Field Deep Convolutional Neural Network (FDCNN). Specifically, we exploit a deep perceptron structure, typically a 6-layer structure, where the first 3 layers remove (learn) a 'style' from a group of samples to map them into a more discriminative space and the last 3 layers are trained to perform classification. For the FDCNN, we modify the AlexNet framework by adding style transformation layers within the hidden layers. We derive a novel learning scheme from a Bayesian framework and design a novel and efficient learning algorithm with guaranteed convergence for training the deep networks. The whole framework is interpreted with visualization features showing that the field deep neural network can better learn the style of a group of samples. Our developed models are also able to achieve transfer learning and learn transformations for newly introduced fields. We conduct extensive comparative experiments on benchmark data (including face, speech, and handwriting data) to validate our learning approach. Experimental results demonstrate that our proposed deep frameworks achieve significant improvements over other state-of-the-art algorithms, attaining new benchmark performance. (C) 2020 Published by Elsevier Ltd.

Author：Jiang, HC;Huang, KZ;Zhang, R

Abstract:In regression tasks for static data, existing methods often assume that they were generated from an identical and independent distribution (i.i.d.). However, violation can be found when input samples may form groups, each affected by a certain different domain. In this case, style consistency exists within a same group, leading to degraded performance when conventional machine learning models were applied due to the violation of the i.i.d. assumption. In this paper, we propose one novel regression model named Field Support Vector Regression (F-SVR) without i.i.d. assumption. Specifically, we perform a style normalization transformation learning and the regression model learning simultaneously. An alternative optimization with final convergence guaranteed is designed, as well as a transductive learning algorithm, enabling extension on unseen styles during the testing phase. Experiments are conducted on two synthetic as well as two real benchmark data sets. Results show that the proposed F-SVR significantly outperforms many other state-of-the-art regression models in all the used data sets.

Author：Fang, YY;Zhang, R;Wang, QF;Huang, KZ

Abstract:Deep Convolutional Neural Networks (CNNs) have achieved great success in object recognition. However, they are difficult to capture the long-range temporal information, which plays an important role for action recognition in videos. To overcome this issue, a two-stream architecture including spatial and temporal segments based CNNs is widely used recently. However, the relationship among the segments is not sufficiently investigated. In this paper, we proposed to combine multiple segments by a fully connected layer in a deep CNN model for the whole action video. Moreover, the four streams (i.e., RGB, RGB differences, optical flow, and warped optical flow) are carefully integrated with a linear combination, and the weights are optimized on the validation datasets. We evaluate the recognition accuracy of the proposed method on two benchmark datasets of UCF101 and HMDB51. The extensive experimental results demonstrate encouraging results of our proposed method. Specifically, the proposed method improves the accuracy of action recognition in videos obviously (e.g., compared with the baseline, the accuracy is improved from 94.20%% to 97.30%% and from 69.40%% to 77.99%% on the dataset UCF101 and HMDB51, respectively). Furthermore, the proposed method can obtain the competitive accuracy to the state-of-the-art method of the 3D convolutional operation, but with much fewer parameters.

Author：Yang, X;Huang, KZ;Zhang, R

Abstract:In this paper, we propose a novel deep density model, called Deep Mixtures of Factor Analyzers with Common Loadings (DMCFA). Employing a mixture of factor analyzers sharing common component loadings, this novel model is more physically meaningful, since the common loadings can be regarded as feature selection or reduction matrices. Importantly, the novel DMCFA model is able to remarkably reduce the number of free parameters, making the involved inferences and learning problem dramatically easier. Despite its simplicity, by engaging learnable Gaussian distributions as the priors, DMCFA does not sacrifice its flexibility in estimating the data density. This is particularly the case when compared with the existing model Deep Mixtures of Factor Analyzers (DMFA), exploiting different loading matrices but simple standard Gaussian distributions for each component prior. We evaluate the performance of the proposed DMCFA in comparison with three other competitive models including Mixtures of Factor Analyzers (MFA), MCFA, and DMFA and their shallow counterparts. Results on four real data sets show that the novel model demonstrates significantly better performance in both density estimation and clustering.

Author：Shao, WT;Wu, XH;Wang, C

Abstract:It is known that spectral methods offer exponential convergence for infinitely smooth solutions. However, they are not applicable for problems presenting singularities or thin layers, especially true for the ones with the location of singularity unknown. An adaptive domain decomposition method (DDM) integrated with Chebyshev tau method based on the highest derivative (CTMHD) is introduced to solve singular perturbed boundary value problems (SPBVPs). The proposed adaptive algorithm uses the refinement indicators based on Chebyshev coefficients to determine which subintervals need to be refined. Numerical experiments have been conducted to demonstrate the superior performance of the method for SPBVPs with a number of singularities including boundary layers, interior layers and dense oscillations. A fourth order nonlinear SPBVP is also concerned. The numerical results illustrate the efficiency and applicability of our adaptive algorithm to capture the locations of singularities, and the higher accuracy in comparison with some existing numerical methods in the literature. (C) 2017 IMACS. Published by Elsevier B.V. All rights reserved.

Author：Yang, X;Huang, KZ;Goulermas, JY;Zhang, R

Abstract:Dimensionality reduction (DR) has been one central research topic in information theory, pattern recognition, and machine learning. Apparently, the performance of many learning models significantly rely on dimensionality reduction: successful DR can largely improve various approaches in clustering and classification, while inappropriate DR may deteriorate the systems. When applied on high-dimensional data, some existing research approaches often try to reduce the dimensionality first, and then input the reduced features to other available models, e.g., Gaussian mixture model (GMM). Such independent learning could however significantly limit the performance, since the optimal subspace given by a particular DR approach may not be appropriate for the following model. In this paper, we focus on investigating how unsupervised dimensionality reduction could be performed together with GMM and if such joint learning could lead to improvement in comparison with the traditional unsupervised method. In particular, we engage the mixture of factor analyzers with the assumption that a common factor loading exists for all the components. Based on that, we then present EM-algorithm that converges to a local optimal solution. Such setting exactly optimizes a dimensionality reduction together with the parameters of GMM. We describe the framework, detail the algorithm, and conduct a series of experiments to validate the effectiveness of our proposed approach. Specifically, we compare the proposed joint learning approach with two competitive algorithms on one synthetic and six real data sets. Experimental results show that the joint learning significantly outperforms the comparison methods in terms of three criteria.

Author：Qian, Z;Huang, KZ;Wang, QF;Xiao, JM;Zhang, R

Abstract:Generative Adversarial Networks (GAN) receive great attention recently due to its excellent performance in image generation, transformation, and super-resolution. However, less emphasis or study has been put on GAN for classification with super-resolution. Moreover, though GANs may fabricate images which perceptually looks realistic, they usually fabricate some fake details especially in character data; this would impose further difficulties when they are input for classification. In this paper, we propose a novel Generative Adversarial Classifier (GAC) for low-resolution handwriting character recognition. Specifically, we design an additional classifier component in GAC, leading to a novel three-player GAN model which is not only able to generate high-quality super-resolved images, but also favorable for classification. Experimental results show that our proposed method can obtain remarkable performance in handwriting characters with 8 x super-resolution, achieving new state-of-the-art on benchmark dataset CASIA-HWDB1.1, and MNIST. (C) 2020 Elsevier Ltd. All rights reserved.

Author：Shao, Wenting ; Wu, Xionghua

Abstract:We propose a new method for the numerical solution of a class of twodimensional parabolic problems. Our algorithm is based on the use of the Alternating Direction Implicit (ADI) approach in conjunction with the Chebyshev tau meshless method based on the highest derivative (CTMMHD). CTMMHD is applied to solve the set of one-dimensional problems resulting from operator-splitting at each time-stage. CTMMHD-ADI yields spectral accuracy in space and second order in time. Several numerical experiments are implemented to verify the efficiency of our method. © 2013 WIT Press.

Author：Zhang, SF;Huang, KZ;Zhang, R;Hussain, A

Abstract:Neural networks (NN) have achieved great successes in pattern recognition and machine learning. However, the success of a NN usually relies on the provision of a sufficiently large number of data samples as training data. When fed with a limited data set, a NN's performance may be degraded significantly. In this paper, a novel NN structure is proposed called a memory network. It is inspired by the cognitive mechanism of human beings, which can learn effectively, even from limited data. Taking advantage of the memory from previous samples, the new model achieves a remarkable improvement in performance when trained using limited data. The memory network is demonstrated here using the multi-layer perceptron (MLP) as a base model. However, it would be straightforward to extend the idea to other neural networks, e.g., convolutional neural networks (CNN). In this paper, the memory network structure is detailed, the training algorithm is presented, and a series of experiments are conducted to validate the proposed framework. Experimental results show that the proposed model outperforms traditional MLP-based models as well as other competitive algorithms in response to two real benchmark data sets.

Author：Yang, GY;Huang, KZ;Zhang, R;Goulermas, JY;Hussain, A

Abstract:Zero-shot learning (ZSL), i.e. classifying patterns where there is a lack of labeled training data, is a challenging yet important research topic. One of the most common ideas for ZSL is to map the data (e.g., images) and semantic attributes to the same embedding space. However, for coarse-grained classification tasks, the samples of each class tend to be unevenly distributed. This leads to the possibility of learned embedding function mapping the attributes to an inappropriate location, and hence limiting the classification performance. In this paper, we propose a novel regularized deep embedding model for ZSL in which a self-focus mechanism, is constructed to constrain the learning of the embedding function. During the training process, the distances of different dimensions in the embedding space will be focused conditioned on the class. Thereby, locations of the prototype mapped from the attributes can be adjusted according to the distribution of the samples for each class. Moreover, over-fitting of the embedding function to known classes will also be mitigated. A series of experiments on four commonly used zero-shot databases show that our proposed method can attain significant improvement in coarse-grained data sets.