Frank Z. Xing
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
The Cerebellar Model Articulation Controller (CMAC) is an influential
brain-inspired computing model in many relevant fields. Since its inception in
the 1970s, the model has been intensively studied and many variants of the
prototype, such as Kernel-CMAC, Self-Organizing Map CMAC, and Linguistic CMAC,
have been proposed. This review article focus on how the CMAC model is
gradually developed and refined to meet the demand of fast, adaptive, and
robust control. Two perspective, CMAC as a neural network and CMAC as a table
look-up technique are presented. Three aspects of the model: the architecture,
learning algorithms and applications are discussed. In the end, some potential
future research directions on this model are suggested.
Eric O. Scott, Kenneth A. De Jong
Comments: 2 pages
Subjects: Neural and Evolutionary Computing (cs.NE)
We introduce a genetic programming method for solving multiple Boolean
circuit synthesis tasks simultaneously. This allows us to solve a set of
elementary logic functions twice as easily as with a direct, single-task
approach.
Moshe Looks, Marcello Herreshoff, DeLesley Hutchins, Peter Norvig
Comments: Published as a conference paper at ICLR 2017
Subjects: Neural and Evolutionary Computing (cs.NE); Learning (cs.LG); Machine Learning (stat.ML)
Neural networks that compute over graph structures are a natural fit for
problems in a variety of domains, including natural language (parse trees) and
cheminformatics (molecular graphs). However, since the computation graph has a
different shape and size for every input, such networks do not directly support
batched training or inference. They are also difficult to implement in popular
deep learning libraries, which are based on static data-flow graphs. We
introduce a technique called dynamic batching, which not only batches together
operations between different input graphs of dissimilar shape, but also between
different nodes within a single input graph. The technique allows us to create
static graphs, using popular libraries, that emulate dynamic computation graphs
of arbitrary shape and size. We further present a high-level library of
compositional blocks that simplifies the creation of dynamic graph models.
Using the library, we demonstrate concise and batch-wise parallel
implementations for a variety of models from the literature.
W. James Murdoch, Arthur Szlam
Comments: ICLR 2017 accepted paper
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Although deep learning models have proven effective at solving problems in
natural language processing, the mechanism by which they come to their
conclusions is often unclear. As a result, these models are generally treated
as black boxes, yielding no insight of the underlying learned patterns. In this
paper we consider Long Short Term Memory networks (LSTMs) and demonstrate a new
approach for tracking the importance of a given input to the LSTM for a given
output. By identifying consistently important patterns of words, we are able to
distill state of the art LSTMs on sentiment analysis and question answering
into a set of representative phrases. This representation is then
quantitatively validated by using the extracted phrases to construct a simple,
rule-based classifier which approximates the output of the LSTM.
Michael Kampffmeyer, Sigurd Løkse, Filippo Maria Bianchi, Robert Jenssen, Lorenzo Livi
Subjects: Machine Learning (stat.ML); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
In this paper we introduce the deep kernelized autoencoder, a neural network
model that allows an explicit approximation of (i) the mapping from an input
space to an arbitrary, user-specified kernel space and (ii) the back-projection
from such a kernel space to input space. The proposed method is based on
traditional autoencoders and is trained through a new unsupervised loss
function. During training, we optimize both the reconstruction accuracy of
input samples and the alignment between a kernel matrix given as prior and the
inner products of the hidden representations computed by the autoencoder.
Kernel alignment provides control over the hidden representation learned by the
autoencoder. Experiments have been performed to evaluate both reconstruction
and kernel alignment performance. Additionally, we applied our method to
emulate kPCA on a denoising task obtaining promising results.
Eric Risser, Pierre Wilmot, Connelly Barnes
Subjects: Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)
Recently, methods have been proposed that perform texture synthesis and style
transfer by using convolutional neural networks (e.g. Gatys et al.
[2015,2016]). These methods are exciting because they can in some cases create
results with state-of-the-art quality. However, in this paper, we show these
methods also have limitations in texture quality, stability, requisite
parameter tuning, and lack of user controls. This paper presents a multiscale
synthesis pipeline based on convolutional neural networks that ameliorates
these issues. We first give a mathematical explanation of the source of
instabilities in many previous approaches. We then improve these instabilities
by using histogram losses to synthesize textures that better statistically
match the exemplar. We also show how to integrate localized style losses in our
multiscale framework. These losses can improve the quality of large features,
improve the separation of content and style, and offer artistic controls such
as paint by numbers. We demonstrate that our approach offers improved quality,
convergence in fewer iterations, and more stability over the optimization.
Patrick Wieschollek, Fabian Groh, Hendrik P.A. Lensch
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Fisher-Vectors (FV) encode higher-order statistics of a set of multiple local
descriptors like SIFT features. They already show good performance in
combination with shallow learning architectures on visual recognitions tasks.
Current methods using FV as a feature descriptor in deep architectures assume
that all original input features are static. We propose a framework to jointly
learn the representation of original features, FV parameters and parameters of
the classifier in the style of traditional neural networks. Our proof of
concept implementation improves the performance of FV on the Pascal Voc 2007
challenge in a multi-GPU setting in comparison to a default SVM setting. We
demonstrate that FV can be embedded into neural networks at arbitrary
positions, allowing end-to-end training with back-propagation.
Olasimbo Ayodeji Arigbabu, Sharifah Mumtazah Syed Ahmad, Wan Azizun Wan Adnan, Salman Yussof, Saif Mahmood
Journal-ref: Journal of Information and Communication Technology (JICT), 2015
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Gender recognition from unconstrained face images is a challenging task due
to the high degree of misalignment, pose, expression, and illumination
variation. In previous works, the recognition of gender from unconstrained face
images is approached by utilizing image alignment, exploiting multiple samples
per individual to improve the learning ability of the classifier, or learning
gender based on prior knowledge about pose and demographic distributions of the
dataset. However, image alignment increases the complexity and time of
computation, while the use of multiple samples or having prior knowledge about
data distribution is unrealistic in practical applications. This paper presents
an approach for gender recognition from unconstrained face images. Our
technique exploits the robustness of local feature descriptor to photometric
variations to extract the shape description of the 2D face image using a single
sample image per individual. The results obtained from experiments on Labeled
Faces in the Wild (LFW) dataset describe the effectiveness of the proposed
method. The essence of this study is to investigate the most suitable functions
and parameter settings for recognizing gender from unconstrained face images.
Markus Höll, Vincent Lepetit
Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Software Engineering (cs.SE)
In this paper, we cover the process of integrating Large-Scale Direct
Simultaneous Localization and Mapping (LSD-SLAM) algorithm into our existing AR
stereo engine, developed for our modified “Augmented Reality Oculus Rift”. With
that, we are able to track one of our realworld cameras which are mounted on
the rift, within a complete unknown environment. This makes it possible to
achieve a constant and full augmentation, synchronizing our 3D movement (x, y,
z) in both worlds, the real world and the virtual world. The development for
the basic AR setup using the Oculus Rift DK1 and two fisheye cameras is fully
documented in our previous paper. After an introduction to image-based
registration, we detail the LSD-SLAM algorithm and document our code
implementing our integration. The AR stereo engine with Oculus Rift support can
be accessed via the GIT repository this https URL and
the modified LSD-SLAM project used for the integration is available here
this https URL
Yi Zhou, Laurent Kneip, Hongdong Li
Comments: ICRA 2017
Subjects: Computer Vision and Pattern Recognition (cs.CV)
This paper presents a robust and efficient semi-dense visual odometry
solution for RGB-D cameras. The core of our method is a 2D-3D ICP pipeline
which estimates the pose of the sensor by registering the projection of a 3D
semi-dense map of the reference frame with the 2D semi-dense region extracted
in the current frame. The processing is speeded up by efficiently implemented
approximate nearest neighbour fields under the Euclidean distance criterion,
which permits the use of compact Gauss-Newton updates in the optimization. The
registration is formulated as a maximum a posterior problem to deal with
outliers and sensor noises, and consequently the equivalent weighted least
squares problem is solved by iteratively reweighted least squares method. A
variety of robust weight functions are tested and the optimum is determined
based on the characteristics of the sensor model. Extensive evaluation on
publicly available RGB-D datasets shows that the proposed method predominantly
outperforms existing state-of-the-art methods.
Corneliu Arsene, Peter Pormann, William Sellers, Siam Bhayro
Comments: 29 February – 2 March 2016, Second International Conference on Natural Sciences and Technology in Manuscript Analysis, Centre for the study of Manuscript Cultures, Hamburg, Germany
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Multispectral and hyperspectral image analysis has experienced much
development in the last decade. The application of these methods to palimpsests
has produced significant results, enabling researchers to recover texts that
would be otherwise lost under the visible overtext, by improving the contrast
between the undertext and the overtext. In this paper we explore an extended
number of multispectral and hyperspectral image analysis methods, consisting of
supervised and unsupervised dimensionality reduction techniques, on a part of
the Syriac Galen Palimpsest dataset (www.digitalgalen.net). Of this extended
set of methods, eight methods gave good results: three were supervised methods
Generalized Discriminant Analysis (GDA), Linear Discriminant Analysis (LDA),
and Neighborhood Component Analysis (NCA); and the other five methods were
unsupervised methods (but still used in a supervised way) Gaussian Process
Latent Variable Model (GPLVM), Isomap, Landmark Isomap, Principal Component
Analysis (PCA), and Probabilistic Principal Component Analysis (PPCA). The
relative success of these methods was determined visually, using color
pictures, on the basis of whether the undertext was distinguishable from the
overtext, resulting in the following ranking of the methods: LDA, NCA, GDA,
Isomap, Landmark Isomap, PPCA, PCA, and GPLVM. These results were compared with
those obtained using the Canonical Variates Analysis (CVA) method on the same
dataset, which showed remarkably accuracy (LDA is a particular case of CVA
where the objects are classified to two classes).
Ziwei Liu, Raymond Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala
Comments: Project page: this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Learning (cs.LG)
We address the problem of synthesizing new video frames in an existing video,
either in-between existing frames (interpolation), or subsequent to them
(extrapolation). This problem is challenging because video appearance and
motion can be highly complex. Traditional optical-flow-based solutions often
fail where flow estimation is challenging, while newer neural-network-based
methods that hallucinate pixel values directly often produce blurry results. We
combine the advantages of these two methods by training a deep network that
learns to synthesize video frames by flowing pixel values from existing ones,
which we call deep voxel flow. Our method requires no human supervision, and
any video can be used as training data by dropping, and then learning to
predict, existing frames. The technique is efficient, and can be applied at any
video resolution. We demonstrate that our method produces results that both
quantitatively and qualitatively improve upon the state-of-the-art.
Hengkai Guo, Guijin Wang, Xinghao Chen, Cairong Zhang, Fei Qiao, Huazhong Yang
Comments: Submitted to ICIP 2017
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Hand pose estimation from monocular depth images is an important and
challenging problem for human-computer interaction. Recently deep convolutional
networks (ConvNet) with sophisticated design have been employed to address it,
but the improvement over traditional methods is not so apparent. To promote the
performance of directly 3D coordinate regression, we propose a tree-structured
Region Ensemble Network (REN), which partitions the convolution outputs into
regions and integrates the results from multiple regressors on each regions.
Compared with multi-model ensemble, our model is completely end-to-end
training. The experimental results demonstrate that our approach achieves the
best performance among state-of-the-arts on two public datasets.
Afonso M. Teodoro, José M. Bioucas-Dias, Mário A. T. Figueiredo
Comments: submitted
Subjects: Computer Vision and Pattern Recognition (cs.CV)
This paper tackles a hyperspectral data fusion problem, using the so-called
plug-and-play approach, which combines an ADMM algorithm with a denoiser based
on a Gaussian mixture prior. We build upon the concept of scene-adapted prior
where, as the name suggests, we learn a model that is targeted to the specific
scene being imaged, and show state-of-the-art results on several hyperspectral
sharpening experiments. Additionally, we prove that the algorithm is guaranteed
to converge.
Mateusz Koziński, Loïc Simon, Frédéric Jurie
Subjects: Computer Vision and Pattern Recognition (cs.CV)
We propose a method for semi-supervised training of structured-output neural
networks. Inspired by the framework of Generative Adversarial Networks (GAN),
we train a discriminator network to capture the notion of a quality of network
output. To this end, we leverage the qualitative difference between outputs
obtained on the labelled training data and unannotated data. We then use the
discriminator as a source of error signal for unlabelled data. This effectively
boosts the performance of a network on a held out test set. Initial experiments
in image segmentation demonstrate that the proposed framework enables achieving
the same network performance as in a fully supervised scenario, while using two
times less annotations.
Lingke Zeng, Xiangmin Xu, Bolun Cai, Suo Qiu, Tong Zhang
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Crowd counting on static images is a challenging problem due to scale
variations. Recently deep neural networks have been shown to be effective in
this task. However, existing neural-networks-based methods often use the
multi-column or multi-network model to extract the scale-relevant features,
which is more complicated for optimization and computation wasting. To this
end, we propose a novel multi-scale convolutional neural network (MSCNN) for
single image crowd counting. Based on the multi-scale blobs, the network is
able to generate scale-relevant features for higher crowd counting performances
in a single-column architecture, which is both accuracy and cost effective for
practical applications. Complemental results show that our method outperforms
the state-of-the-art methods on both accuracy and robustness with far less
number of parameters.
Yi Zhu, Zhenzhong Lan, Shawn Newsam, Alexander G. Hauptmann
Subjects: Computer Vision and Pattern Recognition (cs.CV)
We study the unsupervised learning of CNNs for optical flow estimation using
proxy ground truth data. Supervised CNNs, due to their immense learning
capacity, have shown superior performance on a range of computer vision
problems including optical flow prediction. They however require the ground
truth flow which is usually not accessible except on limited synthetic data.
Without the guidance of ground truth optical flow, unsupervised CNNs often
perform worse as they are naturally ill-conditioned. We therefore propose a
novel framework in which proxy ground truth data generated from classical
approaches is used to guide the CNN learning. The models are further refined in
an unsupervised fashion using an image reconstruction loss. Our guided learning
approach is competitive with or superior to state-of-the-art approaches on
three standard benchmark datasets yet is completely unsupervised and can run in
real time.
Ehsan Jahangiri, Alan L. Yuille
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Machine Learning (stat.ML)
We propose a method to generate multiple hypotheses for human 3D pose all of
them consistent with the 2D detection of joints in a monocular RGB image. To
generate these pose hypotheses we use a novel generative model defined in the
space of anatomically plausible 3D poses satisfying the joint angle limits and
limb length ratios. The proposed generative model is uniform in the space of
anatomically valid poses and as a result, does not suffer from the dataset bias
in existing motion capture datasets such as Human3.6M (H36M), HumanEva, and CMU
MoCap. A good model that spans the full variability of human pose and
generalizes to unseen poses must be compositional i.e., produce a pose by
combining parts. Our model is flexible and compositional and consequently can
generalize to every plausible human 3D pose since it is only limited by
physical constraints. We discuss sampling from this model and use these samples
to generate multiple diverse human 3D pose hypotheses given the 2D detection of
joints. We argue that generating multiple pose hypotheses from a monocular RGB
image is more reasonable than generating only a single 3D pose given the depth
ambiguity and the uncertainty caused by occlusion and imperfect 2D joint
detection. To support this argument, we have performed empirical evaluation on
the popular Human3.6M dataset that confirms that most often, at least one of
our pose hypotheses is closer to the true 3D pose compared to the estimated
pose by other recent baseline methods for 3D pose reconstruction from monocular
RGB images. The idea of generating multiple consistent and valid pose
hypotheses can give rise to a new line of future work that has not previously
been addressed in the literature.
Pei Wang, Guochao Bu, Ronghao Li, Rui Zhao
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Terrestrial laser scanner is a kind of fast, high-precision data acquisition
device, which had been more and more applied to the research areas of forest
inventory. In this study, a kind of automated low-cost terrestrial laser
scanner was designed and implemented based on a two-dimensional laser radar
sensor SICK LMS-511 and a stepper motor. The new scanner was named as BEE,
which can scan the forest trees in three dimension. The BEE scanner and its
supporting software are specifically designed for forest inventory. The
experiments have been performed by using the BEE scanner in an artificial
ginkgo forest which was located in Haidian district of Beijing. Four square
plots were selected to do the experiments. The BEE scanner scanned in the four
plots and acquired the single scan data respectively. The DBH, tree height and
tree position of trees in the four plots were estimated and analyzed. For
comparison, the manual measured data was also collected in the four plots. The
tree stem detection rate for all four plots was 92.75%; the root mean square
error of the DBH estimation was 1.27cm; the root mean square error of the tree
height estimation was 0.24m; the tree position estimation was in line with the
actual position. Experimental results show that the BEE scanner can efficiently
estimate the structure parameters of forest trees and has a good potential in
practical application of forest inventory.
Hongkai Wang, Zongwei Zhou, Yingci Li, Zhonghua Chen, Peiou Lu, Wenzhi Wang, Wanyu Liu, Lijuan Yu
Subjects: Computer Vision and Pattern Recognition (cs.CV); Medical Physics (physics.med-ph)
The present study shows that the performance of CNN is not significantly
different from the best classical methods and human doctors for classifying
mediastinal lymph node metastasis of NSCLC from PET/CT images. Because CNN does
not need tumor segmentation or feature calculation, it is more convenient and
more objective than the classical methods. However, CNN does not make use of
the import diagnostic features, which have been proved more discriminative than
the texture features for classifying small-sized lymph nodes. Therefore,
incorporating the diagnostic features into CNN is a promising direction for
future research.
Anton Kasyanov, Francis Engelmann, Jörg Stückler, Bastian Leibe
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Complementing images with inertial measurements has become one of the most
popular approaches to achieve highly accurate and robust real-time camera pose
tracking. In this paper, we present a keyframe-based approach to
visual-inertial simultaneous localization and mapping (SLAM) for monocular and
stereo cameras. Our method is based on a real-time capable visual-inertial
odometry method that provides locally consistent trajectory and map estimates.
We achieve global consistency in the estimate through online loop-closing and
non-linear optimization. Furthermore, our approach supports relocalization in a
map that has been previously obtained and allows for continued SLAM operation.
We evaluate our approach in terms of accuracy, relocalization capability and
run-time efficiency on public benchmark datasets and on newly recorded
sequences. We demonstrate state-of-the-art performance of our approach towards
a visual-inertial odometry method in recovering the trajectory of the camera.
Eric Risser, Pierre Wilmot, Connelly Barnes
Subjects: Graphics (cs.GR); Computer Vision and Pattern Recognition (cs.CV); Neural and Evolutionary Computing (cs.NE)
Recently, methods have been proposed that perform texture synthesis and style
transfer by using convolutional neural networks (e.g. Gatys et al.
[2015,2016]). These methods are exciting because they can in some cases create
results with state-of-the-art quality. However, in this paper, we show these
methods also have limitations in texture quality, stability, requisite
parameter tuning, and lack of user controls. This paper presents a multiscale
synthesis pipeline based on convolutional neural networks that ameliorates
these issues. We first give a mathematical explanation of the source of
instabilities in many previous approaches. We then improve these instabilities
by using histogram losses to synthesize textures that better statistically
match the exemplar. We also show how to integrate localized style losses in our
multiscale framework. These losses can improve the quality of large features,
improve the separation of content and style, and offer artistic controls such
as paint by numbers. We demonstrate that our approach offers improved quality,
convergence in fewer iterations, and more stability over the optimization.
Clément Carbonnel (LAAS-ROC), Emmanuel Hébrard (LAAS-ROC)
Journal-ref: Michel Rueher. The 22nd International Conference on Principles and
Practice of Constraint Programming, Sep 2016, Toulouse, France. Lecture Notes
in Computer Science, 9892, pp.147 – 156, 2016, Principles and Practice of
Constraint Programming
Subjects: Artificial Intelligence (cs.AI)
The technique of kernelization consists in extracting, from an instance of a
problem, an essentially equivalent instance whose size is bounded in a
parameter k. Besides being the basis for efficient param-eterized algorithms,
this method also provides a wealth of information to reason about in the
context of constraint programming. We study the use of kernelization for
designing propagators through the example of the Vertex Cover constraint. Since
the classic kernelization rules often correspond to dominance rather than
consistency, we introduce the notion of “loss-less” kernel. While our
preliminary experimental results show the potential of the approach, they also
show some of its limits. In particular, this method is more effective for
vertex covers of large and sparse graphs, as they tend to have, relatively,
smaller kernels.
Hyunmin Chae, Chang Mook Kang, ByeoungDo Kim, Jaekyum Kim, Chung Choo Chung, Jun Won Choi
Subjects: Artificial Intelligence (cs.AI)
In this paper, we propose a new autonomous braking system based on deep
reinforcement learning. The proposed autonomous braking system automatically
decides whether to apply the brake at each time step when confronting the risk
of collision using the information on the obstacle obtained by the sensors. The
problem of designing brake control is formulated as searching for the optimal
policy in Markov decision process (MDP) model where the state is given by the
relative position of the obstacle and the vehicle’s speed, and the action space
is defined as whether brake is stepped or not. The policy used for brake
control is learned through computer simulations using the deep reinforcement
learning method called deep Q-network (DQN). In order to derive desirable
braking policy, we propose the reward function which balances the damage
imposed to the obstacle in case of accident and the reward achieved when the
vehicle runs out of risk as soon as possible. DQN is trained for the scenario
where a vehicle is encountered with a pedestrian crossing the urban road.
Experiments show that the control agent exhibits desirable control behavior and
avoids collision without any mistake in various uncertain environments.
W. James Murdoch, Arthur Szlam
Comments: ICLR 2017 accepted paper
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Although deep learning models have proven effective at solving problems in
natural language processing, the mechanism by which they come to their
conclusions is often unclear. As a result, these models are generally treated
as black boxes, yielding no insight of the underlying learned patterns. In this
paper we consider Long Short Term Memory networks (LSTMs) and demonstrate a new
approach for tracking the importance of a given input to the LSTM for a given
output. By identifying consistently important patterns of words, we are able to
distill state of the art LSTMs on sentiment analysis and question answering
into a set of representative phrases. This representation is then
quantitatively validated by using the extracted phrases to construct a simple,
rule-based classifier which approximates the output of the LSTM.
Adrian Benton, Huda Khayrallah, Biman Gujral, Drew Reisinger, Sheng Zhang, Raman Arora
Comments: 14 pages, 6 figures
Subjects: Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
We present Deep Generalized Canonical Correlation Analysis (DGCCA) — a
method for learning nonlinear transformations of arbitrarily many views of
data, such that the resulting transformations are maximally informative of each
other. While methods for nonlinear two-view representation learning (Deep CCA,
(Andrew et al., 2013)) and linear many-view representation learning
(Generalized CCA (Horst, 1961)) exist, DGCCA is the first CCA-style multiview
representation learning technique that combines the flexibility of nonlinear
(deep) representation learning with the statistical power of incorporating
information from many independent sources, or views. We present the DGCCA
formulation as well as an efficient stochastic optimization algorithm for
solving it. We learn DGCCA repre- sentations on two distinct datasets for three
downstream tasks: phonetic transcrip- tion from acoustic and articulatory
measurements, and recommending hashtags and friends on a dataset of Twitter
users. We find that DGCCA representations soundly beat existing methods at
phonetic transcription and hashtag recommendation, and in general perform no
worse than standard linear many-view techniques.
Frank Z. Xing
Subjects: Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI)
The Cerebellar Model Articulation Controller (CMAC) is an influential
brain-inspired computing model in many relevant fields. Since its inception in
the 1970s, the model has been intensively studied and many variants of the
prototype, such as Kernel-CMAC, Self-Organizing Map CMAC, and Linguistic CMAC,
have been proposed. This review article focus on how the CMAC model is
gradually developed and refined to meet the demand of fast, adaptive, and
robust control. Two perspective, CMAC as a neural network and CMAC as a table
look-up technique are presented. Three aspects of the model: the architecture,
learning algorithms and applications are discussed. In the end, some potential
future research directions on this model are suggested.
Ehsan Jahangiri, Alan L. Yuille
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multimedia (cs.MM); Machine Learning (stat.ML)
We propose a method to generate multiple hypotheses for human 3D pose all of
them consistent with the 2D detection of joints in a monocular RGB image. To
generate these pose hypotheses we use a novel generative model defined in the
space of anatomically plausible 3D poses satisfying the joint angle limits and
limb length ratios. The proposed generative model is uniform in the space of
anatomically valid poses and as a result, does not suffer from the dataset bias
in existing motion capture datasets such as Human3.6M (H36M), HumanEva, and CMU
MoCap. A good model that spans the full variability of human pose and
generalizes to unseen poses must be compositional i.e., produce a pose by
combining parts. Our model is flexible and compositional and consequently can
generalize to every plausible human 3D pose since it is only limited by
physical constraints. We discuss sampling from this model and use these samples
to generate multiple diverse human 3D pose hypotheses given the 2D detection of
joints. We argue that generating multiple pose hypotheses from a monocular RGB
image is more reasonable than generating only a single 3D pose given the depth
ambiguity and the uncertainty caused by occlusion and imperfect 2D joint
detection. To support this argument, we have performed empirical evaluation on
the popular Human3.6M dataset that confirms that most often, at least one of
our pose hypotheses is closer to the true 3D pose compared to the estimated
pose by other recent baseline methods for 3D pose reconstruction from monocular
RGB images. The idea of generating multiple consistent and valid pose
hypotheses can give rise to a new line of future work that has not previously
been addressed in the literature.
Baichuan Zhang, Mohammad Al Hasan
Subjects: Social and Information Networks (cs.SI); Computation and Language (cs.CL); Information Retrieval (cs.IR)
In real-world, our DNA is unique but many people share same names. This
phenomenon often causes erroneous aggregation of documents of multiple persons
who are namesake of one another. Such mistakes deteriorate the performance of
document retrieval, web search, and more seriously, cause improper attribution
of credit or blame in digital forensic. To resolve this issue, the name entity
disambiguation task is designed which aims to partition the documents
associated with a name reference such that each partition contains documents
pertaining to a unique real-life person. Existing solutions to this task
substantially rely on feature engineering, such as biographical feature
extraction, or construction of auxiliary features from Wikipedia. However, for
many scenarios, such features may be costly to obtain or unavailable due to the
risk of privacy violation. In this work, we propose a novel name disambiguation
method. Our proposed method is non-intrusive of privacy because instead of
using attributes pertaining to a real-life person, our method leverages only
relational data in the form of anonymized graphs. In the aspect of
methodological novelty, the proposed method uses a representation learning
strategy to embed each document in a low dimensional vector space where name
disambiguation can be solved by a hierarchical agglomerative clustering
algorithm. Our experimental results demonstrate that the proposed method is
significantly better than the existing name entity disambiguation methods
working in a similar setting.
W. James Murdoch, Arthur Szlam
Comments: ICLR 2017 accepted paper
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Although deep learning models have proven effective at solving problems in
natural language processing, the mechanism by which they come to their
conclusions is often unclear. As a result, these models are generally treated
as black boxes, yielding no insight of the underlying learned patterns. In this
paper we consider Long Short Term Memory networks (LSTMs) and demonstrate a new
approach for tracking the importance of a given input to the LSTM for a given
output. By identifying consistently important patterns of words, we are able to
distill state of the art LSTMs on sentiment analysis and question answering
into a set of representative phrases. This representation is then
quantitatively validated by using the extracted phrases to construct a simple,
rule-based classifier which approximates the output of the LSTM.
Ye Zhang, Matthew Lease, Byron C. Wallace
Subjects: Computation and Language (cs.CL)
A fundamental advantage of neural models for NLP is their ability to learn
representations from scratch. However, in practice this often means ignoring
existing external linguistic resources, e.g., WordNet or domain specific
ontologies such as the Unified Medical Language System (UMLS). We propose a
general, novel method for exploiting such resources via weight sharing. Prior
work on weight sharing in neural networks has considered it largely as a means
of model compression. In contrast, we treat weight sharing as a flexible
mechanism for incorporating prior knowledge into neural models. We show that
this approach consistently yields improved performance on classification tasks
compared to baseline strategies that do not exploit weight sharing.
Jiatao Gu, Kyunghyun Cho, Victor O.K. Li
Comments: 10 pages
Subjects: Computation and Language (cs.CL); Learning (cs.LG)
Recent research in neural machine translation has largely focused on two
aspects; neural network architectures and end-to-end learning algorithms. The
problem of decoding, however, has received relatively little attention from the
research community. In this paper, we solely focus on the problem of decoding
given a trained neural machine translation model. Instead of trying to build a
new decoding algorithm for any specific decoding objective, we propose the idea
of trainable decoding algorithm in which we train a decoding algorithm to find
a translation that maximizes an arbitrary decoding objective. More
specifically, we design an actor that observes and manipulates the hidden state
of the neural machine translation decoder and propose to train it using a
variant of deterministic policy gradient. We extensively evaluate the proposed
algorithm using four language pairs and two decoding objectives and show that
we can indeed train a trainable greedy decoder that generates a better
translation (in terms of a target decoding objective) with minimal
computational overhead.
Sebastian Ruder, Parsa Ghaffari, John G. Breslin
Comments: 10 pages, 2 figures, 4 tables
Subjects: Computation and Language (cs.CL); Learning (cs.LG)
Domain adaptation is important in sentiment analysis as sentiment-indicating
words vary between domains. Recently, multi-domain adaptation has become more
pervasive, but existing approaches train on all available source domains
including dissimilar ones. However, the selection of appropriate training data
is as important as the choice of algorithm. We undertake — to our knowledge
for the first time — an extensive study of domain similarity metrics in the
context of sentiment analysis and propose novel representations, metrics, and a
new scope for data selection. We evaluate the proposed methods on two
large-scale multi-domain adaptation settings on tweets and reviews and
demonstrate that they consistently outperform strong random and balanced
baselines, while our proposed selection strategy outperforms instance-level
selection and yields the best score on a large reviews corpus.
Stanislau Semeniuta, Aliaksei Severyn, Erhardt Barth
Subjects: Computation and Language (cs.CL)
In this paper we explore the effect of architectural choices on learning a
Variational Autoencoder (VAE) for text generation. In contrast to the
previously introduced VAE model for text where both the encoder and decoder are
RNNs, we propose a novel hybrid architecture that blends fully feed-forward
convolutional and deconvolutional components with a recurrent language model.
Our architecture exhibits several attractive properties such as faster run time
and convergence, ability to better handle long sequences and, more importantly,
it helps to avoid some of the major difficulties posed by training VAE models
on textual data.
Claudio Greco, Alessandro Suglia, Pierpaolo Basile, Gaetano Rossiello, Giovanni Semeraro
Comments: Paper accepted and presented at the Deep Understanding and Reasoning: A challenge for Next-generation Intelligent Agents (URANIA) workshop, held in the context of the AI*IA 2016 conference
Subjects: Computation and Language (cs.CL)
People have information needs of varying complexity, which can be solved by
an intelligent agent able to answer questions formulated in a proper way,
eventually considering user context and preferences. In a scenario in which the
user profile can be considered as a question, intelligent agents able to answer
questions can be used to find the most relevant answers for a given user. In
this work we propose a novel model based on Artificial Neural Networks to
answer questions with multiple answers by exploiting multiple facts retrieved
from a knowledge base. The model is evaluated on the factoid Question Answering
and top-n recommendation tasks of the bAbI Movie Dialog dataset. After
assessing the performance of the model on both tasks, we try to define the
long-term goal of a conversational recommender system able to interact using
natural language and to support users in their information seeking processes in
a personalized way.
H. Bahadir Sahin, Caglar Tirkaz, Eray Yildiz, Mustafa Tolga Eren, Ozan Sonmez
Comments: 10 page, 1 figure, white paper
Subjects: Computation and Language (cs.CL)
Turkish Wikipedia Named-Entity Recognition and Text Categorization (TWNERTC)
dataset is a collection of automatically categorized and annotated sentences
obtained from Wikipedia. We constructed large-scale gazetteers by using a graph
crawler algorithm to extract relevant entity and domain information from a
semantic knowledge base, Freebase. The constructed gazetteers contains
approximately 300K entities with thousands of fine-grained entity types under
77 different domains. Since automated processes are prone to ambiguity, we also
introduce two new content specific noise reduction methodologies. Moreover, we
map fine-grained entity types to the equivalent four coarse-grained types:
person, loc, org, misc. Eventually, we construct six different dataset versions
and evaluate the quality of annotations by comparing ground truths from human
annotators. We make these datasets publicly available to support studies on
Turkish named-entity recognition (NER) and text categorization (TC).
Kazuma Hashimoto, Yoshimasa Tsuruoka
Subjects: Computation and Language (cs.CL)
This paper presents a novel neural machine translation model which jointly
learns translation and source-side latent graph representations of sentences.
Unlike existing pipelined approaches using syntactic parsers, our end-to-end
model learns a latent graph parser as part of the encoder of an attention-based
neural machine translation model, so the parser is optimized according to the
translation objective. Experimental results show that our model significantly
outperforms the previous best results on the standard English-to-Japanese
translation dataset.
Pramod Bharadwaj Chandrashekar (1), Arjun Magge (1), Abeed Sarker (2), Graciela Gonzalez (2) ((1) Arizona State University, (2) University of Pennsylvania)
Comments: 9 pages
Subjects: Computation and Language (cs.CL)
Widespread use of social media has led to the generation of substantial
amounts of information about individuals, including health-related information.
Social media provides the opportunity to study health-related information about
selected population groups who may be of interest for a particular study. In
this paper, we explore the possibility of utilizing social media to perform
targeted data collection and analysis from a particular population group —
pregnant women. We hypothesize that we can use social media to identify cohorts
of pregnant women and follow them over time to analyze crucial health-related
information. To identify potentially pregnant women, we employ simple
rule-based searches that attempt to detect pregnancy announcements with
moderate precision. To further filter out false positives and noise, we employ
a supervised classifier using a small number of hand-annotated data. We then
collect their posts over time to create longitudinal health timelines and
attempt to divide the timelines into different pregnancy trimesters. Finally,
we assess the usefulness of the timelines by performing a preliminary analysis
to estimate drug intake patterns of our cohort at different trimesters. Our
rule-based cohort identification technique collected 53,820 users over thirty
months from Twitter. Our pregnancy announcement classification technique
achieved an F-measure of 0.81 for the pregnancy class, resulting in 34,895 user
timelines. Analysis of the timelines revealed that pertinent health-related
information, such as drug-intake and adverse reactions can be mined from the
data. Our approach to using user timelines in this fashion has produced very
encouraging results and can be employed for other important tasks where
cohorts, for which health-related information may not be available from other
sources, are required to be followed over time to derive population-based
estimates.
Tarek Sakakini, Suma Bhat, Pramod Viswanath
Subjects: Computation and Language (cs.CL)
We present in this paper a novel framework for morpheme segmentation which
uses the morpho-syntactic regularities preserved by word representations, in
addition to orthographic features, to segment words into morphemes. This
framework is the first to consider vocabulary-wide syntactico-semantic
information for this task. We also analyze the deficiencies of available
benchmarking datasets and introduce our own dataset that was created on the
basis of compositionality. We validate our algorithm across datasets and
present state-of-the-art results.
Tarek Sakakini, Suma Bhat, Pramod Viswanath
Subjects: Computation and Language (cs.CL)
We present an unsupervised and language-agnostic method for learning
root-and-pattern morphology in Semitic languages. This form of morphology,
abundant in Semitic languages, has not been handled in prior unsupervised
approaches. We harness the syntactico-semantic information in distributed word
representations to solve the long standing problem of root-and-pattern
discovery in Semitic languages. Moreover, we construct an unsupervised root
extractor based on the learned rules. We prove the validity of learned rules
across Arabic, Hebrew, and Amharic, alongside showing that our root extractor
compares favorably with a widely used, carefully engineered root extractor:
ISRI.
Zhilin Yang, Junjie Hu, Ruslan Salakhutdinov, William W. Cohen
Subjects: Computation and Language (cs.CL); Learning (cs.LG)
We study the problem of semi-supervised question answering—-utilizing
unlabeled text to boost the performance of question answering models. We
propose a novel training framework, the Generative Domain-Adaptive Nets. In
this framework, we train a generative model to generate questions based on the
unlabeled text, and combine model-generated questions with human-generated
questions for training question answering models. We develop novel domain
adaptation algorithms, based on reinforcement learning, to alleviate the
discrepancy between the model-generated data distribution and the
human-generated data distribution. Experiments show that our proposed framework
obtains substantial improvement from unlabeled text.
Sewon Min, Minjoon Seo, Hannaneh Hajishirzi
Subjects: Computation and Language (cs.CL)
We show that the task of question answering (QA) can significantly benefit
from the transfer learning of models trained on a different large, fine-grained
QA dataset. We achieve the state of the art in two well-studied QA datasets,
WikiQA and SemEval-2016 (Task 3A), through a basic transfer learning technique
from SQuAD. For WikiQA, our model outperforms the previous best model by more
than 8%. We demonstrate that finer supervision provides better guidance for
learning lexical and syntactic information than coarser supervision, through
quantitative results and visual analysis. We also show that a similar transfer
learning procedure achieves the state of the art on an entailment task.
Stanisław Jastrzebski, Damian Leśniak, Wojciech Marian Czarnecki
Subjects: Computation and Language (cs.CL)
Maybe the single most important goal of representation learning is making
subsequent learning faster. Surprisingly, this fact is not well reflected in
the way embeddings are evaluated. In addition, recent practice in word
embeddings points towards importance of learning specialized representations.
We argue that focus of word representation evaluation should reflect those
trends and shift towards evaluating what useful information is easily
accessible. Specifically, we propose that evaluation should focus on data
efficiency and simple supervised tasks, where the amount of available data is
varied and scores of a supervised model are reported for each subset (as
commonly done in transfer learning).
In order to illustrate significance of such analysis, a comprehensive
evaluation of selected word embeddings is presented. Proposed approach yields a
more complete picture and brings new insight into performance characteristics,
for instance information about word similarity or analogy tends to be
non–linearly encoded in the embedding space, which questions the cosine-based,
unsupervised, evaluation methods. All results and analysis scripts are
available online.
Baichuan Zhang, Mohammad Al Hasan
Subjects: Social and Information Networks (cs.SI); Computation and Language (cs.CL); Information Retrieval (cs.IR)
In real-world, our DNA is unique but many people share same names. This
phenomenon often causes erroneous aggregation of documents of multiple persons
who are namesake of one another. Such mistakes deteriorate the performance of
document retrieval, web search, and more seriously, cause improper attribution
of credit or blame in digital forensic. To resolve this issue, the name entity
disambiguation task is designed which aims to partition the documents
associated with a name reference such that each partition contains documents
pertaining to a unique real-life person. Existing solutions to this task
substantially rely on feature engineering, such as biographical feature
extraction, or construction of auxiliary features from Wikipedia. However, for
many scenarios, such features may be costly to obtain or unavailable due to the
risk of privacy violation. In this work, we propose a novel name disambiguation
method. Our proposed method is non-intrusive of privacy because instead of
using attributes pertaining to a real-life person, our method leverages only
relational data in the form of anonymized graphs. In the aspect of
methodological novelty, the proposed method uses a representation learning
strategy to embed each document in a low dimensional vector space where name
disambiguation can be solved by a hierarchical agglomerative clustering
algorithm. Our experimental results demonstrate that the proposed method is
significantly better than the existing name entity disambiguation methods
working in a similar setting.
Petros Giannakopoulos, Michail Gkoumas, Ioannis Diplas, Georgios Voularinos, Theofanis Vlachos, Konstantia Balasi, Ekaterini Tzamariudaki, Christos Filippidis, Yiannis Cotronis, Christos Markou
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
The primary objective of SIRENE is to simulate the response to neutrino
events of any type of high energy neutrino telescope. Additionally, it
implements different geometries for a neutrino detector and different
configurations and characteristics of photo-multiplier tubes (PMTs) inside the
optical modules of the detector through a library of C+ + classes. This could
be considered a massive statistical analysis of photo-electrons. Aim of this
work is the development of a multithreaded version of the SIRENE detector
simulation software for high energy neutrinos. This approach allows utilization
of multiple CPU cores leading to a potentially significant decrease in the
required execution time compared to the sequential code. We are making use of
the OpenMP framework for the production of multithreaded code running on the
CPU. Finally, we analyze the feasibility of a GPU-accelerated implementation.
Dariusz R. Kowalski, William K. Moses Jr., Shailesh Vaya
Comments: 12 pages, 1 table
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Data Structures and Algorithms (cs.DS)
For a given network, a backbone is an overlay network consisting of a
connected dominating set with additional accessibility properties. Once a
backbone is created for a network, it can be utilized for fast communication
amongst the nodes of the network.
The Signal-to-Interference-plus-Noise-Ratio (SINR) model has become the
standard for modeling communication among devices in wireless networks. For
this model, the community has pondered what the most realistic solutions for
communication problems in wireless networks would look like. Such solutions
would have the characteristic that they would make the least number of
assumptions about the availability of information about the participating
nodes. Solving problems when nothing at all is known about the network and
having nodes just start participating would be ideal. However, this is quite
challenging and most likely not feasible. The pragmatic approach is then to
make meaningful assumptions about the available information and present
efficient solutions based on this information.
We present a solution for creation of backbone in the SINR model, when nodes
do not have access to their physical coordinates or the coordinates of other
nodes in the network. This restriction models the deployment of nodes in
various situations for sensing hurricanes, cyclones, and so on, where only
information about nodes prior to their deployment may be known but not their
actual locations post deployment. We assume that nodes have access to knowledge
of their label, the labels of nodes within their neighborhood, the range from
which labels are taken ([N]) and the total number of participating nodes (n).
We also assume that nodes wake up spontaneously. We present an efficient
deterministic protocol to create a backbone with a round complexity of
(O(Delta lg^2 N)).
William K. Moses Jr., Shailesh Vaya
Comments: 37 pages, 1 table, 4 figures
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Data Structures and Algorithms (cs.DS)
Much work has been developed for studying the classical broadcasting problem
in the SINR (Signal-to-Interference-plus-Noise-Ratio) model for wireless device
transmission. The setting typically studied is when all radio nodes transmit a
signal of the same strength. This work studies the challenging problem of
devising a distributed algorithm for multi-broadcasting, assuming a subset of
nodes are initially awake, for the SINR model when each device only has access
to knowledge about the total number of nodes in the network (n), the range from
which each node’s label is taken ([1,dots,N]), and the label of the device
itself. Specifically, we assume no knowledge of the physical coordinates of
devices and also no knowledge of neighborhood of each node.
We present a deterministic protocol for the multi-broadcast problem in (O(n
lg^2 N lg n)) rounds, for this setting, assuming that only a subset of all
(n) nodes are initially awake. A lower bound of (Omega(n lg N)) rounds is
known for this, but even that assumes knowledge of physical coordinates. All
previous results in the literature are either randomized or intensively use the
physical coordinates of the nodes and/or knowledge of the labels of nodes’
neighbors.
In addition to the above result, we present algorithms to achieve
multi-broadcast in (O(n lg^2 N)) rounds and create a backbone in (O(n lg^2
N)) rounds, assuming that all nodes are initially awake. For a given backbone,
messages can be exchanged between every pair of connected nodes in the backbone
in (O(lg N)) rounds and between any node and its leader in the backbone in
(O(Delta lg N)) rounds.
Yu-Fang Chen, Chih-Duo Hong, Ondřej Lengál, Shin-Cheng Mu, Nishant Sinha, Bow-Yaw Wang
Comments: an extended version of a paper accepted at NETYS’17
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Logic in Computer Science (cs.LO)
Spark is a new promising platform for scalable data-parallel computation. It
provides several high-level application programming interfaces (APIs) to
perform parallel data aggregation. Since execution of parallel aggregation in
Spark is inherently non-deterministic, a natural requirement for Spark programs
is to give the same result for any execution on the same data set. We present
PureSpark, an executable formal Haskell specification for Spark aggregate
combinators. Our specification allows us to deduce the precise condition for
deterministic outcomes from Spark aggregation. We report case studies analyzing
deterministic outcomes and correctness of Spark programs.
Anas M. Al-Oraiqat
Comments: 8 pages, 9 Figures
Journal-ref: International Journal of advanced studies in Computer Science and
Engineering (IJASCSE) 2012
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
This research presents a model of a complex dynamic object running on a
multi-core system. Discretization and numerical integration for multibody
models of vehicle rail elements in the vertical longitudinal plane fluctuations
is considered. The implemented model and solution of the motion differential
equations allow estimating the basic processes occurring in the system with
various external influences. Hence the developed programming model can be used
for performing analysis and comparing new vehicle designs.
Keywords-dynamic model; multi-core system; SMP system; rolling stock.
Anas M. Al-Oraiqat
Comments: 7 pages, 5 Figures, International Journal of Engineering Science and Technology 2009
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
This article presents the parallel implementation of the coupled harmonic
oscillator. From the analytical solution of the coupled harmonic oscillator,
the design parameters are obtained. After that, a numerical integration of the
system with MATLAB, which is used as a tool of benchmark evaluation, is
performed. Next, parallel implementation is performed using a well-known
approach like OpenMP and WinAPI. Taking into account the errors of basic
parameters of the simulated process, the generated oscillations of the proposed
parallel realization are almost identical to the actual solution of the
harmonic oscillator model. Test ways to optimize the parallel architecture of
computing processes for software implementations of the considered application
is carried out. The developed model is used to study a fixed priority
scheduling algorithm for real-time parallel threads execution. The proposed
parallel implementation of the considered dynamic system has an independent
value and can be considered as a test for determining the characteristics of
multi-core systems for time-critical simulation problems. Keywords: Harmonic
oscillator, model, SMP, parallel programming, OpenMP;
Travis Desell, Malik Magdon-Ismail, Heidi Newberg, Lee A. Newberg, Boleslaw K. Szymanski, Carlos A. Varela
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
Volunteer computing grids offer super-computing levels of computing power at
the relatively low cost of operating a server. In previous work, the authors
have shown that it is possible to take traditionally iterative evolutionary
algorithms and execute them on volunteer computing grids by performing them
asynchronously. The asynchronous implementations dramatically increase
scalability and decrease the time taken to converge to a solution. Iterative
and asynchronous optimization algorithms implemented using MPI on clusters and
supercomputers, and BOINC on volunteer computing grids have been packaged
together in a framework for generic distributed optimization (FGDO). This paper
presents a new extension to FGDO for an asynchronous Newton method (ANM) for
local optimization. ANM is resilient to heterogeneous, faulty and unreliable
computing nodes and is extremely scalable. Preliminary results show that it can
converge to a local optimum significantly faster than conjugate gradient
descent does.
Xiangyu Guo, Qi Chu, Shin Kee Chung, Zhihui Du, Linqing Wen
Subjects: Instrumentation and Methods for Astrophysics (astro-ph.IM); Distributed, Parallel, and Cluster Computing (cs.DC)
Low-latency detections of gravitational waves (GWs) are crucial to enable
prompt follow-up observations to astrophysical transients by conventional
telescopes. We have developed a low-latency pipeline using a technique called
Summed Parallel Infinite Impulse Response (SPIIR) filtering, realized by a
Graphic Processing Unit (GPU). In this paper, we exploit the new
extit{Maxwell} memory access architecture in NVIDIA GPUs, namely the
read-only data cache, warp-shuffle, and cross-warp atomic techniques. We report
a 3-fold speed-up over our previous implementation of this filtering technique.
To tackle SPIIR with relatively few filters, we develop a new GPU thread
configuration with a nearly 10-fold speedup. In addition, we implement a
multi-rate scheme of SPIIR filtering using Maxwell GPUs. We achieve more than
100-fold speed-up over a single core CPU for the multi-rate filtering scheme.
This results in an overall of 21-fold CPU usage reduction for the entire SPIIR
pipeline.
Adrian Benton, Huda Khayrallah, Biman Gujral, Drew Reisinger, Sheng Zhang, Raman Arora
Comments: 14 pages, 6 figures
Subjects: Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
We present Deep Generalized Canonical Correlation Analysis (DGCCA) — a
method for learning nonlinear transformations of arbitrarily many views of
data, such that the resulting transformations are maximally informative of each
other. While methods for nonlinear two-view representation learning (Deep CCA,
(Andrew et al., 2013)) and linear many-view representation learning
(Generalized CCA (Horst, 1961)) exist, DGCCA is the first CCA-style multiview
representation learning technique that combines the flexibility of nonlinear
(deep) representation learning with the statistical power of incorporating
information from many independent sources, or views. We present the DGCCA
formulation as well as an efficient stochastic optimization algorithm for
solving it. We learn DGCCA repre- sentations on two distinct datasets for three
downstream tasks: phonetic transcrip- tion from acoustic and articulatory
measurements, and recommending hashtags and friends on a dataset of Twitter
users. We find that DGCCA representations soundly beat existing methods at
phonetic transcription and hashtag recommendation, and in general perform no
worse than standard linear many-view techniques.
Wenhao Yu, C. Karen Liu, Greg Turk
Subjects: Learning (cs.LG); Robotics (cs.RO); Systems and Control (cs.SY)
We present a new method of learning control policies that successfully
operate under unknown dynamic models. We create such policies by leveraging a
large number of training examples that are generated using a physical
simulator. Our system is made of two components: a Universal Policy (UP) and a
function for Online System Identification (OSI). We describe our control policy
as universal because it is trained over a wide array of dynamic models. These
variations in the dynamic model may include differences in mass and inertia of
the robots’ components, variable friction coefficients, or unknown mass of an
object to be manipulated. By training the Universal Policy with this variation,
the control policy is prepared for a wider array of possible conditions when
executed in an unknown environment. The second part of our system uses the
recent state and action history of the system to predict the dynamics model
parameters mu. The value of mu from the Online System Identification is then
provided as input to the control policy (along with the system state).
Together, UP-OSI is a robust control policy that can be used across a wide
range of dynamic models, and that is also responsive to sudden changes in the
environment. We have evaluated the performance of this system on a variety of
tasks, including the problem of cart-pole swing-up, the double inverted
pendulum, locomotion of a hopper, and block-throwing of a manipulator. UP-OSI
is effective at these tasks across a wide range of dynamic models. Moreover,
when tested with dynamic models outside of the training range, UP-OSI
outperforms the Universal Policy alone, even when UP is given the actual value
of the model dynamics. In addition to the benefits of creating more robust
controllers, UP-OSI also holds out promise of narrowing the Reality Gap between
simulated and real physical systems.
Sandy Huang, Nicolas Papernot, Ian Goodfellow, Yan Duan, Pieter Abbeel
Subjects: Learning (cs.LG); Cryptography and Security (cs.CR); Machine Learning (stat.ML)
Machine learning classifiers are known to be vulnerable to inputs maliciously
constructed by adversaries to force misclassification. Such adversarial
examples have been extensively studied in the context of computer vision
applications. In this work, we show adversarial attacks are also effective when
targeting neural network policies in reinforcement learning. Specifically, we
show existing adversarial example crafting techniques can be used to
significantly degrade test-time performance of trained policies. Our threat
model considers adversaries capable of introducing small perturbations to the
raw input of the policy. We characterize the degree of vulnerability across
tasks and training algorithms, for a subclass of adversarial-example attacks in
white-box and black-box settings. Regardless of the learned task or training
algorithm, we observe a significant drop in performance, even with small
adversarial perturbations that do not interfere with human perception. Videos
are available at this http URL
Quang N. Tran, Ba-Ngu Vo, Dinh Phung, Ba-Tuong Vo
Comments: Preprint: 23rd Int. Conf. Pattern Recognition (ICPR). Cancun, Mexico, December 2016
Subjects: Learning (cs.LG); Machine Learning (stat.ML)
Clustering is one of the most common unsupervised learning tasks in machine
learning and data mining. Clustering algorithms have been used in a plethora of
applications across several scientific fields. However, there has been limited
research in the clustering of point patterns – sets or multi-sets of unordered
elements – that are found in numerous applications and data sources. In this
paper, we propose two approaches for clustering point patterns. The first is a
non-parametric method based on novel distances for sets. The second is a
model-based approach, formulated via random finite set theory, and solved by
the Expectation-Maximization algorithm. Numerical experiments show that the
proposed methods perform well on both simulated and real data.
Sri Ramana Sekharan, Ramkumar Natarajan, Siddharthan Rajasekaran
Comments: 8 pages, 3 algorithms, 3 figures
Subjects: Learning (cs.LG)
In this paper, we tackle the problem of transferring policy from multiple
partially observable source environments to a partially observable target
environment modeled as predictive state representation. This is an entirely new
approach with no previous work, other than the case of transfer in fully
observable domains. We develop algorithms to successfully achieve policy
transfer when we have the model of both the source and target tasks and discuss
in detail their performance and shortcomings. These algorithms could be a
starting point for the field of transfer learning in partial observability.
Jarek Duda
Comments: 6 pages, 2 figures
Subjects: Learning (cs.LG)
Parametric density estimation, for example as Gaussian distribution, is the
base of the field of statistics. Machine learning requires inexpensive
estimation of much more complex densities, and the basic approach is relatively
costly maximum likelihood estimation (MLE). There will be discussed inexpensive
density estimators, for example literally fitting polynomial to the sample,
which coefficients are calculated from just averaging monomials over the sample
(estimators of moments). Another discussed basic application is fitting
distortion to some standard distribution like Gaussian. The estimated
parameters are approaching the optimal values with error dropping like
(1/sqrt{n}), where (n) is the sample size.
Matt Parker, Colin Parker
Comments: 6 pages
Subjects: Machine Learning (stat.ML); Learning (cs.LG)
The classical construction of a Support Vector Classifier as the classifier
which iden- tifies a hyperplane maximizing the margin between two classes,
given some constraint on slack variables, works well in many common situations.
However when there is a difference between classes in their respective
variances perpendicular to this hyper- plane, the SVC ends up giving the class
with lower variance perpendicular to it an unjustifiably wide berth, while
comparatively tightening up on the high-variance class, resulting in a loss to
predictive performance. This paper outlines an alternate con- struction, which
seeks to adjust the identified hyperplane in such a way that it agrees with the
SVC in the event of a class variance balance along the direction perpendicular
to the optimal hyperplane, and to examine the impact to the dual representation
of the modified constraint equations.
Lukas Machlica, Karel Bartos, Michal Sofka
Subjects: Machine Learning (stat.ML); Learning (cs.LG)
This paper proposes a generic classification system designed to detect
security threats based on the behavior of malware samples. The system relies on
statistical features computed from proxy log fields to train detectors using a
database of malware samples. The behavior detectors serve as basic reusable
building blocks of the multi-level detection architecture. The detectors
identify malicious communication exploiting encrypted URL strings and domains
generated by a Domain Generation Algorithm (DGA) which are frequently used in
Command and Control (C&C), phishing, and click fraud. Surprisingly, very
precise detectors can be built given only a limited amount of information
extracted from a single proxy log. This way, the computational requirements of
the detectors are kept low which allows for deployment on a wide range of
security devices and without depending on traffic context such as DNS logs,
Whois records, webpage content, etc. Results on several weeks of live traffic
from 100+ companies having 350k+ hosts show correct detection with a precision
exceeding 95% of malicious flows, 95% of malicious URLs and 90% of infected
hosts. In addition, a comparison with a signature and rule-based solution shows
that our system is able to detect significant amount of new threats.
Michael Kampffmeyer, Sigurd Løkse, Filippo Maria Bianchi, Robert Jenssen, Lorenzo Livi
Subjects: Machine Learning (stat.ML); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
In this paper we introduce the deep kernelized autoencoder, a neural network
model that allows an explicit approximation of (i) the mapping from an input
space to an arbitrary, user-specified kernel space and (ii) the back-projection
from such a kernel space to input space. The proposed method is based on
traditional autoencoders and is trained through a new unsupervised loss
function. During training, we optimize both the reconstruction accuracy of
input samples and the alignment between a kernel matrix given as prior and the
inner products of the hidden representations computed by the autoencoder.
Kernel alignment provides control over the hidden representation learned by the
autoencoder. Experiments have been performed to evaluate both reconstruction
and kernel alignment performance. Additionally, we applied our method to
emulate kPCA on a denoising task obtaining promising results.
Ziwei Liu, Raymond Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala
Comments: Project page: this https URL
Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Learning (cs.LG)
We address the problem of synthesizing new video frames in an existing video,
either in-between existing frames (interpolation), or subsequent to them
(extrapolation). This problem is challenging because video appearance and
motion can be highly complex. Traditional optical-flow-based solutions often
fail where flow estimation is challenging, while newer neural-network-based
methods that hallucinate pixel values directly often produce blurry results. We
combine the advantages of these two methods by training a deep network that
learns to synthesize video frames by flowing pixel values from existing ones,
which we call deep voxel flow. Our method requires no human supervision, and
any video can be used as training data by dropping, and then learning to
predict, existing frames. The technique is efficient, and can be applied at any
video resolution. We demonstrate that our method produces results that both
quantitatively and qualitatively improve upon the state-of-the-art.
Jiatao Gu, Kyunghyun Cho, Victor O.K. Li
Comments: 10 pages
Subjects: Computation and Language (cs.CL); Learning (cs.LG)
Recent research in neural machine translation has largely focused on two
aspects; neural network architectures and end-to-end learning algorithms. The
problem of decoding, however, has received relatively little attention from the
research community. In this paper, we solely focus on the problem of decoding
given a trained neural machine translation model. Instead of trying to build a
new decoding algorithm for any specific decoding objective, we propose the idea
of trainable decoding algorithm in which we train a decoding algorithm to find
a translation that maximizes an arbitrary decoding objective. More
specifically, we design an actor that observes and manipulates the hidden state
of the neural machine translation decoder and propose to train it using a
variant of deterministic policy gradient. We extensively evaluate the proposed
algorithm using four language pairs and two decoding objectives and show that
we can indeed train a trainable greedy decoder that generates a better
translation (in terms of a target decoding objective) with minimal
computational overhead.
Sebastian Ruder, Parsa Ghaffari, John G. Breslin
Comments: 10 pages, 2 figures, 4 tables
Subjects: Computation and Language (cs.CL); Learning (cs.LG)
Domain adaptation is important in sentiment analysis as sentiment-indicating
words vary between domains. Recently, multi-domain adaptation has become more
pervasive, but existing approaches train on all available source domains
including dissimilar ones. However, the selection of appropriate training data
is as important as the choice of algorithm. We undertake — to our knowledge
for the first time — an extensive study of domain similarity metrics in the
context of sentiment analysis and propose novel representations, metrics, and a
new scope for data selection. We evaluate the proposed methods on two
large-scale multi-domain adaptation settings on tweets and reviews and
demonstrate that they consistently outperform strong random and balanced
baselines, while our proposed selection strategy outperforms instance-level
selection and yields the best score on a large reviews corpus.
David Gamarnik, Quan Li, Hongyi Zhang
Comments: 43 pages, 1 figure
Subjects: Machine Learning (stat.ML); Data Structures and Algorithms (cs.DS); Learning (cs.LG); Optimization and Control (math.OC)
We consider the problem of reconstructing a rank-(k) (n imes n) matrix (M)
from a sampling of its entries. Under a certain incoherence assumption on (M)
and for the case when both the rank and the condition number of (M) are
bounded, it was shown in cite{CandesRecht2009, CandesTao2010, keshavan2010,
Recht2011, Jain2012, Hardt2014} that (M) can be recovered exactly or
approximately (depending on some trade-off between accuracy and computational
complexity) using (O(n , ext{poly}(log n))) samples in super-linear time
(O(n^{a} , ext{poly}(log n))) for some constant (a geq 1).
In this paper, we propose a new matrix completion algorithm using a novel
sampling scheme based on a union of independent sparse random regular bipartite
graphs. We show that under the same conditions w.h.p. our algorithm recovers an
(epsilon)-approximation of (M) in terms of the Frobenius norm using (O(n
log^2(1/epsilon))) samples and in linear time (O(n log^2(1/epsilon))). This
provides the best known bound on the sample complexity and computational cost
for reconstructing an unknown low-rank matrix.
The novelty of our algorithm is a new step of thresholding singular values
and rescaling singular vectors in the application of the “vanilla” alternating
minimization algorithm. The structure of sparse random regular graphs is used
heavily for controlling the impact of these regularization steps.
Cormac Dullaghan, Eleni Rozaki
Comments: 12 pages
Journal-ref: International Journal of Data Mining & Knowledge Management
Process (IJDKP) Vol.7, No.1, January 2017
Subjects: Computers and Society (cs.CY); Learning (cs.LG); Machine Learning (stat.ML)
The telecommunications industry is highly competitive, which means that the
mobile providers need a business intelligence model that can be used to achieve
an optimal level of churners, as well as a minimal level of cost in marketing
activities. Machine learning applications can be used to provide guidance on
marketing strategies. Furthermore, data mining techniques can be used in the
process of customer segmentation. The purpose of this paper is to provide a
detailed analysis of the C.5 algorithm, within naive Bayesian modelling for the
task of segmenting telecommunication customers behavioural profiling according
to their billing and socio-demographic aspects. Results have been
experimentally implemented.
Zhilin Yang, Junjie Hu, Ruslan Salakhutdinov, William W. Cohen
Subjects: Computation and Language (cs.CL); Learning (cs.LG)
We study the problem of semi-supervised question answering—-utilizing
unlabeled text to boost the performance of question answering models. We
propose a novel training framework, the Generative Domain-Adaptive Nets. In
this framework, we train a generative model to generate questions based on the
unlabeled text, and combine model-generated questions with human-generated
questions for training question answering models. We develop novel domain
adaptation algorithms, based on reinforcement learning, to alleviate the
discrepancy between the model-generated data distribution and the
human-generated data distribution. Experiments show that our proposed framework
obtains substantial improvement from unlabeled text.
Moshe Looks, Marcello Herreshoff, DeLesley Hutchins, Peter Norvig
Comments: Published as a conference paper at ICLR 2017
Subjects: Neural and Evolutionary Computing (cs.NE); Learning (cs.LG); Machine Learning (stat.ML)
Neural networks that compute over graph structures are a natural fit for
problems in a variety of domains, including natural language (parse trees) and
cheminformatics (molecular graphs). However, since the computation graph has a
different shape and size for every input, such networks do not directly support
batched training or inference. They are also difficult to implement in popular
deep learning libraries, which are based on static data-flow graphs. We
introduce a technique called dynamic batching, which not only batches together
operations between different input graphs of dissimilar shape, but also between
different nodes within a single input graph. The technique allows us to create
static graphs, using popular libraries, that emulate dynamic computation graphs
of arbitrary shape and size. We further present a high-level library of
compositional blocks that simplifies the creation of dynamic graph models.
Using the library, we demonstrate concise and batch-wise parallel
implementations for a variety of models from the literature.
Lennart Gulikers, Marc Lelarge, Laurent Massoulié
Subjects: Probability (math.PR); Learning (cs.LG); Social and Information Networks (cs.SI); Machine Learning (stat.ML)
We consider community detection in Degree-Corrected Stochastic Block Models
(DC-SBM). We propose a spectral clustering algorithm based on a suitably
normalized adjacency matrix. We show that this algorithm consistently recovers
the block-membership of all but a vanishing fraction of nodes, in the regime
where the lowest degree is of order log((n)) or higher. Recovery succeeds even
for very heterogeneous degree-distributions. The used algorithm does not rely
on parameters as input. In particular, it does not need to know the number of
communities.
Tedros Salih, Elijah Mwangi, Kibet Langat
Comments: 18 pages, 6 figures
Subjects: Information Theory (cs.IT)
The fundamental limitation of massive MIMO technology is pilot contamination
effect. This effect occurs during uplink training when terminals use the same
orthogonal signals. In this paper, a pilot reuse factor with large scale fading
precoding is proposed to mitigate the pilot contamination effect. The pilot
reuse factor is designed to assign unique orthogonal signals to the adjacent
cells. These unique orthogonal signals are reused only within the cell and
hence, intra-pilot contamination is the only concern. Large scale fading
precoding is then used to mitigate the intra-pilot contamination effect. The
average achievable sum rate is computed for different pilot reuse factors.
Experimental results through MATLAB simulation show that a higher pilot reuse
factor gives better average achievable sum rates.
Dalin Zhu, Junil Choi, Robert W. Heath Jr
Comments: Submitted to IEEE Transactions on Wireless Communications
Subjects: Information Theory (cs.IT)
In this paper, a novel two-dimensional super-resolution angle-of-departure
(AoD) and angle-of-arrival (AoA) estimation technique is proposed for wideband
millimeter-wave multiple-input multiple-output systems with cross-polarized
antenna elements. The key ingredient of the proposed method is to form custom
designed beam pairs, and devise an invertible function of the AoD/AoA to be
estimated from the corresponding beam pairs. Further, a new multi-layer
reference signal structure is developed for the proposed method to facilitate
angle estimation for wideband channels with cross-polarized antenna elements.
To facilitate feedback in closed-loop frequency division duplexing systems, a
novel differential feedback strategy is proposed aiming at feedback reduction
for the two-dimensional angle estimation. Numerical results demonstrate that by
using the proposed method, good azimuth/elevation AoD and AoA estimation
performance can be achieved under different levels of signal-to-noise ratio,
channel conditions, and antenna array configurations.
Markus Staudacher, Gerhard Kramer, Wolfgang Zirwas, Berthold Panzner, Rakash Sivasiva Ganesan
Subjects: Information Theory (cs.IT)
A low cost solution for constructing receiver signal points is investigated
that combines a large number of constrained radio frequency (RF) frontends with
a limited number of full RF chains. The constrained RF front ends have low cost
and are limited to on/off switching of antenna elements and a small number of
phases. Severe degradations are typically observed for multi-user MIMO for
these simple on/off antenna arrays. A few full RF frontends are shown to
compensate for the signal errors of the high number of constrained RF frontends
for various scenarios. An algorithm for such a hybrid RF (HRF) system is
developed that achieves performance close to that of exhaustive search with
respect to the mean square error of the constructed receiver signals for
Rayleigh fading and the WINNER 2 Urban Macro channel model.
Anurag Anshu, Rahul Jain, Naqueeb Ahmad Warsi
Comments: 21 pages, version 1. arXiv admin note: text overlap with arXiv:1410.3031
Subjects: Information Theory (cs.IT); Quantum Physics (quant-ph)
We study the problem of state redistribution both in the classical (shared
randomness assisted) and quantum (entanglement assisted) one-shot settings and
provide new upper bounds on the communication required. Our bounds are in terms
of smooth-min and max relative entropies. We also consider a special case of
this problem in the classical setting, previously studied by Braverman and Rao
(2011). We show that their upper bound is optimal. In addition we provide an
alternate protocol achieving a priory different looking upper bound. However,
we show that our upper bound is essentially the same as their upper bound and
hence also optimal.
Valeriya Potapova, Alexey Frolov
Subjects: Information Theory (cs.IT)
We address the problem of constructing of coding schemes for the channels
with high-order modulations. It is known, that non-binary LDPC codes are
especially good for such channels and significantly outperform their binary
counterparts. Unfortunately, their decoding complexity is still large. In order
to reduce the decoding complexity we consider multilevel coding schemes based
on non-binary LDPC codes (NB-LDPC-MLC schemes) over smaller fields. The use of
such schemes gives us a reasonable gain in complexity. At the same time the
performance of NB-LDPC-MLC schemes is practically the same as the performance
of LDPC codes over the field matching the modulation order. In particular by
means of simulations we showed that the performance of NB-LDPC-MLC schemes over
GF(16) is the same as the performance of non-binary LDPC codes over GF(64) and
GF(256) in AWGN channel with QAM64 and QAM256 accordingly. We also perform a
comparison with binary LDPC codes.
A. Saci, A. Al-Dweik, A. Shami
Subjects: Information Theory (cs.IT)
In the literature, channel estimation and synchronization (CE/SY) algorithms
are classified as blind, and hence spectrally efficient, if they do not require
pilot symbols. However, we show in this letter that such classification is not
accurate and can be misleading. Consequently, this letter presents a more
reliable and accurate approach to evaluate the spectral efficiency of
communications systems with various CE/SY algorithms. The proposed approach
allows fair spectral efficiency comparison between various systems with blind
or non-blind CE/SY algorithms. In particular, we evaluate the spectral
efficiency of communications systems that incorporates blind CE/SY algorithms
and compare it to other blind and pilot-based algorithms. The obtained results
reveal that, on the contrary to what is widely accepted, blind CE/SY algorithms
with modulation-type constrain do not necessarily improve the spectral
efficiency as compared to pilot-based techniques. Consequently, such techniques
are classified as conditionally blind, to distinguish them from fully blind
techniques.
Ori Shmuel, Asaf Cohen, Omer Gurewitz
Subjects: Information Theory (cs.IT)
Compute and Forward (CF) is a promising relaying scheme which, instead of
decoding single messages or forwarding/amplifying information at the relay,
decodes linear combinations of the simultaneously transmitted messages. The
current literature includes several coding schemes and results on the degrees
of freedom in CF, yet for systems with a fixed number of transmitters and
receivers.
It is unclear, however, how CF behaves at the limit of a large number of
transmitters. In this paper, we investigate the performance of CF in that
regime. Specifically, we show that as the number of transmitters grows, CF
becomes degenerated, in the sense that a relay prefers to decode only one
(strongest) user instead of any other linear combination of the transmitted
codewords, treating the other users as noise. Moreover, the sum-rate tends to
zero as well. This makes scheduling necessary in order to maintain the superior
abilities CF provides. Indeed, under scheduling, we show that non-trivial
linear combinations are chosen, and the sum-rate does not decay, even without
state information at the transmitters and without interference alignment.
Mohsen Heidari, Farhad Shirani, S. Sandeep Pradhan
Comments: 13 pages, ISIT 2017
Subjects: Information Theory (cs.IT)
The problem of reliable communication over the multiple-access channel (MAC)
with states is investigated. We propose a new coding scheme for this problem
which uses quasi-group codes (QGC). We derive a new computable single-letter
characterization of the achievable rate region. As an example, we investigate
the problem of doubly-dirty MAC with modulo-(4) addition. It is shown that the
sum-rate (R_1+R_2=1) bits per channel use is achievable using the new scheme.
Whereas, the natural extension of the Gel’fand-Pinsker scheme, sum-rates
greater than (0.32) are not achievable.
Ahmed El Alaoui, Aaditya Ramdas, Florent krzakala, Lenka Zdeborova, Michael I. Jordan
Subjects: Information Theory (cs.IT); Data Structures and Algorithms (cs.DS)
We consider the problem of decoding a discrete signal of categorical
variables from the observation of several histograms of pooled subsets of it.
We present an Approximate Message Passing (AMP) algorithm for recovering the
signal in the random dense setting where each observed histogram involves a
random subset of entries of size proportional to n. We characterize the
performance of the algorithm in the asymptotic regime where the number of
observations (m) tends to infinity proportionally to n, by deriving the
corresponding State Evolution (SE) equations and studying their dynamics. We
initiate the analysis of the multi-dimensional SE dynamics by proving their
convergence to a fixed point, along with some further properties of the
iterates. The analysis reveals sharp phase transition phenomena where the
behavior of AMP changes from exact recovery to weak correlation with the signal
as m/n crosses a threshold. We derive formulae for the threshold in some
special cases and show that they accurately match experimental behavior.
Asma Ghorbel, Khac-Hoang Ngo, Richard Combes, Mari Kobayashi, Sheng Yang
Comments: 7 pages, 2 figures, extended version of a paper submitted to ISIT 2017
Subjects: Information Theory (cs.IT)
We consider content delivery over fading broadcast channels. A server wants
to transmit K files to K users, each equipped with a cache of finite size.
Using the coded caching scheme of Maddah-Ali and Niesen, we design an
opportunistic delivery scheme where the long-term sum content delivery rate
scales with K the number of users in the system. The proposed delivery scheme
combines superposition coding together with appropriate power allocation across
sub-files intended to different subsets of users. We analyze the long-term
average sum content delivery rate achieved by two special cases of our scheme:
a) a selection scheme that chooses the subset of users with the largest
weighted rate, and b) a baseline scheme that transmits to K users using the
scheme of Maddah-Ali and Niesen. We prove that coded caching with appropriate
user selection is scalable since it yields a linear increase of the average sum
content delivery rate.
Jafar Banar, S. Mohammad Razavizadeh
Comments: 21 pages, 7 figures, Elsevier journal. arXiv admin note: text overlap with arXiv:1611.02552
Subjects: Information Theory (cs.IT)
This paper is on relay selection in uplink of an in-band full-duplex (IBFD)
cooperative cellular network. Assuming an orthogonal frequency division
multiple access (OFDMA) cellular network, we develop a relay selection and
resource allocation algorithm for this network. The relay selection algorithms
select the best relay based on distance between users and signal to
interference plus noise ratio (SINR) that operate in amplify and forward (AF)
mode. The optimization problem allocates the optimum subcarriers and powers to
all users to maximize the average sum-rate of the network. In addition, the
constraints of the optimization problem are quality of service (QoS) and
transmit power of each user. Simulation results illustrate the good performance
of the proposed method.
Jalaluddin Qureshi
Comments: This paper appears in: Communications (APCC), 2016 22nd Asia-Pacific Conference on
Subjects: Networking and Internet Architecture (cs.NI); Information Theory (cs.IT)
In this paper we study the problem of increasing the decoding success
probability of random linear fountain code over GF(2) for small packet lengths
used in delay-intolerant applications such as multimedia streaming. Such code
over GF(2) are attractive as they have lower decoding complexity than codes
over larger field size, but suffer from high transmission redundancy. In our
proposed coding scheme we construct a codeword which is not a linear
combination of any codewords previously transmitted to mitigate such
transmission redundancy. We then note the observation that the probability of
receiving a linearly dependent codeword is highest when the receiver has
received k-1 linearly independent codewords. We propose using the BlockACK
frame so that the codeword received after k-1 linearly independent codeword is
always linearly independent, this reduces the expected redundancy by a factor
of three.
Farid Shahandeh, Austin P. Lund, Timothy C. Ralph
Comments: 5 pages, 1 figure, comments are very welcomed
Subjects: Quantum Physics (quant-ph); Information Theory (cs.IT)
Determination of the quantum nature of correlations between two spatially
separated systems plays a crucial role in quantum information science. Of
particular interest is the questions of if and how these correlations enable
quantum information protocols to be more powerful. Here, we report on a
distributed quantum computation protocol in which the input and output quantum
states are considered to be classically correlated in quantum informatics.
Nevertheless, we show that the correlations between the outcomes of the
measurements on the output state cannot be efficiently simulated using
classical algorithms. Crucially, at the same time, local measurement outcomes
can be efficiently simulated on classical computers. We show that the only
known classicality criterion violated by the input and output states in our
protocol is the one used in quantum optics, namely, phase-space
nonclassicality. As a result, we argue that the global phase-space
nonclassicality inherent within the output state of our protocol represents
true quantum correlations.