Filippo Maria Bianchi, Michael Kampffmeyer, Enrico Maiorino, Robert Jenssen
Subjects: Neural and Evolutionary Computing (cs.NE); Dynamical Systems (math.DS)
In this work we present a novel recurrent neural network architecture
designed to model systems characterized by multiple characteristic timescales
in their dynamics. The proposed network is composed by several recurrent groups
of neurons that are trained to separately adapt to each timescale, in order to
improve the system identification process. We test our framework on time series
prediction tasks and we show some promising, preliminary results achieved on
synthetic data. To evaluate the capabilities of our network, we compare the
performance with several state-of-the-art recurrent architectures.
Keyan Ghazi-Zahedi
Subjects: Neural and Evolutionary Computing (cs.NE); Robotics (cs.RO)
Modularisation, repetition, and symmetry are structural features shared by
almost all biological neural networks. These features are very unlikely to be
found by the means of structural evolution of artificial neural networks. This
paper introduces NMODE, which is specifically designed to operate on
neuro-modules. NMODE addresses a second problem in the context of evolutionary
robotics, which is incremental evolution of complex behaviours for complex
machines, by offering a way to interface neuro-modules. The scenario in mind is
a complex walking machine, for which a locomotion module is evolved first, that
is then extended by other modules in later stages. We show that NMODE is able
to evolve a locomotion behaviour for a standard six-legged walking machine in
approximately 10 generations and show how it can be used for incremental
evolution of a complex walking machine. The entire source code used in this
paper is publicly available through GitHub.
Volodymyr Turchenko, Eric Chalmers, Artur Luczak
Comments: 21 pages, 11 figures, 5 tables, 62 references
Subjects: Neural and Evolutionary Computing (cs.NE); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG)
This paper presents the development of several models of a deep convolutional
auto-encoder in the Caffe deep learning framework and their experimental
evaluation on the example of MNIST dataset. We have created five models of a
convolutional auto-encoder which differ architecturally by the presence or
absence of pooling and unpooling layers in the auto-encoder’s encoder and
decoder parts. Our results show that the developed models provide very good
results in dimensionality reduction and unsupervised clustering tasks, and
small classification errors when we used the learned internal code as an input
of a supervised linear classifier and multi-layer perceptron. The best results
were provided by a model where the encoder part contains convolutional and
pooling layers, followed by an analogous decoder part with deconvolution and
unpooling layers without the use of switch variables in the decoder part. The
paper also discusses practical details of the creation of a deep convolutional
auto-encoder in the very popular Caffe deep learning framework. We believe that
our approach and results presented in this paper could help other researchers
to build efficient deep neural network architectures in the future.
Ludvig Ericson, Rendani Mbuvha
Comments: 4 Pages, 4 Figures, 1 Table
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Performance (cs.PF); Machine Learning (stat.ML)
Artificial Neural Networks (ANNs) have received increasing attention in
recent years with applications that span a wide range of disciplines including
vital domains such as medicine, network security and autonomous transportation.
However, neural network architectures are becoming increasingly complex and
with an increasing need to obtain real-time results from such models, it has
become pivotal to use parallelization as a mechanism for speeding up network
training and deployment. In this work we propose an implementation of Network
Parallel Training through Cannon’s Algorithm for matrix multiplication. We show
that increasing the number of processes speeds up training until the point
where process communication costs become prohibitive; this point varies by
network complexity. We also show through empirical efficiency calculations that
the speedup obtained is superlinear.
Ardavan Salehi Nobandegani, Thomas R. Shultz
Subjects: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Humans are not only adept in recognizing what class an input instance belongs
to (i.e., classification task), but perhaps more remarkably, they can imagine
(i.e., generate) plausible instances of a desired class with ease, when
prompted. Inspired by this, we propose a framework which allows transforming
Cascade-Correlation Neural Networks (CCNNs) into probabilistic generative
models, thereby enabling CCNNs to generate samples from a category of interest.
CCNNs are a well-known class of deterministic, discriminative NNs, which
autonomously construct their topology, and have been successful in giving
accounts for a variety of psychological phenomena. Our proposed framework is
based on a Markov Chain Monte Carlo (MCMC) method, called the
Metropolis-adjusted Langevin algorithm, which capitalizes on the gradient
information of the target distribution to direct its explorations towards
regions of high probability, thereby achieving good mixing properties. Through
extensive simulations, we demonstrate the efficacy of our proposed framework.
Zetao Chen, Adam Jacobson, Niko Sunderhauf, Ben Upcroft, Lingqiao Liu, Chunhua Shen, Ian Reid, Michael Milford
Comments: 8 pages, 10 figures. Accepted by International Conference on Robotics and Automation (ICRA) 2017. This is the submitted version. The final published version may be slightly different
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
The success of deep learning techniques in the computer vision domain has
triggered a range of initial investigations into their utility for visual place
recognition, all using generic features from networks that were trained for
other types of recognition tasks. In this paper, we train, at large scale, two
CNN architectures for the specific place recognition task and employ a
multi-scale feature encoding method to generate condition- and
viewpoint-invariant features. To enable this training to occur, we have
developed a massive Specific PlacEs Dataset (SPED) with hundreds of examples of
place appearance change at thousands of different places, as opposed to the
semantic place type datasets currently available. This new dataset enables us
to set up a training regime that interprets place recognition as a
classification problem. We comprehensively evaluate our trained networks on
several challenging benchmark place recognition datasets and demonstrate that
they achieve an average 10% increase in performance over other place
recognition algorithms and pre-trained CNNs. By analyzing the network responses
and their differences from pre-trained networks, we provide insights into what
a network learns when training for place recognition, and what these results
signify for future research in this area.
Tony Lindeberg
Comments: 44 pages, 15 figures, 10 tables in Journal of Mathematical Imaging and Vision, 2017
Subjects: Computer Vision and Pattern Recognition (cs.CV)
When designing and developing scale selection mechanisms for generating
hypotheses about characteristic scales in signals, it is essential that the
selected scale levels reflect the extent of the underlying structures in the
signal.
This paper presents a theory and in-depth theoretical analysis about the
scale selection properties of methods for automatically selecting local
temporal scales in time-dependent signals based on local extrema over temporal
scales of scale-normalized temporal derivative responses. Specifically, this
paper develops a novel theoretical framework for performing such temporal scale
selection over a time-causal and time-recursive temporal domain as is necessary
when processing continuous video or audio streams in real time or when
modelling biological perception.
For a recently developed time-causal and time-recursive scale-space concept
defined by convolution with a scale-invariant limit kernel, we show that it is
possible to transfer a large number of the desirable scale selection properties
that hold for the Gaussian scale-space concept over a non-causal temporal
domain to this temporal scale-space concept over a truly time-causal domain.
Specifically, we show that for this temporal scale-space concept, it is
possible to achieve true temporal scale invariance although the temporal scale
levels have to be discrete, which is a novel theoretical construction.
Ni Chen, Zhenbo Ren, Dayan Li, Edmund Y. Lam, Guohai Situ
Subjects: Computer Vision and Pattern Recognition (cs.CV); Optics (physics.optics)
Light field reconstruction from images captured by focal plane sweeping can
achieve high lateral resolution comparable to the modern camera sensor. This is
impossible for the conventional micro-lenslet based light field capture
systems. However, the severe defocus noise and the low depth resolution limit
its applications. In this paper, we analyze the defocus noise and the depth
resolution in the focal plane sweeping based light field reconstruction
technique, and propose a method to reduce the defocus noise and improve the
depth resolution. Both numerical and experimental results verify the proposed
method.
M. A. Khorsandi, N. Karimi, S. Samavi
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Radon transform is a type of transform which is used in image processing to
transfer the image into intercept-slope coordinate. Its diagonal properties
made it appropriate for some applications which need processes in different
degrees. Radon transform computation needs a lot of arithmetic operations which
makes it a compute-intensive algorithm. In literature an approximate algorithm
for computing Radon transform is introduces which reduces the complexity of
computations. But this algorithm is complex and need arbitrary accesses to
memory. In this paper we proposed an algorithm which accesses to memory
sequentially. In the following an architecture is introduced which uses
pipeline to reduce the time complexity of algorithm.
Veronika Cheplygina, Isabel Pino Peña, Jesper Holst Pedersen, David A. Lynch, Lauge Sørensen, Marleen de Bruijne
Comments: Under review
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Chronic obstructive pulmonary disease (COPD) is a lung disease which can be
quantified using chest computed tomography (CT) scans. Recent studies have
shown that COPD can be automatically diagnosed using weakly supervised learning
of intensity and texture distributions. However, up till now such classifiers
have only been evaluated on scans from a single domain, and it is unclear
whether they would generalize across domains, such as different scanners or
scanning protocols. To address this problem, we investigate classification of
COPD in a multi-center dataset with a total of 803 scans from three different
centers, four different scanners, with heterogenous subject distributions. Our
method is based on Gaussian texture features, and a weighted logistic
classifier, which increases the weights of samples similar to the test data. We
show that Gaussian texture features outperform intensity features previously
used in multi-center classification tasks. We also show that a weighting
strategy based on a classifier that is trained to discriminate between scans
from different domains, can further improve the results. To encourage further
research into transfer learning methods for classification of COPD, upon
acceptance of the paper we will release two feature datasets used in this study
on this http URL
Yang Wang, Xuemin Lin, Lin Wu, Wenjie Zhang
Comments: Accepted to Appear in IEEE Trans on Image Processing
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Given a query photo issued by a user (q-user), the landmark retrieval is to
return a set of photos with their landmarks similar to those of the query,
while the existing studies on the landmark retrieval focus on exploiting
geometries of landmarks for similarity matches between candidate photos and a
query photo. We observe that the same landmarks provided by different users
over social media community may convey different geometry information depending
on the viewpoints and/or angles, and may subsequently yield very different
results. In fact, dealing with the landmarks with illshapes caused by the
photography of q-users is often nontrivial and has seldom been studied. In this
paper we propose a novel framework, namely multi-query expansions, to retrieve
semantically robust landmarks by two steps. Firstly, we identify the top-(k)
photos regarding the latent topics of a query landmark to construct multi-query
set so as to remedy its possible illshape. For this purpose, we significantly
extend the techniques of Latent Dirichlet Allocation. Then, motivated by the
typical emph{collaborative filtering} methods, we propose to learn a
emph{collaborative} deep networks based semantically, nonlinear and high-level
features over the latent factor for landmark photo as the training set, which
is formed by matrix factorization over emph{collaborative} user-photo matrix
regarding the multi-query set. The learned deep network is further applied to
generate the features for all the other photos, meanwhile resulting into a
compact multi-query set within such space. Extensive experiments are conducted
on real-world social media data with both landmark photos together with their
user information to show the superior performance over the existing methods.
Bhautik Joshi, Kristen Stewart, David Shapiro
Comments: 3 pages, 6 figures, paper is a case study of how Neural Style Transfer can be used in a movie production context
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Neural Style Transfer is a striking, recently-developed technique that uses
neural networks to artistically redraw an image in the style of a source style
image. This paper explores the use of this technique in a production setting,
applying Neural Style Transfer to redraw key scenes in ‘Come Swim’ in the style
of the impressionistic painting that inspired the film. We document how the
technique can be driven within the framework of an iterative creative process
to achieve a desired look, and propose a mapping of the broad parameter space
to a key set of creative controls. We hope that this mapping can provide
insights into priorities for future research.
Vijay Chandrasekhar, Jie Lin, Qianli Liao, Olivier Morère, Antoine Veillard, Lingyu Duan, Tomaso Poggio
Comments: 10 pages, accepted by DCC 2017
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Image instance retrieval is the problem of retrieving images from a database
which contain the same object. Convolutional Neural Network (CNN) based
descriptors are becoming the dominant approach for generating {it global image
descriptors} for the instance retrieval problem. One major drawback of
CNN-based {it global descriptors} is that uncompressed deep neural network
models require hundreds of megabytes of storage making them inconvenient to
deploy in mobile applications or in custom hardware. In this work, we study the
problem of neural network model compression focusing on the image instance
retrieval task. We study quantization, coding, pruning and weight sharing
techniques for reducing model size for the instance retrieval problem. We
provide extensive experimental results on the trade-off between retrieval
performance and model size for different types of networks on several data sets
providing the most comprehensive study on this topic. We compress models to the
order of a few MBs: two orders of magnitude smaller than the uncompressed
models while achieving negligible loss in retrieval performance.
Forrester Cole, David Belanger, Dilip Krishnan, Aaron Sarna, Inbar Mosseri, William T. Freeman
Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (stat.ML)
We present a method for synthesizing a frontal, neutral-expression image of a
person’s face given an input face photograph. This is achieved by learning to
generate facial landmarks and textures from features extracted from a
facial-recognition network. Unlike previous approaches, our encoding feature
vector is largely invariant to lighting, pose, and facial expression.
Exploiting this invariance, we train our decoder network using only frontal,
neutral-expression photographs. Since these photographs are well aligned, we
can decompose them into a sparse set of landmark points and aligned texture
maps. The decoder then predicts landmarks and textures independently and
combines them using a differentiable image warping operation. The resulting
images can be used for a number of applications, such as analyzing facial
attributes, exposure and white balance adjustment, or creating a 3-D avatar.
Volodymyr Turchenko, Eric Chalmers, Artur Luczak
Comments: 21 pages, 11 figures, 5 tables, 62 references
Subjects: Neural and Evolutionary Computing (cs.NE); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG)
This paper presents the development of several models of a deep convolutional
auto-encoder in the Caffe deep learning framework and their experimental
evaluation on the example of MNIST dataset. We have created five models of a
convolutional auto-encoder which differ architecturally by the presence or
absence of pooling and unpooling layers in the auto-encoder’s encoder and
decoder parts. Our results show that the developed models provide very good
results in dimensionality reduction and unsupervised clustering tasks, and
small classification errors when we used the learned internal code as an input
of a supervised linear classifier and multi-layer perceptron. The best results
were provided by a model where the encoder part contains convolutional and
pooling layers, followed by an analogous decoder part with deconvolution and
unpooling layers without the use of switch variables in the decoder part. The
paper also discusses practical details of the creation of a deep convolutional
auto-encoder in the very popular Caffe deep learning framework. We believe that
our approach and results presented in this paper could help other researchers
to build efficient deep neural network architectures in the future.
Fahimeh Rezazadegan, Sareh Shirazi, Ben Upcroft, Michael Milford
Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
Deep learning models have achieved state-of-the- art performance in
recognizing human activities, but often rely on utilizing background cues
present in typical computer vision datasets that predominantly have a
stationary camera. If these models are to be employed by autonomous robots in
real world environments, they must be adapted to perform independently of
background cues and camera motion effects. To address these challenges, we
propose a new method that firstly generates generic action region proposals
with good potential to locate one human action in unconstrained videos
regardless of camera motion and then uses action proposals to extract and
classify effective shape and motion features by a ConvNet framework. In a range
of experiments, we demonstrate that by actively proposing action regions during
both training and testing, state-of-the-art or better performance is achieved
on benchmarks. We show the outperformance of our approach compared to the
state-of-the-art in two new datasets; one emphasizes on irrelevant background,
the other highlights the camera motion. We also validate our action recognition
method in an abnormal behavior detection scenario to improve workplace safety.
The results verify a higher success rate for our method due to the ability of
our system to recognize human actions regardless of environment and camera
motion.
Ludvig Ericson, Rendani Mbuvha
Comments: 4 Pages, 4 Figures, 1 Table
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Performance (cs.PF); Machine Learning (stat.ML)
Artificial Neural Networks (ANNs) have received increasing attention in
recent years with applications that span a wide range of disciplines including
vital domains such as medicine, network security and autonomous transportation.
However, neural network architectures are becoming increasingly complex and
with an increasing need to obtain real-time results from such models, it has
become pivotal to use parallelization as a mechanism for speeding up network
training and deployment. In this work we propose an implementation of Network
Parallel Training through Cannon’s Algorithm for matrix multiplication. We show
that increasing the number of processes speeds up training until the point
where process communication costs become prohibitive; this point varies by
network complexity. We also show through empirical efficiency calculations that
the speedup obtained is superlinear.
Abir M 'Baya (DISP), Jannik Laval (DISP), Nejib Moalla (DISP), Yacine Ouzrout (DISP), Abdelaziz Bouras
Subjects: Artificial Intelligence (cs.AI)
Internship assignment is a complicated process for universities since it is
necessary to take into account a multiplicity of variables to establish a
compromise between companies’ requirements and student competencies acquired
during the university training. These variables build up a complex relations
map that requires the formulation of an exhaustive and rigorous conceptual
scheme. In this research a domain ontological model is presented as support to
the student’s decision making for opportunities of University studies level of
the University Lumiere Lyon 2 (ULL) education system. The ontology is designed
and created using methodological approach offering the possibility of improving
the progressive creation, capture and knowledge articulation. In this paper, we
draw a balance taking the demands of the companies across the capabilities of
the students. This will be done through the establishment of an ontological
model of an educational learners’ profile and the internship postings which are
written in a free text and using uncontrolled vocabulary. Furthermore, we
outline the process of semantic matching which improves the quality of query
results.
Samuel Albanie, Hillary Shakespeare, Tom Gunter
Comments: NIPS Symposium 2016: Machine Learning and the Law
Subjects: Artificial Intelligence (cs.AI)
For a social networking service to acquire and retain users, it must find
ways to keep them engaged. By accurately gauging their preferences, it is able
to serve them with the subset of available content that maximises revenue for
the site. Without the constraints of an appropriate regulatory framework, we
argue that a sufficiently sophisticated curator algorithm tasked with
performing this process may choose to explore curation strategies that are
detrimental to users. In particular, we suggest that such an algorithm is
capable of learning to manipulate its users, for several qualitative reasons:
1. Access to vast quantities of user data combined with ongoing breakthroughs
in the field of machine learning are leading to powerful but uninterpretable
strategies for decision making at scale. 2. The availability of an effective
feedback mechanism for assessing the short and long term user responses to
curation strategies. 3. Techniques from reinforcement learning have allowed
machines to learn automated and highly successful strategies at an abstract
level, often resulting in non-intuitive yet nonetheless highly appropriate
action selection. In this work, we consider the form that these strategies for
user manipulation might take and scrutinise the role that regulation should
play in the design of such systems.
Zetao Chen, Adam Jacobson, Niko Sunderhauf, Ben Upcroft, Lingqiao Liu, Chunhua Shen, Ian Reid, Michael Milford
Comments: 8 pages, 10 figures. Accepted by International Conference on Robotics and Automation (ICRA) 2017. This is the submitted version. The final published version may be slightly different
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Robotics (cs.RO)
The success of deep learning techniques in the computer vision domain has
triggered a range of initial investigations into their utility for visual place
recognition, all using generic features from networks that were trained for
other types of recognition tasks. In this paper, we train, at large scale, two
CNN architectures for the specific place recognition task and employ a
multi-scale feature encoding method to generate condition- and
viewpoint-invariant features. To enable this training to occur, we have
developed a massive Specific PlacEs Dataset (SPED) with hundreds of examples of
place appearance change at thousands of different places, as opposed to the
semantic place type datasets currently available. This new dataset enables us
to set up a training regime that interprets place recognition as a
classification problem. We comprehensively evaluate our trained networks on
several challenging benchmark place recognition datasets and demonstrate that
they achieve an average 10% increase in performance over other place
recognition algorithms and pre-trained CNNs. By analyzing the network responses
and their differences from pre-trained networks, we provide insights into what
a network learns when training for place recognition, and what these results
signify for future research in this area.
Ardavan Salehi Nobandegani, Thomas R. Shultz
Subjects: Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Humans are not only adept in recognizing what class an input instance belongs
to (i.e., classification task), but perhaps more remarkably, they can imagine
(i.e., generate) plausible instances of a desired class with ease, when
prompted. Inspired by this, we propose a framework which allows transforming
Cascade-Correlation Neural Networks (CCNNs) into probabilistic generative
models, thereby enabling CCNNs to generate samples from a category of interest.
CCNNs are a well-known class of deterministic, discriminative NNs, which
autonomously construct their topology, and have been successful in giving
accounts for a variety of psychological phenomena. Our proposed framework is
based on a Markov Chain Monte Carlo (MCMC) method, called the
Metropolis-adjusted Langevin algorithm, which capitalizes on the gradient
information of the target distribution to direct its explorations towards
regions of high probability, thereby achieving good mixing properties. Through
extensive simulations, we demonstrate the efficacy of our proposed framework.
Gürkan Alpaslan
Subjects: Information Retrieval (cs.IR)
By the growing trend of online shopping and e-commerce websites,
recommendation systems have gained more importance in recent years in order to
increase the sales ratios of companies. Different algorithms on recommendation
systems are used and every one produce different results. Every algorithm on
this area have positive and negative attributes. The purpose of the research is
to test the different algorithms for choosing the best one according as
structure of dataset and aims of developers. For this purpose, threshold and
k-means based collaborative filtering and content-based filtering algorithms
are utilized on the dataset contains 100*73421 matrix length. What are the
differences and effects of these different algorithms on the same dataset? What
are the challenges of the algorithms? What criteria are more important in order
to evaluate a recommendation systems? In the study, we answer these crucial
problems with the case study.
Swati Agarwal, Ashish Sureka
Comments: This paper is an extended and detailed version of our (same authors) previous paper (regular paper) published at COMSNETS2015
Journal-ref: S. Agarwal and A. Sureka, “Using common-sense knowledge-base for
detecting word obfuscation in adversarial communication,” 2015 7th
International Conference on Communication Systems and Networks (COMSNETS),
Bangalore, 2015, pp. 1-6
Subjects: Information Retrieval (cs.IR)
Word obfuscation or substitution means replacing one word with another word
in a sentence to conceal the textual content or communication. Word obfuscation
is used in adversarial communication by terrorist or criminals for conveying
their messages without getting red-flagged by security and intelligence
agencies intercepting or scanning messages (such as emails and telephone
conversations). ConceptNet is a freely available semantic network represented
as a directed graph consisting of nodes as concepts and edges as assertions of
common sense about these concepts. We present a solution approach exploiting
vast amount of semantic knowledge in ConceptNet for addressing the technically
challenging problem of word substitution in adversarial communication. We frame
the given problem as a textual reasoning and context inference task and utilize
ConceptNet’s natural-language-processing tool-kit for determining word
substitution. We use ConceptNet to compute the conceptual similarity between
any two given terms and define a Mean Average Conceptual Similarity (MACS)
metric to identify out-of-context terms. The test-bed to evaluate our proposed
approach consists of Enron email dataset (having over 600000 emails generated
by 158 employees of Enron Corporation) and Brown corpus (totaling about a
million words drawn from a wide variety of sources). We implement word
substitution techniques used by previous researches to generate a test dataset.
We conduct a series of experiments consisting of word substitution methods used
in the past to evaluate our approach. Experimental results reveal that the
proposed approach is effective.
Swati Agarwal, Ashish Sureka
Comments: This paper is an extended and detailed version of our (same authors’) short paper published in EISIC 2016
Subjects: Information Retrieval (cs.IR)
Research shows that many like-minded people use popular microblogging
websites for posting hateful speech against various religions and race.
Automatic identification of racist and hate promoting posts is required for
building social media intelligence and security informatics based solutions.
However, just keyword spotting based techniques cannot be used to accurately
identify the intent of a post. In this paper, we address the challenge of the
presence of ambiguity in such posts by identifying the intent of author. We
conduct our study on Tumblr microblogging website and develop a cascaded
ensemble learning classifier for identifying the posts having racist or
radicalized intent. We train our model by identifying various semantic,
sentiment and linguistic features from free-form text. Our experimental results
shows that the proposed approach is effective and the emotion tone, social
tendencies, language cues and personality traits of a narrative are
discriminatory features for identifying the racist intent behind a post.
Eugénio Ribeiro, Fernando Batista, Isabel Trancoso, José Lopes, Ricardo Ribeiro, David Martins de Matos
Comments: 10 pages
Journal-ref: Advances in Speech and Language Technologies for Iberian
Languages: Third International Conference, IberSPEECH 2016, Lisbon, Portugal,
November 23-25, pp. 245-254
Subjects: Computation and Language (cs.CL)
Identifying the level of expertise of its users is important for a system
since it can lead to a better interaction through adaptation techniques.
Furthermore, this information can be used in offline processes of root cause
analysis. However, not much effort has been put into automatically identifying
the level of expertise of an user, especially in dialog-based interactions. In
this paper we present an approach based on a specific set of task related
features. Based on the distribution of the features among the two classes –
Novice and Expert – we used Random Forests as a classification approach.
Furthermore, we used a Support Vector Machine classifier, in order to perform a
result comparison. By applying these approaches on data from a real system,
Let’s Go, we obtained preliminary results that we consider positive, given the
difficulty of the task and the lack of competing approaches for comparison.
Ofer Feinerman, Amos Kormanë (CNRS, IRIF, GANG)
Journal-ref: The Journal of Experimental Biology, The Company of Biologists
2017, 220, pp.73 – 82
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC)
The concerted responses of eusocial insects to environmental stimuli are
often referred to as collective cognition on the level of the colony.To achieve
collective cognitiona group can draw on two different sources: individual
cognitionand the connectivity between individuals.Computation in
neural-networks, for example,is attributedmore tosophisticated communication
schemes than to the complexity of individual neurons. The case of social
insects, however, can be expected to differ. This is since individual insects
are cognitively capable units that are often able to process information that
is directly relevant at the level of the colony.Furthermore, involved
communication patterns seem difficult to implement in a group of insects since
these lack clear network structure.This review discusses links between the
cognition of an individual insect and that of the colony. We provide examples
for collective cognition whose sources span the full spectrum between
amplification of individual insect cognition and emergent group-level
processes.
Burak C. Civek, Ibrahim Delibalta, Suleyman S. Kozat
Subjects: Learning (cs.LG)
We introduce highly efficient online nonlinear regression algorithms that are
suitable for real life applications. We process the data in a truly online
manner such that no storage is needed, i.e., the data is discarded after being
used. For nonlinear modeling we use a hierarchical piecewise linear approach
based on the notion of decision trees where the space of the regressor vectors
is adaptively partitioned based on the performance. As the first time in the
literature, we learn both the piecewise linear partitioning of the regressor
space as well as the linear models in each region using highly effective second
order methods, i.e., Newton-Raphson Methods. Hence, we avoid the well known
over fitting issues by using piecewise linear models, however, since both the
region boundaries as well as the linear models in each region are trained using
the second order methods, we achieve substantial performance compared to the
state of the art. We demonstrate our gains over the well known benchmark data
sets and provide performance results in an individual sequence manner
guaranteed to hold without any statistical assumptions. Hence, the introduced
algorithms address computational complexity issues widely encountered in real
life applications while providing superior guaranteed performance in a strong
deterministic sense.
Samuel Kadoury, William Mandel, Marie-Lyne Nault, Stefan Parent
Subjects: Learning (cs.LG)
We introduce a novel approach for predicting the progression of adolescent
idiopathic scoliosis from 3D spine models reconstructed from biplanar X-ray
images. Recent progress in machine learning have allowed to improve
classification and prognosis rates, but lack a probabilistic framework to
measure uncertainty in the data. We propose a discriminative probabilistic
manifold embedding where locally linear mappings transform data points from
high-dimensional space to corresponding low-dimensional coordinates. A
discriminant adjacency matrix is constructed to maximize the separation between
progressive and non-progressive groups of patients diagnosed with scoliosis,
while minimizing the distance in latent variables belonging to the same class.
To predict the evolution of deformation, a baseline reconstruction is projected
onto the manifold, from which a spatiotemporal regression model is built from
parallel transport curves inferred from neighboring exemplars. Rate of
progression is modulated from the spine flexibility and curve magnitude of the
3D spine deformation. The method was tested on 745 reconstructions from 133
subjects using longitudinal 3D reconstructions of the spine, with results
demonstrating the discriminatory framework can identify between progressive and
non-progressive of scoliotic patients with a classification rate of 81% and
prediction differences of 2.1(^{o}) in main curve angulation, outperforming
other manifold learning methods. Our method achieved a higher prediction
accuracy and improved the modeling of spatiotemporal morphological changes in
highly deformed spines compared to other learning methods.
Zhao Peng
Subjects: Machine Learning (stat.ML); Learning (cs.LG)
Artificial Neural Networks(ANN) has been phenomenally successful on various
pattern recognition tasks. However, the design of neural networks rely heavily
on the experience and intuitions of individual developers. In this article, the
author introduces a mathematical structure called MLP algebra on the set of all
Multilayer Perceptron Neural Networks(MLP), which can serve as a guiding
principle to build MLPs accommodating to the particular data sets, and to build
complex MLPs from simpler ones.
Volodymyr Turchenko, Eric Chalmers, Artur Luczak
Comments: 21 pages, 11 figures, 5 tables, 62 references
Subjects: Neural and Evolutionary Computing (cs.NE); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG)
This paper presents the development of several models of a deep convolutional
auto-encoder in the Caffe deep learning framework and their experimental
evaluation on the example of MNIST dataset. We have created five models of a
convolutional auto-encoder which differ architecturally by the presence or
absence of pooling and unpooling layers in the auto-encoder’s encoder and
decoder parts. Our results show that the developed models provide very good
results in dimensionality reduction and unsupervised clustering tasks, and
small classification errors when we used the learned internal code as an input
of a supervised linear classifier and multi-layer perceptron. The best results
were provided by a model where the encoder part contains convolutional and
pooling layers, followed by an analogous decoder part with deconvolution and
unpooling layers without the use of switch variables in the decoder part. The
paper also discusses practical details of the creation of a deep convolutional
auto-encoder in the very popular Caffe deep learning framework. We believe that
our approach and results presented in this paper could help other researchers
to build efficient deep neural network architectures in the future.
Chung Chan, Ali Al-Bashabsheh, Qiaoqiao Zhou
Subjects: Information Theory (cs.IT); Learning (cs.LG)
An agglomerative clustering of random variables is proposed, where clusters
of random variables sharing the maximum amount of multivariate mutual
information are merged successively to form larger clusters. Compared to the
previous info-clustering algorithms, the agglomerative approach allows the
computation to stop earlier when clusters of desired size and accuracy are
obtained. An efficient algorithm is also derived based on the submodularity of
entropy and the duality between the principal sequence of partitions and the
principal sequence for submodular functions.
Ansuman Adhikary, Alexei Ashikhmin, Thomas L. Marzetta
Subjects: Information Theory (cs.IT)
A massive MIMO system entails a large number (tens or hundreds) of base
station antennas serving a much smaller number of terminals. These systems
demonstrate large gains in spectral and energy efficiency compared with
conventional MIMO technology. As the number of antennas grows, the performance
of a massive MIMO system gets limited by the interference caused by pilot
contamination. Earlier A. Ashikhmin and T. Marzetta proposed (under the name of
Pilot Contamination Precoding) Large Scale Fading Precoding (LSFP) and Decoding
(LSFD) based on limited cooperation between base stations. They showed that
Zero-Forcing LSFP and LSFD eliminate pilot contamination entirely and lead to
an infinite throughput as the number of antennas grows.
In this paper, we focus on the uplink and show that even in the case of a
finite number of base station antennas, LSFD yields a very large performance
gain. In particular, one of our algorithms gives a more than 140 fold increase
in the 5% outage data transmission rate! We show that the performance can be
improved further by optimizing the transmission powers of the users. Finally,
we present decentralized LSFD that requires limited cooperation only between
neighboring cells.
Yuyi Mao, Jun Zhang, Khaled B. Letaief
Comments: 6 pages, 4 figures, accepted to IEEE Wireless Communications and Networking Conference (WCNC) 2017, San Francisco, CA, March 2017
Subjects: Information Theory (cs.IT)
Mobile-edge computing (MEC) has emerged as a prominent technique to provide
mobile services with high computation requirement, by migrating the
computation-intensive tasks from the mobile devices to the nearby MEC servers.
To reduce the execution latency and device energy consumption, in this paper,
we jointly optimize task offloading scheduling and transmit power allocation
for MEC systems with multiple independent tasks. A low-complexity sub-optimal
algorithm is proposed to minimize the weighted sum of the execution delay and
device energy consumption based on alternating minimization. Specifically,
given the transmit power allocation, the optimal task offloading scheduling,
i.e., to determine the order of offloading, is obtained with the help of flow
shop scheduling theory. Besides, the optimal transmit power allocation with a
given task offloading scheduling decision will be determined using convex
optimization techniques. Simulation results show that task offloading
scheduling is more critical when the available radio and computational
resources in MEC systems are relatively balanced. In addition, it is shown that
the proposed algorithm achieves near-optimal execution delay along with a
substantial device energy saving.
Mustafa Sari, Emre Kolotoglu
Subjects: Information Theory (cs.IT)
This paper is devoted to the study of the construction of new quantum MDS
codes. Based on constacyclic codes over Fq2 , we derive four new families of
quantum MDS codes, one of which is an explicit generalization of the
construction given in Theorem 7 in [22]. We also extend the result of Theorem
3:3 given in [17].
Septimia Sarbu
Comments: 20 pages, submitted to IEEE Transactions on Information Theory. arXiv admin note: substantial text overlap with arXiv:1604.05113
Subjects: Information Theory (cs.IT)
We prove the Courtade-Kumar conjecture, for several classes of n-dimensional
Boolean functions, for all (n geq 2) and for all values of the error
probability of the binary symmetric channel, (0 leq p leq 1/2). This
conjecture states that the mutual information between any Boolean function of
an n-dimensional vector of independent and identically distributed inputs to a
memoryless binary symmetric channel and the corresponding vector of outputs is
upper-bounded by (1-operatorname{H}(p)), where (operatorname{H}(p))
represents the binary entropy function. That is, let (mathbf{X}=[X_1 ldots
X_n]) be a vector of independent and identically distributed Bernoulli(1/2)
random variables, which are the input to a memoryless binary symmetric channel,
with the error probability in the interval (0 leq p leq 1/2) and
(mathbf{Y}=[Y_1 ldots Y_n]) the corresponding output. Let (f:{0,1}^n
ightarrow {0,1}) be an n-dimensional Boolean function. Then,
(operatorname{MI}(f(X),Y) leq 1-operatorname{H}(p)). Our proof employs
Karamata’s theorem, concepts from probability theory, transformations of random
variables and vectors and algebraic manipulations.
Chung Chan, Manuj Mukherjee, Navin Kashyap, Qiaoqiao Zhou
Subjects: Information Theory (cs.IT)
For the multiterminal secret key agreement problem, new single-letter lower
bounds are obtained on the public discussion rate required to achieve any given
secret key rate below the secrecy capacity. The results apply to general source
model without helpers or wiretapper’s side information but can be strengthened
for hypergraphical sources. In particular, for the pairwise independent
network, the results give rise to a complete characterization of the maximum
secret key rate achievable under a constraint on the total discussion rate.
Stas Tiomkin, Daniel Polani, Naftali Tishby
Comments: 11 [ages, 7 figures
Subjects: Information Theory (cs.IT)
Stochastic dynamic control systems relate in a prob- abilistic fashion the
space of control signals to the space of corresponding future states.
Consequently, stochastic dynamic systems can be interpreted as an information
channel between the control space and the state space. In this work we study
this control-to-state informartion capacity of stochastic dynamic systems in
continuous-time, when the states are observed only partially. The
control-to-state capacity, known as empowerment, was shown in the past to be
useful in solving various Artificial Intelligence & Control benchmarks, and was
used to replace problem-specific utilities. The higher the value of empowerment
is, the more optional future states an agent may reach by using its controls
inside a given time horizon. The contribution of this work is that we derive an
efficient solution for computing the control-to-state information capacity for
a linear, partially-observed Gaussian dynamic control system in continuous
time, and discover new relationships between control-theoretic and
information-theoretic properties of dynamic systems. Particularly, using the
derived method, we demonstrate that the capacity between the control signal and
the system output does not grow without limits with the length of the control
signal. This means that only the near-past window of the control signal
contributes effectively to the control-to-state capacity, while most of the
information beyond this window is irrelevant for the future state of the
dynamic system. We show that empowerment depends on a time constant of a
dynamic system.
Luca Barletta, Stefano Rini
Comments: 7 pages, 3 figures. Extended version of a paper submitted to ISIT 2017
Subjects: Information Theory (cs.IT)
The capacity of the discrete-time channel affected by both additive Gaussian
noise and Wiener phase noise is studied. Novel inner and outer bounds are
presented, which differ of at most (6.65) bits per channel use for all channel
parameters. The capacity of this model can be subdivided in three regimes: (i)
for large values of the frequency noise variance, the channel behaves similarly
to a channel with circularly uniform iid phase noise; (ii) when the frequency
noise variance is small, the effects of the additive noise dominate over those
of the phase noise, while (iii) for intermediate values of the frequency noise
variance, the transmission rate over the phase modulation channel has to be
reduced due to the presence of phase noise.
Abhiram Gnanasambandam, Ragini Chaluvadi, Srikrishna Bhashyam
Comments: Longer version of submission to ISIT 2017
Subjects: Information Theory (cs.IT)
In this paper, we obtain new sum capacity results for the Gaussian
many-to-one and one-to-many interference channels. Simple Han-Kobayashi (HK)
schemes, i.e., HK schemes with Gaussian signaling, no time-sharing, and no
common-private power splitting, achieve sum capacity under the channel
conditions for which the new results are obtained. First, by careful
Fourier-Motzkin elimination, we obtain the HK achievable rate region for the
K-user Gaussian many-to-one and one-to-many channels in simplified form, i.e.,
only in terms of the K rates (R_1), (R_2),…, (R_K). We also obtain the
achievable sum rate using Fourier-Motzkin elimination. Then, to obtain sum
capacity results, we derive genie-aided upper bounds that match the achievable
sum rate of simple HK schemes under certain channel conditions.
Nguyen T. Nghia, Hoang D. Tuan, Trung Q. Duong, H. Vincent Poor
Comments: 12 pages, 2 figures
Subjects: Information Theory (cs.IT)
Considering a multiple-user multiple-input multiple-output (MIMO) channel
with an eavesdropper, this letter develops a beamformer design to optimize the
energy efficiency in terms of secrecy bits per Joule under secrecy
quality-of-service constraints. This is a very difficult design problem with no
available exact solution techniques. A path-following procedure, which
iteratively improves its feasible points by using a simple quadratic program of
moderate dimension, is proposed. Under any fixed computational tolerance the
procedure terminates after finitely many iterations, yielding at least a
locally optimal solution. Simulation results show the superior performance of
the obtained algorithm over other existing methods.
Mohammed Karmoose, Linqi Song, Martina Cardone, Christina Fragouli
Subjects: Information Theory (cs.IT)
Using a broadcast channel to transmit clients’ data requests may impose
privacy risks. In this paper, we address such privacy concerns in the index
coding framework. We show how a malicious client can infer some information
about the requests and side information of other clients by learning the
encoding matrix used by the server. We propose an information-theoretic metric
to measure the level of privacy and show how encoding matrices can be designed
to achieve specific privacy guarantees. We then consider a special scenario for
which we design a transmission scheme and derive the achieved levels of privacy
in closed-form. We also derive upper bounds and we compare them to the levels
of privacy achieved by our scheme, highlighting that an inherent trade-off
exists between protecting privacy of the request and of the side information of
the clients.
Anshoo Tandon, Han Mao Kiah, Mehul Motani
Subjects: Information Theory (cs.IT)
The study of subblock-constrained codes has recently gained attention due to
their application in diverse fields. We present bounds on the size and
asymptotic rate for two classes of subblock-constrained codes. The first class
is binary constant subblock-composition codes (CSCCs), where each codeword is
partitioned into equal sized subblocks, and every subblock has the same fixed
weight. The second class is binary subblock energy-constrained codes (SECCs),
where the weight of every subblock exceeds a given threshold. We present novel
upper and lower bounds on the code sizes and asymptotic rates for binary CSCCs
and SECCs. For a fixed subblock length and small relative distance, we show
that the asymptotic rate for CSCCs (resp. SECCs) is strictly lower than the
corresponding rate for constant weight codes (CWCs) (resp. heavy weight codes
(HWCs)). Further, for codes with high weight and low relative distance, we show
that the asymptotic rates for CSCCs is strictly lower than that of SECCs, which
contrasts that the asymptotic rate for CWCs is equal to that of HWCs. We also
provide a correction to an earlier result by Chee et al. (2014) on the
asymptotic CSCC rate. Additionally, we present several numerical examples
comparing the rates for CSCCs and SECCs with those for constant weight codes
and heavy weight codes.
Jun Muramatsu, Shigeki Miyake
Comments: 10 pages. This is the extended version of the paper submitted to ISIT2017
Subjects: Information Theory (cs.IT)
This paper investigates the error probability of a stochastic decision and
the way in which it differs from the error probability of an optimal decision,
i.e., the maximum a posteriori decision. It is shown that the error probability
of a stochastic decision with the a posteriori distribution is at most twice
the error probability of the maximum a posteriori decision. Furthermore, it is
shown that, by generating an independent identically distributed random
sequence subject to the a posteriori distribution and making a decision that
maximizes the a posteriori probability over the sequence, the error probability
approaches exponentially the error probability of the maximum a posteriori
decision as the sequence length increases. Using these ideas as a basis, we can
construct stochastic decoders for source/channel codes.
Chung Chan, Ali Al-Bashabsheh, Qiaoqiao Zhou
Subjects: Information Theory (cs.IT); Learning (cs.LG)
An agglomerative clustering of random variables is proposed, where clusters
of random variables sharing the maximum amount of multivariate mutual
information are merged successively to form larger clusters. Compared to the
previous info-clustering algorithms, the agglomerative approach allows the
computation to stop earlier when clusters of desired size and accuracy are
obtained. An efficient algorithm is also derived based on the submodularity of
entropy and the duality between the principal sequence of partitions and the
principal sequence for submodular functions.
N. Prakash, Vitaly Abdrashitov, Muriel Medard
Comments: 26 pages, 15 figures
Subjects: Information Theory (cs.IT)
We study a generalization of the setting of regenerating codes, motivated by
applications to storage systems consisting of clusters of storage nodes. There
are (n) clusters in total, with (m) nodes per cluster. A data file is coded and
stored across the (mn) nodes, with each node storing (alpha) symbols. For
availability of data, we demand that the file is retrievable by downloading the
entire content from any subset of (k) clusters. Nodes represent entities that
can fail, and we distinguish between intra-cluster and inter-cluster
bandwidth-costs during node repair. Node-repair is accomplished by downloading
(eta) symbols each from any set of (d) other clusters. The replacement-node
also downloads content from any set of (ell) surviving nodes in the same
cluster during the repair process. We first identity the optimal trade-off
between storage-overhead and inter-cluster (IC) repair-bandwidth under
functional repair, and also present optimal exact-repair code constructions for
a class of parameters. The new trade-off is strictly better than what is
achievable via space-sharing existing coding solutions, whenever (ell > 0). We
then obtain lower bounds on the necessary intra-cluster repair-bandwidth to
achieve optimal trade-off. Our bounds reveal the interesting fact that while it
is beneficial to increase the number of local helper nodes (ell) in order to
improve the storage-vs-IC-repair-bandwidth trade-off, a high value of (ell)
also demands a higher amount of intra-cluster repair-bandwidth. Under
functional repair, random linear network codes (RLNCs) simultaneously optimize
usage of both inter and intra-cluster repair bandwidth; simulation results
based on RLNCs suggest optimality of the bounds on intra-cluster
repair-bandwidth. We also analyze resiliency of the storage system against
passive eavesdropping by providing file-size bounds and optimal code
constructions.
Kedar Kulkarni, Adrish Banerjee
Comments: Accepted for publication in Physical Communication
Subjects: Information Theory (cs.IT); Networking and Internet Architecture (cs.NI)
We consider a cognitive radio network in a multi-channel licensed
environment. Secondary user transmits in a channel if the channel is sensed to
be vacant. This results in a tradeoff between sensing time and transmission
time. When secondary users are energy constrained, energy available for
transmission is less if more energy is used in sensing. This gives rise to an
energy tradeoff. For multiple primary channels, secondary users must decide
appropriate sensing time and transmission power in each channel to maximize
average aggregate-bit throughput in each frame duration while ensuring
quality-of-service of primary users. Considering time and energy as limited
resources, we formulate this problem as a resource allocation problem.
Initially a single secondary user scenario is considered and solution is
obtained using decomposition and alternating optimization techniques. Later we
extend the analysis for the case of multiple secondary users. Simulation
results are presented to study effect of channel occupancy, fading and energy
availability on performance of proposed method.
Philipp Walk, Peter Jung, Götz E. Pfander, Babak Hassibi
Comments: 17 pages, 4 figures, 50th Asilomar
Subjects: Information Theory (cs.IT)
In this work we characterize all ambiguities of the linear (aperiodic)
one-dimensional convolution on two fixed finite-dimensional complex vector
spaces. It will be shown that the convolution ambiguities can be mapped
one-to-one to factorization ambiguities in the (z-)domain, which are generated
by swapping the zeros of the input signals. We use this polynomial description
to show a deterministic version of a recently introduced masked Fourier phase
retrieval design. In the noise-free case a (convex) semi-definite program can
be used to recover exactly the input signals if they share no common factors
(zeros). Then, we reformulate the problem as deterministic blind deconvolution
with prior knowledge of the autocorrelations. Numerically simulations show that
our approach is also robust against additive noise.
Ali Moharrer, Shuangqing Wei, George T. Amariucai, Jing Deng
Comments: Part of this work has been submitted to 2017 IEEE International Symposium on Information Theory, Aachen, Germany 10 pages, 6 Figures
Subjects: Information Theory (cs.IT)
A new synthesis scheme is proposed to effectively generate a random vector
with prescribed joint density that induces a (latent) Gaussian tree structure.
The quality of synthesis is measured by total variation distance between the
synthesized and desired statistics. The proposed layered and successive
encoding scheme relies on the learned structure of tree to use minimal number
of common random variables to synthesize the desired density. We characterize
the achievable rate region for the rate tuples of multi-layer latent Gaussian
tree, through which the number of bits needed to simulate such Gaussian joint
density are determined. The random sources used in our algorithm are the latent
variables at the top layer of tree, the additive independent Gaussian noises,
and the Bernoulli sign inputs that capture the ambiguity of correlation signs
between the variables.
Omid Semiari, Walid Saad, Mehdi Bennis
Subjects: Computer Science and Game Theory (cs.GT); Information Theory (cs.IT)
One of the most promising approaches to overcome the uncertainty and dynamic
channel variations of millimeter wave (mmW) communications is to deploy
dual-mode base stations that integrate both mmW and microwave ((mu)W)
frequencies. If properly designed, such dual-mode base stations can enhance
mobility and handover in highly mobile wireless environments. In this paper, a
novel approach for analyzing and managing mobility in joint (mu)W-mmW networks
is proposed. The proposed approach leverages device-level caching along with
the capabilities of dual-mode base stations to minimize handover failures,
reduce inter-frequency measurement energy consumption, and provide seamless
mobility in emerging dense heterogeneous networks. First, fundamental results
on the caching capabilities, including caching probability and cache duration
are derived for the proposed dual-mode network scenario. Second, the average
achievable rate of caching is derived for mobile users. Third, the proposed
cache-enabled mobility management problem is formulated as a dynamic matching
game between mobile user equipments (MUEs) and small base stations (SBSs). The
goal of this game is to find a distributed handover mechanism that subject to
the network constraints on HOFs and limited cache sizes, allows each MUE to
choose between executing an HO to a target SBS, being connected to the
macrocell base station (MBS), or perform a transparent HO by using the cached
content. The formulated matching game allows capturing the dynamics of the
mobility management problem caused by HOFs. To solve this dynamic matching
problem, a novel algorithm is proposed and its convergence to a two-sided
dynamically stable HO policy is proved. Numerical results corroborate the
analytical derivations and show that the proposed solution will provides
significant reductions in both the HOF and energy consumption by MUEs.
Ronan Kerviche, Saikat Guha, Amit Ashok
Comments: 8 pages, 3 figures, submitted to ISIT 2017
Subjects: Optics (physics.optics); Information Theory (cs.IT); Quantum Physics (quant-ph)
Estimating the angular separation between two incoherently radiating
monochromatic point sources is a canonical toy problem to quantify spatial
resolution in imaging. In recent work, Tsang {em et al.} showed, using a
Fisher Information analysis, that Rayleigh’s resolution limit is just an
artifact of the conventional wisdom of intensity measurement in the image
plane. They showed that the optimal sensitivity of estimating the angle is only
a function of the total photons collected during the camera’s integration time
but entirely independent of the angular separation itself no matter how small
it is, and found the information-optimal mode basis, intensity detection in
which achieves the aforesaid performance. We extend the above analysis, which
was done for a Gaussian point spread function (PSF) to a hard-aperture pupil
proving the information optimality of image-plane sinc-Bessel modes, and
generalize the result further to an arbitrary PSF. We obtain new
counterintuitive insights on energy vs. information content in spatial modes,
and extend the Fisher Information analysis to exact calculations of minimum
mean squared error, both for Gaussian and hard aperture pupils.