Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Anastasiia O. Deineko
Journal-ref: I.J. Modern Education and Computer Science, 2015, 2, 1-7
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
An architecture of a new neuro-fuzzy system is proposed. The basic idea of
this approach is to tune both synaptic weights and membership functions with
the help of the supervised learning and self-learning paradigms. The approach
to solving the problem has to do with evolving online neuro-fuzzy systems that
can process data under uncertainty conditions. The results prove the
effectiveness of the developed architecture and the learning procedure.
Zhengbing Hu, Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Olena O. Boiko
Journal-ref: I.J. Information Technology and Computer Science, 2016, 10, 1-10
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
An evolving weighted neuro-neo-fuzzy-ANARX model and its learning procedures
are introduced in the article. This system is basically used for time series
forecasting. This system may be considered as a pool of elements that process
data in a parallel manner. The proposed evolving system may provide online
processing data streams.
Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Daria S. Kopaliani
Journal-ref: I.J. Information Technology and Computer Science, 2014, 08, 11-17
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
A new architecture and learning algorithms for the multidimensional hybrid
cascade neural network with neuron pool optimization in each cascade are
proposed in this paper. The proposed system differs from the well-known cascade
systems in its capability to process multidimensional time series in an online
mode, which makes it possible to process non-stationary stochastic and chaotic
signals with the required accuracy. Compared to conventional analogs, the
proposed system provides computational simplicity and possesses both tracking
and filtering capabilities.
Zhengbing Hu, Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Olena O. Boiko
Journal-ref: I.J. Intelligent Systems and Applications, 2016, 9, 1-7
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
Neo-fuzzy elements are used as nodes for an evolving cascade system. The
proposed system can tune both its parameters and architecture in an online
mode. It can be used for solving a wide range of Data Mining tasks (namely time
series forecasting). The evolving cascade system with neo-fuzzy nodes can
process rather large data sets with high speed and effectiveness.
Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Daria S. Kopaliani
Journal-ref: I.J. Intelligent Systems and Applications, 2015, 02, 21-26
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
A modification of the neo-fuzzy neuron is proposed (an extended neo-fuzzy
neuron (ENFN)) that is characterized by improved approximating properties. An
adaptive learning algorithm is proposed that has both tracking and smoothing
properties. An ENFN distinctive feature is its computational simplicity
compared to other artificial neural networks and neuro-fuzzy systems.
Tsendsuren Munkhdalai, Hong Yu
Comments: initial submission: 9 pages
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Hypothesis testing is an important cognitive process that supports human
reasoning. In this paper, we introduce a computational hypothesis testing
approach based on memory augmented neural networks. Our approach involves a
hypothesis testing loop that reconsiders and progressively refines a previously
formed hypothesis in order to generate new hypotheses to test. We apply the
proposed approach to language comprehension task by using Neural Semantic
Encoders (NSE). Our NSE models achieve the state-of-the-art results showing an
absolute improvement of 1.2% to 2.6% accuracy over previous results obtained by
single and ensemble systems on standard machine comprehension benchmarks such
as the Children’s Book Test (CBT) and Who-Did-What (WDW) news article datasets.
Hao Dong, Akara Supratak, Wei Pan, Chao Wu, Paul M. Matthews, Yike Guo
Comments: Under review of IEEE Transactions on Neural Systems and Rehabilitation Engineering since Jun 2016
Subjects: Neurons and Cognition (q-bio.NC); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
This paper proposes a practical approach to addressing limitations posed by
use of single active electrodes in applications for sleep stage classification.
Electroencephalography (EEG)-based characterizations of sleep stage progression
contribute the diagnosis and monitoring of the many pathologies of sleep.
Several prior reports have explored ways of automating the analysis of sleep
EEG and of reducing the complexity of the data needed for reliable
discrimination of sleep stages in order to make it possible to perform sleep
studies at lower cost in the home (rather than only in specialized clinical
facilities). However, these reports have involved recordings from electrodes
placed on the cranial vertex or occiput, which can be uncomfortable or
difficult for subjects to position. Those that have utilized single EEG
channels which contain less sleep information, have showed poor classification
performance. We have taken advantage of Rectifier Neural Network for feature
detection and Long Short-Term Memory (LSTM) network for sequential data
learning to optimize classification performance with single electrode
recordings. After exploring alternative electrode placements, we found a
comfortable configuration of a single-channel EEG on the forehead and have
shown that it can be integrated with additional electrodes for simultaneous
recording of the electroocuolgram (EOG). Evaluation of data from 62 people
(with 494 hours sleep) demonstrated better performance of our analytical
algorithm for automated sleep classification than existing approaches using
vertex or occipital electrode placements. Use of this recording configuration
with neural network deconvolution promises to make clinically indicated home
sleep studies practical.
Marc Pickett, Rami Al-Rfou, Louis Shao, Chris Tar
Comments: Submission to NIPS workshop on Continual Learning. 4 page extended abstract plus 5 more pages of references, figures, and supplementary material
Subjects: Artificial Intelligence (cs.AI); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
The long-term memory of most connectionist systems lies entirely in the
weights of the system. Since the number of weights is typically fixed, this
bounds the total amount of knowledge that can be learned and stored. Though
this is not normally a problem for a neural network designed for a specific
task, such a bound is undesirable for a system that continually learns over an
open range of domains. To address this, we describe a lifelong learning system
that leverages a fast, though non-differentiable, content-addressable memory
which can be exploited to encode both a long history of sequential episodic
knowledge and semantic knowledge over many episodes for an unbounded number of
domains. This opens the door for investigation into transfer learning, and
leveraging prior knowledge that has been learned over a lifetime of experiences
to new domains.
Georgios P. Spithourakis, Steffen E. Petersen, Sebastian Riedel
Comments: Accepted at the 7th International Workshop on Health Text Mining and Information Analysis (LOUHI) EMNLP 2016
Subjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Neural and Evolutionary Computing (cs.NE)
Assisted text input techniques can save time and effort and improve text
quality. In this paper, we investigate how grounded and conditional extensions
to standard neural language models can bring improvements in the tasks of word
prediction and completion. These extensions incorporate a structured knowledge
base and numerical values from the text into the context used to predict the
next word. Our automated evaluation on a clinical dataset shows extended models
significantly outperform standard models. Our best system uses both
conditioning and grounding, because of their orthogonal benefits. For word
prediction with a list of 5 suggestions, it improves recall from 25.03% to
71.28% and for word completion it improves keystroke savings from 34.35% to
44.81%, where theoretical bound for this dataset is 58.78%. We also perform a
qualitative investigation of how models with lower perplexity occasionally fare
better at the tasks. We found that at test time numbers have more influence on
the document level than on individual word probabilities.
Qiyang Li, Jingxing Qian, Zining Zhu, Xuchan Bao, Mohamed K. Helwa, Angela P. Schoellig
Comments: 8 pages, 13 figures, Preprint submitted to 2017 IEEE International Conference on Robotics and Automation
Subjects: Robotics (cs.RO); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Systems and Control (cs.SY)
Trajectory tracking control for quadrotors is important for applications
ranging from surveying and inspection, to film making. However, designing and
tuning classical controllers, such as proportional-integral-derivative (PID)
controllers, to achieve high tracking precision can be time-consuming and
difficult, due to hidden dynamics and other non-idealities. The Deep Neural
Network (DNN), with its superior capability of approximating abstract,
nonlinear functions, proposes a novel approach for enhancing trajectory
tracking control. This paper presents a DNN-based algorithm that improves the
tracking performance of a classical feedback controller. Given a desired
trajectory, the DNNs provide a tailored input to the controller based on their
gained experience. The input aims to achieve a unity map between the desired
and the output trajectory. The motivation for this work is an interactive
“fly-as-you-draw” application, in which a user draws a trajectory on a mobile
device, and a quadrotor instantly flies that trajectory with the DNN-enhanced
control system. Experimental results demonstrate that the proposed approach
improves the tracking precision for user-drawn trajectories after the DNNs are
trained on selected periodic trajectories, suggesting the method’s potential in
real-world applications. Tracking errors are reduced by around 40-50 % for both
training and testing trajectories from users, highlighting the DNNs’ capability
of generalizing knowledge.
Michiel Hermans, Piotr Antonik, Marc Haelterman, Serge Massar
Comments: 5 pages, 2 figures
Journal-ref: Physical Review Letters 117, 128301 (2016)
Subjects: Emerging Technologies (cs.ET); Neural and Evolutionary Computing (cs.NE)
Delay-coupled electro-optical systems have received much attention for their
dynamical properties and their potential use in signal processing. In
particular it has recently been demonstrated, using the artificial intelligence
algorithm known as reservoir computing, that photonic implementations of such
systems solve complex tasks such as speech recognition. Here we show how the
backpropagation algorithm can be physically implemented on the same
electro-optical delay-coupled architecture used for computation with only minor
changes to the original design. We find that, compared when the backpropagation
algorithm is not used, the error rate of the resulting computing device,
evaluated on three benchmark tasks, decreases considerably. This demonstrates
that electro-optical analog computers can embody a large part of their own
training process, allowing them to be applied to new, more difficult tasks.
Piotr Antonik, François Duport, Michiel Hermans, Anteo Smerieri, Marc Haelterman, Serge Massar
Comments: 13 pages, 10 figures
Journal-ref: IEEE Transactions on Neural Networks and Learning Systems ,
vol.PP, no.99, pp.1-13 (2016)
Subjects: Emerging Technologies (cs.ET); Neural and Evolutionary Computing (cs.NE)
Reservoir Computing is a bio-inspired computing paradigm for processing time
dependent signals. The performance of its analogue implementation are
comparable to other state of the art algorithms for tasks such as speech
recognition or chaotic time series prediction, but these are often constrained
by the offline training methods commonly employed. Here we investigated the
online learning approach by training an opto-electronic reservoir computer
using a simple gradient descent algorithm, programmed on an FPGA chip. Our
system was applied to wireless communications, a quickly growing domain with an
increasing demand for fast analogue devices to equalise the nonlinear distorted
channels. We report error rates up to two orders of magnitude lower than
previous implementations on this task. We show that our system is particularly
well-suited for realistic channel equalisation by testing it on a drifting and
a switching channels and obtaining good performances
Jimmy Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, Catalin Ionescu
Subjects: Machine Learning (stat.ML); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Until recently, research on artificial neural networks was largely restricted
to systems with only two types of variable: Neural activities that represent
the current or recent input and weights that learn to capture regularities
among inputs, outputs and payoffs. There is no good reason for this
restriction. Synapses have dynamics at many different time-scales and this
suggests that artificial neural networks might benefit from variables that
change slower than activities but much faster than the standard weights. These
“fast weights” can be used to store temporary memories of the recent past and
they provide a neurally plausible way of implementing the type of attention to
the past that has recently proved very helpful in sequence-to-sequence models.
By using fast weights we can avoid the need to store copies of neural activity
patterns.
Ahmed Ibrahim, A. Lynn Abbott, Mohamed E. Hussein
Comments: Accepted in the 12th International Symposium on Visual Computing (ISVC’16)
Subjects: Computer Vision and Pattern Recognition (cs.CV)
This paper describes a dataset containing small images of text from everyday
scenes. The purpose of the dataset is to support the development of new
automated systems that can detect and analyze text. Although much research has
been devoted to text detection and recognition in scanned documents, relatively
little attention has been given to text detection in other types of images,
such as photographs that are posted on social-media sites. This new dataset,
known as COCO-Text-Patch, contains approximately 354,000 small images that are
each labeled as “text” or “non-text”. This dataset particularly addresses the
problem of text verification, which is an essential stage in the end-to-end
text detection and recognition pipeline. In order to evaluate the utility of
this dataset, it has been used to train two deep convolution neural networks to
distinguish text from non-text. One network is inspired by the GoogLeNet
architecture, and the second one is based on CaffeNet. Accuracy levels of 90.2%
and 90.9% were obtained using the two networks, respectively. All of the
images, source code, and deep-learning trained models described in this paper
will be publicly available
Tomasz Kornuta, Kamil Rocki
Comments: Paper submitted to special session on Machine Intelligence organized during 23rd International AUTOMATION Conference
Subjects: Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG)
The paper focuses on the problem of learning saccades enabling visual object
search. The developed system combines reinforcement learning with a neural
network for learning to predict the possible outcomes of its actions. We
validated the solution in three types of environment consisting of
(pseudo)-randomly generated matrices of digits. The experimental verification
is followed by the discussion regarding elements required by systems mimicking
the fovea movement and possible further research directions.
Stephanie Allen, David Madras, Ye Ye, Greg Zanotti
Subjects: Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG); Machine Learning (stat.ML)
Body-worn video (BWV) cameras are increasingly utilized by police departments
to provide a record of police-public interactions. However, large-scale BWV
deployment produces terabytes of data per week, necessitating the development
of effective computational methods to identify salient changes in video. In
work carried out at the 2016 RIPS program at IPAM, UCLA, we present a novel
two-stage framework for video change-point detection. First, we employ
state-of-the-art machine learning methods including convolutional neural
networks and support vector machines for scene classification. We then develop
and compare change-point detection algorithms utilizing mean squared-error
minimization, forecasting methods, hidden Markov models, and maximum likelihood
estimation to identify noteworthy changes. We test our framework on detection
of vehicle exits and entrances in a BWV data set provided by the Los Angeles
Police Department and achieve over 90% recall and nearly 70% precision —
demonstrating robustness to rapid scene changes, extreme luminance differences,
and frequent camera occlusions.
Hamed R.-Tavakoli, Ali Borji, Jorma Laaksonen, Esa Rahtu
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
This paper presents a novel fixation prediction and saliency modeling
framework based on inter-image similarities and ensemble of Extreme Learning
Machines (ELM). The proposed framework is inspired by two observations, 1) the
contextual information of a scene along with low-level visual cues modulates
attention, 2) the influence of scene memorability on eye movement patterns
caused by the resemblance of a scene to a former visual experience. Motivated
by such observations, we develop a framework that estimates the saliency of a
given image using an ensemble of extreme learners, each trained on an image
similar to the input image. That is, after retrieving a set of similar images
for a given image, a saliency predictor is learnt from each of the images in
the retrieved image set using an ELM, resulting in an ensemble. The saliency of
the given image is then measured in terms of the mean of predicted saliency
value by the ensemble’s members.
Anne Veenendaal, Eddie Jones, Zhao Gang, Elliot Daly, Sumalini Vartak, Rahul Patwardhan
Comments: 7 pages, 4 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
This paper examines use of dynamic probabilistic networks (DPN) for human
action recognition. The actions of lifting objects and walking in the room,
sitting in the room and neutral standing pose were used for testing the
classification. The research used the dynamic interrelation between various
different regions of interest (ROI) on the human body (face, body, arms, legs)
and the time series based events related to the these ROIs. This dynamic links
are then used to recognize the human behavioral aspects in the scene. First a
model is developed to identify the human activities in an indoor scene and this
model is dependent on the key features and interlinks between the various
dynamic events using DPNs. The sub ROI are classified with DPN to associate the
combined interlink with a specific human activity. The recognition accuracy
performance between indoor (controlled lighting conditions) is compared with
the outdoor lighting conditions. The accuracy in outdoor scenes was lower than
the controlled environment.
Samaneh Abbasi-Sureshjani, Jiong Zhang, Remco Duits, Bart ter Haar Romeny
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Natural images contain often curvilinear structures, which might be
disconnected, or partly occluded. Recovering the missing connection of
disconnected structures is an open issue and needs appropriate geometric
reasoning. We propose to find line co-occurrence statistics from the
centerlines of blood vessels in retinal images and show its remarkable
similarity to a well-known probabilistic model for the connectivity pattern in
the primary visual cortex. Furthermore, the probabilistic model is trained from
the data via statistics and used for automated grouping of interrupted vessels
in a spectral clustering based approach. Several challenging image patches are
investigated around junction points, where successful results indicate the
perfect match of the trained model to the profiles of blood vessels in retinal
images. Also, comparisons among several statistical models obtained from
different datasets reveals their high similarity i.e., they are independent of
the dataset. On top of that, the best approximation of the statistical model
with the symmetrized extension of the probabilistic model on the projective
line bundle is found with a least square error smaller than 2%. Apparently, the
direction process on the projective line bundle is a good continuation model
for vessels in retinal images.
Yusuke Uchida, Shigeyuki Sakazawa, Shin'ichi Satoh
Subjects: Computer Vision and Pattern Recognition (cs.CV)
In this paper, we propose a stand-alone mobile visual search system based on
binary features and the bag-of-visual words framework. The contribution of this
study is three-fold: (1) We propose an adaptive substring extraction method
that adaptively extracts informative bits from the original binary vector and
stores them in the inverted index. These substrings are used to refine visual
word-based matching. (2) A modified local NBNN scoring method is proposed in
the context of image retrieval, which considers the density of binary features
in scoring each feature matching. (3) In order to suppress false positives, we
introduce a convexity check step that imposes a convexity constraint on the
configuration of a transformed reference image. The proposed system improves
retrieval accuracy by 11% compared with a conventional method without
increasing the database size. Furthermore, our system with the convexity check
does not lead to false positive results.
Mustafa Devrim Kaba, Mustafa Gokhan Uzunbas, Ser Nam Lim
Subjects: Computer Vision and Pattern Recognition (cs.CV)
We introduce a novel, fully automated solution method for sensor planning
problem for 3D models. By modeling the human approach to the problem first, we
put the problem into a reinforcement learning (RL) framework and successfully
solve it using the well-known RL algorithms with function approximation. We
compare our method with the greedy algorithm in various test cases and show
that we can out-perform the baseline greedy algorithm in all cases.
Raul Mur-Artal, Juan D. Tardos
Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
We present ORB-SLAM2 a complete SLAM system for monocular, stereo and RGB-D
cameras, including map reuse, loop closing and relocalization capabilities. The
system works in real-time in standard CPUs in a wide variety of environments
from small hand-held indoors sequences, to drones flying in industrial
environments and cars driving around a city. Our backend based on Bundle
Adjustment with monocular and stereo observations allows for accurate
trajectory estimation with metric scale. Our system includes a lightweight
localization mode that leverages visual odometry tracks for unmapped regions
and matches to map points that allow for zero-drift localization. The
evaluation in 29 popular public sequences shows that our method achieves
state-of-the-art accuracy, being in most cases the most accurate SLAM solution.
We publish the source code, not only for the benefit of the SLAM community, but
with the aim of being an out-of-the-box SLAM solution for researchers in other
fields.
Abbas Kazemipour, Ji Liu, Patrick Kanold, Min Wu, Behtash Babadi
Comments: 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Dec. 7-9, 2016, Washington D.C
Subjects: Machine Learning (stat.ML); Computer Vision and Pattern Recognition (cs.CV); Information Theory (cs.IT); Dynamical Systems (math.DS); Statistics Theory (math.ST)
In this paper, we consider linear state-space models with compressible
innovations and convergent transition matrices in order to model
spatiotemporally sparse transient events. We perform parameter and state
estimation using a dynamic compressed sensing framework and develop an
efficient solution consisting of two nested Expectation-Maximization (EM)
algorithms. Under suitable sparsity assumptions on the innovations, we prove
recovery guarantees and derive confidence bounds for the state estimates. We
provide simulation studies as well as application to spike deconvolution from
calcium imaging data which verify our theoretical results and show significant
improvement over existing algorithms.
Hao Dong, Akara Supratak, Wei Pan, Chao Wu, Paul M. Matthews, Yike Guo
Comments: Under review of IEEE Transactions on Neural Systems and Rehabilitation Engineering since Jun 2016
Subjects: Neurons and Cognition (q-bio.NC); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
This paper proposes a practical approach to addressing limitations posed by
use of single active electrodes in applications for sleep stage classification.
Electroencephalography (EEG)-based characterizations of sleep stage progression
contribute the diagnosis and monitoring of the many pathologies of sleep.
Several prior reports have explored ways of automating the analysis of sleep
EEG and of reducing the complexity of the data needed for reliable
discrimination of sleep stages in order to make it possible to perform sleep
studies at lower cost in the home (rather than only in specialized clinical
facilities). However, these reports have involved recordings from electrodes
placed on the cranial vertex or occiput, which can be uncomfortable or
difficult for subjects to position. Those that have utilized single EEG
channels which contain less sleep information, have showed poor classification
performance. We have taken advantage of Rectifier Neural Network for feature
detection and Long Short-Term Memory (LSTM) network for sequential data
learning to optimize classification performance with single electrode
recordings. After exploring alternative electrode placements, we found a
comfortable configuration of a single-channel EEG on the forehead and have
shown that it can be integrated with additional electrodes for simultaneous
recording of the electroocuolgram (EOG). Evaluation of data from 62 people
(with 494 hours sleep) demonstrated better performance of our analytical
algorithm for automated sleep classification than existing approaches using
vertex or occipital electrode placements. Use of this recording configuration
with neural network deconvolution promises to make clinically indicated home
sleep studies practical.
Zhengbing Hu, Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Olena O. Boiko
Journal-ref: I.J. Modern Education and Computer Science, 2016, 5, 12-18
Subjects: Artificial Intelligence (cs.AI)
A new approach to data stream clustering with the help of an ensemble of
adaptive neuro-fuzzy systems is proposed. The proposed ensemble is formed with
adaptive neuro-fuzzy self-organizing Kohonen maps in a parallel processing
mode. A final result is chosen by the best neuro-fuzzy self-organizing Kohonen
map.
Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Anastasiia O. Deineko
Journal-ref: I.J. Modern Education and Computer Science, 2015, 2, 1-7
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
An architecture of a new neuro-fuzzy system is proposed. The basic idea of
this approach is to tune both synaptic weights and membership functions with
the help of the supervised learning and self-learning paradigms. The approach
to solving the problem has to do with evolving online neuro-fuzzy systems that
can process data under uncertainty conditions. The results prove the
effectiveness of the developed architecture and the learning procedure.
Zhengbing Hu, Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Olena O. Boiko
Journal-ref: I.J. Information Technology and Computer Science, 2016, 10, 1-10
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
An evolving weighted neuro-neo-fuzzy-ANARX model and its learning procedures
are introduced in the article. This system is basically used for time series
forecasting. This system may be considered as a pool of elements that process
data in a parallel manner. The proposed evolving system may provide online
processing data streams.
Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Daria S. Kopaliani
Journal-ref: I.J. Information Technology and Computer Science, 2014, 08, 11-17
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
A new architecture and learning algorithms for the multidimensional hybrid
cascade neural network with neuron pool optimization in each cascade are
proposed in this paper. The proposed system differs from the well-known cascade
systems in its capability to process multidimensional time series in an online
mode, which makes it possible to process non-stationary stochastic and chaotic
signals with the required accuracy. Compared to conventional analogs, the
proposed system provides computational simplicity and possesses both tracking
and filtering capabilities.
Zhengbing Hu, Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Olena O. Boiko
Journal-ref: I.J. Intelligent Systems and Applications, 2016, 9, 1-7
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
Neo-fuzzy elements are used as nodes for an evolving cascade system. The
proposed system can tune both its parameters and architecture in an online
mode. It can be used for solving a wide range of Data Mining tasks (namely time
series forecasting). The evolving cascade system with neo-fuzzy nodes can
process rather large data sets with high speed and effectiveness.
Yevgeniy V. Bodyanskiy, Oleksii K. Tyshchenko, Daria S. Kopaliani
Journal-ref: I.J. Intelligent Systems and Applications, 2015, 02, 21-26
Subjects: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
A modification of the neo-fuzzy neuron is proposed (an extended neo-fuzzy
neuron (ENFN)) that is characterized by improved approximating properties. An
adaptive learning algorithm is proposed that has both tracking and smoothing
properties. An ENFN distinctive feature is its computational simplicity
compared to other artificial neural networks and neuro-fuzzy systems.
Benjamin Bedregal, Humberto Bustince, Eduardo Palmeira, Graçaliz Pereira Dimuro, Javier Fernandez
Subjects: Artificial Intelligence (cs.AI)
In this work we extend to the interval-valued setting the notion of an
overlap functions and we discuss a method which makes use of interval-valued
overlap functions for constructing OWA operators with interval-valued weights.
. Some properties of interval-valued overlap functions and the derived
interval-valued OWA operators are analysed. We specially focus on the
homogeneity and migrativity properties.
Marc Pickett, Rami Al-Rfou, Louis Shao, Chris Tar
Comments: Submission to NIPS workshop on Continual Learning. 4 page extended abstract plus 5 more pages of references, figures, and supplementary material
Subjects: Artificial Intelligence (cs.AI); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
The long-term memory of most connectionist systems lies entirely in the
weights of the system. Since the number of weights is typically fixed, this
bounds the total amount of knowledge that can be learned and stored. Though
this is not normally a problem for a neural network designed for a specific
task, such a bound is undesirable for a system that continually learns over an
open range of domains. To address this, we describe a lifelong learning system
that leverages a fast, though non-differentiable, content-addressable memory
which can be exploited to encode both a long history of sequential episodic
knowledge and semantic knowledge over many episodes for an unbounded number of
domains. This opens the door for investigation into transfer learning, and
leveraging prior knowledge that has been learned over a lifetime of experiences
to new domains.
Shubham Toshniwal, Karen Livescu
Comments: Accepted in SLT 2016
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
We propose an attention-enabled encoder-decoder model for the problem of
grapheme-to-phoneme conversion. Most previous work has tackled the problem via
joint sequence models that require explicit alignments for training. In
contrast, the attention-enabled encoder-decoder model allows for jointly
learning to align and convert characters to phonemes. We explore different
types of attention models, including global and local attention, and our best
models achieve state-of-the-art results on three standard data sets (CMUDict,
Pronlex, and NetTalk).
Tsendsuren Munkhdalai, Hong Yu
Comments: initial submission: 9 pages
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Hypothesis testing is an important cognitive process that supports human
reasoning. In this paper, we introduce a computational hypothesis testing
approach based on memory augmented neural networks. Our approach involves a
hypothesis testing loop that reconsiders and progressively refines a previously
formed hypothesis in order to generate new hypotheses to test. We apply the
proposed approach to language comprehension task by using Neural Semantic
Encoders (NSE). Our NSE models achieve the state-of-the-art results showing an
absolute improvement of 1.2% to 2.6% accuracy over previous results obtained by
single and ensemble systems on standard machine comprehension benchmarks such
as the Children’s Book Test (CBT) and Who-Did-What (WDW) news article datasets.
Hamed R.-Tavakoli, Ali Borji, Jorma Laaksonen, Esa Rahtu
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
This paper presents a novel fixation prediction and saliency modeling
framework based on inter-image similarities and ensemble of Extreme Learning
Machines (ELM). The proposed framework is inspired by two observations, 1) the
contextual information of a scene along with low-level visual cues modulates
attention, 2) the influence of scene memorability on eye movement patterns
caused by the resemblance of a scene to a former visual experience. Motivated
by such observations, we develop a framework that estimates the saliency of a
given image using an ensemble of extreme learners, each trained on an image
similar to the input image. That is, after retrieving a set of similar images
for a given image, a saliency predictor is learnt from each of the images in
the retrieved image set using an ELM, resulting in an ensemble. The saliency of
the given image is then measured in terms of the mean of predicted saliency
value by the ensemble’s members.
Anne Veenendaal, Eddie Jones, Zhao Gang, Elliot Daly, Sumalini Vartak, Rahul Patwardhan
Comments: 7 pages, 4 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
This paper examines use of dynamic probabilistic networks (DPN) for human
action recognition. The actions of lifting objects and walking in the room,
sitting in the room and neutral standing pose were used for testing the
classification. The research used the dynamic interrelation between various
different regions of interest (ROI) on the human body (face, body, arms, legs)
and the time series based events related to the these ROIs. This dynamic links
are then used to recognize the human behavioral aspects in the scene. First a
model is developed to identify the human activities in an indoor scene and this
model is dependent on the key features and interlinks between the various
dynamic events using DPNs. The sub ROI are classified with DPN to associate the
combined interlink with a specific human activity. The recognition accuracy
performance between indoor (controlled lighting conditions) is compared with
the outdoor lighting conditions. The accuracy in outdoor scenes was lower than
the controlled environment.
Siwar Jendoubi (CERT, DRUID, LARODEC), Arnaud Martin (DRUID), Ludovic Liétard (IRISA), Hend Hadji (CERT), Boutheina Yaghlane (LARODEC)
Journal-ref: FLINS, Aug 2016, Roubaix, France. pp.168 – 174, 2016
Subjects: Social and Information Networks (cs.SI); Artificial Intelligence (cs.AI)
In this paper, we propose a new data based model for influence maximization
in online social networks. We use the theory of belief functions to overcome
the data imperfection problem. Besides, the proposed model searches to detect
influencer users that adopt a positive opinion about the product, the idea,
etc, to be propagated. Moreover, we present some experiments to show the
performance of our model.
Charles L. A. Clarke, Gordon V. Cormack, Jimmy Lin, Adam Roegiest
Subjects: Information Retrieval (cs.IR); Digital Libraries (cs.DL); Networking and Internet Architecture (cs.NI)
This paper explores a simple question: How would we provide a high-quality
search experience on Mars, where the fundamental physical limit is
speed-of-light propagation delays on the order of tens of minutes? On Earth,
users are accustomed to nearly instantaneous response times from search
engines. Is it possible to overcome orders-of-magnitude longer latency to
provide a tolerable user experience on Mars? In this paper, we formulate the
searching from Mars problem as a tradeoff between “effort” (waiting for
responses from Earth) and “data transfer” (pre-fetching or caching data on
Mars). The contribution of our work is articulating this design space and
presenting two case studies that explore the effectiveness of baseline
techniques, using publicly available data from the TREC Total Recall and
Sessions Tracks. We intend for this research problem to be aspirational and
inspirational – even if one is not convinced by the premise of Mars
colonization, there are Earth-based scenarios such as searching from a rural
village in India that share similar constraints, thus making the problem worthy
of exploration and attention from researchers.
Neela Avudaiappan, Alexander Herzog, Sneha Kadam, Yuheng Du, Jason Thatcher, Ilya Safro
Subjects: Social and Information Networks (cs.SI); Information Retrieval (cs.IR)
Methods for detecting and summarizing emergent keywords have been extensively
studied since social media and microblogging activities have started to play an
important role in data analysis and decision making. We present a system for
monitoring emergent keywords and summarizing a document stream based on the
dynamic semantic graphs of streaming documents. We introduce the notion of
dynamic eigenvector centrality for ranking emergent keywords, and present an
algorithm for summarizing emergent events that is based on the minimum weight
set cover. We demonstrate our system with an analysis of streaming Twitter data
related to public security events.
Andreas M. Wahl, Gregor Endler, Peter K. Schwab, Sebastian Herbst, Richard Lenz
Comments: in German
Subjects: Databases (cs.DB); Information Retrieval (cs.IR); Social and Information Networks (cs.SI)
In larger organizations, multiple teams of data scientists have to integrate
data from heterogeneous data sources as preparation for data analysis tasks.
Writing effective analytical queries requires data scientists to have in-depth
knowledge of the existence, semantics, and usage context of data sources. Once
gathered, such knowledge is informally shared within a specific team of data
scientists, but usually is neither formalized nor shared with other teams.
Potential synergies remain unused. We therefore introduce a novel approach
which extends data management systems with additional knowledge-sharing
capabilities to facilitate user collaboration without altering established data
analysis workflows. Relevant collective knowledge from the query log is
extracted to support data source discovery and incremental data integration.
Extracted knowledge is formalized and provided at query time.
Alexander Rosenberg Johansen, Jonas Meinertz Hansen, Elias Khazen Obeid, Casper Kaae Sønderby, Ole Winther
Comments: 8 pages, 7 figures
Subjects: Computation and Language (cs.CL)
Most existing Neural Machine Translation models use groups of characters or
whole words as their unit of input and output. We propose a model with a
hierarchical char2word encoder, that takes individual characters both as input
and output. We first argue that this hierarchical representation of the
character encoder reduces computational complexity, and show that it improves
translation performance. Secondly, by qualitatively studying attention plots
from the decoder we find that the model learns to compress common words into a
single embedding whereas rare words, such as names and places, are represented
character by character.
Graham Neubig
Comments: To Appear in the Workshop on Asian Translation (WAT). arXiv admin note: text overlap with arXiv:1606.02006
Subjects: Computation and Language (cs.CL)
This year, the Nara Institute of Science and Technology (NAIST)/Carnegie
Mellon University (CMU) submission to the Japanese-English translation track of
the 2016 Workshop on Asian Translation was based on attentional neural machine
translation (NMT) models. In addition to the standard NMT model, we make a
number of improvements, most notably the use of discrete translation lexicons
to improve probability estimates, and the use of minimum risk training to
optimize the MT system for BLEU score. As a result, our system achieved the
highest translation evaluation scores for the task.
Shubham Toshniwal, Karen Livescu
Comments: Accepted in SLT 2016
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
We propose an attention-enabled encoder-decoder model for the problem of
grapheme-to-phoneme conversion. Most previous work has tackled the problem via
joint sequence models that require explicit alignments for training. In
contrast, the attention-enabled encoder-decoder model allows for jointly
learning to align and convert characters to phonemes. We explore different
types of attention models, including global and local attention, and our best
models achieve state-of-the-art results on three standard data sets (CMUDict,
Pronlex, and NetTalk).
Anoop Kunchukuttan, Pushpak Bhattacharyya
Comments: A earlier version of this paper is under review at EACL 2107. (10 pages, 2 figures, 9 tables)
Subjects: Computation and Language (cs.CL)
We explore the use of segments learnt using Byte Pair Encoding (referred to
as BPE units) as basic units for statistical machine translation between
related languages and compare it with orthographic syllables, which are
currently the best performing basic units for this translation task. BPE
identifies the most frequent character sequences as basic units, while
orthographic syllables are linguistically motivated pseudo-syllables. We show
that BPE units outperform orthographic syllables as units of translation,
showing up to 11% increase in BLEU scores. In addition, BPE can be applied to
any writing system, while orthographic syllables can be used only for languages
whose writing systems use vowel representations. We show that BPE units
outperform word and morpheme level units for translation involving languages
like Urdu, Japanese whose writing systems do not use vowels (either completely
or partially). Across many language pairs, spanning multiple language families
and types of writing systems, we show that translation with BPE segments
outperforms orthographic syllables, especially for morphologically rich
languages.
Jeaneth Machicao, Edilson A. Corrêa Jr., Gisele H. B. Miranda, Diego R. Amancio, Odemir M. Bruno
Subjects: Computation and Language (cs.CL)
The authorship attribution is a problem of considerable practical and
technical interest. Several methods have been designed to infer the authorship
of disputed documents in multiple contexts. While traditional statistical
methods based solely on word counts and related measurements have provided a
simple, yet effective solution in particular cases; they are prone to
manipulation. Recently, texts have been successfully modeled as networks, where
words are represented by nodes linked according to textual similarity
measurements. Such models are useful to identify informative topological
patterns for the authorship recognition task. However, there is no consensus on
which measurements should be used. Thus, we proposed a novel method to
characterize text networks, by considering both topological and dynamical
aspects of networks. Using concepts and methods from cellular automata theory,
we devised a strategy to grasp informative spatio-temporal patterns from this
model. Our experiments revealed an outperformance over traditional analysis
relying only on topological measurements. Remarkably, we have found a
dependence of pre-processing steps (such as the lemmatization) on the obtained
results, a feature that has mostly been disregarded in related works. The
optimized results obtained here pave the way for a better characterization of
textual networks.
Tsendsuren Munkhdalai, Hong Yu
Comments: initial submission: 9 pages
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Machine Learning (stat.ML)
Hypothesis testing is an important cognitive process that supports human
reasoning. In this paper, we introduce a computational hypothesis testing
approach based on memory augmented neural networks. Our approach involves a
hypothesis testing loop that reconsiders and progressively refines a previously
formed hypothesis in order to generate new hypotheses to test. We apply the
proposed approach to language comprehension task by using Neural Semantic
Encoders (NSE). Our NSE models achieve the state-of-the-art results showing an
absolute improvement of 1.2% to 2.6% accuracy over previous results obtained by
single and ensemble systems on standard machine comprehension benchmarks such
as the Children’s Book Test (CBT) and Who-Did-What (WDW) news article datasets.
Georgios P. Spithourakis, Steffen E. Petersen, Sebastian Riedel
Comments: Accepted at the 7th International Workshop on Health Text Mining and Information Analysis (LOUHI) EMNLP 2016
Subjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Neural and Evolutionary Computing (cs.NE)
Assisted text input techniques can save time and effort and improve text
quality. In this paper, we investigate how grounded and conditional extensions
to standard neural language models can bring improvements in the tasks of word
prediction and completion. These extensions incorporate a structured knowledge
base and numerical values from the text into the context used to predict the
next word. Our automated evaluation on a clinical dataset shows extended models
significantly outperform standard models. Our best system uses both
conditioning and grounding, because of their orthogonal benefits. For word
prediction with a list of 5 suggestions, it improves recall from 25.03% to
71.28% and for word completion it improves keystroke savings from 34.35% to
44.81%, where theoretical bound for this dataset is 58.78%. We also perform a
qualitative investigation of how models with lower perplexity occasionally fare
better at the tasks. We found that at test time numbers have more influence on
the document level than on individual word probabilities.
Bonggun Shin, Timothy Lee, Jinho D. Choi
Subjects: Computation and Language (cs.CL)
With the advent of word embeddings, lexicons are no longer fully utilized for
sentiment analysis although they still provide important features in the
traditional setting. This paper introduces a novel approach to sentiment
analysis that integrates lexicon embeddings and an attention mechanism into
Convolutional Neural Networks. Our approach performs separate convolutions for
word and lexicon embeddings and provides a global view of the document using
attention. Our models are experimented on both the SemEval’16 Task 4 dataset
and the Stanford Sentiment Treebank, and show comparative or better results
against the existing state-of-the-art systems. Our analysis shows that lexicon
embeddings allow to build high-performing models with much smaller word
embeddings, and the attention mechanism effectively dims out noisy words for
sentiment analysis.
Mohammad Sadegh Rasooli, Michael Collins
Subjects: Computation and Language (cs.CL)
We describe a simple but effective method for cross-lingual syntactic
transfer of dependency parsers, in the scenario where a large amount of
translation data is not available.The method makes use of three steps: 1) a
method for deriving cross-lingual word clusters, that can then be used in a
multilingual parser; 2) a method for transferring lexical information from a
target language to source language treebanks; 3) a method for integrating these
steps with the density-driven annotation projection method of Rasooli and
Collins(2015). Experiments show improvements over the state-of-the-art in
several languages used in previous work (Rasooli and Collins, 2015;Zhang and
Barzilay, 2015; Ammar et al.,2016), in a setting where the only source of
translation data is the Bible, a considerably smaller corpus than the Europarl
corpus used in previous work. Results using the Europarl corpus as a source of
translation data show additional improvements over the results of Rasooli and
Collins (2015). We conclude with results on 38 datasets (26 languages) from the
Universal Dependencies corpora: 13 datasets(10 languages) have unlabeled
attachment ac-curacies of 80% or higher; the average unlabeled accuracy on the
38 datasets is 74.8%.
Rik Koncel-Kedziorski, Ioannis Konstas, Luke Zettlemoyer, Hannaneh Hajishirzi
Comments: To appear EMNLP 2016
Subjects: Computation and Language (cs.CL)
Texts present coherent stories that have a particular theme or overall
setting, for example science fiction or western. In this paper, we present a
text generation method called {it rewriting} that edits existing
human-authored narratives to change their theme without changing the underlying
story. We apply the approach to math word problems, where it might help
students stay more engaged by quickly transforming all of their homework
assignments to the theme of their favorite movie without changing the math
concepts that are being taught. Our rewriting method uses a two-stage decoding
process, which proposes new words from the target theme and scores the
resulting stories according to a number of factors defining aspects of
syntactic, semantic, and thematic coherence. Experiments demonstrate that the
final stories typically represent the new theme well while still testing the
original math concepts, outperforming a number of baselines. We also release a
new dataset of human-authored rewrites of math word problems in several themes.
Markus Fidler, Brenton Walker, Yuming Jiang
Comments: arXiv admin note: text overlap with arXiv:1512.08354
Subjects: Performance (cs.PF); Distributed, Parallel, and Cluster Computing (cs.DC)
Multi-server systems have received increasing attention with important
implementations such as Google MapReduce, Hadoop, and Spark. Common to these
systems are a fork operation, where jobs are first divided into tasks that are
processed in parallel, and a later join operation, where completed tasks wait
until the results of all tasks of a job can be combined and the job leaves the
system. The synchronization constraint of the join operation makes the analysis
of fork-join systems challenging and few explicit results are known. In this
work, we model fork-join systems using a max-plus server model that enables us
to derive statistical bounds on waiting and sojourn times for general arrival
and service time processes. We contribute end-to-end delay bounds for
multi-stage fork-join networks that grow in (mathcal{O}(h ln k)) for (h)
fork-join stages, each with (k) parallel servers. We perform a detailed
comparison of different multi-server configurations and highlight their pros
and cons. We also include an analysis of single-queue fork-join systems that
are non-idling and achieve a fundamental performance gain, and compare these
results to both simulation and a live Spark system.
Alexander Ulanov, Andrey Simanovsky, Manish Marwah
Comments: 6 pages, 4 figures
Subjects: Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)
Present day machine learning is computationally intensive and processes large
amounts of data. It is implemented in a distributed fashion in order to address
these scalability issues. The work is parallelized across a number of computing
nodes. It is usually hard to estimate in advance how many nodes to use for a
particular workload. We propose a simple framework for estimating the
scalability of distributed machine learning algorithms. We measure the
scalability by means of the speedup an algorithm achieves with more nodes. We
propose time complexity models for gradient descent and graphical model
inference. We validate our models with experiments on deep learning training
and belief propagation. This framework was used to study the scalability of
machine learning algorithms in Apache Spark.
Ugo Rosolia, Ashwin Carvalho, Francesco Borrelli
Comments: Submitted to ACC
Subjects: Learning (cs.LG); Optimization and Control (math.OC)
A novel learning Model Predictive Control technique is applied to the
autonomous racing problem. The goal of the controller is to minimize the time
to complete a lap. The proposed control strategy uses the data from previous
laps to improve its performance while satisfying safety requirements. Moreover,
a system identification technique is proposed to estimate the vehicle dynamics.
Simulation results with the high fidelity simulator software CarSim show the
effectiveness of the proposed control scheme.
Alexander Ulanov, Andrey Simanovsky, Manish Marwah
Comments: 6 pages, 4 figures
Subjects: Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC)
Present day machine learning is computationally intensive and processes large
amounts of data. It is implemented in a distributed fashion in order to address
these scalability issues. The work is parallelized across a number of computing
nodes. It is usually hard to estimate in advance how many nodes to use for a
particular workload. We propose a simple framework for estimating the
scalability of distributed machine learning algorithms. We measure the
scalability by means of the speedup an algorithm achieves with more nodes. We
propose time complexity models for gradient descent and graphical model
inference. We validate our models with experiments on deep learning training
and belief propagation. This framework was used to study the scalability of
machine learning algorithms in Apache Spark.
Kien Do, Truyen Tran, Svetha Venkatesh
Comments: 9 pages
Subjects: Learning (cs.LG); Databases (cs.DB)
Anomalies are those deviating from the norm. Unsupervised anomaly detection
often translates to identifying low density regions. Major problems arise when
data is high-dimensional and mixed of discrete and continuous attributes. We
propose MIXMAD, which stands for MIXed data Multilevel Anomaly Detection, an
ensemble method that estimates the sparse regions across multiple levels of
abstraction of mixed data. The hypothesis is for domains where multiple data
abstractions exist, a data point may be anomalous with respect to the raw
representation or more abstract representations. To this end, our method
sequentially constructs an ensemble of Deep Belief Nets (DBNs) with varying
depths. Each DBN is an energy-based detector at a predefined abstraction level.
At the bottom level of each DBN, there is a Mixed-variate Restricted Boltzmann
Machine that models the density of mixed data. Predictions across the ensemble
are finally combined via rank aggregation. The proposed MIXMAD is evaluated on
high-dimensional realworld datasets of different characteristics. The results
demonstrate that for anomaly detection, (a) multilevel abstraction of
high-dimensional and mixed data is a sensible strategy, and (b) empirically,
MIXMAD is superior to popular unsupervised detection methods for both
homogeneous and mixed data.
Mariusz Bojarski, Anna Choromanska, Krzysztof Choromanski, Francois Fagan, Cedric Gouy-Pailler, Anne Morvan, Nouri Sakr, Tamas Sarlos, Jamal Atif
Comments: arXiv admin note: substantial text overlap with arXiv:1605.09046
Subjects: Learning (cs.LG)
We consider an efficient computational framework for speeding up several
machine learning algorithms with almost no loss of accuracy. The proposed
framework relies on projections via structured matrices that we call Structured
Spinners, which are formed as products of three structured matrix-blocks that
incorporate rotations. The approach is highly generic, i.e. i) structured
matrices under consideration can either be fully-randomized or learned, ii) our
structured family contains as special cases all previously considered
structured schemes, iii) the setting extends to the non-linear case where the
projections are followed by non-linear functions, and iv) the method finds
numerous applications including kernel approximations via random feature maps,
dimensionality reduction algorithms, new fast cross-polytope LSH techniques,
deep learning, convex optimization algorithms via Newton sketches, quantization
with random projection trees, and more. The proposed framework comes with
theoretical guarantees characterizing the capacity of the structured model in
reference to its unstructured counterpart and is based on a general theoretical
principle that we describe in the paper. As a consequence of our theoretical
analysis, we provide the first theoretical guarantees for one of the most
efficient existing LSH algorithms based on the HD3HD2HD1 structured matrix
[Andoni et al., 2015]. The exhaustive experimental evaluation confirms the
accuracy and efficiency of structured spinners for a variety of different
applications.
Tomasz Kornuta, Kamil Rocki
Comments: Paper submitted to special session on Machine Intelligence organized during 23rd International AUTOMATION Conference
Subjects: Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG)
The paper focuses on the problem of learning saccades enabling visual object
search. The developed system combines reinforcement learning with a neural
network for learning to predict the possible outcomes of its actions. We
validated the solution in three types of environment consisting of
(pseudo)-randomly generated matrices of digits. The experimental verification
is followed by the discussion regarding elements required by systems mimicking
the fovea movement and possible further research directions.
Stephanie Allen, David Madras, Ye Ye, Greg Zanotti
Subjects: Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG); Machine Learning (stat.ML)
Body-worn video (BWV) cameras are increasingly utilized by police departments
to provide a record of police-public interactions. However, large-scale BWV
deployment produces terabytes of data per week, necessitating the development
of effective computational methods to identify salient changes in video. In
work carried out at the 2016 RIPS program at IPAM, UCLA, we present a novel
two-stage framework for video change-point detection. First, we employ
state-of-the-art machine learning methods including convolutional neural
networks and support vector machines for scene classification. We then develop
and compare change-point detection algorithms utilizing mean squared-error
minimization, forecasting methods, hidden Markov models, and maximum likelihood
estimation to identify noteworthy changes. We test our framework on detection
of vehicle exits and entrances in a BWV data set provided by the Los Angeles
Police Department and achieve over 90% recall and nearly 70% precision —
demonstrating robustness to rapid scene changes, extreme luminance differences,
and frequent camera occlusions.
Hao Dong, Akara Supratak, Wei Pan, Chao Wu, Paul M. Matthews, Yike Guo
Comments: Under review of IEEE Transactions on Neural Systems and Rehabilitation Engineering since Jun 2016
Subjects: Neurons and Cognition (q-bio.NC); Computer Vision and Pattern Recognition (cs.CV); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
This paper proposes a practical approach to addressing limitations posed by
use of single active electrodes in applications for sleep stage classification.
Electroencephalography (EEG)-based characterizations of sleep stage progression
contribute the diagnosis and monitoring of the many pathologies of sleep.
Several prior reports have explored ways of automating the analysis of sleep
EEG and of reducing the complexity of the data needed for reliable
discrimination of sleep stages in order to make it possible to perform sleep
studies at lower cost in the home (rather than only in specialized clinical
facilities). However, these reports have involved recordings from electrodes
placed on the cranial vertex or occiput, which can be uncomfortable or
difficult for subjects to position. Those that have utilized single EEG
channels which contain less sleep information, have showed poor classification
performance. We have taken advantage of Rectifier Neural Network for feature
detection and Long Short-Term Memory (LSTM) network for sequential data
learning to optimize classification performance with single electrode
recordings. After exploring alternative electrode placements, we found a
comfortable configuration of a single-channel EEG on the forehead and have
shown that it can be integrated with additional electrodes for simultaneous
recording of the electroocuolgram (EOG). Evaluation of data from 62 people
(with 494 hours sleep) demonstrated better performance of our analytical
algorithm for automated sleep classification than existing approaches using
vertex or occipital electrode placements. Use of this recording configuration
with neural network deconvolution promises to make clinically indicated home
sleep studies practical.
Marc Pickett, Rami Al-Rfou, Louis Shao, Chris Tar
Comments: Submission to NIPS workshop on Continual Learning. 4 page extended abstract plus 5 more pages of references, figures, and supplementary material
Subjects: Artificial Intelligence (cs.AI); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
The long-term memory of most connectionist systems lies entirely in the
weights of the system. Since the number of weights is typically fixed, this
bounds the total amount of knowledge that can be learned and stored. Though
this is not normally a problem for a neural network designed for a specific
task, such a bound is undesirable for a system that continually learns over an
open range of domains. To address this, we describe a lifelong learning system
that leverages a fast, though non-differentiable, content-addressable memory
which can be exploited to encode both a long history of sequential episodic
knowledge and semantic knowledge over many episodes for an unbounded number of
domains. This opens the door for investigation into transfer learning, and
leveraging prior knowledge that has been learned over a lifetime of experiences
to new domains.
Qiyang Li, Jingxing Qian, Zining Zhu, Xuchan Bao, Mohamed K. Helwa, Angela P. Schoellig
Comments: 8 pages, 13 figures, Preprint submitted to 2017 IEEE International Conference on Robotics and Automation
Subjects: Robotics (cs.RO); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE); Systems and Control (cs.SY)
Trajectory tracking control for quadrotors is important for applications
ranging from surveying and inspection, to film making. However, designing and
tuning classical controllers, such as proportional-integral-derivative (PID)
controllers, to achieve high tracking precision can be time-consuming and
difficult, due to hidden dynamics and other non-idealities. The Deep Neural
Network (DNN), with its superior capability of approximating abstract,
nonlinear functions, proposes a novel approach for enhancing trajectory
tracking control. This paper presents a DNN-based algorithm that improves the
tracking performance of a classical feedback controller. Given a desired
trajectory, the DNNs provide a tailored input to the controller based on their
gained experience. The input aims to achieve a unity map between the desired
and the output trajectory. The motivation for this work is an interactive
“fly-as-you-draw” application, in which a user draws a trajectory on a mobile
device, and a quadrotor instantly flies that trajectory with the DNN-enhanced
control system. Experimental results demonstrate that the proposed approach
improves the tracking precision for user-drawn trajectories after the DNNs are
trained on selected periodic trajectories, suggesting the method’s potential in
real-world applications. Tracking errors are reduced by around 40-50 % for both
training and testing trajectories from users, highlighting the DNNs’ capability
of generalizing knowledge.
Jimmy Ba, Geoffrey Hinton, Volodymyr Mnih, Joel Z. Leibo, Catalin Ionescu
Subjects: Machine Learning (stat.ML); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
Until recently, research on artificial neural networks was largely restricted
to systems with only two types of variable: Neural activities that represent
the current or recent input and weights that learn to capture regularities
among inputs, outputs and payoffs. There is no good reason for this
restriction. Synapses have dynamics at many different time-scales and this
suggests that artificial neural networks might benefit from variables that
change slower than activities but much faster than the standard weights. These
“fast weights” can be used to store temporary memories of the recent past and
they provide a neurally plausible way of implementing the type of attention to
the past that has recently proved very helpful in sequence-to-sequence models.
By using fast weights we can avoid the need to store copies of neural activity
patterns.
Cheng Li, Xiaoxiao Guo, Qiaozhu Mei
Subjects: Social and Information Networks (cs.SI); Learning (cs.LG)
The topological (or graph) structures of real-world networks are known to be
predictive of multiple dynamic properties of the networks. Conventionally, a
graph structure is represented using an adjacency matrix or a set of
hand-crafted structural features. These representations either fail to
highlight local and global properties of the graph or suffer from a severe loss
of structural information. There lacks an effective graph representation, which
hinges the realization of the predictive power of network structures.
In this study, we propose to learn the represention of a graph, or the
topological structure of a network, through a deep learning model. This
end-to-end prediction model, named DeepGraph, takes the input of the raw
adjacency matrix of a real-world network and outputs a prediction of the growth
of the network. The adjacency matrix is first represented using a graph
descriptor based on the heat kernel signature, which is then passed through a
multi-column, multi-resolution convolutional neural network. Extensive
experiments on five large collections of real-world networks demonstrate that
the proposed prediction model significantly improves the effectiveness of
existing methods, including linear or nonlinear regressors that use
hand-crafted features, graph kernels, and competing deep learning methods.
Mansoor I. Yousefi
Comments: The abstract in the PDF file is longer. Arxiv limits the abstract field to 1,920 characters
Subjects: Information Theory (cs.IT)
It is shown that signal energy is the only available degree-of-freedom (DOF)
for fiber-optic transmission as the input power tends to infinity. With (n)
signal DOFs at the input, (n-1) DOFs are asymptotically lost to signal-noise
interactions. The main observation is that, nonlinearity introduces a
multiplicative noise in the channel, similar to fading in wireless channels.
The channel is viewed in the spherical coordinate system, where signal vector
(underline{X}inmathbb{C}^n) is represented in terms of its norm
(|underline{X}|) and direction (underline{hat{X}}). The multiplicative noise
causes signal direction (underline{hat{X}}) to vary randomly on the surface
of the unit ((2n-1))-sphere in (mathbb{C}^{n}), in such a way that the
effective area of the support of (underline{hat{X}}) does not vanish as
(|underline{X}|
ightarrowinfty). On the other hand, the surface area of the
sphere is finite, so that (underline{hat{X}}) carries finite information.
This observation is used to show several results. Firstly, let (mathcal
C(mathcal P)) be the capacity of a discrete-time periodic model of the optical
fiber with distributed noise and frequency-dependent loss, as a function of the
average input power (mathcal P). It is shown that asymptotically as (mathcal
P
ightarrowinfty), (mathcal C=frac{1}{n}logigl(logmathcal Pigr)+c),
where (n) is the dimension of the input signal space and (c) is a bounded
number. In particular, (lim_{mathcal P
ightarrowinfty}mathcal C(mathcal
P)=infty) in finite-dimensional periodic models. Secondly, it is shown that
capacity saturates to a constant in infinite-dimensional models where
(n=infty).
Carlos Galindo, Olav Geil, Fernando Hernando, Diego Ruano
Comments: 29 pages
Subjects: Information Theory (cs.IT); Commutative Algebra (math.AC)
Two new constructions of linear code pairs (C_2 subset C_1) are given for
which the codimension and the relative minimum distances (M_1(C_1,C_2)),
(M_1(C_2^perp,C_1^perp)) are good. By this we mean that for any two out of
the three parameters the third parameter of the constructed code pair is large.
Such pairs of nested codes are indispensable for the determination of good
linear ramp secret sharing schemes [35]. They can also be used to ensure
reliable communication over asymmetric quantum channels [47]. The new
constructions result from carefully applying the Feng-Rao bounds [18,27] to a
family of codes defined from multivariate polynomials and Cartesian product
point sets.
Can Xiang
Subjects: Information Theory (cs.IT)
Recently, a (q)-polynomial approach to the construction and analysis of
cyclic codes over (gf(q)) was given by Ding and Ling. The objective of this
paper is to give another (q)-polynomial approach to all cyclic codes over
(gf(q)).
Can Xiang
Subjects: Information Theory (cs.IT)
Linear codes are widely employed in communication systems, consumer
electronics, and storage devices. All linear codes over finite fields can be
generated by a generator matrix. Due to this, the generator matrix approach is
called a fundamental construction of linear codes. This is the only known
construction method that can produce all linear codes over finite fields.
Recently, a defining-set construction of linear codes over finite fields has
attracted a lot of attention, and have been employed to produce a huge number
of classes of linear codes over finite fields. It was claimed that this
approach can also generate all linear codes over finite fields. But so far, no
proof of this claim is given in the literature. The objective of this paper is
to prove this claim, and confirm that the defining-set approach is indeed a
fundamental approach to constructing all linear codes over finite fields. As a
byproduct, a trace representation of all linear codes over finite fields is
presented.
Wengu Chen, Yaling Li
Subjects: Information Theory (cs.IT)
In this paper, we consider the recovery of block sparse signals, whose
nonzero entries appear in blocks (or clusters) rather than spread arbitrarily
throughout the signal, from incomplete linear measurement. A high order
sufficient condition based on block RIP is obtained to guarantee the stable
recovery of all block sparse signals in the presence of noise, and robust
recovery when signals are not exactly block sparse via mixed (l_{2}/l_{1})
minimization. Moreover, a concrete example is established to ensure the
condition is sharp. The significance of the results presented in this paper
lies in the fact that recovery may be possible under more general conditions by
exploiting the block structure of the sparsity pattern instead of the
conventional sparsity pattern.
Amir Aminjavaheri, Arman Farhang, Linda E. Doyle, Behrouz Farhang-Boroujeny
Subjects: Information Theory (cs.IT)
We perform an asymptotic study on the performance of filter bank multicarrier
(FBMC) in the context of massive multi-input multi-output (MIMO). We show that
the signal-to-interference-plus-noise ratio (SINR) cannot grow unboundedly by
increasing the number of base station (BS) antennas, and is upper bounded by a
certain deterministic value. This is a result of the correlation between the
multi-antenna combining tap values and the channel impulse responses between
the terminals and the BS antennas. To solve this problem, we introduce a simple
FBMC prototype filter design method that removes this correlation, enabling us
to achieve arbitrarily large SINR values by increasing the number of BS
antennas.
Tong-Xing Zheng, Hui-Ming Wang, Qian Yang, Moon Ho Lee
Comments: Journal paper, double column, 15 pages, 11 figures, accepted to appear on IEEE Transactions on Wireless Communications
Subjects: Information Theory (cs.IT)
In this paper, we study the benefits of full-duplex (FD) receiver jamming in
enhancing the physical-layer security of a two-tier decentralized wireless
network with each tier deployed with a large number of pairs of a
single-antenna transmitter and a multi-antenna receiver. In the underlying
tier, the transmitter sends unclassified information, and the receiver works in
the halfduplex (HD) mode receiving the desired signal. In the overlaid tier,
the transmitter deliveries confidential information in the presence of randomly
located eavesdroppers, and the receiver works in the FD mode radiating jamming
signals to confuse eavesdroppers and receiving the desired signal
simultaneously. We provide a comprehensive performance analysis and network
design under a stochastic geometry framework. Specifically, we consider the
scenarios where each FD receiver uses single- and multi-antenna jamming, and
analyze the connection probability and the secrecy outage probability of a
typical FD receiver by providing accurate expressions and more tractable
approximations for the two metrics. We further determine the optimal deployment
of the FD-mode tier in order to maximize networkwide secrecy throughput subject
to constraints including the given dual probabilities and the network-wide
throughput of the HD-mode tier. Numerical results are demonstrated to verify
our theoretical findings, and show that network-wide secrecy throughput is
significantly improved by properly deploying the FD-mode tier.
Abbas Kazemipour, Ji Liu, Patrick Kanold, Min Wu, Behtash Babadi
Comments: 2016 IEEE Global Conference on Signal and Information Processing (GlobalSIP), Dec. 7-9, 2016, Washington D.C
Subjects: Machine Learning (stat.ML); Computer Vision and Pattern Recognition (cs.CV); Information Theory (cs.IT); Dynamical Systems (math.DS); Statistics Theory (math.ST)
In this paper, we consider linear state-space models with compressible
innovations and convergent transition matrices in order to model
spatiotemporally sparse transient events. We perform parameter and state
estimation using a dynamic compressed sensing framework and develop an
efficient solution consisting of two nested Expectation-Maximization (EM)
algorithms. Under suitable sparsity assumptions on the innovations, we prove
recovery guarantees and derive confidence bounds for the state estimates. We
provide simulation studies as well as application to spike deconvolution from
calcium imaging data which verify our theoretical results and show significant
improvement over existing algorithms.