作者：奕欣来源：雷锋网注：本文为雷锋网针对《Deep-Learning-Papers-Reading-Roadmap》整理而成，英文原文首发于 GitHub，作者 songrotek。原文链接：1.深度学习零基础进阶大法2.深度学习零基础进阶第二弹3.深度学习零基础进阶第三弹4.深度学习零基础进阶第四弹1.深度学习零基础进阶大法0. 深度学习的“圣经”1. 调研2. 建立深度学习的知识网3. ImageNet 革命2.深度学习零基础进阶第二弹1. 深度学习模型2. 深度学习优化3. 无监督学习/深层生成模型3.深度学习零基础进阶第三弹4. 循环神经网络/序列到序列模式5. 神经图灵机6. 深度强化学习7. 无监督特征学习8. 一步之遥4.深度学习零基础进阶第四弹0. 主要集中在NLP ...继续阅读 (13)

作者：刘帝伟原文：机器学习算法比较本文主要回顾下几个常用算法的适应场景及其优缺点！（提示：部分内容摘自网络）。机器学习算法太多了，分类、回归、聚类、推荐、图像识别领域等等，要想找到一个合适算法真的不容易，所以在实际应用中，我们一般都是采用启发式学习方式来实验。通常最开始我们都会选择大家普遍认同的算法，诸如SVM，GBDT，Adaboost，现在深度学习很火热，神经网络也是一个不错的选择。假如你在乎精度（accuracy）的话，最好的方法就是通过交叉验证（cross-validation）对各个算法一个个地进行测试，进行比较，然后调整参数确保每个算法达到最优解，最后选择最好的一个。但是如果你只是在寻找一个“足够好”的算法来解决你的问题，或者这里有些技巧可以参考，下面来分析下各个算法的优缺点，基于算法的优缺点，更易于我们去选择它。偏差&方差在统计学中，一个模型好坏，是根据偏差和方差来衡量的，所以我们先来普及一下偏差(bias)和方差(variance)：偏差：描述的是预测值（估计值）的期望E’与真实值Y之间的差距。偏差越大，越偏离真实数据。$$Bias [\hat{f}(x)] = E [\hat{f}(x)] – f(x) \\tag{1} \label{1}$$方差：描述的是预测值P的变化范围，离散程度，是预测值的方差，也就是离其期望值E的距离。方差越大，数据的 ...继续阅读 (29)

机器学习大数据必不可少，大数据的可视化分析一直是难以解决好的问题，Google 今年12月8日开源的 Embedding Projector 期待解决这个问题。这款用于交互式可视化和高维数据分析的网页工具 Embedding Projector 是 TensorFlow 的一部分，同时，谷歌也在 projector.tensorflow.org 上开放了一个可以单独使用的版本，让用户无需安装即可使用高维数据可视化官方报道：https://research.googleblog.com/2016/12/open-sourcing-embedding-projector-tool.htmlEmbedding Projector 官方地址：https://www.tensorflow.org/versions/master/how_tos/embedding_viz/index.html无需安装即可单独使用的版本：http://projector.tensorflow.org/相关论文地址：Embedding Projector: Interactive Visualization andInterpretation of Embeddings论文摘要：​Embeddings are ubiquitous in machine learning, appearing in recomme ...继续阅读 (43)

作者：rushter来源：https://github.com/rushter/MLAlgorithmsMachine learning algorithmsA collection of minimal and clean implementations of machine learning algorithms.Why?This project is targeting people who want to learn internals of ml algorithms or implement them from scratch.The code is much easier to follow than the optimized libraries and easier to play with.All algorithms are implemented in Python, using numpy, scipy and autograd.Implemented:Deep learning (MLP, CNN, RNN, LSTM)Linear regression, logistic regressionRandom ForestsSVM with kernels (Linear, Poly, RBF)K-MeansGaussian Mixture ModelK- ...继续阅读 (6)

Neural and Evolutionary ComputingTowards better decoding and language model integration in sequence to sequence modelsJan Chorowski,Navdeep JaitlySubjects:Neural and Evolutionary Computing (cs.NE); Computation and Language (cs.CL); Learning (cs.LG); Machine Learning (stat.ML)The recently proposed Sequence-to-Sequence (seq2seq) framework advocatesreplacing complex data processing pipelines, such as an entire automatic speechrecognition system, with a single neural network trained in an end-to-endfashion. In this contribution, we analyse an attention-based seq2seq speechrecognition system that ...继续阅读 (11)

Neural and Evolutionary ComputingNeural Turing Machines: Convergence of Copy TasksJanez AlešComments: Predictor weights can be provided upon requestSubjects:Neural and Evolutionary Computing (cs.NE)The architecture of neural Turing machines is differentiable end to end andis trainable with gradient descent methods. Due to their large unfolded depthNeural Turing Machines are hard to train and because of their linear access ofcomplete memory they do not scale. Other architectures have been studied toovercome these difficulties. In this report we focus on improving the qualityof prediction ...继续阅读 (6)

英文官网：http://www.deeplearningbook.org/翻译链接：https://github.com/exacity/deeplearningbook-chinese翻译作者：张志华老师团队Deep Learning 中文翻译经过3个多月，我们终于完成了翻译草稿。当然这是草稿中的草稿，我们会不断改进，就像梯度下降一样，要迭代好几轮才能找到一个不错的解。目前的版本是直译版，尽可能地保留原书中的每一个字。 如Inventors have long dreamed of creating machines that think. This desire dates back to at least the time of ancient Greece. 自古以来,创造者就梦想着创造能思考的机器。这个愿望至少可以追溯到古希腊的时期。之后我们可能会翻译成远在古希腊时期,创造者就梦想着创造能思考的机器。语句流畅度的提高必然伴随精度下降（我们水平有限）。因此我们需要ensemble，需要大家的建议。 对应的翻译者：第1、4、7、10、14、20章及第12.4、12.5节由@swordyork 负责第2、5、8、11、15、18章由@liber145 负责第3、6、9章由@KevinLee1110 负责第13、16、17、19章及第12.1至12.3节由@futianfan 负 ...继续阅读 (4)

官网：http://www.mlyearning.org/初稿下载地址：https://gallery.mailchimp.com/dc3a7ef4d750c0abfc19202a3/files/Machine_Learning_Yearning_V0.5_01.pdfhttps://gallery.mailchimp.com/dc3a7ef4d750c0abfc19202a3/files/Machine_Learning_Yearning_V0.5_02.pdf知乎讨论：如何评价吴恩达的新书machine learning yearning？ ...继续阅读 (3)

Neural and Evolutionary ComputingSemi-Supervised Learning with the Deep Rendering Mixture ModelTan Nguyen,Wanjia Liu,Ethan Perez,Richard G. Baraniuk,Ankit B. PatelSubjects:Machine Learning (stat.ML); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)Semi-supervised learning algorithms reduce the high cost of acquiring labeledtraining data by using both labeled and unlabeled data during learning. DeepConvolutional Networks (DCNs) have achieved great success in supervised tasksand as such have been widely employed in the semi-supervised learning. In thispaper we leverage the recently de ...继续阅读 (6)

Neural and Evolutionary ComputingBrainFrame: A heterogeneous accelerator platform for neuron simulationsGeorgios Smaragdos,Georgios Chatzikonstantis,Rahul Kukreja,Harrys Sidiropoulos,Dimitrios Rodopoulos,Ioannis Sourdis,Zaid Al-Ars,Christoforos Kachris,Dimitrios Soudris,Chris I. De Zeeuw,Christos StrydisComments: 17 pages, 19 figures, 4 tablesSubjects:Neural and Evolutionary Computing (cs.NE); Distributed, Parallel, and Cluster Computing (cs.DC)Objective: The advent of High-Performance Computing (HPC) in recent years hasled to its increasing use in brain study through computational models. The ...继续阅读 (6)

来源：https://mostafa-samir.github.io/作者：Mostafa SamirMachine Learning Theory – Part 1: IntroductionMotivationWho is this Series for?PrerequisitesCautionsFormalizing the Learning ProblemThe Target FunctionThe HypothesisThe Loss FunctionThe Generalization ErrorIs the Learning Problem Solvable?References and Additional ReadingsShareCommentsMachine Learning Theory – Part 2: Generalization BoundsIndependently, and Identically DistributedThe Law of Large NumbersHoeffding’s inequalityGeneralization Bound: 1st AttemptExamining the Independence AssumptionThe Symmetrization LemmaThe Growth Fun ...继续阅读 (3)

Neural and Evolutionary ComputingProbabilistic Neural ProgramsKenton W. Murray,Jayant KrishnamurthyComments: Appears in NAMPI workshop at NIPS 2016Subjects:Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Learning (cs.LG)We present probabilistic neural programs, a framework for program inductionthat permits flexible specification of both a computational model and inferencealgorithm while simultaneously enabling the use of deep neural networks.Probabilistic neural programs combine a computation graph for specifying aneural network with an operator for weighted nondete ...继续阅读 (4)

Neural and Evolutionary ComputingNew Ideas for Brain Modelling 3Kieran GreerSubjects:Neural and Evolutionary Computing (cs.NE)This paper considers a process for the creation and subsequent firing ofsequences of neuronal patterns, as might be found in the human brain. The scaleis one of larger patterns emerging from an ensemble mass, possibly through sometype of energy equation and a reduction procedure. The links between thepatterns can be formed naturally, as a residual effect of the pattern creationitself. If the process is valid, then the pattern creation can be relativelysimplistic and aut ...继续阅读 (5)

Computer Vision and Pattern RecognitionAn Artificial Agent for Robust Image RegistrationRui Liao,Shun Miao,Pierre de Tournemire,Sasa Grbic,Ali Kamen,Tommaso Mansi,Dorin ComaniciuComments: To appear in AAAI Conference 2017Subjects:Computer Vision and Pattern Recognition (cs.CV)3-D image registration, which involves aligning two or more images, is acritical step in a variety of medical applications from diagnosis to therapy.Image registration is commonly performed by optimizing an image matching metricas a cost function. However, this task is challenging due to the non-convexnature of the matchi ...继续阅读 (8)

Neural and Evolutionary ComputingEmergence of foveal image sampling from learning to attend in visual scenesBrian Cheung,Eric Weiss,Bruno OlshausenSubjects:Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Learning (cs.LG)We describe a neural attention model with a learnable retinal samplinglattice. The model is trained on a visual search task requiring theclassification of an object embedded in a visual scene amidst backgrounddistractors using the smallest number of fixations. We explore the tilingproperties that emerge in the model’s retinal sampling lattice ...继续阅读 (4)

Neural and Evolutionary ComputingDense Prediction on Sequences with Time-Dilated Convolutions for Speech RecognitionTom Sercu,Vaibhava GoelComments: Appeared at NIPS 2016 End-to-end Learning for Speech and Audio Processing WorkshopSubjects:Computation and Language (cs.CL); Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)In computer vision pixelwise dense prediction is the task of predicting alabel for each pixel in the image. Convolutional neural networks achieve goodperformance on this task, while being computationally efficient. In this paperwe carry these ideas over to the probl ...继续阅读

Neural and Evolutionary ComputingLearning Python Code Suggestion with a Sparse Pointer NetworkAvishkar Bhoopchand,Tim Rocktäschel,Earl Barr,Sebastian RiedelComments: Under review as a conference paper at ICLR 2017Subjects:Neural and Evolutionary Computing (cs.NE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Software Engineering (cs.SE)To enhance developer productivity, all modern integrated developmentenvironments (IDEs) include code suggestion functionality that proposes likelynext tokens at the cursor. While current IDEs work well for statically-typedlanguages, th ...继续阅读 (3)

Neural and Evolutionary ComputingdMath: Distributed Linear Algebra for DLSteven Eliuk,Cameron Upright,Hars Vardhan,Stephen Walsh,Trevor GaleComments: 5 pages. arXiv admin note: text overlap witharXiv:1604.01416Subjects:Distributed, Parallel, and Cluster Computing (cs.DC); Mathematical Software (cs.MS); Neural and Evolutionary Computing (cs.NE)The paper presents a parallel math library, dMath, that demonstrates leadingscaling when using intranode, internode, and hybrid-parallelism for deeplearning (DL). dMath provides easy-to-use distributed primitives and a varietyof domain-specific algorithms ...继续阅读 (3)

作者：Daniil原文：1.Image Classification and Segmentation with Tensorflow and TF-Slim2.Upsampling and Image Segmentation with Tensorflow and TF-Slim导读：TF-Slim是Google年中刚开源的Tensorflow图像软件包，能使用户快速准确地定义复杂模型，尤其是图像分类和分割任务等，具体可以查看新智元的报导：谷歌开源图像分类工具TF-Slim，定义TensorFlow复杂模型这两篇文章介绍具体的使用，第一篇偏重基本的分类问题，第二篇偏重上采样和反卷积。 ...继续阅读 (20)

Neural and Evolutionary ComputingRecurrent Neural Networks With Limited Numerical PrecisionJoachim Ott,Zhouhan Lin,Ying Zhang,Shih-Chii Liu,Yoshua BengioComments: NIPS 2016 EMDNN Workshop paper, condensed version ofarXiv:1608.06902Subjects:Neural and Evolutionary Computing (cs.NE)Recurrent Neural Networks (RNNs) produce state-of-art performance on manymachine learning tasks but their demand on resources in terms of memory andcomputational power are often high. Therefore, there is a great interest inoptimizing the computations performed with these models especially whenconsidering development o ...继续阅读 (7)

Neural and Evolutionary ComputingA Metaprogramming and Autotuning Framework for Deploying Deep Learning ApplicationsMatthew W. Moskewicz,Ali Jannesari,Kurt KeutzerSubjects:Neural and Evolutionary Computing (cs.NE); Distributed, Parallel, and Cluster Computing (cs.DC); Mathematical Software (cs.MS)In recent years, deep neural networks (DNNs), have yielded strong results ona wide range of applications. Graphics Processing Units (GPUs) have been onekey enabling factor leading to the current popularity of DNNs. However, despiteincreasing hardware flexibility and software programming toolchain mat ...继续阅读 (6)

Neural and Evolutionary ComputingNoiseOut: A Simple Way to Prune Neural NetworksMohammad Babaeizadeh,Paris Smaragdis,Roy H. CampbellSubjects:Neural and Evolutionary Computing (cs.NE); Computer Vision and Pattern Recognition (cs.CV)Neural networks are usually over-parameterized with significant redundancy inthe number of required neurons which results in unnecessary computation andmemory usage at inference time. One common approach to address this issue is toprune these big networks by removing extra neurons and parameters whilemaintaining the accuracy. In this paper, we propose NoiseOut, a ful ...继续阅读 (4)

作者：ANKIT GUPTA原文：Solutions for Skilltest Machine Learning : Revealed注：有答案及解析，本文只给出索引，完整题目及解答请点原文1780人参加了这项测试，最终的分数分布如下：Q1. Which of the following method is best suited to detect outliers in a n-dimensional space, where n > 1:Q2. Logistics regression differs from multiple regression analysis in following ways ?Q3. What does it mean to bootstrap data?Q4. “Overfitting is a challenge with supervised learning, but not with unsupervised learning.” Is the above statement True or False?Q5. Which of the following are true with regards to choosing “k” in k-fold cross validation?Q6. A regression model ...继续阅读 (20)

来源：知乎－UCloud云计算机器学习进阶笔记之一 | TensorFlow安装与入门机器学习进阶笔记之二 | 深入理解Neural Style机器学习进阶笔记之三 | 深入理解Alexnet机器学习进阶笔记之四 | 深入理解GoogLeNet机器学习进阶笔记之五 | 深入理解VGG\Residual Network机器学习进阶笔记之六 | 深入理解Fast Neural Style ...继续阅读 (4)

来源：算法与数学之美原文：计算机科学中最重要的32个算法奥地利符号计算研究所（Research Institute for Symbolic Computation，简称RISC）的Christoph Koutschan博士在自己的页面上发布了一篇文章，提到他做了一个调查，参与者大多数是计算机科学家，他请这些科学家投票选出最重要的算法，以下是这次调查的结果，按照英文名称字母顺序排序。A* 搜索算法——图形搜索算法，从给定起点到给定终点计算出路径。其中使用了一种启发式的估算，为每个节点估算通过该节点的最佳路径，并以之为各个地点排定次序。算法以得到的次序访问这些节点。因此，A*搜索算法是最佳优先搜索的范例。集束搜索（又名定向搜索，Beam Search）——最佳优先搜索算法的优化。使用启发式函数评估它检查的每个节点的能力。不过，集束搜索只能在每个深度中发现最前面的m个最符合条件的节点，m是固定数字——集束的宽度。二分查找（Binary Search）——在线性数组中找特定值的算法，每个步骤去掉一半不符合要求的数据。分支界定算法（Branch and Bound）——在多种最优化问题中寻找特定最优化解决方案的算法，特别是针对离散、组合的最优化。Buchberger算法——一种数学算法，可将其视为针对单变量最大公约数求解的欧几里得算法和线性系统中高斯消元法的泛化。数据压缩——采取特定编码方 ...继续阅读 (9)

作者：冯超原文链接：无痛的机器学习第一季目录经过5个月的努力，我终于完成了40篇不高不低还算有些干货的机器学习文章。回首看看这5个月的努力，每一次的写作都充满了开心与痛苦。说开心是因为当自己完成每一个章节的写作后，自己感觉对这一部分的知识有了更加深刻地认识，而痛苦则是对写作过程中一系列事情的恐惧——找不到好选题，对论文细节的困惑，跑不出想要的结果，难以用通俗易懂的语言描述自己所知……好在这一切就要告一段落了。以下就做一个集合贴，展示一下这一季的所有文章，对本专栏文章感兴趣的童鞋，收藏这一篇就足够了（后续未发布的几篇和番外篇会更新上来）：文章目录CNN网络基础结构神经网络-全连接层（1）神经网络-全连接层（2）神经网络-全连接层（3）卷积层（1）卷积层（2）卷积层（3）CNN网络上层结构CNN——架构上的一些数字CNN–结构上的思考Caffe源码分析Caffe代码阅读——层次结构Caffe源码阅读——Net组装Caffe代码阅读——SolverCNN–两个Loss层计算的数值问题Caffe源码阅读——DataLayer&Data Transformer生成网络DCGAN的小尝试（1）DCGAN的小尝试（2）VAE（1）——从KL说起VAE(2)——基本思想VAE(3)——公式与实现VAE（4）——实现优化算法梯度下降是门手艺活……路遥知马力——Mome ...继续阅读 (2)

Neural and Evolutionary ComputingDesigning and Training Feedforward Neural Networks: A Smooth Optimisation PerspectiveHao ShenComments: 15 pages, 2 figures, submitted for publicationSubjects:Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE); Optimization and Control (math.OC)Despite the recent great success of deep neural networks in variousapplications, designing and training a deep neural network is still among thegreatest challenges in the field. In this work, we present a smoothoptimisation perspective on designing and training multilayer Feedfor ...继续阅读 (1)

来源：https://adeshpande3.github.io作者：Adit DeshpandeWeek 1: Generative Adversarial Nets第一周首先简单介绍了GAN的基本概念，然后读两篇paper：Laplacian Pyramid of Adversarial NetworksGenerative Adversarial Text to Image SynthesisWeek 2: Reinforcement Learning第二周先介绍了增强学习，然后读两篇paper：DQN For Reinforcement Learning (RL With Atari Games)Mastering AlphaGo with RL ...继续阅读 (7)

Neural and Evolutionary ComputingTraining Spiking Deep Networks for Neuromorphic HardwareEric Hunsberger,Chris EliasmithComments: 10 pages, 3 figures, 4 tables; the “methods” section of this article draws heavily onarXiv:1510.08829Subjects:Neural and Evolutionary Computing (cs.NE); Learning (cs.LG)We describe a method to train spiking deep networks that can be run usingleaky integrate-and-fire (LIF) neurons, achieving state-of-the-art results forspiking LIF networks on five datasets, including the large ImageNet ILSVRC-2012benchmark. Our method for transforming deep artificial neur ...继续阅读 (3)

新智元深度 | 美国大选Facebook 误导公众？智能时代，推荐算法主导群体意识机器之心深度 | 谷歌新人李飞飞：击碎玻璃天花板的华裔女科学家无痛的机器学习无痛的机器学习第一季目录聊点轻松的5——这篇写得并不轻松我爱机器学习Machine Learning Daily: 2016-11-17arXiv Paper Daily: Thu, 17 Nov 2016[导读]Deep Learning Research Review ...继续阅读 (3)