AI2019

VMP
Neural Network in Your Browser
Intermediate Python
Tutorials for learning Torch
awesome-deep-learning-papers
awesome-deep-learning2
Deep Learning An MIT Press book
deep learning coursera
Artificial Intelligence: A Modern Approach
Docker container for NeuralTalk
Как и для чего использовать Docker
введение в Docker с нуля. Ваш первый микросервис

Comparison of deep learning software
Benchmarking CNTK on Keras: is it Better at Deep Learning than TensorFlow?
Get Started with TensorFlow
Setup CNTK on your machine
Using CNTK with Keras (Beta)
Installing CNTK for Python on Windows
Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
CNTK Examples

Google AI Publication database
DEEP NEURAL NETWORKS AS GAUSSIANP ROCESSES PDF
Gaussian Process Behaviour in Wide Deep Neural Networks PDF
BAYESIAN DEEP CONVOLUTIONAL NETWORKS WITH MANY CHANNELS ARE GAUSSIAN PROCESSES

Deep Neural Networks as Gaussian Processes
Gaussian Process Behaviour in Wide Deep Neural Networks
A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music
A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music

Magenta: Music and Art Generation with Machine Intelligence
Generating Music and Lyrics using Deep Learning via Long Short-Term Recurrent Networks (LSTMs). Implements a Char-RNN in Python using TensorFlow.
Using Long Short-Term Memory neural networks to generate music
Music generation from midi files based on Long Short Term Memory
Using Long Short Term Memory Keras model with additional libraries to generate Metallica style music
StructLSTM – Structure augmented Long-Short Term Memory Networks for Music Generation
Generating Music and Lyrics using Deep Learning via Long Short-Term Recurrent Networks (LSTMs). Implements a Char-RNN in Python using TensorFlow.
In-depth Summary of Facebook AI’s Music Translation Model
happier: hierarchical polyphonic music generative rnn – OpenReview
Model for Learning Long-Term Structure in Music
Learning a Latent Space of Multitrack Measures – Machine Learning
MusicVAE: Creating a palette for musical scores with machine learning
Awesome-Deep-Learning-Resources
awesome-deeplearning-resources

Grid LSTM
tensorflow-grid-lstm
lstm-neural-networks
Grid LSTM – UvA Deep Learning Course
Tensorflow Grid LSTM RNN
Examples of using GridLSTM (and GridRNN in general) in tensorflow
MusicVAE: A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music.

Modeling Time-Frequency Patterns with LSTM vs
Torch7 implementation of Grid-LSTM as described here: http://arxiv.org/pdf/1507.01526v2.pdf
Grid Long Short-Term Memory
Neural Network Right Here in Your Browser
Recurrent Neural Network – A curated list of resources dedicated to RNN
Generation of poems with a recurrent neural network
Generating Poetry using Neural Networks
Chinese Poetry Generation with Recurrent Neural Networks
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , pages 1919–1924, Lisbon, Portugal, 17-21 September 2015. c © 2015 Association for Computational Linguistics. GhostWriter: Using an LSTM for Automatic Rap Lyric Generation
Generating Sentences from a Continuous Space
PoetRNN A python framework for learning and producing verse poetry
Как научить свою нейросеть генерировать стихи
КлассикAI жанра: ML ищет себя в поэзии
github.com/facebookresearch
deep learning v: рекуррентные сети
Нейросеть сочинила стихи в стиле «Нирваны»
Comparison of deep learning software

Neural Network Right Here in Your Browser
TensorFlow на AWS Удобные возможности для глубокого обучения в облаке. 85% проектов TensorFlow в облачной среде выполняются в AWS.
Deep Learning on ROCm
GitHub Facebook Research
nevergrad A Python toolbox for performing gradient-free optimization

On word embeddings – Part 1
On word embeddings – Part 2: Approximating the Softmax
On word embeddings – Part 3: The secret ingredients of word2vec
A survey of cross-lingual word embedding models
Word embeddings in 2017: Trends and future directions
Keras: The Python Deep Learning library
Deep Learning for humans http://keras.io/
Keras examples directory
Стихи.ру – российский литературный портал
github IlyaGusev
Библиотека для анализа и генерации стихов на русском языке
Морфологический анализатор на основе нейронных сетей и pymorphy2
Задание по курсу NLP
Поэтический корпус русского языка http://poetry-corpus.ru/
Code inspired by Unsupervised Machine Translation Using Monolingual Corpora Only
Open Source Neural Machine Translation in PyTorch http://opennmt.net/
A library for Multilingual Unsupervised or Supervised word Embeddings
PyText A natural language modeling framework based on PyTorch https://fb.me/pytextdocs

RusVectōrēs: семантические модели для русского языка
Sberbank AI
Классик AI: Cоревнование по стихотворному Искуственному Интеллекту
Программа MyStem производит морфологический анализ текста на русском языке. Она умеет строить гипотетические разборы для слов, не входящих в словарь.
A Python wrapper of the Yandex Mystem 3.1 morphological analyzer (http://api.yandex.ru/mystem)
gensim
NTLK Natural Language Toolkit
nltk windows
rusvectores
Национальный корпус русского языка
Flask
Морфологическая разметка с использование обширного описания языка
A receiver operating characteristic curve, i.e., ROC curve

Neural Style Transfer: Creating Art with Deep Learning using tf.keras and eager execution
TF Jam — Shooting Hoops with Machine Learning
Introducing TensorFlow.js: Machine Learning in Javascript
Standardizing on Keras: Guidance on High-level APIs in TensorFlow 2.0

PyText Documentation
PyText A natural language modeling framework based on PyTorch https://fb.me/pytextdocs
Recurrent Neural Network – A curated list of resources dedicated to RNN
Generation of poems with a recurrent neural network
Generating Poetry using Neural Networks
Chinese Poetry Generation with Recurrent Neural Networks
Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing , pages 1919–1924, Lisbon, Portugal, 17-21 September 2015. c © 2015 Association for Computational Linguistics. GhostWriter: Using an LSTM for Automatic Rap Lyric Generation
Generating Sentences from a Continuous Space
PoetRNN A python framework for learning and producing verse poetry
Language modeling a billion words
Deep Learning with Torch: the 60-minute blitz
NNGraph A graph based container for creating deep learning models
Tutorials for learning Torch

Как научить свою нейросеть генерировать стихи
КлассикAI жанра: ML ищет себя в поэзии
github.com/facebookresearch
deep learning v: рекуррентные сети
Нейросеть сочинила стихи в стиле «Нирваны»

Building TensorFlow on Android
Building TensorFlow on iOS
Как использовать TensorFlow Mobile в приложениях для Android
TensorFlow Lite is for mobile and embedded devices
TensorFlow Lite versus TensorFlow Mobile
tensorflow/tensorflow/contrib/lite/
github tensorflow
Android TensorFlow Lite Machine Learning Example
After a TensorFlow model is trained, the TensorFlow Lite converter uses that model to generate a TensorFlow Lite FlatBuffer file (.tflite). The converter supports as input: SavedModels, frozen graphs (models generated by freeze_graph.py), and tf.keras HDF5 models. The TensorFlow Lite FlatBuffer file is deployed to a client device (generally a mobile or embedded device), and the TensorFlow Lite interpreter uses the compressed model for on-device inference.

TF Lite Developer Guide
TensorFlow for Poets 2: TFLite iOS
TensorFlow for Poets 2: TFLite Android
TensorFlow for Poets 2: TFMobile
Tinker With a Neural Network Right Here in Your Browser

TensorFlow Lite Optimizing Converter command-line examples

Differences between L1 and L2 as Loss Function and Regularization
L1 and L2 Regularization
Регуляризация
L1- и L2-регуляризация в машинном обучении
L 1 -регуляризациялинейнойрегрессии. Регрессиянаименьшихуглов(алгоритмLARS)
L1 и L2 регуляризации для линейной регрессии
L1 и L2-регуляризация для логистической регрессии


Обработка естественных языков на языке Python
Практическое глубокое обучение в Theano и TensorFlow

TensorFlow For Poets
Get Started with TensorFlow
Train your own image classifier with Inception in TensorFlow
Google Developers






The Khronos Group
Khronos royalty-free open standards for 3D graphics, Virtual and Augmented Reality, Parallel Computing, Neural Networks, and Vision Processing
характеристики CPU AMD Ryzen 3000: флагманская модель Ryzen 9 3850X предложит 16 ядер и частоту 5,1 ГГц
NeuralTuringMachine
Oxford Deep NLP 2017 course
A Stable Neural-Turing-Machine (NTM) Implementation (Source Code and Pre-Print)
A series of models applying memory augmented neural networks to machine translation

Newly published papers (< 6 months) which are worth reading

  • MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017), Andrew G. Howard et al. [pdf]
  • Convolutional Sequence to Sequence Learning (2017), Jonas Gehring et al. [pdf]
  • A Knowledge-Grounded Neural Conversation Model (2017), Marjan Ghazvininejad et al. [pdf]
  • Accurate, Large Minibatch SGD:Training ImageNet in 1 Hour (2017), Priya Goyal et al. [pdf]
  • TACOTRON: Towards end-to-end speech synthesis (2017), Y. Wang et al. [pdf]
  • Deep Photo Style Transfer (2017), F. Luan et al. [pdf]
  • Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017), T. Salimans et al. [pdf]
  • Deformable Convolutional Networks (2017), J. Dai et al. [pdf]
  • Mask R-CNN (2017), K. He et al. [pdf]
  • Learning to discover cross-domain relations with generative adversarial networks (2017), T. Kim et al. [pdf]
  • Deep voice: Real-time neural text-to-speech (2017), S. Arik et al., [pdf]
  • PixelNet: Representation of the pixels, by the pixels, and for the pixels (2017), A. Bansal et al. [pdf]
  • Batch renormalization: Towards reducing minibatch dependence in batch-normalized models (2017), S. Ioffe. [pdf]
  • Wasserstein GAN (2017), M. Arjovsky et al. [pdf]
  • Understanding deep learning requires rethinking generalization (2017), C. Zhang et al. [pdf]
  • Least squares generative adversarial networks (2016), X. Mao et al. [pdf]

Understanding / Generalization / Transfer

  • Distilling the knowledge in a neural network (2015), G. Hinton et al. [pdf]
  • Deep neural networks are easily fooled: High confidence predictions for unrecognizable images (2015), A. Nguyen et al. [pdf]
  • How transferable are features in deep neural networks? (2014), J. Yosinski et al. [pdf]
  • CNN features off-the-Shelf: An astounding baseline for recognition (2014), A. Razavian et al. [pdf]
  • Learning and transferring mid-Level image representations using convolutional neural networks (2014), M. Oquab et al. [pdf]
  • Visualizing and understanding convolutional networks (2014), M. Zeiler and R. Fergus [pdf]
  • Decaf: A deep convolutional activation feature for generic visual recognition (2014), J. Donahue et al. [pdf]

Optimization / Training Techniques

  • Training very deep networks (2015), R. Srivastava et al. [pdf]
  • Batch normalization: Accelerating deep network training by reducing internal covariate shift (2015), S. Loffe and C. Szegedy [pdf]
  • Delving deep into rectifiers: Surpassing human-level performance on imagenet classification (2015), K. He et al. [pdf]
  • Dropout: A simple way to prevent neural networks from overfitting (2014), N. Srivastava et al. [pdf]
  • Adam: A method for stochastic optimization (2014), D. Kingma and J. Ba [pdf]
  • Improving neural networks by preventing co-adaptation of feature detectors (2012), G. Hinton et al. [pdf]
  • Random search for hyper-parameter optimization (2012) J. Bergstra and Y. Bengio [pdf]

Unsupervised / Generative Models

  • Pixel recurrent neural networks (2016), A. Oord et al. [pdf]
  • Improved techniques for training GANs (2016), T. Salimans et al. [pdf]
  • Unsupervised representation learning with deep convolutional generative adversarial networks (2015), A. Radford et al. [pdf]
  • DRAW: A recurrent neural network for image generation (2015), K. Gregor et al. [pdf]
  • Generative adversarial nets (2014), I. Goodfellow et al. [pdf]
  • Auto-encoding variational Bayes (2013), D. Kingma and M. Welling [pdf]
  • Building high-level features using large scale unsupervised learning (2013), Q. Le et al. [pdf]

Convolutional Neural Network Models

  • Rethinking the inception architecture for computer vision (2016), C. Szegedy et al. [pdf]
  • Inception-v4, inception-resnet and the impact of residual connections on learning (2016), C. Szegedy et al. [pdf]
  • Identity Mappings in Deep Residual Networks (2016), K. He et al. [pdf]
  • Deep residual learning for image recognition (2016), K. He et al. [pdf]
  • Spatial transformer network (2015), M. Jaderberg et al., [pdf]
  • Going deeper with convolutions (2015), C. Szegedy et al. [pdf]
  • Very deep convolutional networks for large-scale image recognition (2014), K. Simonyan and A. Zisserman [pdf]
  • Return of the devil in the details: delving deep into convolutional nets (2014), K. Chatfield et al. [pdf]
  • OverFeat: Integrated recognition, localization and detection using convolutional networks (2013), P. Sermanet et al. [pdf]
  • Maxout networks (2013), I. Goodfellow et al. [pdf]
  • Network in network (2013), M. Lin et al. [pdf]
  • ImageNet classification with deep convolutional neural networks (2012), A. Krizhevsky et al. [pdf]

Image: Segmentation / Object Detection

  • You only look once: Unified, real-time object detection (2016), J. Redmon et al. [pdf]
  • Fully convolutional networks for semantic segmentation (2015), J. Long et al. [pdf]
  • Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks (2015), S. Ren et al. [pdf]
  • Fast R-CNN (2015), R. Girshick [pdf]
  • Rich feature hierarchies for accurate object detection and semantic segmentation (2014), R. Girshick et al. [pdf]
  • Spatial pyramid pooling in deep convolutional networks for visual recognition (2014), K. He et al. [pdf]
  • Semantic image segmentation with deep convolutional nets and fully connected CRFs, L. Chen et al. [pdf]
  • Learning hierarchical features for scene labeling (2013), C. Farabet et al. [pdf]

Image / Video / Etc

  • Image Super-Resolution Using Deep Convolutional Networks (2016), C. Dong et al. [pdf]
  • A neural algorithm of artistic style (2015), L. Gatys et al. [pdf]
  • Deep visual-semantic alignments for generating image descriptions (2015), A. Karpathy and L. Fei-Fei [pdf]
  • Show, attend and tell: Neural image caption generation with visual attention (2015), K. Xu et al. [pdf]
  • Show and tell: A neural image caption generator (2015), O. Vinyals et al. [pdf]
  • Long-term recurrent convolutional networks for visual recognition and description (2015), J. Donahue et al. [pdf]
  • VQA: Visual question answering (2015), S. Antol et al. [pdf]
  • DeepFace: Closing the gap to human-level performance in face verification (2014), Y. Taigman et al. [pdf]:
  • Large-scale video classification with convolutional neural networks (2014), A. Karpathy et al. [pdf]
  • Two-stream convolutional networks for action recognition in videos (2014), K. Simonyan et al. [pdf]
  • 3D convolutional neural networks for human action recognition (2013), S. Ji et al. [pdf]

Natural Language Processing / RNNs

  • Neural Architectures for Named Entity Recognition (2016), G. Lample et al. [pdf]
  • Exploring the limits of language modeling (2016), R. Jozefowicz et al. [pdf]
  • Teaching machines to read and comprehend (2015), K. Hermann et al. [pdf]
  • Effective approaches to attention-based neural machine translation (2015), M. Luong et al. [pdf]
  • Conditional random fields as recurrent neural networks (2015), S. Zheng and S. Jayasumana. [pdf]
  • Memory networks (2014), J. Weston et al. [pdf]
  • Neural turing machines (2014), A. Graves et al. [pdf]
  • Neural machine translation by jointly learning to align and translate (2014), D. Bahdanau et al. [pdf]
  • Sequence to sequence learning with neural networks (2014), I. Sutskever et al. [pdf]
  • Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014), K. Cho et al. [pdf]
  • A convolutional neural network for modeling sentences (2014), N. Kalchbrenner et al. [pdf]
  • Convolutional neural networks for sentence classification (2014), Y. Kim [pdf]
  • Glove: Global vectors for word representation (2014), J. Pennington et al. [pdf]
  • Distributed representations of sentences and documents (2014), Q. Le and T. Mikolov [pdf]
  • Distributed representations of words and phrases and their compositionality (2013), T. Mikolov et al. [pdf]
  • Efficient estimation of word representations in vector space (2013), T. Mikolov et al. [pdf]
  • Recursive deep models for semantic compositionality over a sentiment treebank (2013), R. Socher et al. [pdf]
  • Generating sequences with recurrent neural networks (2013), A. Graves. [pdf]

Speech / Other Domain

  • End-to-end attention-based large vocabulary speech recognition (2016), D. Bahdanau et al. [pdf]
  • Deep speech 2: End-to-end speech recognition in English and Mandarin (2015), D. Amodei et al. [pdf]
  • Speech recognition with deep recurrent neural networks (2013), A. Graves [pdf]
  • Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups (2012), G. Hinton et al. [pdf]
  • Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition (2012) G. Dahl et al. [pdf]
  • Acoustic modeling using deep belief networks (2012), A. Mohamed et al. [pdf]

Reinforcement Learning / Robotics

  • End-to-end training of deep visuomotor policies (2016), S. Levine et al. [pdf]
  • Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection (2016), S. Levine et al. [pdf]
  • Asynchronous methods for deep reinforcement learning (2016), V. Mnih et al. [pdf]
  • Deep Reinforcement Learning with Double Q-Learning (2016), H. Hasselt et al. [pdf]
  • Mastering the game of Go with deep neural networks and tree search (2016), D. Silver et al. [pdf]
  • Continuous control with deep reinforcement learning (2015), T. Lillicrap et al. [pdf]
  • Human-level control through deep reinforcement learning (2015), V. Mnih et al. [pdf]
  • Deep learning for detecting robotic grasps (2015), I. Lenz et al. [pdf]
  • Playing atari with deep reinforcement learning (2013), V. Mnih et al. [pdf])

More Papers from 2016

  • Layer Normalization (2016), J. Ba et al. [pdf]
  • Learning to learn by gradient descent by gradient descent (2016), M. Andrychowicz et al. [pdf]
  • Domain-adversarial training of neural networks (2016), Y. Ganin et al. [pdf]
  • WaveNet: A Generative Model for Raw Audio (2016), A. Oord et al. [pdf] [web]
  • Colorful image colorization (2016), R. Zhang et al. [pdf]
  • Generative visual manipulation on the natural image manifold (2016), J. Zhu et al. [pdf]
  • Texture networks: Feed-forward synthesis of textures and stylized images (2016), D Ulyanov et al. [pdf]
  • SSD: Single shot multibox detector (2016), W. Liu et al. [pdf]
  • SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and< 1MB model size (2016), F. Iandola et al. [pdf]
  • Eie: Efficient inference engine on compressed deep neural network (2016), S. Han et al. [pdf]
  • Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1 (2016), M. Courbariaux et al. [pdf]
  • Dynamic memory networks for visual and textual question answering (2016), C. Xiong et al. [pdf]
  • Stacked attention networks for image question answering (2016), Z. Yang et al. [pdf]
  • Hybrid computing using a neural network with dynamic external memory (2016), A. Graves et al. [pdf]
  • Google’s neural machine translation system: Bridging the gap between human and machine translation (2016), Y. Wu et al. [pdf]

New papers
Newly published papers (< 6 months) which are worth reading

  • MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications (2017), Andrew G. Howard et al. [pdf]
  • Convolutional Sequence to Sequence Learning (2017), Jonas Gehring et al. [pdf]
  • A Knowledge-Grounded Neural Conversation Model (2017), Marjan Ghazvininejad et al. [pdf]
  • Accurate, Large Minibatch SGD:Training ImageNet in 1 Hour (2017), Priya Goyal et al. [pdf]
  • TACOTRON: Towards end-to-end speech synthesis (2017), Y. Wang et al. [pdf]
  • Deep Photo Style Transfer (2017), F. Luan et al. [pdf]
  • Evolution Strategies as a Scalable Alternative to Reinforcement Learning (2017), T. Salimans et al. [pdf]
  • Deformable Convolutional Networks (2017), J. Dai et al. [pdf]
  • Mask R-CNN (2017), K. He et al. [pdf]
  • Learning to discover cross-domain relations with generative adversarial networks (2017), T. Kim et al. [pdf]
  • Deep voice: Real-time neural text-to-speech (2017), S. Arik et al., [pdf]
  • PixelNet: Representation of the pixels, by the pixels, and for the pixels (2017), A. Bansal et al. [pdf]
  • Batch renormalization: Towards reducing minibatch dependence in batch-normalized models (2017), S. Ioffe. [pdf]
  • Wasserstein GAN (2017), M. Arjovsky et al. [pdf]
  • Understanding deep learning requires rethinking generalization (2017), C. Zhang et al. [pdf]
  • Least squares generative adversarial networks (2016), X. Mao et al. [pdf]

>Old Papers
Classic papers published before 2012

  • An analysis of single-layer networks in unsupervised feature learning (2011), A. Coates et al. [pdf]
  • Deep sparse rectifier neural networks (2011), X. Glorot et al. [pdf]
  • Natural language processing (almost) from scratch (2011), R. Collobert et al.