Pytorch

VMP 1.10.2020

PyTorch, No Tears
Python
Intermediate Python
Python Numpy Tutorial
Библиотека программиста Подборка книг по машинному обучению
Туториал по PyTorch
TheAlgorithms/Python
pytorch.org
PyTorch, No Tears
Pytorch.org tutorials
Ttutorials beginner deep_learning_60min_blitz
Pytorch
Pytorch examples
Pytorch tutorials
github DeepFaceLab

PyTorch, No Tears
Pytorch
yunjey pytorch-tutorial
deeplearningzerotoall PyTorch
zergtant pytorch-handbook
chenyuntc pytorch-book
pytorch_geometric
PyTorch-GAN
MorvanZhou PyTorch-Tutorial
ГЛУБОКОЕ ОБУЧЕНИЕ
Книга «Программируем с PyTorch: Создание приложений глубокого обучения»
Как подружить PyTorch и C++. Используем TorchScript
TorchScriptTutorial
Deep_Learning_for_Vision_Systems_by_Mohamed_Elgendy_z_lib_org.pdf

The uWSGI project
Введение в WSGI-серверы: Часть первая
Flask Documentation (1.1.x)
Настройка mod_wsgi (Apache) для Flask
WSGI Servers Full Stack Python


Нейронные сети и компьютерное зрение – 66 урок. Строим первую нейронную сеть

Нейронные сети и компьютерное зрение – 75 урок. Классификация в PyTorch
PyTorch-YOLOv3
YOLOv3-in-PyTorch
yolov3

PyTorch documentation
torch.nn
Convolution Layers
Conv2d
A Beginner’s Guide To Understanding Convolutional Neural Networks 1
A Beginner’s Guide To Understanding Convolutional Neural Networks Part 2
CS231n: Convolutional Neural Networks for Visual Recognition
Understanding of Convolutional Neural Network (CNN) — Deep Learning
Padding and Stride

Нейронные сети и компьютерное зрение – 1 урок

Нейронные сети и компьютерное зрение – 103 урок.Свёртка, каскад свёрток

Нейронные сети и компьютерное зрение – 104 урок.Свёртка, каскад свёрток

Нейронные сети и компьютерное зрение – 105 урок.Свёртка, каскад свёрток





Нейронные сети и компьютерное зрение – 110 урок. Архитектура LeNet (1998)













Нейронные сети и компьютерное зрение – 116 – 124 урок. AlexNet (2012) и VGG (2014)





Нейронные сети и компьютерное зрение – 125-130 урок. GoogLeNet и ResNet (2015)

Нейронные сети и компьютерное зрение – 131 урок. Распознавание рукописных чисел свёрточной сетью

torch.nn.Conv2d(in_channels: int, out_channels: int, kernel_size: Union[T, Tuple[T, T]], stride: Union[T, Tuple[T, T]] = 1, padding: Union[T, Tuple[T, T]] = 0, dilation: Union[T, Tuple[T, T]] = 1, groups: int = 1, bias: bool = True, padding_mode: str = ‘zeros’)
Parameters
in_channels (int) – Number of channels in the input image
out_channels (int) – Number of channels produced by the convolution
kernel_size (int or tuple) – Size of the convolving kernel
stride (int or tuple, optional) – Stride of the convolution. Default: 1
padding (int or tuple, optional) – Zero-padding added to both sides of the input. Default: 0
padding_mode (string, optional) – ‘zeros’, ‘reflect’, ‘replicate’ or ‘circular’. Default: ‘zeros’
dilation (int or tuple, optional) – Spacing between kernel elements. Default: 1
groups (int, optional) – Number of blocked connections from input channels to output channels. Default: 1
bias (bool, optional) – If True, adds a learnable bias to the output. Default: True
Pytorch-how-and-when-to-use-Module-Sequential-ModuleList-and-ModuleDict
>>> # With square kernels and equal stride
>>> m = nn.Conv2d(16, 33, 3, stride=2)
>>> # non-square kernels and unequal stride and with padding
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2))
>>> # non-square kernels and unequal stride and with padding and dilation
>>> m = nn.Conv2d(16, 33, (3, 5), stride=(2, 1), padding=(4, 2), dilation=(3, 1))
>>> input = torch.randn(20, 16, 50, 100)
>>> output = m(input)

import torch.nn.functional as F

class MyCNNClassifier(nn.Module):
def __init__(self, in_c, n_classes):
super().__init__()
self.conv1 = nn.Conv2d(in_c, 32, kernel_size=3, stride=1, padding=1)
self.bn1 = nn.BatchNorm2d(32)

self.conv2 = nn.Conv2d(32, 64, kernel_size=3, stride=1, padding=1)
self.bn2 = nn.BatchNorm2d(64)

self.fc1 = nn.Linear(64 * 28 * 28, 1024)
self.fc2 = nn.Linear(1024, n_classes)

def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = F.relu(x)

x = self.conv2(x)
x = self.bn2(x)
x = F.relu(x)

x = x.view(x.size(0), -1) # flat

x = self.fc1(x)
x = F.sigmoid(x)
x = self.fc2(x)

return x

model = MyCNNClassifier(1, 10)
print(model)

MyCNNClassifier(
(conv1): Conv2d(1, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(conv2): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(fc1): Linear(in_features=50176, out_features=1024, bias=True)
(fc2): Linear(in_features=1024, out_features=10, bias=True)
)

PyTorch-VAE
implementing-an-autoencoder-in-pytorch
Building Autoencoder in Pytorch
L1aoXingyu pytorch-beginner
kaggle.com autoencoders-with-pytorch
pytorch-beginner 08-AutoEncoder
kaggle Convolutional Autoencoder
Denoising-Autoencoder-in-Pytorch
github.com Autoencoders+pytorch

pytorch mobile flutter
torch_mobile flutter plugin

GitHub PyTorch
Pytorch3d
PyTorch GitHub
PyTorch Geometric

Туториал по PyTorch: от установки до готовой нейронной сети
NVIDIA выпустили обертку над PyTorch для обучения моделей
Какие ошибки чаще всего совершают при обучении нейросетей
StyleGAN2: улучшенная нейросеть для генерации лиц людей
StyleGAN2 — Official TensorFlow Implementation
PyTorch — Краткое руководство 2019
Понимание PyTorch на примере
PyTorch Tutorial: How to Develop Deep Learning Models with Python

torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode=’zeros’)
Parameters
• in_channels (int) – Number of channels in the input image
• out_channels (int) – Number of channels produced by the convolution
• kernel_size (int or tuple) – Size of the convolving kernel
• stride (int or tuple, optional) – Stride of the convolution. (Default: 1)
• padding (int or tuple, optional) – Zero-padding added to both sides of the input (Default: 0)
• padding_mode (string, optional) – zeros
• dilation (int or tuple, optional) – Spacing between kernel elements. (Default: 1)
• groups (int, optional) – Number of blocked connections from input to output channels. (Default: 1)
• bias (bool, optional) – If True, adds a learnable bias to the output. (Default: True)
And this URL has helpful visualization of the process.
So the in_channels in the beginning is 3 for images with 3 channels (colored images). For images black and white it should be 1. Some satellite images should have 4.
The out_channels is what convolution will produce so these are the number of filters.
Let’s create an example to “prove” that.
import torch
import torch.nn as nn
c = nn.Conv2d(1,3, stride = 1, kernel_size=(4,5))
print(c.weight.shape)
print(c.weight)
Out
torch.Size([3, 1, 4, 5])

Нейронные сети и компьютерное зрение – 1 урок

PyTorch at Tesla – Andrej Karpathy, Tesla

Pytorch Bidirectional LSTM example
Популярность PyTorch в среднем выросла на 243% за год

Pytorch & related libraries

  1. pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration.


NLP & Speech Processing:

  1. pytorch text: Torch text related contents.
  2. pytorch-seq2seq: A framework for sequence-to-sequence (seq2seq) models implemented in PyTorch.
  3. anuvada: Interpretable Models for NLP using PyTorch.
  4. audio: simple audio I/O for pytorch.
  5. loop: A method to generate speech across multiple speakers
  6. fairseq-py: Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
  7. speech: PyTorch ASR Implementation.
  8. OpenNMT-py: Open-Source Neural Machine Translation in PyTorch http://opennmt.net
  9. neuralcoref: State-of-the-art coreference resolution based on neural nets and spaCy huggingface.co/coref
  10. sentiment-discovery: Unsupervised Language Modeling at scale for robust sentiment classification.
  11. MUSE: A library for Multilingual Unsupervised or Supervised word Embeddings
  12. nmtpytorch: Neural Machine Translation Framework in PyTorch.
  13. pytorch-wavenet: An implementation of WaveNet with fast generation
  14. Tacotron-pytorch: Tacotron: Towards End-to-End Speech Synthesis.
  15. AllenNLP: An open-source NLP research library, built on PyTorch.
  16. PyTorch-NLP: Text utilities and datasets for PyTorch pytorchnlp.readthedocs.io
  17. quick-nlp: Pytorch NLP library based on FastAI.
  18. TTS: Deep learning for Text2Speech
  19. LASER: Language-Agnostic SEntence Representations
  20. pyannote-audio: Neural building blocks for speaker diarization: speech activity detection, speaker change detection, speaker embedding
  21. gensen: Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning.
  22. translate: Translate – a PyTorch Language Library.
  23. espnet: End-to-End Speech Processing Toolkit espnet.github.io/espnet
  24. pythia: A software suite for Visual Question Answering
  25. UnsupervisedMT: Phrase-Based & Neural Unsupervised Machine Translation.
  26. jiant: The jiant sentence representation learning toolkit.
  27. BERT-PyTorch: Pytorch implementation of Google AI’s 2018 BERT, with simple annotation
  28. InferSent: Sentence embeddings (InferSent) and training code for NLI.
  29. uis-rnn:This is the library for the Unbounded Interleaved-State Recurrent Neural Network (UIS-RNN) algorithm, corresponding to the paper Fully Supervised Speaker Diarization. arxiv.org/abs/1810.04719
  30. flair: A very simple framework for state-of-the-art Natural Language Processing (NLP)
  31. pytext: A natural language modeling framework based on PyTorch fb.me/pytextdocs
  32. voicefilter: Unofficial PyTorch implementation of Google AI’s VoiceFilter system http://swpark.me/voicefilter
  33. BERT-NER: Pytorch-Named-Entity-Recognition-with-BERT.
  34. transfer-nlp: NLP library designed for flexible research and development
  35. texar-pytorch: Toolkit for Machine Learning and Text Generation, in PyTorch texar.io
  36. pytorch-kaldi: pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.
  37. NeMo: Neural Modules: a toolkit for conversational AI nvidia.github.io/NeMo
  38. pytorch-struct: A library of vectorized implementations of core structured prediction algorithms (HMM, Dep Trees, CKY, ..,)
  39. espresso: Espresso: A Fast End-to-End Neural Speech Recognition Toolkit
  40. transformers: huggingface Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch. huggingface.co/transformers
  41. reformer-pytorch: Reformer, the efficient Transformer, in Pytorch

CV:

  1. pytorch vision: Datasets, Transforms and Models specific to Computer Vision.
  2. pt-styletransfer: Neural style transfer as a class in PyTorch.
  3. OpenFacePytorch: PyTorch module to use OpenFace’s nn4.small2.v1.t7 model
  4. img_classification_pk_pytorch: Quickly comparing your image classification models with the state-of-the-art models (such as DenseNet, ResNet, …)
  5. SparseConvNet: Submanifold sparse convolutional networks.
  6. Convolution_LSTM_pytorch: A multi-layer convolution LSTM module
  7. face-alignment: ? 2D and 3D Face alignment library build using pytorch adrianbulat.com
  8. pytorch-semantic-segmentation: PyTorch for Semantic Segmentation.
  9. RoIAlign.pytorch: This is a PyTorch version of RoIAlign. This implementation is based on crop_and_resize and supports both forward and backward on CPU and GPU.
  10. pytorch-cnn-finetune: Fine-tune pretrained Convolutional Neural Networks with PyTorch.
  11. detectorch: Detectorch – detectron for PyTorch
  12. Augmentor: Image augmentation library in Python for machine learning. http://augmentor.readthedocs.io
  13. s2cnn: This library contains a PyTorch implementation of the SO(3) equivariant CNNs for spherical signals (e.g. omnidirectional cameras, signals on the globe)
  14. TorchCV: A PyTorch-Based Framework for Deep Learning in Computer Vision.
  15. maskrcnn-benchmark: Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch.
  16. image-classification-mobile: Collection of classification models pretrained on the ImageNet-1K.
  17. medicaltorch: A medical imaging framework for Pytorch http://medicaltorch.readthedocs.io
  18. albumentations: Fast image augmentation library.
  19. kornia: Differentiable computer vision library.
  20. pytorch-text-recognition: Text recognition combo – CRAFT + CRNN.
  21. facenet-pytorch: Pretrained Pytorch face detection and recognition models ported from davidsandberg/facenet.
  22. detectron2: Detectron2 is FAIR’s next-generation research platform for object detection and segmentation.
  23. vedaseg: A semantic segmentation framework by pyotrch
  24. ClassyVision: An end-to-end PyTorch framework for image and video classification.
  25. detecto:Computer vision in Python with less than 10 lines of code
  26. pytorch3d: PyTorch3D is FAIR’s library of reusable components for deep learning with 3D data pytorch3d.org
  27. MMDetection: MMDetection is an open source object detection toolbox, a part of the OpenMMLab project.
  28. neural-dream: A PyTorch implementation of the DeepDream algorithm. Creates dream-like hallucinogenic visuals.
  29. FlashTorch: Visualization toolkit for neural networks in PyTorch!
  30. Lucent: Tensorflow and OpenAI Clarity’s Lucid adapted for PyTorch.
  31. MMDetection3D: MMDetection3D is OpenMMLab’s next-generation platform for general 3D object detection, a part of the OpenMMLab project.
  32. MMSegmentation: MMSegmentation is a semantic segmentation toolbox and benchmark, a part of the OpenMMLab project.
  33. MMEditing: MMEditing is a image and video editing toolbox, a part of the OpenMMLab project.
  34. MMAction2: MMAction2 is OpenMMLab’s next generation action understanding toolbox and benchmark, a part of the OpenMMLab project.
  35. MMPose: MMPose is a pose estimation toolbox and benchmark, a part of the OpenMMLab project.

Probabilistic/Generative Libraries:

  1. ptstat: Probabilistic Programming and Statistical Inference in PyTorch
  2. pyro: Deep universal probabilistic programming with Python and PyTorch http://pyro.ai
  3. probtorch: Probabilistic Torch is library for deep generative models that extends PyTorch.
  4. paysage: Unsupervised learning and generative models in python/pytorch.
  5. pyvarinf: Python package facilitating the use of Bayesian Deep Learning methods with Variational Inference for PyTorch.
  6. pyprob: A PyTorch-based library for probabilistic programming and inference compilation.
  7. mia: A library for running membership inference attacks against ML models.
  8. pro_gan_pytorch: ProGAN package implemented as an extension of PyTorch nn.Module.
  9. botorch: Bayesian optimization in PyTorch

Other libraries:

  1. pytorch extras: Some extra features for pytorch.
  2. functional zoo: PyTorch, unlike lua torch, has autograd in it’s core, so using modular structure of torch.nn modules is not necessary, one can easily allocate needed Variables and write a function that utilizes them, which is sometimes more convenient. This repo contains model definitions in this functional way, with pretrained weights for some models.
  3. torch-sampling: This package provides a set of transforms and data structures for sampling from in-memory or out-of-memory data.
  4. torchcraft-py: Python wrapper for TorchCraft, a bridge between Torch and StarCraft for AI research.
  5. aorun: Aorun intend to be a Keras with PyTorch as backend.
  6. logger: A simple logger for experiments.
  7. PyTorch-docset: PyTorch docset! use with Dash, Zeal, Velocity, or LovelyDocs.
  8. convert_torch_to_pytorch: Convert torch t7 model to pytorch model and source.
  9. pretrained-models.pytorch: The goal of this repo is to help to reproduce research papers results.
  10. pytorch_fft: PyTorch wrapper for FFTs
  11. caffe_to_torch_to_pytorch
  12. pytorch-extension: This is a CUDA extension for PyTorch which computes the Hadamard product of two tensors.
  13. tensorboard-pytorch: This module saves PyTorch tensors in tensorboard format for inspection. Currently supports scalar, image, audio, histogram features in tensorboard.
  14. gpytorch: GPyTorch is a Gaussian Process library, implemented using PyTorch. It is designed for creating flexible and modular Gaussian Process models with ease, so that you don’t have to be an expert to use GPs.
  15. spotlight: Deep recommender models using PyTorch.
  16. pytorch-cns: Compressed Network Search with PyTorch
  17. pyinn: CuPy fused PyTorch neural networks ops
  18. inferno: A utility library around PyTorch
  19. pytorch-fitmodule: Super simple fit method for PyTorch modules
  20. inferno-sklearn: A scikit-learn compatible neural network library that wraps pytorch.
  21. pytorch-caffe-darknet-convert: convert between pytorch, caffe prototxt/weights and darknet cfg/weights
  22. pytorch2caffe: Convert PyTorch model to Caffemodel
  23. pytorch-tools: Tools for PyTorch
  24. sru: Training RNNs as Fast as CNNs (arxiv.org/abs/1709.02755)
  25. torch2coreml: Torch7 -> CoreML
  26. PyTorch-Encoding: PyTorch Deep Texture Encoding Network http://hangzh.com/PyTorch-Encoding
  27. pytorch-ctc: PyTorch-CTC is an implementation of CTC (Connectionist Temporal Classification) beam search decoding for PyTorch. C++ code borrowed liberally from TensorFlow with some improvements to increase flexibility.
  28. candlegp: Gaussian Processes in Pytorch.
  29. dpwa: Distributed Learning by Pair-Wise Averaging.
  30. dni-pytorch: Decoupled Neural Interfaces using Synthetic Gradients for PyTorch.
  31. skorch: A scikit-learn compatible neural network library that wraps pytorch
  32. ignite: Ignite is a high-level library to help with training neural networks in PyTorch.
  33. Arnold: Arnold – DOOM Agent
  34. pytorch-mcn: Convert models from MatConvNet to PyTorch
  35. simple-faster-rcnn-pytorch: A simplified implemention of Faster R-CNN with competitive performance.
  36. generative_zoo: generative_zoo is a repository that provides working implementations of some generative models in PyTorch.
  37. pytorchviz: A small package to create visualizations of PyTorch execution graphs.
  38. cogitare: Cogitare – A Modern, Fast, and Modular Deep Learning and Machine Learning framework in Python.
  39. pydlt: PyTorch based Deep Learning Toolbox
  40. semi-supervised-pytorch: Implementations of different VAE-based semi-supervised and generative models in PyTorch.
  41. pytorch_cluster: PyTorch Extension Library of Optimised Graph Cluster Algorithms.
  42. neural-assembly-compiler: A neural assembly compiler for pyTorch based on adaptive-neural-compilation.
  43. caffemodel2pytorch: Convert Caffe models to PyTorch.
  44. extension-cpp: C++ extensions in PyTorch
  45. pytoune: A Keras-like framework and utilities for PyTorch
  46. jetson-reinforcement: Deep reinforcement learning libraries for NVIDIA Jetson TX1/TX2 with PyTorch, OpenAI Gym, and Gazebo robotics simulator.
  47. matchbox: Write PyTorch code at the level of individual examples, then run it efficiently on minibatches.
  48. torch-two-sample: A PyTorch library for two-sample tests
  49. pytorch-summary: Model summary in PyTorch similar to model.summary() in Keras
  50. mpl.pytorch: Pytorch implementation of MaxPoolingLoss.
  51. scVI-dev: Development branch of the scVI project in PyTorch
  52. apex: An Experimental PyTorch Extension(will be deprecated at a later point)
  53. ELF: ELF: a platform for game research.
  54. Torchlite: A high level library on top of(not only) Pytorch
  55. joint-vae: Pytorch implementation of JointVAE, a framework for disentangling continuous and discrete factors of variation star2
  56. SLM-Lab: Modular Deep Reinforcement Learning framework in PyTorch.
  57. bindsnet: A Python package used for simulating spiking neural networks (SNNs) on CPUs or GPUs using PyTorch
  58. pro_gan_pytorch: ProGAN package implemented as an extension of PyTorch nn.Module
  59. pytorch_geometric: Geometric Deep Learning Extension Library for PyTorch
  60. torchplus: Implements the + operator on PyTorch modules, returning sequences.
  61. lagom: lagom: A light PyTorch infrastructure to quickly prototype reinforcement learning algorithms.
  62. torchbearer: torchbearer: A model training library for researchers using PyTorch.
  63. pytorch-maml-rl: Reinforcement Learning with Model-Agnostic Meta-Learning in Pytorch.
  64. NALU: Basic pytorch implementation of NAC/NALU from Neural Arithmetic Logic Units paper by trask et.al arxiv.org/pdf/1808.00508.pdf
  65. QuCumber: Neural Network Many-Body Wavefunction Reconstruction
  66. magnet: Deep Learning Projects that Build Themselves http://magnet-dl.readthedocs.io/
  67. opencv_transforms: OpenCV implementation of Torchvision’s image augmentations
  68. fastai: The fast.ai deep learning library, lessons, and tutorials
  69. pytorch-dense-correspondence: Code for “Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation” arxiv.org/pdf/1806.08756.pdf
  70. colorization-pytorch: PyTorch reimplementation of Interactive Deep Colorization richzhang.github.io/ideepcolor
  71. beauty-net: A simple, flexible, and extensible template for PyTorch. It’s beautiful.
  72. OpenChem: OpenChem: Deep Learning toolkit for Computational Chemistry and Drug Design Research mariewelt.github.io/OpenChem
  73. torchani: Accurate Neural Network Potential on PyTorch aiqm.github.io/torchani
  74. PyTorch-LBFGS: A PyTorch implementation of L-BFGS.
  75. gpytorch: A highly efficient and modular implementation of Gaussian Processes in PyTorch.
  76. hessian: hessian in pytorch.
  77. vel: Velocity in deep-learning research.
  78. nonechucks: Skip bad items in your PyTorch DataLoader, use Transforms as Filters, and more!
  79. torchstat: Model analyzer in PyTorch.
  80. QNNPACK: Quantized Neural Network PACKage – mobile-optimized implementation of quantized neural network operators.
  81. torchdiffeq: Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.
  82. redner: A differentiable Monte Carlo path tracer
  83. pixyz: a library for developing deep generative models in a more concise, intuitive and extendable way.
  84. euclidesdb: A multi-model machine learning feature embedding database http://euclidesdb.readthedocs.io
  85. pytorch2keras: Convert PyTorch dynamic graph to Keras model.
  86. salad: Semi-Supervised Learning and Domain Adaptation.
  87. netharn: Parameterized fit and prediction harnesses for pytorch.
  88. dgl: Python package