2017 The year of Deep Learning frameworks

2017 the year of Deep Learning frameworks b

2017 was a special year for Deep Learning. In addition to the great experimental results obtained thanks to the algorithms developed, the Deep Learning this year has seen its glory in the release of many frameworks. These are very useful tools for developing numerous projects. In the article you will see an overview of many new frameworks that have been proposed as excellent tools for the development of Deep Learning projects.

2017 the year of Deep Learning frameworks

PyTorch

Among all the frameworks, perhaps the most successful and surprised everyone for its potential is PyTorch, the framework for Deep Learning proposed by Facebook.

pyTorch

PyTorch is a Python package that mainly provides two high-level features:

  • Tensor calculus (like Numpy) with GPU acceleration
  • Deep Neural Networks built on a tape-based autograd system

In fact, if you choose to use this package, certainly you do it for these two reasons. The first is to use it instead of Numpy, in order to make the most of the computational power of the GPUs. The second is to use a Deep Learning research platform that can provide maximum performance and flexibility.

PyTorch is a library that is mainly based on tensors. Certainly if you have already used numpy you will certainly have had to deal with tensors and tensor calculus. But this library will allow you to make the most of both your CPUs and GPUs available, to the best it can, thus having a huge calculation speed.

The dynamic neural networks used are based on a tape-based autograd system. In fact PyTorch has only one way to construct the neural networks, that is to use and re-run a tape recorder. Other very famous frameworks like Theano, TensorFlow have a static view of the world: they build a neural network once and for all and then reuse the same structure over and over again.

With PyTorch, a technique called Reverse-mode auto-differentiation is used, which allows to modify the behavior of the neural network in an arbitrary way.

During the last year, PyTorch has received a lot of attention from many researchers especially in the Natural Language Processing,  preferring it to other frameworks for its high dynamism.

Deep Learning with PyTorch: 
Hands-On Deep Learning with PyTorch

Tensorflow

Like Facebook, Google recently introduced TensorFlow, releasing its first Tensorflow 1.0  version in February 2017 with a stable and compatible API. Currently Tensorflow is at version 1.5.

tensorflow logo

In addition to the main framework, several additional libraries have been released, including Tensorflow Fold  for dynamic calculation charts, Tensorflow Transform for input data pipelines, and DeepMind’s higher-level Sonnet library.

And given the great success of PyTorch, the Tensorflow development team recently announced that it will soon be released a new version that will work very similar to PyTorch.

tensorflow how it works

TensorFlow is a library uses graphs (data flow graphs) to perform the mathematical calculations, in these graphs the data flow being gradually processed. Each node (nodes) represents mathematical operations, while the connections (edges) between the nodes represent the multidimensional (tensor) data arrays. From here comes his name TensorFlow.

The flexible architecture of this library allows you to perform calculations using one or more CPUs or GPUs using only one API.

Getting Started with TensorFlow
Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow, 2nd Edition
Learning Tensorflow: A Guide to Building Deep Learning Systems

Other frameworks

Like Google and Facebook, many other companies are developing and working on Machine Learning frameworks.

CoreML logo

Apple has announced that its reference library for machine learning will be CoreML.

Pyro Deep universal probabilistic programming

Uber has released Pyro, , a programming language “Deep Probabilistic” together with  Michelangelo , which will be its internal platform for Machine Learning

Uber Michelangelo machine learning platform
Gluon

Amazon has instead presented  Gluon, a high-level API for machine learning available for MXNet, the Apache Deep Learning framework.

Apache MxNet

Framework for Reinforcement Learning

But the new frameworks do not end there. In fact, what we have seen so far have a general purpose for Deep Learning, there are also more specialized frameworks, in particular for Reinforcement Learning

  • OpenAI Roboschool is an open-source software for robot simulation
  • OpenAI Baselines is a set of Reinforcement Learning algorithms.
  • Tensorflow Agents contains an optimized infrastructure to instruct Reinforcement Learning agents using Tensorflow.
  • Unity ML Agents allows researchers and developers to create games and simulators using Unity Editor and instructing them using Reinforcement Learning.
  • Nervana Coach allows experimentation with the Reinforcement Learning algorithms.
  • Facebook’s ELF is the game search platform.
  • DeepMind Pycolab is a game engine for a customizable gridworld.
  • Geek.ai MAgent is a research platform for many reinforcement learning agents.

Other frameworks

With the aim of making Deep Learning more accessible, there are some very useful frameworks on the web, such as Google’s deeplearn.js and MIL WebDNN.

Theano’s death

Theano library

2017 was the year of Deep Learning, and it brought us great innovations with the introduction and release of so many frameworks and libraries. But a very popular framework for all is dead: Theano. In fact it has been announced to everyone that 1.0 will be the last released release.

ONNX

But the news certainly does not end here. Given the large number of frameworks present for Deep Learning and Machine Learning in general, Facebook and Microsoft have announced  ONNX an open format for sharing deep learning models through frameworks.

ONNX open neural network exchange format

In fact ONNX all developers can develop and represent Deep Learning models using the different tools or libraries present today and choose the combination that best suits your needs. It is also possible to develop using a model and then replace it with another later.

Conclusions

As you can see from the article, 2017 was the year of Deep Learning. Every big company (Uber, Facebook, Google, Amazon) has developed and released one or more frameworks that allow to develop and implement Deep Learning. So 2018 promises a year full of development for the use of all these tools made available.

 [:]



Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.