What's the difference between convolutional and recurrent neural networks?

Question

I'm new to the topic of neural networks. I came across the two terms convolutional neural network and recurrent neural network.

I'm wondering if these two terms are referring to the same thing, or, if not, what would be the difference between them?

Solution 1

Difference between CNN and RNN are as follows:

CNN:

1. CNN takes a fixed size inputs and generates fixed-size outputs.

2. CNN is a type of feed-forward artificial neural network - are variations of multilayer perceptrons which are designed to use minimal amounts of preprocessing.

3. CNNs use connectivity pattern between its neurons and is inspired by the organization of the animal visual cortex, whose individual neurons are arranged in such a way that they respond to overlapping regions tiling the visual field.

4. CNNs are ideal for images and video processing.

RNN:

1. RNN can handle arbitrary input/output lengths.

2. RNN unlike feedforward neural networks - can use their internal memory to process arbitrary sequences of inputs.

3. Recurrent neural networks use time-series information. i.e. what I spoke last will impact what I will speak next.

4. RNNs are ideal for text and speech analysis.

Solution 2

Convolutional neural networks (CNN) are designed to recognize images. It has convolutions inside, which see the edges of an object recognized on the image. Recurrent neural networks (RNN) are designed to recognize sequences, for example, a speech signal or a text. The recurrent network has cycles inside that implies the presence of short memory in the net. We have applied CNN as well as RNN choosing an appropriate machine learning algorithm to classify EEG signals for BCI: http://rnd.azoft.com/classification-eeg-signals-brain-computer-interface/

Solution 3

These architectures are completely different, so it is rather hard to say "what is the difference", as the only thing in common is the fact, that they are both neural networks.

Convolutional networks are networks with overlapping "reception fields" performing convolution tasks.

Recurrent networks are networks with recurrent connections (going in the opposite direction of the "normal" signal flow) which form cycles in the network's topology.

Solution 4

Apart from others, in CNN we generally use a 2d squared sliding window along an axis and convolute (with original input 2d image) to identify patterns.

In RNN we use previously calculated memory. If you are interested you can see, LSTM (Long Short-Term Memory) which is a special kind of RNN.

Both CNN and RNN have one point in common, as they detect patterns and sequences, that is you can't shuffle your single input data bits.

View more solutions

Comments

[ozgur]

The important point would be where they are used, which kind of problems each has advantage on another.

[rkellerm]

RNN definitely isn't restricted to natural language processing, although this is the field where it was first used. It's mainly (but not only) - for now - for sequences.