> For the complete documentation index, see [llms.txt](https://lauradang.gitbook.io/notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://lauradang.gitbook.io/notes/machine-learning/recurrent-neural-networks/recurrent-neural-networks.md).

# Recurrent Neural Networks

**What is a neural network?**

* set of algorithms designed to recognize patterns

### Artificial Neural Network (ANN)

* Has layers
* First layer receives raw input information
* Inner layers process raw input
* Last tier produces output

### How does RNN make decisions?

* Performs same function for every input of data
* Output of input depends on past one computation
* After output is produced, output is copied and sent back into RNN
* Makes decision by considering current input and output that it has learned form previous input
  * Decisions based on what it learned from past

**Example**: Sequence of input: $$X(n)$$

1. Takes $$X(0)$$ from sequence of input
2. Outputs $$h(0)$$
3. $$h(0)$$ and $$X(1)$$ is the input for the next step (i.e. $$\[h(0), X(1)]$$ is 1 vector)
4. Goes through activation function
5. Outputs $$h(1)$$
6. $$h(1)$$ and $$X(2)$$ is the input for the next step
7. Continues..

### Compare NN to RNN

| network | input              | output                              |
| ------- | ------------------ | ----------------------------------- |
| NN      | Fixed sized vector | Fixed sized vector                  |
| RNN     |                    | Series of vectors (no pre-set size) |