RAG
Retrieval-Augmented Generation (RAG) gives the model access to an external knowledge base at the moment it needs to answer a question. Instead of relying only on what it memorized during training, the system first searches a database for relevant documents, then includes those documents in the prompt alongside your question. The model reads the retrieved passages and uses them to give a more accurate, up-to-date answer.
Knowledge Base (3 documents)
How Neural Networks Work
Neural networks are computational models loosely inspired by the human brain. They consist of layers of interconnected nodes (neurons). Each connection carries a weight that is adjusted during training. The network learns by minimizing a loss function using gradient descent and backpropagation.
The History of Transformers
The Transformer architecture was introduced by Vaswani et al. in the 2017 paper "Attention Is All You Need." It replaced recurrent neural networks for sequence tasks by relying entirely on self-attention mechanisms. BERT (2018) demonstrated bidirectional pre-training; GPT showed autoregressive scaling works well.
Attention Mechanisms Explained
Attention allows a model to focus on the most relevant parts of the input. Scaled dot-product attention computes Query, Key, and Value projections. The attention score between position i and j is: score(i,j) = dot(Q_i, K_j) / sqrt(d_k). Softmax converts scores into a probability distribution, then a weighted sum of Values is returned.
Enter a query above or in the pipeline header to begin.