Using RAG with local documents

lewtun
Tue, 06 Apr 2021 21:19:23 +0000

hey @saichandra, one possibility would be to use Haystack’s implementation of RAG (which is based on HF transformers), e.g. see here: https://haystack.deepset.ai/docs/latest/tutorial7md

one advantage of using Haystack is that they provide a nice API for FAISS (and other document stores) so you can store the embeddings locally with just a few lines of code.

i’ve had mixed results from using RAG with the Natural Questions checkpoints (the answers are often gibberish). if you’re doing QA on a specialised corpus, you might be better off using the classic Retriever-Reader architecture or fine-tuning RAG on your domain