A Hacker's Guide to Language Models

21 Jun, 2024

I watched this video by Jeremy Howard and I really really liked it.

Some thoughts, first on Jeremy Howard:

• He co-founded Kaggle! And Fast AI. And FastMail. And he wrote the ULMFit paper that all current language models are based on. The guy is incredible!

Then, on the video:

• There are so many interesting ways you can prompt (or prime) language models into generating the responses you're looking for. In particular, the prompt to get GPT-4 to correctly answer the river crossing problem is quite verbose, but ultimately gets the job done.
• Reasoning and logic tasks that are outside the box - not great.
• Sentence Transformers can be used to create an embedding for some sentences. In his example, this is the first paragraph of the Wikipedia page.
  • These embeddings can be prepared in a one-off job and stored in a vector database. That can then be queried to find the most appropriate document for a particular question.
  • Appropriateness of the document can be established using cosine similarity.
  • I'd heard these terms bandied about a lot, but I never really understood them. This video demonstrated them really neatly.
• It's surprisingly straightforward to use these tools, even locally. I particularly liked the RAG (Retrieval Augmented Generation) example he used to show how a self-hosted solution (h2ogpt) would work.