THE AIGNOSTIC

Author: Thomas

  • Adding Company Data to LLMs with Retrieval Augmented Generation (RAG)

    Adding Company Data to LLMs with Retrieval Augmented Generation (RAG)

    Customization Options for LLMs

    Before we look into Retrieval Augmented Generation a short overview of customisation options of LLMs.

    • Prompt Engineering: Customizes input for better responses.
    • RAG: Integrates external data for immediate context.
    • Fine-Tuning: Alters the model for specific tasks or languages.
    • Pretraining: Involves training from scratch with custom data, very costly and time-intensive.

    What is retrieval-augmented generation?

    Retrieval Augmented Generation (RAG) Framework

    Instead of relying solely on its pre-trained knowledge, the model uses this fresh data to generate responses that are more accurate, up-to-date, and tailored to the context, effectively turning static AI into a dynamic information wizard.

    Challenges Solved by Retrieval Augmented Generation (RAG)

    Problem 1: LLMs Lack Access to Your / New Data

    • LLMs are trained on vast public datasets but can’t access new or private data post-training, leading to outdated or incorrect responses.

    Problem 2: Need for Custom Data in AI Applications

    • Effective AI applications, like customer support bots or internal Q&A systems, require domain-specific knowledge, which static LLMs lack without additional training.

    RAG integrates specific data into the LLM’s prompts, allowing the model to use real-time or custom data for more accurate responses without retraining.

    Use Cases for RAG:

    • Question and Answer Chatbots: Enhances chatbot accuracy by using company documents.
    • Search Augmentation: Improves search results with LLM-generated answers.
    • Knowledge Engine: Enables quick answers from internal data like HR documents.

    Benefits of RAG:

    • Up-to-Date Responses: Provides current information by integrating external data sources.
    • Reduces Hallucinations: Minimizes incorrect or fabricated answers with verified external context.
    • Domain-Specific Answers: Tailors responses to fit organizational data needs.
    • Cost-Effective: Does not require model retraining, saving time and resources.

    RAG vs. Fine-Tuning

    When to Use RAG vs. Fine-Tuning the Model

    Start with RAG if you want a quick, effective solution that leverages real-time data without altering the model’s core behavior. Opt for fine-tuning when you need to modify the model’s behavior or teach it a new domain specific “language.” Remember, these methods can complement each other. Consider fine-tuning for deeper understanding and output precision, while using RAG for up-to-date, contextually relevant responses.

    • Use RAG when you need quick, relevant responses without model changes.
    • Fine-Tune when you want to alter the model’s behavior or language understanding. Both can be used together for optimal results.

  • Python xAI Chatbot Tutorial

    Python xAI Chatbot Tutorial

    Make your own xAI chatbot in 3 minutes

    All the information I found was incomplete or simply did not work. ChatGPT was not much help either. All AI overlord code did not work.

    So here it is. How to make your own xAI Chatbot?

    1. Install Python and OpenAi library
    pip install openai
    1. Create your xAI account and get your API key
      Visit the API Keys page within the console. Here, you’ll create your very own API key, the magical token that grants you access to the Grok API.
    2. Run Python in Visual Studio Code (or wherever)
    import os
from openai import OpenAI

XAI_API_KEY = "xai-YOUR_KEY_COMES_HERE_MATE"
client = OpenAI(
    api_key=XAI_API_KEY,
    base_url="https://api.x.ai/v1",
)

def chat_with_gpt(prompt):
    response = client.chat.completions.create(
        model = "grok-beta",
        messages=[{"role": "user", "content": prompt}],
        #temperature = 0.8,
    )
    return response.choices[0].message.content.strip()

if __name__ == "__main__":
    while True:
        user_input = input("Tom: ")
        if user_input.lower() in ["quit", "exit", "bye"]:
            break
        
        response = chat_with_gpt(user_input)
        print("AIgnostic: ", response)

    That’s all. You got your own little chatbot my friend.

  • What is the difference between custom GPTs and Fine-Tuning a GPT?

    What is the difference between custom GPTs and Fine-Tuning a GPT?

    In November 2023 OpenAi rolled out custom GPTs.

    We’re rolling out custom versions of ChatGPT that you can create for a specific purpose—called GPTs. Anyone can easily build their own GPT—no coding is required.

    So the question arises.

    What is the difference between custom GPTs and Fine-Tuning?

    • Custom GPTS are all based on the same AI model that remains unaltered. You « just » give the GPT instructions and documents, they can be modified at anytime to update the GPT to your convenience. Basically you are uploading some PDFs to improve answers.




      Imagine giving a new employee some print outs in order to better answer some basic client phone calls. These manuals might help answer some specific question but will not train your new employee.

    • Fine-tuning is giving new knowledge to an AI by retraining it. You « feed it » new data that will now be part of the AI, therefore changing the core of that specific AI. The whole model needs to be retrained. You are looking at costs of thousands.


      It’s like sending a person someone to some course or university. It will deeply alter that persons knowledge and understanding.

    So in the second case the AI is modified in its core, while in the first case it is all about providing instructions to guide the existing AI. Fine-tuning is more complex and expensive, you need new quality data in the right format.

    More here: https://community.openai.com/t/custom-gpts-vs-fine-tuning-whats-the-difference/477738/2

  • Python ChatGPT Chatbot Tutorial

    Python ChatGPT Chatbot Tutorial

    How to set up your own Chatbot interface with Python in 3 minutes?

    1. Get an OpenAI account and get your API key

      https://platform.openai.com
    2. Install Python and the OpenAI Python library
    pip install openai

    3. Run Python in Visual Studio Code (or wherever)

    import openai
openai.api_key = "sk-YOUR_KEY_COMES_HERE_MATE"

def chat_with_gpt(prompt):
    response = openai.chat.completions.create(
        model = "o1-mini",
        messages=[
            {"role": "assistant", "content": "You are a helpful assistant." },
            {"role": "user", "content": prompt}
            ],
        #temperature = 0.7,
    )
    return response.choices[0].message.content.strip()

if __name__ == "__main__":
    while True:
        user_input = input("Tom: ")
        if user_input.lower() in ["quit", "exit", "bye"]:
            break
        
        response = chat_with_gpt(user_input)
        print("AIgnostic: ", response)

    You can specify any model you would like to test.

  • Hello world!

    Hello world!

    The AIgnostic – They don’t just follow AI; they philosophize about its societal impact with an almost religious fervor.

    This is my notebook on the sociological, socioeconomic and technical impact of AI.

    I can only assume the next few years will be a wild ride…