How To Train An OpenAI Language Model With Your Own Data

OpenAI’s GPT language models are great, but sometimes the response you get from chatbots like ChatGPT doesn’t always align with your expectations. It’s like ordering a burger and getting a salad instead — it’s not cool.

Problems like this can be solved by adding more specific instructions to the text prompt you feed into the chatbot. For example, if you want the AI to respond sarcastically, you would need to add something like “respond in a sarcastic way” at the end of the prompt.

But let’s be real, who has time to babysit an AI with constant instructions?

What if instead of always asking the AI to respond in a specific way, you can train the AI by giving a few examples to learn how to perform a task?

This is called fine-tuning.

What is fine-tuning?

Fine-tuning a Language Model, or LLM fine-tuning, is the process of taking a pre-trained large language model (LLM) and further training it on a specific dataset to adapt it for a more specialized task or for use in a specific domain.

In this article, I am going to show you how you can train an AI model with your own data. Let’s get started.

Step #1: Get Started with OpenAI Playground

Head over to OpenAI, create an account, and open the playground dashboard.

Just a heads up, using the playground to use OpenAI features isn’t free. On sign-up, you’ll receive a $5 free credit, which should be enough to test out some features including the fine-tuning.

Monitor the remaining credits you have under your Profile page and Billing tab. You can also link a credit card so you don’t have to worry about credit limits. Monitor your usage stats under the Usage tab of your Org’s dashboard.

Now that your OpenAI account is set up, we can start preparing the training data.

Step #2: Prepare the Training Data

Right now, you can’t just upload any file as your training data for the training model. OpenAI requires you to transform them into a JSONL format like the below:

{"messages": [{"role": "system", "content": "Article Intro is a chatbot that generates human written introduction for articles"}, {"role": "user", "content": "Write an introduction for an article with title Scarlet Johansson Is Not Happy With OpenAI Using Her Voice For ChatGPT"}, {"role": "assistant", "content": "OpenAI is nothing without its drama.\n\nIt\u2019s been a week full of exciting announcements and shocking resignations in OpenAI. Days after the GPT-4o release, Jan Leike, the super alignment lead, and executive, announced his resignation. It doesn't end there as the actress of the 2014 hit film \u201cHer\u201d, Scarlet Johansson, who voiced the character of Samantha, has expressed anger over the ChatGPT\u2019s chatbot voice that \u201csounded so eerily similar\u201d to hers."}]}
{"messages": [{"role": "system", "content": "Article Intro is a chatbot that generates human written introduction for articles"}, {"role": "user", "content": "Write an introduction for an article with title Is Prompt Engineering Dead?"}, {"role": "assistant", "content": "Remember when prompt engineering for AI chatbots was a huge buzz, hundreds of courses sprouted like mushrooms on the internet? Some even considered it a profession and called themselves prompt engineers.\n\nAlthough I never called myself a prompt engineer, I was one of those wannabe cool guys who downloaded a bunch of prompt engineering guides to get the most out of popular language and image models like ChatGPT and Stable Diffusion.\n\nSome examples of these guides are:\n\n\u201cA Guide To Better Prompt Engineering\u201d\n\u201cPrompt Engineering Tips For Stable Diffusion\u201d\nWhat I didn\u2019t know back then was how fast these AI models can evolve. In just a few months, prompts for language and image models have become driven by natural language and are no longer bound to specific rules.\n\nToday, one of the leading AI companies, Anthropic, is taking prompt engineering to a new level by letting you generate production-ready prompts simply by describing what you want to achieve."}]}
{"messages": [{"role": "system", "content": "Article Intro is a chatbot that generates human written introduction for articles"}, {"role": "user", "content": "Write an introduction for an article with title ChatGPT Now Supports Interactive Tables And Charts Directly From Google Drive"}, {"role": "assistant", "content": "Less than a week after announcing GPT-4o, OpenAI has unveiled another cool feature. This new capability not only allows direct file uploads from Google Drive and Microsoft OneDrive but also enables users to work with interactive charts and tables.\n\nTo be clear, file support isn\u2019t new in ChatGPT. However, the ability to add files directly from other sources is new. This eliminates the need to download files to a user\u2019s desktop before uploading them to ChatGPT."}]}
{"messages": [{"role": "system", "content": "Article Intro is a chatbot that generates human written introduction for articles"}, {"role": "user", "content": "Write an introduction for an article with title Google Imagen 3 VS Midjourney V6"}, {"role": "assistant", "content": "At the Google IO 2024 event, Google announced a slew of brand-new products and huge AI updates. One of the major announcements was the brand new version of its text-to-image AI tool, Imagen 3.\n\nBased on what they showcased during the announcement, there has been a significant improvement in visual quality. Imagen 3 has reached a level where it can easily compete with Midjourney v6.\n\nBut how do these two AI image generators compare side by side?\n\nLet\u2019s dive in and find out."}]}

How many examples do you need?

The training actually just requires at least 10 examples to work. However, it is recommended that more data be collected to have an observable improvement in the output results. Here’s what OpenAI has to say:

We recommend starting with 50 well-crafted demonstrations and seeing if the model shows signs of improvement after fine-tuning. In some cases that may be sufficient, but even if the model is not yet production quality, clear improvements are a good sign that providing more data will continue to improve the model. No improvement suggests that you may need to rethink how to set up the task for the model or restructure the data before scaling beyond a limited example set.

I know, I know, this sounds like a tedious task, especially if you do it manually. So, it’s better to use tools or AI to automate this process for us. Efficiency is the name of the game.

First, I created a Google Sheets file and on the first column I added sample topics and on the second column, I added sample results.

Open up ChatGPT, upload the Google Sheets file, and ask the AI to transform it into a JSONL file using the prompt below:

Convert the content to a .jsonl file for fine-tuning. Follow this format
{"messages": [{"role": "system", "content": "Article Intro is a chatbot that generates human written introduction for articles"}, {"role": "user", "content": "Write an introduction for an article with the title [column 1]"}, {"role": "assistant", "content": "[column 2]"}]}

Hit enter and ChatGPT should be able to transform the data into something like this:

{'messages': [{'role': 'system', 'content': 'Article Intro is a chatbot that generates human written introduction for articles'}, {'role': 'user', 'content': 'Write an introduction for an article with the title Scarlet Johansson Is Not Happy With OpenAI Using Her Voice For ChatGPT'}, {'role': 'assistant', 'content': "OpenAI is nothing without its drama.\n\nIt’s been a week full of exciting announcements and shocking resignations in OpenAI. Days after the GPT-4o release, Jan Leike, the super alignment lead, and executive, announced his resignation. It doesn't end there as the actress of the 2014 hit film “Her”, Scarlet Johansson, who voiced the character of Samantha, has expressed anger over the ChatGPT’s chatbot voice that “sounded so eerily similar” to hers."}]}
{'messages': [{'role': 'system', 'content': 'Article Intro is a chatbot that generates human written introduction for articles'}, {'role': 'user', 'content': 'Write an introduction for an article with the title Is Prompt Engineering Dead?'}, {'role': 'assistant', 'content': 'Remember when prompt engineering for AI chatbots was a huge buzz, hundreds of courses sprouted like mushrooms on the internet? Some even considered it a profession and called themselves prompt engineers.\n\nAlthough I never called myself a prompt engineer, I was one of those wannabe cool guys who downloaded a bunch of prompt engineering guides to get the most out of popular language and image models like ChatGPT and Stable Diffusion.\n\nSome examples of these guides are:\n\n“A Guide To Better Prompt Engineering”\n“Prompt Engineering Tips For Stable Diffusion”\nWhat I didn’t know back then was how fast these AI models can evolve. In just a few months, prompts for language and image models have become driven by natural language and are no longer bound to specific rules.\n\nToday, one of the leading AI companies, Anthropic, is taking prompt engineering to a new level by letting you generate production-ready prompts simply by describing what you want to achieve.'}]}
{'messages': [{'role': 'system', 'content': 'Article Intro is a chatbot that generates human written introduction for articles'}, {'role': 'user', 'content': 'Write an introduction for an article with the title ChatGPT Now Supports Interactive Tables And Charts Directly From Google Drive'}, {'role': 'assistant', 'content': 'Less than a week after announcing GPT-4o, OpenAI has unveiled another cool feature. This new capability not only allows direct file uploads from Google Drive and Microsoft OneDrive but also enables users to work with interactive charts and tables.\n\nTo be clear, file support isn’t new in ChatGPT. However, the ability to add files directly from other sources is new. This eliminates the need to download files to a user’s desktop before uploading them to ChatGPT.'}]}
{'messages': [{'role': 'system', 'content': 'Article Intro is a chatbot that generates human written introduction for articles'}, {'role': 'user', 'content': 'Write an introduction for an article with the title Google Imagen 3 VS Midjourney V6'}, {'role': 'assistant', 'content': 'At the Google IO 2024 event, Google announced a slew of brand-new products and huge AI updates. One of the major announcements was the brand new version of its text-to-image AI tool, Imagen 3.\n\nBased on what they showcased during the announcement, there has been a significant improvement in visual quality. Imagen 3 has reached a level where it can easily compete with Midjourney v6.\n\nBut how do these two AI image generators compare side by side?\n\nLet’s dive in and find out.'}]}
{'messages': [{'role': 'system', 'content': 'Article Intro is a chatbot that generates human written introduction for articles'}, {'role': 'user', 'content': 'Write an introduction for an article with the title Freepik’s New Upscaler Feature Can Upscale Your Images Up To 4K'}, {'role': 'assistant', 'content': 'Freepik, one of the largest databases for high-quality stock photos, videos, vector illustrations, and more, has been going all-in with AI with the recent introduction of its AI-powered tools. In the past couple of weeks, Freepik has released innovative AI tools like Sketch-to-Image and Reimagine.\n\nToday, the company dropped another AI-powered tool, Upscaler, which can scale up low-resolution images by 400%.'}]}
{'messages': [{'role': 'system', 'content': 'Article Intro is a chatbot that generates human written introduction for articles'}, {'role': 'user', 'content': 'Write an introduction for an article with the title 5 Major Announcements In Google IO 2024'}, {'role': 'assistant', 'content': 'At this very moment, I am still mind-blown by the recently announced GPT-4o model from OpenAI. The voice assistant feature is incredibly impressive — the way it laughs, giggles, and sings makes Siri and Google Assistant seem primitive in comparison.\n\nWhile the much-anticipated voice assistant feature is set to roll out over the next couple of weeks, the text and image capabilities are already publicly available for eager early users to experiment with. In this article, I will share five FREE ways for you to dive in and try the GPT-4o model yourself.\n\nOh, and in case you missed it, the “o” in GPT-4o stands for “omni,” representing the model’s incredible multimodality support for both inputs and outputs. It can process and generate text, audio, and images in real-time. How cool is that?'}]}

Awesome, now we are ready to feed this file into the fine-tuning model.

Step #3: Create a Fine-Tuning Job

Go back to the OpenAI playground dashboard and click on the fine-tuning tab. Click on the “+Create” button to start creating a fine-tuning job.

In the base model list, select a model.

• GPT-3.5-turbo-0125 (recommended)
• GPT-3.5-turbo-1106
• GPT-3.5-turbo-0613
• Babbage-002
• DaVinci-002

You can also fine-tune a fine-tuned model which is useful if you acquire additional data and don’t want to repeat the previous training steps.

Give it a name, set the other settings as default, and click on the Create button.

Depending on the size of your JSON file, it may take a few minutes or several hours for the training to complete. You can monitor the progress on the right side of the dashboard.

You’ll also receive an email like this once the training is completed.

Great. Now we need to test the new model to see if it’s working as expected.

Step #4: Testing the Fine-tuned Model

The final step is to test the fine-tuned model. Click on the Playground button in the lower right corner, and you’ll be redirected to a screen with two chat instances.

On the left side is a chat instance with a GPT base model like GPT-3.5-turbo and on the right is a chant instance with your fine-tuned model. On the left side, make sure to select the same model you used in fine-tuning.

Simply send a query and you’ll notice the responses between the two are different. The result on the right should be more aligned with the training data you used to train the model.

Note that you can use this fine-tuned model to power your web apps by accessing them via API. I am not going to explain how to do that in this article, but if you’re interested in that topic, please let me know in the comments.

Final Thoughts

Even though this topic might seem a bit technical, it’s totally worth wrapping your head around. Knowing about AI techniques like fine-tuning can be super useful, even if you’re not an expert.

Just think about it — you could offer your skills to other companies and make some serious cash. Imagine training a model with their data and creating a custom chatbot that knows all about their business.

Plus, once you’ve got the hang of fine-tuning, you can use those skills for your own web apps. So, good luck on your journey to learning about generative AI.

‍