How to beginner · 3 min read

How to build test generator with LLM

Quick answer

Use a large language model like gpt-4o via the OpenAI API to generate test questions by prompting it with your desired topic and question format. Implement a Python script that sends prompts and parses the generated questions for automated test creation.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works)
pip install openai>=1.0

Setup

Install the openai Python package and set your API key as an environment variable for secure access.

bash

pip install openai>=1.0

Step by step

This example uses gpt-4o to generate multiple-choice test questions on a given topic. The prompt instructs the model to create questions with four options and the correct answer indicated.

python

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

def generate_test(topic: str, num_questions: int = 5) -> str:
    prompt = (
        f"Generate {num_questions} multiple-choice questions on the topic '{topic}'. "
        "Each question should have 4 options labeled A to D, and specify the correct answer. "
        "Format the output as numbered questions with options and answers."
    )
    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

if __name__ == "__main__":
    topic = "Python programming basics"
    test_content = generate_test(topic)
    print(test_content)

output

1. What is the output of print(len('Hello'))?
A) 4
B) 5
C) 6
D) Error
Answer: B

2. Which keyword is used to define a function in Python?
A) func
B) define
C) def
D) function
Answer: C

3. What data type is returned by the expression 3 / 2 in Python 3?
A) int
B) float
C) str
D) bool
Answer: B

4. Which of the following is a mutable data type?
A) tuple
B) list
C) string
D) int
Answer: B

5. How do you start a comment in Python?
A) //
B) <!--
C) #
D) /*
Answer: C

Common variations

Use gpt-4o-mini for faster, cheaper generation with smaller context.
Implement async calls with asyncio and await client.chat.completions.create(...) for concurrency.
Stream responses by setting stream=True to display questions as they generate.
Customize prompts to generate different question types like true/false, fill-in-the-blank, or coding exercises.

python

import asyncio
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

async def generate_test_async(topic: str, num_questions: int = 3) -> str:
    prompt = (
        f"Generate {num_questions} true/false questions on the topic '{topic}'. "
        "Include the correct answer after each question."
    )
    response = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content

async def main():
    test = await generate_test_async("Data structures")
    print(test)

if __name__ == "__main__":
    asyncio.run(main())

output

1. A linked list is a linear data structure. True
Answer: True

2. A stack follows FIFO order. True
Answer: False

3. A hash table uses key-value pairs for storage. True
Answer: True

Troubleshooting

If you get authentication errors, verify your OPENAI_API_KEY environment variable is set correctly.
If the model returns incomplete questions, increase max_tokens in the API call.
For rate limit errors, implement exponential backoff retries or reduce request frequency.
Ensure your prompt is clear and specific to get consistent question formatting.

✅

Key Takeaways

Use gpt-4o or gpt-4o-mini with clear prompts to generate test questions automatically.
Implement async and streaming calls for scalable and responsive test generation.
Customize prompts to generate various question types and formats for diverse assessments.

Verified 2026-04 · gpt-4o, gpt-4o-mini

Verify ↗