How to beginner · 3 min read

How to collect user feedback on AI outputs

Quick answer

Collect user feedback on AI outputs by integrating explicit feedback prompts in your application UI or by logging user interactions and ratings programmatically. Use OpenAI or Anthropic APIs to generate outputs and capture user responses for continuous improvement.

PREREQUISITES

Python 3.8+
OpenAI API key (free tier works)
pip install openai>=1.0

Setup

Install the openai Python package and set your API key as an environment variable to authenticate requests.

bash

pip install openai>=1.0

Step by step

This example shows how to generate AI output using gpt-4o and collect user feedback via a simple console prompt. The feedback is then logged for analysis.

python

import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

# Generate AI output
response = client.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Explain the benefits of AI in healthcare."}]
)

ai_output = response.choices[0].message.content
print("AI Output:\n", ai_output)

# Collect user feedback
user_feedback = input("Please rate this output from 1 (poor) to 5 (excellent): ")

# Log feedback (here we just print, but you can save to a database or file)
print(f"User feedback recorded: {user_feedback}")

output

AI Output:
 AI can improve healthcare by enabling faster diagnosis, personalized treatment, and efficient data management.
Please rate this output from 1 (poor) to 5 (excellent): 4
User feedback recorded: 4

Common variations

You can collect feedback asynchronously by integrating feedback forms in your web or mobile app UI, or use streaming APIs to gather real-time reactions. Different models like claude-3-5-sonnet-20241022 can be used similarly with their respective SDKs.

python

import anthropic
import os

client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=300,
    system="You are a helpful assistant.",
    messages=[{"role": "user", "content": "Summarize the impact of AI on education."}]
)

ai_output = response.content[0].text
print("AI Output:\n", ai_output)

user_feedback = input("Rate this summary from 1 to 5: ")
print(f"User feedback recorded: {user_feedback}")

output

AI Output:
 AI has transformed education by enabling personalized learning, automating grading, and providing access to global resources.
Rate this summary from 1 to 5: 5
User feedback recorded: 5

Troubleshooting

If you receive authentication errors, verify your API key is correctly set in the environment variable.
If user feedback input is invalid, implement validation to ensure ratings are within the expected range.
For network issues, check your internet connection and API endpoint availability.

✅

Key Takeaways

Embed explicit feedback prompts in your app to collect user ratings on AI outputs.
Log feedback data systematically for continuous AI model improvement.
Use SDKs like OpenAI and Anthropic to generate outputs and collect feedback programmatically.

Verified 2026-04 · gpt-4o, claude-3-5-sonnet-20241022

Verify ↗