Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

o1-mini not working for GPTEvaluater #2841

Open
sam-semillia opened this issue Dec 31, 2024 · 2 comments
Open

o1-mini not working for GPTEvaluater #2841

sam-semillia opened this issue Dec 31, 2024 · 2 comments

Comments

@sam-semillia
Copy link

Step 1: Have you search for this issue before posting it?

yes

Step 2: Describe your environment

running docker, version latest

Step 3: Describe the problem:

Describe the bug
set GPT Evaluater GPT Model: the GPT model to use. to "o1-mini" gives the following error:
"ERROR | GPTEvaluator | Invalid GPT request: Error when running request with model o1-mini (invalid request): Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'system' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}}

Expected behavior
It does not give an error, as with gpt-4, gpt-4o etc.

Steps to reproduce:

  1. Navigate to Confiuration of GPTEvaluator
  2. Set GPT Model to "o1-mini"

Observed Results:

  • What happened? an error occured using chat gpt
  • What did you expect to happen? no error occurs and chatgpt works as expected

Relevant code exceptions or logs:

"ERROR | GPTEvaluator | Invalid GPT request: Error when running request with model o1-mini (invalid request): Error code: 400 - {'error': {'message': "Unsupported value: 'messages[0].role' does not support 'system' with this model.", 'type': 'invalid_request_error', 'param': 'messages[0].role', 'code': 'unsupported_value'}}

@sam-semillia sam-semillia changed the title o1-mini o1-mini not working for GPTEvaluater Jan 1, 2025
@ruidazeng
Copy link

The o1-mini model does not support the system role in the Chat Completion format

@GuillaumeDSM
Copy link
Member

Thanks for reporting this issue @sam-semillia
Unfortunately, as o1-mini doesn't yet support system prompts, it can't be used for predictions. The next version of OctoBot will display a better error message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants