Skip to content

chore: add page documenting use of alternative model providers #193

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions docs/docs/04-alternative-model-providers.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
# Alternative Model Providers


## Usage

GPTScript can be used against alternative models that expose an OpenAI compatible API or have a provider shim available.

To use a model with an OpenAI compatible API:

```gptscript
model: mistral-large-latest from https://api.mistral.ai/v1

Say hello world
```

To use a model that requires a provider shim:

```gptscript
model: claude-3-haiku-20240307 from github.com/gptscript-ai/anthropic-provider

Say hello world
```

### Authentication

For OpenAI compatible providers, GPTScript will look for an API key to be configured with the prefix `GPTSCRIPT_PROVIDER_`, the base domain converted to environment variable format, and a suffix of `_API_KEY`.
As an example if you are using `mistral-large-latest from https://api.mistral.ai/v1`, the environment variable would be `GPTSCRIPT_PROVIDER_API_MISTRAL_AI_API_KEY`

Each provider shim has different requirements for authentication. Please check the readme for the provider you are trying to use.

## Available Model Providers

The following shims are currently available:
* [github.com/gptscript-ai/azure-provider](https://github.com/gptscript-ai/azure-provider)
* [github.com/gptscript-ai/anthropic-provider](https://github.com/gptscript-ai/anthropic-provider)
* [github.com/gptscript-ai/google-provider](https://github.com/gptscript-ai/google-provider)

## Listing available models

For any provider that supports listing models, you can use this command:

```bash
# With a shim
gptscript --list-models github.com/gptscript-ai/anthropic-provider

# To OpenAI compatible endpoint
gptscript --list-models https://api.mistral.ai/v1
```


## Compatibility

While the shims provide support for using GPTScript with other models, the effectiveness of using a
different model will depend on a combination of prompt engineering and the quality of the model. You may need to change
wording or add more description if you are not getting the results you want. In some cases, the model might not be
capable of intelligently handling the complex function calls.
File renamed without changes.
File renamed without changes.