From aa74b53283de1ddb82f7534508d57fead00e6a95 Mon Sep 17 00:00:00 2001 From: Taylor Price Date: Wed, 27 Mar 2024 13:39:48 -0700 Subject: [PATCH] chore: add page documenting use of alternative model providers Signed-off-by: Taylor Price --- docs/docs/04-alternative-model-providers.md | 56 +++++++++++++++++++ .../docs/{04-use-cases.md => 06-use-cases.md} | 0 ...-reference.md => 07-gpt-file-reference.md} | 0 3 files changed, 56 insertions(+) create mode 100644 docs/docs/04-alternative-model-providers.md rename docs/docs/{04-use-cases.md => 06-use-cases.md} (100%) rename docs/docs/{06-gpt-file-reference.md => 07-gpt-file-reference.md} (100%) diff --git a/docs/docs/04-alternative-model-providers.md b/docs/docs/04-alternative-model-providers.md new file mode 100644 index 00000000..b76b7f1a --- /dev/null +++ b/docs/docs/04-alternative-model-providers.md @@ -0,0 +1,56 @@ +# Alternative Model Providers + + +## Usage + +GPTScript can be used against alternative models that expose an OpenAI compatible API or have a provider shim available. + +To use a model with an OpenAI compatible API: + +```gptscript +model: mistral-large-latest from https://api.mistral.ai/v1 + +Say hello world +``` + +To use a model that requires a provider shim: + +```gptscript +model: claude-3-haiku-20240307 from github.com/gptscript-ai/anthropic-provider + +Say hello world +``` + +### Authentication + +For OpenAI compatible providers, GPTScript will look for an API key to be configured with the prefix `GPTSCRIPT_PROVIDER_`, the base domain converted to environment variable format, and a suffix of `_API_KEY`. +As an example if you are using `mistral-large-latest from https://api.mistral.ai/v1`, the environment variable would be `GPTSCRIPT_PROVIDER_API_MISTRAL_AI_API_KEY` + +Each provider shim has different requirements for authentication. Please check the readme for the provider you are trying to use. + +## Available Model Providers + +The following shims are currently available: +* [github.com/gptscript-ai/azure-provider](https://github.com/gptscript-ai/azure-provider) +* [github.com/gptscript-ai/anthropic-provider](https://github.com/gptscript-ai/anthropic-provider) +* [github.com/gptscript-ai/google-provider](https://github.com/gptscript-ai/google-provider) + +## Listing available models + +For any provider that supports listing models, you can use this command: + +```bash +# With a shim +gptscript --list-models github.com/gptscript-ai/anthropic-provider + +# To OpenAI compatible endpoint +gptscript --list-models https://api.mistral.ai/v1 +``` + + +## Compatibility + +While the shims provide support for using GPTScript with other models, the effectiveness of using a +different model will depend on a combination of prompt engineering and the quality of the model. You may need to change +wording or add more description if you are not getting the results you want. In some cases, the model might not be +capable of intelligently handling the complex function calls. diff --git a/docs/docs/04-use-cases.md b/docs/docs/06-use-cases.md similarity index 100% rename from docs/docs/04-use-cases.md rename to docs/docs/06-use-cases.md diff --git a/docs/docs/06-gpt-file-reference.md b/docs/docs/07-gpt-file-reference.md similarity index 100% rename from docs/docs/06-gpt-file-reference.md rename to docs/docs/07-gpt-file-reference.md