Orion-14B chat template is not support #6009
Description
The chat template for Orion's models is missing, and applying chatml format will give wrong response.
./build/bin/server -m ./Orion-14B-Chat.gguf -c 2048
curl --request POST --url http://localhost:8080/completion --header "Content-Type: application/json" --data '{"prompt": "
Write a c++ program printing G'day.","n_predict": 512}'
output:
{"content":"\nHere is a simple C++ program that prints \"Hello, World!\": \n\n```c++ \n#include <iostream> \n#include <iostream> Hello, this code snippet.\n#include hello world program to print \"Hello, using c++ Hello!","generation_settings":{"dynatemp_exponent":1.0,"dynatemp_range":0.0,"frequency_penalty":0.0,"grammar":"","ignore_eos":false,"logit_bias":[],"min_keep":0,"min_p":0.05000000074505806,"mirostat":0,"mirostat_eta":0.10000000149011612,"mirostat_tau":5.0,"model":"../ollama_wks/Orion-14B-Chat-Q2_K.gguf","n_ctx":2048,"n_keep":0,"n_predict":-1,"n_probs":0,"penalize_nl":true,"penalty_prompt_tokens":[],"presence_penalty":0.0,"repeat_last_n":64,"repeat_penalty":1.100000023841858,"samplers":["top_k","tfs_z","typical_p","top_p","min_p","temperature"],"seed":4294967295,"stop":[],"stream":false,"temperature":0.800000011920929,"tfs_z":1.0,"top_k":40,"top_p":0.949999988079071,"typical_p":1.0,"use_penalty_prompt_tokens":false},"id_slot":0,"model":"../ollama_wks/Orion-14B-Chat-Q2_K.gguf","prompt":"Write a c++ program printing hello world.","stop":true,"stopped_eos":true,"stopped_limit":false,"stopped_word":false,"stopping_word":"","timings":{"predicted_ms":25227.774,"predicted_n":69,"predicted_per_second":2.7350807883406594,"predicted_per_token_ms":365.6199130434783,"prompt_ms":1169.981,"prompt_n":9,"prompt_per_second":7.692432612153531,"prompt_per_token_ms":129.9978888888889},"tokens_cached":77,"tokens_evaluated":9,"tokens_predicted":69,"truncated":false}
gg model and chat template can be found here.
"{% for message in messages %}{% if loop.first %}{{ bos_token }}{% endif %}{% if message['role'] == 'user' %}{{ 'Human: ' + message['content'] + '\n\nAssistant: ' + eos_token }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token }}{% endif %}{% endfor %}"