Skip to content
This repository was archived by the owner on Jun 24, 2024. It is now read-only.
This repository was archived by the owner on Jun 24, 2024. It is now read-only.

Implement SuperHOT/interpolated RoPE support #378

@philpax

Description

@philpax

Another llama.cpp feature that seems to have shrunk the paper-to-implementation pipeline to less than one week!

This allows for a much longer context (assuming you have the (V)RAM for it)

We can probably close out #77 if this is done.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions