Module: llama_llm

Classes

class LlamaLlm (*, llm_id: str = None)

Abstract base to cloud provider implementations of the LLAMA API, includes settings shared by all providers.

Expand source code

@dataclass(slots=True, kw_only=True)
class LlamaLlm(Llm, ABC):
    """Abstract base to cloud provider implementations of the LLAMA API, includes settings shared by all providers."""

    @classmethod
    def create_prompt_from_messages_v2(cls, messages: list[dict]) -> str:
        """
        Transforms list of messages in the following format:
        [
            {"role": "system", "content": "System Prompt"},
            {"role": "user", "content": "What is 2 + 2?"},
            {"role": "assistant", "content": "2+2 is equals to 4"},
            {"role": "user", "content": "Answer only with resulting number"},
        ]
        Into format supported by llama2:
        <s><<SYS>>System Prompt<</SYS>>
        [INST]What is 2 + 2?[/INST]
        2+2 is equals to 4
        </s>
        <s>[INST]Answer only with resulting number[/INST]</s>
        """
        system_prompt = None
        user_messages = []
        assistant_messages = []
        for message in messages:
            if message.role == "system":
                system_prompt = message.content
            elif message.role == "user":
                user_messages.append(message.content)
            elif message.role == "assistant":
                assistant_messages.append(message.content)

        if len(assistant_messages) < len(user_messages):
            assistant_messages.append("")
        message_pairs = list(zip(user_messages, assistant_messages))
        if system_prompt is not None:
            pair = message_pairs[0]
            message_pairs[0] = (f"<<SYS>>n{system_prompt}<</SYS>>nn" + pair[0], pair[1])
        message_pairs = [f"[INST] {pair[0]}[/INST]" + pair[1] for pair in message_pairs]
        prompt = "<s>" + "</s>n<s>".join(message_pairs)
        return prompt

    @classmethod
    def create_prompt_from_messages(cls, messages: list[dict]) -> str:
        """
        Transforms list of messages in the following format:
        [
            {"role": "system", "content": "System Prompt"},
            {"role": "user", "content": "What is 2 + 2?"},
            {"role": "assistant", "content": "2+2 is equals to 4"},
            {"role": "user", "content": "Answer only with resulting number"},
        ]
        Into format supported by llama3:
        <|begin_of_text|><|start_header_id|>system<|end_header_id|>

        You are a helpful AI assistant<|eot_id|>
        <|start_header_id|>user<|end_header_id|>

        What is 2 + 2?<|eot_id|>
        <|start_header_id|>assistant<|end_header_id|>

        2+2 is equals to 4<|eot_id|>
        <|start_header_id|>user<|end_header_id|>
        Answer only with resulting number<|eot_id|>
        """
        system_prompt = None
        user_messages = []
        assistant_messages = []
        for message in messages:
            if message.role == "system":
                system_prompt = message.content
            elif message.role == "user":
                user_messages.append(message.content)
            elif message.role == "assistant":
                assistant_messages.append(message.content)

        if len(assistant_messages) < len(user_messages):
            assistant_messages.append("")
        prompt = "<|begin_of_text|>"
        if system_prompt is not None:
            prompt += f"<|start_header_id|>system<|end_header_id|>nn{system_prompt}<|eot_id|>n"
        for user, assistant in zip(user_messages, assistant_messages):
            prompt += f"<|start_header_id|>user<|end_header_id|>nn{user}<|eot_id|>n"
            prompt += f"<|start_header_id|>assistant<|end_header_id|>"
            if assistant != "":
                prompt += f"nn{assistant}<|eot_id|>n"
        return prompt

Ancestors

• Llm
• LlmKey
• KeyMixin
• RecordMixin
• abc.ABC
• typing.Generic

Subclasses

• FireworksLlamaLlm

Static methods

def create_prompt_from_messages(messages: list[dict]) -> str

Transforms list of messages in the following format: [ {“role”: “system”, “content”: “System Prompt”}, {“role”: “user”, “content”: “What is 2 + 2?”}, {“role”: “assistant”, “content”: “2+2 is equals to 4”}, {“role”: “user”, “content”: “Answer only with resulting number”}, ] Into format supported by llama3: <|begin_of_text|><|start_header_id|>system<|end_header_id|>

You are a helpful AI assistant<|eot_id|> <|start_header_id|>user<|end_header_id|>

What is 2 + 2?<|eot_id|> <|start_header_id|>assistant<|end_header_id|>

2+2 is equals to 4<|eot_id|> <|start_header_id|>user<|end_header_id|> Answer only with resulting number<|eot_id|>

def create_prompt_from_messages_v2(messages: list[dict]) -> str

Transforms list of messages in the following format: [ {“role”: “system”, “content”: “System Prompt”}, {“role”: “user”, “content”: “What is 2 + 2?”}, {“role”: “assistant”, “content”: “2+2 is equals to 4”}, {“role”: “user”, “content”: “Answer only with resulting number”}, ] Into format supported by llama2: <>System Prompt<> [INST]What is 2 + 2?[/INST] 2+2 is equals to 4 [INST]Answer only with resulting number[/INST]

def get_key_type() -> Type

Inherited from: Llm.get_key_type

Return key type even when called from a record.

Fields

var llm_id -> str

Inherited from: Llm.llm_id

Unique LLM identifier.

Methods

def completion(self, query: str, *, trial_id: str | int | None = None) -> str

Inherited from: Llm.completion

Text-in, text-out single query completion without model-specific tags (uses response caching).

def get_key(self) -> LlmKey

Inherited from: Llm.get_key

Return a new key object whose fields populated from self, do not return self.

def init_all(self) -> None

Inherited from: Llm.init_all

Invoke ‘init’ for each class in the order from base to derived, then validate against schema.

def uncached_completion(self, request_id: str, query: str) -> str

Inherited from: Llm.uncached_completion

Perform completion without CompletionCache lookup, call completion instead.