fabricatio_core.capabilities.usages

Module for defining LLM (Large Language Model) usage capabilities.

This module contains classes and methods to manage interactions with LLMs, including: - Deploying models - Querying responses - Validating outputs - Generating embeddings - Selecting tools and toolboxes

It provides structured functionality for managing language model operations, embedding generation, and tool selection workflows.

Classes

UseLLM

Class that manages LLM (Large Language Model) usage parameters and methods.

UseEmbedding

A class representing the embedding model.

UseReranker

A class for reranking documents using a reranker model.

Module Contents

class fabricatio_core.capabilities.usages.UseLLM(/, **data: Any)[source]

Bases: fabricatio_core.models.generic.LLMScopedConfig, abc.ABC

Class that manages LLM (Large Language Model) usage parameters and methods.

This class provides methods to deploy LLMs, query them for responses, and handle various configurations related to LLM usage such as API keys, endpoints, and rate limits.

async aask(question: List[str], send_to: str | None = None, no_cache: bool | None = None, temperature: float | None = None, top_p: float | None = None, max_completion_tokens: int | None = None, stream: bool | None = None, presence_penalty: float | None = None, frequency_penalty: float | None = None) → List[str][source]
async aask(question: str, send_to: str | None = None, no_cache: bool | None = None, temperature: float | None = None, top_p: float | None = None, max_completion_tokens: int | None = None, stream: bool | None = None, presence_penalty: float | None = None, frequency_penalty: float | None = None) → str

Asynchronously asks the language model a question and returns the response content.

Parameters:

  • question (str | List[str]) – The question or list of questions to ask the model.

  • send_to (Optional[str]) – The target namespace for the request. If None, uses the default llm_send_to.

  • no_cache (Optional[bool]) – Whether to use cached responses. If None, uses llm_no_cache.

  • temperature (Optional[float]) – Sampling temperature for response generation. If None, uses llm_temperature.

  • top_p (Optional[float]) – Nucleus sampling parameter. If None, uses llm_top_p.

  • max_completion_tokens (Optional[int]) – Maximum number of tokens to generate. If None, uses llm_max_completion_tokens.

  • stream (Optional[bool]) – Whether to stream the response. If None, uses llm_stream.

  • presence_penalty (Optional[float]) – Presence penalty for response generation. If None, uses llm_presence_penalty.

  • frequency_penalty (Optional[float]) – Frequency penalty for response generation. If None, uses llm_frequency_penalty.

Returns:

The content of the model’s response message. Returns a single string if input is a string,

or a list of strings if input is a list of strings.

Return type:

str | List[str]

async aask_validate[T](question: str, validator: Callable[[str], T | None], default: T = ..., max_validations: pydantic.PositiveInt = 3, **kwargs: Unpack[fabricatio_core.models.kwargs_types.LLMKwargs]) → T[source]
async aask_validate(question: List[str], validator: Callable[[str], T | None], default: T = ..., max_validations: pydantic.PositiveInt = 3, **kwargs: Unpack[fabricatio_core.models.kwargs_types.LLMKwargs]) → List[T]
async aask_validate(question: str, validator: Callable[[str], T | None], default: None = None, max_validations: pydantic.PositiveInt = 3, **kwargs: Unpack[fabricatio_core.models.kwargs_types.LLMKwargs]) → T | None
async aask_validate(question: List[str], validator: Callable[[str], T | None], default: None = None, max_validations: pydantic.PositiveInt = 3, **kwargs: Unpack[fabricatio_core.models.kwargs_types.LLMKwargs]) → List[T | None]

Asynchronously asks a question and validates the response using a given validator.

Parameters:

  • question (str | List[str]) – The question to ask.

  • validator (Callable[[str], T | None]) – A function to validate the response.

  • default (T | None) – Default value to return if validation fails. Defaults to None.

  • max_validations (PositiveInt) – Maximum number of validation attempts. Defaults to 3.

  • **kwargs (Unpack[LLMKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The validated response.

Return type:

Optional[T] | List[T | None] | List[T] | T

async amapping_str(requirement: str, k: pydantic.NonNegativeInt = 0, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[Dict[str, str]]]) → Dict[str, str] | None[source]
async amapping_str(requirement: List[str], k: pydantic.NonNegativeInt = 0, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[Dict[str, str]]]) → List[Dict[str, str] | None] | None

Asynchronously generates a mapping of strings based on a given requirement.

Parameters:

  • requirement (str) – The requirement for the mapping of strings.

  • k (NonNegativeInt) – The number of choices to select, 0 means infinite. Defaults to 0.

  • **kwargs (Unpack[ValidateKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The validated response as a mapping of strings.

Return type:

Optional[Dict[str, str]]

async alist_str(requirement: str, k: pydantic.NonNegativeInt = 0, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[List[str]]]) → List[str] | None[source]
async alist_str(requirement: List[str], k: pydantic.NonNegativeInt = 0, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[List[str]]]) → List[List[str] | None] | None

Asynchronously generates a list of strings based on a given requirement.

Parameters:

  • requirement (str) – The requirement for the list of strings.

  • k (NonNegativeInt) – The number of choices to select, 0 means infinite. Defaults to 0.

  • **kwargs (Unpack[ValidateKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The validated response as a list of strings.

Return type:

Optional[List[str]]

async apathstr(requirement: str, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ChooseKwargs[str]]) → List[str] | None[source]

Asynchronously generates a list of strings based on a given requirement.

Parameters:

  • requirement (str) – The requirement for the list of strings.

  • **kwargs (Unpack[ChooseKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The validated response as a list of strings.

Return type:

Optional[List[str]]

async awhich_pathstr(requirement: str, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[List[str]]]) → str | None[source]

Asynchronously generates a single path string based on a given requirement.

Parameters:

  • requirement (str) – The requirement for the list of strings.

  • **kwargs (Unpack[ValidateKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The validated response as a single string.

Return type:

Optional[str]

async ageneric_string(requirement: str, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[str]]) → str | None[source]
async ageneric_string(requirement: List[str], **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[str]]) → List[str | None] | None

Asynchronously generates a generic string based on a given requirement.

Parameters:

  • requirement (str) – The requirement for the string.

  • **kwargs (Unpack[LLMKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The generated string.

Return type:

Optional[str]

async acode_string(requirement: str, code_language: str | None = None, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[str]]) → str | None[source]
async acode_string(requirement: List[str], code_language: str | None = None, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[str]]) → List[str | None]

Asynchronously generates code strings based on given requirements and code language.

Parameters:

  • requirement (str | List[str]) – The requirement(s) for generating code strings.

  • code_language (str) – The programming language for the generated code.

  • **kwargs (Unpack[ValidateKwargs[str]]) – Additional keyword arguments for the LLM usage.

Returns:

The generated code string(s). Returns a single string if requirement is a string, or a list of strings/None values if requirement is a list.

Return type:

None | str | List[str | None]

async acode_snippets(requirement: str, code_language: str | None = None, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[List[fabricatio_core.rust.CodeSnippet]]]) → List[fabricatio_core.rust.CodeSnippet] | None[source]
async acode_snippets(requirement: List[str], code_language: str | None = None, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[List[fabricatio_core.rust.CodeSnippet]]]) → List[List[fabricatio_core.rust.CodeSnippet] | None] | None

Asynchronously generates code snippets based on given requirements and code language.

Parameters:

  • requirement (str | List[str]) – The requirement(s) for generating code snippets.

  • code_language (Optional[str]) – The programming language for the generated code. Defaults to None.

  • **kwargs (Unpack[ValidateKwargs[List[CodeSnippet]]]) – Additional keyword arguments for the LLM usage.

Returns:

The generated code snippet(s). Returns a list of CodeSnippet objects if requirement is a string, or a list of lists of CodeSnippet objects or None if requirement is a list.

Return type:

None | List[CodeSnippet] | List[List[CodeSnippet] | None]

async achoose[T: fabricatio_core.models.generic.WithBriefing](instruction: str, choices: List[T], k: pydantic.NonNegativeInt = 0, is_included_fn: Callable[[Set[str], T], bool] | None = None, **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[List[T]]]) → List[T] | None[source]

Asynchronously executes a multi-choice decision-making process, generating a prompt based on the instruction and options, and validates the returned selection results.

Parameters:

  • instruction (str) – The user-provided instruction/question description.

  • choices (List[T]) – A list of candidate options, requiring elements to have name and briefing fields.

  • k (NonNegativeInt) – The number of choices to select, 0 means infinite. Defaults to 0.

  • is_included_fn (Optional[Callable[[Set[str],T], bool]] = None) – A function to check whether a choice is included in the query.

  • **kwargs (Unpack[ValidateKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The final validated selection result list, with element types matching the input choices.

Return type:

Optional[List[T]]

async apick[T: fabricatio_core.models.generic.WithBriefing](instruction: str, choices: List[T], **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[List[T]]]) → T[source]

Asynchronously picks a single choice from a list of options using AI validation.

Parameters:

  • instruction (str) – The user-provided instruction/question description.

  • choices (List[T]) – A list of candidate options, requiring elements to have name and briefing fields.

  • **kwargs (Unpack[ValidateKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The single selected item from the choices list.

Return type:

T

Raises:

ValueError – If validation fails after maximum attempts or if no valid selection is made.

async ajudge(prompt: str, affirm_case: str = '', deny_case: str = '', **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[bool]]) → bool | None[source]
async ajudge(prompt: List[str], affirm_case: str = '', deny_case: str = '', **kwargs: Unpack[fabricatio_core.models.kwargs_types.ValidateKwargs[bool]]) → List[bool | None] | None

Asynchronously judges a prompt using AI validation.

Parameters:

  • prompt (str) – The input prompt to be judged.

  • affirm_case (str) – The affirmative case for the AI model. Defaults to an empty string.

  • deny_case (str) – The negative case for the AI model. Defaults to an empty string.

  • **kwargs (Unpack[ValidateKwargs]) – Additional keyword arguments for the LLM usage.

Returns:

The judgment result (True or False) based on the AI’s response.

Return type:

bool

class fabricatio_core.capabilities.usages.UseEmbedding(/, **data: Any)[source]

Bases: fabricatio_core.models.generic.EmbeddingScopedConfig, abc.ABC

A class representing the embedding model.

This class extends LLMUsage and provides methods to generate embeddings for input text using various models.

async vectorize(input_text: List[str], **kwargs: Unpack[fabricatio_core.models.kwargs_types.EmbeddingKwargs]) → List[List[float]][source]
async vectorize(input_text: str, **kwargs: Unpack[fabricatio_core.models.kwargs_types.EmbeddingKwargs]) → List[float]
async vectorize(input_text: List[str] | str, **kwargs: Unpack[fabricatio_core.models.kwargs_types.EmbeddingKwargs]) → List[List[float]] | List[float]

Asynchronously generates vector embeddings for the given input text.

Parameters:

  • input_text (List[str] | str) – A string or list of strings to generate embeddings for.

  • **kwargs (Unpack[EmbeddingKwargs]) – Additional keyword arguments for the embedding model.

Returns:

The generated embeddings.

Return type:

List[List[float]] | List[float]

class fabricatio_core.capabilities.usages.UseReranker(/, **data: Any)[source]

Bases: fabricatio_core.models.generic.RerankerScopedConfig, abc.ABC

A class for reranking documents using a reranker model.

async arank(query: str, documents: List[str], **kwargs: Unpack[fabricatio_core.models.kwargs_types.RerankerKwargs]) → List[Tuple[int, float]][source]

Reranks a list of documents based on their relevance to the query.

Parameters:

  • query – The query text to rank documents against.

  • documents – A list of document texts to rerank.

  • **kwargs – Additional keyword arguments for the reranker model.

Returns:

A list of (document_index, score) pairs sorted by relevance descending.

Return type:

List[Tuple[int, float]]