Skip to content

API Reference

This reference is generated automatically from docstrings via mkdocstrings. See the Quickstart for a narrative walkthrough.

Functions

ask(prompt, *, provider, model, reasoning=False, **kwargs)

Call provider's model synchronously and return its response.

**kwargs is split: construction keys (api_key, endpoint, api_version, deployment, region, project, compartment_id) go to the provider's constructor; everything else (temperature, max_tokens, ...) is forwarded to generate_sync() unchanged.

reasoning=True requests extended/deliberate reasoning using each provider's own real mechanism (Anthropic extended thinking, OpenAI/ OpenAI-compatible reasoning_effort, Gemini/Vertex AI thinking budgets, Bedrock's Claude thinking field). Providers with no such mechanism (Azure, Mistral, Cohere, OCI) raise ProviderNotFoundError rather than silently ignoring it - see docs/providers/index.md for the current per-provider support matrix.

Source code in anyask/api.py 
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
def ask(
    prompt: str,
    *,
    provider: str,
    model: str,
    reasoning: bool = False,
    **kwargs: Any,
) -> AskResponse:
    """Call `provider`'s `model` synchronously and return its response.

    `**kwargs` is split: construction keys (api_key, endpoint, api_version,
    deployment, region, project, compartment_id) go to the provider's
    constructor; everything else (temperature, max_tokens, ...) is
    forwarded to `generate_sync()` unchanged.

    `reasoning=True` requests extended/deliberate reasoning using each
    provider's own real mechanism (Anthropic extended thinking, OpenAI/
    OpenAI-compatible `reasoning_effort`, Gemini/Vertex AI thinking
    budgets, Bedrock's Claude thinking field). Providers with no such
    mechanism (Azure, Mistral, Cohere, OCI) raise `ProviderNotFoundError`
    rather than silently ignoring it - see `docs/providers/index.md` for
    the current per-provider support matrix.
    """
    provider_cls = _resolve_provider_cls(provider)
    config, call_kwargs = _split_kwargs(kwargs)
    return provider_cls(**config).generate_sync(
        prompt, model=model, reasoning=reasoning, **call_kwargs
    )

ask_async(prompt, *, provider, model, reasoning=False, **kwargs) async

Async counterpart of ask() - calls generate() instead of generate_sync(). See ask() for reasoning.

Source code in anyask/api.py 
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
async def ask_async(
    prompt: str,
    *,
    provider: str,
    model: str,
    reasoning: bool = False,
    **kwargs: Any,
) -> AskResponse:
    """Async counterpart of `ask()` - calls `generate()` instead of
    `generate_sync()`. See `ask()` for `reasoning`."""
    provider_cls = _resolve_provider_cls(provider)
    config, call_kwargs = _split_kwargs(kwargs)
    return await provider_cls(**config).generate(
        prompt, model=model, reasoning=reasoning, **call_kwargs
    )

list_models(provider, **kwargs)

Return the live list of model IDs provider currently serves.

Raises ProviderNotFoundError if the resolved provider class doesn't implement ModelListingMixin (e.g. it has no live listing endpoint).

Source code in anyask/api.py 
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
def list_models(provider: str, **kwargs: Any) -> List[str]:
    """Return the live list of model IDs `provider` currently serves.

    Raises `ProviderNotFoundError` if the resolved provider class doesn't
    implement `ModelListingMixin` (e.g. it has no live listing endpoint).
    """
    provider_cls = _resolve_provider_cls(provider)
    if not issubclass(provider_cls, ModelListingMixin):
        raise ProviderNotFoundError(
            f"Provider {provider!r} does not support model listing"
        )

    config, _ = _split_kwargs(kwargs)
    return provider_cls(**config).fetch_latest_models()

list_models_async(provider, **kwargs) async

Async counterpart of list_models().

No provider SDK exposes a genuinely async model-listing call today, so this just runs list_models() in a worker thread.

Source code in anyask/api.py 
110
111
112
113
114
115
116
async def list_models_async(provider: str, **kwargs: Any) -> List[str]:
    """Async counterpart of `list_models()`.

    No provider SDK exposes a genuinely async model-listing call today,
    so this just runs `list_models()` in a worker thread.
    """
    return await asyncio.to_thread(list_models, provider, **kwargs)

get_provider(provider, **config)

Construct and return a live, reusable Provider instance.

This is not routing/fallback - it's the same object ask() builds internally, just handed back instead of discarded, so callers who want to construct a provider once (avoiding per-call SDK client reconstruction, e.g. re-reading ~/.oci/config or a fresh boto3.client()) and call generate_sync()/generate() many times can do so.

Source code in anyask/api.py 
119
120
121
122
123
124
125
126
127
128
129
130
def get_provider(provider: str, **config: Any) -> Provider:
    """Construct and return a live, reusable `Provider` instance.

    This is not routing/fallback - it's the same object `ask()` builds
    internally, just handed back instead of discarded, so callers who
    want to construct a provider once (avoiding per-call SDK client
    reconstruction, e.g. re-reading ~/.oci/config or a fresh
    boto3.client()) and call `generate_sync()`/`generate()` many times
    can do so.
    """
    provider_cls = _resolve_provider_cls(provider)
    return provider_cls(**config)

Types

TokenUsage dataclass

Token accounting for a single generation call.

Any field the underlying provider SDK doesn't report is left as None rather than coerced to 0, so callers can distinguish "not reported" from "reported as zero".

Source code in anyask/provider.py 
10
11
12
13
14
15
16
17
18
19
20
21
@dataclass(frozen=True)
class TokenUsage:
    """Token accounting for a single generation call.

    Any field the underlying provider SDK doesn't report is left as
    `None` rather than coerced to 0, so callers can distinguish
    "not reported" from "reported as zero".
    """

    prompt_tokens: Optional[int]
    completion_tokens: Optional[int]
    total_tokens: Optional[int]

AskResponse dataclass

The normalized result of a single generate/generate_sync call.

finish_reason is intentionally left as the RAW, provider-specific value (e.g. "end_turn" for Anthropic, "stop" for OpenAI-family, a Gemini FinishReason enum member, ...) - it is never normalized or collapsed to a bool, so callers can detect truncated completions themselves.

raw holds the untouched original SDK response object, for callers that need something this dataclass doesn't expose.

Source code in anyask/provider.py 
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
@dataclass(frozen=True)
class AskResponse:
    """The normalized result of a single `generate`/`generate_sync` call.

    `finish_reason` is intentionally left as the RAW, provider-specific
    value (e.g. "end_turn" for Anthropic, "stop" for OpenAI-family, a
    Gemini `FinishReason` enum member, ...) - it is never normalized or
    collapsed to a bool, so callers can detect truncated completions
    themselves.

    `raw` holds the untouched original SDK response object, for callers
    that need something this dataclass doesn't expose.
    """

    content: str
    usage: TokenUsage
    finish_reason: Optional[str]
    provider: str
    model: str
    raw: Any = field(repr=False)

Provider

Bases: ABC

Base interface for all LLM providers.

Construction is kwargs-only: every concrete provider's __init__ accepts **kwargs and pulls only the keys it needs (e.g. api_key, region), ignoring the rest. This lets callers pass one uniform config dict to any provider class without an if/elif dispatch on provider name. See anyask/api.py's _CONSTRUCTION_KEYS for the known keyword vocabulary.

A provider instance is reusable across multiple model= values of the same vendor - model is always passed to generate/ generate_sync, never fixed at construction time.

Source code in anyask/provider.py 
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
class Provider(ABC):
    """Base interface for all LLM providers.

    Construction is kwargs-only: every concrete provider's `__init__`
    accepts `**kwargs` and pulls only the keys it needs (e.g. `api_key`,
    `region`), ignoring the rest. This lets callers pass one uniform
    config dict to any provider class without an if/elif dispatch on
    provider name. See `anyask/api.py`'s `_CONSTRUCTION_KEYS` for the
    known keyword vocabulary.

    A provider instance is reusable across multiple `model=` values of
    the same vendor - `model` is always passed to `generate`/
    `generate_sync`, never fixed at construction time.
    """

    name: str

    def __init__(self, **kwargs: Any) -> None: ...

    @abstractmethod
    def supports(self, model: str) -> bool:
        """Return True if this provider can serve the given model identifier."""
        raise NotImplementedError

    @abstractmethod
    async def generate(
        self, prompt: str, *, model: str, reasoning: bool = False, **kwargs: Any
    ) -> AskResponse:
        """Async text generation call.

        `reasoning=True` requests extended/deliberate reasoning on
        providers that support it (Anthropic, OpenAI and the
        OpenAI-compatible vendors, Gemini, Vertex AI, Bedrock), using each
        provider's own real mechanism - never simulated. Providers with no
        such mechanism (Azure, Mistral, Cohere, OCI) raise
        `ProviderNotFoundError` rather than silently ignoring it.
        """
        raise NotImplementedError

    @abstractmethod
    def generate_sync(
        self, prompt: str, *, model: str, reasoning: bool = False, **kwargs: Any
    ) -> AskResponse:
        """Sync text generation call. See `generate()` for `reasoning`."""
        raise NotImplementedError

    @classmethod
    def from_env(cls) -> "Provider":
        raise NotImplementedError

supports(model) abstractmethod

Return True if this provider can serve the given model identifier.

Source code in anyask/provider.py 
65
66
67
68
@abstractmethod
def supports(self, model: str) -> bool:
    """Return True if this provider can serve the given model identifier."""
    raise NotImplementedError

generate(prompt, *, model, reasoning=False, **kwargs) abstractmethod async

Async text generation call.

reasoning=True requests extended/deliberate reasoning on providers that support it (Anthropic, OpenAI and the OpenAI-compatible vendors, Gemini, Vertex AI, Bedrock), using each provider's own real mechanism - never simulated. Providers with no such mechanism (Azure, Mistral, Cohere, OCI) raise ProviderNotFoundError rather than silently ignoring it.

Source code in anyask/provider.py 
70
71
72
73
74
75
76
77
78
79
80
81
82
83
@abstractmethod
async def generate(
    self, prompt: str, *, model: str, reasoning: bool = False, **kwargs: Any
) -> AskResponse:
    """Async text generation call.

    `reasoning=True` requests extended/deliberate reasoning on
    providers that support it (Anthropic, OpenAI and the
    OpenAI-compatible vendors, Gemini, Vertex AI, Bedrock), using each
    provider's own real mechanism - never simulated. Providers with no
    such mechanism (Azure, Mistral, Cohere, OCI) raise
    `ProviderNotFoundError` rather than silently ignoring it.
    """
    raise NotImplementedError

generate_sync(prompt, *, model, reasoning=False, **kwargs) abstractmethod

Sync text generation call. See generate() for reasoning.

Source code in anyask/provider.py 
85
86
87
88
89
90
@abstractmethod
def generate_sync(
    self, prompt: str, *, model: str, reasoning: bool = False, **kwargs: Any
) -> AskResponse:
    """Sync text generation call. See `generate()` for `reasoning`."""
    raise NotImplementedError

Errors

AskLLMError

Bases: Exception

Base class for all anyask errors.

Source code in anyask/errors.py 
6
7
class AskLLMError(Exception):
    """Base class for all anyask errors."""

ProviderNotFoundError

Bases: AskLLMError

Raised when an unknown provider name is requested, or when the requested operation needs a capability (e.g. model listing) the resolved provider class doesn't implement.

Source code in anyask/errors.py 
10
11
12
13
class ProviderNotFoundError(AskLLMError):
    """Raised when an unknown provider name is requested, or when the
    requested operation needs a capability (e.g. model listing) the
    resolved provider class doesn't implement."""

ProviderError

Bases: AskLLMError

Raised when a provider call fails. Always raised with from exc so err.__cause__ is the original SDK exception.

Source code in anyask/errors.py 
16
17
18
class ProviderError(AskLLMError):
    """Raised when a provider call fails. Always raised with `from exc`
    so `err.__cause__` is the original SDK exception."""

ProviderAuthError

Bases: ProviderError

Raised when a provider is missing required credentials/config at construction time (before any network call is made).

Source code in anyask/errors.py 
21
22
23
class ProviderAuthError(ProviderError):
    """Raised when a provider is missing required credentials/config at
    construction time (before any network call is made)."""