> ## Documentation Index
> Fetch the complete documentation index at: https://docs.abv.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Guaranteed Availability

> Ensure 100% prompt availability with pre-fetching and fallback strategies for mission-critical applications

ABV's prompt management is highly available through multi-layer caching and robust infrastructure. For most applications, the default SDK caching provides sufficient resilience. However, mission-critical systems requiring absolute guarantees can implement pre-fetching or fallback prompts to eliminate any dependency on ABV's availability.

<Warning>
  Implementing guaranteed availability is usually unnecessary and adds complexity to your application. ABV Prompt Management is highly available due to multiple [caching layers](/developer/prompt-management/caching-prompts), and we closely monitor performance ([status page](https://status.abv.dev)). Consider these strategies only for mission-critical applications with zero-downtime requirements.
</Warning>

# Understanding the Failure Scenario

Before implementing guaranteed availability, understand when and why prompt fetching can fail:

<AccordionGroup>
  <Accordion title="Normal Operation with Caching" icon="check">
    **Typical request flow**:

    1. Application starts, first `get_prompt()` call fetches from ABV API
    2. Prompt is cached locally in SDK (60-second TTL default)
    3. All subsequent `get_prompt()` calls return from cache (`<1ms`, zero network)
    4. After TTL expiry, background revalidation updates cache
    5. Cycle repeats

    **Network dependency**: Only the first fetch and background revalidations require network access.

    **ABV outage impact**: If ABV becomes unavailable, cached prompts continue working until the cache expires. Even after expiry, stale-while-revalidate means the cached prompt is still served.

    **Conclusion**: Most applications experience zero impact from brief ABV outages due to caching.
  </Accordion>

  <Accordion title="The Failure Scenario" icon="xmark">
    Prompt fetching fails only when **all** of these conditions are true simultaneously:

    1. **No cached prompt**: Fresh application startup, or fetching a prompt name for the first time
    2. **Network request to ABV fails**: After retries (typically 3 attempts)
    3. **No fallback configured**: Application didn't provide a fallback prompt

    **When this occurs**:

    * Application deployment to new instances during ABV outage
    * Kubernetes pod restart during ABV outage
    * First use of a new prompt name when ABV is unreachable

    **Frequency**: Extremely rare. Requires both ABV unavailability and cold start timing.

    **Without guaranteed availability**: `get_prompt()` raises an exception, application must handle error.
  </Accordion>

  <Accordion title="Why Most Applications Don't Need Guaranteed Availability" icon="circle-info">
    **ABV's high availability**:

    * Multi-region deployment with automatic failover
    * > 99.9% uptime SLA
    * [Public status page](https://status.abv.dev) with real-time monitoring
    * Multiple caching layers (SDK cache, API Redis cache, database fallback)

    **SDK resilience**:

    * Caching eliminates network dependency for most requests
    * Stale-while-revalidate ensures zero downtime during cache updates
    * Automatic retries with exponential backoff

    **Practical reality**: Most applications have other single points of failure (database, payment gateway, auth service) with similar availability profiles. Adding guaranteed availability for prompts while tolerating failures elsewhere provides minimal benefit.

    **When to implement**: Mission-critical systems (healthcare, finance, safety) with explicit zero-downtime requirements and comprehensive failure handling across all dependencies.
  </Accordion>
</AccordionGroup>

# Option 1: Pre-Fetch Prompts on Startup

Fetch prompts during application initialization and exit if fetching fails:

<AccordionGroup>
  <Accordion title="Python Implementation (Flask)" icon="python">
    **Install dependencies**:

    ```bash theme={null}
    pip install abvdev flask
    ```

    **Implementation**:

    ```python theme={null}
    from flask import Flask, jsonify
    from abvdev import ABV
    import sys

    # Initialize Flask app and ABV client
    app = Flask(__name__)
    abv = ABV(
        api_key="sk-abv-...",
        host="https://app.abv.dev",  # or "https://eu.app.abv.dev" for EU
    )

    def fetch_prompts_on_startup():
        """
        Pre-fetch all prompts used by the application.
        Exit if any prompt fetch fails, preventing degraded startup.
        """
        try:
            # Fetch and cache all prompts used by your application
            abv.get_prompt("movie-critic")
            abv.get_prompt("summarizer")
            abv.get_prompt("translator")
            print("All prompts successfully fetched and cached")
        except Exception as e:
            print(f"CRITICAL: Failed to fetch prompts on startup: {e}")
            print("Application will not start in degraded state. Exiting.")
            sys.exit(1)  # Exit with error code

    # Call during application startup (before accepting traffic)
    fetch_prompts_on_startup()

    @app.route('/get-movie-prompt/<movie>', methods=['GET'])
    def get_movie_prompt(movie):
        # Prompt is guaranteed to be in cache (fetched at startup)
        prompt = abv.get_prompt("movie-critic")
        compiled_prompt = prompt.compile(criticlevel="expert", movie=movie)
        return jsonify({"prompt": compiled_prompt})

    if __name__ == '__main__':
        app.run(debug=True)
    ```

    **Behavior**:

    * **ABV available at startup**: Prompts cached, application starts normally
    * **ABV unavailable at startup**: Application exits with error code, orchestration system can retry or alert

    **Health checks**: Container orchestration systems (Kubernetes, Docker Swarm) detect failed startup and prevent routing traffic to unhealthy instances.
  </Accordion>

  <Accordion title="JavaScript/TypeScript Implementation (Express)" icon="js">
    **Install dependencies**:

    ```bash theme={null}
    npm install @abvdev/client express dotenv
    ```

    **Environment variables** (`.env`):

    ```bash theme={null}
    ABV_API_KEY=sk-abv-...
    ABV_BASEURL=https://app.abv.dev  # US region
    # ABV_BASEURL=https://eu.app.abv.dev  # EU region
    ```

    **Implementation**:

    ```typescript theme={null}
    import express from "express";
    import { ABVClient } from "@abvdev/client";
    import dotenv from "dotenv";
    dotenv.config();

    // Initialize Express app and ABV client
    const app = express();
    const abv = new ABVClient();

    async function fetchPromptsOnStartup() {
      /**
       * Pre-fetch all prompts used by the application.
       * Exit if any prompt fetch fails.
       */
      try {
        // Fetch and cache all prompts
        await abv.prompt.get("movie-critic");
        await abv.prompt.get("summarizer");
        await abv.prompt.get("translator");
        console.log("All prompts successfully fetched and cached");
      } catch (error) {
        console.error("CRITICAL: Failed to fetch prompts on startup:", error);
        console.error("Application will not start in degraded state. Exiting.");
        process.exit(1);  // Exit with error code
      }
    }

    // Call during application startup
    await fetchPromptsOnStartup();

    app.get("/get-movie-prompt/:movie", async (req, res) => {
      const movie = req.params.movie;
      // Prompt guaranteed to be in cache (fetched at startup)
      const prompt = await abv.prompt.get("movie-critic");
      const compiledPrompt = prompt.compile({ criticlevel: "expert", movie });
      res.json({ prompt: compiledPrompt });
    });

    app.listen(3000, () => {
      console.log("Server running on port 3000");
    });
    ```

    **Run**:

    ```bash theme={null}
    npx tsx server.ts
    ```

    **Test**:

    ```bash theme={null}
    curl http://localhost:3000/get-movie-prompt/The-Lord-of-the-Rings
    ```
  </Accordion>

  <Accordion title="Kubernetes Health Check Integration" icon="heart-pulse">
    Configure Kubernetes to detect pre-fetch failures:

    **Deployment manifest** (`deployment.yaml`):

    ```yaml theme={null}
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: my-app
    spec:
      replicas: 3
      template:
        spec:
          containers:
          - name: app
            image: my-app:latest
            env:
            - name: ABV_API_KEY
              valueFrom:
                secretKeyRef:
                  name: abv-secret
                  key: api-key
            # Startup probe: Wait for app to successfully start
            startupProbe:
              httpGet:
                path: /health
                port: 3000
              initialDelaySeconds: 5
              periodSeconds: 5
              failureThreshold: 3
            # Liveness probe: Restart if app becomes unhealthy
            livenessProbe:
              httpGet:
                path: /health
                port: 3000
              periodSeconds: 10
            # Readiness probe: Remove from service if unhealthy
            readinessProbe:
              httpGet:
                path: /health
                port: 3000
              periodSeconds: 5
    ```

    **Health endpoint implementation**:

    ```python theme={null}
    @app.route('/health', methods=['GET'])
    def health():
        return jsonify({"status": "healthy"}), 200
    ```

    **Behavior**: If pre-fetch fails (app exits with code 1), Kubernetes:

    1. Detects container exit
    2. Doesn't route traffic to failed pod
    3. Attempts restart with backoff
    4. Alerts if restart limit exceeded
  </Accordion>
</AccordionGroup>

# Option 2: Fallback Prompts

Provide hardcoded fallback prompts when ABV is unreachable:

<AccordionGroup>
  <Accordion title="Python SDK Fallback" icon="python">
    **Text prompt with fallback**:

    ```python theme={null}
    from abvdev import ABV

    abv = ABV(
        api_key="sk-abv-...",
        host="https://app.abv.dev",
    )

    # Fetch prompt with fallback
    prompt = abv.get_prompt(
        "movie-critic",
        fallback="Do you like {{movie}}?"  # Used if ABV unreachable
    )

    # Check if fallback was used
    if prompt.is_fallback:
        print("WARNING: Using fallback prompt (ABV unavailable)")
        # Optional: Log alert, increment metric, etc.

    # Compile and use (works same way for ABV or fallback)
    compiled = prompt.compile(movie="Dune 2")
    ```

    **Chat prompt with fallback**:

    ```python theme={null}
    # Fetch chat prompt with fallback
    chat_prompt = abv.get_prompt(
        "movie-critic-chat",
        type="chat",
        fallback=[
            {"role": "system", "content": "You are an expert on {{movie}}"},
            {"role": "user", "content": "Provide a review"}
        ]
    )

    # Check if fallback
    if chat_prompt.is_fallback:
        print("WARNING: Using fallback chat prompt")

    # Compile and use
    compiled_messages = chat_prompt.compile(movie="Dune 2")
    ```

    **Key properties**:

    * `prompt.is_fallback` (bool): `True` if fallback prompt is being used
    * `prompt.compile(**vars)`: Works identically for ABV and fallback prompts
  </Accordion>

  <Accordion title="JavaScript/TypeScript SDK Fallback" icon="js">
    **Text prompt with fallback**:

    ```typescript theme={null}
    import { ABVClient } from "@abvdev/client";

    const abv = new ABVClient();

    async function main() {
      // Fetch prompt with fallback
      const prompt = await abv.prompt.get("movie-critic", {
        fallback: "Do you like {{movie}}?"  // Used if ABV unreachable
      });

      // Check if fallback was used
      if (prompt.isFallback) {
        console.warn("WARNING: Using fallback prompt (ABV unavailable)");
        // Optional: Log alert, increment metric, etc.
      }

      // Compile and use
      const compiled = prompt.compile({ movie: "Dune 2" });
      console.log(compiled);
    }

    main();
    ```

    **Chat prompt with fallback**:

    ```typescript theme={null}
    async function main() {
      // Fetch chat prompt with fallback
      const chatPrompt = await abv.prompt.get("movie-critic-chat", {
        type: "chat",
        fallback: [
          { role: "system", content: "You are an expert on {{movie}}" },
          { role: "user", content: "Provide a review" }
        ]
      });

      // Check if fallback
      if (chatPrompt.isFallback) {
        console.warn("WARNING: Using fallback chat prompt");
      }

      // Compile and use
      const compiledMessages = chatPrompt.compile({ movie: "Dune 2" });
      console.log(compiledMessages);
    }

    main();
    ```

    **Key properties**:

    * `prompt.isFallback` (boolean): `true` if fallback prompt is being used
    * `prompt.compile(vars)`: Works identically for ABV and fallback prompts
  </Accordion>

  <Accordion title="Fallback Best Practices" icon="lightbulb">
    **Keep fallbacks in sync with production**:

    ```python theme={null}
    # Bad: Hardcoded fallback that diverges over time
    prompt = abv.get_prompt(
        "movie-critic",
        fallback="Review {{movie}}."  # Doesn't match current production prompt
    )

    # Better: Periodically update fallbacks to match production prompts
    # (e.g., copy production prompt to fallback during deployment)
    FALLBACK_PROMPTS = {
        "movie-critic": "As a {{criticLevel}} critic, review {{movie}}.",  # Matches v5
        # Updated: 2025-01-15
    }

    prompt = abv.get_prompt(
        "movie-critic",
        fallback=FALLBACK_PROMPTS["movie-critic"]
    )
    ```

    **Monitor fallback usage**:

    ```python theme={null}
    prompt = abv.get_prompt("movie-critic", fallback="...")

    if prompt.is_fallback:
        # Increment metric
        metrics.increment("prompts.fallback.used", tags=["prompt:movie-critic"])

        # Log warning
        logger.warning(
            "Using fallback prompt for movie-critic. "
            "ABV may be unavailable. Check status page."
        )

        # Optional: Alert on-call if fallbacks are used frequently
        if fallback_usage_rate > 0.01:  # >1% fallback usage
            alert("High fallback prompt usage detected")
    ```

    **Understand fallback limitations**:

    * **No metrics tracking**: Fallback prompts aren't linked to traces, so you lose version-specific metrics
    * **No config**: Fallback prompts don't include `config` field (model parameters, tools, etc.)
    * **Maintenance burden**: Must keep fallbacks updated manually
    * **Version mismatch risk**: Fallback may not match current production prompt

    **When to use fallbacks**: Only for truly mission-critical applications where any downtime is unacceptable. For most applications, pre-fetching or accepting occasional startup failures is simpler.
  </Accordion>
</AccordionGroup>

# Comparing Approaches

<AccordionGroup>
  <Accordion title="Pre-Fetching vs. Fallbacks" icon="scale-balanced">
    | Aspect                    | Pre-Fetching                             | Fallbacks                          |
    | ------------------------- | ---------------------------------------- | ---------------------------------- |
    | **Startup behavior**      | Fails if ABV unreachable                 | Always starts successfully         |
    | **Operational mode**      | Fail closed (won't run degraded)         | Fail open (runs with fallbacks)    |
    | **Metrics tracking**      | Full metrics (uses ABV prompts)          | No metrics during fallback         |
    | **Maintenance**           | No additional maintenance                | Must keep fallbacks updated        |
    | **Version mismatch risk** | None (always uses latest from ABV)       | High (fallbacks can diverge)       |
    | **Complexity**            | Low (one-time fetch)                     | Medium (manage fallbacks)          |
    | **Best for**              | Applications that shouldn't run degraded | Applications requiring 100% uptime |

    **Recommendation**: Prefer pre-fetching for most mission-critical applications. It's simpler and provides clearer failure modes. Use fallbacks only if your application absolutely must keep running during ABV outages.
  </Accordion>

  <Accordion title="Do You Really Need Guaranteed Availability?" icon="question">
    **Questions to ask**:

    1. **What's the impact of a startup failure?**
       * If deployment tools retry automatically, brief startup failures are harmless
       * If manual intervention is required, pre-fetching adds risk

    2. **What's your uptime requirement?**
       * > 99% (two nines): SDK caching is sufficient
       * > 99.9% (three nines): Consider pre-fetching
       * > 99.99% (four nines): Consider fallbacks or pre-fetching with multiple retries

    3. **Do you have other single points of failure?**
       * Database, auth service, payment gateway all have similar availability
       * If you tolerate those failures, why special-case prompts?

    4. **Can you tolerate fallback degradation?**
       * Fallback prompts may have lower quality than ABV-managed prompts
       * If quality is critical, pre-fetching (fail closed) is better

    **Most applications**: Standard SDK caching provides sufficient availability without additional complexity.

    **Mission-critical applications**: Pre-fetch prompts at startup for clear failure modes.

    **Absolute zero-downtime requirements**: Use fallbacks, but understand the tradeoffs and maintenance burden.
  </Accordion>
</AccordionGroup>

# Next Steps

<CardGroup cols={2}>
  <Card title="Client-Side Caching" icon="bolt" href="/developer/prompt-management/caching-prompts">
    Understand how SDK caching provides resilience
  </Card>

  <Card title="Get Started with Prompts" icon="rocket" href="/developer/prompt-management/get-started">
    Create and fetch prompts with ABV SDKs
  </Card>

  <Card title="Status Page" icon="heart-pulse" href="https://status.abv.dev">
    Monitor ABV's real-time availability
  </Card>

  <Card title="Version Control" icon="code-branch" href="/developer/prompt-management/version-control">
    Manage prompt versions and deployments
  </Card>
</CardGroup>