Self-Hosted Decision Models

​

Overview

​

Laya

​

1. Run the server

pip install "laya[serve]"
laya-serve

​

2. Add the provider

• Web UI
• config.json

1. Navigate to Models > Model Providers and click Add New Provider.
2. In Add Custom Provider, set Name (for example laya) and Base Format to Typesafe.
3. Set Base URL to your server, for example http://localhost:8000.
4. Under Allowed Request Types, turn on Decisions and turn List Models off.
5. If LAYA_API_KEY is set, add a key with that value. Otherwise, turn on Is Keyless?.
6. In the key’s Allowed Models, type the checkpoints you use: english, multilingual, typed-decisions.
7. Save.

{
  "providers": {
    "laya": {
      "keys": [
        {
          "name": "laya-key",
          "value": "env.LAYA_API_KEY",
          "models": ["english", "multilingual", "typed-decisions"],
          "weight": 1
        }
      ],
      "network_config": {
        "base_url": "http://localhost:8000"
      },
      "custom_provider_config": {
        "base_provider_type": "typesafe",
        "allowed_requests": {
          "list_models": false,
          "decisions": true
        }
      }
    }
  }
}

Without LAYA_API_KEY, drop keys and add "is_key_less": true to custom_provider_config.

​

3. Send a decision

curl http://localhost:8080/v1/decisions \
  -H "Content-Type: application/json" \
  -H "x-bf-passthrough-extra-params: true" \
  -d '{
    "model": "laya/english",
    "input": "Customer message: I was double charged and nobody replied to my two emails.",
    "questions": [
      { "type": "predicate", "name": "is_frustrated", "instructions": "Is the customer frustrated?" }
    ],
    "min_confidence": 0.8
  }'

• Choosing a checkpoint. english reads 512 tokens, multilingual 1,024, and typed-decisions 1,024; longer input is truncated (see usage.truncated). Any other name without a slash, such as auto, lets Laya’s router pick by language. A name with a slash is a 422.
• Laya-only fields (min_confidence, lang, max_len, task) need the x-bf-passthrough-extra-params: true header on /v1/decisions. On /typesafe/v1/systemone they pass through without it.
• Extra response fields such as answer_confidence, abstention, and routing are listed under Provider-Specific Fields. A server started with LAYA_JEV_STRICT=1 leaves them out.
• Images are rejected with a 400 before the call, so a fallback can serve the request.

​

Nimble

​

1. Run the server

​

2. Add the provider

• Web UI
• config.json

1. Navigate to Models > Model Providers and click Add New Provider.
2. In Add Custom Provider, set Name (for example nimble) and Base Format to Typesafe.
3. Set Base URL to your deployment’s URL.
4. Under Allowed Request Types, turn on Decisions and List Models.
5. Turn on Is Keyless?. For a private deployment, send its auth headers through Network > Extra Headers instead.
6. Save.

{
  "providers": {
    "nimble": {
      "network_config": {
        "base_url": "https://bespokelabs--nimble-sglang-nimble.us-west.modal.direct"
      },
      "custom_provider_config": {
        "base_provider_type": "typesafe",
        "is_key_less": true,
        "allowed_requests": {
          "list_models": true,
          "decisions": true
        }
      }
    }
  }
}

​

3. Send a decision

curl http://localhost:8080/v1/decisions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nimble/nimble-latest",
    "input": "Customer message: I was double charged and nobody replied to my two emails.",
    "questions": [
      { "type": "predicate", "name": "is_frustrated", "instructions": "Is the customer frustrated?" }
    ]
  }'

• Limits. Choice and score questions take 2-26 options, a request takes at most 64 questions, and each question’s prompt must fit in 8,192 tokens. Oversized prompts are rejected, not truncated.
• Question names matter. Nimble keeps names and option keys in its prompt, so renaming a question can change its answer.
• Cold starts. A Modal deployment scales to zero after 2 idle minutes and takes a few minutes to wake, returning 503 meanwhile. Under load it can return 529; retry.
• Images are rejected with a 400 before the call.

​

Pricing

• Scope: the provider (optionally one key)
• Match: Exact on a model name such as english, or Wildcard for several
• Request types: decisions
• Rates: input_cost_per_token_decisions and output_cost_per_token_decisions, in USD per token. For $0.50 per million input tokens, set 0.0000005.

{
  "governance": {
    "pricing_overrides": [
      {
        "id": "8a1f6c2e-3b7d-4e59-9c1a-2f4d6e8b0a13",
        "name": "Laya english",
        "scope_kind": "provider",
        "provider_id": "laya",
        "match_type": "exact",
        "pattern": "english",
        "request_types": ["decisions"],
        "pricing_patch": "{\"input_cost_per_token_decisions\":0.0000005}"
      }
    ]
  }
}

​

Using them in the Complexity Router

• Keep guidance short on Laya. All tier guidance is sent on every request, and english reads only 512 tokens. multilingual reads 1,024.
• Keep the model loaded. A request that waits for a model to load goes unclassified once it passes decision.timeout (1.5s by default). For Laya, keep LAYA_PRELOAD=1 with every checkpoint you use, or raise LAYA_MAX_LOADED. For Nimble on Modal, keep the deployment warm, since waking takes minutes.

​

Troubleshooting

​

Next Steps

• Decisions - Request and response format
• TypeSafe - How System One requests are converted
• Complexity Router - Route requests by complexity with a decision model
• Custom Pricing - Price self-hosted models