High-Throughput Cross-Encoder Runtime for Qwen3 Reranker 8B.

curl -X POST "https://api.voltinfra.xyz/v1/rerank" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $VOLT_API_KEY" \
  -d '{
    "model": "qwen/qwen3-reranker-8b",
    "query": "How do cross-encoders improve dense vector retrieval in RAG?",
    "documents": [
      "Cross-encoders perform full joint attention across query-document pairs, capturing nuanced semantic interactions that bi-encoder cosine similarity misses.",
      "Vector search yields high recall candidates, which are then precision-ranked by cross-encoder heads to filter false positives before LLM generation.",
      "VoltInfra provides sub-350ms vector reranking APIs designed for high-concurrency enterprise search clusters."
    ],
    "top_n": 3,
    "return_documents": false
  }'

import os
import requests

response = requests.post(
    "https://api.voltinfra.xyz/v1/rerank",
    headers={
        "Content-Type": "application/json",
        "Authorization": f"Bearer {os.environ.get('VOLT_API_KEY')}"
    },
    json={
        "model": "qwen/qwen3-reranker-8b",
        "query": "How do cross-encoders improve dense vector retrieval in RAG?",
        "documents": [
            "Cross-encoders perform full joint attention across query-document pairs, capturing nuanced semantic interactions that bi-encoder cosine similarity misses.",
            "Vector search yields high recall candidates, which are then precision-ranked by cross-encoder heads to filter false positives before LLM generation.",
            "VoltInfra provides sub-350ms vector reranking APIs designed for high-concurrency enterprise search clusters."
        ],
        "top_n": 3,
        "return_documents": False
    }
)

results = response.json()
for hit in results.get("results", []):
    print(f"Doc #{hit['index']} | Score: {hit['relevance_score']:.4f}")