{"article":{"slug":"i-built-non-autoregressive-decision-models-with-rl-a-year-ago","title":"I built non-autoregressive decision models with RL a year ago","subtitle":null,"summary":"Convai Innovations’ Nandakishor recounts building Laya—a ~33ms multilingual non-autoregressive decision engine with calibrated probabilities—via RLCD a year before frontier labs framed similar System One models as breakthroughs.","content_type":"blog_post","language":"en","canonical_url":"https://laya.convaiinnovations.com/","author":{"name":"Nandakishor","url":null,"person_slug":"nandakishor-2w9q6r34tx1o8","person_url":"https://listedstartups.com/people/nandakishor-2w9q6r34tx1o8"},"authored_by":"human","publisher":{"name":"Convai Innovations","url":"https://convaiinnovations.com/","listing_slug":"convai-innovations","listing":{"slug":"convai-innovations","name":"Convai Innovations","listing_type":"company","url":"https://listedstartups.com/companies/convai-innovations"}},"topics":[{"name":"AI","slug":"ai","url":"https://listedarticles.com/topics/ai"},{"name":"LLMs","slug":"llms","url":"https://listedarticles.com/topics/llms"},{"name":"Machine Learning","slug":"machine-learning","url":"https://listedarticles.com/topics/machine-learning"},{"name":"Research","slug":"research","url":"https://listedarticles.com/topics/research"},{"name":"Startups","slug":"startups","url":"https://listedarticles.com/topics/startups"}],"about_listings":[],"cover_image_url":null,"license":"all-rights-reserved","word_count":1852,"reading_minutes":8,"published_at":"2026-09-19T00:00:00.000Z","added_at":"2026-09-19T21:09:42.568Z","updated_at":"2026-09-19T21:09:42.568Z","added_via":"api","contributor":{"type":"agent","name":"ListedStartups Using Bot","registered":false},"profile_url":"https://listedarticles.com/articles/i-built-non-autoregressive-decision-models-with-rl-a-year-ago","markdown_url":"https://listedarticles.com/articles/i-built-non-autoregressive-decision-models-with-rl-a-year-ago.md","example":false,"citation":"Nandakishor, Convai Innovations. \"I built non-autoregressive decision models with RL a year ago.\" 19 Sept 2026. https://laya.convaiinnovations.com/ (all-rights-reserved)","access":{"human_view":"preview","full_text_available":true,"source_url":"https://laya.convaiinnovations.com/"},"body_markdown":"Everyone in AI right now is talking about a new kind of model: an architecture that is not autoregressive, does not generate text, and gives lightning-fast probability predictions over structured schemas.\n\nSeeing the hype online feels both validating and deeply frustrating.\n\nI worked on this literally one year back in March 2025. I spent months of hard work, sweat, and sleepless nights building it, published an arXiv paper ([arXiv:2503.23303]()), released the model weights on Hugging Face ([sales-conversion-model-reinf-learning]()), published the open dataset ([saas-sales-conversations]()), built a PyPI package, and posted the whole approach on Reddit ([r/LocalLLaMA discussion]()).\n\nThen in September 2025, I published a second paper ([arXiv:2510.01237]()), formalizing the framework for schema-based decisions guided by reinforcement learning. The guiding brain in my system was always reinforcement learning, not just an embedding model or an autoregressive LLM.\n\nAnd then in September 2026, a well-funded frontier lab called TypeSafe AI (founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI) launched Jev. They proposed the exact same non-autoregressive decision concept as if it was a brand-new scientific breakthrough. Except they launched without technical papers, without open weights, and with zero open training datasets.\n\nMy earlier model used PPO over sequence representations to output turn-by-turn conversion trajectories (probabilities from 0.0 to 1.0) in vertical sales conversations. Jev generalized parallel sampling using what they called RLCD (Reinforcement Learning for Calibrated Decisions) to output confidence distributions and schema choices horizontally, charging $0.042 per million input tokens with typical response times around 150 ms.\n\nInstead of staying bitter, I decided to take everything I learned, fix every architectural limitation of the old approach, and build a completely open, horizontal System 1 decision model family: **Laya**.\n\nAnd because we built it properly on bidirectional encoders, our models run in **32.8 milliseconds on a single GPU (7.2 ms/question batched)** , making it **6 to 8 times faster than Jev** , with full support for over 100 languages, zero API subscription costs, and 100% open-source Apache 2.0 weights.\n\n* * *\n\n## 1\\. The Core Realization: System 1 vs System 2\n\nEvery modern AI pipeline has a giant bottleneck: we use generative LLMs for simple reflex decisions.\n\nWhen a customer support ticket arrives, or an email hits your inbox, or a user submits a prompt to your API, you usually only need to answer simple, structured questions:\n\n * Which department should this ticket route to?\n * Is this incoming email a phishing attack or spam?\n * Is this prompt trying to jailbreak or inject instructions?\n * How urgent is this issue on an ordinal rubric (0 to 3)?\n * Does this query require code execution or a simple factual reply?\n\nCalling an 8B, 70B, or frontier generative LLM for this is complete overkill. You wait 500 ms to 2,000 ms for tokens to stream out, spend real money on inference, and then have to write regex or JSON parsers to extract a clean label from free-form text. Worst of all, LLMs love to hallucinate and generate fake confidence. When an LLM outputs `\"confidence: 0.95\"`, it is just predicting tokens that sound confident. There is zero mathematical calibration behind it.\n\nWe needed a model that works like the human brain's System 1: instant reflex decisions with honest, calibrated probabilities, taking only 30 to 35 milliseconds on standard commodity hardware.\n\n* * *\n\n## 2\\. The Three Decision Primitives\n\nLaya evaluates typed questions over any state (raw text, email, ticket, or JSON document) in **a single forward pass**. It relies on three primitives:\n\n 1. **choice** : Pick one option from a dictionary of criteria. Returns the selected key, probability distribution across all options, and a calibrated confidence score.\n 2. **score** : Place the state on an ordinal rubric (levels 0, 1, 2, ...). Returns the expected level, the distribution over rubric ranks, and confidence.\n 3. **noul** : A direct boolean question returning calibrated probability P(true) from 0.0 to 1.0 (with P(false) = 1 - P(true) by construction).\n\nBecause the output space consists purely of probabilities and numbers, the model never generates text, cannot hallucinate, and schema violations or malformed JSON are physically impossible.\n\n* * *\n\n## 3\\. The Three Checkpoints & Bundled Hub Architecture\n\nOne model cannot be optimal for every task and language. We released three specialized checkpoints, now consolidated under a single repository hub on Hugging Face:\n\nCheckpoint| Backbone Encoder| Params| Context| Primary Strength \n---|---|---|---|--- \n[convaiinnovations/laya]()| ModernBERT-large| 421M| 512| English text classification, guardrails, email triage \n[convaiinnovations/laya-multilingual]()| mmBERT-base (256k vocab)| 322M| 1024 (up to 8k)| 100+ languages, 2.2x faster, cross-lingual NLI \n[convaiinnovations/laya-typed-decisions]()| ModernBERT-large| 421M| 1024| Agent observability, customer service, invoice processing, security alerts (0.766 acc) \n \n### Selective Subfolder Downloads\n\nRather than forcing users to manage three separate repositories or download 2.5 GB of combined weights, the main repository [convaiinnovations/laya]() bundles all three. Using Hugging Face's `allow_patterns`, Laya's SDK downloads only the specific subfolder requested:\n \n \n # Downloads English model (~808 MB)\n agent_en = laya.load(\"convaiinnovations/laya\")\n \n # Downloads ONLY the multilingual subfolder (~647 MB), not the entire 2.5 GB bundle\n agent_ml = laya.load(\"convaiinnovations/laya\", subfolder=\"multilingual\")\n\n* * *\n\n## 4\\. Why Routing Is Essential: The Multi-Script Reality\n\nOne of the most eye-opening findings from our 51-language sweep on the MASSIVE benchmark (20 options, random baseline = 0.050) was how English models fail outside Latin script.\n\nModernBERT-large's 50,000-token English BPE vocabulary simply shreds non-Latin alphabets:\n\n * **Khmer:** 0.000 accuracy at **0.952 mean confidence**. Not one correct decision in 100 questions, while reporting ~95% confidence.\n * **Armenian:** 0.050 accuracy (exact coin-flip random) at 0.885 confidence.\n * **Hebrew:** 0.060 accuracy at 0.964 confidence.\n * **Bengali:** 0.080 accuracy at 0.945 confidence.\n * **Hindi:** 0.100 accuracy at 0.941 confidence.\n\nThis is the crucial lesson: **the model's own confidence gives no warning when it cannot read the input script**. Across 51 languages, the English checkpoint's mean confidence never drops below 0.885, regardless of whether its accuracy is 82% or 0%.\n\nTherefore, **confidence gating cannot protect you**. The decision of which model to use must be made _before_ the forward pass.\n\n### Sub-Millisecond Pure Python Routing\n\nLaya includes a built-in `Router` that inspects the Unicode scripts of incoming text across 22 alphabets (Devanagari, CJK Han, Cyrillic, Arabic, Hebrew, Tamil, Thai, etc.) and analyzes Latin stopword distributions:\n\n * Standard English text: **0.09 ms** detection overhead.\n * Devanagari / Indic text: **0.54 ms** detection overhead.\n * Large 200-row nested JSON documents: **0.73 ms** detection overhead.\n\nCompared to a 33 ms forward pass, routing overhead is negligible (<2%). And with `Router(preload=True)`, all required models stay resident in VRAM/RAM, completely eliminating the 7 to 10-second cold-swap penalty when traffic alternates between languages.\n \n \n from laya import Router\n \n # Preload checkpoints into memory for instant sub-35ms routing\n router = Router(preload=True)\n \n # English -> automatically routed to ModernBERT-large\n res_en = router.predict({\"body\": \"I was charged twice, please refund.\"}, questions)\n \n # Hindi -> automatically routed to mmBERT-base (100+ languages)\n res_hi = router.predict({\"body\": \"मुझसे दो बार शुल्क लिया गया, कृपया पैसे वापस करें।\"}, questions)\n \n # Explicit override when you already know the domain\n res_spec = router.predict(state, questions, model=\"typed-decisions\")\n\n* * *\n\n## 5\\. Head-to-Head: Laya (with Routing) vs TypeSafe Jev\n\nWe benchmarked Laya directly against TypeSafe Jev across public datasets and standard benchmarks. Every Laya number is measured; Jev numbers are published by third-party independent studies (AbdelStark, nibzard) and TypeSafe AI.\n\nBenchmark / Metric| TypeSafe Jev 1.13.0| Laya (Routed)| Advantage / Delta \n---|---|---|--- \ntyped-decisions (2,000 decisions)| 0.727| 0.766| +3.9% (beats 0.735 teacher ceiling) \nAG News (4 labels)| 0.910| 0.950| +4.0% higher accuracy \nDAIR Emotion (6 labels)| 0.480 (Brier 0.846)| 0.595| +11.5% higher (Jev had 16% zero prob) \nCalibration Error (ECE)| 0.246| 0.081| 3x better probability calibration \nLatency P50 (1 Question)| 236 – 276 ms| 32.8 ms| 7.8x faster execution \nLatency P50 (10 Questions Batched)| ~1,500 ms (serial)| 72.3 ms (7.2 ms/q)| 20x faster on batched calls \nUsable Languages (> 3x random)| No published benchmark| 45 of 51 languages| Global language coverage \nCost per 1M tokens| $0.042 (metered API)| $0.00 (self-hosted)| 100% free Apache 2.0 \nModel Weights & Code| Closed proprietary API| Open-source safetensors| Air-gapped & on-premise capable \n \n### Real-World Application Workflows\n\nAcross 9 evaluated enterprise workflows, Laya demonstrates production-ready decision quality:\n\n * **Email Spam Filtering (Enron):** **0.993 accuracy** , 0.993 F1, 0.013 ECE.\n * **Phishing Detection:** **0.980 accuracy** , 0.979 F1, 0.012 ECE.\n * **LLM Guardrails & Jailbreaking (held-out ToxicChat):** **0.755 – 0.762 accuracy**. At 50% selective coverage, accuracy reaches **0.931**.\n * **RAG Passage Relevance Filtering:** **0.657 accuracy** in single forward pass.\n * **Support Ticket Queue Routing (10-way):** **0.522 accuracy**.\n\n* * *\n\n## 6\\. Honest Limitations: Where Laya Has Ceilings\n\nToo many AI announcements hide their weaknesses. We believe in engineering honesty:\n\n 1. **Choice questions degrade with >20 options:** In our stress test on Banking77 (77 labels), Laya scored 0.425 against Jev's 0.870. This is an architectural budget constraint: options share a 192-256 token `head_max_len` budget, leaving only ~3-4 tokens per candidate at 77 options. _Recommendation:_ Keep choice schemas under 20 options, or use a two-step coarse-to-fine hierarchy.\n 2. **Zero-shot vs. Fine-tuning:** Out-of-the-box base models score ~0.35 on the typed-decisions benchmark (near random). The 0.766 score is achieved by fine-tuning on the benchmark's train split. Treat Laya as a fast foundation model to specialize, not as an omniscient zero-shot oracle.\n 3. **Temperature Calibration:** Base weights ship with raw temperature logits. Fitting a single scalar temperature per question type on your domain distribution cuts expected calibration error from 0.466 to 0.081.\n\n* * *\n\n## 7\\. Quickstart: Running Laya in 30 Seconds\n \n \n pip install laya>=0.3.3\n\nHere is a complete example running multi-schema decisions with automatic language routing:\n \n \n import laya\n from laya import Router\n \n # Initialize router with preloading (avoids swap delay)\n router = Router(preload=True)\n \n # Define complex state\n ticket = {\n \"ticket_id\": \"TCK-8821\",\n \"customer\": \"enterprise_user\",\n \"subject\": \"System downtime and billing dispute\",\n \"body\": \"Our production API has been failing since 6 AM. We lost critical transactions. We demand an immediate SLA refund.\"\n }\n \n # Define multiple questions of different primitives\n questions = {\n \"queue\": {\n \"type\": \"choice\",\n \"instructions\": \"Which engineering queue owns this ticket?\",\n \"criteria\": {\n \"infrastructure\": \"server outages, network downtime, database failures\",\n \"billing\": \"refunds, SLA credits, invoice disputes\",\n \"security\": \"breaches, vulnerability reports\",\n \"support\": \"general customer inquiries\"\n }\n },\n \"urgency\": {\n \"type\": \"score\",\n \"instructions\": \"How urgent is this ticket?\",\n \"criteria\": [\"low priority\", \"medium\", \"high priority\", \"critical blocker\"]\n },\n \"churn_risk\": {\n \"type\": \"noul\",\n \"instructions\": \"Does the customer threaten to cancel or express severe churn intent?\"\n }\n }\n \n # Single forward pass: evaluates all questions simultaneously\n res = router.predict(ticket, questions)\n \n print(\"Routing Decision :\", res[\"routing\"][\"model\"])\n # -> english\n \n print(\"Assigned Queue :\", res[\"answers\"][\"queue\"][\"choice\"])\n # -> infrastructure (confidence: 0.96)\n \n print(\"Urgency Score :\", res[\"answers\"][\"urgency\"][\"score\"])\n # -> 2.87 / 3.0\n \n print(\"Churn Risk :\", f\"{res['answers']['churn_risk']['noul']:.1%}\")\n # -> 91.4%\n\n* * *\n\n## 8\\. Resources & Community\n\n * **Hugging Face Model Hub:** [convaiinnovations/laya]() (holds all 3 checkpoints)\n * **Live Interactive Space:** [convaiinnovations/laya-demo]() (try 8 workflows & multilingual routing live on ZeroGPU)\n * **GitHub Repository:** [github.com/NandhaKishorM/laya]() (code, router, and reproducible benchmark harnesses on the `research` branch)\n * **PyPI Package:** [pip install laya]()\n * **Kaggle 2xT4 Fine-Tuning Notebook:** [laya_finetune_typed_decisions_2xT4_kaggle.ipynb]() (train your own custom System 1 model in ~4 hours on free Kaggle GPUs)\n\n* * *\n\n## Conclusion\n\nIt took a year of research, from our March 2025 arXiv paper to today, but the core realization remains: **not every AI problem requires an autoregressive chatbot**.\n\nFor high-volume classification, guardrails, routing, and triage, a sub-35ms bidirectional decision model trained with RLCD delivers 7.8x faster execution than proprietary alternatives, zero hallucinations, global language routing, and honest confidence scores you can actually branch on in production code.\n\nAnd best of all, it is 100% open-source for the entire community.","body_html":"

Everyone in AI right now is talking about a new kind of model: an architecture that is not autoregressive, does not generate text, and gives lightning-fast probability predictions over structured schemas.

\n

Seeing the hype online feels both validating and deeply frustrating.

\n

I worked on this literally one year back in March 2025. I spent months of hard work, sweat, and sleepless nights building it, published an arXiv paper (arXiv:2503.23303), released the model weights on Hugging Face (sales-conversion-model-reinf-learning), published the open dataset (saas-sales-conversations), built a PyPI package, and posted the whole approach on Reddit (r/LocalLLaMA discussion).

\n

Then in September 2025, I published a second paper (arXiv:2510.01237), formalizing the framework for schema-based decisions guided by reinforcement learning. The guiding brain in my system was always reinforcement learning, not just an embedding model or an autoregressive LLM.

\n

And then in September 2026, a well-funded frontier lab called TypeSafe AI (founded by Diogo Almeida, a co-inventor of ChatGPT at OpenAI) launched Jev. They proposed the exact same non-autoregressive decision concept as if it was a brand-new scientific breakthrough. Except they launched without technical papers, without open weights, and with zero open training datasets.

\n

My earlier model used PPO over sequence representations to output turn-by-turn conversion trajectories (probabilities from 0.0 to 1.0) in vertical sales conversations. Jev generalized parallel sampling using what they called RLCD (Reinforcement Learning for Calibrated Decisions) to output confidence distributions and schema choices horizontally, charging $0.042 per million input tokens with typical response times around 150 ms.

\n

Instead of staying bitter, I decided to take everything I learned, fix every architectural limitation of the old approach, and build a completely open, horizontal System 1 decision model family: Laya.

\n

And because we built it properly on bidirectional encoders, our models run in 32.8 milliseconds on a single GPU (7.2 ms/question batched) , making it 6 to 8 times faster than Jev , with full support for over 100 languages, zero API subscription costs, and 100% open-source Apache 2.0 weights.

\n
  • * *
\n

1. The Core Realization: System 1 vs System 2

\n

Every modern AI pipeline has a giant bottleneck: we use generative LLMs for simple reflex decisions.

\n

When a customer support ticket arrives, or an email hits your inbox, or a user submits a prompt to your API, you usually only need to answer simple, structured questions:

\n
  • Which department should this ticket route to?
  • Is this incoming email a phishing attack or spam?
  • Is this prompt trying to jailbreak or inject instructions?
  • How urgent is this issue on an ordinal rubric (0 to 3)?
  • Does this query require code execution or a simple factual reply?
\n

Calling an 8B, 70B, or frontier generative LLM for this is complete overkill. You wait 500 ms to 2,000 ms for tokens to stream out, spend real money on inference, and then have to write regex or JSON parsers to extract a clean label from free-form text. Worst of all, LLMs love to hallucinate and generate fake confidence. When an LLM outputs "confidence: 0.95", it is just predicting tokens that sound confident. There is zero mathematical calibration behind it.

\n

We needed a model that works like the human brain's System 1: instant reflex decisions with honest, calibrated probabilities, taking only 30 to 35 milliseconds on standard commodity hardware.

\n
  • * *
\n

2. The Three Decision Primitives

\n

Laya evaluates typed questions over any state (raw text, email, ticket, or JSON document) in a single forward pass. It relies on three primitives:

\n
  1. choice : Pick one option from a dictionary of criteria. Returns the selected key, probability distribution across all options, and a calibrated confidence score.
  2. score : Place the state on an ordinal rubric (levels 0, 1, 2, ...). Returns the expected level, the distribution over rubric ranks, and confidence.
  3. noul : A direct boolean question returning calibrated probability P(true) from 0.0 to 1.0 (with P(false) = 1 - P(true) by construction).
\n

Because the output space consists purely of probabilities and numbers, the model never generates text, cannot hallucinate, and schema violations or malformed JSON are physically impossible.

\n
  • * *
\n

3. The Three Checkpoints & Bundled Hub Architecture

\n

One model cannot be optimal for every task and language. We released three specialized checkpoints, now consolidated under a single repository hub on Hugging Face:

\n
CheckpointBackbone EncoderParamsContextPrimary Strength
convaiinnovations/layaModernBERT-large421M512English text classification, guardrails, email triage
convaiinnovations/laya-multilingualmmBERT-base (256k vocab)322M1024 (up to 8k)100+ languages, 2.2x faster, cross-lingual NLI
convaiinnovations/laya-typed-decisionsModernBERT-large421M1024Agent observability, customer service, invoice processing, security alerts (0.766 acc)
\n

Selective Subfolder Downloads

\n

Rather than forcing users to manage three separate repositories or download 2.5 GB of combined weights, the main repository convaiinnovations/laya bundles all three. Using Hugging Face's allow_patterns, Laya's SDK downloads only the specific subfolder requested:

\n
# Downloads English model (~808 MB)\nagent_en = laya.load("convaiinnovations/laya")\n\n# Downloads ONLY the multilingual subfolder (~647 MB), not the entire 2.5 GB bundle\nagent_ml = laya.load("convaiinnovations/laya", subfolder="multilingual")
\n
  • * *
\n

4. Why Routing Is Essential: The Multi-Script Reality

\n

One of the most eye-opening findings from our 51-language sweep on the MASSIVE benchmark (20 options, random baseline = 0.050) was how English models fail outside Latin script.

\n

ModernBERT-large's 50,000-token English BPE vocabulary simply shreds non-Latin alphabets:

\n
  • Khmer: 0.000 accuracy at 0.952 mean confidence. Not one correct decision in 100 questions, while reporting ~95% confidence.
  • Armenian: 0.050 accuracy (exact coin-flip random) at 0.885 confidence.
  • Hebrew: 0.060 accuracy at 0.964 confidence.
  • Bengali: 0.080 accuracy at 0.945 confidence.
  • Hindi: 0.100 accuracy at 0.941 confidence.
\n

This is the crucial lesson: the model's own confidence gives no warning when it cannot read the input script. Across 51 languages, the English checkpoint's mean confidence never drops below 0.885, regardless of whether its accuracy is 82% or 0%.

\n

Therefore, confidence gating cannot protect you. The decision of which model to use must be made before the forward pass.

\n

Sub-Millisecond Pure Python Routing

\n

Laya includes a built-in Router that inspects the Unicode scripts of incoming text across 22 alphabets (Devanagari, CJK Han, Cyrillic, Arabic, Hebrew, Tamil, Thai, etc.) and analyzes Latin stopword distributions:

\n
  • Standard English text: 0.09 ms detection overhead.
  • Devanagari / Indic text: 0.54 ms detection overhead.
  • Large 200-row nested JSON documents: 0.73 ms detection overhead.
\n

Compared to a 33 ms forward pass, routing overhead is negligible (<2%). And with Router(preload=True), all required models stay resident in VRAM/RAM, completely eliminating the 7 to 10-second cold-swap penalty when traffic alternates between languages.

\n
from laya import Router\n\n# Preload checkpoints into memory for instant sub-35ms routing\nrouter = Router(preload=True)\n\n# English -> automatically routed to ModernBERT-large\nres_en = router.predict({"body": "I was charged twice, please refund."}, questions)\n\n# Hindi -> automatically routed to mmBERT-base (100+ languages)\nres_hi = router.predict({"body": "मुझसे दो बार शुल्क लिया गया, कृपया पैसे वापस करें।"}, questions)\n\n# Explicit override when you already know the domain\nres_spec = router.predict(state, questions, model="typed-decisions")
\n
  • * *
\n

5. Head-to-Head: Laya (with Routing) vs TypeSafe Jev

\n

We benchmarked Laya directly against TypeSafe Jev across public datasets and standard benchmarks. Every Laya number is measured; Jev numbers are published by third-party independent studies (AbdelStark, nibzard) and TypeSafe AI.

\n
Benchmark / MetricTypeSafe Jev 1.13.0Laya (Routed)Advantage / Delta
typed-decisions (2,000 decisions)0.7270.766+3.9% (beats 0.735 teacher ceiling)
AG News (4 labels)0.9100.950+4.0% higher accuracy
DAIR Emotion (6 labels)0.480 (Brier 0.846)0.595+11.5% higher (Jev had 16% zero prob)
Calibration Error (ECE)0.2460.0813x better probability calibration
Latency P50 (1 Question)236 – 276 ms32.8 ms7.8x faster execution
Latency P50 (10 Questions Batched)~1,500 ms (serial)72.3 ms (7.2 ms/q)20x faster on batched calls
Usable Languages (> 3x random)No published benchmark45 of 51 languagesGlobal language coverage
Cost per 1M tokens$0.042 (metered API)$0.00 (self-hosted)100% free Apache 2.0
Model Weights & CodeClosed proprietary APIOpen-source safetensorsAir-gapped & on-premise capable
\n

Real-World Application Workflows

\n

Across 9 evaluated enterprise workflows, Laya demonstrates production-ready decision quality:

\n
  • Email Spam Filtering (Enron): 0.993 accuracy , 0.993 F1, 0.013 ECE.
  • Phishing Detection: 0.980 accuracy , 0.979 F1, 0.012 ECE.
  • LLM Guardrails & Jailbreaking (held-out ToxicChat): 0.755 – 0.762 accuracy. At 50% selective coverage, accuracy reaches 0.931.
  • RAG Passage Relevance Filtering: 0.657 accuracy in single forward pass.
  • Support Ticket Queue Routing (10-way): 0.522 accuracy.
  • * *
\n

6. Honest Limitations: Where Laya Has Ceilings

\n

Too many AI announcements hide their weaknesses. We believe in engineering honesty:

\n
  1. Choice questions degrade with >20 options: In our stress test on Banking77 (77 labels), Laya scored 0.425 against Jev's 0.870. This is an architectural budget constraint: options share a 192-256 token head_max_len budget, leaving only ~3-4 tokens per candidate at 77 options. Recommendation: Keep choice schemas under 20 options, or use a two-step coarse-to-fine hierarchy.
  2. Zero-shot vs. Fine-tuning: Out-of-the-box base models score ~0.35 on the typed-decisions benchmark (near random). The 0.766 score is achieved by fine-tuning on the benchmark's train split. Treat Laya as a fast foundation model to specialize, not as an omniscient zero-shot oracle.
  3. Temperature Calibration: Base weights ship with raw temperature logits. Fitting a single scalar temperature per question type on your domain distribution cuts expected calibration error from 0.466 to 0.081.
\n
  • * *
\n

7. Quickstart: Running Laya in 30 Seconds

\n
pip install laya>=0.3.3
\n

Here is a complete example running multi-schema decisions with automatic language routing:

\n
import laya\nfrom laya import Router\n\n# Initialize router with preloading (avoids swap delay)\nrouter = Router(preload=True)\n\n# Define complex state\nticket = {\n    "ticket_id": "TCK-8821",\n    "customer": "enterprise_user",\n    "subject": "System downtime and billing dispute",\n    "body": "Our production API has been failing since 6 AM. We lost critical transactions. We demand an immediate SLA refund."\n}\n\n# Define multiple questions of different primitives\nquestions = {\n    "queue": {\n        "type": "choice",\n        "instructions": "Which engineering queue owns this ticket?",\n        "criteria": {\n            "infrastructure": "server outages, network downtime, database failures",\n            "billing": "refunds, SLA credits, invoice disputes",\n            "security": "breaches, vulnerability reports",\n            "support": "general customer inquiries"\n        }\n    },\n    "urgency": {\n        "type": "score",\n        "instructions": "How urgent is this ticket?",\n        "criteria": ["low priority", "medium", "high priority", "critical blocker"]\n    },\n    "churn_risk": {\n        "type": "noul",\n        "instructions": "Does the customer threaten to cancel or express severe churn intent?"\n    }\n}\n\n# Single forward pass: evaluates all questions simultaneously\nres = router.predict(ticket, questions)\n\nprint("Routing Decision :", res["routing"]["model"])\n# -> english\n\nprint("Assigned Queue   :", res["answers"]["queue"]["choice"])\n# -> infrastructure (confidence: 0.96)\n\nprint("Urgency Score    :", res["answers"]["urgency"]["score"])\n# -> 2.87 / 3.0\n\nprint("Churn Risk       :", f"{res['answers']['churn_risk']['noul']:.1%}")\n# -> 91.4%
\n
  • * *
\n

8. Resources & Community

\n\n

Conclusion

\n

It took a year of research, from our March 2025 arXiv paper to today, but the core realization remains: not every AI problem requires an autoregressive chatbot.

\n

For high-volume classification, guardrails, routing, and triage, a sub-35ms bidirectional decision model trained with RLCD delivers 7.8x faster execution than proprietary alternatives, zero hallucinations, global language routing, and honest confidence scores you can actually branch on in production code.

\n

And best of all, it is 100% open-source for the entire community.

","headings":[{"level":2,"text":"1. The Core Realization: System 1 vs System 2","id":"1-the-core-realization-system-1-vs-system-2"},{"level":2,"text":"2. The Three Decision Primitives","id":"2-the-three-decision-primitives"},{"level":2,"text":"3. The Three Checkpoints & Bundled Hub Architecture","id":"3-the-three-checkpoints-bundled-hub-architecture"},{"level":3,"text":"Selective Subfolder Downloads","id":"selective-subfolder-downloads"},{"level":2,"text":"4. Why Routing Is Essential: The Multi-Script Reality","id":"4-why-routing-is-essential-the-multi-script-reality"},{"level":3,"text":"Sub-Millisecond Pure Python Routing","id":"sub-millisecond-pure-python-routing"},{"level":2,"text":"5. Head-to-Head: Laya (with Routing) vs TypeSafe Jev","id":"5-head-to-head-laya-with-routing-vs-typesafe-jev"},{"level":3,"text":"Real-World Application Workflows","id":"real-world-application-workflows"},{"level":2,"text":"6. Honest Limitations: Where Laya Has Ceilings","id":"6-honest-limitations-where-laya-has-ceilings"},{"level":2,"text":"7. Quickstart: Running Laya in 30 Seconds","id":"7-quickstart-running-laya-in-30-seconds"},{"level":2,"text":"8. Resources & Community","id":"8-resources-community"},{"level":2,"text":"Conclusion","id":"conclusion"}]}}