Introduction

Overview

SDialog is a modular Python toolkit for generating, steering, evaluating, and analyzing synthetic dialogues with instruction-tuned Large Language Models (LLMs). It standardizes core abstractions (Dialog, Turn, Event, Persona, Context, Agent, BaseOrchestrator) and supplies composable components for:

  • Persona-driven multi-agent role-play

  • Dynamic orchestration (rule / probabilistic / semantic response suggestion)

  • Attribute & dialogue generation (LLM + light rule hybrids)

  • Evaluation (linguistic, flow / coherence, embedding similarity, LLM judges)

  • Mechanistic interpretability & activation steering

  • Dataset integration (e.g. STAR) and reproducible experiment pipelines

If you are building controlled conversational simulations, benchmarking dialog models, producing synthetic training corpora, or probing internal model behavior, SDialog provides an end-to-end workflow.

Architecture at a Glance

SDialog is divided into many submodules, each depicted as a circle in the below diagram describing the overall architecture:

../_images/sdialog-modules.png

Note: Attribute & Dialog generators may produce Personas / Contexts / prototype Dialogs that seed Agents; Agents then produce the canonical Dialog objects evaluated downstream.


Core Concepts

Dialog

A Dialog object contains an ordered list of Turn instances and an optional list of Event objects, plus metadata and utility methods:

Creating Dialog from Text:

Dialog objects can be created programmatically, but let’s explore how we can also easily create them from plain text! Whether you have a string or a text file, we can use the convenient from_str() or from_file() methods respectively. Both methods accept the same arguments and Dialog.from_str(text) is equivalent to Dialog.from_file("file.txt") when the file contains plain text. Let’s walk through three common scenarios you might encounter:

from sdialog import Dialog, Turn

# 1) Basic usage - Text in default "{speaker}: {text}" format
dialog_text = """Alice: Hello there! How are you today?
Bob: I'm doing great, thanks for asking.
Alice: That's wonderful to hear!
Bob: What about you? How's your day going?"""

dialog = Dialog.from_str(dialog_text)
dialog.print()

# 2) Text in custom format
chat_log = """[2024-01-15 14:30] @user123: Hey everyone!
[2024-01-15 14:31] @moderator: Welcome to the chat
[2024-01-15 14:32] @user123: Thanks, excited to be here!
[2024-01-15 14:33] @helper_bot: How can I assist you today?"""

# Define your custom template to parse each turn
dialog_from_chat = Dialog.from_str(
    chat_log,
    template="[{timestamp}] @{speaker}: {text}"
)
dialog_from_chat.print()

# 3) Text with no speaker tags
simple_conversation = """Hello there!
Hi, how are you?
I'm doing well, thanks!
That's great to hear."""

# Provide default speakers to assign alternately
dialog_with_defaults = Dialog.from_str(
    simple_conversation,
    template="{text}",  # No speaker in text turns
    default_speakers=["Alice", "Bob"]  # Alternating assignment
)
dialog_with_defaults.print()

Operations Example:

Now that we understand how to create dialogs, let’s explore the powerful operations we can perform with them! Here’s a hands-on walkthrough demonstrating common Dialog manipulations we’ll be working with—creating a dialog programmatically, slicing (which preserves lineage), chaining text transformations, selective speaker filtering, speaker renaming, length/statistics queries, and safe iteration over derived copies.

from sdialog import Dialog, Turn

# Let's first create a sample dialog programatically
dialog = Dialog(turns=[
    Turn(speaker="Alice", text="Hello there! How are you doing today?"),
    Turn(speaker="Bob", text="I'm doing great, thanks for asking."),
    Turn(speaker="Alice", text="That's wonderful to hear!"),
    Turn(speaker="Bob", text="What about you? How's YOUR DAY going?")
])

# Slicing creates new Dialog with fresh ID and parentId linkage
first_half = dialog[:2]  # First 2 turns → new Dialog with dialog.id as parentId
print(f"Original ID: {dialog.id}")
print(f"Slice ID: {first_half.id}, Parent: {first_half.parentId}")

# Text transformations examples
normalized = dialog.lower().replace("!", ".")             # Chain operations
filtered_alice = dialog.filter("Alice")                   # Only Alice's turns
dialog_alicia = dialog.rename_speaker("Alice", "Alicia")  # Rename speaker

# Print some information
print(f"Length: {len(dialog)} turns")  # len(dialog) == dialog.length('turns')
print(f"Length: {dialog.length('words')} words, approx. {dialog.length('time')} minutes")
print(f"Speakers: {dialog.get_speakers()}")

# Iterate over dialog turns
for ix, turn in enumerate(dialog_alicia):
    print(f"Turn {ix+1}: {turn.speaker} - {turn.text}")

# Save to JSON with metadata
dialog_alicia.to_file("dialog_alicia.json")

Personas & Context

Personas are structured, typed attribute bundles that specify role, style, goals, background knowledge, and behavioral constraints used to condition LLM prompts for Agents in a reproducible, inspectable way. Context objects complement Personas with shared situational grounding so multiple agents can coordinate.

SDialog formalizes this socio-cognitive conditioning through attribute models:

All inherit BaseAttributeModel and, as such, they support:

Creating your own Personas:

Sometimes the built-in personas don’t quite fit your specific use case—and that’s perfectly fine! Let’s create custom personas by inheriting from BasePersona (equivalent to BaseAttributeModel) or from any existing persona class to extend it further.

from sdialog.personas import BasePersona

class TravelAgentPersona(BasePersona):
    role: str = ""
    expertise: str = ""
    tone: str = ""
    goals: str = ""
    constraints: str = "Make sure to provide realistic travel options"

# Instantiate with defaults or override specific attributes
my_persona = TravelAgentPersona(
    role="Travel Agent",
    expertise="European rail itineraries",
    tone="friendly and concise",
    goals="suggest routes, optimize cost",
)
my_persona.print()  # Pretty-print my travel agent persona

# Clone with overrides while preserving lineage metadata
variant = my_persona.clone(tone="enthusiastic and energetic")
assert variant.parentId == my_persona.id

Advanced Persona Creation with Field Documentation:

Here’s where things get even more interesting! We can use Pydantic Field descriptions to document each attribute in our custom persona. These descriptions serve as guides for LLMs when generating persona objects via PersonaGenerator. Let’s see this in action:

from sdialog.generators import PersonaGenerator
from sdialog.personas import BasePersona
from pydantic import Field


class TravelAgentPersona(BasePersona):
    role: str = Field("", description="The role of the persona, for example 'Travel Agent' or 'Tour Guide'")
    expertise: str = Field("", description="The area of expertise")
    tone: str = Field("", description="The tone of the persona")
    goals: str = Field("", description="A short description of the goals")
    constraints: str = Field("", description="Operational constraints, e.g., 'budget limitations'")

# Alternatively, use PersonaGenerator to create personas based on field descriptions
generator = PersonaGenerator(TravelAgentPersona, model="openai:gpt-4")
generated_persona = generator.generate()
generated_persona.print()  # Pretty-print the generated travel agent persona

Agents & Orchestration

Agents

Agent encapsulates an LLM-backed conversational actor:

Features:

  • Persona + optional Context + exemplar Dialogs → prompt assembly.

  • Memory (list of role-tagged messages) with additive system instructions.

  • First utterance selection (fixed or random from list).

  • Orchestrator pipeline (dynamic instruction injection).

  • Optional tools (callable functions) integrated into LLM responses (if backend supports function/tool calling).

  • Optional “thinking” (hidden reasoning segments) extraction & filtering.

  • Lookahead capability (response_lookahead()) without mutating memory.

  • JSON serialization of configuration and persona.

Key Methods:

  • __call__(): invoke the agent with a message and get a response (e.g. resp = agent("Hello!")).

  • dialog_with() / alias talk_with(): multi-turn role-play.

  • instruct(): inject immediate or persistent system instructions.

  • add_orchestrators() / pipeline composition via agent | orchestrator.

  • add_inspectors() or agent | inspector for activation capture (mechanistic interpretability).

  • reset(): reproducible restart to mark the beginning of a new conversation.

  • memory_dump(): snapshot conversation for logging.

  • prompt(): get the underlying system prompt used by the agent.

  • json(): export the agent as a JSON object.

Orchestration

Orchestrators are lightweight controllers that examine the current dialog state and the last utterance from the other agent, optionally returning an instruction. They can be ephemeral (one-time) or persistent (lasting across multiple turns). Orchestrators are composed using the pipe operator:

# Instantiate orchestrators first
length_orch = LengthOrchestrator(min=8, max=12)
reflex_orch = SimpleReflexOrchestrator(
    condition=lambda utt: "confused" in utt.lower() or "don't understand" in utt.lower(),
    instruction="Slow down and explain with a concrete example to clarify."
)
agent = agent | length_orch | reflex_orch

Built-in Orchestrators:

Creating your own Orchestrators:

Ready to build your own orchestration logic? You can create custom orchestrators by inheriting from BaseOrchestrator (for one-time instructions) or BasePersistentOrchestrator (for persistent instructions). The key is implementing the instruct(self, dialog, utterance) method to define your orchestration logic. The method receives the current Dialog and the last utterance from the opposite agent (or None if it’s the first turn).

Let’s explore both types with practical examples:

  • One-time Orchestrator Example:

from sdialog.orchestrators import BaseOrchestrator

class EmpathyBooster(BaseOrchestrator):
    def __init__(self, emotion_keywords):
        self.emotion_keywords = emotion_keywords

    def instruct(self, dialog, utterance):
        if utterance and any(keyword in utterance.lower() for keyword in self.emotion_keywords):
            return ("The user seems to be experiencing difficult emotions. "
                    "Acknowledge their feelings with empathy and offer gentle support.")
        return None

empathy_booster = EmpathyBooster(["worried", "stressed", "frustrated",
                                  "upset", "anxious", "sad", "disappointed", "overwhelmed"])
agent = agent | empathy_booster
  • Persistent Orchestrator Example:

This simple example shows the minimal pattern: return an instruction once when a condition is first met; the instruction then persists automatically in the agent’s system messages and does not need to be returned again. A more advanced “LLM-as-a-Judge”-based persistent orchestrator is provided here.

from sdialog.orchestrators import BasePersistentOrchestrator

class EarlyRecapOrchestrator(BasePersistentOrchestrator):
    """Inject one persistent instruction once it's clear the user is asking too many questions."""
    def __init__(self, question_threshold=3):
        super().__init__()
        self.question_threshold = question_threshold
        self.instructed = False

    def instruct(self, dialog, utterance):
        if self.instructed or len(dialog.turns) < 4:
            return None
        # Count '?' in all turns from the other speaker (not this agent)
        other_qmarks = sum(t.text.count('?') for t in dialog.turns
                           if t.speaker.lower() != self.agent.name.lower())
        if other_qmarks >= self.question_threshold:
            self.instructed = True
            return ("From now on begin each reply with a concise bullet-style recap of the key questions so far, "
                    "then answer the latest question directly, and ask ONE clarifying follow-up only if needed.")
        return None

    def reset(self):  # allow reuse in later conversations
        super().reset()
        self.instructed = False

recap_orch = EarlyRecapOrchestrator(question_threshold=3)
agent = agent | recap_orch

Serving Agents via REST API

You can expose any Agent over an OpenAI/Ollama-compatible REST API using the serve() method to talk to it from tools like Open WebUI, Ollama GUI, or simple HTTP clients.

from sdialog.agents import Agent

# Let's create an example agent
support = Agent(name="Support")

# And serve it on port 1333 (default host 0.0.0.0)
support.serve(port=1333)
# Connect client to base URL localhost:1333

For example, to run Open WebUI in docker locally, just set OLLAMA_BASE_URL to point to port 1333 in the same machine when launching the container:

docker run -d -e OLLAMA_BASE_URL=http://host.docker.internal:1333 \
              -p 3030:8080 \
              -v open-webui:/app/backend/data --name open-webui \
              --restart always ghcr.io/open-webui/open-webui:main

Then open http://localhost:3030 in your browser to chat with the agent!

For serving multiple agents under a single endpoint, use Server’s serve(), as in the following example:

from sdialog.agents import Agent
from sdialog.server import Server

# Create multiple agents
agent1 = Agent(name="AgentOne")
agent2 = Agent(name="AgentTwo")

# Serve both agents on port 1333 (select them by name from the GUI)
Server.serve([agent1, agent2], port=1333)

Rate limits. To keep anyone who can reach the server from flooding it or running up LLM costs, each client (IP address) can send up to max_requests_per_minute chat requests per minute (30 by default) and max_requests_per_hour per hour (300 by default). Extra requests get an HTTP 429 “Too Many Requests” error with a Retry-After header, which clients like the OpenAI Python SDK retry automatically after a short wait. Only chat requests count, not model listings or health checks. Some things to keep in mind:

  • Clients may send several requests per user message (Open WebUI also asks the agent for a chat title, tags and follow-up suggestions), so the defaults are higher than they may seem to need to be.

  • Behind a reverse proxy on the same machine (e.g. nginx), the real client IP is used, but all users of an app like Open WebUI reach the server from the app’s address and share its limits. Raise them if the app serves several people.

  • For local scripts that call the server in a loop (e.g. evaluations or simulated users), disable the limits with None:

support.serve(port=1333, max_requests_per_minute=None, max_requests_per_hour=None)

Parallel requests. Each agent processes up to max_workers requests at the same time (8 by default), each on a fresh clone of the agent that shares its model (clone()), e.g. support.serve(port=1333, max_workers=16). Stateful agents (stateless=False) keep a single conversation, so they process one request at a time, and so do agents with a local Hugging Face model or with inspectors, since parallel calls would not be faster. With several workers, the agent’s tools and custom functions (e.g. postprocess_fn) may run at the same time for different requests, so they must be thread-safe; pass max_workers=1 otherwise.

Serving Agents on WhatsApp

Agents can also chat with real users over WhatsApp, through the Meta WhatsApp Cloud API, with serve_whatsapp(). Meta sends each incoming message to a webhook served by SDialog, the agent replies, and the reply is sent back through Meta’s API. Each user (phone number) has their own conversation, saved in a local SQLite database and restored on their next message with get_state() / set_state(). A user’s conversation starts over when they send /reset, when they open a new chat (see below), or after session_timeout seconds of inactivity.

To try it with a free Meta test account:

  1. At developers.facebook.com, create an app of type “Business” and add the WhatsApp product. Meta provides a test phone number, its Phone Number ID, and a temporary access token. Add your own phone number as an allowed recipient.

  2. Copy the App Secret from App settings → Basic. It is required, since it is used to check that webhook requests really come from Meta.

  3. Expose the server on a public HTTPS URL. For local testing, tunnel=True opens an ngrok tunnel (pip install pyngrok).

  4. In the app’s WhatsApp → Configuration page, set the callback URL (tunnel URL + /webhook) and the verify token, and subscribe to the messages field.

import os
from sdialog.agents import Agent

os.environ["WHATSAPP_ACCESS_TOKEN"] = "..."
os.environ["WHATSAPP_PHONE_NUMBER_ID"] = "..."
os.environ["WHATSAPP_APP_SECRET"] = "..."
os.environ["WHATSAPP_VERIFY_TOKEN"] = "any-random-string"  # same value as in Meta's webhook settings

support = Agent(name="Support")
support.serve_whatsapp(tunnel=True)

Then send a WhatsApp message to the test number from your phone.

Some things to keep in mind:

  • Only text messages are supported; users sending other media get a short notice.

  • Incoming messages are marked as read, and users see “typing…” until the reply arrives (WhatsAppConfig(mark_as_read=False) and typing_indicator=False disable them).

  • Messages sent more than max_message_age seconds ago (2 minutes by default) are ignored. Meta keeps retrying webhooks it could not deliver (e.g. while the server or tunnel was down) for days, and answering them hours later would be out of context. Set it to None to answer all messages.

  • Up to max_workers users (8 by default) are served at the same time, each message on a fresh clone of the agent that shares its model (clone()); a user’s own messages are still processed one at a time, in order. Agents with a local Hugging Face model or with inspectors are served one message at a time, since parallel calls would not be faster. With several workers, the agent’s tools and custom functions (e.g. postprocess_fn) may run at the same time for different users, so they must be thread-safe; pass max_workers=1 otherwise.

  • Each user can send up to max_messages_per_minute messages per minute (6 by default, enough for a few quick messages in a row) and max_messages_per_hour per hour (60 by default). Extra messages are dropped without calling the LLM, and the user gets a single notice; set either limit to None to disable it.

  • Conversations are stored in plain text in db_path (whatsapp_sessions.db by default), so treat that file as personal data. Conversations of users inactive for more than session_retention_days (30 by default) are deleted. Keep db_path on a local disk (SQLite does not work reliably on network filesystems such as NFS), and use one server process per database file: message ordering, duplicate detection and rate limits are kept per process.

  • With auto_setup=True (and tunnel=True), SDialog overwrites the webhook URL of your WhatsApp Business Account and phone number with the tunnel URL. Use it only with test accounts.

Starting new conversations. WhatsApp has no “new conversation” button, and Meta does not tell the business when a user clears or deletes a chat. If welcome messages are enabled on the phone number, though, Meta sends a request_welcome event when a user opens a chat with it for the first time. SDialog then starts a fresh conversation for that user and greets them, with the agent’s first_utterance if it has one, or else a greeting written by the agent. Welcome messages are off by default; to enable them on a phone number:

curl -X POST "https://graph.facebook.com/v21.0/$WHATSAPP_PHONE_NUMBER_ID/conversational_automation" \
     -H "Authorization: Bearer $WHATSAPP_ACCESS_TOKEN" -d "enable_welcome_message=true"

With the default session_timeout="auto", SDialog checks this setting at startup (and notices it when the first request_welcome event arrives): if welcome messages are enabled, conversations start over on new chats or after 24 hours of inactivity, in case a new chat goes unnoticed; otherwise they start over after 30 minutes of inactivity. Pass a number of seconds to always use that timeout, or None to never time out.

Note

Some Meta documentation says the event is also sent when a user deletes the chat and opens it again, but this is not reliable (in our tests with a Meta test number it was not sent). Users can always send /reset to start over.

Reacting to events. To act on what happens in a conversation, e.g. to alert a person when a safety orchestrator fires, pass an on_event function. It is called as on_event(user_id, event) with each Event of a user’s turn, in order, once the reply is sent: orchestrator instructions (with the orchestrator’s event_label as actionLabel), tool calls, and the reply (action="utter"). user_id is the user’s phone number.

safety = SimpleReflexOrchestrator(condition=lambda utterance: "help" in utterance.lower(),
                                  instruction="Take it seriously and point to emergency services.",
                                  event_label="safety")
agent = agent | safety

def on_event(user_id, event):
    if event.actionLabel == "safety":
        send_alert("Safety orchestrator triggered in a WhatsApp conversation")  # e.g. an email or a push notification

agent.serve_whatsapp(on_event=on_event)

on_event runs in the server’s threads, so it can block (e.g. to send an email) without holding up other users, but it may run at the same time for different users, so it must be thread-safe. Errors raised by it are logged and do not affect the conversation. Events contain the conversation’s text and user_id is personal data, so pass on only what is needed (e.g. that an alert was triggered rather than the messages).


Generation

Generation Components

The generation system in SDialog provides powerful tools for creating synthetic content at different levels of complexity. Whether you need to generate individual personas, contextual settings, or complete dialogues, these components work together to produce diverse and realistic outputs.

Attribute Generators

Let’s dive into one of SDialog’s most powerful features! Attribute generators combine LLM intelligence with rule-based patterns to create structured objects with randomized or AI-generated content. They are perfect for creating diverse personas and contexts for your dialogue simulations.

Both generators derive from BaseAttributeModelGenerator and support flexible attribute generation rules:

PersonaGenerator (PersonaGenerator)

Creates diverse character profiles with demographic, behavioral, and professional attributes. Ideal for generating varied participants in dialogue scenarios.

Let’s see how we can create teacher personas using only attributes defined in personas.py. For example, we can generate teachers with different subjects and years of experience:

import random
from sdialog.personas import Teacher
from sdialog.generators import PersonaGenerator

# Define a custom function to sample communication style based on years of experience
def get_communication_style(years_experience=None, **kwargs):
    # Example: simple mapping for demonstration
    if not years_experience:
        return "clear"
    return "enthusiastic" if years_experience < 10 else "authoritative"

# 1) Create a generator for teacher personas
teacher_gen = PersonaGenerator(Teacher)

# 2) Setup generation with interdependent attributes
teacher_gen.set(
    subject=["mathematics", "history", "biology"],
    years_experience="{2-25}",
    communication_style=get_communication_style,  # Depends on years_experience
    politeness=["polite", "neutral", "strict"]
)

# 3) Generate diverse teachers with contextually appropriate communication styles
teacher1 = teacher_gen.generate()
teacher2 = teacher_gen.generate()

# 4) Let's generate 3 more teachers in one shot
teachers_batch = teacher_gen.generate(n=3)  # Returns list of 3 teachers
ContextGenerator (ContextGenerator)

Generates rich contextual frameworks that define the setting, environment, and situational constraints for dialogues. Essential for creating realistic and consistent conversation backgrounds.

Now let’s create varied classroom contexts to set the stage for our educational conversations:

from sdialog import Context
from sdialog.generators import ContextGenerator

# Create varied classroom contexts
ctx_gen = ContextGenerator(Context(location="classroom"))
ctx_gen.set(
    environment="{llm:Describe a realistic educational environment}",
    constraints=["time pressure", "distraction", "group activity"]
)
context = ctx_gen.generate()

Attribute Rule Patterns

The set() method accepts various rule types for flexible content generation:

  • LLM Delegation: "*" or "{llm}" - Let the AI decide the value (default)

  • Guided LLM: "{llm:Create a professional background}" - AI with specific instructions

  • Numeric Ranges: "{5-25}" - Random integer between bounds (inclusive)

  • File Sources: "{txt:names.txt}" - Random line from text file

  • Data Tables: "{csv:specialty:doctors.csv}" - Random value from CSV/TSV column

  • Python Functions: callable(**attributes) - Execute function (receives partial object if compatible)

  • Choice Lists: ["option1", "option2"] - Random selection from list

  • Fixed Values: "specific_value" - Direct assignment

Dialogue Generators

Now let’s move on to creating complete conversations! Dialogue generators create full dialogues using different approaches, from direct LLM instruction to sophisticated persona-driven interactions.

DialogGenerator (DialogGenerator)

The foundational dialogue generator that creates conversations based on free-form instructions. Great for quick prototyping and simple dialogue generation tasks.

Let’s start with a simple example—generating a medical consultation:

from sdialog.generators import DialogGenerator

# Generate a consultation dialogue
gen = DialogGenerator("Generate a brief medical consultation about headaches")

dialog = gen.generate()
dialog.print()
PersonaDialogGenerator (PersonaDialogGenerator)

Creates sophisticated dialogues by having two distinct personas or agents interact naturally. This generator produces more realistic and character-consistent conversations.

Here’s how we can create a dialogue between a mentor and student with their unique characteristics:

from sdialog.personas import Teacher, Student
from sdialog.generators import PersonaDialogGenerator

teacher = Teacher(name="Ms. Lee", subject="mathematics")
student = Student(name="Alex", interests="algebra")

# Generate persona-driven dialogue
gen = PersonaDialogGenerator(
    teacher, student,
    dialogue_details="Discuss learning challenges and provide guidance"
)

dialog = gen.generate()
dialog.print()
Paraphraser (Paraphraser)

Transforms existing dialogues following user-provided instructions. Useful for improving synthetic dialogues, adapting content for different styles, or data augmentation.

Let’s see how we can make automated responses sound more natural and empathetic:

from sdialog.generators import Paraphraser

# Make bot responses sound more natural
paraphraser = Paraphraser(
    extra_instructions="Make responses sound more empathetic and natural",
    target_speaker="Bot"  # Only paraphrase "Bot"'s turns
)

improved_dialog = paraphraser(original_dialog)
improved_dialog.print()

Reproducibility & Seeding

Every generation entry point accepts seed where feasible. Metadata persists seed and model so output artifacts remain auditable. Identical model + parameters + seed ⇒ deterministic attribute choices & (backend permitting) stable dialog trajectories. Use clone(new_id=...) to branch derived objects with recorded lineage.


Evaluation & Interpretability

Evaluation Suite

Categories:

  1. Linguistic / Readability: LinguisticFeatureScore - mean-turn-length, hesitation-rate, gunning-fog, flesch-reading-ease.

  2. Flow Graph Based: DialogFlowScore (likelihood) & DialogFlowPPL (perplexity-like) against reference dialog graph.

  3. Embedding Similarity: SentenceTransformerDialogEmbedder + evaluators such as ReferenceCentroidEmbeddingEvaluator.

  4. Distribution Divergence (score-level): KDEDistanceEvaluator, FrechetDistanceEvaluator.

  5. LLM Judges: Binary or numeric scoring with structured output
  6. Advanced Embedding Evaluators: FrechetBERTDistanceEvaluator (Fréchet distance with BERT embeddings), PrecisionRecallDistanceEvaluator (Precision-Recall curves based on BERT).

  7. Dataset Aggregators: DatasetComparator to compare multiple evaluators.

  8. Statistics / Frequency: StatsEvaluator (mean/std/min/max/median), MeanEvaluator, FrequencyEvaluator.

Evaluation Examples:

Let’s explore how to evaluate our generated dialogues using SDialog’s comprehensive evaluation suite. We’ll walk through different evaluation approaches to assess dialogue quality from multiple perspectives:

from sdialog.evaluation import (
    LinguisticFeatureScore, DialogFlowScore, LLMJudgeRealDialog, LLMJudgeRealDialogLikertScore,
    MeanEvaluator, FrequencyEvaluator, StatsEvaluator,
    DatasetComparator
)

# Linguistic analysis
hesitation_score = LinguisticFeatureScore(feature="hesitation-rate")
readability_score = LinguisticFeatureScore(feature="flesch-reading-ease")

# Flow-based evaluation (requires reference dialogues)
flow_evaluator = DialogFlowScore(reference_dialogs)

# LLM-based judgment
realism_judge = LLMJudgeRealDialog(reason=True)
realism_judge_likert = LLMJudgeRealDialogLikertScore()  # 1-5 scale

# Multi-metric comparison across different models
comparator = DatasetComparator([
    FrequencyEvaluator(realism_judge),
    StatsEvaluator(readability_score),
    MeanEvaluator(hesitation_score),
    MeanEvaluator(flow_evaluator)
])

# Evaluate multiple dialog sets
results = comparator({
    "model_A": dialogs_a,
    "model_B": dialogs_b,
    "human_baseline": human_dialogs
})
comparator.plot()

Caching: Some score computations, so a good practice is to enable caching (sdialog.config.cache(True)) to accelerate repeated runs across large corpora.

Interpretability & Steering

SDialog provides advanced tools for mechanistic interpretability and activation steering to understand and control model behavior during dialogue generation.

Activation Inspection:

Let’s start exploring the inner workings of our models! Inspector registers PyTorch forward hooks on specified model layers to capture per-token activations during generation. This allows us to see what’s happening inside the model as it generates responses:

from sdialog.interpretability import Inspector
from sdialog.agents import Agent

# Create agent and inspector
agent = Agent(persona=my_persona)
inspector = Inspector(target='model.layers.15.post_attention_layernorm')

# Attach inspector to agent
agent = agent | inspector

# Generate responses (activations are captured automatically)
response1 = agent("Tell me about the weather")
response2 = agent("What's your favorite color?")

# Access captured activations
print(f"Captured {len(inspector)} responses")
first_token_activation = inspector[-1][0].act  # Last response, first token
print(f"Activation shape: {first_token_activation.shape}")

# Inspect system instructions
inspector.find_instructs(verbose=True)

Activation Steering:

Now for the exciting part—actually steering the model’s behavior! DirectionSteerer enables targeted manipulation of model activations. This means we can guide the model’s responses in specific directions:

import torch
from sdialog.interpretability import DirectionSteerer

# Create a direction vector (e.g., from activation differences)
empathy_direction = torch.randn(4096)  # Model hidden size

# Create steerer and attach to inspector
steerer = DirectionSteerer(empathy_direction)
inspector = inspector + steerer  # Add direction
# or: inspector = inspector - steerer  # Subtract direction

# Generate with steering applied
steered_response = agent("I'm feeling really sad today")

# Compare with baseline
inspector.clear()  # Reset to remove steering
baseline_response = agent("I'm feeling really sad today")

Advanced Usage:

Ready to dive deeper? Here’s how we can inspect multiple layers simultaneously and apply fine-grained steering control:

# Multi-layer inspection
multi_inspector = Inspector({
    'early': 'model.layers.5.post_attention_layernorm',
    'middle': 'model.layers.15.post_attention_layernorm',
    'late': 'model.layers.25.post_attention_layernorm'
},
steering_interval=(0, 2))  # steer only the first two generated tokens

agent = agent | multi_inspector - direction

Audio Generation

SDialog turns any two-speaker dialogue into realistic audio with a single call, dialog.to_audio(), and returns the audio together with its ground-truth annotations: who speaks when, what is said, and word timestamps. With control over voices, timing and acoustics, the same text dialogues can produce audio at different levels of difficulty, for ASR, speaker diarization, speech LLMs, call-center analytics and other speech tasks.

Setup and Installation

pip install sdialog[audio]

The default TTS engine, Dia2, requires Python >= 3.10, and a GPU is strongly recommended. It is installed with sdialog[audio], and its weights are downloaded from Hugging Face on first use. On Python 3.9, sdialog[audio] installs without Dia2; use another engine instead (see Using Other TTS Engines).

Quick Start

audio = dialog.to_audio()
audio.save("my_dialog")  # my_dialog.wav + my_dialog.json (annotations) + my_dialog.rttm

To make it sound like a phone call, a meeting room or a noisy far-field recording, use a preset (see Presets):

audio = dialog.to_audio(preset="call_center")

The returned object holds the audio and its annotations:

audio.audio, audio.sample_rate        # numpy waveform and sample rate
audio.display()                       # audio player (Jupyter)

for seg in audio.segments:            # one segment per turn
    print(f"{seg.start:6.2f} - {seg.end:6.2f}  {seg.speaker}: {seg.text}")

Pass seed= to get the same audio every time. Without it, a random seed is used and saved in audio.config["seed"]. The sections below cover the options for advanced use.

Note

Looking for the AudioPipeline and room API described in the SDialog paper? See Coming from the Legacy Audio Pipeline.

Voices

The voices argument selects how each speaker’s voice is chosen:

  • "db" (default): picked from a voice database to match each persona’s gender, age and language. The default database is sdialog/voices-libritts; pass voice_database= to use another one (see Voice Database Management). Ages such as "middle-aged" or "30s" are understood. Note that sdialog/voices-libritts has no age labels, so with it voices are matched by gender and language only.

  • "design": designed from each persona’s description with Qwen3-TTS VoiceDesign (requires pip install qwen-tts). The description comes from a Jinja2 template over the persona fields, which you can change with voice_description=.

  • "random": made up by Dia2, giving new voices for every seed. Each speaker keeps their voice for the whole dialogue.

  • A dictionary {speaker: voice} with explicit voices, keyed by speaker name or index. A voice is a reference audio path, an (array, sample_rate) tuple, or a Voice from a database. Speakers missing from the dictionary get a random voice (a database voice if the engine cannot make up voices).

audio = dialog.to_audio(voices="random", seed=42)
audio = dialog.to_audio(voices="design", voice_description="A {{ gender }} voice, {{ age }} years old, calm.")
audio = dialog.to_audio(voices={"Doctor": "voices/doctor.wav", "Patient": "voices/patient.wav"})
print(audio.voices)                       # which voice each speaker got

Timing and Acoustics

Argument

Effect

pauses

"natural" (default) keeps the timing produced by the TTS engine. A number is the median pause in seconds (log-normal). A (min, max) tuple draws pauses uniformly.

overlap

Probability (0-1) that a turn starts before the previous one ends. max_overlap caps the overlap as a fraction of the previous turn (default 0.3). A speaker never overlaps themselves.

reverb

"small", "medium", "large" (or True): a simulated room (pyroomacoustics) with a random size, RT60 and speaker positions. A Room: a room of your choice (see Custom Rooms). Alternatively, a path to an impulse-response file or folder of measured RIRs. Speech onsets are kept aligned, so the annotations stay valid.

noise

An SNR in dB, a (min, max) SNR range, a noise type ("white", "pink", "brown") or a noise audio file/folder, or {"type": ..., "snr": ...}.

channel

A telephone codec: "telephone" (= "g711"), "g711", "g729", "amr", "amr-wb".

level_jitter_db

Maximum random loudness difference (dB) between the two speakers.

stereo

True for one channel per speaker (in order of appearance), False (default) for mono.

sample_rate

Output sample rate (default 16000 Hz).

reverb, noise and channel accept one value for the whole recording, or a dictionary per speaker, keyed by speaker name or index (0 = first speaker). A list of values means “pick one at random”:

# both speakers in the same large room (different positions), noisy
audio = dialog.to_audio(reverb="large", noise=(5, 15))

# a phone call: only the caller goes through a (random) codec, and only their side is noisy
audio = dialog.to_audio(channel={"Caller": ["g711", "amr"]},
                        noise={"Caller": {"type": "pink", "snr": (10, 20)}},
                        overlap=0.15, stereo=True)

Effects for a single speaker are applied to that speaker’s own track. Shared effects are applied to the recording: the mono mix, or each stereo channel.

Custom Rooms

For full control over the room, pass a Room as reverb. It describes a shoebox room simulated with pyroomacoustics (image source method). It is the legacy pipeline’s Room (see Coming from the Legacy Audio Pipeline), reworked to run without the legacy pipeline’s dependencies: pyroomacoustics is already installed with sdialog[audio], so nothing else is needed.

A Room sets:

  • dimensions: width (x), length (y) and height (z), in meters.

  • Absorption: either a target rt60 in seconds, or materials: a pyroomacoustics material name (e.g. "brickwork", "plasterboard", "carpet_thin") or absorption coefficient for every surface, or a dictionary by surface ("walls", "floor", "ceiling", or a single wall: "east", "west", "north", "south"). The default is wooden walls, a carpet and an absorbing ceiling.

  • mic: (x, y, z) coordinates, "ceiling" (center of the ceiling, the default), "wall", "between_speakers" (e.g. a table microphone), or a speaker name or index (a lapel microphone on that speaker’s chest).

  • mic_pattern: "omni" (default), "cardioid", "subcardioid", "hypercardioid" or "figure8", and mic_direction: a speaker name or index, "speakers" (their midpoint, the default) or (azimuth, colatitude) in degrees.

  • speakers: (x, y, z) positions by speaker name or index. Speakers without a position are placed at random, from the to_audio seed.

  • air_absorption and ray_tracing (more realistic late reverberation, but slower) are off by default.

from sdialog.audio import Room

room = Room(dimensions=(5, 4, 2.8),
            materials={"walls": "plasterboard", "floor": "linoleum_on_concrete"},
            mic="between_speakers", mic_pattern="cardioid", mic_direction="Doctor",
            speakers={"Doctor": (1.5, 2, 1.2), "Patient": (3.5, 2, 1.2)})
audio = dialog.to_audio(reverb=room, noise=30)
print(audio.config["room"])   # the simulated room, with the microphone and speaker positions

# Each speaker in their own room, e.g. both ends of a phone call
audio = dialog.to_audio(reverb={"Agent": Room(dimensions=(3, 3, 2.5), mic="Agent"),
                                "Caller": Room(dimensions=(8, 6, 3), rt60=0.9)},
                        channel={"Caller": "g711"}, stereo=True)

To get the impulse responses only (e.g. to use them elsewhere), call room.compute_rirs(["Alice", "Bob"], sample_rate=16000).

Presets

Presets bundle realistic values for the timing and acoustic arguments:

Preset

Scene

clean

Natural TTS timing, no effects (the default).

conversational

Shorter pauses, 15% overlapping turns, small loudness differences.

room

Meeting/office: both speakers in a medium room, light pink noise (SNR 20-30 dB).

call_center

Agent (first speaker) on a headset; caller (second speaker) through a G.711/AMR phone line with noise (SNR 10-25 dB).

far_field

Distant microphone in a large reverberant room, SNR 5-15 dB, 20% overlaps.

hard

Large room, SNR 0-10 dB, G.711 channel, 30% overlaps, ±3 dB loudness differences.

A preset only sets the scene, never the output format, so any preset can be rendered in mono or stereo. Explicit arguments override the preset, and False turns off one of its effects:

audio = dialog.to_audio(preset="call_center", stereo=True)   # agent on channel 0, caller on channel 1
audio = dialog.to_audio(preset="call_center")                 # same scene, mono
audio = dialog.to_audio(preset="hard", channel=False)          # hard, without the telephone channel

from sdialog.audio import register_preset
register_preset("noisy_phone", channel="g711", noise={"type": "brown", "snr": (0, 10)}, overlap=0.2)
audio = dialog.to_audio(preset="noisy_phone")

Annotations

to_audio() returns a DialogAudio object:

  • audio.segments: one Segment per turn, with turn (index in the dialogue), speaker, text, start, end, channel (stereo only) and words (word start times, when the engine provides them).

  • audio.voices: the voice of each speaker, e.g. the voice database identifier.

  • audio.config: every parameter used, including the values drawn at random (seed, room size and RT60, SNRs, codecs, …).

  • audio.to_rttm() (speaker diarization ground truth), audio.to_dict() and audio.to_json().

Batch Generation

dialogs_to_audio converts many dialogues in parallel, with one TTS model per worker process. By default it uses every available GPU (or the CPU if there is none). devices can also be "cpu", "cuda", a single device like "cuda:1", or a list of devices.

Each dialogue gets its own seed. With seed set, the seed is derived from it and the dialogue’s position, so the whole batch is reproducible and results don’t depend on the number of workers. Without it, a random seed is used. Either way, the seed of every dialogue is saved in its annotations and in index.jsonl:

from sdialog.audio import dialogs_to_audio

rows = dialogs_to_audio(dialogs, "audios/", preset="call_center", stereo=True)
# audios/<dialog_id>.wav/.json/.rttm and audios/index.jsonl

Using Other TTS Engines

Any classic, one-utterance-at-a-time engine can be passed as tts; it is wrapped with PerTurnTTS. This also works on Python 3.9. Each turn is then generated independently:

from sdialog.audio.tts import Dia2TTS, KokoroTTS, Qwen3TTSVoiceClone

audio = dialog.to_audio(tts=KokoroTTS())                         # voices from sdialog/voices-kokoro
audio = dialog.to_audio(tts=Qwen3TTSVoiceClone())                # voice cloning from sdialog/voices-libritts
audio = dialog.to_audio(tts=Dia2TTS("nari-labs/Dia2-1B", cfg_scale=3.0))   # smaller / tuned Dia2

New dialogue-level engines can be added by subclassing sdialog.audio.tts.BaseDialogTTS.

Voice Database Management

SDialog supports multiple voice database types for flexible voice selection:

HuggingFace Voice Databases - Use pre-trained voice collections:

from sdialog.audio.voice_database import HuggingfaceVoiceDatabase

# LibriTTS voices
voices_libritts = HuggingfaceVoiceDatabase("sdialog/voices-libritts")

# Kokoro voices
voices_kokoro = HuggingfaceVoiceDatabase("sdialog/voices-kokoro")

# Get voice statistics
print(voices_kokoro.get_statistics(pretty=True))

# Select voices based on characteristics
female_voice = voices_libritts.get_voice(gender="female", age=25, seed=42)
# Prevent voice reuse
male_voice = voices_libritts.get_voice(gender="male", age=30, keep_duplicate=False)

# Reset used voices for reuse
voices_libritts.reset_used_voices()

Local Voice Databases - Use your own voice files:

from sdialog.audio.voice_database import LocalVoiceDatabase

# Create database from local files with CSV metadata
voice_database = LocalVoiceDatabase(
    directory_audios="./my_custom_voices/",
    metadata_file="./my_custom_voices/metadata.csv"
)

# Add custom voices programmatically
voice_database.add_voice(
    gender="female",
    age=42,
    identifier="french_female_42",
    voice="./my_custom_voices/french_female_42.wav",
    lang="french",
    language_code="f"
)

# Get voice by language and prevent voice reuse (it will give you the closest age for the specified gender)
french_voice = voice_database.get_voice(gender="female", age=20, lang="french", keep_duplicate=False)

# Get statistics
print(voice_database.get_statistics(pretty=True))

Quick Voice Database - Create databases from dictionaries:

from sdialog.audio.voice_database import VoiceDatabase

# Create database from predefined voice list
quick_voices = VoiceDatabase(
    data=[
        {
            "voice": "am_echo",
            "language": "english",
            "language_code": "a",
            "identifier": "am_echo",
            "gender": "male",
            "age": 20
        },
        {
            "voice": "af_heart",
            "language": "english",
            "language_code": "a",
            "identifier": "af_heart",
            "gender": "female",
            "age": 25
        }
    ]
)

# Use the voices
male_voice = quick_voices.get_voice(gender="male", age=20)
female_voice = quick_voices.get_voice(gender="female", age=25)

# Unavailable voice for this language (an error will be raised)
female_voice_spanish = quick_voices.get_voice(gender="female", age=25, lang="spanish")

Coming from the Legacy Audio Pipeline

The audio pipeline described in the SDialog paper (AudioPipeline, AudioDialog, perform_room_acoustics=True, room generators, dSCAPER background and foreground effects) was deprecated in v0.5.0 and removed in v0.7.0. It is still available in earlier versions, with its documentation:

pip install "sdialog[audio,audio-legacy]<0.7"   # v0.6.0 is the last version with the legacy pipeline

dialog.to_audio() covers the same use cases without dSCAPER or system packages (sdialog[audio] is all you need, custom rooms included):

Legacy pipeline

dialog.to_audio()

to_audio(dialog, perform_room_acoustics=True, room=room)

dialog.to_audio(reverb=Room(...)) (see Custom Rooms)

Room(dimensions=Dimensions3D(...), materials=RoomMaterials(...)) or reverberation_time_ratio=

Room(dimensions=(w, l, h), materials={...}) or Room(..., rt60=...)

room.place_speaker(...), mic_position=MicrophonePosition.CUSTOM

Room(speakers={name: (x, y, z)}, mic=(x, y, z))

MicrophonePosition.CEILING_CENTERED, WALL_MOUNTED, MIDDLE_SPEAKERS, CHEST_POCKET_SPEAKER_1

mic="ceiling", "wall", "between_speakers", or the speaker’s name or index

room.set_directivity(DirectivityType.SPEAKER_1)

Room(mic_pattern="cardioid", mic_direction=0)

BasicRoomGenerator().generate({"room_size": ...}), MedicalRoomGenerator

reverb="small", "medium" or "large" (random rooms), or a list of Room objects to pick from

Background and foreground effects (dSCAPER)

noise= with a noise type or a folder of noise files, and an SNR

Impulse response databases (microphone simulation)

reverb="path/to/irs/" (a file or folder of impulse responses)

Telecommunications backend (two rooms and a phone line)

Per-speaker reverb={...} (one Room each) and channel=

AudioDialog with per-step audio files

DialogAudio with the waveform, the annotations (segments, RTTM) and config

Furniture, room images (room.to_image()) and speaker placement around furniture have no equivalent: set the positions directly, as furniture did not change the acoustics.


Configuration & Control

Configuration Layer

Centralized configuration lives in sdialog.config:

Supported Backend Formats:

  • openai:MODEL - OpenAI models (GPT-3.5, GPT-4, etc.)

  • huggingface:MODEL - HuggingFace transformers models

  • ollama:MODEL - Local/remote Ollama models

  • amazon:MODEL or aws:MODEL - AWS Bedrock models (Anthropic Claude, etc.)

  • google:MODEL or google-genai:MODEL - Google Gen AI models (Gemini, etc.)

  • anthropic:MODEL - Anthropic API models (Claude, etc.)

  • azure:MODEL or azure-openai:MODEL - Azure OpenAI models

  • Local model instances passed directly

Configuration Examples:

Let’s configure SDialog to work with your preferred LLM backend. Here are some common configuration patterns:

import sdialog.config as config

# Set global LLM backend and model with specific parameters
config.llm("openai:gpt-4",
           temperature=0.7,
           max_tokens=1000)

# Use local Ollama model with default parameters
config.llm("ollama:llama2")

# Configure Amazon Bedrock with default parameters
# plus specific temperature value
config.llm("amazon:anthropic.claude-v2",
           temperature=0.5)

# Enable evaluation caching with default parameters
config.cache(True)

# Enable caching in specific path
config.set_cache("/path/to/cache", enable=True)

Component-Level Overrides:

Any component that uses LLMs (Agents, Generators, Judges, Orchestrators) accepts model followed by its parameters to override global config on a per-instance basis.

from sdialog.agents import Agent

# Create an agent with custom model and parameters
custom_agent = Agent(
    name="CustomAgent",
    model="ollama:llama3",
    temperature=0.7,
    base_url="http://localhost:11434"
)

For instance, you can set a default model globally that fits your GPU via sdialog.config.llm(), and then override specific components, like an LLM-as-a-Judge to use a powerful API-based model as a judge (e.g. OpenAI models).

Tools & Function Calling

Provide a list of Python callables to an Agent via tools=[fn_a, fn_b]. For backends that expose tool/function-calling semantics, outputs may be (a) executed and inserted into memory, (b) used to refine final completions depending on backend support. Ensure return values are JSON-serializable.

Thinking Segments

If think=True on Agent (and backend supports surrogate reasoning tokens), SDialog can preserve or strip these segments. If not natively supported by the underlying, SDialog can still recognize thinking segments using the provided value in thinking_pattern (regex capturing group). Suitable for experiments analyzing chain-of-thought style reasoning without leaking it downstream.


Extensibility & I/O

Serialization & Persistence

All core objects support JSON serialization with metadata. Typical patterns:

dialog.to_file("session.json")
restored = Dialog.from_file("session.json")

context.to_file("ccontext.json")
restored = context.from_file("ccontext.json")

persona.to_file("persona.json")
restored = Persona.from_file("persona.json")

For plain text or CSV import/export: to_file() (type='auto'|'txt'|'csv'|'json'); similarly from_file() (type='auto'|'txt'|'csv'|'json'). The default is 'auto' (infers the format from the file extension).

Extensibility Patterns

Create new components by subclassing:


Operations & Recipes

Testing & Reproducibility Tips

  • Use prompt() to get the underlying prompt sent to the LLM for any LLM-dependant component (Agent and generators) for paper reporting.

  • Use json() to export any LLM-dependant component (Agent, Persona, Context, Generators) for reproducibility.

  • Use clone() to derive variations while preserving ancestry for auditing/controllability.

  • Cache expensive score computations (flow graph building, embedding passes) across experimental sweeps.

Performance Considerations

  • Batch evaluation: Pre-build flow graphs once (reuse in multiple scorers) or supply precomputed graph / nodes to Flow scores.

  • Embedding evaluators: Control batch size & device in SentenceTransformer / BERT evaluators to optimize GPU utilization.

  • Caching: Enable disk cache for repeated LLMJudge or flow evaluations over large static corpora.

Common Recipes

  1. Generate Multi-Variant Persona Set: PersonaGenerator with rule lists and numeric ranges + seed cycle.

  2. Scenario Simulation at Scale: For each scenario → build Agents → attach orchestrators → run dialog_with for N seeds.

  3. Quality Filtering Pipeline: Generate dialogs → apply LLM judges (realism > threshold) → compute flow score percentile → retain top quantile.

  4. Style Harmonization: Paraphrase dialogs targeting only system speaker with controlled extra instructions.

  5. Activation Steering Study: Attach Inspector → collect baseline activations → compute direction (e.g., mean difference) → apply DirectionSteerer → compare linguistic + refusal metrics pre/post.