Build a salary negotiation coach with a LemonSlice avatar

This tutorial walks you through building a salary negotiation practice app that uses a talking avatar powered by LemonSlice. Users pick a boss personality, practice their pitch out loud, and can pause at any point to ask the same avatar for coaching feedback. When they do, the agent breaks character, gives specific advice, then steps back into the boss role.

For the full source code, see the lemonslice example.

What you'll build

By the end of this tutorial, you'll have a working app that:

• Renders a talking LemonSlice avatar in the browser, lip-synced to the agent's voice
• Lets users pick from three boss personalities with distinct voices and behaviors
• Switches between roleplay and coaching mode using LiveKit Agents function tools
• Syncs the current mode to the frontend via participant attributes so the UI updates in real time

Prerequisites

Before you start, make sure you have:

• Python 3.11 or later and uv installed
• Node.js and pnpm installed
• A LiveKit Cloud account (the free tier works)
• A LemonSlice API key

The agent uses LiveKit's inference API for STT, LLM, and TTS, so you don't need separate API keys from Deepgram, OpenAI, or Cartesia.

Step 1: Get the code

Use pnpm dlx degit to download just the lemonslice folder without cloning the entire repository:

1
pnpm dlx degit livekit-examples/python-agents-examples/complex-agents/avatars/lemonslice lemonslice
2
cd lemonslice

The directory has two parts:

• agent/ — the Python LiveKit agent
• frontend/ — a Next.js frontend

Step 2: Set up the agent

Install dependencies from the existing pyproject.toml:

1
cd agent
2
uv sync

Copy .env.example to .env.local in the agent/ directory and fill in your values:

1
cp .env.example .env.local

1
LIVEKIT_API_KEY=<your_api_key>
2
LIVEKIT_API_SECRET=<your_api_secret>
3
LIVEKIT_URL=wss://<project-subdomain>.livekit.cloud
4
LEMONSLICE_API_KEY=<your_lemonslice_api_key>
5

6
# Optional: use your own publicly accessible avatar images
7
EASY_BOSS_IMAGE_URL=https://...
8
MEDIUM_BOSS_IMAGE_URL=https://...
9
HARD_BOSS_IMAGE_URL=https://...

Get your LiveKit credentials from the LiveKit Cloud dashboard under Settings > API Keys. Default boss avatar images are already provided in the code, so the image URL variables are only needed if you want to swap them out.

Step 3: Understand the agent

This section walks through the key parts of agent/lemonslice-agent.py.

Boss configurations

Each boss has a unique Cartesia voice ID, avatar image URL, and a prompt for the avatar's behavior stored in a BossConfig dataclass:

1
@dataclass
2
class BossConfig:
3
    voice_id: str
4
    avatar_image_url: str
5
    avatar_prompt: str
6

7
BOSS_CONFIGS = {
8
    "easy": BossConfig(
9
        voice_id="f786b574-daa5-4673-aa0c-cbe3e8534c02",
10
        avatar_image_url=os.getenv("EASY_BOSS_IMAGE_URL", "https://iili.io/frL9tuj.png"),
11
        avatar_prompt="Be warm and encouraging in your movements. Use open gestures and smile naturally.",
12
    ),
13
    "medium": BossConfig(
14
        voice_id="228fca29-3a0a-435c-8728-5cb483251068",
15
        avatar_image_url=os.getenv("MEDIUM_BOSS_IMAGE_URL", "https://iili.io/frL9L8u.png"),
16
        avatar_prompt="Be professional and thoughtful in your movements. Use controlled gestures.",
17
    ),
18
    "hard": BossConfig(
19
        voice_id="66c6b81c-ddb7-4892-bdd5-19b5a7be38e7",
20
        avatar_image_url=os.getenv("HARD_BOSS_IMAGE_URL", "https://iili.io/frL9Qyb.png"),
21
        avatar_prompt="Be direct and professional in your movements. Show confidence through body language.",
22
    ),
23
}

Because the agent uses inference.TTS("cartesia/sonic-3", voice=config.voice_id), LiveKit routes to Cartesia automatically without you needing to manage a Cartesia API key directly.

Session state

A UserData dataclass tracks the state of each session and gets passed to AgentSession via userdata=userdata:

1
@dataclass
2
class UserData:
3
    ctx: Optional[JobContext] = None
4
    boss_type: str = "easy"
5
    mode: str = "roleplay"  # "coaching" or "roleplay"
6
    session_start_time: float = 0.0
7
    timer_task: Optional[asyncio.Task] = None
8
    session_ended: bool = False

Any agent method or function tool can access it through context.userdata.

Attaching the LemonSlice avatar

In the entrypoint, after the room connects and the correct boss agent is ready, create an AvatarSession and start it before starting the agent session:

1
avatar = lemonslice.AvatarSession(
2
    agent_image_url=boss_config.avatar_image_url,
3
    agent_prompt=boss_config.avatar_prompt,
4
)
5
await avatar.start(session, room=ctx.room)
6

7
await session.start(
8
    agent=boss_agent,
9
    room=ctx.room,
10
    room_options=room_io.RoomOptions(delete_room_on_close=True),
11
)

avatar.start() must come before session.start(). LemonSlice joins the room as a separate participant and publishes a video track. The LiveKit session routes TTS audio to LemonSlice, which drives lip-sync.

The agent_prompt parameter influences how the avatar moves and expresses itself. "Be warm and encouraging" vs "Be direct and professional" produces noticeably different body language.

Switching modes with function tools

The core design here is a single agent that plays two roles and switches between them on demand. BaseBossAgent defines two function tools:

1
@function_tool()
2
async def how_am_i_doing(self, context: RunContext_T) -> str:
3
    """User is asking for coaching feedback on their negotiation performance."""
4
    userdata = context.userdata
5
    userdata.mode = "coaching"
6
    userdata.coaching_requests += 1
7

8
    await userdata.ctx.room.local_participant.set_attributes({"mode": "coaching"})
9

10
    await self.update_instructions(
11
        f"{self.instructions}\n\nIMPORTANT: You are now in COACHING MODE. "
12
        "Break character from the boss role completely and provide honest, specific "
13
        "feedback on their negotiation performance so far. After giving feedback, "
14
        "call the return_to_roleplay function to switch back."
15
    )
16
    return "Switching to coaching mode to provide feedback."
17

18
@function_tool()
19
async def return_to_roleplay(self, context: RunContext_T) -> str:
20
    """Return from coaching mode back to the boss role-play."""
21
    userdata = context.userdata
22
    userdata.mode = "roleplay"
23

24
    await userdata.ctx.room.local_participant.set_attributes({"mode": "roleplay"})
25
    await self.update_instructions(self.instructions)
26
    return "Returning to boss role-play mode."

update_instructions() swaps the system prompt at runtime without interrupting the session or restarting the model. When the user clicks "How am I doing?" in the frontend, a chat message is sent. The LLM sees it in context and calls how_am_i_doing(). After giving feedback, the LLM calls return_to_roleplay() to reset.

The set_attributes() call broadcasts the current mode to all room participants. The frontend reads this and updates the avatar border color.

Reading boss type from participant attributes

The entrypoint is registered with @server.rtc_session(agent_name="lemonslice-salary-coach"). That string is how LiveKit identifies which agent to dispatch when a room is created — the frontend must reference this exact name. The frontend passes boss_type as a participant attribute when connecting, and the agent reads it here to initialize the correct boss:

1
@server.rtc_session(agent_name="lemonslice-salary-coach")
2
async def entrypoint(ctx: JobContext):
3
    await ctx.connect(auto_subscribe=AutoSubscribe.AUDIO_ONLY)
4
    participant = await ctx.wait_for_participant()
5
    boss_type = participant.attributes.get("boss_type", "easy")
6

7
    if boss_type == "easy":
8
        boss_agent = EasyBossAgent()
9
    elif boss_type == "medium":
10
        boss_agent = MediumBossAgent()
11
    else:
12
        boss_agent = HardBossAgent()

Session timer

A 3-minute timer runs as a background asyncio.Task. When it fires, the agent delivers a closing message and shuts down the room:

1
await self.session.say("Our practice session time is up. I hope you have a great day.")
2
await asyncio.sleep(2.0)
3
userdata.ctx.shutdown("Session timer expired")

The timer starts in on_enter and is cancelled in on_exit if the user disconnects early.

Step 4: Set up the frontend

Copy the example env file and fill in your credentials:

1
cd frontend
2
cp .env.example .env.local

1
LIVEKIT_API_KEY=<your_api_key>
2
LIVEKIT_API_SECRET=<your_api_secret>
3
LIVEKIT_URL=wss://<project-subdomain>.livekit.cloud
4
AGENT_NAME=lemonslice-salary-coach

AGENT_NAME must match the agent_name in @server.rtc_session from the Python file — if you change one, change both.

Then install dependencies:

1
pnpm install

How the frontend connects the pieces

The App component builds a custom TokenSource that POSTs the selected boss_type to /api/connection-details, which embeds it as a participant attribute in the LiveKit JWT:

1
const tokenSource = useMemo(() => {
2
  const participantAttributes = { boss_type: selectedBoss };
3

4
  return TokenSource.custom(async () => {
5
    const res = await fetch('/api/connection-details', {
6
      method: 'POST',
7
      headers: { 'Content-Type': 'application/json' },
8
      body: JSON.stringify({
9
        room_config: { agents: [{ agent_name: appConfig.agentName }] },
10
        participant_attributes: participantAttributes,
11
      }),
12
    });
13
    return res.json();
14
  });
15
}, [appConfig, selectedBoss]);

When the user joins, the agent reads boss_type from their participant attributes and starts the matching boss.

The "How am I doing?" button sends a plain chat message using the useChat() hook:

1
const { send } = useChat();
2
await send('How am I doing?');

The LLM receives this as part of the conversation and calls the how_am_i_doing() function tool automatically.

TileLayout reads the mode attribute from the agent participant using useParticipantAttributes and switches the avatar border color: blue for roleplay, green for coaching. It uses useVoiceAssistant() to get agentVideoTrack — once LemonSlice connects and publishes its video, a <VideoTrack> renders the avatar in the browser.

Step 5: Run the app

Start the agent in one terminal:

1
cd agent
2
uv run lemonslice-agent.py dev

Start the frontend in another terminal:

1
cd frontend
2
pnpm dev

Open http://localhost:3000, select a boss personality, and click Start Session. The LemonSlice avatar will appear and begin the negotiation practice.

How it works

Here's the full session flow:

1. The user selects a boss on the welcome screen. The frontend passes boss_type as a participant attribute in the LiveKit JWT.
2. The agent reads boss_type on connect and initializes the matching boss agent with its voice ID and avatar image.
3. AvatarSession.start() connects LemonSlice to the room. LemonSlice publishes a video track; the frontend renders it.
4. The user speaks. LiveKit's inference API routes audio through Deepgram Nova 3 (STT), OpenAI gpt-4o-mini (LLM), and Cartesia Sonic 3 (TTS). LemonSlice receives the TTS audio and lip-syncs the avatar.
5. The user clicks "How am I doing?". A chat message is sent, the LLM calls how_am_i_doing(), and the agent's instructions switch to coaching mode. The agent publishes mode: coaching as a participant attribute and the frontend border turns green.
6. After giving feedback, the LLM calls return_to_roleplay(). Instructions reset, the attribute reverts to roleplay, and the border turns blue again.
7. When the 3-minute timer fires, the agent delivers a closing message and calls ctx.shutdown(), which tears down the room.

Summary

The key techniques in this app:

• Attaching a LemonSlice avatar with lemonslice.AvatarSession, started before session.start()
• Using update_instructions() inside function tools to switch agent behavior at runtime without restarting the session
• Broadcasting agent state to the frontend in real time using set_attributes() on the local participant
• Passing user choices as participant attributes in the LiveKit JWT so the agent can read them on connect

For more information, see:

• LemonSlice plugin docs for the full parameter reference
• Full source code including the complete Next.js frontend