Guardrails

https://github.com/NVIDIA/NeMo-Guardrails/blob/aa07d889e9437dc687cd1c0acf51678ad435516e/docs/architecture/README.md#the-guardrails-process

Guardrails Runtime

1. Generate canonical user message

2. Decide next step and execute

3. Generate bot utterance

Each of the above stages can involve one or more calls to the LLM.

Generate canonical user message

• generate canonical form for user utterance (captures 1. user intent 2. allows GR to trigger flows)

define flow generate user intent
  """Turn the raw user utterance into a canonical form."""

  event UtteranceUserActionFinished(final_transcript="...")
  execute generate_user_intent

Generate_user_intent action:

1. vector search on all canonical form examples in GR configs

2. Take top 5

3. Include them in prompt

4. Ask LLM to generate canonical form for current user utterance

5. new UserIntent even created

Decide Next Steps

potential next paths:

1. pre-defined flow already defined

2. LLM used to decide next step

   1. vector search performed for most relevant flows from GR config

   2. take top 5 flows

   3. include in prompt

   define flow generate next step
       """Generate the next step when there isn't any.
   
       We set the priority at 0.9 so it is lower than the default which is 1. So, if there
       is a flow that has a next step, it will have priority over this one.
       """
       priority 0.9
   
       user ...
       execute generate_next_step

This will happen for either step:

bot should say something (BotIntent events)

-> invokes `generate_bot_message` action (explained in next step)

Generate Bot Utterances

1. vector search for most relevant bot utterance examples in GR configs

   • also vector search on knowledge base if provided to include in prompt (retrieve_relevant_chunks action)

2. included in prompt

3. ask LLM to generate utterance for current bot intent

define extension flow generate bot message
  """Generate the bot utterance for a bot message.

  We always want to generate an utterance after a bot intent, hence the high priority.
  """
  priority 100

  bot ...
  execute retrieve_relevant_chunks
  execute generate_bot_message

Overall

- type: UtteranceUserActionFinished
  final_transcript: "how many unemployed people were there in March?"

# Stage 1: generate canonical form
- type: StartInternalSystemAction
  action_name: generate_user_intent

- type: InternalSystemActionFinished
  action_name: generate_user_intent
  status: success

- type: UserIntent
  intent: ask about headline numbers

# Stage 2: generate next step
- type: StartInternalSystemAction
  action_name: generate_next_step

- type: InternalSystemActionFinished
  action_name: generate_next_step
  status: success

- type: BotIntent
  intent: response about headline numbers

# Stage 3: generate bot utterance
- type: StartInternalSystemAction
  action_name: retrieve_relevant_chunks

- type: ContextUpdate
  data:
    relevant_chunks: "The number of persons not in the labor force who ..."

- type: InternalSystemActionFinished
  action_name: retrieve_relevant_chunks
  status: success

- type: StartInternalSystemAction
  action_name: generate_bot_message

- type: InternalSystemActionFinished
  action_name: generate_bot_message
  status: success

- type: StartInternalSystemAction
  content: "According to the US Bureau of Labor Statistics, there were 8.4 million unemployed people in March 2021."

- type: Listen