Skip to content

flow

Hooks to modify the Cat's flow of execution.

Here is a collection of methods to hook into the Cat execution pipeline.

after_cat_bootstrap(cat)

Hook into the end of the Cat start up.

Bootstrapping is the process of loading the plugins, the natural language objects (e.g. the LLM), the memories, the Agent Manager and the Rabbit Hole.

This hook allows to intercept the end of such process and is executed right after the Cat has finished loading its components.

This can be used to set or store variables to be shared further in the pipeline.

Parameters:

Name Type Description Default
cat CheshireCat

Cheshire Cat instance.

 required
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
@hook(priority=0)
def after_cat_bootstrap(cat) -> None:
    """Hook into the end of the Cat start up.

    Bootstrapping is the process of loading the plugins, the natural language objects (e.g. the LLM),
    the memories, the *Agent Manager* and the *Rabbit Hole*.

    This hook allows to intercept the end of such process and is executed right after the Cat has finished loading
    its components.

    This can be used to set or store variables to be shared further in the pipeline.

    Parameters
    ----------
    cat : CheshireCat
        Cheshire Cat instance.
    """
    pass # do nothing

after_cat_recalls_memories(cat)

Hook after semantic search in memories.

The hook is executed just after the Cat searches for the meaningful context in memories and stores it in the Working Memory.

Parameters:

Name Type Description Default
cat CheshireCat

Cheshire Cat instance.

 required
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
243
244
245
246
247
248
249
250
251
252
253
254
255
256
@hook(priority=0)
def after_cat_recalls_memories(cat) -> None:
    """Hook after semantic search in memories.

    The hook is executed just after the Cat searches for the meaningful context in memories
    and stores it in the *Working Memory*.

    Parameters
    ----------
    cat : CheshireCat
        Cheshire Cat instance.

    """
    pass # do nothing

before_cat_bootstrap(cat)

Hook into the Cat start up.

Bootstrapping is the process of loading the plugins, the natural language objects (e.g. the LLM), the memories, the Agent Manager and the Rabbit Hole.

This hook allows to intercept such process and is executed in the middle of plugins and natural language objects loading.

This hook can be used to set or store variables to be propagated to subsequent loaded objects.

Parameters:

Name Type Description Default
cat CheshireCat

Cheshire Cat instance.

 required
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@hook(priority=0)
def before_cat_bootstrap(cat) -> None:
    """Hook into the Cat start up.

    Bootstrapping is the process of loading the plugins, the natural language objects (e.g. the LLM),
    the memories, the *Agent Manager* and the *Rabbit Hole*.

    This hook allows to intercept such process and is executed in the middle of plugins and
    natural language objects loading.

    This hook can be used to set or store variables to be propagated to subsequent loaded objects.

    Parameters
    ----------
    cat : CheshireCat
        Cheshire Cat instance.
    """
    pass # do nothing

before_cat_reads_message(user_message_json, cat)

Hook the incoming user's JSON dictionary.

Allows to edit and enrich the incoming message received from the WebSocket connection.

For instance, this hook can be used to translate the user's message before feeding it to the Cat. Another use case is to add custom keys to the JSON dictionary.

The incoming message is a JSON dictionary with keys: { "text": message content }

Parameters:

Name Type Description Default
user_message_json dict

JSON dictionary with the message received from the chat.

 required
cat CheshireCat

Cheshire Cat instance.

 required

Returns:

Name Type Description
user_message_json dict

Edited JSON dictionary that will be fed to the Cat.
Notes

For example:

{
    "text": "Hello Cheshire Cat!",
    "custom_key": True
}

where "custom_key" is a newly added key to the dictionary to store any data.

Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
@hook(priority=0)
def before_cat_reads_message(user_message_json: dict, cat) -> dict:
    """Hook the incoming user's JSON dictionary.

    Allows to edit and enrich the incoming message received from the WebSocket connection.

    For instance, this hook can be used to translate the user's message before feeding it to the Cat.
    Another use case is to add custom keys to the JSON dictionary.

    The incoming message is a JSON dictionary with keys:
        {
            "text": message content
        }

    Parameters
    ----------
    user_message_json : dict
        JSON dictionary with the message received from the chat.
    cat : CheshireCat
        Cheshire Cat instance.


    Returns
    -------
    user_message_json : dict
        Edited JSON dictionary that will be fed to the Cat.

    Notes
    -----
    For example:

        {
            "text": "Hello Cheshire Cat!",
            "custom_key": True
        }

    where "custom_key" is a newly added key to the dictionary to store any data.

    """
    return user_message_json

before_cat_recalls_declarative_memories(declarative_recall_config, cat)

Hook into semantic search in memories.

Allows to intercept when the Cat queries the memories using the embedded user's input.

The hook is executed just before the Cat searches for the meaningful context in both memories and stores it in the Working Memory.

The hook return the values for maximum number (k) of items to retrieve from memory and the score threshold applied to the query in the vector memory (items with score under threshold are not retrieved) It also returns the embedded query (embedding) and the conditions on recall (metadata).

Parameters:

Name Type Description Default
declarative_recall_config dict

Dictionary with data needed to recall declarative memories

 required
cat CheshireCat

Cheshire Cat instance.

 required

Returns:

Name Type Description
declarative_recall_config dict

Edited dictionary that will be fed to the embedder.
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
@hook(priority=0)
def before_cat_recalls_declarative_memories(declarative_recall_config: dict, cat) -> dict:
    """Hook into semantic search in memories.

    Allows to intercept when the Cat queries the memories using the embedded user's input.

    The hook is executed just before the Cat searches for the meaningful context in both memories
    and stores it in the *Working Memory*.

    The hook return the values for maximum number (k) of items to retrieve from memory and the score threshold applied
    to the query in the vector memory (items with score under threshold are not retrieved)
    It also returns the embedded query (embedding) and the conditions on recall (metadata).

    Parameters
    ----------
    declarative_recall_config: dict
        Dictionary with data needed to recall declarative memories
    cat : CheshireCat
        Cheshire Cat instance.

    Returns
    -------
    declarative_recall_config: dict
        Edited dictionary that will be fed to the embedder.

    """
    return declarative_recall_config

before_cat_recalls_episodic_memories(episodic_recall_config, cat)

Hook into semantic search in memories.

Allows to intercept when the Cat queries the memories using the embedded user's input.

The hook is executed just before the Cat searches for the meaningful context in both memories and stores it in the Working Memory.

The hook return the values for maximum number (k) of items to retrieve from memory and the score threshold applied to the query in the vector memory (items with score under threshold are not retrieved). It also returns the embedded query (embedding) and the conditions on recall (metadata).

Parameters:

Name Type Description Default
episodic_recall_config dict

Dictionary with data needed to recall episodic memories

 required
cat CheshireCat

Cheshire Cat instance.

 required

Returns:

Name Type Description
episodic_recall_config dict

Edited dictionary that will be fed to the embedder.
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
@hook(priority=0)
def before_cat_recalls_episodic_memories(episodic_recall_config: dict, cat) -> dict:
    """Hook into semantic search in memories.

    Allows to intercept when the Cat queries the memories using the embedded user's input.

    The hook is executed just before the Cat searches for the meaningful context in both memories
    and stores it in the *Working Memory*.

    The hook return the values for maximum number (k) of items to retrieve from memory and the score threshold applied
    to the query in the vector memory (items with score under threshold are not retrieved).
    It also returns the embedded query (embedding) and the conditions on recall (metadata).

    Parameters
    ----------
    episodic_recall_config : dict
        Dictionary with data needed to recall episodic memories
    cat : CheshireCat
        Cheshire Cat instance.

    Returns
    -------
    episodic_recall_config: dict
        Edited dictionary that will be fed to the embedder.

    """
    return episodic_recall_config

before_cat_recalls_memories(cat)

Hook into semantic search in memories.

Allows to intercept when the Cat queries the memories using the embedded user's input.

The hook is executed just before the Cat searches for the meaningful context in both memories and stores it in the Working Memory.

Parameters:

Name Type Description Default
cat CheshireCat

Cheshire Cat instance.

 required
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
@hook(priority=0)
def before_cat_recalls_memories(cat) -> None:
    """Hook into semantic search in memories.

    Allows to intercept when the Cat queries the memories using the embedded user's input.

    The hook is executed just before the Cat searches for the meaningful context in both memories
    and stores it in the *Working Memory*.

    Parameters
    ----------
    cat : CheshireCat
        Cheshire Cat instance.

    """
    pass # do nothing

before_cat_recalls_procedural_memories(procedural_recall_config, cat)

Hook into semantic search in memories.

Allows to intercept when the Cat queries the memories using the embedded user's input.

The hook is executed just before the Cat searches for the meaningful context in both memories and stores it in the Working Memory.

The hook return the values for maximum number (k) of items to retrieve from memory and the score threshold applied to the query in the vector memory (items with score under threshold are not retrieved) It also returns the embedded query (embedding) and the conditions on recall (metadata).

Parameters:

Name Type Description Default
procedural_recall_config dict

Dictionary with data needed to recall tools from procedural memory

 required
cat CheshireCat

Cheshire Cat instance.

 required

Returns:

Name Type Description
procedural_recall_config dict

Edited dictionary that will be fed to the embedder.
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
@hook(priority=0)
def before_cat_recalls_procedural_memories(procedural_recall_config: dict, cat) -> dict:
    """Hook into semantic search in memories.

    Allows to intercept when the Cat queries the memories using the embedded user's input.

    The hook is executed just before the Cat searches for the meaningful context in both memories
    and stores it in the *Working Memory*.

    The hook return the values for maximum number (k) of items to retrieve from memory and the score threshold applied
    to the query in the vector memory (items with score under threshold are not retrieved)
    It also returns the embedded query (embedding) and the conditions on recall (metadata).

    Parameters
    ----------
    procedural_recall_config: dict
        Dictionary with data needed to recall tools from procedural memory
    cat : CheshireCat
        Cheshire Cat instance.

    Returns
    -------
    procedural_recall_config: dict
        Edited dictionary that will be fed to the embedder.

    """
    return procedural_recall_config

before_cat_sends_message(message, cat)

Hook the outgoing Cat's message.

Allows to edit the JSON dictionary that will be sent to the client via WebSocket connection.

This hook can be used to edit the message sent to the user or to add keys to the dictionary.

Parameters:

Name Type Description Default
message dict

JSON dictionary to be sent to the WebSocket client.

 required
cat CheshireCat

Cheshire Cat instance.

 required

Returns:

Name Type Description
message dict

Edited JSON dictionary with the Cat's answer.
Notes

Default message is::

    {
        "type": "chat",
        "content": cat_message["output"],
        "why": {
            "input": cat_message["input"],
            "output": cat_message["output"],
            "intermediate_steps": cat_message["intermediate_steps"],
            "memory": {
                "vectors": {
                    "episodic": episodic_report,
                    "declarative": declarative_report
                }
            },
        },
    }
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
@hook(priority=0)
def before_cat_sends_message(message: dict, cat) -> dict:
    """Hook the outgoing Cat's message.

    Allows to edit the JSON dictionary that will be sent to the client via WebSocket connection.

    This hook can be used to edit the message sent to the user or to add keys to the dictionary.

    Parameters
    ----------
    message : dict
        JSON dictionary to be sent to the WebSocket client.
    cat : CheshireCat
        Cheshire Cat instance.

    Returns
    -------
    message : dict
        Edited JSON dictionary with the Cat's answer.

    Notes
    -----
    Default `message` is::

            {
                "type": "chat",
                "content": cat_message["output"],
                "why": {
                    "input": cat_message["input"],
                    "output": cat_message["output"],
                    "intermediate_steps": cat_message["intermediate_steps"],
                    "memory": {
                        "vectors": {
                            "episodic": episodic_report,
                            "declarative": declarative_report
                        }
                    },
                },
            }

    """

    return message

before_cat_stores_episodic_memory(doc, cat)

Hook the user message Document before is inserted in the vector memory.

Allows editing and enhancing a single Document before the Cat add it to the episodic vector memory.

Parameters:

Name Type Description Default
doc Document

Langchain Document to be inserted in memory.

 required
cat CheshireCat

Cheshire Cat instance.

 required

Returns:

Name Type Description
doc Document

Langchain Document that is added in the episodic vector memory.
Notes

The Document has two properties::

`page_content`: the string with the text to save in memory;
`metadata`: a dictionary with at least two keys:
    `source`: where the text comes from;
    `when`: timestamp to track when it's been uploaded.
Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
@hook(priority=0)
def before_cat_stores_episodic_memory(doc: Document, cat) -> Document:
    """Hook the user message `Document` before is inserted in the vector memory.

    Allows editing and enhancing a single `Document` before the Cat add it to the episodic vector memory.

    Parameters
    ----------
    doc : Document
        Langchain `Document` to be inserted in memory.
    cat : CheshireCat
        Cheshire Cat instance.

    Returns
    -------
    doc : Document
        Langchain `Document` that is added in the episodic vector memory.

    Notes
    -----
    The `Document` has two properties::

        `page_content`: the string with the text to save in memory;
        `metadata`: a dictionary with at least two keys:
            `source`: where the text comes from;
            `when`: timestamp to track when it's been uploaded.

    """
    return doc

cat_recall_query(user_message, cat)

Hook the semantic search query.

This hook allows to edit the user's message used as a query for context retrieval from memories. As a result, the retrieved context can be conditioned editing the user's message.

Parameters:

Name Type Description Default
user_message str

String with the text received from the user.

 required
cat CheshireCat

Cheshire Cat instance to exploit the Cat's methods.

 required

Returns:

Type Description
Edited string to be used for context retrieval in memory. The returned string is further stored in the
Working Memory at `cat.working_memory.recall_query`.
Notes

For example, this hook is a suitable to perform Hypothetical Document Embedding (HyDE). HyDE [1]_ strategy exploits the user's message to generate a hypothetical answer. This is then used to recall the relevant context from the memory. An official plugin is available to test this technique.

References

[1] Gao, L., Ma, X., Lin, J., & Callan, J. (2022). Precise Zero-Shot Dense Retrieval without Relevance Labels. arXiv preprint arXiv:2212.10496.

Source code in cat/mad_hatter/core_plugin/hooks/flow.py 
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
@hook(priority=0)
def cat_recall_query(user_message: str, cat) -> str:
    """Hook the semantic search query.

    This hook allows to edit the user's message used as a query for context retrieval from memories.
    As a result, the retrieved context can be conditioned editing the user's message.

    Parameters
    ----------
    user_message : str
        String with the text received from the user.
    cat : CheshireCat
        Cheshire Cat instance to exploit the Cat's methods.

    Returns
    -------
    Edited string to be used for context retrieval in memory. The returned string is further stored in the
    Working Memory at `cat.working_memory.recall_query`.

    Notes
    -----
    For example, this hook is a suitable to perform Hypothetical Document Embedding (HyDE).
    HyDE [1]_ strategy exploits the user's message to generate a hypothetical answer. This is then used to recall
    the relevant context from the memory.
    An official plugin is available to test this technique.

    References
    ----------
    [1] Gao, L., Ma, X., Lin, J., & Callan, J. (2022). Precise Zero-Shot Dense Retrieval without Relevance Labels.
       arXiv preprint arXiv:2212.10496.

    """

    # here we just return the latest user message as is
    return user_message