Skip to content
Snippets Groups Projects
Unverified Commit 8d2de77e authored by Siraj R Aizlewood's avatar Siraj R Aizlewood
Browse files

Bug Fix Saving JSON/YAML of Layer Config

Saving Layer Configs as JSON or YAML config files by representing the LayerConfig object as a dict and then turning into JSON/YAML wasn't working.

See issue here: https://github.com/aurelio-labs/semantic-router/issues/144

The issue was that, by attempting this, we were attempting to serialize all objects included in the Layer, including Routes, and the LLMs that those Routes use.

In the case of the above issue, the LLM was a Cohere one, which included a Client as one of its attributes, and this Client is not serializable.

So, instead of attempting to represent the entire LLM object a dict, to then be converted into JSON/YAML, we only keep key information about the LLM:

'module': self.llm.__module__,
'class': self.llm.__class__.__name__,
'model': self.llm.name

This is what's saved in route.py, and then sent to layer.py to be serialized (in LayerConfig.to_file()).

Then, when it comes time to load from the file via LayerConfig.from_file, the LLM is re-initialized dynamically.
parent 8d7579f8
No related branches found
No related tags found
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment