-
Siraj R Aizlewood authored
Saving Layer Configs as JSON or YAML config files by representing the LayerConfig object as a dict and then turning into JSON/YAML wasn't working. See issue here: https://github.com/aurelio-labs/semantic-router/issues/144 The issue was that, by attempting this, we were attempting to serialize all objects included in the Layer, including Routes, and the LLMs that those Routes use. In the case of the above issue, the LLM was a Cohere one, which included a Client as one of its attributes, and this Client is not serializable. So, instead of attempting to represent the entire LLM object a dict, to then be converted into JSON/YAML, we only keep key information about the LLM: 'module': self.llm.__module__, 'class': self.llm.__class__.__name__, 'model': self.llm.name This is what's saved in route.py, and then sent to layer.py to be serialized (in LayerConfig.to_file()). Then, when it comes time to load from the file via LayerConfig.from_file, the LLM is re-initialized dynamically.
Siraj R Aizlewood authoredSaving Layer Configs as JSON or YAML config files by representing the LayerConfig object as a dict and then turning into JSON/YAML wasn't working. See issue here: https://github.com/aurelio-labs/semantic-router/issues/144 The issue was that, by attempting this, we were attempting to serialize all objects included in the Layer, including Routes, and the LLMs that those Routes use. In the case of the above issue, the LLM was a Cohere one, which included a Client as one of its attributes, and this Client is not serializable. So, instead of attempting to represent the entire LLM object a dict, to then be converted into JSON/YAML, we only keep key information about the LLM: 'module': self.llm.__module__, 'class': self.llm.__class__.__name__, 'model': self.llm.name This is what's saved in route.py, and then sent to layer.py to be serialized (in LayerConfig.to_file()). Then, when it comes time to load from the file via LayerConfig.from_file, the LLM is re-initialized dynamically.
Code owners
Assign users and groups as approvers for specific file changes. Learn more.