Note: This post was composed with Mistral 7B by using LM Studio and edited by a human.
In the ever-evolving world of language models (LLMs), Shawn, the chief software developer at Knight Technologies LLC, has been conducting fascinating experiments in world of LLM and NLP using LlamaSharp. This open-source package allows developers to work with various LLM’s, including OpenAI’s GPT-3, along with offering integration with Microsoft’s semantic kernel. In this blog post, we will delve into Shawn’s journey as he navigates the complexities of leveraging these models and shares his insights on creating a stable version of SwankBOT for our upcoming Garden Guru project.
LlamaSharp: A Convenient Library for LLM Experimentation
LlamaSharp, an open-source C# wrapper around LLama.cpp, has proven to be a convenient tool for Shawn’s experiments with LLMs. This versatile library allows developers to work with multiple models and use them according to their specific needs. By using LlamaSharp, Shawn was able to explore the capabilities of various models from HuggingFace and gain valuable insights into how they can be applied to real-world scenarios.
Microsoft’s Semantic Kernel: A Bleeding Edge Option with OpenAI Leanings
During his experiments, Shawn also dabbled in Microsoft’s semantic kernel, which will work with other LLM’s but is bleeding edge and has a slant towards OpenAI considering Microsoft’s investment in it. While this presented some limitations, it did provide a unique opportunity for Shawn to delve deeper into the intricacies of LLMs and their potential applications. However, as OpenAI’s services come with a cost, Shawn had to find alternative solutions that would support semantic kernel functions without incurring additional expenses. Although this approach seemed encouraging, Shawn quickly found that the features that semantic kernel offered require a participating AI service to act on them. An attempt to use kernel memory also proved to be frustrating as the “memories” never finished loading.
Introducing LM Studio: A Powerful Platform for LLM Experimentation
After exploring various options, Shawn turned his attention to LM Studio, an innovative platform that can host various LLMs. This powerful tool allowed Shawn to further his experiments with attempting to create a stable version of SwankBOT. By leveraging the capabilities of LM Studio, Shawn was able to overcome the challenges posed by Microsoft’s semantic kernel and continue his experiments with LLMs by continuing running his experiments locally without incurring a charge for every conversation. Thus far this approach is great for experimenting but is not at all portable and not suited to run within an application.
The Transcriber Solution: Capturing Thoughts in MongoDB
It is worth noting that one of Shawn’s experiments for capturing data from a hand-written journal was a complete success. In an effort to streamline the data collection process, he developed a transcriber solution that utilized Microsoft’s Cognitive Services. This innovative tool captured transcribed material as “thoughts” in a MongoDB database, allowing for easy access and analysis. However, attempting to access this information without passing in a prompt has proven to be problematic. Many great things came from this experiment which included several private NuGet packages such as a MongoDB API to help encapsulate features required by the projects using this particular datastore paradigm.
Anything LLM: A Versatile Tool for Chat-Based Interactions
To effectively mine the data captured by the transcriber solution, Shawn turned to Anything LLM, which is a platform that can read documents from the file system and store them in a vector database. This versatile tool allowed Shawn to “chat” with the data stored in the vector database and retrieve relevant information on demand. By leveraging the capabilities of Anything LLM, Shawn was able to create a more efficient and user-friendly system for accessing and analyzing data which is helping to shape his future approaches to handling LLM concerns for SwankBOT.
The Road Ahead: Developing a Stable Version of SwankBOT for Garden Guru
As Shawn continues his journey exploring the world of LLMs, he remains focused on developing a stable version of SwankBOT for our upcoming Garden Guru project. By leveraging the insights gained from his experiments with LlamaSharp, LM Studio, and Anything LLM, Shawn is well-positioned to create a powerful tool that will revolutionize the way we interact with gardening information.
In conclusion, Shawn’s experiments with various LLMs have provided valuable insights into the potential applications of these cutting-edge technologies. As we continue to refine and improve our tools, we are confident that SwankBOT will become an invaluable resource for gardeners everywhere. Stay tuned for more updates as we progress towards the launch of Garden Guru!
