Skip to main content

Integration with Local Ollama Server for Custom LLM Models

Current Situation:
Pastai currently operates with
LLM (llama 3, 7b), limiting users to the default capabilities provided by the software.

Proposed Feature:
Implement support for connecting to a local Ollama server, allowing users to utilize custom large language models (LLMs) of their choice.

Rationale:
While Pastai may have been initially designed for users without extensive LLM knowledge, adding this feature would significantly enhance its versatility and appeal to a broader user base, including those with LLM expertise. This integration aligns with industry trends, as seen in applications like LM Studio and AnythingLLM.

1 comment

Log in to comment and vote

Comments1

  • Jeff@Pastai

    Team•

    Jul 3, 2024

    Embarrassingly - i attempted

    This on my Mac but the run times were very poor. It’s probably time I upgraddd my hardware