Open-source LLM
Open-source LLM - open language models (Llama 3, Mistral, Qwen) for launching without dependence on the API.
Open-source LLM - language models with open weights that can be downloaded and run locally. Main players: Meta Llama 3 (70B - a really strong model), Mistral (French company, very compact and fast models), Qwen from Alibaba (excellent Chinese + English), DeepSeek (competitive with GPT-4 when launched locally).
To run locally, I use Ollama - it simplifies the process as much as possible: one command in the terminal, and any model from the catalog works as a local API. On MacBook Pro M2 Llama 3 8B responds in 2-3 seconds. The quality is sufficient for the tasks of summarization, classification, and generation of primary drafts.
The main reason to choose open-source in marketing projects is not ideology, but pragmatics: data confidentiality (no API calls to external servers), cost (no fees for tokens), manageability (you fix the version of the model, it does not change). For tasks involving personal customer data, this is often the only reasonable choice.
Related terms
Need to set this up on your project?
I analyze metrics, calculate unit economics and collect funnels on real budgets. 30 minutes on call - free.