Community Workshop | Small is beautiful — training small language models

This is a continuation of our earlier workshops on small language models (SLM, https://huggingface.co/blog/jjokah/small-language-model).
In this new workshop we go beyond using SLMs as they are to training and adapting them for specific purposes.
This hands-on workshop is for those with prior experience in SLM and Python, who would like to have a first experience creating and customizing SLM for specific needs.

The goals of this workshop are not only technical but much more importantly:

• To raise awareness on alternatives to online solutions.
• To give agency and choices back to users of AI through such alternatives.
• To live the Code of Conduct Democratic AI (https://demokratische-ki.de/).

Our rundown:

• Motivation and goals.
• Train a SLM on your laptop from scratch in Python.
• Supplement a SLM with organization-specifics after initial training using Python.
• Catch hallucinations as in large language models (LLM), and attempt to reduce them.
• Open exchange on how SLM helps in living the Code of Conduct Democratic AI.

Sylvi and Chris will be leading this workshop. Sylvi is an economist gone data scientist, currently serves on the Board of CorrelAid and is interested in establishing the code of conduct of democratic AI into CorrelAid’s structure. Chris is one of CorrelAid's Local Chapter Heads in Cologne, a career IT person with over 30 years experience working worldwide.

Please install:

• Git at https://git-scm.com/
• Thonny at https://thonny.org/
• LM Studio at https://lmstudio.ai/
• Model "mistralai/ministral-3-3b" using LM Studio

on your laptop before the workshop, and note the followings:

• This event will be conducted in English.
• This event follows the CorrelAid Code of Conduct (/en/coc).
• There will be NO recording of the session.
• AI bots for transcription are not allowed.
• Participants are limited to 15.