Slm Mini Chatbot

A lightweight chatbot application powered by Microsoft's Phi-2 small language model. This chatbot runs entirely on CPU and features a user-friendly Gradio web interface with optional context/memory loading.

🌟 Features

• Small Language Model: Uses Microsoft Phi-2 (2.7B parameters) - efficient and runs on CPU
• Web Interface: Clean and intuitive Gradio UI
• Context Loading: Upload text files to provide context/memory to the chatbot
• Adjustable Settings: Control response length and creativity (temperature)
• Conversation History: Maintains chat history for contextual responses
• CPU Compatible: No GPU required - runs on standard hardware

🚀 Quick Start

Prerequisites

• Python 3.8 or higher
• 8GB+ RAM recommended
• Internet connection (for first-time model download)

Installation

1. Clone the repository

git clone https://github.com/RasikhAli/SLM-Mini-Chatbot.git
cd SLM-Mini-Chatbot

1. Create a virtual environment (recommended)

python -m venv venv

# On Windows
venv\Scripts\activate

# On macOS/Linux
source venv/bin/activate

1. Install dependencies

pip install -r requirements.txt

Running the Chatbot

Simply run:

python chatbot.py

On first run, the model will be downloaded automatically (~5GB) and cached in the models/ directory within the project. This may take several minutes depending on your internet connection.

Once loaded, the Gradio interface will open in your browser at http://127.0.0.1:7860

📖 Usage

Basic Chat

1. Type your message in the text box
2. Click "Send" or press Enter
3. The chatbot will generate a response

Loading Context

1. Click "Browse" under "Context/Memory" section
2. Upload a .txt file containing context information
3. The chatbot will use this context when generating responses
4. Click "Clear Context" to remove the loaded context

Adjusting Settings

• Max Response Length: Control how long the responses can be (50-500 tokens)
• Temperature: Adjust creativity (0.1 = focused, 1.5 = creative)

🛠️ Technical Details

Model Information

• Model: Microsoft Phi-2
• Size: 2.7 billion parameters
• Source: Hugging Face Transformers
• Device: CPU (no GPU required)
• Precision: FP32 for CPU compatibility

Architecture

• Framework: PyTorch + Transformers
• UI: Gradio 4.0+
• Pipeline: Text generation with custom prompt formatting

File Structure

SLM-Mini-Chatbot/
├── chatbot.py              # Main application
├── requirements.txt        # Python dependencies
├── README.md              # This file
├── sample_context.txt     # Example context file
├── models/                # Model cache directory (auto-created)
└── venv/                  # Virtual environment (created during setup)

🎯 Use Cases

• Personal assistant for quick questions
• Learning tool for AI/ML concepts
• Prototype for chatbot applications
• Testing prompt engineering techniques
• Document Q&A with context loading

⚙️ Configuration

You can modify the model by editing chatbot.py:

# Change the model (line 238)
bot = MiniChatbot(model_name="microsoft/phi-2")

Alternative small models you can try:

• microsoft/phi-1_5 (1.3B - faster, less capable)
• TinyLlama/TinyLlama-1.1B-Chat-v1.0 (1.1B - very fast)
• stabilityai/stablelm-2-zephyr-1_6b (1.6B - good balance)

⚡ Performance Optimization

Speed Tips

The chatbot has been optimized for CPU performance, but response times will vary based on your hardware:

Expected Response Times (on typical CPU):

• Short responses (30-50 tokens): 10-20 seconds
• Medium responses (100 tokens): 30-60 seconds
• Long responses (200 tokens): 60-120 seconds

To Get Faster Responses:

1. Reduce Max Response Length: Set to 50-100 tokens instead of 200
2. Lower Temperature: Use 0.5-0.7 for more focused, faster responses
3. Keep Prompts Short: Shorter questions = faster answers
4. Limit Context: Only load essential context (first 200 chars are used)
5. Close Other Apps: Free up CPU resources

Optimizations Already Applied:

• ✅ Direct model.generate() instead of pipeline (faster)
• ✅ torch.no_grad() for inference (reduced memory)
• ✅ Multi-threading enabled (uses all CPU cores)
• ✅ Top-k sampling (faster than beam search)
• ✅ Reduced conversation history (last 2 exchanges only)
• ✅ Prompt truncation (max 512 tokens)

🔧 Troubleshooting

Model Download Issues

If the model fails to download, ensure you have:

• Stable internet connection
• At least 10GB free disk space
• Proper firewall/proxy settings

Memory Issues

If you encounter out-of-memory errors:

• Close other applications
• Try a smaller model (see Configuration section)
• Reduce max_length in the UI settings to 30-50

Slow Response Times

If responses are taking too long:

• Reduce max response length to 50-100 tokens (most important!)
• Lower temperature to 0.5
• Keep your questions concise
• First response after startup is always slower (model warmup)
• Consider upgrading to a faster CPU or using a smaller model

📝 Example Context File

Create a context.txt file with information you want the chatbot to know:

Company: TechCorp
Founded: 2020
Products: AI Solutions, Cloud Services
Mission: Making AI accessible to everyone

Upload this file to give the chatbot specific knowledge about your topic.

🤝 Contributing

This is a demonstration project. Feel free to fork and modify for your needs.

📄 License

This project is open source and available for educational and commercial use.

🔗 Links

• Repository: https://github.com/RasikhAli/SLM-Mini-Chatbot.git
• Phi-2 Model: https://huggingface.co/microsoft/phi-2
• Gradio Documentation: https://gradio.app/docs

👨‍💻 Author

Built as a demonstration of small language model deployment with user-friendly interfaces.

---

Note: First run will download ~5GB model files. Subsequent runs will use cached files and start much faster.