From ce1532644816189d033aa13260e62b4489ee5870 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 30 Oct 2025 16:07:59 +0000 Subject: [PATCH 1/2] Add automated Ubuntu/Debian installation script with documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This commit adds a comprehensive, automated installation system for Ubuntu/Debian users: ## New Files: - install-ubuntu.sh: Fully automated installation script with: * Interactive installation type selection (Docker/Local/Both) * Automatic system requirement checks (RAM, disk space, GPU) * Intelligent dependency installation * GPU/CUDA support detection and configuration * Comprehensive error handling and logging * Progress indicators and colored output * Convenience startup/stop scripts generation - INSTALLATION-DEUTSCH.md: German quick-start guide with: * Simple step-by-step instructions * Common troubleshooting solutions * Performance optimization tips * Training monitoring guidance - docs/INSTALLATION-UBUNTU.md: Detailed English documentation with: * Complete installation guide * System requirements breakdown * Extensive troubleshooting section * Performance tuning recommendations * Useful command reference ## Modified Files: - README.md: Added reference to automated installation option ## Features: ✅ One-command installation: curl | bash ✅ Detects Ubuntu/Debian automatically ✅ Checks system resources (RAM, disk, GPU) ✅ Installs all required dependencies ✅ Offers Docker or local installation ✅ Sets up NVIDIA Container Toolkit for GPU support ✅ Creates convenience scripts (start/stop) ✅ Optimizes for low-memory systems ✅ Comprehensive logging to install-ubuntu.log ✅ Beautiful, colored terminal output This solves common installation pain points reported by users, especially around training integration and dependency management. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- INSTALLATION-DEUTSCH.md | 300 +++++++++++++++++++ README.md | 18 ++ docs/INSTALLATION-UBUNTU.md | 362 ++++++++++++++++++++++ install-ubuntu.sh | 579 ++++++++++++++++++++++++++++++++++++ 4 files changed, 1259 insertions(+) create mode 100644 INSTALLATION-DEUTSCH.md create mode 100644 docs/INSTALLATION-UBUNTU.md create mode 100755 install-ubuntu.sh diff --git a/INSTALLATION-DEUTSCH.md b/INSTALLATION-DEUTSCH.md new file mode 100644 index 00000000..90383242 --- /dev/null +++ b/INSTALLATION-DEUTSCH.md @@ -0,0 +1,300 @@ +# 🇩🇪 Second-Me Installation für Ubuntu/Debian (Deutsch) + +## ⚡ Schnellinstallation (Empfohlen) + +### Eine Zeile - Fertig! + +```bash +curl -fsSL https://raw.githubusercontent.com/mindverse/Second-Me/master/install-ubuntu.sh | bash +``` + +### Oder Repository zuerst klonen: + +```bash +git clone https://github.com/mindverse/Second-Me.git +cd Second-Me +chmod +x install-ubuntu.sh +./install-ubuntu.sh +``` + +Das Script führt Sie durch eine **vollautomatische Installation** mit interaktiven Fragen. + +--- + +## 📋 Was macht das Installations-Script? + +### ✨ Automatische Erkennung: +- ✅ Ubuntu/Debian Version +- ✅ RAM und Festplatten-Platz +- ✅ NVIDIA GPU (für CUDA-Beschleunigung) +- ✅ Bestehende Software (Docker, Python, Node.js) + +### 🎯 Interaktive Installation: + +Das Script fragt Sie: + +**1. Welche Installation möchten Sie?** +- **Docker** (Empfohlen - Am einfachsten) +- **Lokal** (Bessere Performance) +- **Beides** (Maximale Flexibilität) + +**2. GPU-Support aktivieren?** (Falls NVIDIA GPU vorhanden) + +### 🔧 Was wird installiert: + +#### Docker-Installation: +- Docker Engine & Docker Compose +- NVIDIA Container Toolkit (bei GPU) +- Alle Second-Me Container + +#### Lokale Installation: +- Python 3.12+ +- Node.js 20.x LTS +- Poetry (Python Package Manager) +- CMake & Build-Tools +- Alle Second-Me Abhängigkeiten + +--- + +## 💻 System-Anforderungen + +### Minimum: +- **OS:** Ubuntu 20.04+ oder Debian 11+ +- **RAM:** 8 GB +- **Disk:** 20 GB frei +- **CPU:** 4 Kerne + +### Empfohlen: +- **RAM:** 16 GB +- **Disk:** 50 GB frei +- **GPU:** NVIDIA mit 6+ GB VRAM (optional) +- **CPU:** 8 Kerne + +--- + +## 🚀 Nach der Installation + +### Starten: +```bash +./start-second-me.sh +``` + +### Zugriff: +- **Web-Interface:** http://localhost:3000 +- **Backend API:** http://localhost:8002 + +### Stoppen: +```bash +./stop-second-me.sh +``` + +--- + +## 🔍 Training überwachen + +Das Training kann **1-6 Stunden** dauern, je nach Datenmenge und Hardware. + +### Fortschritt anzeigen: + +**Docker:** +```bash +docker logs -f second-me-backend +``` + +**Lokal:** +```bash +tail -f logs/train/train.log +``` + +Sie sollten Ausgaben wie diese sehen: +``` +Training progress: 15% (36/240) +Training progress: 25% (60/240) +... +``` + +--- + +## 🛠️ Häufige Probleme & Lösungen + +### Problem: "Training hängt fest" + +**Ursache:** Der Prozess läuft wahrscheinlich noch, braucht aber Zeit. + +**Lösung:** +1. **Log-Datei prüfen** (siehe oben) +2. Wenn wirklich hängt: Mit kleinen Daten starten +3. RAM-Optimierung in `.env`: + ```bash + DATA_SYNTHESIS_MODE=low + CONCURRENCY_THREADS=1 + ``` + +### Problem: "Permission denied" bei Docker + +**Lösung:** +```bash +newgrp docker +# oder ausloggen und wieder einloggen +``` + +### Problem: "Port 3000 or 8002 already in use" + +**Lösung:** +```bash +# Prüfen welcher Prozess den Port verwendet +sudo lsof -i :3000 +sudo lsof -i :8002 + +# Prozess beenden oder andere Ports konfigurieren +``` + +### Problem: Nicht genug RAM (nur 8 GB) + +**Lösung:** +Bearbeiten Sie die `.env` Datei: +```bash +# Öffnen +nano .env + +# Hinzufügen/Ändern: +DATA_SYNTHESIS_MODE=low +CONCURRENCY_THREADS=1 + +# Speichern: Ctrl+O, Enter, Ctrl+X +``` + +Dann neu starten: +```bash +./stop-second-me.sh +./start-second-me.sh +``` + +--- + +## 💡 Performance-Tipps + +### Bei 8 GB RAM: +- ✓ `DATA_SYNTHESIS_MODE=low` setzen +- ✓ Mit 1-2 kleinen Dokumenten starten +- ✓ Kleineres Modell wählen (0.5B oder 1.5B) + +### Bei 16+ GB RAM: +- ✓ `DATA_SYNTHESIS_MODE=medium` +- ✓ Mittlere Modelle möglich (2B-3B) + +### Bei 32+ GB RAM + GPU: +- ✓ `DATA_SYNTHESIS_MODE=high` +- ✓ Große Modelle möglich (7B+) +- ✓ GPU-Beschleunigung aktivieren + +--- + +## 📚 Nützliche Befehle + +```bash +# Alle Befehle anzeigen +make help + +# Status prüfen +make status + +# Neu starten +make restart + +# GPU-Support prüfen (Docker) +make docker-check-cuda + +# Logs anzeigen +docker logs -f second-me-backend # Docker +tail -f logs/train/train.log # Lokal +``` + +--- + +## 📖 Dokumentation & Hilfe + +- **Homepage:** https://home.second.me/ +- **Ausführliche Anleitung (Englisch):** [docs/INSTALLATION-UBUNTU.md](docs/INSTALLATION-UBUNTU.md) +- **User Tutorial:** https://secondme.gitbook.io/secondme/getting-started +- **FAQ:** https://secondme.gitbook.io/secondme/faq +- **Discord Community:** https://discord.gg/GpWHQNUwrg +- **GitHub Issues:** https://github.com/mindverse/Second-Me/issues + +--- + +## 🎯 Schritt-für-Schritt Guide + +### 1. Installation starten +```bash +./install-ubuntu.sh +``` + +### 2. Fragen beantworten +- Installationstyp wählen: **1** (Docker - Empfohlen) +- GPU aktivieren: **Y** (wenn vorhanden) + +### 3. Warten +- Installation dauert ca. 20-45 Minuten +- Kaffee holen ☕ + +### 4. Starten +```bash +./start-second-me.sh +``` + +### 5. Browser öffnen +- Gehen Sie zu: http://localhost:3000 + +### 6. Erste Schritte +1. ✓ Account erstellen / Einloggen +2. ✓ Profil einrichten +3. ✓ **Kleine** Dokumente hochladen (1-2 PDFs, je <10 Seiten) +4. ✓ Training starten +5. ✓ Logs überwachen (siehe oben) +6. ✓ Nach 1-2 Stunden: Ihr AI-Selbst ist bereit! + +### 7. Mehr Daten hinzufügen +Nach erfolgreichem ersten Training: +- Laden Sie mehr Dokumente hoch +- Starten Sie erneut das Training +- Ihr AI-Selbst wird besser und persönlicher + +--- + +## ✅ Installations-Checkliste + +- [ ] System-Anforderungen geprüft +- [ ] Repository geklont oder Script heruntergeladen +- [ ] `./install-ubuntu.sh` ausgeführt +- [ ] Installation erfolgreich +- [ ] `./start-second-me.sh` ausgeführt +- [ ] http://localhost:3000 erreichbar +- [ ] Profil erstellt +- [ ] Erste Dokumente hochgeladen +- [ ] Training gestartet +- [ ] Logs zeigen Fortschritt + +--- + +## 🎉 Fertig! + +Willkommen bei Second-Me! Sie haben erfolgreich Ihr eigenes AI-System installiert. + +**Nächste Schritte:** +1. Erkunden Sie die Web-Oberfläche +2. Laden Sie Ihre Dokumente hoch +3. Trainieren Sie Ihr AI-Selbst +4. Testen Sie die Features (Roleplay, AI Space) +5. Teilen Sie Ihre Erfahrungen in der Community! + +**Viel Spaß mit Second-Me!** 🚀 + +--- + +## 🐛 Probleme? + +1. **Prüfen Sie:** `install-ubuntu.log` +2. **Lesen Sie:** [docs/INSTALLATION-UBUNTU.md](docs/INSTALLATION-UBUNTU.md) (Englisch) +3. **Fragen Sie:** [Discord Community](https://discord.gg/GpWHQNUwrg) +4. **Melden Sie:** [GitHub Issues](https://github.com/mindverse/Second-Me/issues) diff --git a/README.md b/README.md index 49668eed..cc1011f2 100644 --- a/README.md +++ b/README.md @@ -81,6 +81,24 @@ Star and join us, and you will receive all release notifications from GitHub wit ### ⚡ Get your Second Me running in just 3 steps: +#### Option 1: Automated Ubuntu/Debian Installation (Recommended for Linux) + +```bash +# One-command automated installation +curl -fsSL https://raw.githubusercontent.com/mindverse/Second-Me/master/install-ubuntu.sh | bash +``` + +This script provides: +- ✅ Interactive installation (Docker or Local setup) +- ✅ Automatic dependency installation +- ✅ GPU detection and setup (NVIDIA CUDA) +- ✅ System optimization for your hardware +- ✅ Comprehensive error handling + +📖 **[Deutsche Anleitung](INSTALLATION-DEUTSCH.md)** | **[Detailed Guide](docs/INSTALLATION-UBUNTU.md)** + +#### Option 2: Manual Docker Installation (All Platforms) + ```bash # 1. Clone the repository git clone https://github.com/mindverse/Second-Me.git diff --git a/docs/INSTALLATION-UBUNTU.md b/docs/INSTALLATION-UBUNTU.md new file mode 100644 index 00000000..2fe241c4 --- /dev/null +++ b/docs/INSTALLATION-UBUNTU.md @@ -0,0 +1,362 @@ +# Second-Me Installation Guide für Ubuntu/Debian + +Eine vollautomatische, benutzerfreundliche Installation für Ubuntu und Debian Linux-Systeme. + +## 🚀 Schnellstart (1-Befehl-Installation) + +```bash +curl -fsSL https://raw.githubusercontent.com/mindverse/Second-Me/master/install-ubuntu.sh | bash +``` + +**Oder** wenn Sie das Repository bereits geklont haben: + +```bash +cd Second-Me +chmod +x install-ubuntu.sh +./install-ubuntu.sh +``` + +## 📋 Was das Script macht + +Das Installations-Script führt automatisch folgende Schritte durch: + +### 1. **System-Prüfung** +- ✅ Betriebssystem-Erkennung (Ubuntu/Debian) +- ✅ RAM-Prüfung (Minimum 8 GB empfohlen) +- ✅ Festplatten-Platz (Minimum 20 GB empfohlen) +- ✅ GPU-Erkennung (NVIDIA CUDA) + +### 2. **Interaktive Installation** + +Das Script fragt Sie nach Ihrer bevorzugten Installationsmethode: + +**Option 1: Docker Installation (Empfohlen)** +- ✓ Einfachste Installation +- ✓ Alle Abhängigkeiten vorinstalliert +- ✓ Isolierte Umgebung +- ✓ Automatische GPU-Erkennung + +**Option 2: Lokale Installation** +- ✓ Bessere Performance +- ✓ Mehr Kontrolle +- ✓ Direkter System-Zugriff + +**Option 3: Beides** +- ✓ Maximale Flexibilität + +### 3. **Automatische Abhängigkeits-Installation** + +#### Für Docker: +- Docker Engine +- Docker Compose +- NVIDIA Container Toolkit (wenn GPU vorhanden) + +#### Für Lokale Installation: +- Python 3.12+ +- Node.js 20.x LTS +- Poetry (Python Dependency Management) +- CMake +- SQLite3 +- Build-Tools + +### 4. **Projekt-Setup** +- Repository-Konfiguration +- Abhängigkeiten installieren +- Umgebungsvariablen einrichten +- Optimierungen für Ihr System + +### 5. **Convenience Scripts** +- `start-second-me.sh` - Startet Second-Me +- `stop-second-me.sh` - Stoppt Second-Me + +## 💻 System-Anforderungen + +### Minimum (CPU-Only): +- **OS:** Ubuntu 20.04+ oder Debian 11+ +- **RAM:** 8 GB (mit "low" Datensynthese-Modus) +- **Disk:** 20 GB freier Speicher +- **CPU:** 4 Kerne empfohlen + +### Empfohlen: +- **RAM:** 16 GB+ +- **Disk:** 50 GB+ freier Speicher +- **GPU:** NVIDIA GPU mit 6+ GB VRAM (optional, aber empfohlen) +- **CPU:** 8+ Kerne + +### Optimal (für größere Modelle): +- **RAM:** 32 GB+ +- **GPU:** NVIDIA GPU mit 12+ GB VRAM +- **Disk:** 100 GB+ NVMe SSD + +## 📖 Detaillierte Installationsanleitung + +### Schritt 1: Repository klonen (optional) + +Wenn Sie das Script nicht direkt von GitHub ausführen möchten: + +```bash +git clone https://github.com/mindverse/Second-Me.git +cd Second-Me +``` + +### Schritt 2: Installations-Script ausführen + +```bash +chmod +x install-ubuntu.sh +./install-ubuntu.sh +``` + +### Schritt 3: Interaktive Fragen beantworten + +Das Script wird Sie fragen: + +1. **Installationstyp wählen** (Docker/Lokal/Beides) +2. **GPU-Support aktivieren?** (Nur wenn NVIDIA GPU erkannt wurde) +3. Das Script führt dann alle erforderlichen Installationen durch + +### Schritt 4: Nach der Installation + +#### Docker-Installation: +```bash +# Starten +./start-second-me.sh +# oder +make docker-up + +# Zugriff +# Web-Interface: http://localhost:3000 +# Backend API: http://localhost:8002 + +# Stoppen +./stop-second-me.sh +# oder +make docker-down +``` + +#### Lokale Installation: +```bash +# Starten +./start-second-me.sh +# oder +make start + +# Zugriff +# Web-Interface: http://localhost:3000 +# Backend API: http://localhost:8002 + +# Stoppen +./stop-second-me.sh +# oder +make stop +``` + +## 🔧 Fehlerbehebung + +### Problem: "Permission denied" bei Docker + +**Lösung:** +```bash +# Nach Docker-Installation: +newgrp docker +# oder ausloggen und wieder einloggen +``` + +### Problem: "Port already in use" + +**Lösung:** +```bash +# Prüfen, welcher Prozess Port 3000/8002 verwendet +sudo lsof -i :3000 +sudo lsof -i :8002 + +# Prozess beenden oder anderen Port in .env konfigurieren +``` + +### Problem: Installation schlägt fehl + +**Lösung:** +```bash +# Log-Datei überprüfen +cat install-ubuntu.log + +# Manuelle Installation versuchen +make setup +``` + +### Problem: Training bricht ab oder wirkt "hängengeblieben" + +**Lösungen:** + +1. **Log-Datei überwachen:** +```bash +# Docker +docker logs -f second-me-backend + +# Lokal +tail -f logs/train/train.log +``` + +2. **Speicher-Optimierung (bei wenig RAM):** + +Bearbeiten Sie `.env`: +```bash +DATA_SYNTHESIS_MODE=low +CONCURRENCY_THREADS=1 +``` + +3. **Starten Sie mit kleinen Daten:** +- Laden Sie zuerst nur 1-2 kleine Dokumente (< 10 Seiten) hoch +- Lassen Sie den kompletten Training-Prozess durchlaufen +- Fügen Sie dann mehr Daten hinzu + +### Problem: GPU wird nicht erkannt (Docker) + +**Lösung:** +```bash +# NVIDIA Container Toolkit installieren +distribution=$(. /etc/os-release;echo $ID$VERSION_ID) +curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg +curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \ + sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ + sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list + +sudo apt-get update +sudo apt-get install -y nvidia-container-toolkit +sudo systemctl restart docker + +# GPU-Modus aktivieren +make docker-use-gpu +make docker-up +``` + +## 🎯 Performance-Tipps + +### Für 8 GB RAM Systeme: +```bash +# In .env setzen: +DATA_SYNTHESIS_MODE=low +CONCURRENCY_THREADS=1 + +# Kleines Modell wählen (0.5B oder 1.5B) +# In der Web-UI beim Training +``` + +### Für 16+ GB RAM Systeme: +```bash +DATA_SYNTHESIS_MODE=medium +CONCURRENCY_THREADS=2 +``` + +### Für 32+ GB RAM Systeme mit GPU: +```bash +DATA_SYNTHESIS_MODE=high +CONCURRENCY_THREADS=4 +``` + +## 📝 Nützliche Befehle + +```bash +# Alle verfügbaren Befehle anzeigen +make help + +# Service-Status prüfen +make status + +# Services neustarten +make restart + +# Docker-Container prüfen +docker ps + +# GPU-Support in Docker prüfen +make docker-check-cuda + +# Logs anzeigen (Docker) +docker logs -f second-me-backend +docker logs -f second-me-frontend + +# Logs anzeigen (Lokal) +tail -f logs/train/train.log +tail -f logs/app.log +``` + +## 🔄 Updates + +Um Second-Me zu aktualisieren: + +```bash +# Repository aktualisieren +git pull origin master + +# Docker: Container neu bauen +make docker-down +make docker-build +make docker-up + +# Lokal: Abhängigkeiten aktualisieren +make setup +make restart +``` + +## 🆘 Hilfe bekommen + +Wenn Sie auf Probleme stoßen: + +1. **Prüfen Sie die Log-Dateien:** + - Installation: `install-ubuntu.log` + - Docker Backend: `docker logs second-me-backend` + - Lokales Training: `logs/train/train.log` + +2. **Suchen Sie in der FAQ:** + - https://secondme.gitbook.io/secondme/faq + +3. **Fragen Sie in der Community:** + - Discord: https://discord.gg/GpWHQNUwrg + - GitHub Issues: https://github.com/mindverse/Second-Me/issues + +4. **Dokumentation:** + - GitBook: https://secondme.gitbook.io/secondme/ + - Homepage: https://home.second.me/ + +## 📊 Installations-Zeitplan + +Ungefähre Installationszeiten: + +| Schritt | Docker | Lokal | +|---------|--------|-------| +| System-Abhängigkeiten | 5-10 Min | 5-10 Min | +| Docker Installation | 5 Min | - | +| Python/Node.js | - | 10 Min | +| Image Build / Setup | 10-30 Min | 15-30 Min | +| **Gesamt** | **20-45 Min** | **30-50 Min** | + +*Zeiten abhängig von Internet-Geschwindigkeit und System-Performance* + +## ✅ Installations-Checkliste + +- [ ] System-Anforderungen geprüft (8+ GB RAM, 20+ GB Disk) +- [ ] Installations-Script heruntergeladen/geklont +- [ ] Script ausgeführt (`./install-ubuntu.sh`) +- [ ] Installationstyp gewählt (Docker/Lokal) +- [ ] Installation erfolgreich abgeschlossen +- [ ] Second-Me gestartet (`./start-second-me.sh`) +- [ ] Web-Interface erreichbar (http://localhost:3000) +- [ ] Erste Dokumente hochgeladen +- [ ] Training-Prozess gestartet und überwacht + +## 🎉 Nach der Installation + +Herzlichen Glückwunsch! Second-Me ist jetzt installiert. + +### Nächste Schritte: + +1. **Öffnen Sie die Web-Oberfläche:** http://localhost:3000 +2. **Folgen Sie dem User Tutorial:** https://secondme.gitbook.io/secondme/getting-started +3. **Laden Sie Ihre ersten Dokumente hoch** +4. **Starten Sie das Training Ihres AI-Selbst** +5. **Erkunden Sie die Features:** + - Roleplay Mode + - AI Space (Network Features) + - Memory Management + +**Viel Erfolg mit Second-Me!** 🚀 diff --git a/install-ubuntu.sh b/install-ubuntu.sh new file mode 100755 index 00000000..9caca893 --- /dev/null +++ b/install-ubuntu.sh @@ -0,0 +1,579 @@ +#!/bin/bash + +################################################################################ +# Second-Me Automated Installation Script for Ubuntu/Debian +# Version: 1.0.0 +# Description: Automated installation with Docker and local setup options +################################################################################ + +set -e # Exit on error + +# Color definitions for output +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' # No Color +BOLD='\033[1m' + +# Installation log file +INSTALL_LOG="install-ubuntu.log" +exec > >(tee -a "$INSTALL_LOG") +exec 2>&1 + +################################################################################ +# Helper Functions +################################################################################ + +print_header() { + echo -e "\n${CYAN}╔══════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${CYAN}║${BOLD} Second-Me Automated Installer (Ubuntu) ${NC}${CYAN}║${NC}" + echo -e "${CYAN}╚══════════════════════════════════════════════════════════════════╝${NC}\n" +} + +print_section() { + echo -e "\n${BLUE}═══════════════════════════════════════════════════════════════${NC}" + echo -e "${BOLD}${BLUE}▶ $1${NC}" + echo -e "${BLUE}═══════════════════════════════════════════════════════════════${NC}\n" +} + +print_success() { + echo -e "${GREEN}✓${NC} $1" +} + +print_error() { + echo -e "${RED}✗${NC} $1" +} + +print_warning() { + echo -e "${YELLOW}⚠${NC} $1" +} + +print_info() { + echo -e "${CYAN}ℹ${NC} $1" +} + +print_step() { + echo -e "${BOLD}→${NC} $1" +} + +# Check if running as root +check_root() { + if [ "$EUID" -eq 0 ]; then + print_error "Please do not run this script as root or with sudo" + print_info "The script will ask for sudo password when needed" + exit 1 + fi +} + +# Check Ubuntu/Debian version +check_ubuntu_version() { + if [ -f /etc/os-release ]; then + . /etc/os-release + OS=$ID + VER=$VERSION_ID + + if [[ "$OS" != "ubuntu" ]] && [[ "$OS" != "debian" ]]; then + print_warning "This script is optimized for Ubuntu/Debian" + print_info "Detected OS: $OS $VER" + read -p "Continue anyway? (y/N): " -n 1 -r + echo + if [[ ! $REPLY =~ ^[Yy]$ ]]; then + exit 1 + fi + else + print_success "Detected: $OS $VER" + fi + fi +} + +# Check system resources +check_system_resources() { + print_step "Checking system resources..." + + # Check RAM + TOTAL_RAM=$(free -g | awk '/^Mem:/{print $2}') + if [ "$TOTAL_RAM" -lt 8 ]; then + print_warning "Only ${TOTAL_RAM}GB RAM detected. Minimum 8GB recommended." + print_info "You can still continue with 'low' data synthesis mode" + else + print_success "RAM: ${TOTAL_RAM}GB" + fi + + # Check disk space + AVAILABLE_SPACE=$(df -BG . | awk 'NR==2 {print $4}' | sed 's/G//') + if [ "$AVAILABLE_SPACE" -lt 20 ]; then + print_warning "Only ${AVAILABLE_SPACE}GB disk space available. 20GB+ recommended." + else + print_success "Disk space: ${AVAILABLE_SPACE}GB available" + fi + + # Check for NVIDIA GPU + if command -v nvidia-smi &> /dev/null; then + GPU_INFO=$(nvidia-smi --query-gpu=name --format=csv,noheader | head -1) + print_success "NVIDIA GPU detected: $GPU_INFO" + HAS_GPU=true + else + print_info "No NVIDIA GPU detected (CPU-only mode)" + HAS_GPU=false + fi +} + +# Ask user for installation type +ask_installation_type() { + print_section "Installation Type Selection" + + echo -e "${BOLD}Please choose your installation method:${NC}\n" + echo " 1) Docker Installation (Recommended - Easiest)" + echo " └─ All dependencies pre-installed" + echo " └─ Isolated environment" + echo " └─ Automatic GPU detection" + echo "" + echo " 2) Local/Native Installation" + echo " └─ Better performance" + echo " └─ More control" + echo " └─ Requires manual dependency management" + echo "" + echo " 3) Both (Docker + Local)" + echo " └─ Maximum flexibility" + echo "" + + read -p "Enter your choice (1/2/3) [1]: " INSTALL_TYPE + INSTALL_TYPE=${INSTALL_TYPE:-1} + + case $INSTALL_TYPE in + 1) + INSTALL_DOCKER=true + INSTALL_LOCAL=false + print_success "Selected: Docker Installation" + ;; + 2) + INSTALL_DOCKER=false + INSTALL_LOCAL=true + print_success "Selected: Local Installation" + ;; + 3) + INSTALL_DOCKER=true + INSTALL_LOCAL=true + print_success "Selected: Both Docker and Local" + ;; + *) + print_error "Invalid choice. Defaulting to Docker." + INSTALL_DOCKER=true + INSTALL_LOCAL=false + ;; + esac +} + +# Install system dependencies +install_system_dependencies() { + print_section "Installing System Dependencies" + + print_step "Updating package lists..." + sudo apt-get update -qq + + print_step "Installing base dependencies..." + sudo apt-get install -y \ + curl \ + wget \ + git \ + build-essential \ + cmake \ + sqlite3 \ + libsqlite3-dev \ + ca-certificates \ + gnupg \ + lsb-release \ + unzip \ + software-properties-common + + print_success "System dependencies installed" +} + +# Install Docker +install_docker() { + print_section "Installing Docker" + + if command -v docker &> /dev/null; then + print_info "Docker is already installed" + docker --version + return + fi + + print_step "Adding Docker repository..." + + # Add Docker's official GPG key + sudo mkdir -p /etc/apt/keyrings + curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg + + # Set up the repository + echo \ + "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \ + $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null + + print_step "Installing Docker Engine..." + sudo apt-get update -qq + sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin + + print_step "Adding user to docker group..." + sudo usermod -aG docker $USER + + print_success "Docker installed successfully" + print_warning "You may need to log out and back in for docker group changes to take effect" +} + +# Install Python +install_python() { + print_section "Installing Python 3.12+" + + if command -v python3 &> /dev/null; then + PYTHON_VERSION=$(python3 --version | cut -d' ' -f2) + PYTHON_MAJOR=$(echo $PYTHON_VERSION | cut -d'.' -f1) + PYTHON_MINOR=$(echo $PYTHON_VERSION | cut -d'.' -f2) + + if [ "$PYTHON_MAJOR" -ge 3 ] && [ "$PYTHON_MINOR" -ge 12 ]; then + print_success "Python $PYTHON_VERSION already installed" + return + else + print_warning "Python $PYTHON_VERSION found, but 3.12+ required" + fi + fi + + print_step "Adding deadsnakes PPA for Python 3.12..." + sudo add-apt-repository -y ppa:deadsnakes/ppa + sudo apt-get update -qq + + print_step "Installing Python 3.12..." + sudo apt-get install -y python3.12 python3.12-venv python3.12-dev python3-pip + + # Set Python 3.12 as default python3 + sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1 + + print_success "Python 3.12 installed" + python3 --version +} + +# Install Node.js +install_nodejs() { + print_section "Installing Node.js" + + if command -v node &> /dev/null; then + NODE_VERSION=$(node --version) + print_success "Node.js $NODE_VERSION already installed" + return + fi + + print_step "Installing Node.js 20.x LTS..." + curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - + sudo apt-get install -y nodejs + + print_success "Node.js installed" + node --version + npm --version +} + +# Install Poetry +install_poetry() { + print_section "Installing Poetry" + + if command -v poetry &> /dev/null; then + print_success "Poetry already installed" + poetry --version + return + fi + + print_step "Installing Poetry..." + curl -sSL https://install.python-poetry.org | python3 - + + # Add Poetry to PATH for current session + export PATH="$HOME/.local/bin:$PATH" + + # Add to shell RC files + for rc_file in ~/.bashrc ~/.zshrc; do + if [ -f "$rc_file" ]; then + if ! grep -q 'export PATH="$HOME/.local/bin:$PATH"' "$rc_file"; then + echo 'export PATH="$HOME/.local/bin:$PATH"' >> "$rc_file" + fi + fi + done + + print_success "Poetry installed" + poetry --version +} + +# Setup local installation +setup_local_installation() { + print_section "Setting Up Local Installation" + + print_step "Running setup script..." + bash ./scripts/setup.sh + + print_success "Local setup completed" +} + +# Configure environment +configure_environment() { + print_section "Configuring Environment" + + if [ ! -f .env ]; then + if [ -f .env.example ]; then + print_step "Creating .env file from template..." + cp .env.example .env + print_success ".env file created" + else + print_warning ".env.example not found, skipping .env creation" + fi + else + print_info ".env file already exists" + fi + + # Ask for optimization settings + if [ "$TOTAL_RAM" -lt 16 ]; then + print_step "Optimizing for low memory system..." + if [ -f .env ]; then + # Add or update settings for low memory + if ! grep -q "DATA_SYNTHESIS_MODE" .env; then + echo "DATA_SYNTHESIS_MODE=low" >> .env + fi + if ! grep -q "CONCURRENCY_THREADS" .env; then + echo "CONCURRENCY_THREADS=1" >> .env + fi + print_success "Low memory optimizations applied" + fi + fi +} + +# Setup Docker installation +setup_docker_installation() { + print_section "Setting Up Docker Installation" + + # Check for GPU and ask user + if [ "$HAS_GPU" = true ]; then + echo "" + read -p "Enable GPU support (CUDA) in Docker? (Y/n): " -n 1 -r + echo + if [[ $REPLY =~ ^[Yy]$ ]] || [ -z "$REPLY" ]; then + print_step "Configuring GPU support..." + + # Install NVIDIA Container Toolkit if not present + if ! command -v nvidia-container-toolkit &> /dev/null; then + print_step "Installing NVIDIA Container Toolkit..." + + distribution=$(. /etc/os-release;echo $ID$VERSION_ID) + curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg + curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list | \ + sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \ + sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list + + sudo apt-get update -qq + sudo apt-get install -y nvidia-container-toolkit + sudo systemctl restart docker + + print_success "NVIDIA Container Toolkit installed" + fi + + # Mark GPU preference + echo "gpu" > .gpu_selected + print_success "GPU support enabled" + fi + fi + + print_step "Building and starting Docker containers..." + print_info "This may take 10-30 minutes depending on your internet speed..." + + make docker-up + + print_success "Docker containers are running" +} + +# Create startup scripts +create_startup_scripts() { + print_section "Creating Convenience Scripts" + + # Create start script + cat > start-second-me.sh << 'EOF' +#!/bin/bash +# Second-Me Startup Script + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$SCRIPT_DIR" + +echo "Starting Second-Me..." + +if [ -f ".docker_installed" ]; then + echo "Starting Docker containers..." + make docker-up + echo "" + echo "✓ Second-Me is running!" + echo " Web Interface: http://localhost:3000" + echo " Backend API: http://localhost:8002" + echo "" + echo "To stop: make docker-down" +elif [ -f ".local_installed" ]; then + echo "Starting local installation..." + make start + echo "" + echo "✓ Second-Me is running!" + echo " Web Interface: http://localhost:3000" + echo " Backend API: http://localhost:8002" + echo "" + echo "To stop: make stop" +else + echo "Error: Installation not found" + exit 1 +fi +EOF + chmod +x start-second-me.sh + + # Create stop script + cat > stop-second-me.sh << 'EOF' +#!/bin/bash +# Second-Me Stop Script + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +cd "$SCRIPT_DIR" + +echo "Stopping Second-Me..." + +if [ -f ".docker_installed" ]; then + make docker-down + echo "✓ Docker containers stopped" +elif [ -f ".local_installed" ]; then + make stop + echo "✓ Local services stopped" +fi +EOF + chmod +x stop-second-me.sh + + print_success "Startup scripts created (start-second-me.sh, stop-second-me.sh)" +} + +# Print final instructions +print_final_instructions() { + print_section "Installation Complete!" + + echo -e "${GREEN}╔══════════════════════════════════════════════════════════════════╗${NC}" + echo -e "${GREEN}║${BOLD} Installation Successful! ${NC}${GREEN}║${NC}" + echo -e "${GREEN}╚══════════════════════════════════════════════════════════════════╝${NC}\n" + + echo -e "${BOLD}Next Steps:${NC}\n" + + if [ "$INSTALL_DOCKER" = true ]; then + echo -e "${CYAN}Docker Installation:${NC}" + echo " • Access Second-Me at: ${BOLD}http://localhost:3000${NC}" + echo " • Backend API at: ${BOLD}http://localhost:8002${NC}" + echo "" + echo " Start: ${BOLD}./start-second-me.sh${NC} or ${BOLD}make docker-up${NC}" + echo " Stop: ${BOLD}./stop-second-me.sh${NC} or ${BOLD}make docker-down${NC}" + echo " Logs: ${BOLD}docker logs -f second-me-backend${NC}" + echo "" + fi + + if [ "$INSTALL_LOCAL" = true ]; then + echo -e "${CYAN}Local Installation:${NC}" + echo " • Access Second-Me at: ${BOLD}http://localhost:3000${NC}" + echo " • Backend API at: ${BOLD}http://localhost:8002${NC}" + echo "" + echo " Start: ${BOLD}./start-second-me.sh${NC} or ${BOLD}make start${NC}" + echo " Stop: ${BOLD}./stop-second-me.sh${NC} or ${BOLD}make stop${NC}" + echo " Logs: ${BOLD}tail -f logs/train/train.log${NC}" + echo "" + fi + + echo -e "${YELLOW}Important Notes:${NC}" + echo " • First startup may take a few minutes" + echo " • Training process can take 1-6 hours depending on data" + echo " • Start with small documents (1-2 PDFs) for testing" + echo " • Check logs if something seems stuck" + + if [ "$INSTALL_DOCKER" = true ] && [ "$INSTALL_LOCAL" = false ]; then + echo "" + print_warning "You may need to log out and back in for Docker permissions" + echo " Or run: ${BOLD}newgrp docker${NC}" + fi + + echo "" + echo -e "${BOLD}Useful Commands:${NC}" + echo " • View all commands: ${BOLD}make help${NC}" + echo " • Check service status: ${BOLD}make status${NC}" + echo " • Restart services: ${BOLD}make restart${NC}" + + echo "" + echo -e "${CYAN}Documentation:${NC}" + echo " • Homepage: https://home.second.me/" + echo " • GitBook: https://secondme.gitbook.io/secondme/" + echo " • Discord: https://discord.gg/GpWHQNUwrg" + + echo "" + echo -e "${GREEN}Installation log saved to: ${BOLD}$INSTALL_LOG${NC}\n" +} + +# Error handler +error_handler() { + print_error "An error occurred during installation" + print_info "Check the log file for details: $INSTALL_LOG" + print_info "You can also ask for help on Discord: https://discord.gg/GpWHQNUwrg" + exit 1 +} + +trap error_handler ERR + +################################################################################ +# Main Installation Flow +################################################################################ + +main() { + print_header + + print_info "Installation log: $INSTALL_LOG" + print_info "Started at: $(date)" + + # Pre-flight checks + check_root + check_ubuntu_version + check_system_resources + + # Ask user preferences + ask_installation_type + + # Install system dependencies + install_system_dependencies + + # Install Docker if requested + if [ "$INSTALL_DOCKER" = true ]; then + install_docker + fi + + # Install local dependencies if requested + if [ "$INSTALL_LOCAL" = true ]; then + install_python + install_nodejs + install_poetry + fi + + # Configure environment + configure_environment + + # Setup installations + if [ "$INSTALL_LOCAL" = true ]; then + setup_local_installation + touch .local_installed + fi + + if [ "$INSTALL_DOCKER" = true ]; then + setup_docker_installation + touch .docker_installed + fi + + # Create convenience scripts + create_startup_scripts + + # Print final instructions + print_final_instructions + + print_info "Installation completed at: $(date)" +} + +# Run main installation +main "$@" From 1246751779761e70fcd08d0761e7870c4f255eb1 Mon Sep 17 00:00:00 2001 From: DYAI Date: Sat, 1 Nov 2025 05:36:21 +0100 Subject: [PATCH 2/2] Update install-ubuntu.sh Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- install-ubuntu.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/install-ubuntu.sh b/install-ubuntu.sh index 9caca893..6eed4cd1 100755 --- a/install-ubuntu.sh +++ b/install-ubuntu.sh @@ -247,11 +247,14 @@ install_python() { print_step "Installing Python 3.12..." sudo apt-get install -y python3.12 python3.12-venv python3.12-dev python3-pip - # Set Python 3.12 as default python3 - sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.12 1 + # Do NOT change the system default python3, as this can break system tools. + # If you need to use Python 3.12, use 'python3.12' explicitly or create a virtual environment: + print_warning "System default python3 NOT changed. To use Python 3.12, use 'python3.12' or create a virtual environment:" + print_info "python3.12 -m venv ~/secondme-venv" + print_info "source ~/secondme-venv/bin/activate" print_success "Python 3.12 installed" - python3 --version + python3.12 --version } # Install Node.js