A WhatsApp chatbot for task management, designed to work with Evolution API in a secure internal network environment.
## 📌 Overview
This service provides a WhatsApp interface for task management within WhatsApp groups. It:
- Listens for `/tarea` commands in WhatsApp groups via Evolution API webhooks.
- Stores tasks, users, and groups in a SQLite database.
- Synchronizes group information periodically from the Evolution API.
- Manages user permissions and group membership (partially implemented).
- Integrates with Evolution API for WhatsApp connectivity.
- Sends direct messages to acknowledge the creator and notify each assignee; includes mentions with phone numbers for quick action.
- DM-only responses: the bot does not post messages in groups; all outputs are sent via DM to the author (except the optional creation summary below).
- DM reminders: opt-in daily/weekly configured via "/t configurar"; runs at local TZ time (default Europe/Madrid), skips when no tasks and avoids duplicates per day.
- Advanced listings: "/t ver" supports scopes "grupo", "mis", "sin" and "todos" with pagination ("… y X más").
- Complete tasks: "/t x <id>" (aliases: hecho, completar, done) with compact DM feedback.
- Claim/unassign tasks: "/t tomar <id>" and "/t soltar <id>" with compact DM feedback.
- Natural date parsing: supports "hoy"/"mañana" with TZ (env TZ; default Europe/Madrid).
- Unified date display: all dates are rendered as dd/MM; help available via DM with "/t" or "ayuda".
- Optional: group notification on task creation controlled by NOTIFY_GROUP_ON_CREATE (default false), including proper mentions for visibility.
## 🔐 Security Model
- **Internal Networking**: The webhook should ideally only accept connections from Evolution API via internal Docker networking (configuration dependent).
- **Environment Variables**: Sensitive configuration (API keys, URLs) is managed through environment variables.
- **Group Restrictions**: Designed to operate within pre-approved WhatsApp groups (validation logic pending integration).
- **Input Validation**: Basic validation exists for webhook structure; needs enhancement for command arguments and user/group IDs.
## 🧱 Architecture
```mermaid
graph TD
A[Webhook Received] --> B{Valid Payload?}
B -->|No| C[Ignore]
B -->|Yes| D{Normalize IDs & Check Group Active?}
D -->|No| C[Ignore/Log]
D -->|Yes| E[Ensure User Exists in DB]
E --> G{/tarea Command?}
G -->|No| C
G -->|Yes| J[Process Command Logic]
J -- Success/Error --> K[Queue Response(s)]
K --> L[Process Queue & Send Response via API]
subgraph Database Interaction
E --> DB[(SQLite DB)]
J --> DB
end
subgraph Evolution API
L --> EA((Evolution API))
EA -- Webhook --> A
end
```
*(Diagram updated for planned flow)*
## Decisiones de diseño de la cola de respuestas (estado actual)
1) Persistencia: 100% persistente (tabla única).
2) Worker: continuo en background.
3) Locking: status=processing (sin lease).
4) Orden: sin orden estricto por chat.
5) Reintentos: activados con backoff exponencial + jitter (configurables).
6) Errores: 4xx = fallo definitivo; 5xx/red = reintento hasta RQ_MAX_ATTEMPTS con `next_attempt_at`.
7) Idempotencia: no.
8) Esquema DB: tabla única response_queue.
9) Transporte: envío desde ResponseQueue (fetch directo a Evolution API).
10) Concurrencia: N workers globales.
11) Integración: encolar y worker continuo (arranca con el servidor).
12) Configuración: defaults fijos (sin nuevas env vars por ahora).
13) Limpieza: retención periódica de historiales aplicada (sent y failed antiguos).
14) Seguridad: no enviar al número del bot (CHATBOT_PHONE_NUMBER).
15) Pruebas: unitarias de cola con mocks de fetch.
Actualización Phase 4 — Etapa 1 (Completada):
- Reintentos activados con backoff exponencial + jitter.
- Sin garantía de orden cross-chat; fair scheduling pendiente.
## Plan incremental posterior
- Añadir reintentos con backoff exponencial y jitter. (Completado)
- Garantizar orden por chat (serialización por recipient).
- Introducir lease (lease_until) para tolerancia a fallos y recuperación.
- Limpieza/retención y métricas/observabilidad.
- Opcional: idempotencia e índices adicionales.
## ✅ Current Status (as of latest commit)
### Implemented
- Webhook server setup (`src/server.ts`) receiving Evolution API events.
- Corrección: extracción robusta de remitente en DMs vs grupos para evitar 'Invalid sender ID'.
- Sistema de migraciones up-only para SQLite (`src/db/migrator.ts`, `src/db/migrations/`), ejecutado al arranque; backup automático con VACUUM INTO y baseline cuando existe esquema previo.
- Group synchronization service (`src/services/group-sync.ts`) to fetch/store/cache groups. Includes active group caching and `isGroupActive` checks in server.
- Webhook registration and verification with Evolution API (`src/services/webhook-manager.ts`).
- WhatsApp ID normalization utility (`src/utils/whatsapp.ts`).
- User creation/update function (`src/db.ts::ensureUserExists`) integrated into the main flow.
- Command handling for `/tarea nueva` end-to-end (`src/services/command.ts`):
- Parses description and selects the last future date (YYYY-MM-DD) as due date.
- Extracts assignees from explicit mentions and plain-text @tokens; defaults to creator only when none found.
- Cleans description to remove @mentions tokens.
- Persists task and assignments atomically via `TaskService`.
- Builds response with assignment list and includes Evolution API “mentioned” JIDs via `ResponseQueue`.
- Command handling for listings, completion and assignment (tomar/soltar):
- `/t ver` with scopes `grupo`, `mis`, `sin` and `todos` with top-N and “… y X más”; DM-only policy enforced.
- `/t x <id>` to complete tasks (aliases: `hecho`, `completar`, `done`) with compact feedback.
- Task persistence service (`src/tasks/service.ts`) with `created_by` and assignment inserts in a transaction; supports DB injection for tests.
- Response queue persistente con workers en background y envío vía Evolution API (`src/services/response-queue.ts`), persistiendo metadata `{ mentioned: [...] }` y enviándola como `mentioned` en el payload, con reintentos (backoff exponencial + jitter), recuperación de `processing` y limpieza/retención.
- Contacts service and friendly names: `ContactsService` resolves display names via webhooks (CONTACTS_UPDATE/CHATS_UPDATE) and Evolution API fallback; used to render names in outgoing texts (falls back to numbers). Skips network calls under NODE_ENV=test for fast and isolated unit tests.
- Reminders by DM: user preferences in `user_preferences` (reminder_freq: off|daily|weekly; reminder_time; last_reminded_on), `RemindersService` running minutely (disabled in tests), daily and weekly (Mondays 08:30) summaries, skips when no tasks, compact message with “… y X más”. Command “/t configurar daily|weekly|off” with DM confirmation. Unit tests included.
- Notification UX: Always send DM acknowledgment to the creator (header: 📝 <id> “description”, plus date 📅 and owner line), DM to each assignee with header “📬 Tarea <id> — 📅 dd/MM” (excluding the creator); optional group notification controlled by `NOTIFY_GROUP_ON_CREATE` (default false) with proper mentions; unassigned shown as “🚫👤 sin dueño”; overdue dates flagged with “⚠️” before “📅”.
- Natural date tokens support (“hoy”/“mañana”) with TZ configurable (env TZ; default Europe/Madrid) in task creation.
- Unified dd/MM date formatting across outgoing texts.
- Help by DM for "/t" and "/t ayuda" with a concise guide and examples.
- Rate limiting per user: 15 commands/min (configurable via RATE_LIMIT_PER_MIN and RATE_LIMIT_BURST); disabled under NODE_ENV=test; polite DM notice throttled to 1/min.
- **Database isolation in unit tests**: Using in-memory instances to avoid conflicts.
### Incomplete / Missing Core Functionality
- ResponseQueue reliability: reintentos con backoff, recuperación de `processing`, métricas y limpieza/retención.
- ContactsService improvements (optional): refine caching policy and endpoints; basic friendly-name resolution is already implemented and used in outgoing texts.
- More robust error handling and observability around API/DB operations.
### Known Issues
- Mentions UX: In group chats, mentions are highlighted and each client resolves the chip to their local contact name; we include friendly names in text when available. In direct messages, WhatsApp does not render mention chips for third parties, so we include the number as @digits for quick action; no per-recipient name rewriting occurs.
- Test suite: 170 unit tests passing.
## 🛠️ Setup
### Environment Variables
*(Ensure these are set correctly)*
```env
# Evolution API Connection
EVOLUTION_API_URL=http://evolution-api:3000 # Or your API URL
EVOLUTION_API_KEY=your-api-key
EVOLUTION_API_INSTANCE=main # Your instance name
# WhatsApp Specific
WHATSAPP_COMMUNITY_ID=your-community-id # ID of the main community to sync groups from
CHATBOT_PHONE_NUMBER=1234567890 # Bot's normalized phone number (e.g., for assigning tasks)
# Webhook Configuration
WEBHOOK_URL=http://your-service-internal-url:3007 # URL Evolution API calls *back* to this service
PORT=3007 # Port this service listens on
# Runtime Environment
NODE_ENV=production # Or development
TZ=Europe/Madrid # Default timezone used for "hoy/mañana" parsing and date displays
# Optional
# GROUP_SYNC_INTERVAL_MS=3600000 # Sync interval in ms (default: 24h)
# NOTIFY_GROUP_ON_CREATE=false # If 'true', also post a brief summary with mentions to the group
# RATE_LIMIT_PER_MIN=15 # Commands per minute per user (token refill rate)
# RATE_LIMIT_BURST=15 # Bucket capacity (defaults to RATE_LIMIT_PER_MIN)
```
### Development Setup
```bash
# Install dependencies
bun install
# Copy .env.example to .env and fill in values
cp .env.example .env
# Start development server (watches for changes)
bun run dev
# Run tests
bun test
```
## 📅 Roadmap & Priorities (Updated Plan)
### MVP — Próximos pasos (prioridad actual)
1) Afinar recordatorios por DM (daily/weekly): hora configurable por usuario y pequeños ajustes de formato.
2) Refinar ContactsService: mejorar heurística de nombre, invalidación temprana en CHATS_UPDATE/CONTACTS_UPDATE, TTL configurable, robustez ante fallos, sin llamadas de red en tests.
3) Sincronización mínima de miembros: obtener y cachear miembros de grupos activos con TTL, uso no bloqueante; base para futuras validaciones/UX.
4) Ampliar test suite: actualización de nombres, sync de miembros y casos extremos de ResponseQueue.
### Phase 1: User & Group Foundation (Highest Priority - In Progress)
- [x] **Create WhatsApp ID Normalization Utility:** (`src/utils/whatsapp.ts`) Handle different ID formats.
- [x] **Implement `ensureUserExists`:** (`src/db.ts`) Add users to DB on first interaction.
- [x] **Implement `isGroupActive` Check:** (`src/services/group-sync.ts`, `src/server.ts`) Cache exists in `group-sync.ts`. **Needs integration** into `src/server.ts`.
- [x] **Integrate Validation in Server:** (`src/server.ts`) Use normalization, `ensureUserExists`, and active group check before processing commands.
- [x] **Implement DB isolation in tests:** (`tests/unit/services/group-sync.test.ts`) Use in-memory instances to avoid conflicts.
- Lógica: 2xx → sent; 4xx → failed definitivo; 5xx/red → reintento con `next_attempt_at` hasta MAX_ATTEMPTS.
- Etapa 2 — Recuperación de items en `processing` mediante lease/expiración (Completada)
- Etapa 3 — Métricas y observabilidad (Pospuesta post-MVP)
- Etapa 4 — Limpieza/retención (Completada)
### Phase 5: Advanced Features (Low Priority)
- [x] Add task reminders system (daily/weekly) with user preferences and scheduler.
- [ ] Implement user permissions system.
- [x] Add rate limiting.
- [ ] Create task history tracking.
### Etapas siguientes (priorización propuesta)
1) Phase 4 — Etapa 4: Limpieza y retención
- Objetivo: evitar crecimiento indefinido y mantener la BD sana.
- Implica: retención/archivado de response_queue, posible VACUUM, config RQ_RETENTION_DAYS y job periódico.
2) Fiabilidad avanzada de ResponseQueue
- Objetivo: mejorar entrega y evitar duplicados/ruido.
- Implica: orden por destinatario, idempotency_key con índice único, límites de inflight/backpressure.
3) UX Iteración A (MVP centrado en valor)
- Objetivo: fluidez de uso y silencio en grupos.
- Implica: alias y sinónimos, “solo DM”, soporte “hoy/mañana”, comandos tomar/soltar, ayuda breve.
4) UX Iteración B (listas y completar)
- Objetivo: visibilidad y cierre rápido.
- Implica: “ver grupo” y “ver mis” con tope y “y X más…”, completar (x/hecho/done) robusto, formatos compactos.
5) Recordatorios (Phase 5)
- Objetivo: nudge suave por DM.
- Implica: tabla user_preferences, comando “configurar”, job diario de resumen.
Un chatbot de WhatsApp para gestionar tareas en grupos, integrado con Evolution API. Diseño “solo DM”: el bot no publica en grupos; todas las respuestas se envían por mensaje directo al autor (opcionalmente puede enviarse un breve resumen al grupo al crear, configurable).
## Cómo se usa (mini guía para usuarios)
- Principios
- Comandos con prefijo “/t” o “/tarea”.
- En grupo: el bot responde por DM al autor (no escribe en el grupo).
- Fechas en formato dd/MM en mensajes; puedes escribir “hoy” o “mañana” al crear; la zona horaria se controla con la variable de entorno TZ (por defecto Europe/Madrid).
- Comandos y alias principales
- Crear: “/t nueva Acta reunión mañana @600123456”
- Ver pendientes del grupo: “/t ver grupo”
- Ver tus pendientes: “/t ver mis”
- Completar: “/t x 26” (alias: hecho, completar, done)
