taskbot/README.md

# Task WhatsApp Chatbot

A WhatsApp chatbot for task management, designed to work with Evolution API in a secure internal network environment.

## 📌 Overview
This service provides a WhatsApp interface for task management within WhatsApp groups. It:
- Listens for `/tarea` commands in WhatsApp groups via Evolution API webhooks.
- Stores tasks, users, and groups in a SQLite database.
- Synchronizes group information periodically from the Evolution API.
- Manages user permissions and group membership (partially implemented).
- Integrates with Evolution API for WhatsApp connectivity.

## 🔐 Security Model
- **Internal Networking**: The webhook should ideally only accept connections from Evolution API via internal Docker networking (configuration dependent).
- **Environment Variables**: Sensitive configuration (API keys, URLs) is managed through environment variables.
- **Group Restrictions**: Designed to operate within pre-approved WhatsApp groups (validation logic pending integration).
- **Input Validation**: Basic validation exists for webhook structure; needs enhancement for command arguments and user/group IDs.

## 🧱 Architecture
```mermaid
graph TD
    A[Webhook Received] --> B{Valid Payload?}
    B -->|No| C[Ignore]
    B -->|Yes| D{Normalize IDs & Check Group Active?}
    D -->|No| C[Ignore/Log]
    D -->|Yes| E[Ensure User Exists in DB]
    E --> G{/tarea Command?}
    G -->|No| C
    G -->|Yes| J[Process Command Logic]
    J -- Success/Error --> K[Queue Response(s)]
    K --> L[Process Queue & Send Response via API]

    subgraph Database Interaction
        E --> DB[(SQLite DB)]
        J --> DB
    end
    subgraph Evolution API
        L --> EA((Evolution API))
        EA -- Webhook --> A
    end
```
*(Diagram updated for planned flow)*

## Decisiones de diseño de la cola de respuestas (MVP)
1) Persistencia: 100% persistente (tabla única).
2) Worker: continuo en background.
3) Locking: status=processing (sin lease).
4) Orden: sin orden estricto por chat.
5) Reintentos: sin reintentos en MVP.
6) Errores: 4xx = fallo definitivo; 5xx/red = fallo (sin reintentos).
7) Idempotencia: no.
8) Esquema DB: tabla única response_queue.
9) Transporte: envío desde ResponseQueue (fetch directo a Evolution API).
10) Concurrencia: N workers globales.
11) Integración: encolar y worker continuo (arranca con el servidor).
12) Configuración: defaults fijos (sin nuevas env vars por ahora).
13) Limpieza: sin limpieza/retención de historiales (por ahora).
14) Seguridad: no enviar al número del bot (CHATBOT_PHONE_NUMBER).
15) Pruebas: unitarias de cola con mocks de fetch.

## Arquitectura de la cola persistente (MVP)
- Estados: queued | processing | sent | failed.
- Campos mínimos por mensaje: id (PK), recipient, type (text), message, status, attempts (0), last_error (nullable), created_at, updated_at.
- Índices recomendados: (status, created_at) para seleccionar pendientes rápidamente.
- Sin orden estricto por chat; el envío puede intercalarse entre destinatarios.
- Concurrencia: N workers globales operando en bucle, cada uno toma mensajes en estado queued y los marca processing.
Estado: la tabla response_queue ya está creada e incluida en los tests de DB.

## Flujo del worker continuo (MVP)
- Se inicia al arrancar el servidor (desactivado en tests).
- Ciclo: seleccionar hasta un pequeño batch de mensajes queued, marcar processing, enviar a Evolution API, marcar sent o failed según respuesta.
- Sin reintentos; logs mínimos y no sensibles.

## Limitaciones explícitas del MVP
- Sin backoff ni reintentos.
- Sin orden garantizado por chat.
- Sin idempotencia ni limpieza automática.
- Sin lease; en caso de crash podrían quedar mensajes en processing que requerirán recuperación manual en una iteración futura.

## Plan incremental posterior
- Añadir reintentos con backoff exponencial y jitter.
- Garantizar orden por chat (serialización por recipient).
- Introducir lease (lease_until) para tolerancia a fallos y recuperación.
- Limpieza/retención y métricas/observabilidad.
- Opcional: idempotencia e índices adicionales.

## ✅ Current Status (as of latest commit)
### Implemented
- Webhook server setup (`src/server.ts`) receiving Evolution API events.
- Database schema definition and initialization (`src/db.ts`), including lightweight migration to add `response_queue.metadata`.
- Group synchronization service (`src/services/group-sync.ts`) to fetch/store/cache groups. Includes active group caching and `isGroupActive` checks in server.
- Webhook registration and verification with Evolution API (`src/services/webhook-manager.ts`).
- WhatsApp ID normalization utility (`src/utils/whatsapp.ts`).
- User creation/update function (`src/db.ts::ensureUserExists`) integrated into the main flow.
- Command handling for `/tarea nueva` end-to-end (`src/services/command.ts`):
  - Parses description and selects the last future date (YYYY-MM-DD) as due date.
  - Extracts assignees from explicit mentions and plain-text @tokens; defaults to creator only when none found.
  - Cleans description to remove @mentions tokens.
  - Persists task and assignments atomically via `TaskService`.
  - Builds response with assignment list and includes Evolution API “mentioned” JIDs via `ResponseQueue`.
- Task persistence service (`src/tasks/service.ts`) with `created_by` and assignment inserts in a transaction; supports DB injection for tests.
- Response queue persistente con workers en background y envío vía Evolution API (`src/services/response-queue.ts`), persistiendo metadata `{ mentioned: [...] }` y enviándola como `mentioned` en el payload.
- Environment variable validation (`src/server.ts`, `src/services/webhook-manager.ts`).
- Health check endpoint (`/health`).
- **Database isolation in unit tests**: Using in-memory instances to avoid conflicts.

### Incomplete / Missing Core Functionality
- Additional commands: `/tarea mostrar` (list) y `/tarea completar`.
- ResponseQueue reliability: reintentos con backoff, recuperación de `processing`, métricas y limpieza/retención.
- Contact names: optional service/cache to resolve friendly names from Evolution API for nicer “@Nombre” rendering in text (the app already sends `mentioned` JIDs).
- Database migrations system (beyond current lightweight on-boot checks).
- More robust error handling and observability around API/DB operations.

### Known Issues
- Mentions UX: although we send `mentioned` JIDs, WhatsApp still shows “@número” in the text. This is expected unless we replace the text with a friendly name known to the bot; the client renders the highlight but doesn’t rewrite the text to each user’s local contact name.
- Test suite: currently 1 failing test — “Database > Table Schemas > response_queue table should have required columns”.

## 🛠️ Setup
### Environment Variables
*(Ensure these are set correctly)*
```env
# Evolution API Connection
EVOLUTION_API_URL=http://evolution-api:3000 # Or your API URL
EVOLUTION_API_KEY=your-api-key
EVOLUTION_API_INSTANCE=main # Your instance name

# WhatsApp Specific
WHATSAPP_COMMUNITY_ID=your-community-id # ID of the main community to sync groups from
CHATBOT_PHONE_NUMBER=1234567890 # Bot's normalized phone number (e.g., for assigning tasks)

# Webhook Configuration
WEBHOOK_URL=http://your-service-internal-url:3007 # URL Evolution API calls *back* to this service
PORT=3007 # Port this service listens on

# Runtime Environment
NODE_ENV=production # Or development

# Optional
# GROUP_SYNC_INTERVAL_MS=3600000 # Sync interval in ms (default: 24h)
```

### Development Setup
```bash
# Install dependencies
bun install

# Copy .env.example to .env and fill in values
cp .env.example .env

# Start development server (watches for changes)
bun run dev

# Run tests
bun test
```

## 📅 Roadmap & Priorities (Updated Plan)

### Phase 1: User & Group Foundation (Highest Priority - In Progress)
- [x] **Create WhatsApp ID Normalization Utility:** (`src/utils/whatsapp.ts`) Handle different ID formats.
- [x] **Implement `ensureUserExists`:** (`src/db.ts`) Add users to DB on first interaction.
- [x] **Implement `isGroupActive` Check:** (`src/services/group-sync.ts`, `src/server.ts`) Cache exists in `group-sync.ts`. **Needs integration** into `src/server.ts`.
- [x] **Integrate Validation in Server:** (`src/server.ts`) Use normalization, `ensureUserExists`, and active group check before processing commands.
- [x] **Implement DB isolation in tests:** (`tests/unit/services/group-sync.test.ts`) Use in-memory instances to avoid conflicts.

### Phase 2: Refinar `/tarea nueva` (Alta prioridad)
- [ ] Completar `TaskService.createTask` con `created_by` y asignaciones (incluyendo `ensureUserExists` para asignados) y transacción atómica.
- [ ] Mejorar `CommandService` para parsear menciones y devolver el id de la tarea creada y resumen de asignados.

### Phase 3: Comandos adicionales y refinamientos (Alta prioridad)
- [ ] Implementar `/tarea mostrar [group|mine]` para listar pendientes.
- [ ] Implementar `/tarea completar <task_id>` con validaciones básicas.
- [ ] Soportar mensajes de texto extendido y captions de media (además de conversation).

### Phase 4: Fiabilidad de la cola y observabilidad (Media)
- [ ] Añadir reintentos con backoff exponencial y jitter.
- [ ] Recuperar ítems en estado `processing` tras reinicios (lease o expiración y requeue).
- [ ] Métricas y logging mejorado (contadores de enviados/fallidos, tiempos).
- [ ] Limpieza/retención de historiales.

### Phase 5: Advanced Features (Low Priority)
- [ ] Add task reminders system.
- [ ] Implement user permissions system.
- [ ] Add rate limiting.
- [ ] Create task history tracking.

## 🔑 Key Considerations & Caveats
*   **WhatsApp ID Normalization:** Crucial for consistently identifying users and groups. Needs careful implementation to handle edge cases. (Utility function exists).
*   **Response Latency:** Sending responses requires an API call back to Evolution. Ensure the `ResponseQueue` processing is efficient.
*   **Cola de respuestas:** Persistente en DB; pendiente añadir reintentos/backoff y limpieza/retención.
*   **Group Sync:** The current full sync might be slow or rate-limited with many groups. Delta updates are recommended long-term.
*   **Error Handling:** Failures in command processing or response sending should be logged clearly and potentially reported back to the user. Database operations should use transactions for atomicity (especially task+assignment creation).
*   **State Management:** The current design is stateless. Complex interactions might require state persistence later.
*   **Security:** Ensure group/user validation logic is robust once integrated.

## 🧪 Testing
### Running Tests
```bash
bun test
```

### Test Coverage
- Database initialization and basic operations (`db.test.ts`).
- Webhook validation (basic) (`webhook-manager.test.ts`).
- Command parsing (basic structure) (`command.test.ts`).
- Environment checks (`server.test.ts`).
- Basic error handling (`server.test.ts`).
- WhatsApp ID normalization (`whatsapp.test.ts`).
- Group sync operations (`group-sync.test.ts`).
- **Needed:** Tests for `ensureUserExists` integration, `isGroupActive` integration, `CommandService` logic, `ResponseQueue` processing (mocking API), `TaskService` operations.
- Tests mostly pass; currently 1 failing test: “Database > Table Schemas > response_queue table should have required columns”.

## 🧑‍💻 Contributing
1. Fork the repository
2. Create a feature branch (`git checkout -b feature/implement-user-validation`)
3. Add/update tests for new functionality
4. Ensure tests pass (`bun test`)
5. Submit a pull request

## 📚 Documentation
For detailed API documentation and architecture decisions, see the [docs/](docs/) directory (if created).