fix(calendar): TZID Europe/Berlin for timed events/tasks, resolve real resource href on delete/update, fix _report_tasks XML; sync deployed servers (owamail, openwearables, ghostfolio), docs and tests
This commit is contained in:
1 parent
7c01e7be53
commit
75185ef2ba
41 files changed
+8834
-324
No files matched your search
@@ -0,0 +1,209 @@
|
||||
# mcp-home
|
||||
|
||||
Self-hosted MCP (Model Context Protocol) servers for the OpenClaw assistants (mycroft/jarvis). Provides access to personal data services via HTTP endpoints with OAuth (client_credentials).
|
||||
|
||||
## Architecture
|
||||
|
||||
8 independent MCP servers, each on its own port, bound to 0.0.0.0:
|
||||
|
||||
- **Mail** (Port 5100) — searches IMAP backup Maildirs, reads attachments, creates drafts
|
||||
- **Calendar** (Port 5101) — CalDAV against Baikal (Produktiv; Radicale-Fallback nur fuer Test-User mcptest) (events + tasks/reminders, travel time, geocoding, alarms)
|
||||
- **Contacts** (Port 5102) — CardDAV against Baikal (Produktiv; Radicale-Fallback nur fuer Test-User mcptest) (contacts + photos)
|
||||
- **Files** (Port 5103) — WebDAV against oCIS (full CRUD, images inline, documents as blobs)
|
||||
- **Notes** (Port 5104) — Joplin Data API (notes + attachments)
|
||||
- **OpenWearables** (Port 5105) — read-only Wearable-Daten (Sleep, Activity, Workouts, Timeseries) aus Open Wearables (`/opt/openwearables`), nur eigene Daten (s. u.)
|
||||
- **Ghostfolio** (Port 5120) — Portfolio-Tracker via Ghostfolio-REST-API (Port 3333); read+write (Konten, Activities, Performance, Dividenden)
|
||||
- **ActualBudget** (Port 5121) — Haushaltsbudget; read+write via Node-Sidecar `/opt/actualbudget-mcp/sidecar` (systemd `actualbudget-sidecar`, 127.0.0.1:5106) auf ActualBudget-Server (Port 5006)
|
||||
|
||||
Zugriff intern (Standard): Die OpenClaw-Container (mycroft/jarvis) erreichen die Server
|
||||
ueber das Docker-Gateway `172.26.0.1:<port>/mcp` (openclaw-net). Die URLs stehen in den
|
||||
Container-Configs `/mnt/ssd/openclaw/<agent>/data/openclaw.json` (mcp.servers.*.url).
|
||||
|
||||
Zugriff oeffentlich (seit 2026-09-18): notes, calendar, mail, contacts, files, openwearables,
|
||||
ghostfolio und owamail sind zusaetzlich unter `https://<name>.mcp.slohmaier.de/mcp`
|
||||
erreichbar (nginx -> 127.0.0.1:<port>, Let's-Encrypt-SAN, Rate-Limit auf /authorize+/token).
|
||||
OpenClaw nutzt fuer diese die oeffentlichen URLs (bewusste Abweichung von der Regel "lokale
|
||||
Routen", damit die Config bei einem Umzug unveraendert bleibt). Weitere Server bei Bedarf nach
|
||||
demselben Muster (vhost + Cert-SAN).
|
||||
Details: README.md, Abschnitt "Oeffentliche Endpunkte".
|
||||
|
||||
Auth fuer oeffentliche Clients: OAuth (claude.ai) ODER nicht-ablaufende Service-Tokens
|
||||
(`service_tokens.json`, Verwaltung `./mint-service-token.sh`); common.py prueft Service-Tokens
|
||||
vor den OAuth-Access-Tokens. Zukunfts-Option OAuth/DCR (ChatGPT) siehe README.md.
|
||||
|
||||
Firewall: iptables raw-PREROUTING blockt die Ports 5100-5105,5120,5121 fuer alle Quellen
|
||||
ausserhalb `172.26.0.0/16` (openclaw-net). nginx erreicht die Ports via 127.0.0.1 (ACCEPT-Regel).
|
||||
Regel in `/etc/iptables/rules.v4` (netfilter-persistent).
|
||||
|
||||
### OpenWearables (Port 5105)
|
||||
|
||||
Server: `/opt/mcp-servers/openwearables/server.py`, Config `/opt/mcp-servers/openwearables/config.json`
|
||||
(chmod 600, enthaelt den gemeinsamen OpenWearables-API-Key und das User-ID-Mapping).
|
||||
|
||||
Tools: `get_users`, `get_sleep_summary`, `get_activity_summary`, `get_workout_events`, `get_timeseries`.
|
||||
Alle read-only, gegen die interne Open-Wearables-API (`http://127.0.0.1:8000`), NIE ueber Cloudflare.
|
||||
|
||||
**User-Trennung (wichtig):** Der OpenWearables-API-Key ist GLOBAL (nicht pro User gescoped). Die
|
||||
Trennung passiert im Server: `config.json` mappt den OAuth-User (`stefan`/`kati`) auf die jeweilige
|
||||
OpenWearables-User-UUID (`user_id_map`). Die OpenWearables-User-ID wird IMMER aus dem Login des
|
||||
Clients bestimmt, nie aus Tool-Argumenten. mycroft sieht also nur Stefans Daten, jarvis nur Katis.
|
||||
|
||||
- MCP-Server in openclaw.json: `openwearables` -> `http://172.26.0.1:5105/mcp`
|
||||
- systemd: `mcp-openwearables.service`
|
||||
|
||||
All servers share `common.py` for OAuth + user resolution.
|
||||
|
||||
## Auth Flow
|
||||
|
||||
claude.ai uses OAuth 2.0 **Authorization Code + PKCE**. client_credentials is also supported (used by tests).
|
||||
|
||||
1. claude.ai discovers `/.well-known/oauth-authorization-server`
|
||||
2. `GET /authorize` → auto-approves valid client_id, redirects with code
|
||||
3. `POST /token` (authorization_code + code_verifier, optional client_secret) → access_token
|
||||
4. `Authorization: Bearer <token>` on MCP requests
|
||||
|
||||
Access tokens are stored in a shared file `.active_tokens.json` so all 5 services validate tokens issued by any service. client_secret is verified at token exchange when provided.
|
||||
|
||||
## User Separation
|
||||
|
||||
`tokens.json` (not in git, chmod 600) maps client_id (`stefan`, `kati`, `test`) → secret. `USER_ALIASES` in `common.py` maps `test` → `stefan`. Each user sees only their own data; shared CalDAV calendars are visible to both.
|
||||
|
||||
## Config
|
||||
|
||||
- `tokens.json` — OAuth client_id/secret per user (gitignored)
|
||||
- `config.json` — all backend credentials (Baikal + Radicale, oCIS, Joplin, IMAP), gitignored. See `config.json.example`.
|
||||
- `.active_tokens.json` — runtime OAuth tokens (gitignored)
|
||||
|
||||
## Backends
|
||||
|
||||
- **Mail**: reads Maildirs under `/mnt/ssd/Backup/{stefan,kati}/imap/`. Drafts via IMAP APPEND (1blu / Gmail).
|
||||
- **Calendar/Contacts**: Baikal CalDAV/CardDAV on `https://baikal.slohmaier.de` (Produktiv, seit 2026-09-11). Basen:
|
||||
`dav.php/calendars/<user>/` und `dav.php/addressbooks/<user>/`; Baikal liefert hrefs pfad-absolut (`/dav.php/...`) und
|
||||
calendar-data/address-data mit Namespace-Prefix `cal:`/`card:` — der Parser ist Namespace-agnostisch
|
||||
(`(?:[a-zA-Z][a-zA-Z0-9_]*):calendar-data|address-data`). sabre inbox/outbox werden herausgefiltert.
|
||||
Backend-Auswahl pro User ueber `_serve(user)`: User in `baikal_users` (stefan/kati = mycroft/jarvis) → Baikal;
|
||||
andere (Test-User `mcptest`) → Radicale-Fallback `http://127.0.0.1:5232` (Radicale laeuft weiter fuer Tests,
|
||||
mirror-first bis Geraete-Umstellung abgeschlossen). Radicale-URL/Creds/Pfade bleiben in `config.json`
|
||||
(`radicale_url`, `radicale_users`, `radicale_calendar_paths`, `radicale_addressbook_paths`).
|
||||
- **Files**: oCIS WebDAV on `127.0.0.1:9200/remote.php/dav/files/<user>/`. oCIS username may differ from MCP user (mapping in config).
|
||||
- **Notes**: Joplin **Data API** via joplin-cli sync clients on `127.0.0.1:41184` (stefan) / `41185` (kati). See `/opt/joplin-mcp/README.md`. NOT the Joplin Server sync API (that has no clean REST/search).
|
||||
- **Ghostfolio**: Ghostfolio-REST-API on `127.0.0.1:3333` (JWT-Token `/opt/ghostfolio/migration/.auth-token`). Read+write.
|
||||
- **ActualBudget**: Node-Sidecar (`/opt/actualbudget-mcp/sidecar`, systemd `actualbudget-sidecar`) on `127.0.0.1:5106` (RPC), der `@actual-app/api` kapselt und den ActualBudget-Server (Port 5006) anspricht.
|
||||
|
||||
## Files
|
||||
|
||||
- `common.py` — OAuth (authorize/token/metadata), Bearer middleware, user resolution (contextvars), config loader
|
||||
- `mail/server.py` — Maildir reader; tools: list_accounts, list_folders, search_mail, read_mail (lists attachments), read_attachment, create_draft
|
||||
- `calendar/server.py` — CalDAV REPORT + vobject; events/tasks CRUD, travel time, geocoding, reminders. Timed events/tasks werden mit TZID=Europe/Berlin (pytz) geschrieben (naive Eingaben lokalisiert, Offsets/Z konvertiert; Bugfix 2026-09-30, vorher floating -> Baikal las +2h). Delete/Update loesen den echten Ressourcenpfad per UID auf (`_find_href`), da der Dateiname nicht die UID sein muss (Importe wie medatixx).
|
||||
- `contacts/server.py` — CardDAV; search/get/create, contact photos
|
||||
- `files/server.py` — WebDAV; list/read/write/delete/move/search, images + documents
|
||||
- `notes/server.py` — Joplin Data API; list/search/read/create notes, list_note_resources, read_resource, add/update/delete_attachment
|
||||
- `ghostfolio/server.py` — Ghostfolio-REST-API; portfolio_details, get_performance, get_dividends, list_accounts, create_account, import_activities, list_activities, delete_activity. Schreib-Tools stoessen automatisch `update-report.sh` an -> accessible-dashboard-Report wird sofort neu gebaut (Details im ghostfolio/README)
|
||||
- `actualbudget/server.py` — Node-Sidecar-RPC; budgets/accounts/transactions/categories/budget/payees CRUD + aql_query
|
||||
- `tests/` — Schichten: `test_smoke.py` (NIGHTLY via `mcp-tests.timer`, nur up+Auth+1 Read/Connector), `test_edge.py`+`test_stress.py`+`test_all.py` (ON-DEMAND via `run_full_tests.sh`). Doku `tests/MCPTEST.md`
|
||||
|
||||
## Dependencies
|
||||
|
||||
```
|
||||
pip install mcp[cli] httpx vobject python-dateutil pytest
|
||||
```
|
||||
|
||||
Python 3.12+, venv at `./venv/`. joplin-cli (npm global) for Notes backend.
|
||||
|
||||
## Deployment
|
||||
|
||||
Each server is a systemd unit `mcp-<name>.service` (uvicorn), bound to 0.0.0.0. Zugriff intern via openclaw-net (172.26.0.1:<port>/mcp); einzelne Server zusaetzlich oeffentlich via nginx/TLS (notes/calendar, s. o.). Joplin: `joplin-cli-{stefan,kati}.service` + `joplin-sync.timer` (15 min).
|
||||
|
||||
## Content Types (returning binary)
|
||||
|
||||
- Text → `TextContent`
|
||||
- Images (jpg/png/gif/webp) → `ImageContent` (base64 inline)
|
||||
- PDFs → server-seitig zu Text extrahiert bzw. gescannte Seiten als PNG+OCR gerendert (`pdfutil.py`). claude.ai nimmt KEINE PDF/Dokument-Blobs aus Tool-Results nativ entgegen (nur Text + Bild). Andere Binaerdateien → `EmbeddedResource`/`BlobResourceContents` (Blob; wird von claude.ai nicht als Dokument verarbeitet)
|
||||
|
||||
## Robustheit / Limits
|
||||
|
||||
Alle Such-/List-Tools sind gegen grosse Datenmengen abgesichert (single-threaded Server darf nicht haengen): Mail per-Mail try/except + `DEADLINE_S`/`MAX_SCAN`; Files `search_files` `SEARCH_DEADLINE_S`/`SEARCH_MAX_DIRS`; Calendar/Contacts `PARSE_DEADLINE_S` + `MAX_RESULTS`-Output-Cap. Details + Messwerte: `tests/MCPTEST.md`.
|
||||
|
||||
## Adding a New Service
|
||||
|
||||
1. `<name>/server.py` with `FastMCP` instance + `create_app()` (copy mail/server.py pattern)
|
||||
2. Import `get_current_user`, `OAUTH_ROUTES`, `BearerAuthMiddleware`, `load_config` from `common.py`
|
||||
3. systemd unit `mcp-<name>.service` (bind 0.0.0.0) + iptables-Regel fuer den Port (nur openclaw-net 172.26.0.0/16) + URL in den Container-Configs eintragen
|
||||
4. systemd unit `mcp-<name>.service`
|
||||
5. Backend creds in `config.json`
|
||||
6. Tests: Smoke-Read in `test_smoke.py` + Edge in `test_edge.py` + Stress in `test_stress.py` (Funktional in `test_all.py`); Dashboard + dieses File aktualisieren
|
||||
|
||||
## Tool Design Guidelines
|
||||
|
||||
- Docstring is the first thing Claude sees — make it actionable ("Call this first", "Use the UID from search results")
|
||||
- `Annotated[str, Field(description="...")]` with example values for every parameter
|
||||
- Mark optional params ("Leave empty for all")
|
||||
- Reference other tools ("Use read_attachment with the index from read_mail")
|
||||
- Return structured text, not JSON
|
||||
- Include IDs/keys in output so Claude can chain to detail tools
|
||||
|
||||
## Test-/Dev-User mcptest
|
||||
|
||||
Die Test-Suite laeuft auf einem isolierten `mcptest`-User (NICHT Stefans Daten):
|
||||
`USER_ALIASES = {"test": "mcptest"}`. Eigene Backends: Radicale-User mcptest (Calendar/Contacts laufen ueber den
|
||||
Radicale-Fallback in `_serve(user)`, da es keinen Baikal-mcptest gibt), oCIS-User
|
||||
mcptest, lokales Joplin-Profil (:41186, `joplin-cli-mcptest.service`), Test-Maildir
|
||||
`tests/testdata/maildir/`. Dient auch als Dev-Sandbox. Vollstaendige Doku + Provisioning:
|
||||
`tests/MCPTEST.md`. Creds: `/root/.mcptest-creds` (600) + `config.json`.
|
||||
|
||||
## User nova (stefan@slohmaier.com, fuer OpenClaw-Agent Nova)
|
||||
|
||||
Nova (Agent im mycroft-Gateway) braucht Business-Mails statt Stefans privater .de-Mails.
|
||||
Dafuer:
|
||||
- tokens.json: User `nova` mit eigenem Client-Secret (token).
|
||||
- config.json mail_roots.nova -> `/mnt/ssd/Backup/stefan/nova-imap` (Symlink auf das
|
||||
.com-Maildir d370128_0-stefanlohmaier). Nova sieht damit NUR das .com-Postfach.
|
||||
- Im mycroft-Gateway gibt es einen zusaetzlichen MCP-Server `mail-com` (Bearer des nova-
|
||||
Users) auf demselben Port 5100. Nova nutzt mail-com, Mycroft nutzt `mail` (.de).
|
||||
- Zugriffstoken fuer nova per `mcp-token-refresh`-Mechanik oder manuell:
|
||||
POST /token client_credentials mit client_id=nova + secret aus tokens.json.
|
||||
|
||||
## Calendar: Namens-Aufloesung (exakt vor Teilstring)
|
||||
|
||||
Seit 2026-08-20 loest `_match_cols` Kalender-/Listen-Namen bei allen
|
||||
Schreib-/Loesch-Operationen (create_event, create_task, delete_event,
|
||||
delete_task) so auf: exakter Namens-Treffer zuerst, sonst Teilstring.
|
||||
Grund: Die Shared-Kalender "Kati Stefan" (beschreibbar) und "Kati Stefan
|
||||
JaSi" (read-only, Google-Sync) kollidierten bei Teilstring-Match — ein PUT
|
||||
auf "Kati Stefan" traf sonst den read-only JaSi-Kalender (HTTP 403), obwohl
|
||||
der User (kati/stefan) Schreibrechte auf den richtigen Kalender hat.
|
||||
|
||||
## Calendar: Loeschen
|
||||
|
||||
Der Calendar-MCP hat seit 2026-08-13 `delete_event` und `delete_task` (Loeschen per UID).
|
||||
UID kommt aus `get_events` / `get_tasks`. Loeschung direkt via CalDAV an das aktive Backend (Baikal fuer stefan/kati,
|
||||
Radicale-Fallback fuer mcptest).
|
||||
|
||||
## Mail-Drafts mit Anhang
|
||||
|
||||
`create_draft` unterstuetzt seit 2026-08-13 Anhaenge ueber Base64:
|
||||
- `attachment_filenames` (Liste), `attachment_contents` (Liste, Base64-kodierte Bytes),
|
||||
optional `attachment_mimetypes` (Liste; sonst application/octet-stream).
|
||||
- Der Agent erzeugt einen mehrteiligen (multipart) Draft im IMAP-Drafts-Ordner.
|
||||
- Test ok: "Entwurf erstellt mit 1 Anhang/Anhaengen".
|
||||
|
||||
## Notes: Anhaenge (Attachments)
|
||||
|
||||
Der Notes-MCP kann seit 2026-09-16 Anhaenge an Notizen anlegen, aendern und
|
||||
loeschen (vorher nur lesen: `list_note_resources` / `read_resource`):
|
||||
|
||||
- `add_attachment(note_id, filename, content_base64, mime?, title?)` — laedt die
|
||||
Datei via `POST /resources` hoch (multipart/form-data: Datei im Feld `data`,
|
||||
Properties als JSON im Feld `props`) und haengt `[title](:/resourceId)` ans
|
||||
Ende des Notiz-Bodys an (bei Bildern `![...]`, damit es inline rendert).
|
||||
Max 25 MB; Inhalt Base64 (data-URL-Prefix und Zeilenumbrueche erlaubt).
|
||||
- `update_attachment(resource_id, content_base64?, filename?, mime?, title?)` —
|
||||
ersetzt den Datei-Inhalt (`PUT /resources/:id`, multipart) und/oder aendert
|
||||
Titel/MIME (JSON-PUT, wenn kein Inhalt uebergeben wird).
|
||||
- `delete_attachment(resource_id, note_id?, remove_links?)` — loescht das
|
||||
Resource endgueltig (Joplin hat fuer Resources keinen Papierkorb) und entfernt
|
||||
die Markdown-/HTML-Referenzen aus dem Notiz-Body. `note_id` angeben, damit der
|
||||
Link sicher bereinigt wird: Joplins Endpoint `/resources/:id/notes` ist
|
||||
unzuverlaessig (Link-Tabelle kann leer bleiben, dann bliebe der Link verwaist).
|
||||
- MIME wird aus der Dateiendung geraten, wenn nicht uebergeben.
|
||||
- Getestet in `tests/test_all.py::TestNotes::test_attachment_crud` (mcptest).
|
||||
Reference in new issue
Block a user