Migrating an MCP-based memory setup from a local workstation to a central server sounded simple at first: move the data, expose the MCP endpoint securely, and point the clients to the new URL.
In practice, it was a valuable operational exercise. The migration touched Docker, systemd, Apache httpd, SELinux, DNS, TLS certificates, backup strategy, client configuration, and the Model Context Protocol itself.
The goal was clear: move our local AI memory service from a single Arch Linux workstation to a central server, so that multiple Codex clients — workstation and laptop — could use the same persistent memory.
The result is a centralized MCP memory service running on our server, reachable via HTTPS, backed by persistent storage, protected by a bearer token, and maintained through backups and systemd timers.
Starting Point
Initially, the memory service ran locally on the workstation.
The local setup looked roughly like this:
Codex on workstation
-> MCP over HTTP
-> http://127.0.0.1:3456/mcp
-> ai-rem Docker container
-> local persistent data directory
The local Codex configuration used an MCP server entry similar to this:
[mcp_servers.ai_rem]
url = "http://127.0.0.1:3456/mcp"
bearer_token_env_var = "AI_REM_API_TOKEN"
startup_timeout_sec = 20
tool_timeout_sec = 60
enabled = true
required = false
default_tools_approval_mode = "prompt"
The token itself was kept outside the Codex configuration:
# ~/.config/codex-memory/ai-rem.env
export AI_REM_API_TOKEN="local-token-value"
This was important from the beginning: the Codex config references the name of the environment variable, not the token value itself.
Why We Migrated
The local setup worked well, but it had one major limitation: it belonged to one machine.
If the workstation was off, the memory service was unavailable. If another device used Codex, it either had no memory or needed a separate local memory database. That would eventually lead to fragmentation.
We wanted one shared memory backend:
Codex on workstation
Codex on laptop
|
v
https://ai-rem.jr-it-services.de/mcp
|
v
central ai-rem service on server
|
v
persistent memory database
The main goals were:
- one central memory service
- access from multiple Codex clients
- HTTPS endpoint
- bearer token authentication
- no public exposure of the raw container port
- persistent server-side storage
- safe backup and restore process
- controlled update workflow
Target Architecture
The final architecture uses Apache httpd as the reverse proxy, because Apache was already running on the server for other web applications.
The target setup:
Codex client
-> HTTPS
-> https://ai-rem.jr-it-services.de/mcp
-> Apache httpd reverse proxy
-> http://127.0.0.1:3456/mcp
-> ai-rem Docker container
-> /srv/ai-rem/data
The public endpoint is HTTPS-only. The container port is bound only to localhost:
ports:
- "127.0.0.1:3456:3456"
This was an important security decision. Port 3456 is never exposed directly to the internet.
DNS First
Before configuring TLS, the DNS record had to exist.
We used:
ai-rem.jr-it-services.de -> 202.61.199.155
This could have been either an A record or a CNAME. We chose an explicit DNS record for clarity.
Verification was straightforward:
dig +short ai-rem.jr-it-services.de A
dig @1.1.1.1 +short ai-rem.jr-it-services.de A
dig @8.8.8.8 +short ai-rem.jr-it-services.de A
Expected result:
202.61.199.155
Only after DNS was working did we continue with Apache and Let’s Encrypt.
Running the Service with a Dedicated User
On the server we created a dedicated service user:
sudo useradd \
--system \
--home-dir /srv/ai-rem \
--shell /sbin/nologin \
--create-home \
--user-group \
airem
The service data lives under:
/srv/ai-rem
/srv/ai-rem/data
/srv/ai-rem/backups
Permissions:
sudo mkdir -p /srv/ai-rem/data /srv/ai-rem/backups
sudo chown -R airem:airem /srv/ai-rem
sudo chmod 750 /srv/ai-rem /srv/ai-rem/data /srv/ai-rem/backups
The Docker access was granted to the service user:
sudo usermod -aG docker airem
This is a pragmatic compromise. The Docker group is powerful, so the service user has no login shell and is only used for this controlled service.
Environment File Placement Matters
Initially, the environment file lived under /srv/ai-rem/.env. On Rocky Linux with systemd and SELinux, this caused a permission issue:
Failed to load environment files: Permission denied
The cleaner solution was to move the environment file to /etc:
/etc/ai-rem/ai-rem.env
We installed it like this:
sudo install -d -m 0750 -o root -g airem /etc/ai-rem
sudo install -m 0640 -o root -g airem \
/srv/ai-rem/.env \
/etc/ai-rem/ai-rem.env
The environment file contains the server-side token and runtime configuration:
AI_REM_API_TOKEN="server-token-value"
PORT=3456
MAX_BACKUPS=30
KUZU_POOL_SIZE=4
AI_REM_UID=...
AI_REM_GID=...
KG_PUBLIC_URL=https://ai-rem.jr-it-services.de
The token is not committed to source control, not copied into the Codex config, and not exposed in logs.
Docker Compose Setup
The service is run as a Docker container using Compose.
A simplified version of the final docker-compose.yml:
services:
ai-rem:
image: magic3arkus/ai-rem:latest
container_name: ai-rem
restart: unless-stopped
user: "${AI_REM_UID}:${AI_REM_GID}"
ports:
- "127.0.0.1:3456:3456"
volumes:
- ./data:/data:Z
- ./backups:/backups:Z
environment:
HOST: "0.0.0.0"
PORT: "3456"
KG_PUBLIC_URL: "${KG_PUBLIC_URL}"
KUZU_DB_PATH: "/data/kg.db"
BACKUP_DIR: "/backups"
MAX_BACKUPS: "${MAX_BACKUPS:-30}"
KUZU_POOL_SIZE: "${KUZU_POOL_SIZE:-4}"
AI_REM_API_TOKEN: "${AI_REM_API_TOKEN:?AI_REM_API_TOKEN is required}"
mem_limit: 2g
memswap_limit: 2g
security_opt:
- no-new-privileges:true
cap_drop:
- ALL
healthcheck:
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:3456/health',timeout=3)\""]
interval: 30s
timeout: 5s
retries: 3
start_period: 15s
logging:
driver: "json-file"
options:
max-size: "20m"
max-file: "10"
The :Z suffix on the bind mounts is important on SELinux systems. It allows Docker to relabel the mounted directories correctly.
systemd Service
We use systemd to manage the Compose-based service:
[Unit]
Description=ai-rem MCP memory server
Requires=docker.service
After=docker.service network-online.target
Wants=network-online.target
[Service]
Type=oneshot
RemainAfterExit=yes
User=airem
Group=airem
SupplementaryGroups=docker
WorkingDirectory=/srv/ai-rem
EnvironmentFile=/etc/ai-rem/ai-rem.env
ExecStartPre=/usr/bin/sh -c '/usr/bin/docker info >/dev/null 2>&1'
ExecStart=/usr/bin/docker compose --env-file /etc/ai-rem/ai-rem.env -f /srv/ai-rem/docker-compose.yml up -d
ExecStartPost=/usr/bin/sh -c 'sleep 2; for i in $(seq 1 30); do /usr/bin/curl -fsS http://127.0.0.1:3456/health >/dev/null 2>&1 && exit 0; sleep 1; done; exit 1'
ExecStop=/usr/bin/docker compose --env-file /etc/ai-rem/ai-rem.env -f /srv/ai-rem/docker-compose.yml down
TimeoutStartSec=120
TimeoutStopSec=60
[Install]
WantedBy=multi-user.target
Enable and start:
sudo systemctl daemon-reload
sudo systemctl enable --now ai-rem.service
Verification:
sudo systemctl status ai-rem.service --no-pager
docker ps --filter name=ai-rem
curl -fsS http://127.0.0.1:3456/health && echo
Expected result:
ok
Apache Reverse Proxy
The raw container port is local-only. Apache publishes the HTTPS endpoint.
The final Apache virtual host:
<VirtualHost *:80>
ServerName ai-rem.jr-it-services.de
RewriteEngine On
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [R=301,L]
</VirtualHost>
<VirtualHost *:443>
ServerName ai-rem.jr-it-services.de
SSLEngine on
SSLCertificateFile /etc/letsencrypt/live/ai-rem.jr-it-services.de/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/ai-rem.jr-it-services.de/privkey.pem
ProxyPreserveHost On
ProxyRequests Off
ProxyTimeout 600
RequestHeader unset Proxy early
ProxyPass / http://127.0.0.1:3456/ retry=0 timeout=600
ProxyPassReverse / http://127.0.0.1:3456/
ErrorLog /var/log/httpd/ai-rem-error.log
CustomLog /var/log/httpd/ai-rem-access.log combined
</VirtualHost>
SELinux required one additional setting:
sudo setsebool -P httpd_can_network_connect 1
Without this, Apache may be unable to connect to the local backend even if the backend itself is healthy.
The final checks:
curl -fsS http://127.0.0.1:3456/health && echo
curl -fsS https://ai-rem.jr-it-services.de/health && echo
Expected result:
ok
ok
Let’s Encrypt Bootstrap
Before the final reverse proxy was active, we used a temporary webroot virtual host for the ACME challenge.
Bootstrap test file:
sudo mkdir -p /var/www/ai-rem-bootstrap/.well-known/acme-challenge
echo "ok" | sudo tee /var/www/ai-rem-bootstrap/.well-known/acme-challenge/ping >/dev/null
Temporary Apache config:
<VirtualHost *:80>
ServerName ai-rem.jr-it-services.de
DocumentRoot /var/www/ai-rem-bootstrap
Alias "/.well-known/acme-challenge/" "/var/www/ai-rem-bootstrap/.well-known/acme-challenge/"
<Directory "/var/www/ai-rem-bootstrap">
Options None
AllowOverride None
Require all granted
</Directory>
<Directory "/var/www/ai-rem-bootstrap/.well-known/acme-challenge">
Options None
AllowOverride None
Require all granted
</Directory>
ErrorLog /var/log/httpd/ai-rem-bootstrap-error.log
CustomLog /var/log/httpd/ai-rem-bootstrap-access.log combined
</VirtualHost>
Verification:
curl -fsS http://ai-rem.jr-it-services.de/.well-known/acme-challenge/ping && echo
Expected result:
ok
Then the certificate could be obtained:
sudo certbot certonly \
--webroot \
-w /var/www/ai-rem-bootstrap \
-d ai-rem.jr-it-services.de \
--email johannes@jr-it-services.de \
--agree-tos \
--no-eff-email
One lesson here was that checking the ACME challenge path before running Certbot saves time. If the test file does not work, Certbot will not work either.
Exporting the Existing Memory
The local memory data was exported from the workstation.
The migration archive intentionally did not include secrets:
mkdir -p "$HOME/backups/codex-memory"
STAMP="$(date +%Y%m%d-%H%M%S)"
MIGRATION_DIR="$(mktemp -d)"
ARCHIVE="$HOME/backups/codex-memory/ai-rem-migration-$STAMP.tar.gz"
mkdir -p "$MIGRATION_DIR/payload/data"
sudo systemctl stop ai-rem.service 2>/dev/null || true
cp -a "$HOME/services/ai-rem/data" "$MIGRATION_DIR/payload/data/data"
cat > "$MIGRATION_DIR/manifest.txt" <<EOF
created_at=$(date --iso-8601=seconds)
source_host=$(cat /proc/sys/kernel/hostname 2>/dev/null || echo unknown)
source_path=$HOME/services/ai-rem/data
target_host=gemini.jr-it-services.de
target_path=/srv/ai-rem/data
purpose=ai-rem migration to server
contains_secrets=no
EOF
tar -C "$MIGRATION_DIR" -czf "$ARCHIVE" .
sha256sum "$ARCHIVE" > "$ARCHIVE.sha256"
rm -rf "$MIGRATION_DIR"
sudo systemctl start ai-rem.service 2>/dev/null || true
echo "$ARCHIVE"
Verification:
gzip -t "$ARCHIVE"
sha256sum -c "$ARCHIVE.sha256"
Copy to the server:
scp "$ARCHIVE" "$ARCHIVE.sha256" johannes@gemini.jr-it-services.de:/tmp/
Importing the Memory on the Server
On the server, the archive was moved to the backup directory:
sudo mkdir -p /var/backups/ai-rem
sudo chmod 700 /var/backups/ai-rem
sudo mv /tmp/ai-rem-migration-*.tar.gz /var/backups/ai-rem/
sudo mv /tmp/ai-rem-migration-*.tar.gz.sha256 /var/backups/ai-rem/ 2>/dev/null || true
sudo chmod 600 /var/backups/ai-rem/ai-rem-migration-*.tar.gz*
Because the checksum file had been generated on the workstation, it contained the original absolute path. We compared the hash values directly:
ARCHIVE="/var/backups/ai-rem/ai-rem-migration-20260817-130210.tar.gz"
SHAFILE="/var/backups/ai-rem/ai-rem-migration-20260817-130210.tar.gz.sha256"
EXPECTED="$(sudo awk '{print $1}' "$SHAFILE")"
ACTUAL="$(sudo sha256sum "$ARCHIVE" | awk '{print $1}')"
echo "EXPECTED=$EXPECTED"
echo "ACTUAL=$ACTUAL"
if [ "$EXPECTED" = "$ACTUAL" ]; then
echo "sha256 ok"
else
echo "sha256 mismatch - STOP"
exit 1
fi
Then we rewrote a server-local checksum file:
sudo bash -lc 'cd /var/backups/ai-rem && sha256sum ai-rem-migration-20260817-130210.tar.gz > ai-rem-migration-20260817-130210.tar.gz.sha256'
sudo chmod 600 /var/backups/ai-rem/ai-rem-migration-20260817-130210.tar.gz.sha256
sudo bash -lc 'cd /var/backups/ai-rem && sha256sum -c ai-rem-migration-20260817-130210.tar.gz.sha256'
The actual import:
MIGRATION_ARCHIVE="/var/backups/ai-rem/ai-rem-migration-20260817-130210.tar.gz"
sudo systemctl stop ai-rem.service
RESTORE_DIR="$(mktemp -d)"
sudo tar -xzf "$MIGRATION_ARCHIVE" -C "$RESTORE_DIR"
sudo test -d "$RESTORE_DIR/payload/data/data"
sudo mv /srv/ai-rem/data "/srv/ai-rem/data.before-import.$(date +%Y%m%d-%H%M%S)"
sudo cp -a "$RESTORE_DIR/payload/data/data" /srv/ai-rem/data
sudo chown -R airem:airem /srv/ai-rem/data
sudo restorecon -Rv /srv/ai-rem >/dev/null 2>&1 || true
sudo rm -rf "$RESTORE_DIR"
sudo systemctl start ai-rem.service
Verification:
curl -fsS http://127.0.0.1:3456/health && echo
curl -fsS https://ai-rem.jr-it-services.de/health && echo
docker ps --filter name=ai-rem
Expected result:
ok
ok
ai-rem ... healthy
Client Configuration
Each Codex client now points to the central MCP endpoint.
The environment file on each client:
# ~/.config/codex-memory/ai-rem.env
export AI_REM_API_TOKEN="server-token-value"
The Codex MCP configuration:
[mcp_servers.ai_rem]
url = "https://ai-rem.jr-it-services.de/mcp"
bearer_token_env_var = "AI_REM_API_TOKEN"
startup_timeout_sec = 20
tool_timeout_sec = 60
enabled = true
required = false
default_tools_approval_mode = "prompt"
This detail is easy to get wrong:
bearer_token_env_var = "AI_REM_API_TOKEN"
This field must contain the variable name, not the token value.
Wrong:
bearer_token_env_var = "actual-long-secret-token"
Right:
bearer_token_env_var = "AI_REM_API_TOKEN"
A quick shell check:
source "$HOME/.config/codex-memory/ai-rem.env"
[[ -n "${AI_REM_API_TOKEN:-}" ]] \
&& echo "TOKEN_SET=yes" \
|| echo "TOKEN_SET=no"
Codex validation:
/mcp
Expected result:
ai_rem
Auth: Bearer token
Tools: memory_add, memory_get_context, memory_relate, memory_search
Launcher Wrappers and Environment Variables
A subtle issue appeared when Codex was started from a desktop launcher instead of an interactive shell.
The shell had the token, but the launcher did not. The solution was a small wrapper script:
#!/usr/bin/env bash
set -euo pipefail
ENV_FILE="$HOME/.config/codex-memory/ai-rem.env"
if [ -f "$ENV_FILE" ]; then
. "$ENV_FILE"
fi
if [ -z "${AI_REM_API_TOKEN:-}" ]; then
echo "AI_REM_API_TOKEN is not set."
echo "Expected env file: $ENV_FILE"
echo
echo "Press Enter to close..."
read -r _ || true
exit 1
fi
exec /usr/bin/codex "$@"
The desktop entry then starts this wrapper instead of calling Codex directly.
This was one of the most practical lessons of the migration: graphical launchers do not necessarily inherit the same environment as an interactive shell.
Validation Inside Codex
After the migration, we validated that the imported memory was available:
Use ai_rem. Search memory for "Codex Memory Setup", "Johannes", and "ai-rem". Briefly summarize what you find.
The result showed entries from the previous local setup, proving that the import worked.
Then we stored a new central migration decision:
Use ai_rem. Store as a Decision in context work:
"The central Codex memory now runs on gemini.jr-it-services.de via ai-rem,
available at https://ai-rem.jr-it-services.de/mcp. The original local memory
data was imported from the Arch workstation setup."
This validated both read and write access.
Backup and Maintenance Framework
After the migration, we added a maintenance command:
sudo ai-rem-maint status
sudo ai-rem-maint backup --reason "post-migration-server-ok"
sudo ai-rem-maint list
sudo ai-rem-maint verify /var/backups/ai-rem/ai-rem-YYYYMMDD-HHMMSS.tar.gz
sudo ai-rem-maint update
The maintenance script performs safe backups by temporarily stopping the service, copying the persistent data, restarting the service, and writing a checksum.
A successful backup looked like this:
Backup created: /var/backups/ai-rem/ai-rem-20260817-134437.tar.gz
Checksum: /var/backups/ai-rem/ai-rem-20260817-134437.tar.gz.sha256
The update flow is intentionally conservative:
1. Create pre-update backup.
2. Tag current Docker image as fallback.
3. Pull new image.
4. Restart service.
5. Run healthcheck.
6. Roll back on failure.
For a memory service, safe updates matter more than fast updates.
systemd Timers
We added timers for backups, pruning, and health checks.
Daily backup:
[Unit]
Description=Create ai-rem backup
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/ai-rem-maint backup --reason scheduled-daily
[Unit]
Description=Daily ai-rem backup timer
[Timer]
OnCalendar=*-*-* 03:30:00
Persistent=true
RandomizedDelaySec=15m
[Install]
WantedBy=timers.target
Weekly pruning:
[Unit]
Description=Prune old ai-rem backups
[Service]
Type=oneshot
ExecStart=/usr/local/sbin/ai-rem-maint prune 30
[Unit]
Description=Weekly ai-rem backup pruning timer
[Timer]
OnCalendar=Sun *-*-* 04:15:00
Persistent=true
RandomizedDelaySec=30m
[Install]
WantedBy=timers.target
Healthcheck timer:
[Unit]
Description=Check ai-rem health
[Service]
Type=oneshot
ExecStart=/usr/bin/curl -fsS http://127.0.0.1:3456/health
[Unit]
Description=ai-rem healthcheck timer
[Timer]
OnCalendar=*:0/15
Persistent=true
[Install]
WantedBy=timers.target
Timer status:
sudo systemctl list-timers | grep ai-rem
This gives us scheduled backups, automatic cleanup, and regular operational checks without running unattended updates.
Problems We Hit
Several small issues appeared during the migration.
Environment file permission problem
The systemd service initially failed because the environment file was stored under /srv/ai-rem.
Fix:
Move runtime secrets/config to /etc/ai-rem/ai-rem.env.
Apache challenge path returned 404
The ACME challenge URL returned 404 until the bootstrap virtual host was made explicit with an Alias.
Fix:
Alias "/.well-known/acme-challenge/" "/var/www/ai-rem-bootstrap/.well-known/acme-challenge/"
Checksum file used the old path
The .sha256 file contained the workstation path.
Fix:
EXPECTED="$(sudo awk '{print $1}' "$SHAFILE")"
ACTUAL="$(sudo sha256sum "$ARCHIVE" | awk '{print $1}')"
Compare hashes instead of relying on the stored file path.
Desktop launcher did not load the token
Codex worked in the shell but failed from the launcher.
Fix:
Use a wrapper script that sources ~/.config/codex-memory/ai-rem.env before starting Codex.
bearer_token_env_var was misunderstood
The value must be the variable name, not the token itself.
Fix:
bearer_token_env_var = "AI_REM_API_TOKEN"
What Went Well
The migration worked because we validated each layer independently.
DNS was checked before TLS.
The local container health was checked before Apache.
Apache was tested with a simple ACME file before Certbot.
HTTPS health was checked before importing data.
The archive checksum was verified before restoring.
Codex MCP visibility was checked before disabling the local service.
This step-by-step approach made the migration much less stressful.
What We Would Do Again
We would use the same general pattern again:
1. Define the target architecture.
2. Create a dedicated service user.
3. Keep secrets outside the application directory.
4. Bind the container port only to localhost.
5. Put Apache or another reverse proxy in front.
6. Use HTTPS.
7. Verify DNS before TLS.
8. Export data without secrets.
9. Verify checksums before import.
10. Validate MCP access from the client.
11. Disable the old local service only after successful validation.
12. Add backup and maintenance automation.
The migration was successful because no single step was trusted blindly.
What We Would Improve
Next time, we would prepare the maintenance framework earlier. It would have been useful to have backup, restore, and update commands already available before the import.
We would also standardize the migration archive format from the beginning, including a server-independent checksum file.
Finally, we would document the client launcher behavior upfront. Environment handling between shell sessions and desktop launchers is easy to overlook, but it matters for token-based MCP services.
Résumé
The MCP memory migration turned a local workstation-bound setup into a central, shared service.
The final result is more useful and more maintainable:
- one memory backend for multiple Codex clients
- HTTPS access via Apache
- bearer-token authentication
- Dockerized service with persistent storage
- dedicated service user
- SELinux-aware deployment
- backup and restore workflow
- systemd timers for backup, pruning, and health checks
The most important lesson was that MCP itself was not the hard part. The hard part was the operational envelope around it: DNS, TLS, environment variables, service users, permissions, SELinux, backups, and client startup behavior.
A reliable AI memory service is not just a container with an HTTP endpoint. It is a small production service. It deserves the same discipline as any other service that stores valuable state.
In the end, the migration gave us more than shared memory for Codex. It gave us a cleaner operating model, better resilience, and a central foundation we can reuse from multiple machines.
Views: 0
