shopify-ai-backup/CONTAINER_LOGGING.md

# Container Diagnostics and Logging

## Overview
This application includes comprehensive container diagnostics and logging to help troubleshoot issues in production.

## Log Files

### Diagnostic Logs
Located in: `/var/log/shopify-ai/`

Files:
- `diagnostics.log` - System startup, configuration, and runtime diagnostics
- `healthcheck.log` - Health check results and service status

### Accessing Logs

#### Via Docker
```bash
# View diagnostic logs
docker logs -f shopify-ai-builder

# Follow log volume
docker exec -it shopify-ai-builder tail -f /var/log/shopify-ai/diagnostics.log

# View health check results
docker exec -it shopify-ai-builder cat /var/log/shopify-ai/healthcheck.log
```

#### Via Named Volume
```bash
# Access logs volume
docker run --rm -v shopify_ai_logs:/data alpine cat /data/diagnostics.log

# Copy logs to local machine
docker run --rm -v shopify_ai_logs:/logs -v $(pwd):/output alpine cp -a /logs /output/
```

## Log Levels

Logs use the following severity levels:
- **INFO** - Normal operation and informational messages
- **WARN** - Warning messages (non-critical issues)
- **ERROR** - Error messages (critical failures)
- **DEBUG** - Detailed debugging information

## What's Logged

### Startup Diagnostics
- Container ID and hostname
- OS and kernel information
- CPU count and model
- Total/used/available memory
- Disk usage
- Network interfaces and IP addresses
- Environment variable validation
- Filesystem permissions

### Runtime Monitoring
Every 2 minutes:
- CPU usage percentage
- Memory usage (used/total/percentage)
- Disk usage percentage
- System load average

Every 5 minutes:
- Chat service status (port 4000)
- TTYD service status (port 4001)
- Process health checks
- HTTP endpoint responsiveness

### Service Health Checks
- Port listening status
- HTTP endpoint response
- Process memory and CPU usage
- Process uptime

## Troubleshooting

### High Memory Usage
If logs show `⚠ High memory usage`:
1. Check current usage: `docker stats shopify-ai-builder`
2. Review recent message sizes
3. Check for memory leaks in server.js
4. Consider increasing container memory limits

### Service Not Responding
If health check fails:
1. View diagnostic logs for errors
2. Check if ports are accessible: `netstat -tuln | grep -E ':(4000|4001)'`
3. Check process status: `ps aux | grep -E 'node|ttyd'`
4. Restart container: `docker restart shopify-ai-builder`

### Disk Space Issues
If logs show `⚠ High disk usage`:
1. Check workspace size: `du -sh /home/web/data`
2. Clean up old sessions
3. Check log file sizes: `du -sh /var/log/shopify-ai`
4. Rotate logs manually if needed

## Log Rotation

Diagnostic logs are automatically rotated when they exceed 10 MB:
- Original file is backed up with timestamp: `diagnostics.log.20250114_120000.bak`
- New log file is started
- Old logs are retained until manually cleaned

## Enhancing Logging

To add custom logging:

In `entrypoint.sh`:
```bash
log "Your custom message"

# With diagnostic logger
if type diag_log &>/dev/null; then
    diag_log "INFO" "Your diagnostic message"
fi
```

In `healthcheck.sh`:
```bash
health_log "INFO" "Your health check message"
```

## Monitoring Dashboard

For a real-time monitoring dashboard:

```bash
# Follow diagnostic logs with color highlighting
docker exec -it shopify-ai-builder tail -f /var/log/shopify-ai/diagnostics.log | grep --color=auto -E 'ERROR|WARN|INFO'

# Monitor all logs
docker logs -f shopify-ai-builder 2>&1 | grep --color=auto -E 'ERROR|WARN|usage|tokens'
```

## Common Issues and Log Patterns

### Token Usage Tracking
Look for: `[USAGE]` tags
```
[2025-01-14 10:30:00] [INFO] [USAGE] Usage summary loaded: {...}
[2025-01-14 10:30:05] [INFO] [USAGE] Started aggressive usage polling
[2025-01-14 10:30:10] [INFO] [USAGE] Stopped usage polling
```

### Service Startup
Look for service startup messages:
```
[2025-01-14 10:00:00] [INFO] Chat service started with PID: 123
[2025-01-14 10:00:02] [INFO] ✓ chat: Port 4000 listening
[2025-01-14 10:00:02] [INFO] ✓ chat: HTTP endpoint responding
```

### Resource Issues
Look for warning markers:
```
[2025-01-14 10:00:00] [WARN] ⚠ High memory usage: 95%
[2025-01-14 10:00:00] [WARN] ⚠ High disk usage: 85%
```

## Export Logs for Support

To export all logs for debugging:

```bash
# Create a logs export directory
mkdir -p ./logs-export
docker cp shopify-ai-builder:/var/log/shopify-ai/. ./logs-export/

# Export Docker container logs
docker logs shopify-ai-builder > ./logs-export/container-logs.txt 2>&1

# Create a compressed archive
tar -czf logs-export-$(date +%Y%m%d).tar.gz logs-export/

# Send to support
# Attach logs-export-20250114.tar.gz to your support ticket
```