shopify-ai-backup/CONTAINER_LOGGING.md

Container Diagnostics and Logging

Overview

This application includes comprehensive container diagnostics and logging to help troubleshoot issues in production.

Log Files

Diagnostic Logs

Located in: /var/log/shopify-ai/

Files:

• diagnostics.log - System startup, configuration, and runtime diagnostics
• healthcheck.log - Health check results and service status

Accessing Logs

Via Docker

# View diagnostic logs
docker logs -f shopify-ai-builder

# Follow log volume
docker exec -it shopify-ai-builder tail -f /var/log/shopify-ai/diagnostics.log

# View health check results
docker exec -it shopify-ai-builder cat /var/log/shopify-ai/healthcheck.log

Via Named Volume

# Access logs volume
docker run --rm -v shopify_ai_logs:/data alpine cat /data/diagnostics.log

# Copy logs to local machine
docker run --rm -v shopify_ai_logs:/logs -v $(pwd):/output alpine cp -a /logs /output/

Log Levels

Logs use the following severity levels:

• INFO - Normal operation and informational messages
• WARN - Warning messages (non-critical issues)
• ERROR - Error messages (critical failures)
• DEBUG - Detailed debugging information

What's Logged

Startup Diagnostics

• Container ID and hostname
• OS and kernel information
• CPU count and model
• Total/used/available memory
• Disk usage
• Network interfaces and IP addresses
• Environment variable validation
• Filesystem permissions

Runtime Monitoring

Every 2 minutes:

• CPU usage percentage
• Memory usage (used/total/percentage)
• Disk usage percentage
• System load average

Every 5 minutes:

• Chat service status (port 4000)
• TTYD service status (port 4001)
• Process health checks
• HTTP endpoint responsiveness

Service Health Checks

• Port listening status
• HTTP endpoint response
• Process memory and CPU usage
• Process uptime

Troubleshooting

High Memory Usage

If logs show ⚠ High memory usage:

1. Check current usage: docker stats shopify-ai-builder
2. Review recent message sizes
3. Check for memory leaks in server.js
4. Consider increasing container memory limits

Service Not Responding

If health check fails:

1. View diagnostic logs for errors
2. Check if ports are accessible: netstat -tuln | grep -E ':(4000|4001)'
3. Check process status: ps aux | grep -E 'node|ttyd'
4. Restart container: docker restart shopify-ai-builder

Disk Space Issues

If logs show ⚠ High disk usage:

1. Check workspace size: du -sh /home/web/data
2. Clean up old sessions
3. Check log file sizes: du -sh /var/log/shopify-ai
4. Rotate logs manually if needed

Log Rotation

Diagnostic logs are automatically rotated when they exceed 10 MB:

• Original file is backed up with timestamp: diagnostics.log.20250114_120000.bak
• New log file is started
• Old logs are retained until manually cleaned

Enhancing Logging

To add custom logging:

In entrypoint.sh:

log "Your custom message"

# With diagnostic logger
if type diag_log &>/dev/null; then
    diag_log "INFO" "Your diagnostic message"
fi

In healthcheck.sh:

health_log "INFO" "Your health check message"

Monitoring Dashboard

For a real-time monitoring dashboard:

# Follow diagnostic logs with color highlighting
docker exec -it shopify-ai-builder tail -f /var/log/shopify-ai/diagnostics.log | grep --color=auto -E 'ERROR|WARN|INFO'

# Monitor all logs
docker logs -f shopify-ai-builder 2>&1 | grep --color=auto -E 'ERROR|WARN|usage|tokens'

Common Issues and Log Patterns

Token Usage Tracking

Look for: [USAGE] tags

[2025-01-14 10:30:00] [INFO] [USAGE] Usage summary loaded: {...}
[2025-01-14 10:30:05] [INFO] [USAGE] Started aggressive usage polling
[2025-01-14 10:30:10] [INFO] [USAGE] Stopped usage polling

Service Startup

Look for service startup messages:

[2025-01-14 10:00:00] [INFO] Chat service started with PID: 123
[2025-01-14 10:00:02] [INFO] ✓ chat: Port 4000 listening
[2025-01-14 10:00:02] [INFO] ✓ chat: HTTP endpoint responding

Resource Issues

Look for warning markers:

[2025-01-14 10:00:00] [WARN] ⚠ High memory usage: 95%
[2025-01-14 10:00:00] [WARN] ⚠ High disk usage: 85%

Export Logs for Support

To export all logs for debugging:

# Create a logs export directory
mkdir -p ./logs-export
docker cp shopify-ai-builder:/var/log/shopify-ai/. ./logs-export/

# Export Docker container logs
docker logs shopify-ai-builder > ./logs-export/container-logs.txt 2>&1

# Create a compressed archive
tar -czf logs-export-$(date +%Y%m%d).tar.gz logs-export/

# Send to support
# Attach logs-export-20250114.tar.gz to your support ticket