implementation of db
This commit is contained in:
southseact-3d
392
.opencode/plans/IMPLEMENTATION_PLAN_1.2_1.3.md
Normal file
392
.opencode/plans/IMPLEMENTATION_PLAN_1.2_1.3.md
Normal file
@@ -0,0 +1,392 @@
|
||||
# Implementation Plan: Phases 1.2 & 1.3
|
||||
## Database Encryption at Rest + Session Revocation
|
||||
|
||||
**Version:** 1.0
|
||||
**Date:** February 9, 2026
|
||||
**Estimated Total Effort:** 80 hours
|
||||
**Risk Level:** High (requires zero-downtime migration)
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
This plan details the implementation of **Phase 1.2** (Database with Encryption at Rest) and **Phase 1.3** (Session Revocation and Token Management) from the Security Remediation Plan.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1.2: Database with Encryption at Rest
|
||||
|
||||
### Overview
|
||||
Replace in-memory JSON file storage with SQLite database using SQLCipher for transparent encryption at rest.
|
||||
|
||||
**Current State:**
|
||||
- Data stored in `.data/` directory as JSON files (users.json, user-sessions.json, affiliates.json, etc.)
|
||||
- No encryption - plaintext readable by anyone with filesystem access
|
||||
- In-memory Maps with periodic persistence to JSON
|
||||
|
||||
**Target State:**
|
||||
- SQLite database with AES-256 encryption (SQLCipher)
|
||||
- All sensitive fields encrypted at application layer
|
||||
- Audit logging for all data access
|
||||
- Transaction support for data integrity
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
#### 1. Dependencies (4 hours)
|
||||
```bash
|
||||
cd chat/
|
||||
npm install better-sqlite3@^9.4.3
|
||||
npm install migrate@^9.2.1
|
||||
```
|
||||
|
||||
**Environment Variables:**
|
||||
```bash
|
||||
DATABASE_PATH=./.data/shopify_ai.db
|
||||
DATABASE_ENCRYPTION_KEY=<64-char-hex-key>
|
||||
DATABASE_BACKUP_ENABLED=1
|
||||
DATABASE_WAL_MODE=1
|
||||
```
|
||||
|
||||
#### 2. Database Schema
|
||||
Key tables:
|
||||
- `users` - User accounts with encrypted email, name, 2FA secrets
|
||||
- `sessions` - Active sessions for revocation
|
||||
- `refresh_tokens` - Refresh tokens with device fingerprinting
|
||||
- `token_blacklist` - Immediate token revocation
|
||||
- `affiliates`, `withdrawals`, `feature_requests`, `contact_messages`
|
||||
- `audit_log` - Comprehensive audit trail
|
||||
|
||||
#### 3. Encryption Layer
|
||||
- **Algorithm:** AES-256-GCM with authenticated encryption
|
||||
- **Key Derivation:** PBKDF2 with 100,000 iterations
|
||||
- **Per-field encryption:** Sensitive fields encrypted individually
|
||||
- **Token hashing:** PBKDF2 for secure token storage
|
||||
|
||||
#### 4. Data Migration Strategy
|
||||
**Zero-downtime approach:**
|
||||
1. Create SQLite database alongside existing JSON files
|
||||
2. Run migration script to copy and encrypt data
|
||||
3. Deploy new code with backward compatibility layer
|
||||
4. Switch to database mode via environment variable
|
||||
5. Keep JSON files as backup for 30 days
|
||||
|
||||
**Migration Script:** `scripts/migrate-to-database.js`
|
||||
- Backs up all JSON files
|
||||
- Transforms and encrypts sensitive fields
|
||||
- Inserts into database with verification
|
||||
- Generates migration report
|
||||
|
||||
#### 5. Backward Compatibility
|
||||
**File:** `src/database/compat.js`
|
||||
- Dual-mode operation (JSON or Database)
|
||||
- Same API for both modes
|
||||
- Environment variable `USE_JSON_DATABASE` controls mode
|
||||
- Allows gradual rollout and easy rollback
|
||||
|
||||
---
|
||||
|
||||
## Phase 1.3: Session Revocation and Token Management
|
||||
|
||||
### Overview
|
||||
Replace simple UUID session tokens with JWT access tokens + refresh tokens.
|
||||
|
||||
**Current State:**
|
||||
- Simple UUID tokens stored in memory/JSON
|
||||
- No expiration on tokens (only cookie expiration)
|
||||
- Cannot revoke individual sessions
|
||||
- No device tracking
|
||||
|
||||
**Target State:**
|
||||
- Short-lived JWT access tokens (15 minutes)
|
||||
- Long-lived refresh tokens (7 days) stored in database
|
||||
- Ability to revoke specific sessions or all sessions
|
||||
- Device fingerprinting for security
|
||||
|
||||
### Implementation Steps
|
||||
|
||||
#### 1. JWT Token Manager
|
||||
**File:** `src/utils/tokenManager.js`
|
||||
|
||||
**Access Token (JWT):**
|
||||
- Expires: 15 minutes
|
||||
- Contains: user ID, email, role, plan, unique JTI
|
||||
- Signed with JWT_SECRET
|
||||
|
||||
**Refresh Token:**
|
||||
- Random 128-byte hex string
|
||||
- Stored hashed in database
|
||||
- Expires: 7 days
|
||||
- Linked to device fingerprint
|
||||
|
||||
**Key Features:**
|
||||
- Token rotation on refresh
|
||||
- Device fingerprint verification
|
||||
- Blacklist for immediate access token revocation
|
||||
- Automatic cleanup of expired tokens
|
||||
|
||||
#### 2. Device Fingerprinting
|
||||
```javascript
|
||||
function generateFingerprint(req) {
|
||||
const components = [
|
||||
req.headers['user-agent'],
|
||||
req.headers['accept-language'],
|
||||
req.ip,
|
||||
req.headers['x-forwarded-for']
|
||||
];
|
||||
return crypto.createHash('sha256')
|
||||
.update(components.join('|'))
|
||||
.digest('hex').substring(0, 32);
|
||||
}
|
||||
```
|
||||
|
||||
If fingerprint mismatch detected → revoke all tokens (potential theft).
|
||||
|
||||
#### 3. Updated Authentication Flow
|
||||
|
||||
**Login:**
|
||||
1. Validate credentials
|
||||
2. Check account lockout status
|
||||
3. Check 2FA if enabled
|
||||
4. Generate JWT access token (15 min)
|
||||
5. Generate refresh token (7 days)
|
||||
6. Store refresh token hash in database
|
||||
7. Set HTTP-only cookies
|
||||
|
||||
**Token Refresh:**
|
||||
1. Verify refresh token hash in database
|
||||
2. Verify device fingerprint
|
||||
3. Mark old refresh token as used
|
||||
4. Generate new access + refresh tokens
|
||||
5. Return new tokens
|
||||
|
||||
**Logout:**
|
||||
1. Revoke refresh token in database
|
||||
2. Clear cookies
|
||||
3. Optional: Blacklist access token
|
||||
|
||||
**Logout All:**
|
||||
1. Revoke all refresh tokens for user
|
||||
2. Clear cookies
|
||||
3. Blacklist all active access tokens
|
||||
|
||||
#### 4. New API Endpoints
|
||||
|
||||
```
|
||||
POST /auth/login → Returns access + refresh tokens
|
||||
POST /auth/refresh → Rotate tokens
|
||||
POST /auth/logout → Revoke current session
|
||||
POST /auth/logout-all → Revoke all sessions
|
||||
GET /auth/sessions → List active sessions
|
||||
DELETE /auth/sessions/:id → Revoke specific session
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Transfer Process
|
||||
|
||||
### Pre-Migration
|
||||
1. **Backup Creation:**
|
||||
```bash
|
||||
tar -czf backup-$(date +%Y%m%d-%H%M%S).tar.gz .data/
|
||||
```
|
||||
|
||||
2. **Environment Setup:**
|
||||
```bash
|
||||
export USE_JSON_DATABASE=1 # Keep using JSON initially
|
||||
export DATABASE_ENCRYPTION_KEY=$(openssl rand -hex 32)
|
||||
```
|
||||
|
||||
### Migration Execution
|
||||
```bash
|
||||
# 1. Setup database schema
|
||||
node scripts/setup-database.js
|
||||
|
||||
# 2. Run migration
|
||||
node scripts/migrate-to-database.js
|
||||
|
||||
# 3. Verify data integrity
|
||||
node scripts/verify-migration.js
|
||||
```
|
||||
|
||||
### Post-Migration
|
||||
```bash
|
||||
# 1. Switch to database mode
|
||||
unset USE_JSON_DATABASE
|
||||
|
||||
# 2. Restart application
|
||||
pm2 restart server
|
||||
|
||||
# 3. Monitor logs
|
||||
tail -f logs/server.log
|
||||
```
|
||||
|
||||
### Rollback (if needed)
|
||||
```bash
|
||||
# 1. Switch back to JSON mode
|
||||
export USE_JSON_DATABASE=1
|
||||
|
||||
# 2. Restart application
|
||||
pm2 restart server
|
||||
|
||||
# All data still in JSON files, no data loss
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
chat/
|
||||
├── src/
|
||||
│ ├── database/
|
||||
│ │ ├── connection.js # Database connection
|
||||
│ │ ├── schema.sql # Database schema
|
||||
│ │ └── compat.js # Backward compatibility
|
||||
│ ├── repositories/
|
||||
│ │ ├── userRepository.js # User data access
|
||||
│ │ ├── sessionRepository.js # Session data access
|
||||
│ │ └── index.js # Repository exports
|
||||
│ ├── utils/
|
||||
│ │ ├── encryption.js # Field-level encryption
|
||||
│ │ └── tokenManager.js # JWT + refresh tokens
|
||||
│ ├── middleware/
|
||||
│ │ └── auth.js # Updated auth middleware
|
||||
│ └── routes/
|
||||
│ └── auth.js # New auth endpoints
|
||||
├── scripts/
|
||||
│ ├── setup-database.js # Initial schema setup
|
||||
│ ├── migrate-to-database.js # Data migration
|
||||
│ └── verify-migration.js # Integrity verification
|
||||
└── .data/
|
||||
├── shopify_ai.db # Encrypted SQLite database
|
||||
└── migration_backup_*/ # Backup directories
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Features
|
||||
|
||||
### Encryption at Rest
|
||||
- **Database-level:** SQLCipher AES-256 encryption
|
||||
- **Field-level:** Sensitive fields individually encrypted
|
||||
- **Key management:** Master key derived via PBKDF2
|
||||
- **Token storage:** Hashed with PBKDF2 (not encrypted)
|
||||
|
||||
### Session Security
|
||||
- **Short-lived tokens:** 15-minute access tokens
|
||||
- **Token rotation:** New refresh token on every use
|
||||
- **Device binding:** Tokens bound to device fingerprint
|
||||
- **Theft detection:** Automatic revocation on fingerprint mismatch
|
||||
- **Immediate revocation:** Token blacklist for instant logout
|
||||
|
||||
### Audit Trail
|
||||
- All logins/logouts logged
|
||||
- Token refresh events logged
|
||||
- Session revocations logged
|
||||
- Data access logged
|
||||
- IP address and user agent captured
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
- Encryption/decryption round-trip
|
||||
- Token generation and verification
|
||||
- Device fingerprint generation
|
||||
- Password hashing and verification
|
||||
|
||||
### Integration Tests
|
||||
- Login flow with JWT
|
||||
- Token refresh rotation
|
||||
- Session revocation
|
||||
- Device fingerprint mismatch
|
||||
- Migration script with test data
|
||||
|
||||
### Security Tests
|
||||
- Token theft simulation
|
||||
- Replay attack prevention
|
||||
- Session fixation protection
|
||||
- Encryption key rotation
|
||||
|
||||
### Load Tests
|
||||
- Database query performance
|
||||
- Token generation under load
|
||||
- Concurrent session handling
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Alerting
|
||||
|
||||
### Metrics
|
||||
- Database query latency (p95, p99)
|
||||
- Authentication success rate
|
||||
- Token refresh frequency
|
||||
- Session revocation count
|
||||
- Device fingerprint mismatches
|
||||
- Database size growth
|
||||
|
||||
### Alerts
|
||||
- Database connection failures
|
||||
- Migration errors
|
||||
- High authentication failure rate
|
||||
- Unusual token revocation patterns
|
||||
- Encryption errors
|
||||
|
||||
---
|
||||
|
||||
## Timeline
|
||||
|
||||
| Phase | Task | Hours |
|
||||
|-------|------|-------|
|
||||
| 1.2 | Dependencies & setup | 4 |
|
||||
| 1.2 | Database schema | 6 |
|
||||
| 1.2 | Encryption layer | 8 |
|
||||
| 1.2 | Repositories | 12 |
|
||||
| 1.2 | Migration script | 8 |
|
||||
| 1.2 | Backward compatibility | 6 |
|
||||
| 1.2 | Testing | 10 |
|
||||
| 1.3 | Token manager | 8 |
|
||||
| 1.3 | Auth middleware | 6 |
|
||||
| 1.3 | API endpoints | 6 |
|
||||
| 1.3 | Testing | 6 |
|
||||
| Both | Deployment & rollback testing | 10 |
|
||||
| **Total** | | **90 hours** |
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| Data loss | Multiple backups, transaction support, rollback capability |
|
||||
| Migration failure | Backward compatibility layer, easy rollback |
|
||||
| Performance issues | WAL mode, indexing, query optimization |
|
||||
| Security vulnerabilities | Security review, penetration testing |
|
||||
| Downtime | Zero-downtime migration strategy |
|
||||
| Key compromise | Key rotation plan, secure key storage |
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] All data migrated without loss
|
||||
- [ ] Encryption verified (data unreadable without key)
|
||||
- [ ] Sessions can be revoked individually and globally
|
||||
- [ ] Token rotation working correctly
|
||||
- [ ] Device fingerprinting detecting mismatches
|
||||
- [ ] Rollback tested and working
|
||||
- [ ] Performance meets or exceeds JSON storage
|
||||
- [ ] Audit logging capturing all security events
|
||||
|
||||
---
|
||||
|
||||
## Next Steps
|
||||
|
||||
1. **Review this plan** with security team and stakeholders
|
||||
2. **Set up staging environment** with production-like data
|
||||
3. **Implement Phase 1.2** (database + encryption)
|
||||
4. **Test migration** on staging
|
||||
5. **Implement Phase 1.3** (token management)
|
||||
6. **Security review** and penetration testing
|
||||
7. **Deploy to production** with monitoring
|
||||
Reference in New Issue
Block a user