implementation of db
This commit is contained in:
southseact-3d
392
.opencode/plans/IMPLEMENTATION_PLAN_1.2_1.3.md
Normal file
392
.opencode/plans/IMPLEMENTATION_PLAN_1.2_1.3.md
Normal file
@@ -0,0 +1,392 @@
|
|||||||
|
# Implementation Plan: Phases 1.2 & 1.3
|
||||||
|
## Database Encryption at Rest + Session Revocation
|
||||||
|
|
||||||
|
**Version:** 1.0
|
||||||
|
**Date:** February 9, 2026
|
||||||
|
**Estimated Total Effort:** 80 hours
|
||||||
|
**Risk Level:** High (requires zero-downtime migration)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Executive Summary
|
||||||
|
|
||||||
|
This plan details the implementation of **Phase 1.2** (Database with Encryption at Rest) and **Phase 1.3** (Session Revocation and Token Management) from the Security Remediation Plan.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1.2: Database with Encryption at Rest
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
Replace in-memory JSON file storage with SQLite database using SQLCipher for transparent encryption at rest.
|
||||||
|
|
||||||
|
**Current State:**
|
||||||
|
- Data stored in `.data/` directory as JSON files (users.json, user-sessions.json, affiliates.json, etc.)
|
||||||
|
- No encryption - plaintext readable by anyone with filesystem access
|
||||||
|
- In-memory Maps with periodic persistence to JSON
|
||||||
|
|
||||||
|
**Target State:**
|
||||||
|
- SQLite database with AES-256 encryption (SQLCipher)
|
||||||
|
- All sensitive fields encrypted at application layer
|
||||||
|
- Audit logging for all data access
|
||||||
|
- Transaction support for data integrity
|
||||||
|
|
||||||
|
### Implementation Steps
|
||||||
|
|
||||||
|
#### 1. Dependencies (4 hours)
|
||||||
|
```bash
|
||||||
|
cd chat/
|
||||||
|
npm install better-sqlite3@^9.4.3
|
||||||
|
npm install migrate@^9.2.1
|
||||||
|
```
|
||||||
|
|
||||||
|
**Environment Variables:**
|
||||||
|
```bash
|
||||||
|
DATABASE_PATH=./.data/shopify_ai.db
|
||||||
|
DATABASE_ENCRYPTION_KEY=<64-char-hex-key>
|
||||||
|
DATABASE_BACKUP_ENABLED=1
|
||||||
|
DATABASE_WAL_MODE=1
|
||||||
|
```
|
||||||
|
|
||||||
|
#### 2. Database Schema
|
||||||
|
Key tables:
|
||||||
|
- `users` - User accounts with encrypted email, name, 2FA secrets
|
||||||
|
- `sessions` - Active sessions for revocation
|
||||||
|
- `refresh_tokens` - Refresh tokens with device fingerprinting
|
||||||
|
- `token_blacklist` - Immediate token revocation
|
||||||
|
- `affiliates`, `withdrawals`, `feature_requests`, `contact_messages`
|
||||||
|
- `audit_log` - Comprehensive audit trail
|
||||||
|
|
||||||
|
#### 3. Encryption Layer
|
||||||
|
- **Algorithm:** AES-256-GCM with authenticated encryption
|
||||||
|
- **Key Derivation:** PBKDF2 with 100,000 iterations
|
||||||
|
- **Per-field encryption:** Sensitive fields encrypted individually
|
||||||
|
- **Token hashing:** PBKDF2 for secure token storage
|
||||||
|
|
||||||
|
#### 4. Data Migration Strategy
|
||||||
|
**Zero-downtime approach:**
|
||||||
|
1. Create SQLite database alongside existing JSON files
|
||||||
|
2. Run migration script to copy and encrypt data
|
||||||
|
3. Deploy new code with backward compatibility layer
|
||||||
|
4. Switch to database mode via environment variable
|
||||||
|
5. Keep JSON files as backup for 30 days
|
||||||
|
|
||||||
|
**Migration Script:** `scripts/migrate-to-database.js`
|
||||||
|
- Backs up all JSON files
|
||||||
|
- Transforms and encrypts sensitive fields
|
||||||
|
- Inserts into database with verification
|
||||||
|
- Generates migration report
|
||||||
|
|
||||||
|
#### 5. Backward Compatibility
|
||||||
|
**File:** `src/database/compat.js`
|
||||||
|
- Dual-mode operation (JSON or Database)
|
||||||
|
- Same API for both modes
|
||||||
|
- Environment variable `USE_JSON_DATABASE` controls mode
|
||||||
|
- Allows gradual rollout and easy rollback
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Phase 1.3: Session Revocation and Token Management
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
Replace simple UUID session tokens with JWT access tokens + refresh tokens.
|
||||||
|
|
||||||
|
**Current State:**
|
||||||
|
- Simple UUID tokens stored in memory/JSON
|
||||||
|
- No expiration on tokens (only cookie expiration)
|
||||||
|
- Cannot revoke individual sessions
|
||||||
|
- No device tracking
|
||||||
|
|
||||||
|
**Target State:**
|
||||||
|
- Short-lived JWT access tokens (15 minutes)
|
||||||
|
- Long-lived refresh tokens (7 days) stored in database
|
||||||
|
- Ability to revoke specific sessions or all sessions
|
||||||
|
- Device fingerprinting for security
|
||||||
|
|
||||||
|
### Implementation Steps
|
||||||
|
|
||||||
|
#### 1. JWT Token Manager
|
||||||
|
**File:** `src/utils/tokenManager.js`
|
||||||
|
|
||||||
|
**Access Token (JWT):**
|
||||||
|
- Expires: 15 minutes
|
||||||
|
- Contains: user ID, email, role, plan, unique JTI
|
||||||
|
- Signed with JWT_SECRET
|
||||||
|
|
||||||
|
**Refresh Token:**
|
||||||
|
- Random 128-byte hex string
|
||||||
|
- Stored hashed in database
|
||||||
|
- Expires: 7 days
|
||||||
|
- Linked to device fingerprint
|
||||||
|
|
||||||
|
**Key Features:**
|
||||||
|
- Token rotation on refresh
|
||||||
|
- Device fingerprint verification
|
||||||
|
- Blacklist for immediate access token revocation
|
||||||
|
- Automatic cleanup of expired tokens
|
||||||
|
|
||||||
|
#### 2. Device Fingerprinting
|
||||||
|
```javascript
|
||||||
|
function generateFingerprint(req) {
|
||||||
|
const components = [
|
||||||
|
req.headers['user-agent'],
|
||||||
|
req.headers['accept-language'],
|
||||||
|
req.ip,
|
||||||
|
req.headers['x-forwarded-for']
|
||||||
|
];
|
||||||
|
return crypto.createHash('sha256')
|
||||||
|
.update(components.join('|'))
|
||||||
|
.digest('hex').substring(0, 32);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
If fingerprint mismatch detected → revoke all tokens (potential theft).
|
||||||
|
|
||||||
|
#### 3. Updated Authentication Flow
|
||||||
|
|
||||||
|
**Login:**
|
||||||
|
1. Validate credentials
|
||||||
|
2. Check account lockout status
|
||||||
|
3. Check 2FA if enabled
|
||||||
|
4. Generate JWT access token (15 min)
|
||||||
|
5. Generate refresh token (7 days)
|
||||||
|
6. Store refresh token hash in database
|
||||||
|
7. Set HTTP-only cookies
|
||||||
|
|
||||||
|
**Token Refresh:**
|
||||||
|
1. Verify refresh token hash in database
|
||||||
|
2. Verify device fingerprint
|
||||||
|
3. Mark old refresh token as used
|
||||||
|
4. Generate new access + refresh tokens
|
||||||
|
5. Return new tokens
|
||||||
|
|
||||||
|
**Logout:**
|
||||||
|
1. Revoke refresh token in database
|
||||||
|
2. Clear cookies
|
||||||
|
3. Optional: Blacklist access token
|
||||||
|
|
||||||
|
**Logout All:**
|
||||||
|
1. Revoke all refresh tokens for user
|
||||||
|
2. Clear cookies
|
||||||
|
3. Blacklist all active access tokens
|
||||||
|
|
||||||
|
#### 4. New API Endpoints
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /auth/login → Returns access + refresh tokens
|
||||||
|
POST /auth/refresh → Rotate tokens
|
||||||
|
POST /auth/logout → Revoke current session
|
||||||
|
POST /auth/logout-all → Revoke all sessions
|
||||||
|
GET /auth/sessions → List active sessions
|
||||||
|
DELETE /auth/sessions/:id → Revoke specific session
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Data Transfer Process
|
||||||
|
|
||||||
|
### Pre-Migration
|
||||||
|
1. **Backup Creation:**
|
||||||
|
```bash
|
||||||
|
tar -czf backup-$(date +%Y%m%d-%H%M%S).tar.gz .data/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Environment Setup:**
|
||||||
|
```bash
|
||||||
|
export USE_JSON_DATABASE=1 # Keep using JSON initially
|
||||||
|
export DATABASE_ENCRYPTION_KEY=$(openssl rand -hex 32)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Migration Execution
|
||||||
|
```bash
|
||||||
|
# 1. Setup database schema
|
||||||
|
node scripts/setup-database.js
|
||||||
|
|
||||||
|
# 2. Run migration
|
||||||
|
node scripts/migrate-to-database.js
|
||||||
|
|
||||||
|
# 3. Verify data integrity
|
||||||
|
node scripts/verify-migration.js
|
||||||
|
```
|
||||||
|
|
||||||
|
### Post-Migration
|
||||||
|
```bash
|
||||||
|
# 1. Switch to database mode
|
||||||
|
unset USE_JSON_DATABASE
|
||||||
|
|
||||||
|
# 2. Restart application
|
||||||
|
pm2 restart server
|
||||||
|
|
||||||
|
# 3. Monitor logs
|
||||||
|
tail -f logs/server.log
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rollback (if needed)
|
||||||
|
```bash
|
||||||
|
# 1. Switch back to JSON mode
|
||||||
|
export USE_JSON_DATABASE=1
|
||||||
|
|
||||||
|
# 2. Restart application
|
||||||
|
pm2 restart server
|
||||||
|
|
||||||
|
# All data still in JSON files, no data loss
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## File Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
chat/
|
||||||
|
├── src/
|
||||||
|
│ ├── database/
|
||||||
|
│ │ ├── connection.js # Database connection
|
||||||
|
│ │ ├── schema.sql # Database schema
|
||||||
|
│ │ └── compat.js # Backward compatibility
|
||||||
|
│ ├── repositories/
|
||||||
|
│ │ ├── userRepository.js # User data access
|
||||||
|
│ │ ├── sessionRepository.js # Session data access
|
||||||
|
│ │ └── index.js # Repository exports
|
||||||
|
│ ├── utils/
|
||||||
|
│ │ ├── encryption.js # Field-level encryption
|
||||||
|
│ │ └── tokenManager.js # JWT + refresh tokens
|
||||||
|
│ ├── middleware/
|
||||||
|
│ │ └── auth.js # Updated auth middleware
|
||||||
|
│ └── routes/
|
||||||
|
│ └── auth.js # New auth endpoints
|
||||||
|
├── scripts/
|
||||||
|
│ ├── setup-database.js # Initial schema setup
|
||||||
|
│ ├── migrate-to-database.js # Data migration
|
||||||
|
│ └── verify-migration.js # Integrity verification
|
||||||
|
└── .data/
|
||||||
|
├── shopify_ai.db # Encrypted SQLite database
|
||||||
|
└── migration_backup_*/ # Backup directories
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security Features
|
||||||
|
|
||||||
|
### Encryption at Rest
|
||||||
|
- **Database-level:** SQLCipher AES-256 encryption
|
||||||
|
- **Field-level:** Sensitive fields individually encrypted
|
||||||
|
- **Key management:** Master key derived via PBKDF2
|
||||||
|
- **Token storage:** Hashed with PBKDF2 (not encrypted)
|
||||||
|
|
||||||
|
### Session Security
|
||||||
|
- **Short-lived tokens:** 15-minute access tokens
|
||||||
|
- **Token rotation:** New refresh token on every use
|
||||||
|
- **Device binding:** Tokens bound to device fingerprint
|
||||||
|
- **Theft detection:** Automatic revocation on fingerprint mismatch
|
||||||
|
- **Immediate revocation:** Token blacklist for instant logout
|
||||||
|
|
||||||
|
### Audit Trail
|
||||||
|
- All logins/logouts logged
|
||||||
|
- Token refresh events logged
|
||||||
|
- Session revocations logged
|
||||||
|
- Data access logged
|
||||||
|
- IP address and user agent captured
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Strategy
|
||||||
|
|
||||||
|
### Unit Tests
|
||||||
|
- Encryption/decryption round-trip
|
||||||
|
- Token generation and verification
|
||||||
|
- Device fingerprint generation
|
||||||
|
- Password hashing and verification
|
||||||
|
|
||||||
|
### Integration Tests
|
||||||
|
- Login flow with JWT
|
||||||
|
- Token refresh rotation
|
||||||
|
- Session revocation
|
||||||
|
- Device fingerprint mismatch
|
||||||
|
- Migration script with test data
|
||||||
|
|
||||||
|
### Security Tests
|
||||||
|
- Token theft simulation
|
||||||
|
- Replay attack prevention
|
||||||
|
- Session fixation protection
|
||||||
|
- Encryption key rotation
|
||||||
|
|
||||||
|
### Load Tests
|
||||||
|
- Database query performance
|
||||||
|
- Token generation under load
|
||||||
|
- Concurrent session handling
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Monitoring & Alerting
|
||||||
|
|
||||||
|
### Metrics
|
||||||
|
- Database query latency (p95, p99)
|
||||||
|
- Authentication success rate
|
||||||
|
- Token refresh frequency
|
||||||
|
- Session revocation count
|
||||||
|
- Device fingerprint mismatches
|
||||||
|
- Database size growth
|
||||||
|
|
||||||
|
### Alerts
|
||||||
|
- Database connection failures
|
||||||
|
- Migration errors
|
||||||
|
- High authentication failure rate
|
||||||
|
- Unusual token revocation patterns
|
||||||
|
- Encryption errors
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Timeline
|
||||||
|
|
||||||
|
| Phase | Task | Hours |
|
||||||
|
|-------|------|-------|
|
||||||
|
| 1.2 | Dependencies & setup | 4 |
|
||||||
|
| 1.2 | Database schema | 6 |
|
||||||
|
| 1.2 | Encryption layer | 8 |
|
||||||
|
| 1.2 | Repositories | 12 |
|
||||||
|
| 1.2 | Migration script | 8 |
|
||||||
|
| 1.2 | Backward compatibility | 6 |
|
||||||
|
| 1.2 | Testing | 10 |
|
||||||
|
| 1.3 | Token manager | 8 |
|
||||||
|
| 1.3 | Auth middleware | 6 |
|
||||||
|
| 1.3 | API endpoints | 6 |
|
||||||
|
| 1.3 | Testing | 6 |
|
||||||
|
| Both | Deployment & rollback testing | 10 |
|
||||||
|
| **Total** | | **90 hours** |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Risk Mitigation
|
||||||
|
|
||||||
|
| Risk | Mitigation |
|
||||||
|
|------|------------|
|
||||||
|
| Data loss | Multiple backups, transaction support, rollback capability |
|
||||||
|
| Migration failure | Backward compatibility layer, easy rollback |
|
||||||
|
| Performance issues | WAL mode, indexing, query optimization |
|
||||||
|
| Security vulnerabilities | Security review, penetration testing |
|
||||||
|
| Downtime | Zero-downtime migration strategy |
|
||||||
|
| Key compromise | Key rotation plan, secure key storage |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Success Criteria
|
||||||
|
|
||||||
|
- [ ] All data migrated without loss
|
||||||
|
- [ ] Encryption verified (data unreadable without key)
|
||||||
|
- [ ] Sessions can be revoked individually and globally
|
||||||
|
- [ ] Token rotation working correctly
|
||||||
|
- [ ] Device fingerprinting detecting mismatches
|
||||||
|
- [ ] Rollback tested and working
|
||||||
|
- [ ] Performance meets or exceeds JSON storage
|
||||||
|
- [ ] Audit logging capturing all security events
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Next Steps
|
||||||
|
|
||||||
|
1. **Review this plan** with security team and stakeholders
|
||||||
|
2. **Set up staging environment** with production-like data
|
||||||
|
3. **Implement Phase 1.2** (database + encryption)
|
||||||
|
4. **Test migration** on staging
|
||||||
|
5. **Implement Phase 1.3** (token management)
|
||||||
|
6. **Security review** and penetration testing
|
||||||
|
7. **Deploy to production** with monitoring
|
||||||
Reference in New Issue
Block a user