Restore to commit 74e578279624c6045ca440a3459ebfa1f8d54191
This commit is contained in:
southseact-3d
215
TESTING_MISTRAL_LOGGING.md
Normal file
215
TESTING_MISTRAL_LOGGING.md
Normal file
@@ -0,0 +1,215 @@
|
||||
# Testing Mistral Response Parsing with Enhanced Logging
|
||||
|
||||
## How to Test
|
||||
|
||||
### 1. Set Up Mistral as Planning Provider
|
||||
|
||||
1. Log in to the admin panel at `/admin`
|
||||
2. Go to the "Provider Configuration" section
|
||||
3. Set "Planning Provider" to "Mistral"
|
||||
4. Make sure you have a valid `MISTRAL_API_KEY` environment variable set
|
||||
5. Configure at least one Mistral model in the models list
|
||||
|
||||
### 2. Create a Test Plan
|
||||
|
||||
1. Go to `/apps` (or `/builder`)
|
||||
2. Create a new app/session
|
||||
3. Send a message that should trigger the planning phase
|
||||
4. Watch the logs
|
||||
|
||||
### 3. View the Logs
|
||||
|
||||
**Docker Logs:**
|
||||
```bash
|
||||
# View all logs
|
||||
docker logs <container-id> -f
|
||||
|
||||
# Filter for Mistral logs only
|
||||
docker logs <container-id> -f 2>&1 | grep "\[MISTRAL\]"
|
||||
|
||||
# Filter for plan logs
|
||||
docker logs <container-id> -f 2>&1 | grep "\[PLAN\]"
|
||||
|
||||
# Filter for both
|
||||
docker logs <container-id> -f 2>&1 | grep -E "\[MISTRAL\]|\[PLAN\]"
|
||||
```
|
||||
|
||||
**ttyd Terminal (if available):**
|
||||
```bash
|
||||
# In the container terminal
|
||||
tail -f /path/to/server.log | grep -E "\[MISTRAL\]|\[PLAN\]"
|
||||
```
|
||||
|
||||
## What to Look For
|
||||
|
||||
### 1. Request Being Made
|
||||
|
||||
Look for:
|
||||
```
|
||||
[MISTRAL] Starting API request { url: '...', model: '...', ... }
|
||||
[MISTRAL] Request payload: { model: '...', messagesCount: 3, ... }
|
||||
```
|
||||
|
||||
**Check:**
|
||||
- Is the URL correct? (`https://api.mistral.ai/v1/chat/completions`)
|
||||
- Is there an API key? (should show first 8 characters)
|
||||
- Is the model name correct?
|
||||
- Are messages being sent?
|
||||
|
||||
### 2. Response Received
|
||||
|
||||
Look for:
|
||||
```
|
||||
[MISTRAL] Response received { status: 200, ok: true, ... }
|
||||
[MISTRAL] Full API response: { "choices": [...], ... }
|
||||
```
|
||||
|
||||
**Check:**
|
||||
- Is status 200?
|
||||
- Is ok: true?
|
||||
- What does the full response look like?
|
||||
|
||||
### 3. Content Extraction
|
||||
|
||||
Look for:
|
||||
```
|
||||
[MISTRAL] Choices array: [...]
|
||||
[MISTRAL] First choice: { ... }
|
||||
[MISTRAL] Message object: { ... }
|
||||
[MISTRAL] Content value: "actual content here"
|
||||
[MISTRAL] Content type: "string"
|
||||
```
|
||||
|
||||
**Check:**
|
||||
- Does choices array exist?
|
||||
- Does first choice exist?
|
||||
- Does message object exist?
|
||||
- Does content value exist and is it a string?
|
||||
|
||||
### 4. Reply Extraction
|
||||
|
||||
Look for:
|
||||
```
|
||||
[MISTRAL] Extracted reply: {
|
||||
reply: "...",
|
||||
replyType: "string",
|
||||
replyLength: 1234,
|
||||
isEmpty: false,
|
||||
isNull: false,
|
||||
isUndefined: false,
|
||||
isFalsy: false
|
||||
}
|
||||
```
|
||||
|
||||
**Check:**
|
||||
- Is reply a string?
|
||||
- Is replyLength > 0?
|
||||
- Are isEmpty/isNull/isUndefined all false?
|
||||
|
||||
### 5. Plan Processing
|
||||
|
||||
Look for:
|
||||
```
|
||||
[PLAN] Mistral result received (raw): { hasResult: true, ... }
|
||||
[PLAN] Before sanitization: { rawReply: "...", ... }
|
||||
[PLAN] After sanitization: { sanitizedReply: "...", ... }
|
||||
```
|
||||
|
||||
**Check:**
|
||||
- Is hasResult: true?
|
||||
- Does rawReply contain content?
|
||||
- Does sanitizedReply match rawReply? (or is content being removed?)
|
||||
|
||||
### 6. Message Creation
|
||||
|
||||
Look for:
|
||||
```
|
||||
[PLAN] Creating message object with reply: { reply: "...", ... }
|
||||
[PLAN] Final message details: { messageId: "...", messageReply: "...", ... }
|
||||
[PLAN] Plan message completed successfully { ... }
|
||||
```
|
||||
|
||||
**Check:**
|
||||
- Does the message object have a reply?
|
||||
- Is messageReplyLength > 0?
|
||||
- Does it complete successfully?
|
||||
|
||||
## Common Issues and What Logs Will Show
|
||||
|
||||
### Issue: "No API key"
|
||||
```
|
||||
[MISTRAL] API key missing
|
||||
```
|
||||
**Fix:** Set MISTRAL_API_KEY environment variable
|
||||
|
||||
### Issue: "API returns error"
|
||||
```
|
||||
[MISTRAL] Response received { status: 401, ok: false, ... }
|
||||
[MISTRAL] Error response body: "Unauthorized"
|
||||
```
|
||||
**Fix:** Check API key validity
|
||||
|
||||
### Issue: "Empty response"
|
||||
```
|
||||
[MISTRAL] Full API response: { "choices": [] }
|
||||
or
|
||||
[MISTRAL] Content value: undefined
|
||||
```
|
||||
**Fix:** Check API documentation, may need different model or parameters
|
||||
|
||||
### Issue: "Content removed by sanitization"
|
||||
```
|
||||
[PLAN] Before sanitization: { rawReply: "...", rawReplyLength: 1234 }
|
||||
[PLAN] After sanitization: { sanitizedReply: "", sanitizedReplyLength: 0, wasModified: true }
|
||||
```
|
||||
**Fix:** Check sanitizeAiOutput() function, may be too aggressive
|
||||
|
||||
### Issue: "Reply lost in message creation"
|
||||
```
|
||||
[PLAN] After sanitization: { sanitizedReply: "...", sanitizedReplyLength: 1234 }
|
||||
[PLAN] Creating message object with reply: { reply: "", replyLength: 0 }
|
||||
```
|
||||
**Fix:** Check the assignment between sanitization and message creation
|
||||
|
||||
## Expected Successful Output
|
||||
|
||||
For a successful Mistral plan request, you should see:
|
||||
|
||||
```
|
||||
[PLAN] Starting plan message handling { sessionId: '...', ... }
|
||||
[PLAN] Trying provider { provider: 'mistral', model: '...' }
|
||||
[PLAN] Using Mistral provider { modelHint: '...', messagesCount: 3 }
|
||||
[MISTRAL] Starting fallback chain { preferredModel: '...', chainLength: 1, models: [...] }
|
||||
[MISTRAL] Trying model: mistral-...
|
||||
[MISTRAL] Starting API request { url: '...', model: '...', ... }
|
||||
[MISTRAL] Request payload: { model: '...', messagesCount: 3, ... }
|
||||
[MISTRAL] Response received { status: 200, ok: true, ... }
|
||||
[MISTRAL] Full API response: { "choices": [{ "message": { "content": "..." } }], ... }
|
||||
[MISTRAL] Response data structure: { hasChoices: true, choicesLength: 1, ... }
|
||||
[MISTRAL] Choices array: [{ ... }]
|
||||
[MISTRAL] First choice: { message: { content: '...' }, ... }
|
||||
[MISTRAL] Message object: { content: '...' }
|
||||
[MISTRAL] Content value: Plan details here...
|
||||
[MISTRAL] Content type: string
|
||||
[MISTRAL] Extracted reply: { reply: '...', replyType: 'string', replyLength: 1234, isEmpty: false, ... }
|
||||
[MISTRAL] Successfully extracted reply { replyLength: 1234, replyPreview: 'Plan details...' }
|
||||
[MISTRAL] Plan succeeded on first try { model: '...' }
|
||||
[PLAN] Mistral result received (raw): { hasResult: true, hasReply: true, replyLength: 1234, ... }
|
||||
[PLAN] Before sanitization: { rawReply: '...', rawReplyLength: 1234 }
|
||||
[PLAN] After sanitization: { sanitizedReply: '...', sanitizedReplyLength: 1234, wasModified: false }
|
||||
[PLAN] Provider succeeded { provider: 'mistral', model: '...', replyLength: 1234, ... }
|
||||
[PLAN] Creating message object with reply: { reply: '...', replyLength: 1234, ... }
|
||||
[PLAN] Final message details: { messageId: '...', messageReplyLength: 1234, cleanReplyLength: 1234, ... }
|
||||
[PLAN] Plan message completed successfully { sessionId: '...', provider: 'mistral', ... }
|
||||
```
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
When reporting issues, please include:
|
||||
|
||||
1. The full log output filtered by `[MISTRAL]` and `[PLAN]`
|
||||
2. Screenshots of the UI showing the empty response
|
||||
3. The admin panel configuration (provider settings, model settings)
|
||||
4. Environment variables (without exposing the actual API key)
|
||||
|
||||
This will help identify exactly where in the chain the response is being lost.
|
||||
Reference in New Issue
Block a user