Testing Functions

​

Testing Strategy

​

Unit Testing Your API

​

Using curl

curl -X POST https://api.yourcompany.com/orders/status \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "order_id": "TEST_123"
  }'

​

Using Postman

1. Create a new request
2. Set method and URL
3. Add headers
4. Add request body
5. Send and verify response

​

Expected Response

• Consistent JSON structure
• Appropriate status codes
• Clear error messages
• Fast response time (< 2s)

​

Testing Function Definitions

​

Validate JSON Schema

• JSON Schema Validator
• Swagger Editor

​

Test with Sample Parameters

{
  "function": "check_order_status",
  "test_parameters": {
    "order_id": "TEST_ORDER_123"
  },
  "expected_response": {
    "status": "shipped",
    "tracking_number": "1Z999AA10123456784"
  }
}

​

Conversation Testing

​

Create Test Scenarios

User: "I want to check my order status"
Agent: "I'd be happy to help. What's your order number?"
User: "It's ORDER-123"
Agent: [Calls check_order_status function]
Agent: "Your order is currently in transit and will arrive on Friday"

User: "Check my order"
Agent: "I'll check that for you. What's your order number?"
User: "I don't have it"
Agent: "No problem. Can you provide your email address?"

User: "Order number ABC-999"
Agent: [Calls function, receives 404 error]
Agent: "I couldn't find that order number. Could you verify it's correct?"

​

Test Checklist

• ✓ Function triggers at the right time
• ✓ Parameters are extracted correctly
• ✓ API call succeeds
• ✓ Response is interpreted correctly
• ✓ Agent continues conversation naturally
• ✓ Errors are handled gracefully

​

Debugging

​

Enable Logging

app.post('/api/orders/status', (req, res) => {
  console.log('Function call received:', {
    timestamp: new Date().toISOString(),
    callId: req.headers['x-call-id'],
    body: req.body,
    source: req.headers['user-agent']
  });
  
  // Process request
  
  console.log('Function response:', {
    status: 200,
    data: responseData
  });
  
  res.json(responseData);
});

​

Common Issues

​

Function Not Triggering

• Agent doesn’t call the function when expected

1. Check function description is clear
2. Add more trigger examples
3. Make conditions more explicit
4. Test with direct requests

​

Wrong Parameters Extracted

• Function called with incorrect parameter values

1. Improve parameter descriptions
2. Add format examples
3. Use enums to constrain values
4. Add validation in prompt

​

API Call Fails

• Function triggers but API returns error

1. Test API endpoint with curl
2. Check authentication headers
3. Verify parameter format
4. Review API logs

​

Monitoring Production

​

Track Function Calls

• Function call frequency
• Success rate
• Average response time
• Error rate by type

​

Set Up Alerts

• High error rate (> 5%)
• Slow response time (> 5s)
• Authentication failures
• Unexpected errors

​

Review Logs

• Function call logs
• API error logs
• Agent conversation logs
• User feedback

​

Testing Tools

​

ngrok for Local Testing

# Start your local API
node server.js

# In another terminal, start ngrok
ngrok http 3000

# Use ngrok URL in function definition
https://abc123.ngrok.io/api/orders/status

​

Mock APIs

// mock-api.js
const express = require('express');
const app = express();

app.post('/api/orders/status', (req, res) => {
  const { order_id } = req.body;
  
  // Return mock data
  res.json({
    order_id: order_id,
    status: 'shipped',
    tracking_number: 'MOCK123',
    estimated_delivery: '2025-11-20'
  });
});

app.listen(3000);

​

Best Practices

​

Start Simple

1. Single parameter
2. Simple GET request
3. Clear success/failure states

​

Iterate Based on Data

1. Deploy with logging
2. Monitor for issues
3. Adjust based on patterns
4. Refine descriptions and parameters

​

Document Everything

• Function purpose and triggers
• Parameter formats and examples
• Expected responses
• Error scenarios
• Testing procedures

​

Next Steps