When troubleshooting Delayed Device Telemetry on a Palo Alto Networks firewall, the goal is to verify four things:
- Telemetry is enabled and healthy.
- The firewall can reach the telemetry servers.
- Telemetry files are being generated and uploaded.
- There are no certificate, connectivity, or software issues.
1. Verify Telemetry Status
> show device-telemetry settings
Example output:
Device Telemetry Settings:
device-health-performance: yes
product-usage: yes
threat-prevention: yes
region: Americas
status: Device Certificate is valid
Check for:
- Status = Device Certificate is valid
- All required telemetry categories = yes
- Correct region configured
2. Check Telemetry Statistics
> show device-telemetry stats all
Look for:
- Last successful upload
- Last attempt
- Upload failures
- Queue size
3. Force Telemetry Collection
> request device-telemetry collect-now
This immediately triggers telemetry collection and is useful for testing.
4. Check System Logs
> show log system
Filter for messages such as:
Failed to send: file
Certificate Does Not Exist
Send File to CDL Receiver Failed
These indicate upload or certificate problems.
5. Review Telemetry Log File
> less mp-log device_telemetry_send.log
Common errors include:
Send File to CDL Receiver Failed
Certificate Does Not Exist
curl cmd status: 11
curl cmd status: 18
These help identify whether the failure is due to network connectivity or certificates.
6. Verify Device Certificate
> show device-telemetry settings
If you see:
status: Device Certificate does not exist
Telemetry uploads will fail until the device certificate is restored.
7. Test DNS Resolution
> ping host telemetry.paloaltonetworks.com
or
> show dns-proxy cache all
Verify the firewall can resolve required cloud endpoints.
8. Verify Internet Connectivity
> ping source <management-ip> host 8.8.8.8
or
> traceroute host telemetry.paloaltonetworks.com
If management traffic uses a service route, verify it is correctly configured.
9. Verify Service Route
> show deviceconfig system route
or in the GUI:
Device → Setup → Services → Service Route Configuration
Ensure telemetry traffic exits through the expected interface.
10. Check Management Plane Resource Usage
> show system resources
High CPU or memory utilization on the management plane can delay telemetry generation and uploads.
11. Verify Current Software Version
> show system info
Some PAN-OS releases contained telemetry defects (for example, repeated "Failed to send: file" messages), which were resolved in later maintenance releases. If you're seeing known telemetry errors, compare your version against the fixes documented by Palo Alto Networks.
Common Causes of Delayed Telemetry
| Symptom | Possible Cause | Verification |
|---|---|---|
| Last upload several hours old | No Internet connectivity | ping, traceroute |
| Failed to send file | CDL connectivity issue | device_telemetry_send.log |
| Certificate Does Not Exist | Missing device certificate | show device-telemetry settings |
| Region mismatch | Incorrect telemetry region | show device-telemetry settings |
| Upload queue increasing | Management plane busy | show system resources |
| Upload retries every hour | PAN-OS software defect | Check PAN-OS version and release notes |
Useful Troubleshooting Workflow
show device-telemetry settings
show device-telemetry stats all
request device-telemetry collect-now
show log system
less mp-log device_telemetry_send.log
show system resources
show system info
ping host telemetry.paloaltonetworks.com
This sequence verifies telemetry configuration, checks for upload attempts, reviews errors, confirms management plane health, and tests connectivity to the telemetry service.
No comments:
Post a Comment