If someone says their export failed, I want to find the job without asking them to reproduce everything. A job ID carried from the interface through the worker and provider request does a lot of useful work here.
For a small product, I’d begin with the main workflow: requested, queued, started, completed or failed. Record the dependency involved, duration and a safe error summary. Keep credentials and raw private content out of logs.
For model-heavy work, attach usage and cost to the job too. Cost per successful export is more revealing than the monthly API bill alone. A cheap request that fails three times may be the expensive route.
I don’t want every unusual metric to become an alert. If there is no action to take, a daily summary may be enough. A broken payment path or an exhausted provider balance needs a different level of attention.
The test I use for a dashboard is simple: what question can it answer, and what would I do with the answer? If neither is clear, I probably don’t need another chart.