Production software monitoring: alerts that matter vs noise

How to design observability for business applications in Mexico: metrics, logs, traces, and on-call without waking the team on false positives.