I have a few nodes (especially net-snmp Linux) which are finicky when coming back up and often need SNMP service restarted. While it's easy for me to spot them, Node UP but hardware health, volumes, and interfaces all not responding, I am yet to find a script to locate them. I can identify which nodes are UP, and SNMP object type and child status down but those could also be legitimate child object notifications.
I know the new NPM might be coming with SNMP polling for up status but I would prefer a method to both leave ICMP as up/down but notify when just SNMP is down.
Thoughts?