Quantcast
Channel: THWACK: All Content - Network Performance Monitor
Viewing all 21870 articles
Browse latest View live

SQL block, lock, deadlock - System Down

$
0
0

Community,

 

I have been experiencing a large number of system down incidents (29 as of yesterday) over the past couple months. These are caused by SQL deadlocks. I need assistance determining root cause. Before you ask, Orion is completely updated, SQL server 2016, 12 cpu cores per VM, and minimum 128MB RAM. These servers are fully updated and overpowered. The queries are the same every time, pulling from the events table. Any assistance or guidance would be appreciated. Please do not ask me to put in a support ticket, I already have two with no progress made, including one for system down right now (been on hold too long).

 

 

 

Blocking Session (From Primary Server):

  Application name: SolarWinds.Collector.Service@domain-SWCollectorService-0

 

    First 2KB of SQL: (@engineId int,@eventType int,@eventDate int)SELECT MAX(e.EventTime), e.NetObjectID FROM Events e

                                               INNER JOIN Nodes n ON n.NodeID = e.NetObjectID

                                               WHERE n.EngineID = @engineId AND e.EventType = @eventType AND e.EventTime >= DATEADD(D, -@eventDate, GETDATE()) GROUP BY e.NetObjectID

 

 

 

 

Blocked session (From additional poller):

  Application name: SolarWinds.Collector.Service@domain-SWCollectorService-0

 

    First 2KB of SQL:

CREATE PROCEDURE [dbo].[apm_AddEvent]

@NetworkNode int,

@NetObjectID int,

@NetObjectType char(10),

@EventType int,

@Message ntext,

@EventTimeUtc datetime = NULL

               AS

BEGIN

               IF ((@EventType IS NOT NULL) AND ((@Message IS NOT NULL) AND (DATALENGTH(@Message) > 0)) AND ((Select Record From EventTypes where EventType = @EventType) = 1))

                              BEGIN

                                             SET NOCOUNT ON;

 

                                             DECLARE @EventTimeLocal datetime

 

                                             IF @EventTimeUtc IS NULL

                                                            SET @EventTimeLocal = GETDATE()

                                             ELSE

                                                            SET @EventTimeLocal = DATEADD(MINUTE, DATEDIFF(MINUTE, GETUTCDATE(), GETDATE()), @EventTimeUtc)

 

                                             INSERT INTO Events (EventTime, NetworkNode, NetObjectID, NetObjectType, EventType, Message, Acknowledged)

                                             VALUES (@EventTimeLocal, @NetworkNode, @NetObjectID, @NetObjectType, @EventType, @Message, 0)

                              END

END


Running Solarwinds on Alwayson SQL cluster (database issues)

$
0
0

Hello Everyone

 

We are running Solarwinds NPM, NTA, and NCM on an alwayson SLQ cluster.   We have been having tons of issues.  Database has tons of errors and needs to be repaired constantly, performance has not been great.   We were reading on some other posts that Alwayson is supported by SW but does is not recommended to have full recovery mode.  The problem with that is full recovery mode is required for alwayson cluster.  Cannot have one without the other. 

 

Is anyone else running SW on alwayson?  Are there certain items that need to be tuned?   Does anyone have a opinion about this?

 

Thanks

Deploying Virtual Appliances

$
0
0

Are you comfortable with deploying virtual appliances?

 

We're discussing ways to deploy new products that may interact with your NPM (and other SolarWinds) deployments and one of our options is a virtual appliance.

Using Your Custom HTML Resource To Build A Better Way To Navigate Your Custom Views

What We're Working on for NPM (Updated June 1st, 2018)

$
0
0

NPM 12.3 has shipped and we're hard at work building the next release.  Here's what we're working on, in no particularly order.

 

Tracking API usage

$
0
0

I have a need to track API requests made to Solarwinds. Any ideas on how this can be accomplished?

Netpath to a meraki Z3 Gateway

$
0
0

I am interested in using the netpath service to troubleshoot some remote ISP issues but have not been able to find a way to get the Z3 to respond to my probe? I have tried every port that Meraki documentation says needs to be open for the device to talk to the web dashboard. Has anyone else tried to work with the Z1 or Z3 devices?

Nodes in Unknown state after unmanage/manage.

$
0
0

Hello,

After putting some nodes in unmanaged mode and putting them back into Managed mode we have a strange behavior: some of them still into "Unknown state" even if we can see status on interfaces.... We have also this strange behavior on ICMP devices...


Multiple IP Addresses for Border Devices.

$
0
0

Dear colleagues,

 

What workaround or workflow you use for monitoring border devices with mutiple IPs?

 

In my case: we have 2+ channels for each branch (main + 1+ reserve, each of them can be used as main in case of some issue on Internet provider side) and I want to have alerts about down device only when all channels/IPs are unavailable.

I've seen the post in KB Multiple IP Addresses for One Node - SolarWinds Worldwide, LLC. Help and Support, it's nice, but from my point of view, more modern and flexible is to have a possibility to add more than 1 IP address for a device.

SQL AlwaysOn High availability " Not Synchronization/Critical" state monitoring

$
0
0

Hi All,

 

I wondering to create an alert for SQL AlwaysOn High availability " Not Synchronization" state monitoring. Can you please help me out complete the same?

Orion Module Engine stops and restart continuously after DB connection failure

$
0
0

I'm running NPM 10.2.2 with NCM 7.0.2.  Whenever the DB server is unavailable (e.g a machine restart), the NPM server seems to get confused and NPM will no longer serve pages.  I've had to restart IIS or the "Solarwinds Orion Module Engine" to get it working again.  This NPM confusion can be caused in as little as a couple of minutes of inability to access the DB server.  Is this a known issue?
Thanks in advance.

Bug with Manage Accounts - Groups

$
0
0

A few releases ago I noticed that under Manage Accounts - Groups section our groups that we put into the system no longer show up.  Individual accounts still show up fine. When we go to the Account list, we can still see all of the groups in the system.  So we know they are still there and the access is still working.   But, we can no longer see them on the Manage Accounts page.

 

We recently upgraded to Orion Platform 2018.2 HF5, but that didn't resolve the problem. I see that HF6 is now out, but it doesn't appear like this is something that was addressed in it based on the release notes.

 

Is this a known issue already?  Is there an easy fix for this? Is it included in HF6?

Automated port shutdown - what would be your reasons?

$
0
0

There was a feature request for a UDT recently for an automated port shutdown, if the connected endpoint is on the "black list".

What would be your other reasons, for which would you want to shut down the port? Lets's go our imagination wild and imagine events like user device infected by a non-deletable virus, peer-to-peer software installed, firewall disabled? Feel free to vote for Other and leave comment. By liking the specific comments you can vote for other ideas.

Thanks!

rebooted events that keep showing on how to stop that?

$
0
0

Hi

 

Does any 1 know how I stop that from  showing up ?

It was normal that you get 1 event that a box was rebooted ...

Now it's the same event keep showing 

Deploying Virtual Appliances

$
0
0

Are you comfortable with deploying virtual appliances?

 

We're discussing ways to deploy new products that may interact with your NPM (and other SolarWinds) deployments and one of our options is a virtual appliance.


Tell Us Your Unknown Devices v2.0

$
0
0

Those that have been part of the Thwack Community a while may be familiar with the long running Tell us your "Unknown" devices! thread which had been active since 2007. That thread had become too unwieldy, and most of the user submissions had been implemented many years ago. I recently reviewed each and every posting in that thread, verified what had been implemented in-product, and which ones had not so they could be included in a forthcoming release. With that done, it was time to lock that thread for good and start anew. This time, providing a bit more guidance along the way to ensure everyone is successful in providing the necessary information required to properly identify these devices.

 

What is an 'Unknown' Device anyway?

 

Orion does its best to automatically identify and classify nodes as they're added to Orion. There are however, new device types and models released all the time. It's entirely possible you might be managing a device right now that Orion is unable to properly identify. You can find these easily by going to [Settings - Manage Nodes], changing the 'Group by:' option to 'Machine Type' and clicking on the 'Unknown' category. It's also helpful to add the 'Polling Method' column to the layout, as this thread pertains exclusively to SNMP managed nodes.

 

Any SNMP managed nodes listed under the 'Unknown' Machine Type category are prime candidates for submission to this thread. All that's required is that you provide the devices SNMP System Object Identifier (SysObjectID), as well as the Make & Model of the device associated with that SysObjectID. This post is an excellent example of the perfect submission.

 

What Exactly is a SysObjectID?

 

I have yet to find a clearer definition for what the SysObjectID (System Object Identifier) is then the following excerpt which can typically be found written in virtually every vendor's MIB file verbatim.

 

Object Name: sysObjectID
Object ID: 1.3.6.1.2.1.1.2.0
Object Syntax: OBJECT IDENTIFIER
Object Access: read-only
Object Status: mandatory
Object Description: The vendor's authoritative identification of the  network management subsystem contained in the  entity. This value is allocated within the SMI  enterprises subtree (1.3.6.1.4.1) and provides an  easy and unambiguous means for determining `what  kind of box' is being managed. For example, if  vendor `Flintstones, Inc.' was assigned the  subtree 1.3.6.1.4.1.4242, it could assign the  identifier 1.3.6.1.4.1.4242.1.1 to its `Fred  Router'.

 

Essentially, it's a string of numbers in dotted notation that is (hopefully) unique to at least the manufacturer, and in most cases, to the specific make and model of the device being monitored. It's how we identify for example, that the device vendor is 'Cisco' and the model is a 'Nexus C7018'. All System Object ID's begin with '1.3.6.1.4.1' followed by a number which uniquely identifies the manufacturer. The numbers which then follow typically identify the specific model of the device.

 

Where Can I Locate the SysObjectID?

 

If the device is already managed as a Node in Orion then you can locate the SysObjectID in the 'Node Details' resource as shown below, when viewing the node in the Orion web interface.

 

Node DetailsNET-SNMP

Alternatively, you can use NET-SNMP to query the following SNMP OID to return the unique SysObjectID.

 

1.3.6.1.2.1.1.2.0

 

Below is an example of the 'snmpget' command line arguments which will return you the SysObjectID for the device.

 

 snmpget -v2c -On -c public 10.199.5.103 1.3.6.1.2.1.1.2.0

 

The example above is executed against a device with the IP address of '10.199.5.103' using SNMPv2c, with the community string 'public'. Below is a screenshot of the resulting output from that command. The string of numbers and periods highlighted in yellow below is this device's unique SysObjectID.

 

My Device Incorrectly Appears Listed as 'NET-SNMP'

 

Linux hosts, virtual appliances, and even some network equipment built on Linux, FreeBSD, etc. are often identified as 'NET-SNMP'. This is because the SNMP Daemon running on those hosts is, you guessed it, NET-SNMP. Unfortunately, these vendors for some reason, have chosen not to implement their own unique SysObjectID, and instead kept the default SysObjectID '1.3.6.1.4.1.8072.3.2.10' which is designated for NET-SNMP. If you have a device such as this, fret not. There are a few options available to you if you'd like these devices to be properly identified by their appropriate vendor's make & model within Orion.

 

Install The Orion Linux Agent

 

The easiest solution would be to install the Orion Linux Agent on the device which is reporting itself to be 'NET-SNMP'. The Linux Agent does not rely upon SNMP to identify the machine type or vendor. Instead, the Agent will report the Vendor as 'Linux' and the 'Machine Type' as the Linux distribution running on the device as depicted in the screenshots below.

 

Red HatCitrix XenServer

 

 

Modify NET-SNMP Configuration

 

Another approach is to customize NET-SNMP and Orion to properly reflect the Vendor and Machine Type. Simply following the steps outlined by adatole's post entitled No More Net-SNMP Nodes. This method uses a script osname.sh which is executed when a particular OID is is queried. Next, you would create a custom Device Poller to query that newly created OID and populate the Machine Type value in Orion for that device.

 

If you find it more fun to follow along, you can watch adatole walk you through the entire process in the following video.

 

 

 

Can't I Just Upload My Vendor's MIB File Here And You figure it Out?

 

While it would be nice if that's how it worked, unfortunately many (or most) vendors don't include this information within their MIB files. MIB files include a listing of all possible OIDs which could be polled across a wide variety of different devices (typically an entire product family), but it doesn't include the values which are returned by the devices (Enums notwithstanding). For that reason we need users, such as yourself, to post the SysObjectID's in this thread, along with the device vendor and model information so it can be included in our database.

 

If you'd still like your device's MIB file included in the Orion MIB database, for use with Network Performance Monitor's Universal Device Poller, or the Orion Platform's SNMP Trap Receiver, simply follow the steps outlined in KB article at the link below.  The latest version of the MIB database, containing your submissions, can always be downloaded from within the Customer Portal.

 

Request additional MIBs to the SNMP MIB browser database - SolarWinds Worldwide, LLC. Help and Support

Can Solarwinds be integrated with huawei ticketing system?

$
0
0

Need some help guys, is it possible to integrate our Solarwinds environment with a Huawei ticketing system? i know its compatible with Jira but is it with Huawei?

Silver Peak UnDP

SilverPeak UnDP With Alarm Table

Autobalance APE's Part ONE -SQL script to update custom properties

$
0
0

I apologize in advance for my not so friendly formatting. The following is a script my DBA and I wrote to assist with balancing nodes. This is not the script that balances them, however its the first part. This script updates a custom property whose values represent the polling engine desired, marks it as in the DMZ or not, and defines its logical location in the network (which can be used for maps and dynamic groups). It only runs on nodes with custom property value Update_Script is yes. My query balances off of subnet and logical location to designate which poller it goes to, among other things. You can add in whatever conditions you like, I have removed most of mine. At the very end, this script outputs a change log. This script runs nightly on the SQL agent.  I am confident someone better than me can optimize this.

 

Note the following custom properties need to exist:

CP_Location

InDMZ

Polling_Engine

Update_Script

 

In order to run this query on a node, you can use the web interface to set custom property Update_Script to yes or null. I have it as the default when a new node is discovered. Script sets this value back to no so it will not run twice. I like Automation.

-------------------------------------------------------------------------------------

 

 

BEGIN TRANSACTION;

 

--Define conditions to update logical location, temporary value to use in second part of the query

 

WITH cte_locations

  AS (SELECT d.NodeID,

             d.IP_Address,

             CASE

                  WHEN d.IP_Address LIKE '<<dmz subnet>>' THEN '<<your_location value here>>'

                

 

<<additional conditions here>>

 

 

                  END AS your_Location,

--set in dmz or not below

             CASE

                  WHEN d.IP_Address LIKE '<<dmz subnet>>' THEN 1

                  ELSE 0 END AS In_DMZ

        FROM dbo.NodesData d

       INNER JOIN dbo.NodesCustomProperties c

          ON c.NodeID = d.NodeID

     ),

 

 

--part DEUX  below is where the magic happens, update polling engine property

 

     cte_poll

  AS (SELECT d.NodeID,

             d.your_Location,

             d.In_DMZ,

 

  d.IP_Address,

             CASE

                  WHEN d.In_DMZ = 1 THEN '<<poll engine CP value>>'

  WHEN d.IP_Address LIKE '<<subnet>>' OR d.IP_Address LIKE '<<subnet>>' THEN '<<poll engine cp value>>'

WHEN d.your_Location IN ( '<<cp_location value>>' ) AND d.IP_Address LIKE '<<subnet>>' THEN  '<<cp polling engine value>>'

 

 

 

<<additional conditions here>>

 

.

 

 

                  ELSE CASE

  WHEN d.IP_Address LIKE '<<subnet>>' OR d.IP_Address LIKE '<<subnet>>' THEN '<<poll engine cp value>>'

 

 

 

<<additional conditions here>>

 

 

 

 

                            ELSE '<<cp poll engine value>>'

END

END AS Polling_Engine

        FROM cte_locations d)

 

 

--ACTUAL changes below

 

UPDATE NodesCustomProperties

   SET CP_Location = z.your_Location, --logical location of the node

       InDMZ = z.In_DMZ, --is this node in the dmz

       Polling_Engine = z.Polling_Engine, --what polling engine you want it on

       Update_Script = 'No' --do not modify this node next time.

 

--output change log

   OUTPUT Deleted.NodeID, Deleted.CP_Location, Inserted.CP_Location, Deleted.Polling_Engine, Inserted.Polling_Engine

 

  FROM cte_poll z

  WHERE z.NodeID = NodesCustomProperties.NodeID

 

 

ROLLBACK

--commit

Viewing all 21870 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>