When adding new nodes to NPM, do you use Network Sonar or you add your nodes directly?
Do you use the Network Sonar Discovery?
My Orion Website & Server is very slow / CPU & Memory Spike what should i check quickly ?
This article provides quick information about your current environment and health check , Further it will help you address the most common reasons of performance issues on to your server without sending the diagnostics to SolarWinds support .
In this article you can also Audit your own environment quickly if its been setup as recommended by the Solarwinds MINIMUM requirements or according to the settings eliminating bottlenecks creating performance issues within the set environment .
This article also help to save time to upload the diagnostics for Support where you have air gap between the server and you can check the basic health check on the actual server itself .
There are lots of other benefits to check the environment health within the internal due to security procedure not allowing uploading the diagnostics for Solarwinds support .
Your check list
Server Hardware
Total Elements (Nodes /Interfaces/ Volumes) been polled per server
Check free disk space on the Orion Server and SQL server
Check Your Server Polling Rate
SQL Server / Orion DB Size / Settings / Options
Check SQL Server Disk Performance
Orion Antivirus directory exclusion
Webpages Customization
Lets Go!
Collect System diagnostics as below.
Navigate to Start -> SolarWinds Orion -> Documentation and Support
Launch the gray icon for Orion Diagnostics.> Click "Start"
This program will generate a .zip file as output.
( Unzip in a folder ) Right Click > Select Extract Here .
Server Hardware
Lets check your System Hardware first if this even near to the Solarwinds MINIM recommended.
Go to the SystemInformation folder > Open the SystemInfo.txt file
You will be able to find the System hardware specification below is an example where system is only assigned with 2 Physical CPU Sockets /
below is an example where the system is only assigned two CPU PHYSICAL SOCKETS which is below Solarwinds MINIMUM recommendation .
You must have to have MINIMUM 4 PHYSICAL CPU SOCKETS here .
System Type: x64-based PC
✘Processor(s): 2 Processor(s) Installed.
[01]: Intel64 Family 6 Model 45 Stepping 7 GenuineIntel ~1600 Mhz
[02]: Intel64 Family 6 Model 45 Stepping 7 GenuineIntel ~1400 Mhz
Total Physical Memory: 49.082 MB
Available Physical Memory: 39.408 MB
Virtual Memory: Max Size: 56.250 MB
Virtual Memory: Available: 45.376 MB
Virtual Memory: In Use: 10.874 MB
Now open the SysInfo.csv file and check the further current CPU load on the System and CPU GHz level .
Below in an example where the CPU load in around 70% on the current system due to two main reasons .
Parameter | Value |
OSVersion | Windows Server 2012 R2 (Microsoft Windows NT 6.2.9200.0) |
CPUInformation | Intel(R) Xeon(R) CPU E5-2420 0 @ 1.90GHz |
CurrentCPUUssage | 70 % |
TotalPhysicalMemmory | 49152 MB |
FreePhysicalMemmory | 39802 MB |
FreeVirtualMemmory | 45843 MB |
FreeSpaceInPagingFiles | 7109 MB |
CurrentTimeZone | xxxx Standard Time (UTC+01:00:00) |
Low Physical Sockets assigned
Low CPU power less then 3.0 GHz
You should be able to see MINIMUM 4 Physical Processors Sockets as below .
Strongly recommend : NOT to use lower then 3.0 GHz processor you will never get the performance what you are looking for even the Host and Guest wont show the CPU is busy .
Most likely you will see CPU spikes / Orion Services consuming High CPU and Memory . Once you will move the same VM to higher then 3.0 GHz process all the above symptoms will be resolved.
With lower then 3.0 GHz processor there might be other issues such as SQL Server TCP connections TimeOut Errors and High amount of data stored under MSMQ on the system .
Make sure you have MINIMUM 3. Ghz host with Hyperthreading Active it will improve the Guest performance significantly and you will have full performance out of Solarwinds application
This is how you setup your VM in ESX
Here is an Example when you assign the numbers of CPU SOCKETS to the VM
System Model: VMware Virtual Platform
System Type: x64-based PC
Processor(s): 4 Processor(s) Installed.
[01]: Intel64 Family 6 Model 15 Stepping 1 GenuineIntel ~3493 Mhz
[02]: Intel64 Family 6 Model 15 Stepping 1 GenuineIntel ~3493 Mhz
[03]: Intel64 Family 6 Model 15 Stepping 1 GenuineIntel ~3493 Mhz
[04]: Intel64 Family 6 Model 15 Stepping 1 GenuineIntel ~3493 Mhz
BIOS Version: Phoenix Technologies LTD 6.00, 4/14/2014
Further check how much memory is assigned and available for the system and check the TaskManager which application is consuming high memory .
In above case the System Hardware is not even near to the recommended SolarWinds production deployment therefor the CPU load will remain high therefor System resources.
The following table lists minimum hardware requirements and recommendations for your SolarWinds Orion server.
Installing multiple SolarWindsOrion Platform products on the same computer may change the requirements.
Hardware requirements are listed by SolarWinds NPM license level.
These minimum requirements are for the Orion Platform. Products that run on the Orion Platform may have different requirements, such as different OS or memory requirements.
Consult your product-specific documentation for the exact requirements.
Hardware | SL100, SL250, SL500 | SL2000 | SLX |
---|---|---|---|
CPU speed | Quad core processor, 2.5 GHz or better | Quad core processor, 2.5 GHz or better | Quad core processor, 3.0 GHz or better |
For more details see below guide
NPM 12.0 system requirements - SolarWinds Worldwide, LLC. Help and Support
Check free disk space on the Orion Sever and SQL Server
Make sure you have Good free space available on the Orion Server disks C Drive and installed Directory .
Make sure you have Good free space available on the SQL Server where the actual DB is stored.
Total Elements (Nodes /Interfaces/ Volumes) been polled per server
Go to folder "DB" > Open file "AllEngines.csv"
Check how many Elements you are polling per server
EngineID | Elements | Nodes | Interfaces | Volumes |
1 | 15828 | 934 | 6823 | 1071 |
2 | 16084 | 202 | 1305 | 77 |
With only SolarWinds SLX license you can montior up to 12000 Elements and beyond this you will need an Additional Polling Engine to monitor.
More Details see the Server Sizing guide .
Use additional polling engines for 12,000 or more monitored elements
If you plan to monitor 12,000 or more elements, SolarWinds recommends that you install additional polling engines on separate servers to help distribute the work load.
I would also strongly advise you to check the blog post for any other questions if you are polling beyond 12000 Elements with single SLX Server.
Boost your server polling capacity with Stackable Poller license
Multi-module system guidelines
Check your Server Polling Rate
Go to Settings > Polling Engines .
Check if any of the Polling Rate is increased ?
Make sure none of the Polling Rate exceeded above 100%
POLLING COMPLETION | 100 |
ELEMENTS | 225 |
NETWORK NODE ELEMENTS | 18 |
VOLUME ELEMENTS | 50 |
INTERFACE ELEMENTS | 157 |
POLLING RATE | 2% of its maximum rate. |
ROUTING POLLING RATE | 0% of its maximum rate. |
HARDWARE HEALTH POLLING RATE | 0% of its maximum rate. |
VIM.VMWARE.POLLING | 2 |
F5 POLLING RATE | 0% of its maximum rate. |
WIRELESS HEAT MAP POLLING RATE | 0% of its maximum rate. |
WIRELESS POLLING RATE | 0% of its maximum rate. |
UNDP POLLING RATE | 0% of its maximum rate. |
SAM APPLICATION POLLING RATE | 170% of its maximum rate. |
If you have any polling rate increased above the 100% you will notice high CPU / Memory Utilization on the System which could effect the System and application Performance .
Orion DB Size and settings
Go to the DBInfo Folder > Open DatabaseInfo.csv file
Check the Database Recovery Mode
Check the Total Database Size
Default DB Recovery should be SIMPLE (Strongly recommended)
name | db_size | status |
SolarWindsOrion | 889274.25 MB | ✘Recovery=FULL |
- I have a very large Orion database. What should I check?
- I have database performance issues. How can i improve my database performance?
- Why is my Orion database growing so quickly?
- Why does my Orion performance decrease each day?
- Why is my Orion Web Console loading data at a slow rate?
For more details please see the post below and follow all the steps one by one to check your Orion Database Health and settings.
This guide will help you address the most common questions and issues related to the Orion database performance check and configuration without using the SolarWinds Database Administrator (DBA).
Quick Orion database health check guide
Check SQL Server Disk Performance
Orion Antivirus directory exclusion for NPM
Web pages recommended settings
Still have any question / need assistance ?
Please feel free to submit a new support ticket in relation to your question/error. Our support lines are available 24/7.
http://www.solarwinds.com/support/ticket
You can also contact the support by 24/7 phone support .
Custom report to pull power supply SN's from Cisco Devices
We're working on trying to get a list of the PSU SN's from our Cisco devices, and we want to see if reporting can pull this info from the devices.
Does anyone know if this can be done/has been done before? I've already got a good report for the chassis PSU, but trying to get one for the PSU Sn's
specifically, looking to do this to the 2960x series.
Orion Polling Engine Report
Updated to use hover-overs & expose more details.
By default, this sorts the Primary Polling Engine first and then all Add'l Polling Engines by Name.
If you want to put the SWQL into a Custom Query widget, the code is:
SELECT CASE WHEN [N].Caption IS NULL THEN [E].ServerName ELSE CONCAT('<a href="', [N].DetailsUrl, '">', [N].[Caption], '</a>') END AS [Caption] , [E].ServerType , [N].DetailsUrl AS [_LinkFor_Caption] , ROUND([N].SystemUpTime / 60 / 60 / 24, 2 ) AS [Uptime (Days)] , MinuteDiff([E].KeepAlive, GETUTCDATE()) AS [Last Checkin (Minutes)] , [E].Elements AS [Monitored Elements] , [E].Nodes AS [Monitored Nodes] , [E].Interfaces AS [Monitored Interfaces] , [E].Volumes AS [Monitored Volumes] , [E].Pollers AS [Monitored UnDP] -- Comment out the follow line if you don't own SAM , [SAM].ComponentCount AS [Monitored Components] , [E].EngineVersion , CONCAT([E].WindowsVersion, '/', [E].ServicePack ) AS [OS/SP] , [N].CPULoad AS [CPU %] , [N].PercentMemoryUsed AS [Mem %] , [E].PollingCompletion AS [Polling Completion %] FROM Orion.Engines AS [E] -- Use of LEFT JOIN so that we can show Engines even if we aren't monitoring them... but we should be monitoring them LEFT JOIN Orion.Nodes AS [N] ON [E].IP = [N].IP_Address -- Comment out the follow block if you don't own SAM -- [BEGIN] SAM Information... INNER JOIN ( SELECT COUNT([AA].ComponentID) AS ComponentCount , [N].EngineID FROM Orion.APM.Component AS [AA] INNER JOIN Orion.APM.Application AS [A] ON [AA].ApplicationID = [A].ApplicationID INNER JOIN Orion.Nodes AS [N] ON [A].NodeID = [N].NodeID WHERE [AA].Disabled = 'False' GROUP BY [N].EngineID ) AS [SAM] ON [SAM].EngineID = [E].EngineID -- [END] SAM Information ORDER BY [E].ServerType DESC, [N].Caption
Inspired by Orion_Polling_Engine_Load.OrionReport, I decided to take that and move it to the Web Report.
Feedback is appreciated!
NPM 12.3 SNOW - Create ServiceNow Incident action not working
Hi All, I am trying to test SNOW Integration with NPM 12.3 and Create ServiceNow Incident action not working as expected:
1. I have added SNOW instance as mentioned on the documentation into SolarWinds.
2. I gave in the required details and created an alert for -> Node Status is equal to Unmanage (which is the trigger condition)
3. Under trigger actions I created a action 'Create ServiceNow Incident' assigned the SNOW instance added in NPM
4. Gave the default fields and submitted the same, without proceeding further from that step i tried to simulate it from trigger actions page, clicked on the lightning symbol which is used for simulate and it successfully created a SNOW ticket, so far so good, hence i assumed this should work fine as simulating the action is working fine and creating a ticket in SNOW.
5. Later i went on to Summary step and saved the alert.
6. An alert fired within a minute and i could see it under Active alerts
ISSUE -> A TICKET WASN'T CREATED ON SNOW
What should i look for now ? Which log in SolarWinds would provide some additional info on the same, would MacroParser log help or ?
If someone has faced a similar issue, please do help me with the remediation, thanks in advance.
Polling Engine Shows as DOWN with Last Database Sync "2760 minutes ago"
Hi All,
We are having NPM 12.2 and Orion Platform as 2017.3. We have only one polling engine and it shows as below. Almost, since 48 hours, the sync is not happening. We checked and found the time zone is same on both polling engine and DB server. We have not changed any password for Orion DB account and also got confirmation from DB team, that password was not changed recently (in last 3 days).
Also, we are experiencing one more issue, since the above was observed. When we do a List Resources on an existing devices, where some changes happend on the device, it was taking hell lot of time and never show the resources. Just showing resources being discovered. Was it related to the above issue?
Note: QUICK RESPONSE WOULD BE MUCH APPRECIATED.
What We're Working on for NPM (Updated June 1st, 2018)
NPM 12.3 has shipped and we're hard at work building the next release. Here's what we're working on, in no particularly order.
- Cisco ACI Monitoring
- Remote Collector - New, agent based collector for distributed environments and hybrid deployments
- Next Generation Orion Mapping - First version delivered in NPM 12.3 via Orion Platform 2018.2. Working on the next version.
- Centralized Upgrades
- Website & Database Performance Improvements
- Windows Device Guard Support
- SAML Authentication Support
- Replace syslog/trap with the functionality from our new Log Manager product
VMWARE ESXI 6.5 MONITORING
Hello. I have a question on monitoring for my ESXi hosts on Solarwinds. Details for solarwinds as follows:
NPM: 12.2
SAM: 6.5
I have port 161 open for SNMP polling only from Solarwinds to ESXi due to strict security requirements.
Current configuration done is I am able to see the nodes status of the ESXi and the VMs.
What I would like to configure is to monitor the physical components of the ESXi hosts. I am currently using Fujitsu servers with custom image. I would like to know if I should be using CIM or SNMP/OID to poll the ESXi, or should I send snmp traps to Solarwinds?
how to monitor Tunnel health
Anyone has any idea, how to monitor tunnel health?
I have tunnel configured with /30 IP. Can ping end to end. I want to know the tunnel health/ stability before taking it to production.
Alert me when a component goes down question
Hello everyone,
I am very new to SolarWinds.
We get a ton of alerts under the "alert me when a component goes down." The alert lists all the components for a few nodes.
Is there any way to structure this alert so that the alert gives you the node and when selected you can see the many components within it?
I appreciate any help.
Network Discovery automatically add missing volumes to existing nodes
I can see a few have tried to solve this over the last few years and wanted to see if there are any new solutions based on the latest releases.
The goal is to automatically add new volumes to existing nodes. I am resigned to the fact Solarwinds is not going to detect that a volume's label has changed and make the update automatically so I have created a custom alert that deletes 'stale' volumes directly from SQL. The next stage is to use Network discover to bring those disks or newly added disks back in.
Currently I am trying to get Network Discover to run against a list of servers pulled from AD. Under the automatic monitoring settings I have deselected everything except fixed disks.
The issue is that although the nodes scanned, about 400, are already monitored by NPM the auto-import will fail because I only have 200 free licences. It errors with License Exceeded. I can do a manual import after the scan and it will pass the licence error and proceed to import missing disks but the goal is to have this be a background automatic process.
Has anyone faced the specific licence issue I have or does anyone have a fresh angle for adding volumes to existing nodes?
After NPM 12.3 upgradation getting an error & How to fix this error permanently and what can be the reason behind it?
Hi All,
I am getting an error after "2018.2" NPM 12.3 upgradation. Database connectivity being the loss and getting below error. What can be the solution?
Service was unable to open new database connection when requested. SqlException: A connection was successfully established with the server, but then an error occurred during the pre-login handshake. (provider: SSL Provider, error: 0 - The wait operation timed out.) Connection string - Data Source=vmswdbp01.hersheys.com;Initial Catalog=SolarWindsOrion;Persist Security Info=False;User ID=swdbuser;Password=*******;Max Pool Size=1000;Connect Timeout=20;Load Balance Timeout=120;Packet Size=4096;Application Name=SolarWinds.ServiceHost.Process@Service-SolarWinds.Cortex.DataCollection;Workstation ID=vmswprodserver08
Orion Platform 2018.2 HF3, UDT 3.3.1, VNQM 4.5.0, SRM 6.6.0, WPM 2.2.2, DPAIM 11.1.0, NPM 12.3, VMAN 8.2.1 HF1, NetPath 1.1.3, CloudMonitoring 2.0.1, SAM 6.6.1, NTA 4.2.3
How ICMP polling works? Node is down (not pinging as well)but still status is up
Group Membership
Hi,
Most of our NPM alerts are applied to groups, with most devices added to the relevant groups by means of a dynamic query.
For assurance purposes I'd like to create a report that shows each device and which group(s) each is assigned to. Is there an easy way to do this, or an 'out of the box' report that I can run?
Alternatively, if the above is difficult to create, a report showing any device which doesn't belong to be a group would be helpful.
Any ideas?
Uninstall of default database after migration
Hello,
I recently installed SolarWinds Orion in my environment. The design plan was to have the majority of components such as NPM installed on one server with its database on a separate server. Then to have the NTA database installed on a third server. Everything seems to have installed correctly but during the install of NPM, it installed a SQL instance.
Since NPM and NTA have their own dedicated DB servers, I would like to remove the SQL instance which was installed on the NPM server to reduce the amount of DBs floating around. Is it possible to remove the database or does SolarWinds still require it to be there even if other DBs are being utilized?
Thanks!
Top_50_CPU_Growth_-_Last_30_Days.zip
Netpath Probe talking to the APE rather than the Core server.
Hi is it possible for the Netpath probe to be installed so that it talks back to one of the polling engines rather than the core Server. Don't really want to expose the Core server to our customer networks if I can use the APE instead. to send the netpath traffic back to the core the same as it does for normal device polling.
Thanks
How to rebuild an APE
Hi Team,
I have to rebuild an APE. The platform version is 2017.3 HF3
While reinstalling the APE on a new box, we are getting the error: Algorithm has stuck! Unable to install any product on setup.
Is there any other way to install or I have to choose the legacy version?
SNMP v3 traps failing "Bad trap packet received: Decryption of incoming packet failed"
Hi all,
We are having trouble receiving traps from Aruba 207 access points. We configure the devices with SNMP v3 and when we add the APs as nodes the credentials test successfully and the nodes are added/polled without issue. However, we continue to receive the following error messages in regards to traps: Bad trap packet received from Node 10.1.1.2. Error description : Decryption of incoming packet failed (local user credentials are not the right ones or the packet is malformed).
I've tried changing the credentials for individual APs and re-adding them but the bad trap error continues to appear.
Here's the GUI for the SNMP config on the Arubas
Any help is appreciated, thanks.
Cisco ASA as a Default gateway?
There has been over 1 million of Cisco ASAs built and deployed around the world in a past years (good job Cisco!). As not supporting SNMP protocol properly, they often complicate our lives when configured to be Default gateways, not allowing products like SolarWinds User Device Tracker to monitor what's behind. Do you use your ASAs as a Default gateways too? If so, what percentage of your network they route?