![]() Starting with Splunk ITSI version 4.0.x, the ITSI Health Check dashboard provides these statistics: How do I know how many KPIs are associated with a single KPI base search: Limit = (number of KPIs * number of entities for each service) + (number of services) * 2Įx: A KPI base search is powering 5,000 KPIs across 500 services. * The maximum number of results to load when triggering an alert action. You must increase the value of the following setting in nf: This number is not enough in a large-scale environment and will produce mysterious “N/A” results on your service or KPI tiles. By default, the system only transports 50,000 rows of results to the summary index. Splunk ITSI transports the processed search results to the ITSI summary index (itsi_summary) through alert_nf. A single KPI base search produces more rows of results when more services and entities are involved.This will cause delayed KPI alert values and health score results, and means you have too many KPIs tied to a single KPI base search. If the KPI base search is scheduled to run every minute but the actual search execution takes longer than a minute, the next scheduled search will be skipped. Go to the search inspector and check the search execution stats. When configuring a KPI base search, consider the following recommendations: ![]() Use the following guidelines to decide on the correct number of KPIs to be powered by a single KPI base search. In Splunk ITSI, KPI base searches are recommended to minimize the overall search load at the Splunk Enterprise level. ![]() Best Practice #3: Use KPI base searches to power multiple KPIs ![]() The recommendation is be mindful of the performance implication when you have a lot of entities matched for a single service. Splunk ITSI does not limit the number of matching entities for a service. If you’re matching service-level entity rules to tens and thousands of entities, it can be difficult to monitor the entities that are of interest, and can slow internal operations. Use entity rules that are prescriptive enough that you’re catching the entities you care about for that service. Best Practice #2: Use entity rules to filter to the entities you care about within your serviceĮntity rules within a service ensure that you’re dynamically filtering to the entities that matter in your environment. It’s best to have no more than 20 KPIs per individual service-more than enough to capture the key metrics you care about (like CPU, IO, disk free, and response time). So what is the recommended number of KPIs for a single service? You’ll save yourself time troubleshooting later. So spend time crafting and fostering the KPIs that you really care about and want to measure. Part of the beauty of Splunk ITSI is that it makes it easy to focus on what matters in your environment. ![]() How do you effectively monitor and troubleshoot the service when there are that many KPIs involved? I’ve seen cases where the customer configured more than 50 KPIs in a service. It's not good to have so many KPIs in a single service that you can barely keep track of them all. Best Practice #1: Focus on the KPIs that matter It's NOT a complete list of Splunk ITSI configuration guidelines check out the Splunk ITSI Documentation for more in-depth information about that. This blog post provides a sample of best practices for configuring a large-scale Splunk ITSI deployment. Additionally, Splunk ITSI can scale to support monitoring of thousands of services and tens of thousands of entities. Traceback ( most recent call last ) : File " /opt/splunk/etc/apps/SA-ITOA/lib/itsi/backup_restore/itsi_backup_restore_utils.py ", line 678, in run worker.execute () File " /opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py ", line 1172, in execute self.restore () File " /opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py ", line 1148, in restore raise e File " /opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py ", line 1140, in restore self.restore_from_folder () File " /opt/splunk/etc/apps/SA-ITOA/lib/itsi/upgrade/kvstore_backup_restore.py ", line 978, in restore_from_folder raise Exception ( failure_msg ) Exception: Restore failed, ].Part of the beauty of Splunk IT Service Intelligence (ITSI) is that it provides users with flexible models of their entities and services. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |