Frankly speaking, OpenStack Ceilometer will suffer some kind of performance issues sooner or later if you don't modify or tune the configuration. The issues has two parts that need you to consider. One is the message bus and API loading, and the other is database. However, I find some best practices which are easy and quick for us to adopt. Here you go:
1. Telemetry(Ceilometer) best practices
a. Data collection
- Based on your needs, you can edit the pipeline.yaml configuration file to include a selected number of meters while disregarding the rest.
- By default, Telemetry service polls the service APIs every 10 minutes. You can change the polling interval on a per meter basis by editing the pipeline.yaml configuration file.
for example:
vim /etc/ceilometer/ceilometer.conf=> evaluation_interval=120vim /etc/ceilometer/pipeline.yaml=> interval: 120 - you can delay or adjust polling requests by enabling the jitter support. This adds a random delay on how the polling agents send requests to the service APIs. To enable jitter, set shuffle_time_before_polling_task in the ceilometer.conf configuration file to an integer greater than 0.
b. Data storage
- We recommend that you avoid open-ended queries.
- You can install the API behind mod_wsgi, as it provides more settings to tweak, likethreads and processes in case of WSGIDaemon. a. For more information on how to configure mod_wsgi, see the Telemetry Install Documentation.
- The collection service provided by the Telemetry project is not intended to be an archival service. Set a Time to Live (TTL) value to expire data and minimize the database size.
for example:
vi /etc/ceilometer/ceilometer.conf=> time_to_live=302400 - Use replica sets in MongoDB. Replica sets provide high availability through automatic failover. If your primary node fails, MongoDB will elect a secondary node to replace the primary node, and your cluster will remain functional.
- For more information on replica sets, see the MongoDB replica sets docs.
- Use sharding in MongoDB. Sharding helps in storing data records across multiple machines and is the MongoDB’s approach to meet the demands of data growth.
2. Metering Service (Ceilometer): Best Practices and Optimization
a. Modifying the List of Meters
sources:
- name: meter_source
interval: 604800
meters:
- "instance"
- "image"
- "image.size"
- "image.upload"
- "image.delete"
- "volume"
- "volume.size"
- "snapshot"
- "snapshot.size"
- "ip.floating"
- "network.*"
- "compute.instance.create.end"
- "compute.instance.delete.end"
- "compute.instance.update"
- "compute.instance.exists"
sinks:
- meter_sink
sinks:
- name: meter_sink
transformers:
publishers:
- notifier://
b. Modifying the Polling Intervals
The interval attribute is the time between polls. Meters that are available as both notification and polling are going to be polled at the specified interval. To rely on notifications rather than polling, set the interval attribute to 604800 seconds, or once a week.
Reference
One of the main issues operators relayed was the polling that Ceilometer was running on Nova to gather instance information. It had a highly negative impact on the Nova API CPU usage, as it retrieves all the information about instances on regular intervals.
