prometheus
This sink publishes events processed by stream worker into Prometheus metrics and exposes them to the Prometheus server at the specified URL. The created metrics can be published to Prometheus via server or pushGateway, depending on your preference. The metric types that are supported by the Prometheus sink are counter, gauge, histogram, and summary. The values and labels of the Prometheus metrics can be updated through the events.
Syntax
CREATE SINK <NAME> WITH (type="prometheus", map.type="<STRING>", job="<STRING>", publish.mode="<STRING>", push.url="<STRING>", server.url="<STRING>", metric.type="<STRING>", metric.help="<STRING>", metric.name="<STRING>", buckets="<STRING>", quantiles="<STRING>", quantile.error="<DOUBLE>", value.attribute="<STRING>", push.operation="<STRING>", grouping.key="<STRING>")
Query Parameters
| Name | Description | Default Value | Possible Data Types | Optional | Dynamic |
|---|---|---|---|---|---|
| job | This parameter specifies the job name of the metric. This must be the same job name that is defined in the Prometheus configuration file. | stream processor Job | STRING | Yes | No |
| publish.mode | The mode in which the metrics need to be exposed to the Prometheus server.The possible publishing modes are server and pushgateway. The server mode exposes the metrics through an HTTP server at the specified URL, and the pushGateway mode pushes the metrics to the pushGateway that needs to be running at the specified URL. | server | STRING | Yes | No |
| push.url | This parameter specifies the target URL of the Prometheus pushGateway. This is the URL at which the pushGateway must be listening. This URL needs to be defined in the Prometheus configuration file as a target before it can be used here. | http://localhost:9091 | STRING | Yes | No |
| server.url | This parameter specifies the URL where the HTTP server is initiated to expose metrics in the server publish mode. This URL needs to be defined in the Prometheus configuration file as a target before it can be used here. | http://localhost:9080 | STRING | Yes | No |
| metric.type | The type of Prometheus metric that needs to be created at the sink. The supported metric types are counter, gauge, histogram and summary. | STRING | No | No | |
| metric.help | A brief description of the metric and its purpose. | STRING | Yes | No | |
| metric.name | This parameter allows you to assign a preferred name for the metric. The metric name must match the regex format, i.e., [a-zA-Z:][a-zA-Z0-9:]*. | STRING | Yes | No | |
| buckets | The bucket values preferred by the user for histogram metrics. The bucket values must be in the string format with each bucket value separated by a comma as shown in the example below. "2,4,6,8" | null | STRING | Yes | No |
| quantiles | This parameter allows you to specify quantile values for summary metrics as preferred. The quantile values must be in the string format with each quantile value separated by a comma as shown in the example below. "0.5,0.75,0.95" | null | STRING | Yes | No |
| quantile.error | The error tolerance value for calculating quantiles in summary metrics. This must be a positive value, but less than 1. | 0.001 | DOUBLE | Yes | No |
| value.attribute | The name of the attribute in the stream definition that specifies the metric value. The defined value attribute must be included in the stream definition. The system increases the metric value for the counter and gauge metric types by the value of the value attribute. The system observes the value of the value attribute for the calculations of summary and histogram metric types. | value | STRING | Yes | No |
| push.operation | This parameter defines the mode for pushing metrics to the pushGateway. The available push operations are push and pushadd. The operations differ according to the existing metrics in pushGateway where push operation replaces the existing metrics, and pushadd operation only updates the newly created metrics. | pushadd | STRING | Yes | No |
| grouping.key | This parameter specifies the grouping key of created metrics in key-value pairs. The grouping key is used only in pushGateway mode in order to distinguish the metrics from already existing metrics. The expected format of the grouping key is as follows: "key1:value1,key2:value2" | STRING | Yes | No |
System Parameters
| Name | Description | Default Value | Possible Parameters |
|---|---|---|---|
| jobName | This property specifies the default job name for the metric. This job name must be the same as the job name defined in the Prometheus configuration file. | stream processor Job | Any string |
| publishMode | The default publish mode for the Prometheus sink for exposing metrics to the Prometheus server. The mode can be either server or pushgateway. | server | server or pushgateway |
| serverURL | This property configures the URL where the HTTP server is initiated to expose metrics. This URL needs to be defined in the Prometheus configuration file as a target to be identified by Prometheus before it can be used here. By default, the HTTP server is initiated at http://localhost:9080. | http://localhost:9080 | Any valid URL |
| pushURL | This property configures the target URL of the Prometheus pushGateway (where the pushGateway needs to listen). This URL needs to be defined in the Prometheus configuration file as a target to be identified by Prometheus before it can be used here. | http://localhost:9091 | Any valid URL |
| groupingKey | This property configures the grouping key of created metrics in key-value pairs. Grouping key is used only in pushGateway mode in order to distinguish these metrics from already existing metrics under the same job. The expected format of the grouping key is as follows: "key1:value1,key2:value2" . | null | Any key value pairs in the supported format |
Example 1
CREATE SINK FooCountStream WITH (type='prometheus',job='fooOrderCount', server.url ='http://localhost:9080', publish.mode='server', metric.type='counter', metric.help= 'Number of foo orders', map.type='keyvalue') (Name String, quantity int, value int);
In the above example, the Prometheus-sink creates a counter metric with the stream name and defined attributes as labels. The metric is exposed through an HTTP server at the target URL.
Example 2
CREATE SINK InventoryLevelStream WITH (type='prometheus',job='inventoryLevel', push.url='http://localhost:9080', publish.mode='pushGateway', metric.type='gauge', metric.help= 'Current level of inventory', map.type='keyvalue') (Name String, value int);
In the above example, the Prometheus-sink creates a gauge metric with the stream name and defined attributes as labels.The metric is pushed to the Prometheus pushGateway at the target URL.