Skip to content

Commit ee8eb02

Browse files
Update upstream alerts
1 parent fb4131e commit ee8eb02

File tree

2 files changed

+28
-27
lines changed

2 files changed

+28
-27
lines changed

component/extracted_alerts/master/lokistack_prometheus_alerts.yaml

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ groups:
44
rules:
55
- alert: LokiRequestErrors
66
annotations:
7-
message: |-
7+
description: |-
88
{{ $labels.job }} {{ $labels.route }} is experiencing {{ printf "%.2f" $value }}% errors.
99
summary: "At least 10% of requests are responded by 5xx server errors."
1010
runbook_url: "[[ .RunbookURL ]]#Loki-Request-Errors"
@@ -23,7 +23,7 @@ groups:
2323
severity: critical
2424
- alert: LokiStackWriteRequestErrors
2525
annotations:
26-
message: |-
26+
description: |-
2727
{{ printf "%.2f" $value }}% of write requests from {{ $labels.job }} in {{ $labels.namespace }} are returned with server errors.
2828
summary: "At least 10% of write requests to the lokistack-gateway are responded with 5xx server errors."
2929
runbook_url: "[[ .RunbookURL ]]#LokiStack-Write-Request-Errors"
@@ -42,7 +42,7 @@ groups:
4242
severity: critical
4343
- alert: LokiStackReadRequestErrors
4444
annotations:
45-
message: |-
45+
description: |-
4646
{{ printf "%.2f" $value }}% of query requests from {{ $labels.job }} in {{ $labels.namespace }} are returned with server errors.
4747
summary: "At least 10% of query requests to the lokistack-gateway are responded with 5xx server errors."
4848
runbook_url: "[[ .RunbookURL ]]#LokiStack-Read-Request-Errors"
@@ -61,7 +61,7 @@ groups:
6161
severity: critical
6262
- alert: LokiRequestPanics
6363
annotations:
64-
message: |-
64+
description: |-
6565
{{ $labels.job }} is experiencing an increase of {{ $value }} panics.
6666
summary: "A panic was triggered."
6767
runbook_url: "[[ .RunbookURL ]]#Loki-Request-Panics"
@@ -76,7 +76,7 @@ groups:
7676
severity: critical
7777
- alert: LokiRequestLatency
7878
annotations:
79-
message: |-
79+
description: |-
8080
{{ $labels.job }} {{ $labels.route }} is experiencing {{ printf "%.2f" $value }}s 99th percentile latency.
8181
summary: "The 99th percentile is experiencing high latency (higher than 1 second)."
8282
runbook_url: "[[ .RunbookURL ]]#Loki-Request-Latency"
@@ -94,7 +94,7 @@ groups:
9494
severity: critical
9595
- alert: LokiTenantRateLimit
9696
annotations:
97-
message: |-
97+
description: |-
9898
{{ $labels.job }} {{ $labels.route }} is experiencing 429 errors.
9999
summary: "At least 10% of requests are responded with the rate limit error code."
100100
runbook_url: "[[ .RunbookURL ]]#Loki-Tenant-Rate-Limit"
@@ -113,7 +113,7 @@ groups:
113113
severity: warning
114114
- alert: LokiStorageSlowWrite
115115
annotations:
116-
message: |-
116+
description: |-
117117
The storage path is experiencing slow write response rates.
118118
summary: "The storage path is experiencing slow write response rates."
119119
runbook_url: "[[ .RunbookURL ]]#Loki-Storage-Slow-Write"
@@ -129,7 +129,7 @@ groups:
129129
severity: warning
130130
- alert: LokiStorageSlowRead
131131
annotations:
132-
message: |-
132+
description: |-
133133
The storage path is experiencing slow read response rates.
134134
summary: "The storage path is experiencing slow read response rates."
135135
runbook_url: "[[ .RunbookURL ]]#Loki-Storage-Slow-Read"
@@ -145,7 +145,7 @@ groups:
145145
severity: warning
146146
- alert: LokiWritePathHighLoad
147147
annotations:
148-
message: |-
148+
description: |-
149149
The write path is experiencing high load.
150150
summary: "The write path is experiencing high load, causing backpressure storage flushing."
151151
runbook_url: "[[ .RunbookURL ]]#Loki-Write-Path-High-Load"
@@ -159,7 +159,7 @@ groups:
159159
severity: warning
160160
- alert: LokiReadPathHighLoad
161161
annotations:
162-
message: |-
162+
description: |-
163163
The read path is experiencing high load.
164164
summary: "The read path has high volume of queries, causing longer response times."
165165
runbook_url: "[[ .RunbookURL ]]#Loki-Read-Path-High-Load"
@@ -177,7 +177,7 @@ groups:
177177
severity: warning
178178
- alert: LokiDiscardedSamplesWarning
179179
annotations:
180-
message: |-
180+
description: |-
181181
Loki in namespace {{ $labels.namespace }} is discarding samples in the "{{ $labels.tenant }}" tenant during ingestion.
182182
Samples are discarded because of "{{ $labels.reason }}" at a rate of {{ .Value | humanize }} samples per second.
183183
summary: Loki is discarding samples during ingestion because they fail validation.
@@ -195,7 +195,7 @@ groups:
195195
severity: warning
196196
- alert: LokistackSchemaUpgradesRequired
197197
annotations:
198-
message: |-
198+
description: |-
199199
The LokiStack "{{ $labels.stack_name }}" in namespace "{{ $labels.stack_namespace }}" is using a storage schema
200200
configuration that does not contain the latest schema version. It is recommended to update the schema
201201
configuration to update the schema version to the latest version in the future.

tests/golden/master/openshift4-logging/openshift4-logging/60_lokistack_alerts.yaml

Lines changed: 16 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ spec:
1111
rules:
1212
- alert: SYN_LokiRequestErrors
1313
annotations:
14-
message: '{{ $labels.job }} {{ $labels.route }} is experiencing {{ printf
15-
"%.2f" $value }}% errors.'
14+
description: '{{ $labels.job }} {{ $labels.route }} is experiencing {{
15+
printf "%.2f" $value }}% errors.'
1616
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Request-Errors
1717
summary: At least 10% of requests are responded by 5xx server errors.
1818
expr: |
@@ -32,7 +32,7 @@ spec:
3232
syn_component: openshift4-logging
3333
- alert: SYN_LokiStackWriteRequestErrors
3434
annotations:
35-
message: '{{ printf "%.2f" $value }}% of write requests from {{ $labels.job
35+
description: '{{ printf "%.2f" $value }}% of write requests from {{ $labels.job
3636
}} in {{ $labels.namespace }} are returned with server errors.'
3737
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#LokiStack-Write-Request-Errors
3838
summary: At least 10% of write requests to the lokistack-gateway are responded
@@ -54,7 +54,7 @@ spec:
5454
syn_component: openshift4-logging
5555
- alert: SYN_LokiStackReadRequestErrors
5656
annotations:
57-
message: '{{ printf "%.2f" $value }}% of query requests from {{ $labels.job
57+
description: '{{ printf "%.2f" $value }}% of query requests from {{ $labels.job
5858
}} in {{ $labels.namespace }} are returned with server errors.'
5959
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#LokiStack-Read-Request-Errors
6060
summary: At least 10% of query requests to the lokistack-gateway are responded
@@ -76,8 +76,8 @@ spec:
7676
syn_component: openshift4-logging
7777
- alert: SYN_LokiRequestPanics
7878
annotations:
79-
message: '{{ $labels.job }} is experiencing an increase of {{ $value }}
80-
panics.'
79+
description: '{{ $labels.job }} is experiencing an increase of {{ $value
80+
}} panics.'
8181
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Request-Panics
8282
summary: A panic was triggered.
8383
expr: |
@@ -93,8 +93,8 @@ spec:
9393
syn_component: openshift4-logging
9494
- alert: SYN_LokiRequestLatency
9595
annotations:
96-
message: '{{ $labels.job }} {{ $labels.route }} is experiencing {{ printf
97-
"%.2f" $value }}s 99th percentile latency.'
96+
description: '{{ $labels.job }} {{ $labels.route }} is experiencing {{
97+
printf "%.2f" $value }}s 99th percentile latency.'
9898
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Request-Latency
9999
summary: The 99th percentile is experiencing high latency (higher than
100100
1 second).
@@ -114,7 +114,8 @@ spec:
114114
syn_component: openshift4-logging
115115
- alert: SYN_LokiTenantRateLimit
116116
annotations:
117-
message: '{{ $labels.job }} {{ $labels.route }} is experiencing 429 errors.'
117+
description: '{{ $labels.job }} {{ $labels.route }} is experiencing 429
118+
errors.'
118119
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Tenant-Rate-Limit
119120
summary: At least 10% of requests are responded with the rate limit error
120121
code.
@@ -135,7 +136,7 @@ spec:
135136
syn_component: openshift4-logging
136137
- alert: SYN_LokiStorageSlowWrite
137138
annotations:
138-
message: The storage path is experiencing slow write response rates.
139+
description: The storage path is experiencing slow write response rates.
139140
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Storage-Slow-Write
140141
summary: The storage path is experiencing slow write response rates.
141142
expr: |
@@ -152,7 +153,7 @@ spec:
152153
syn_component: openshift4-logging
153154
- alert: SYN_LokiStorageSlowRead
154155
annotations:
155-
message: The storage path is experiencing slow read response rates.
156+
description: The storage path is experiencing slow read response rates.
156157
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Storage-Slow-Read
157158
summary: The storage path is experiencing slow read response rates.
158159
expr: |
@@ -169,7 +170,7 @@ spec:
169170
syn_component: openshift4-logging
170171
- alert: SYN_LokiWritePathHighLoad
171172
annotations:
172-
message: The write path is experiencing high load.
173+
description: The write path is experiencing high load.
173174
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Write-Path-High-Load
174175
summary: The write path is experiencing high load, causing backpressure
175176
storage flushing.
@@ -185,7 +186,7 @@ spec:
185186
syn_component: openshift4-logging
186187
- alert: SYN_LokiReadPathHighLoad
187188
annotations:
188-
message: The read path is experiencing high load.
189+
description: The read path is experiencing high load.
189190
runbook_url: https://github.com/grafana/loki/blob/main/operator/docs/lokistack/sop.md#Loki-Read-Path-High-Load
190191
summary: The read path has high volume of queries, causing longer response
191192
times.
@@ -205,7 +206,7 @@ spec:
205206
syn_component: openshift4-logging
206207
- alert: SYN_LokiDiscardedSamplesWarning
207208
annotations:
208-
message: |-
209+
description: |-
209210
Loki in namespace {{ $labels.namespace }} is discarding samples in the "{{ $labels.tenant }}" tenant during ingestion.
210211
Samples are discarded because of "{{ $labels.reason }}" at a rate of {{ .Value | humanize }} samples per second.
211212
runbook_url: '[[ .RunbookURL]]#Loki-Discarded-Samples-Warning'
@@ -226,7 +227,7 @@ spec:
226227
syn_component: openshift4-logging
227228
- alert: SYN_LokistackSchemaUpgradesRequired
228229
annotations:
229-
message: |-
230+
description: |-
230231
The LokiStack "{{ $labels.stack_name }}" in namespace "{{ $labels.stack_namespace }}" is using a storage schema
231232
configuration that does not contain the latest schema version. It is recommended to update the schema
232233
configuration to update the schema version to the latest version in the future.

0 commit comments

Comments
 (0)