阿里云Elasticsearch团队自研的TimeStream时序引擎增强插件,支持使用PromQL查询Elasticsearch数据,无缝对接Prometheus+Grafana。本文介绍如何基于Elasticsearch TimeStream使用Prometheus集成接口。
背景信息
TimeStream是阿里云Elasticsearch团队自研,并结合Elastic社区时序类产品特性共建的时序引擎。阿里云Elasticsearch提供的TimeStream时序增强功能插件,优化了Elasticsearch在存储指标数据方面的DSL(Domain-Specific Language)查询复杂且慢以及存储成本过高等问题。详细信息请参见TimeStream时序增强引擎介绍。
前提条件
已创建阿里云Elasticsearch实例,且实例版本为7.10(内核版本为1.8.0及以上)或7.16及以上(内核版本为1.7.0及以上)。具体操作请参见创建阿里云Elasticsearch实例。
Prometheus API说明
Prometheus API除了可以访问通过Prometheus Remote Write方式写入的数据,也可以访问任意TimeStream索引。
Prometheus API支持去掉返回的指标和维度的前后缀关键字。如果使用默认的数据模型,指标前缀metric.会被去掉,维度前缀label.会被去掉。
{
"labels": {
"instance": "127.0.0.1:9114",
"job": "elasticsearch"
},
"metrics": {
"up": 0.0,
"scrape_samples_post_metric_relabeling": 0.0,
"scrape_samples_scraped": 0.0,
"scrape_duration_seconds": 4.45999E-4,
"scrape_series_added": 0.0
},
"@timestamp": "1655717638795"
}
{
"status" : "success",
"data" : [
"__name__",
"instance",
"job"
]
}
{
"status" : "success",
"data" : {
"scrape_samples_post_metric_relabeling" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_samples_scraped" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_duration_seconds" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"scrape_series_added" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
],
"up" : [
{
"type" : "gauge",
"help" : "",
"unit" : ""
}
]
}
}
PUT _time_stream/{name}
{
"time_stream": {
"labels_fields": "@labels.*_l",
"metrics_fields": "@metrics.*_m",
"label_prefix": "@labels.",
"label_suffix": "_l",
"metric_prefix": "@metrics.",
"metric_suffix": "_m"
}
}
RemoteWrite接口
API
POST /_time_stream/prom_write/{index}
---PB data---
使用说明
remote_write:
- url: "http://127.0.0.1:9200/_time_stream/prom_write/prom_index"
basic_auth:
username: elastic
password: xxxx
Prometheus通过Remote Write写入数据到TimeStream索引,只支持默认的时序数据模型,即维度前缀是label.,指标前缀是metric.。
Instant Query接口
API
GET /_time_stream/prom/{index}/query
POST /_time_stream/prom/{index}/query
使用说明
Prometheus Instant Query接口,详细信息请参见Prometheus instant queries。
Range Query接口
API
GET /_time_stream/prom/{index}/query_range
POST /_time_stream/prom/{index}/query_range
使用说明
Prometheus Range Query接口,详细信息请参见Prometheus range queries。
Series Metadata接口
API
GET /_time_stream/prom/{index}/series
POST /_time_stream/prom/{index}/series
使用说明
Prometheus查询时间线的接口,详细信息请参见Prometheus finding series by label matchers。
Labels Metadata接口
API
GET /_time_stream/prom/{index}/labels
POST /_time_stream/prom/{index}/labels
使用说明
Prometheus查询维度(label)列表的接口,详细信息请参见Prometheus getting label names。
Label Values Metadata接口
API
GET /_time_stream/prom/{index}/label/<label_name>/values
使用说明
Prometheus查询维度对应values列表的接口,详细信息请参见Prometheus querying label values。
Metric Metadata接口
API
GET /_time_stream/prom/{index}/metadata
使用说明
Prometheus查询指标(Metrics)列表的接口,详细信息请参见Prometheus querying metric metadata。
TimeStream PromQL支持情况
表达式支持情况
表达式 | 类型 | 是否支持 | 说明 |
powOp | binaryOperator | 是 | 幂乘 |
unaryOp | binaryOperator | 是 | 正、负 |
multOp | binaryOperator | 是 | 乘、除、取模 |
addOp | binaryOperator | 是 | 加、减 |
compareOp | binaryOperator | 是 | 比较(==,>=, >, <=, <, !=) |
andUnlessOp | binaryOperator | 是 | 集合运算(and、or、unless) |
orOp | binaryOperator | 是 | 集合运算(or) |
function | vector | 是 | 函数 |
aggregation | vector | 是 | 聚合运算 |
instantSelector | vector | 是 | 瞬时选择器 |
matrixSelector | vector | 是 | 向量选择器 |
offset | vector | 是 | 偏移量 |
literal | vector | 是 | 常量 |
labelMatcher | vector | 是 | 标签匹配 |
grouping | group | 否 | group聚合,包括on、ignoring、group_left、group_right |
Aggregation operator支持情况
Aggregation operator | 是否支持 |
sum(calculate sum over dimensions) | 是 |
min(select minimum over dimensions) | 是 |
max(select maximum over dimensions) | 是 |
avg(calculate the average over dimensions) | 是 |
group(all values in the resulting vector are 1) | 否 |
stddev(calculate population standard deviation over dimensions) | 否 |
stdvar(calculate population standard variance over dimensions) | 否 |
count(count number of elements in the vector) | 是 |
count_values(count number of elements with the same value) | 是 |
bottomk(smallest k elements by sample value) | 是 |
topk(largest k elements by sample value) | 是 |
quantile(calculate φ-quantile (0 ≤ φ ≤ 1) over dimensions) | 是 |
Function支持情况
Function | 是否支持 |
abs(v instant-vector) | 是 |
absent(v instant-vector) | 否 |
absent_over_time(v range-vector) | 否 |
ceil(v instant-vector) | 是 |
changes(v range-vector) | 否 |
clamp(v instant-vector, min scalar, max scalar) | 是 |
clamp_max(v instant-vector, max scalar) | 是 |
clamp_min(v instant-vector, min scalar) | 是 |
day_of_month(v=vector(time()) instant-vector) | 是 |
day_of_week(v=vector(time()) instant-vector) | 是 |
days_in_month(v=vector(time()) instant-vector) | 否 |
delta(v range-vector) | 是 |
deriv(v range-vector) | 否 |
exp(v instant-vector) | 是 |
floor(v instant-vector) | 是 |
histogram_quantile(φ scalar, b instant-vector) | 否 |
holt_winters(v range-vector, sf scalar, tf scalar) | 否 |
hour(v=vector(time()) instant-vector) | 是 |
idelta(v range-vector) | 是 |
increase(v range-vector) | 是 |
irate(v range-vector) | 是 |
label_join(v instant-vector, dst_label string, separator string, src_label_1 string, src_label_2 string, ...) | 否 |
label_replace(v instant-vector, dst_label string, replacement string, src_label string, regex string) | 否 |
ln(v instant-vector) | 是 |
log2(v instant-vector) | 否 |
log10(v instant-vector) | 是 |
minute(v=vector(time()) instant-vector) | 是 |
month(v=vector(time()) instant-vector) | 是 |
predict_linear(v range-vector, t scalar) | 否 |
rate(v range-vector) | 是 |
resets(v range-vector) | 否 |
round(v instant-vector, to_nearest=1 scalar) | 否 |
scalar(v instant-vector) | 否 |
sgn(v instant-vector) | 是 |
sort(v instant-vector) | 否 |
sort_desc() | 否 |
sqrt(v instant-vector) | 是 |
time() | 否 |
timestamp(v instant-vector) | 是 |
vector(s scalar) | 否 |
year(v=vector(time()) instant-vector) | 是 |
avg_over_time(range-vector) | 是 |
min_over_time(range-vector) | 是 |
max_over_time(range-vector) | 是 |
sum_over_time(range-vector) | 是 |
count_over_time(range-vector) | 是 |
quantile_over_time(scalar, range-vector) | 是 |
stddev_over_time(range-vector) | 否 |
stdvar_over_time(range-vector) | 否 |
last_over_time(range-vector) | 是 |
present_over_time(range-vector) | 否 |
acos(v instant-vector) | 是 |
acosh(v instant-vector) | 是 |
asin(v instant-vector) | 是 |
asinh(v instant-vector) | 否 |
atan(v instant-vector) | 是 |
atanh(v instant-vector) | 否 |
cos(v instant-vector) | 是 |
cosh(v instant-vector) | 是 |
sin(v instant-vector) | 是 |
sinh(v instant-vector) | 是 |
tan(v instant-vector) | 是 |
tanh(v instant-vector) | 是 |
deg(v instant-vector) | 是 |
pi() | 是 |
rad(v instant-vector) | 是 |