全部產品
Search
文件中心

Application Real-Time Monitoring Service:通過OpenTelemetry上報Python應用資料

更新時間:Aug 10, 2024

通過OpenTelemetry為應用埋點並上報鏈路資料至Managed Service for OpenTelemetry後,Managed Service for OpenTelemetry即可開始監控應用,您可以查看應用拓撲、調用鏈路、異常事務、慢事務和SQL分析等一系列監控資料。本文介紹如何使用OpenTelemetry Python Agent/SDK進行自動或手動埋點並上報資料。

前提條件

擷取存取點資訊

  1. 登入ARMS控制台,在左側導覽列單擊接入中心

  2. 服務端應用地區單擊OpenTelemetry卡片。

  3. 在彈出的OpenTelemetry面板中選擇資料需要上報的地區。

    說明

    初次接入的地區將會自動進行資源初始化。

  4. 選擇串連方式上報方式,然後複製存取點資訊。

    • 串連方式:若您的服務部署在阿里雲上,且所屬地區與選擇的接入地區一致,推薦使用阿里雲內網方式,否則選擇公網方式。

    • 上報方式:根據用戶端支援的協議類型選擇HTTP或gRPC協議上報資料。

    image.png

背景資訊

OpenTelemetry提供了若干自動埋點外掛程式,支援為常見架構自動建立Span,支援的架構列表如下,完整資訊請參見OpenTelemetry官方文檔

說明

版本~= V.N表示≥ V.N== V.*,例如,aiohttp ~= 3.0表示aiohttp版本要求aiohttp ≥ 3.0aiohttp == 3.*

展開查看支援的Python架構

OpenTelemetry外掛程式

支援的Package以及版本

opentelemetry-instrumentation-aio-pika

aio_pika - [7.2.0, 10.0.0)

opentelemetry-instrumentation-aiohttp-client

aiohttp ~= 3.0

opentelemetry-instrumentation-aiohttp-server

aiohttp ~= 3.0

opentelemetry-instrumentation-aiopg

aiopg - [0.13.0, 2.0.0)

opentelemetry-instrumentation-asgi

asgiref ~= 3.0

opentelemetry-instrumentation-asyncpg

asyncpg ≥ 0.12.0

opentelemetry-instrumentation-aws-lambda

aws_lambda

opentelemetry-instrumentation-boto

boto ~= 2.0

opentelemetry-instrumentation-boto3sqs

boto3 ~= 1.0

opentelemetry-instrumentation-botocore

botocore ~= 1.0

opentelemetry-instrumentation-cassandra

  • cassandra-driver ~= 3.25

  • scylla-driver ~= 3.25

opentelemetry-instrumentation-celery

celery - [4.0, 6.0)

opentelemetry-instrumentation-confluent-kafka

confluent-kafka - [1.8.2, 2.2.0]

opentelemetry-instrumentation-dbapi

dbapi

opentelemetry-instrumentation-django

django ≥ 1.10

opentelemetry-instrumentation-elasticsearch

elasticsearch ≥ 2.0

opentelemetry-instrumentation-falcon

falcon - [1.4.1, 4.0.0)

opentelemetry-instrumentation-fastapi

fastapi ~= 0.58

opentelemetry-instrumentation-flask

flask - [1.0, 3.0)

opentelemetry-instrumentation-grpc

grpcio ~= 1.27

opentelemetry-instrumentation-httpx

httpx ≥ 0.18.0

opentelemetry-instrumentation-jinja2

jinja2 - [2.7, 4.0)

opentelemetry-instrumentation-kafka-python

kafka-python ≥ 2.0

opentelemetry-instrumentation-logging

logging

opentelemetry-instrumentation-mysql

mysql-connector-python ~= 8.0

opentelemetry-instrumentation-mysqlclient

mysqlclient < 3

opentelemetry-instrumentation-pika

pika≥ 0.12.0

opentelemetry-instrumentation-psycopg2

psycopg2 ≥ 2.7.3.1

opentelemetry-instrumentation-pymemcache

pymemcache - [1.3.5, 5)

opentelemetry-instrumentation-pymongo

pymongo - [3.1, 5.0)

opentelemetry-instrumentation-pymysql

PyMySQL < 2

opentelemetry-instrumentation-pyramid

pyramid ≥ 1.7

opentelemetry-instrumentation-redis

redis ≥ 2.6

opentelemetry-instrumentation-remoulade

remoulade ≥ 0.50

opentelemetry-instrumentation-requests

requests ~= 2.0

opentelemetry-instrumentation-sqlalchemy

sqlalchemy

opentelemetry-instrumentation-sqlite3

sqlite3

opentelemetry-instrumentation-starlette

starlette ~= 0.13.0

opentelemetry-instrumentation-system-metrics

psutil ≥ 5

opentelemetry-instrumentation-tornado

tornado ≥ 5.1.1

opentelemetry-instrumentation-tortoiseorm

tortoise-orm ≥ 0.17.0

opentelemetry-instrumentation-urllib

urllib

opentelemetry-instrumentation-urllib3

urllib3 - [1.0.0, 3.0.0)

opentelemetry-instrumentation-wsgi

wsgi

樣本Demo

範例程式碼倉庫地址:python-opentelemetry-demo

方法一:通過Agent自動埋點並上報

  1. 下載所需包。

    pip install django
    pip install requests
    pip install opentelemetry-distro \
    	opentelemetry-exporter-otlp
     
    opentelemetry-bootstrap -a install
  2. 建立AutoAndManualDemo專案並建立HelloWorld應用。

    1. 建立AutoAndManualDemo專案。

      django-admin startproject AutoAndManualDemo
    2. 在專案中建立HelloWorld應用。

      cd AutoAndManualDemo
      
      # 在專案中建立helloworld app
      python manage.py startapp helloworld
  3. 修改HelloWorld應用代碼。

    1. 在AutoAndManualDemo/helloworld/views.py檔案中添加以下代碼。

      from django.http import HttpResponse
      from datetime import datetime
      
      # Create your views here.
      def hello_world_view(request):
          result = "Hello World! Current Time =" + str(get_time())
          return HttpResponse(result)
      
      def get_time():
          now = datetime.now()
          return now.strftime("%H:%M:%S")
    2. 建立AutoAndManualDemo/helloworld/urls.py檔案,並在urls.py檔案中添加以下代碼。

      from django.urls import path
      
      from . import views
      
      urlpatterns = [
          path('', views.hello_world_view, name='helloworld')
      ]
    3. 修改AutoAndManualDemo/AutoAndManualDemo/urls.py檔案,添加helloworld的URL。

      from django.contrib import admin
      from django.urls import path, include
      
      urlpatterns = [
       path('admin/', admin.site.urls),
       path('helloworld/', include('helloworld.urls')),
      ]
  4. 運行專案。

    通過HTTP上報,請將<your-service-name>替換為您的應用程式名稱,將<http-endpoint>替換為HTTP存取點。

    opentelemetry-instrument \
        --traces_exporter console,otlp_proto_http \
        --metrics_exporter none \
        --service_name <your-service-name> \
        --exporter_otlp_traces_endpoint <http-endpoint> \
        python manage.py runserver --noreload
    說明
    • --noreload:避免manage.main方法執行兩次。

    • 如果運行報錯CommandError: You must set settings.ALLOWED_HOSTS if DEBUG is False.,但AutoAndManualDemo/AutoAndManualDemo/settings.py中的DEBUGALLOWED_HOSTS均已配置正確,這是因為使用opentelemetry-instrument啟動時使用了Django架構的預設設定檔(django/conf/global_settings.py), 因此需要添加export DJANGO_SETTINGS_MODULE=AutoAndManualDemo.settings環境變數。

  5. 在瀏覽器中訪問http://127.0.0.1:8000/helloworld/,控制台會列印Trace,同時也會將Trace上報至阿里雲Managed Service for OpenTelemetry

    如需關閉控制台列印Trace,只需將--traces_exporter參數配置為--traces_exporter otlp_proto_http

方法二:手動埋點並上報

  1. 下載所需包。

    pip install opentelemetry-api
    pip install opentelemetry-sdk
    pip install opentelemetry-exporter-otlp
  2. manual.py檔案中設定OpenTelemetry初始化代碼。

    • 請將代碼中的<token><endpoint>替換成前提條件中擷取的存取點資訊。

    • 請根據實際情況替換代碼中的<service-name>(服務名)和<host-name>(主機名稱)。

    from opentelemetry import trace, baggage
    from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter as OTLPSpanGrpcExporter
    from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter as OTLPSpanHttpExporter
    from opentelemetry.sdk.resources import SERVICE_NAME, Resource, HOST_NAME
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import BatchSpanProcessor
    
    def init_opentelemetry():
        # 設定服務名、主機名稱
        resource = Resource(attributes={
            SERVICE_NAME: "<service-name>",
            HOST_NAME: "<host-name>"
        })
        # 使用GRPC協議上報
        span_processor = BatchSpanProcessor(OTLPSpanGrpcExporter(
            endpoint="<endpoint>",
            headers=("Authentication=<token>")
        ))
        # 使用HTTP協議上報
        # span_processor = BatchSpanProcessor(OTLPSpanHttpExporter(
        #     endpoint="<endpoint>",
        # ))
        trace_provider = TracerProvider(resource=resource, active_span_processor=span_processor)
        trace.set_tracer_provider(trace_provider)
                            
  3. 建立Span。

    tracer = trace.get_tracer(__name__)  # 擷取tracer
    with tracer.start_as_current_span("child_span") as child_span:  # 建立名為child_span的span
        print("hello world")
  4. 使用OpenTelemetry Baggage API透傳業務自訂標籤。

    建立baggage_parent_span時通過指定attributes參數來設定屬性。

    def baggage_and_attribute_usage():
        tracer = trace.get_tracer(__name__)
        global_ctx = baggage.set_baggage("key", "value_from_global_ctx")  # 使用baggage api,在不同span之間傳遞資料
        with tracer.start_as_current_span(name='baggage_parent_span', attributes={'attribute_key': 'value'}) as baggage_parent_span:
            parent_ctx = baggage.set_baggage("key", "value_from_parent_ctx")
            with tracer.start_as_current_span(name='baggage_child_span', context=parent_ctx) as baggage_child_span:
                child_ctx = baggage.set_baggage("key", "value_from_child_ctx")
    
        # 擷取不同contex下key對應的值
        print(baggage.get_baggage("key", global_ctx))
        print(baggage.get_baggage("key", parent_ctx))
        print(baggage.get_baggage("key", child_ctx))
        

    展開查看完整範例程式碼

    from opentelemetry import trace, baggage
    from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter as OTLPSpanGrpcExporter
    from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter as OTLPSpanHttpExporter
    from opentelemetry.sdk.resources import SERVICE_NAME, Resource, HOST_NAME
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import BatchSpanProcessor
    
    def inner_method():
        tracer = trace.get_tracer(__name__)
        with tracer.start_as_current_span("child_span") as child_span:
            print("hello world")
    
    
    def outer_method():
        tracer = trace.get_tracer(__name__)
        with tracer.start_as_current_span("parent_span") as parent_span:
            inner_method()
    
    def baggage_and_attribute_usage():
        tracer = trace.get_tracer(__name__)
        global_ctx = baggage.set_baggage("key", "value_from_global_ctx")  # 使用baggage api,在不同span之間傳遞資料
        with tracer.start_as_current_span(name='baggage_parent_span', attributes={'attribute_key': 'value'}) as baggage_parent_span:
            parent_ctx = baggage.set_baggage("key", "value_from_parent_ctx")
            with tracer.start_as_current_span(name='baggage_child_span', context=parent_ctx) as baggage_child_span:
                child_ctx = baggage.set_baggage("key", "value_from_child_ctx")
    
        print(baggage.get_baggage("key", global_ctx))
        print(baggage.get_baggage("key", parent_ctx))
        print(baggage.get_baggage("key", child_ctx))
    
    
    
    def init_opentelemetry():
        # 設定服務名、主機名稱
        resource = Resource(attributes={
            SERVICE_NAME: "<service-name>",
            HOST_NAME: "<host-name>"
        })
        # 使用GRPC協議上報
        span_processor = BatchSpanProcessor(OTLPSpanGrpcExporter(
            endpoint="<endpoint>",
            headers=("Authentication=<token>")
        ))
        # 使用HTTP協議上報
        # span_processor = BatchSpanProcessor(OTLPSpanHttpExporter(
        #     endpoint="<endpoint>",
        # ))
        trace_provider = TracerProvider(resource=resource, active_span_processor=span_processor)
        trace.set_tracer_provider(trace_provider)
    
    
    if __name__ == '__main__':
        init_opentelemetry()
        outer_method()
        baggage_and_attribute_usage()
  5. 運行程式。

    python manual.py

方法三:在自動埋點基礎上手動埋點

如果您需要在使用OpenTelemetry獲得自動埋點能力的同時,添加自訂業務埋點,請在方法一的基礎上完成以下操作。

  1. 下載包。

    pip install django
    pip install requests
    pip install opentelemetry-sdk
    pip install opentelemetry-instrumentation-django
    pip install opentelemetry-exporter-otlp 
  2. 建立AutoAndManualDemo專案並建立HelloWorld應用。

    1. 建立AutoAndManualDemo專案。

      django-admin startproject AutoAndManualDemo
    2. 在專案中建立HelloWorld應用。

      cd AutoAndManualDemo
      
      # 在專案中建立helloworld app
      python manage.py startapp helloworld
  3. 修改helloworld/views.py檔案代碼。

    擷取tracer並手動建立span,同時設定span名稱。

    from django.http import HttpResponse
    from opentelemetry import trace
    from datetime import datetime
    
    
    # Create your views here.
    def hello_world_view(request):
        tracer = trace.get_tracer(__name__)
    
        with tracer.start_as_current_span("hello_world_span") as hello_world_span:
            result = "Hello World! Current Time =" + str(get_time())
            return HttpResponse(result)
    
    
    def get_time():
        now = datetime.now()
        tracer = trace.get_tracer(__name__)
        # 建立新的span
        with tracer.start_as_current_span("time_span") as time_span:
            return now.strftime("%H:%M:%S")
  4. 修改urls.py檔案。

    1. 建立helloworld/urls.py檔案,在urls.py中添加以下代碼。

      from django.urls import path
      
      from . import views
      
      urlpatterns = [
          path('', views.hello_world_view, name='helloworld')
      ]
    2. 修改AutoAndManualDemo/AutoAndManualDemo/urls.py檔案,添加helloworld的URL。

      from django.contrib import admin
      from django.urls import path, include
      
      urlpatterns = [
       path('admin/', admin.site.urls),
       path('helloworld/', include('helloworld.urls')),
      ]
  5. 修改manage.py檔案代碼,在應用初始化的代碼中添加以下內容。

    from opentelemetry import trace
    from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter  # 通過gRPC接入
    # from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter # 通過HTTP接入
    from opentelemetry.sdk.resources import SERVICE_NAME, Resource, HOST_NAME
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
    
    
    
    resource = Resource(attributes={
            SERVICE_NAME: "<service-name>",
            HOST_NAME: "<host-name>"
    })
    trace.set_tracer_provider(TracerProvider(resource=resource))
    trace.get_tracer_provider().add_span_processor(
        BatchSpanProcessor(OTLPSpanExporter(
            endpoint="<endpoint>",
            headers="Authentication=<token>" # 通過gRPC接入時需要headers參數,通過HTTP接入時不需要此參數
    )))  # 通過 OTLPSpanExporter 上報Trace
    trace.get_tracer_provider().add_span_processor(BatchSpanProcessor(ConsoleSpanExporter()))  # 在控制台輸出Trace

    請替換以下內容,並根據接入方式(gRPC或者HTTP)修改代碼:

    • <service-name>: 需要上報的服務名

    • <host-name>:主機名稱

    • <endpoint>:通過HTTP/gRPC上報資料的存取點

    • <token>:通過gRPC上報資料的鑒權Token

  6. 運行專案。

    python manage.py runserver --noreload
    • --noreload防止manage.main方法執行兩次。

    • 如果運行時出現ImportError(symbol not found in flat namespace '_CFRelease'),請下載並安裝grpcio包。

      pip install grpcio
  7. 在瀏覽器中訪問127.0.0.1:8000/helloworld,鏈路資料便會上報至Managed Service for OpenTelemetry控制台。

查看監控資料

登入ARMS控制台後,在應用監控 > 應用列表頁面選擇目標應用,查看鏈路資料。

說明

語言列顯示image表徵圖的應用為接入應用監控的應用,顯示-表徵圖的應用為接入可觀測鏈路 OpenTelemetry 版的應用。