diff --git a/doc/source/internal/apimon_training/notifications.rst b/doc/source/internal/apimon_training/notifications.rst index 4b01b70..f8e6dac 100644 --- a/doc/source/internal/apimon_training/notifications.rst +++ b/doc/source/internal/apimon_training/notifications.rst @@ -13,8 +13,12 @@ Every stream contains topics based on the service type (if represented by standalone Ansible playbook) and general apimon_endpoint_monitor topic which contains alerts of GET queries towards all services. + +.. image:: training_images/zulip_notifications.png + + If the error has been acknowledged on Alerta, the new notification message for -repeating error wont get posted again on Zulip. +repeating error won't get posted again on Zulip. Notifications contain further details which help to identify root cause faster and more effectively. @@ -59,4 +63,3 @@ Th EpMon notification consists of several fields: -.. image:: training_images/zulip_notifications.png diff --git a/doc/source/internal/apimon_training/training_images/apimon_data_flow.svg b/doc/source/internal/apimon_training/training_images/apimon_data_flow.svg index c1b1b7c..a57638b 100644 --- a/doc/source/internal/apimon_training/training_images/apimon_data_flow.svg +++ b/doc/source/internal/apimon_training/training_images/apimon_data_flow.svg @@ -1,4 +1,4 @@ -

Scheduler


running
8 parallel
threads
Scheduler...
Add next playbook to the queue when thread is free
Add next playbook...
Influx DB


Influx DB...
Fill in playbooks to the queue of threads
Fill in playboo...
Execute ansible playbooks
Execute ansible...
Remove completed playbook from the thread
Remove complete...

Telegraf


Collects the
metrics
Telegraf...

Ansible


Ansible
Ansible...
Send metrics to Influx DB
Send metrics to...
Apimon
Scenario Developers
Apimon...
If playbook/thread failed raise alert
If playbook/thread...
Store the job logs
to object storage
Store the job...
Visualize data
Visualize...
Create Alerts based on Thresholds
Create Alerts...
O/M
O/M

Github


apimon tests
repository
Github...
Pull
repository

Pull...
Management
Manage...
Endless loop
Endless loop

Grafana


Dashboard
Grafana...

Alerta


Dashboard
Alerta...
Send notifications to Zulip
Send notifica...

Zulip


Alerts
Alerts-Hybrid
Alerts-Preprod
Zulip...
Swift

Swift
Text is not SVG - cannot display
\ No newline at end of file +

Scheduler


running
8 parallel
threads
Scheduler...
Add next playbook from the queue when thread is free
Add next playbook...
Graphite TSDB



Graphite TSDB...
Fill in playbooks to the queue of threads
Fill in playboo...
Execute ansible playbooks
Execute ansible...
Remove completed playbook from the thread
Remove complete...

Statsd


Collects the
metrics
Statsd...

Executor


Ansible
Executor...
Send metrics to graphite
Send metrics to...
Service Squad
Servic...
If playbook
failed raise alert
If playbook...
Store the
job logs to
object storage
Store the...
Data
Sources
Data...
Create Alerts based on Thresholds
Create Alerts...
O/M
O/M

Github


apimon tests
repository
Github...
Pull
repository

Pull...
Management
Manage...
Endless loop
Endless loop

Grafana


Dashboard
Grafana...

Alerta


Dashboard
Alerta...
Send notifications to Zulip
Send notificati...

Zulip


Alerts
Alerts-Hybrid
Alerts-Preprod
Zulip...
Swift

Swift
Postgresql RDB



Postgresql RDB...
Test results
Test resul...
Metrics
Metrics

Scheduler


running
8 parallel
threads
Scheduler...
1
1
2
2
3
3
4
4
6
6
5
5
7
7
8
8
9
9
10
10
11
11
Text is not SVG - cannot display
\ No newline at end of file diff --git a/doc/source/internal/apimon_training/workflow.rst b/doc/source/internal/apimon_training/workflow.rst index 1078a6c..3fa1c6d 100644 --- a/doc/source/internal/apimon_training/workflow.rst +++ b/doc/source/internal/apimon_training/workflow.rst @@ -7,3 +7,22 @@ ApiMon Flow Process .. image:: training_images/apimon_data_flow.svg :target: training_images/apimon_data_flow.svg :alt: apimon_data_flow + + +#. Service squad adds test scenario to github repository. +#. Scheduler fetches test scenarios from Github and add them to queue. +#. Executor plays Ansible test scenario playbooks. Up to 8 parallel threads are enabled +#. Test scenario which has finished is being removed from the thread and next + playbook in the queue is added to the free thread. The previous playbook is + added to the queue on the last position. +#. Test scenario statistics are stored in the Postgresql database. +#. Metrics from HTTP requests are collected by Statsd. +#. Collected metrics are stored in time-series database Graphite. +#. Grafana uses metrics and statistics databases as the data sources for the + dashboards. The dashboard with various panels show the real-time status of + the platform. Grafana supports also historical views and trends. +#. Breached thresholds as well as failed test scenarios result in generated + alerts on Alerta. +#. Notifications containing alert details are sent to Zulip +#. Every test scenario stores it's job output log into Swift object storage for further analysis and investigation. +