Initial data

This commit is contained in:
Artem Goncharov 2023-03-29 13:35:19 +02:00
parent 05f7b6cdad
commit ab29626444
502 changed files with 17976 additions and 0 deletions

50
README.rst Normal file
View File

@ -0,0 +1,50 @@
=========================================
OpenTelekomCloud SCS System Configuration
=========================================
This is the machinery that drives the configuration, testing, continuous
integration and deployment of services provided by the OpenTelekomCloud
project. It heavily copies OpenDev configuration approach with some extensions
and deviations.
Services are driven by Ansible playbooks and associated roles stored here. If
you are interested in the configuration of a particular service, starting at
``playbooks/service-<name>.yaml`` will show you how it is configured.
Most services are deployed via containers; many of them are built or customised
in this repository; see ``docker/``.
Bootstrap
=========
Bootstraping new installation is connected with usual
chicken-egg problem. Generally having system up and running it
is required to maintain certain secrets. But providing those
secrets requires infrastructure to be up and running. Addressing
this requres certain steps.
TLS Certificates
----------------
Most systems require valid TLS certificates. Initial bootstraping also requires valid TLS certificates. System that require those will typically support providing of initial certificates through inventory variables.
Vault
-----
Managing secrets securely is possible in few different ways.
Ansible vault is a good tool, but it is complex to manage unseal
and to implement rotations (of both vault password as well as
secrets inside the vault).
HashiCorp Vault is in that sense a much more flexible system that also provides support for infrastructure based authorization.
Deploying Vault on the other side is also requiring SSL certificates. Since during bootstraping it is most likely not possible to rely on the `playbooks/acme-certs.yaml` since it requires bootstrapped bridge host first it is required to provide initial valid certificates through host variables (`vault_tls_cert_content` and `vault_tls_key_content`). It makes sense not to commit those variables under the git and only provide them during the bootstraping phase.
Bootstraping Vault therefore requires following steps
1. Login to the host having access to all nodes which will host HashiCorp vault
2. Checkout this repository and ensure
`inventory/service/hosts.yaml` contain proper IP addresses as
well as those hosts are member of vault group as `inventory/service/groups.yaml`
3. execute `ansible-playbook playbooks/service-vault.yaml` playbook.

5
bindep.txt Normal file
View File

@ -0,0 +1,5 @@
libffi-dev [platform:dpkg]
libffi-devel [platform:rpm]
libssl-dev [platform:dpkg]
openssl-devel [platform:rpm]
graphviz [doc]

6
doc/requirements.txt Normal file
View File

@ -0,0 +1,6 @@
docutils>=0.11 # OSI-Approved Open Source, Public Domain
beautifulsoup4>=4.6.0 # MIT
reno>=3.1.0 # Apache-2.0
sphinx>=4.0.0 # BSD
zuul-sphinx>=0.1.1
graphviz

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

BIN
doc/source/_images/git.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 30 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

BIN
doc/source/_images/helm.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 18 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.6 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

BIN
doc/source/_images/loki.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 15 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 24 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

BIN
doc/source/_images/nova.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.1 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.8 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 KiB

View File

@ -0,0 +1,13 @@
digraph HelpCenter {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
user [label=Clients fixedsize=true fontsize=10 height=1.4 image="../_images/users.png" imagescale=true labelloc=b shape=none width=1]
web [label=WebServer fixedsize=true fontsize=10 height=1.4 image="../_images/nginx.png" imagescale=true labelloc=b shape=none width=1]
github [label="GitHub Projects" fixedsize=true fontsize=10 height=1.4 href="https://github.com/opentelekomcloud-docs" image="../_images/github.png" imagescale=true labelloc=b shape=none width=1]
zuul [label="Zuul CI/CD" fixedsize=true fontsize=10 height=1.4 href="https://docs.otc-service.com/system-config/zuul.html" image="../_images/zuulci.png" imagescale=true labelloc=b shape=none width=1]
swift [label="Swift Object Store" fixedsize=true fontsize=10 height=1.4 image="../_images/swift.png" imagescale=true labelloc=b shape=none width=1]
user -> web [label=Pull color=black fontsize=8]
web -> swift [label=Pull color=black fontsize=8]
github -> zuul [label=Push color=red fontsize=8]
zuul -> swift [label=Push color=red fontsize=8]
}

View File

@ -0,0 +1,76 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 2.50.0 (20211204.2007)
-->
<!-- Title: HelpCenter Pages: 1 -->
<svg width="328pt" height="228pt"
viewBox="0.00 0.00 328.00 228.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 224)">
<title>HelpCenter</title>
<!-- user -->
<g id="node1" class="node">
<title>user</title>
<image xlink:href="../_images/users.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="0" y="-205.5"/>
<text text-anchor="middle" x="36" y="-122" font-family="Times,serif" font-size="10.00">Clients</text>
</g>
<!-- web -->
<g id="node2" class="node">
<title>web</title>
<image xlink:href="../_images/nginx.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="124" y="-205.5"/>
<text text-anchor="middle" x="160" y="-122" font-family="Times,serif" font-size="10.00">WebServer</text>
</g>
<!-- user&#45;&gt;web -->
<g id="edge1" class="edge">
<title>user&#45;&gt;web</title>
<path fill="none" stroke="black" d="M72.29,-169.5C85.19,-169.5 99.96,-169.5 113.62,-169.5"/>
<polygon fill="black" stroke="black" points="113.79,-173 123.79,-169.5 113.79,-166 113.79,-173"/>
<text text-anchor="middle" x="98" y="-172.1" font-family="Times,serif" font-size="8.00">Pull</text>
</g>
<!-- swift -->
<g id="node5" class="node">
<title>swift</title>
<image xlink:href="../_images/swift.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="248" y="-145.5"/>
<text text-anchor="middle" x="284" y="-62" font-family="Times,serif" font-size="10.00">Swift Object Store</text>
</g>
<!-- web&#45;&gt;swift -->
<g id="edge2" class="edge">
<title>web&#45;&gt;swift</title>
<path fill="none" stroke="black" d="M196.29,-152.15C209.44,-145.68 224.54,-138.25 238.42,-131.43"/>
<polygon fill="black" stroke="black" points="240.36,-134.37 247.79,-126.82 237.27,-128.09 240.36,-134.37"/>
<text text-anchor="middle" x="222" y="-145.1" font-family="Times,serif" font-size="8.00">Pull</text>
</g>
<!-- github -->
<g id="node3" class="node">
<title>github</title>
<g id="a_node3"><a xlink:href="https://github.com/opentelekomcloud-docs" xlink:title="GitHub Projects">
<image xlink:href="../_images/github.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="0" y="-86.5"/>
<text text-anchor="middle" x="36" y="-3" font-family="Times,serif" font-size="10.00">GitHub Projects</text>
</a>
</g>
</g>
<!-- zuul -->
<g id="node4" class="node">
<title>zuul</title>
<g id="a_node4"><a xlink:href="https://docs.otc-service.com/system-config/zuul.html" xlink:title="Zuul CI/CD">
<image xlink:href="../_images/zuulci.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="124" y="-86.5"/>
<text text-anchor="middle" x="160" y="-3" font-family="Times,serif" font-size="10.00">Zuul CI/CD</text>
</a>
</g>
</g>
<!-- github&#45;&gt;zuul -->
<g id="edge3" class="edge">
<title>github&#45;&gt;zuul</title>
<path fill="none" stroke="red" d="M72.29,-50.5C85.19,-50.5 99.96,-50.5 113.62,-50.5"/>
<polygon fill="red" stroke="red" points="113.79,-54 123.79,-50.5 113.79,-47 113.79,-54"/>
<text text-anchor="middle" x="98" y="-53.1" font-family="Times,serif" font-size="8.00">Push</text>
</g>
<!-- zuul&#45;&gt;swift -->
<g id="edge4" class="edge">
<title>zuul&#45;&gt;swift</title>
<path fill="none" stroke="red" d="M196.29,-67.57C209.44,-73.93 224.54,-81.23 238.42,-87.94"/>
<polygon fill="red" stroke="red" points="237.27,-91.27 247.79,-92.47 240.31,-84.97 237.27,-91.27"/>
<text text-anchor="middle" x="222" y="-86.1" font-family="Times,serif" font-size="8.00">Push</text>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.7 KiB

View File

@ -0,0 +1,35 @@
digraph "Documentation Portal Security diagram" {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
subgraph cluster_web {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label="Web Server(s)"
web1 [label="WebServer 1"]
web2 [label="WebServer 2"]
web3 [label="WebServer XX"]
}
subgraph cluster_storage {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label=Storage
swift [label="Swift Object Store"]
web1 -> swift [label=HTTPS color=black dir=back fontsize=8]
web2 -> swift [label=HTTPS color=black dir=back fontsize=8]
web3 -> swift [label=HTTPS color=black dir=back fontsize=8]
}
subgraph cluster_zuul {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label="Zuul CI/CD"
zuul [label="Zuul CI/CD" href="https://docs.otc-service.com/system-config/zuul.html"]
zuul -> swift [label=HTTPS color=black fontsize=8]
}
subgraph cluster_git {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label="Git Hosting"
github1 [label="Project 1"]
github2 [label="Project 2"]
github3 [label="Project XX"]
github1 -> zuul [label=HTTPS color=black fontsize=8]
github2 -> zuul [label=HTTPS color=black fontsize=8]
github3 -> zuul [label=HTTPS color=black fontsize=8]
}
}

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 7.5 KiB

View File

@ -0,0 +1,13 @@
digraph HelpCenter {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
user [label=Clients fixedsize=true fontsize=10 height=1.4 image="../_images/users.png" imagescale=true labelloc=b shape=none width=1]
web [label=WebServer fixedsize=true fontsize=10 height=1.4 image="../_images/nginx.png" imagescale=true labelloc=b shape=none width=1]
github [label="GitHub Projects" fixedsize=true fontsize=10 height=1.4 href="https://github.com/opentelekomcloud-docs" image="../_images/github.png" imagescale=true labelloc=b shape=none width=1]
zuul [label="Zuul CI/CD" fixedsize=true fontsize=10 height=1.4 href="https://docs.otc-service.com/system-config/zuul.html" image="../_images/zuulci.png" imagescale=true labelloc=b shape=none width=1]
swift [label="Swift Object Store" fixedsize=true fontsize=10 height=1.4 image="../_images/swift.png" imagescale=true labelloc=b shape=none width=1]
user -> web [label=Pull color=black fontsize=8]
web -> swift [label=Pull color=black fontsize=8]
github -> zuul [label=Push color=red fontsize=8]
zuul -> swift [label=Push color=red fontsize=8]
}

View File

@ -0,0 +1,76 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 2.50.0 (20211204.2007)
-->
<!-- Title: HelpCenter Pages: 1 -->
<svg width="328pt" height="228pt"
viewBox="0.00 0.00 328.00 228.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 224)">
<title>HelpCenter</title>
<!-- user -->
<g id="node1" class="node">
<title>user</title>
<image xlink:href="../_images/users.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="0" y="-205.5"/>
<text text-anchor="middle" x="36" y="-122" font-family="Times,serif" font-size="10.00">Clients</text>
</g>
<!-- web -->
<g id="node2" class="node">
<title>web</title>
<image xlink:href="../_images/nginx.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="124" y="-205.5"/>
<text text-anchor="middle" x="160" y="-122" font-family="Times,serif" font-size="10.00">WebServer</text>
</g>
<!-- user&#45;&gt;web -->
<g id="edge1" class="edge">
<title>user&#45;&gt;web</title>
<path fill="none" stroke="black" d="M72.29,-169.5C85.19,-169.5 99.96,-169.5 113.62,-169.5"/>
<polygon fill="black" stroke="black" points="113.79,-173 123.79,-169.5 113.79,-166 113.79,-173"/>
<text text-anchor="middle" x="98" y="-172.1" font-family="Times,serif" font-size="8.00">Pull</text>
</g>
<!-- swift -->
<g id="node5" class="node">
<title>swift</title>
<image xlink:href="../_images/swift.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="248" y="-145.5"/>
<text text-anchor="middle" x="284" y="-62" font-family="Times,serif" font-size="10.00">Swift Object Store</text>
</g>
<!-- web&#45;&gt;swift -->
<g id="edge2" class="edge">
<title>web&#45;&gt;swift</title>
<path fill="none" stroke="black" d="M196.29,-152.15C209.44,-145.68 224.54,-138.25 238.42,-131.43"/>
<polygon fill="black" stroke="black" points="240.36,-134.37 247.79,-126.82 237.27,-128.09 240.36,-134.37"/>
<text text-anchor="middle" x="222" y="-145.1" font-family="Times,serif" font-size="8.00">Pull</text>
</g>
<!-- github -->
<g id="node3" class="node">
<title>github</title>
<g id="a_node3"><a xlink:href="https://github.com/opentelekomcloud-docs" xlink:title="GitHub Projects">
<image xlink:href="../_images/github.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="0" y="-86.5"/>
<text text-anchor="middle" x="36" y="-3" font-family="Times,serif" font-size="10.00">GitHub Projects</text>
</a>
</g>
</g>
<!-- zuul -->
<g id="node4" class="node">
<title>zuul</title>
<g id="a_node4"><a xlink:href="https://docs.otc-service.com/system-config/zuul.html" xlink:title="Zuul CI/CD">
<image xlink:href="../_images/zuulci.png" width="72px" height="72px" preserveAspectRatio="xMinYMin meet" x="124" y="-86.5"/>
<text text-anchor="middle" x="160" y="-3" font-family="Times,serif" font-size="10.00">Zuul CI/CD</text>
</a>
</g>
</g>
<!-- github&#45;&gt;zuul -->
<g id="edge3" class="edge">
<title>github&#45;&gt;zuul</title>
<path fill="none" stroke="red" d="M72.29,-50.5C85.19,-50.5 99.96,-50.5 113.62,-50.5"/>
<polygon fill="red" stroke="red" points="113.79,-54 123.79,-50.5 113.79,-47 113.79,-54"/>
<text text-anchor="middle" x="98" y="-53.1" font-family="Times,serif" font-size="8.00">Push</text>
</g>
<!-- zuul&#45;&gt;swift -->
<g id="edge4" class="edge">
<title>zuul&#45;&gt;swift</title>
<path fill="none" stroke="red" d="M196.29,-67.57C209.44,-73.93 224.54,-81.23 238.42,-87.94"/>
<polygon fill="red" stroke="red" points="237.27,-91.27 247.79,-92.47 240.31,-84.97 237.27,-91.27"/>
<text text-anchor="middle" x="222" y="-86.1" font-family="Times,serif" font-size="8.00">Push</text>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.7 KiB

View File

@ -0,0 +1,34 @@
digraph "HelpCenter Security diagram" {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
subgraph cluster_web {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label="Web Server(s)"
web1 [label="WebServer 1"]
web2 [label="WebServer 2"]
web3 [label="WebServer XX"]
}
subgraph cluster_storage {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label=Storage
swift [label="Swift Object Store"]
web1 -> swift [label=HTTPS color=black dir=back fontsize=8]
web2 -> swift [label=HTTPS color=black dir=back fontsize=8]
web3 -> swift [label=HTTPS color=black dir=back fontsize=8]
}
subgraph cluster_zuul {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label="Zuul CI/CD"
zuul [label="Zuul CI/CD"]
zuul -> swift [label=HTTPS color=black fontsize=8]
}
subgraph cluster_git {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
github1 [label="Project 1"]
github2 [label="Project 2"]
github3 [label="Project XX"]
github1 -> zuul [label=HTTPS color=black fontsize=8]
github2 -> zuul [label=HTTPS color=black fontsize=8]
github3 -> zuul [label=HTTPS color=black fontsize=8]
}
}

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 7.2 KiB

View File

@ -0,0 +1,41 @@
digraph "Reverse Proxy" {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
user [label=Clients fixedsize=true fontsize=10 height=1.4 image="../_images/users.png" imagescale=true labelloc=b shape=none width=1]
lb [label="Load Balancer" imagescale=true shape=box tooltip="Load Balancer in OTC"]
gw [label="Network Gateway" imagescale=true shape=box tooltip="Network Gateway in vCloud"]
user -> lb
user -> gw
lb -> proxy1
lb -> proxy2
gw -> web3
subgraph cluster_proxy {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label="Reverse Proxy"
proxy1 [label=proxy1 fixedsize=true fontsize=10 height=1.4 image="../_images/haproxy.png" imagescale=true labelloc=b shape=none tooltip="proxy1.eco.tsi-dev.otc-service.com" width=1]
proxy2 [label=proxy2 fixedsize=true fontsize=10 height=1.4 image="../_images/haproxy.png" imagescale=true labelloc=b shape=none tooltip="proxy2.eco.tsi-dev.otc-service.com" width=1]
web3 [label=web3 fixedsize=true fontsize=10 height=1.4 image="../_images/haproxy.png" imagescale=true labelloc=b shape=none tooltip="web3.eco.tsi-dev.otc-service.com" width=1]
}
proxy2 -> alerta [ltail=cluster_proxy]
proxy2 -> dashboard [ltail=cluster_proxy]
proxy2 -> "dashboard-eco" [ltail=cluster_proxy]
proxy2 -> docs [ltail=cluster_proxy]
proxy2 -> "graphite-apimon" [ltail=cluster_proxy]
proxy2 -> "graphite-ca" [ltail=cluster_proxy]
proxy2 -> influx [ltail=cluster_proxy]
proxy2 -> matrix [ltail=cluster_proxy]
proxy2 -> vault [ltail=cluster_proxy]
subgraph cluster_apps {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
label=Applications
alerta
dashboard
"dashboard-eco"
docs
"graphite-apimon"
"graphite-ca"
influx
matrix
vault
}
}

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 10 KiB

33
doc/source/_svg/zuul Normal file
View File

@ -0,0 +1,33 @@
digraph "Zuul CI/CD" {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
user [label=Clients fixedsize=true fontsize=10 height=1.4 image="../_images/users.png" imagescale=true labelloc=b shape=none width=1]
git [label="Git Provider" fixedsize=true fontsize=10 height=1.4 image="../_images/git.png" imagescale=true labelloc=b shape=none width=1]
subgraph cluster_zuul {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
node [fontsize=8]
label="Zuul CI/CD"
"zuul-web" [label="Zuul Web"]
"zuul-merger" [label="Zuul Merger"]
"zuul-executor" [label="Zuul Executor"]
"zuul-scheduler" [label="Zuul Scheduler"]
"nodepool-launcher" [label="Nodepool Launcher"]
"nodepool-builder" [label="Nodepool Builder"]
}
zookeeper [label=Zookeeper fixedsize=true fontsize=10 height=1.4 image="../_images/zookeeper.png" imagescale=true labelloc=b shape=none width=1]
"zuul-web" -> zookeeper
"zuul-merger" -> zookeeper
"zuul-executor" -> zookeeper
"zuul-scheduler" -> zookeeper
"nodepool-launcher" -> zookeeper
"nodepool-builder" -> zookeeper
db [label="SQL Database" fixedsize=true fontsize=10 height=1.4 image="../_images/postgresql.png" imagescale=true labelloc=b shape=none width=1]
cloud [label="Clouds resources" fixedsize=true fontsize=10 height=1.4 image="../_images/openstack.png" imagescale=true labelloc=b shape=none width=1]
user -> "zuul-web"
"zuul-merger" -> git
"zuul-executor" -> git
"zuul-web" -> db
"nodepool-launcher" -> cloud
"nodepool-builder" -> cloud
"zuul-executor" -> cloud
}

161
doc/source/_svg/zuul.svg Normal file

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 8.6 KiB

38
doc/source/_svg/zuul_dpl Normal file
View File

@ -0,0 +1,38 @@
digraph "Zuul CI/CD Deployment Design" {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
vault [label=Vault fixedsize=true fontsize=10 height=1.4 image="../_images/vault.png" imagescale=true labelloc=b shape=none width=1]
"zuul-web" -> vault [label=TLS color=blue fontsize=8]
"zuul-merger" -> vault [label=TLS color=blue fontsize=8]
"zuul-executor" -> vault [label=TLS color=blue fontsize=8]
"zuul-scheduler" -> vault [label=TLS color=blue fontsize=8]
"nodepool-launcher" -> vault [label=TLS color=blue fontsize=8]
"nodepool-builder" -> vault [label=TLS color=blue fontsize=8]
zookeeper -> vault [label=TLS color=blue fontsize=8]
"zuul-web" -> zookeeper [label=TLS color=red fontsize=8]
"zuul-merger" -> zookeeper [label=TLS color=red fontsize=8]
"zuul-executor" -> zookeeper [label=TLS color=red fontsize=8]
"zuul-scheduler" -> zookeeper [label=TLS color=red fontsize=8]
"nodepool-launcher" -> zookeeper [label=TLS color=red fontsize=8]
"nodepool-builder" -> zookeeper [label=TLS color=red fontsize=8]
subgraph cluster_k8 {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
node [fontsize=8]
label="Kubernetes Cluster"
subgraph cluster_zuul {
node [fontsize=8]
label="Zuul Namespace"
"zuul-web" [label="Zuul Web"]
"zuul-merger" [label="Zuul Merger"]
"zuul-executor" [label="Zuul Executor"]
"zuul-scheduler" [label="Zuul Scheduler"]
"nodepool-launcher" [label="Nodepool Launcher"]
"nodepool-builder" [label="Nodepool Builder"]
}
subgraph cluster_zk {
node [fontsize=8]
label="Zuul Namespace"
zookeeper [label=Zookeeper fixedsize=true fontsize=10 height=1.4 image="../_images/zookeeper.png" imagescale=true labelloc=b shape=none width=1]
}
}
}

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 11 KiB

39
doc/source/_svg/zuul_sec Normal file
View File

@ -0,0 +1,39 @@
digraph "Zuul CI/CD Security Design" {
graph [bgcolor=transparent compound=true fontcolor="#2D3436" fontname="Sans-Serif" fontsize=10 rankdir=LR]
node [fixedsize=false]
git [label="Git Provider" fixedsize=true fontsize=10 height=1.4 image="../_images/git.png" imagescale=true labelloc=b shape=none width=1]
db [label="SQL Database" fixedsize=true fontsize=10 height=1.4 image="../_images/postgresql.png" imagescale=true labelloc=b shape=none width=1]
cloud [label="Clouds resources" fixedsize=true fontsize=10 height=1.4 image="../_images/openstack.png" imagescale=true labelloc=b shape=none width=1]
"zuul-web" -> zookeeper [label=TLS color=red fontsize=8]
"zuul-merger" -> zookeeper [label=TLS color=red fontsize=8]
"zuul-executor" -> zookeeper [label=TLS color=red fontsize=8]
"zuul-scheduler" -> zookeeper [label=TLS color=red fontsize=8]
"nodepool-launcher" -> zookeeper [label=TLS color=red fontsize=8]
"nodepool-builder" -> zookeeper [label=TLS color=red fontsize=8]
subgraph cluster_k8 {
graph [bgcolor="#E5F5FD" shape=box style=rounded]
node [fontsize=8]
label="Kubernetes Cluster"
subgraph cluster_zuul {
node [fontsize=8]
label="Zuul Namespace"
"zuul-web" [label="Zuul Web"]
"zuul-merger" [label="Zuul Merger"]
"zuul-executor" [label="Zuul Executor"]
"zuul-scheduler" [label="Zuul Scheduler"]
"nodepool-launcher" [label="Nodepool Launcher"]
"nodepool-builder" [label="Nodepool Builder"]
}
subgraph cluster_zk {
node [fontsize=8]
label="Zuul Namespace"
zookeeper [label=Zookeeper fixedsize=true fontsize=10 height=1.4 image="../_images/zookeeper.png" imagescale=true labelloc=b shape=none width=1]
}
}
"zuul-merger" -> git [label=SSH color=blue fontsize=8]
"zuul-executor" -> git [label=SSH color=blue fontsize=8]
"zuul-web" -> db [label=TLS fontsize=8]
"nodepool-launcher" -> cloud [label=HTTPS color=green fontsize=8]
"nodepool-builder" -> cloud [label=HTTPS color=green fontsize=8]
"zuul-executor" -> cloud [label=SSH color=blue fontsize=8]
}

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 11 KiB

66
doc/source/bridge.rst Normal file
View File

@ -0,0 +1,66 @@
:title: Bridge
.. _bridge:
Bridge
######
Bridge is a bastion host that is the starting point for ops operations in
OpenTelekomCloudEco. It is the server from which Ansible is run, and contains
decrypted secure information such as passwords. The bridge server contains all
of the ansible playbooks as well as the scripts to create new servers.
Sensitive information like passwords is stored encrypted in the private git and
are pulled by the bridge host on a cron basis.
At a Glance
===========
:Projects:
* https://ansible.com/
:Bugs:
:Resources:
Ansible Hosts
-------------
In OTC Eco, all host configuration is done via ansible playbooks.
Adding a node
-------------
In principle hosts in the inventory (``inventory/base/hosts.yaml``) contain
required variables so that playbooks are able to provision the infrastructure.
This is not yet implemented for all hosts/systems.
.. _running-ansible-on-nodes:
Running Ansible on Nodes
------------------------
Each service that has been migrated fully to Ansible has its own playbook in
:git_file:`playbooks` named ``service_{ service_name }.yaml``.
Because the playbooks are normally run by zuul, to run them manually, first run
the utility ``disable-ansible`` as root. That will touch the file
``/home/zuul/DISABLE-ANSIBLE``. We use the utility to avoid mistyping the
lockfile name. Then make sure no jobs are currently executing ansible. Ensure
that ``/home/zuul/src/github.com/opentelekomcloud-infra/system-config`` is in
the appropriate state, then run:
.. code-block:: bash
cd /home/zuul/src/github.com/opentelekomcloud-infra/system-config
ansible-playbook --limit="$HOST:localhost" playbooks/service-$SERVICE.yaml
as root, where `$HOST` is the host you want to run puppet on.
The `:localhost` is important as some of the plays depend on performing a task
on the localhost before continuing to the host in question, and without it in
the limit section, the tasks for the host will have undefined values.
When done, don't forget to remove ``/home/zuul/DISABLE-ANSIBLE``
Disabling Ansible on Nodes
--------------------------
In the case of needing to disable the running of ansible on a node, it's a
simple matter of adding an entry to the ansible inventory "disabled" group.

Some files were not shown because too many files have changed in this diff Show More