From abd290d92d8ce0468c5def015054c78e7280431d Mon Sep 17 00:00:00 2001 From: Kyriakos Akriotis Date: Thu, 23 Nov 2023 14:54:18 +0100 Subject: [PATCH] - added categories structure and summaries --- doc/source/best-practices/computing/index.rst | 10 +- .../best-practices/data-analysis/index.rst | 14 - .../best-practices/data-analytics/index.rst | 14 + doc/source/best-practices/index.rst | 13 +- doc/source/best-practices/network/index.rst | 9 +- .../best-practices/security}/cce_vault.rst | 0 doc/source/best-practices/security/index.rst | 12 +- doc/source/best-practices/storage/index.rst | 10 +- doc/source/industry/automotive/index.rst | 12 + doc/source/industry/education/index.rst | 12 + doc/source/industry/finance/index.rst | 11 + doc/source/industry/government/index.rst | 10 + doc/source/industry/healthcare/index.rst | 11 + doc/source/industry/index.rst | 20 ++ doc/source/industry/media/index.rst | 10 + doc/source/industry/retail/index.rst | 11 + doc/source/use-cases/analytics/index.rst | 13 + doc/source/use-cases/hybrid/index.rst | 11 + doc/source/use-cases/index.rst | 18 + doc/source/use-cases/migration/index.rst | 13 + .../use-cases/migration/rds_migration.rst | 328 ++++++++++++++++++ doc/source/use-cases/networking/index.rst | 12 + doc/source/use-cases/security/index.rst | 11 + 23 files changed, 550 insertions(+), 35 deletions(-) delete mode 100644 doc/source/best-practices/data-analysis/index.rst create mode 100644 doc/source/best-practices/data-analytics/index.rst rename doc/{best-practice/source => source/best-practices/security}/cce_vault.rst (100%) create mode 100644 doc/source/industry/automotive/index.rst create mode 100644 doc/source/industry/education/index.rst create mode 100644 doc/source/industry/finance/index.rst create mode 100644 doc/source/industry/government/index.rst create mode 100644 doc/source/industry/healthcare/index.rst create mode 100644 doc/source/industry/index.rst create mode 100644 doc/source/industry/media/index.rst create mode 100644 doc/source/industry/retail/index.rst create mode 100644 doc/source/use-cases/analytics/index.rst create mode 100644 doc/source/use-cases/hybrid/index.rst create mode 100644 doc/source/use-cases/index.rst create mode 100644 doc/source/use-cases/migration/index.rst create mode 100644 doc/source/use-cases/migration/rds_migration.rst create mode 100644 doc/source/use-cases/networking/index.rst create mode 100644 doc/source/use-cases/security/index.rst diff --git a/doc/source/best-practices/computing/index.rst b/doc/source/best-practices/computing/index.rst index 166fdd6..1503e6c 100644 --- a/doc/source/best-practices/computing/index.rst +++ b/doc/source/best-practices/computing/index.rst @@ -1,10 +1,12 @@ Computing ========= -Discover a wealth of expert insights and strategies to optimize your cloud computing experience. -From efficient resource allocation to performance tuning, our curated collection empowers you to harness the full potential of the Open Telekom Cloud. -Stay ahead of the curve with industry-leading techniques, ensuring seamless scalability, security, and cost-effectiveness for your projects. -Unlock the true power of Compute services of Open Telekom Cloud with our comprehensive resources. +The Computing section offers essential insights for optimizing computing resources. Discover guidelines for selecting +appropriate instance types, managing virtual machines efficiently, and leveraging auto-scaling capabilities for dynamic +workloads. Learn best practices for designing resilient and high-performance computing architectures, ensuring optimal +utilization of resources while maintaining cost-effectiveness. This section serves as a comprehensive guide for architects +and developers to fine-tune their computing strategies, enhancing the overall efficiency and reliability of applications +in the Open Telekom Cloud environment. .. toctree:: :maxdepth: 1 diff --git a/doc/source/best-practices/data-analysis/index.rst b/doc/source/best-practices/data-analysis/index.rst deleted file mode 100644 index 21cf5c4..0000000 --- a/doc/source/best-practices/data-analysis/index.rst +++ /dev/null @@ -1,14 +0,0 @@ -Data Analysis -============= - -Dive into a world of data-driven excellence with our expert guidance. -Discover cutting-edge methodologies and techniques to extract valuable insights from your data, leveraging the power of the Open Telekom Cloud. -Optimize data processing, visualization, and storage to unlock the full potential of your analytics projects. -Stay at the forefront of the data analytics landscape with our curated collection of resources, ensuring your organization makes informed decisions and stays competitive. -Propel your data analytics journey to new heights with our comprehensive and invaluable insights. - - -.. toctree:: - :maxdepth: 1 - - diff --git a/doc/source/best-practices/data-analytics/index.rst b/doc/source/best-practices/data-analytics/index.rst new file mode 100644 index 0000000..51031a5 --- /dev/null +++ b/doc/source/best-practices/data-analytics/index.rst @@ -0,0 +1,14 @@ +Data Analytics +============== + +This section provides strategic guidance for optimizing data processing workflows. Explore recommendations for selecting +and configuring data analytics services, ensuring efficient and scalable processing of large datasets. Learn about best +practices for data storage, retrieval, and integration to enhance overall analytics performance. This section is a +valuable resource for architects and data professionals, offering insights into designing robust and cost-effective data +analytics architectures within the Open Telekom Cloud, fostering informed decision-making and actionable insights. + + +.. toctree:: + :maxdepth: 1 + + diff --git a/doc/source/best-practices/index.rst b/doc/source/best-practices/index.rst index 45fae56..17aa768 100644 --- a/doc/source/best-practices/index.rst +++ b/doc/source/best-practices/index.rst @@ -1,11 +1,14 @@ Best Practices ============== -Welcome to our Open Telekom Cloud Best Practices. Explore a curated collection of -cutting-edge techniques and methodologies to optimize your cloud journey in Open Telekom Cloud. -Streamline your organization's migration process and enhance scalability, security, -and cost-efficiency with our expert guidance. Unleash the full potential of leading Open Telekom Cloud, -while avoiding common pitfalls, through our comprehensive resources. +Welcome to the Best Practices section of Open Telekom Cloud's Architecture Center. +Here we provides crucial guidelines for optimizing cloud-based solutions. +Best Practices emphasize architectural principles that enhance reliability, scalability, and security. +Explore our recommended strategies for resource management, such as efficient utilization of compute +and storage resources. Gain insights into designing for high availability and fault tolerance +to ensure robust system performance. This section serves as a valuable resource for architects and developers +to implement cloud solutions that align with industry best practices and maximize the benefits of the public cloud +infrastructure. .. toctree:: :maxdepth: 1 diff --git a/doc/source/best-practices/network/index.rst b/doc/source/best-practices/network/index.rst index cfa1a73..13ad2c9 100644 --- a/doc/source/best-practices/network/index.rst +++ b/doc/source/best-practices/network/index.rst @@ -1,9 +1,12 @@ Network ======= -Discover optimal strategies for designing, deploying, and managing high-performance networks on the Open Telekom Cloud. -Enhance security, scalability, and reliability to ensure seamless communication and data flow by following the tested and -validated recommendations from our experts. +Network Best Practices outline key strategies for optimizing network configurations. Explore guidelines for designing +resilient and high-performance network architectures, including considerations for security and scalability. +Learn about best practices for leveraging Virtual Private Clouds (VPCs), network segmentation, and load balancing to +enhance overall network efficiency. This section serves as a valuable resource for architects and network administrators, +providing insights into building robust and secure network infrastructures within the Open Telekom Cloud environment, +ensuring reliable and seamless connectivity for applications and services. .. toctree:: diff --git a/doc/best-practice/source/cce_vault.rst b/doc/source/best-practices/security/cce_vault.rst similarity index 100% rename from doc/best-practice/source/cce_vault.rst rename to doc/source/best-practices/security/cce_vault.rst diff --git a/doc/source/best-practices/security/index.rst b/doc/source/best-practices/security/index.rst index 3b06010..eef8f40 100644 --- a/doc/source/best-practices/security/index.rst +++ b/doc/source/best-practices/security/index.rst @@ -1,12 +1,16 @@ Security ======== -Explore comprehensive resources to protect your data, applications, and systems from evolving cyber threats. -Stay compliant with the latest security standards and optimize your security posture on the Open Telekom Cloud. -From encryption to access control, discover invaluable insights to fortify your defenses and ensure a robust and resilient environment. - +The Security Best Practices offer comprehensive guidance on fortifying cloud environments. Explore recommendations for +implementing robust identity and access management, encryption protocols, and network security measures. Learn about +best practices for securing data at rest and in transit, as well as strategies for monitoring and responding to +security incidents. This section is a crucial resource for architects and cybersecurity professionals, providing +insights into designing and maintaining resilient security postures within the Open Telekom Cloud, ensuring the +confidentiality, integrity, and availability of sensitive information. .. toctree:: :maxdepth: 1 + cce_vault.rst + diff --git a/doc/source/best-practices/storage/index.rst b/doc/source/best-practices/storage/index.rst index 34d6d0c..2d3edc9 100644 --- a/doc/source/best-practices/storage/index.rst +++ b/doc/source/best-practices/storage/index.rst @@ -1,11 +1,11 @@ Storage ======= -Discover the most effective and efficient ways to manage your data in the cloud. -Explore expert insights and industry-leading techniques to optimize storage performance, cost, and scalability on the Open Telekom Cloud. -From data migration to backup and recovery, access a curated collection of resources to meet your storage needs. -Stay ahead of the curve with our comprehensive guidance and unlock the full potential of Open Telekom Cloud Storage offerings for your organization. - +Storage Best Practices offer key insights into optimizing storage solutions. Explore guidelines for selecting +appropriate storage types, managing data lifecycle, and implementing redundancy for enhanced durability. Learn about +best practices for achieving optimal performance and cost-effectiveness in storage configurations. This section is a +valuable resource for architects and storage administrators, providing essential strategies to design resilient and +scalable storage architectures within the Open Telekom Cloud environment, ensuring efficient data management and retrieval. .. toctree:: :maxdepth: 1 diff --git a/doc/source/industry/automotive/index.rst b/doc/source/industry/automotive/index.rst new file mode 100644 index 0000000..6b1311a --- /dev/null +++ b/doc/source/industry/automotive/index.rst @@ -0,0 +1,12 @@ +Automotive +========== + +The Automotive section showcases tailored solutions for the automotive sector. Explore practical examples demonstrating +how the platform supports the industry's unique requirements, from connected car technologies to manufacturing +processes. This section provides architects with insights into designing scalable and secure cloud architectures to +enhance innovation and efficiency in the automotive domain. Discover recommended best practices, empowering users to +leverage Open Telekom Cloud for optimized performance and transformative capabilities within the automotive industry's +dynamic landscape. + +.. toctree:: + :maxdepth: 1 diff --git a/doc/source/industry/education/index.rst b/doc/source/industry/education/index.rst new file mode 100644 index 0000000..334678e --- /dev/null +++ b/doc/source/industry/education/index.rst @@ -0,0 +1,12 @@ +Education +========= + +The Education section highlights tailored cloud solutions for the education sector. Explore practical examples +showcasing how the platform supports e-learning applications, research initiatives, and administrative processes. +This section provides architects with insights into designing scalable and cost-effective cloud architectures to meet +the diverse needs of educational institutions. Discover recommended best practices, empowering users to leverage +Open Telekom Cloud for enhanced collaboration, resource efficiency, and innovation within the dynamic landscape of the +education industry. + +.. toctree:: + :maxdepth: 1 diff --git a/doc/source/industry/finance/index.rst b/doc/source/industry/finance/index.rst new file mode 100644 index 0000000..a3cbef3 --- /dev/null +++ b/doc/source/industry/finance/index.rst @@ -0,0 +1,11 @@ +Finance +======= + +The Finance section offers targeted cloud solutions for the financial sector. Explore practical examples demonstrating +how the platform supports secure and compliant financial applications, from digital banking to risk management. This +section provides architects with insights into designing robust, scalable, and regulatory-compliant cloud architectures +tailored to financial industry requirements. Discover recommended best practices, empowering users to leverage +Open Telekom Cloud for optimized performance, security, and innovation within the dynamic landscape of the financial sector. + +.. toctree:: + :maxdepth: 1 diff --git a/doc/source/industry/government/index.rst b/doc/source/industry/government/index.rst new file mode 100644 index 0000000..6614ebb --- /dev/null +++ b/doc/source/industry/government/index.rst @@ -0,0 +1,10 @@ +Government +========== + +The Government section focuses on tailored cloud solutions for the public sector. Explore practical examples showcasing +how the platform supports secure and compliant government applications, from citizen services to data management. This +section provides architects with insights into designing resilient, scalable, and regulatory-compliant cloud +architectures tailored to governmental requirements. + +.. toctree:: + :maxdepth: 1 diff --git a/doc/source/industry/healthcare/index.rst b/doc/source/industry/healthcare/index.rst new file mode 100644 index 0000000..bfc62a9 --- /dev/null +++ b/doc/source/industry/healthcare/index.rst @@ -0,0 +1,11 @@ +Healthcare +========== + +The Healthcare section showcases specialized cloud solutions for the healthcare sector. Explore practical examples +illustrating how the platform supports secure and compliant healthcare applications, from electronic health records to +medical research. This section provides architects with insights into designing robust, scalable, and +regulatory-compliant cloud architectures tailored to the unique needs of the healthcare industry. + +.. toctree:: + :maxdepth: 1 + diff --git a/doc/source/industry/index.rst b/doc/source/industry/index.rst new file mode 100644 index 0000000..12907c1 --- /dev/null +++ b/doc/source/industry/index.rst @@ -0,0 +1,20 @@ +By Industry +========== + +The "Use Cases By Industry" section provides tailored solutions for diverse sectors, including healthcare, finance, +automotive, media, education, retail and government. Explore industry-specific scenarios showcasing the platform's adaptability +to unique requirements and challenges. This section offers architects valuable insights into designing cloud solutions +that align with the specific needs and compliance standards of their respective industries. Discover best practices and +recommended architectures, empowering users to leverage Open Telekom Cloud effectively for industry-specific use cases, +ensuring optimal performance and regulatory adherence. + +.. toctree:: + :maxdepth: 1 + + government/index.rst + media/index.rst + automotive/index.rst + education/index.rst + finance/index.rst + healthcare/index.rst + retail/index.rst \ No newline at end of file diff --git a/doc/source/industry/media/index.rst b/doc/source/industry/media/index.rst new file mode 100644 index 0000000..fe1639f --- /dev/null +++ b/doc/source/industry/media/index.rst @@ -0,0 +1,10 @@ +Media +===== + +The Media section highlights tailored cloud solutions for the media and entertainment sector. Explore practical examples +showcasing how the platform supports content delivery, media processing, and collaborative production workflows. +This section provides architects with insights into designing scalable and cost-effective cloud architectures to +meet the dynamic demands of the media industry. + +.. toctree:: + :maxdepth: 1 diff --git a/doc/source/industry/retail/index.rst b/doc/source/industry/retail/index.rst new file mode 100644 index 0000000..b296be7 --- /dev/null +++ b/doc/source/industry/retail/index.rst @@ -0,0 +1,11 @@ +Retail +====== + +The Retail section offers tailored cloud solutions for the retail sector. Explore practical examples showcasing how the +platform supports e-commerce applications, supply chain management, and customer engagement solutions. This section +provides architects with insights into designing scalable and customer-focused cloud architectures to meet the evolving +needs of the retail industry. + +.. toctree:: + :maxdepth: 1 + diff --git a/doc/source/use-cases/analytics/index.rst b/doc/source/use-cases/analytics/index.rst new file mode 100644 index 0000000..368cab4 --- /dev/null +++ b/doc/source/use-cases/analytics/index.rst @@ -0,0 +1,13 @@ +Data Analytics & AI +=================== + +This section showcases practical applications of cloud analytics solutions. Explore real-world scenarios demonstrating +the platform's capabilities in processing and deriving insights from large datasets. From business intelligence to +predictive analytics, this section provides tailored examples and best practices for architects and data professionals. +Discover how Open Telekom Cloud's analytics services can empower organizations to extract meaningful information, +optimize decision-making processes, and drive innovation through advanced data analysis. + +.. toctree:: + :maxdepth: 1 + + diff --git a/doc/source/use-cases/hybrid/index.rst b/doc/source/use-cases/hybrid/index.rst new file mode 100644 index 0000000..19a39f6 --- /dev/null +++ b/doc/source/use-cases/hybrid/index.rst @@ -0,0 +1,11 @@ +Hybrid +====== + +Hybrid section highlights the versatility of hybrid cloud solutions. Explore practical examples showcasing seamless +integration between on-premises infrastructure and the Open Telekom Cloud platform. This section provides insights for +architects on optimizing hybrid architectures, ensuring flexibility, scalability, and streamlined management. +Learn how organizations can leverage the hybrid model to balance performance and compliance requirements, fostering +a dynamic and efficient IT environment tailored to specific business needs. + +.. toctree:: + :maxdepth: 1 diff --git a/doc/source/use-cases/index.rst b/doc/source/use-cases/index.rst new file mode 100644 index 0000000..0ede5e9 --- /dev/null +++ b/doc/source/use-cases/index.rst @@ -0,0 +1,18 @@ +By Use Case +========== + +Welcome to the Use Cases section of Open Telekom Cloud Architecture Center. Here you can find tailored solutions and +practical implementations for a range of scenarios. Explore real-world examples demonstrating the versatility of our +cloud services, spanning industries such as e-commerce, healthcare, and finance. This section serves as a comprehensive +resource for architects, offering insights into how to adapt and optimize cloud solutions for specific business needs. +Discover recommended architectures and best practices for diverse use cases, empowering users to leverage the full +potential of Open Telekom Cloud platform in addressing their unique requirements and challenges. + +.. toctree:: + :maxdepth: 1 + + hybrid/index.rst + security/index.rst + networking/index.rst + migration/index.rst + analytics/index.rst \ No newline at end of file diff --git a/doc/source/use-cases/migration/index.rst b/doc/source/use-cases/migration/index.rst new file mode 100644 index 0000000..f749160 --- /dev/null +++ b/doc/source/use-cases/migration/index.rst @@ -0,0 +1,13 @@ +Migration +========= + +Migration section offers practical guidance for seamless migration strategies. Explore real-world scenarios +demonstrating how organizations can efficiently move their applications and data to the Open Telekom Cloud. +This section provides architects with best practices, ensuring a smooth transition while minimizing downtime and +optimizing resource utilization. Discover insights into planning, executing, and validating successful migrations, +empowering users to harness the benefits of the cloud environment effectively. + +.. toctree:: + :maxdepth: 1 + + cce_vault.rst diff --git a/doc/source/use-cases/migration/rds_migration.rst b/doc/source/use-cases/migration/rds_migration.rst new file mode 100644 index 0000000..a367825 --- /dev/null +++ b/doc/source/use-cases/migration/rds_migration.rst @@ -0,0 +1,328 @@ +=========================================================================== +Migrate a MySQL/MariaDB database to Open Telekom Cloud RDS with Apache NiFi +=========================================================================== + +`Apache NiFi `__ (**nye-fye**) is a project from the Apache Software Foundation aiming +to automate the flow of data between systems in form of workflows. It supports powerful and scalable directed graphs +of data routing, transformation, and system mediation logic. We are going to utilize these features in order to +consolidate all the manual, mundane and error prone steps of an on-premises MySQL/MariaDb database migration +to Open Telekom Cloud RDS in a form of a highly scalable workflow. + +.. note:: + Historically, NiFi is based on the "*NiagaraFiles*", a system that was developed by the US National Security Agency (NSA). + It was open-sourced as a part of NSA's technology transfer program in 2014. + +Overview +======== + +With zero cost in 3rd party components and in less than 15 minutes we are going to transform a highly error prone and +demanding use-case, as the migration of an MySQL or MariaDB to the cloud, to a fully automated, repeatable and scalable procedure. + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/placeholder-image.jpg + +.. image:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20221004-etc.png + :scale: 75 + :target: https://docs.otc.t-systems.com/en-us/usermanual/rds/en-us_topic_dashboard.html + :alt: Relational Database Service (RDS) +.. image:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20221004-etn.png + :scale: 75 + :target: https://docs.otc.t-systems.com/en-us/usermanual/rds/en-us_topic_dashboard.html +.. image:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20221004-f0m.png + :scale: 75 + :target: https://docs.otc.t-systems.com/en-us/usermanual/rds/en-us_topic_dashboard.html +.. image:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20221004-f0q.png + :scale: 75 + :target: https://docs.otc.t-systems.com/en-us/usermanual/rds/en-us_topic_dashboard.html +.. image:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20221004-f0u.png + :scale: 75 + :target: https://docs.otc.t-systems.com/en-us/usermanual/rds/en-us_topic_dashboard.html + +.. seealso:: + + `Github Repository `_ + + `Apache Nifi Workflow Template `_ + +Provision a MySQL instance in RDS +================================== + +If you don't have an RDS instance in place, let's create one in order to demonstrace this use-case. +Under Relational Database Service in Open Telekom Cloud Console, + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20221004-fw6.png + +choose *Create DB Instance* and go through the creation wizard: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220926-9b5.png + +1. Choose the basic details of your database engine. You need to stick for this use-case to MySQL engine v8.0. +Whether you create a single instance database or a replicated one with primary and standby instances is fairly +irrelevant in regards to our use-case. + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220926-9c5.png + +2. Create a new Security Group that will allow port 3306 in its inbound ports, and assign this Security Group +as Security Group of the ECS instances of your database (still in the database creation wizard) + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220926-9d4.png + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220926-9h7.png + +3. After the database is successfully created, enable SSL support: + +.. warning:: + It's not recommended transfering production data without having SSL enalbed + +Provision an Apache Nifi Server +================================ + +We are going to deploy the Apache NiFi server as a **docker container** using the following command +(replace first the required credentials with the ones of your choice): + +.. code-block:: shell + + docker run --name nifi \ + -p 8443:8443 \ + -d \ + -e SINGLE_USER_CREDENTIALS_USERNAME={{USERNAME}} \ + -e SINGLE_USER_CREDENTIALS_PASSWORD={{PASSWORD}} \ + apache/nifi:latest + +and then open your browser and navigate to the following URL address: + +.. code-block:: shell + + https://localhost:8443/nifi/ + +enter your credentials and you will land on an empty workflow canvas: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220810-lt4.png + +Create the migration workflow +============================= + +1. Add a **Processor** of type **GenerateFlowFile**, as the entry point of our workflow (as is instructed in the following picture): + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220810-lvz.png + +2. Add a **Processor** of type **ExecuteStreamCommand**, as the step that will dump and export our source database — and call it ExportMysqlDump: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220810-m0k.png + +and let’s configure the external command we want this component to execute: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220810-m2m.png + +go to **Properties** from the tab menu: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220810-m44.png + +As **Command Path** set : + +.. code-block:: shell + + /usr/bin/mysqldump + +and as **Command Arguments** fill in the mysql-client arguments, but separated by a semicolon +(replace the highlighted values with your own): + +.. code-block:: shell + + -u;root;-P;3306;-h;{{HOSTNAME_OR_CONTAINER_IP}};-p{{PASSWORD}}; + --databases;employees;--routines;--triggers;--single-transaction; + --order-by-primary;--gtid;--force + +Connect the two Processors by dragging a connector line from the first to the latter. +You should be able to observe now that a **Queue** component is injected between them: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220810-m8y.png + +We will see later how these Queues contribute to the workflow and how we can use them +to gain useful insights or debug our workflows. + +3. Open Telekom Cloud RDS for MySql will **not** permit SUPER privileges or the SET_USER_ID privilege to any user, +and this will lead to the following error when you will try to run the migration workflow for the first time: + +.. code-block:: shell + + ERROR 1227 (42000) at line 295: Access denied; + you need (at least one of) the SUPER or SET_USER_ID privilege(s) for this operation + +The error above may occur while executing CREATE VIEW, FUNCTION, PROCEDURE, TRIGGER OR EVENT with DEFINER statements +as part of importing a dump file or running a script. In order to preactively mitigate this situation, we are going to add +a second **Processor** of type **ExecuteStreamCommand**. This Processor (let’s call it ReplaceDefinersCommand) +will edit the dump file script and replace the DEFINER values with the appropriate user with admin permissions +who is going to perform the import or execute the script file. + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-ni2.png + +As **Command Path** set : + +.. code-block:: shell + + sed + +and as **Command Arguments** (*in one line*): + +.. code-block:: shell + + -e;"s/DEFINER[ ]*=[ ]*[^*]*\*/\*/"; + -e;"s/DEFINER[ ]*=.*FUNCTION/FUNCTION/"; + -e;"s/DEFINER[ ]*=.*PROCEDURE/PROCEDURE/"; + -e;"s/DEFINER[ ]*=.*TRIGGER/TRIGGER/"; + -e;"s/DEFINER[ ]*=.*EVENT/EVENT/" + +Connect the two ExecuteCommandStream Processors, by dragging a connector line from the first to the second. +You should be able to observe now that a second Queue component is added between them on the canvas. + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-ngs.png + +4. Add a third **Processor** of type **ExecuteStreamCommand** (same drill as with ExportMysqlDump). +This step will import the dump to our target database — call it ImportMysqlDump. Let’s configure it: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220810-mf6.png + +As **Command Path** set : + +.. code-block:: shell + + /usr/bin/mysql + +and as **Command Arguments** (*in one line*): + +.. code-block:: shell + + -u;root;-P;3306;-h;{{EIP}};-p{{PASSWORD}};--ssl-ca;/usr/bin/ca-bundle.pem;--force + +Connect the ReplaceDefinersCommand with this new Processor, by dragging a connector line from the first to the second. +You should be able to observe now that a second Queue component is added between them on the canvas: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-nfj.png + +5. Add a **Processor** of type **LogAttribute**; this component will emit attributes of the FlowFile for a predefined log level. + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-dsr.png + +Then drag a connection between the ExportMysqlDump and the LogAttribute Processors, and in the Create Connection popup +let’s define two new relationships: *original* and *nonzero status*. The former is the original queue message that was +processed from the Processor and the latter bears the potential errors (*non zero results*) that were thrown during +this step of the workflow. Every relationship will inject a dedicated queue in the workflow. Repeat the same steps for +the ReplaceDefinersCommand Processor. For ImportMySqlDump and LogAttribute Processors, activate all 3 available relationship options. +The output stream will log the successful results of our import workflow step. + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-dum.png + +Eventually, our LogAttribute Processor and its dependencies should now look like this on the canvas: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-nk1.png + +6. Start the Processors. As you will notice on the left-hand upper corner of every Processor on the canvas appears a stop sign. +That means that the Processors will not execute any commands even if we kick off a new instance of the workflow. +In order to start them press, for every single one of them — except LogAttribute, the start button marked with blue in the picture below: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-e7c.png + +Configure the Apache Nifi Server +================================ + +At this point we are not ready yet to run our workflow. The Apache Nifi server is lacking two additional resources. +The two ExecuteStreamCommand Processors will execute an export and import from and to remote MySQL instances using +the mysql-client, but the Apache NiFi container doesn’t have any knowledge of this package. We have to connect to our +container and install the required client. + +Let's connect first to the Apache Nifi container as root: + +.. code-block:: shell + + docker exec -it -u 0 nifi /bin/bash + +and install the client (in this case is the *mariadb-client* package): + +.. code-block:: shell + + apt-get update -y + apt-get install -y mariadb-client + +A quick sanity check to make sure that everything is in place. For that matter go to `/usr/bin/` and make sure you +that `mysqldump` and `mysql` are properly symlinked: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-eii.png + +Next we have to copy to the Apache Nifi container the SSL certificate we downloaded from the Open Telekom Cloud console. + +.. code-block:: shell + + docker cp ca-bundle.pem nifi:/usr/bin + +.. attention:: + For the time being, let's skip the step above in order to simulate an error in the migration workflow and we will + come back later to this. + +Start a Migration Workflow +========================= + +Open the cascading menu of the *GenerateFlowFile* component and click *Run Once*: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-f0t.png + +The current active Processor will be marked with this sign on right-hand upper corner on the canvas: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-f32.png + +Let’s see what happened and if the migration went through, and if no how could we debug and trace the source of our problem. +The canvas now will be updated with some more data in every *Processor* and *Queue*: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-nsn.png + +*GenerateFlowFile* Processor is informing us that has sent 1 request down the pipeline (*Out* 1 — in box marked in blue). +The *ExecuteMysqlDump* Processor ran successfully and wrote out a dump in the size of 160.59MB. Its logging queues show +us that we have a new entry in *original* and zero entries in *nonzero status*. (The latter indicates that the Processor ran **without any error**). +Let’s see what was written in the original queue. Open the queue: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-fap.png + +and under the *Properties* tab of the Queue, we can see which command was executed by our Processor: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-fc21.png + +Now let's focus on the second ExecuteStreamCommand Processor, the one that is responsible to import the dump to the target database. +We can see that it received an input of 160.59MB (that is our dump file, generated from the previous Processor); +it pushed it down in the *original* queue but it seems that migration didn’t go through as planned, +because we have items in the *nonzero status* queue. As a first step finding the culprit, we will inspect in the original queue +(open the *List Queue* and pick the element that corresponds to this very workflow instance under the *Details* tab). +We can either inspect the generated dump file that was handed over by the ExportMysqlDump Processor by either viewing or download it, + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-fhz.png + +or inspect the command that was executed to see if there is a helpful error message (in our case there is one): + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-fhm1.png + +A faster way though, figuring out what went wrong, is hovering over the red sign (that will appear in case of error) +in the upper right-hand corner of our Processor that threw the error: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220812-flv.png + +Now that we saw how we can, in principle, debug and investigate errors during the execution of our workflows, go back +to previous chapter guidelines and, this time, do copy the SSL certificate to the Apache Nifi container. + +We are now set to start a new migration instance. You will observe that after a while the *ImportMysqlDump* Processor goes +in execution mode, for the small sign on the right upper-hand corner that indicates the active threads currently running +on this component. After a while, when the workflow will: + +* not have any more active threads in any processor +* have an additional message in the outcome queue of the ImportMysqlDump Processor +* have no additional messages in the nonzero status queue of the ImportMysqlDump Processor + +then check your database — the migration would have successfully completed: + +.. figure:: https://architecture-center-poc-images.obs.eu-de.otc.t-systems.com/rds-migration/SCR-20220926-bhx.png + +References +========== + +.. seealso:: + + `Relational Database Service: Accessing RDS `_ + + `Database Services Overview with RDS Deep Dive `_ \ No newline at end of file diff --git a/doc/source/use-cases/networking/index.rst b/doc/source/use-cases/networking/index.rst new file mode 100644 index 0000000..c8c5604 --- /dev/null +++ b/doc/source/use-cases/networking/index.rst @@ -0,0 +1,12 @@ +Networking +========== + +Networking section illustrates how organizations can optimize their network infrastructures for diverse scenarios. +Explore practical examples showcasing the flexibility and scalability of networking solutions within the +Open Telekom Cloud. This section provides architects with insights into designing resilient and high-performance +networks, addressing specific use cases such as multi-tier applications or distributed architectures. +Discover recommended best practices for network security, load balancing, and connectivity, empowering users to tailor +their networking strategies to meet the unique requirements of their applications and workloads. + +.. toctree:: + :maxdepth: 1 diff --git a/doc/source/use-cases/security/index.rst b/doc/source/use-cases/security/index.rst new file mode 100644 index 0000000..d84b7b6 --- /dev/null +++ b/doc/source/use-cases/security/index.rst @@ -0,0 +1,11 @@ +Security & Compliance +===================== + +The Security & Compliance section emphasizes strategies for fortifying cloud environments. Explore practical examples +demonstrating how the platform addresses security and compliance challenges across industries. +This section provides architects with insights into implementing robust identity and access management, encryption +protocols, and compliance controls. Discover recommended best practices for securing sensitive data and ensuring +regulatory adherence, empowering users to design and deploy secure, compliant solutions on the Open Telekom Cloud. + +.. toctree:: + :maxdepth: 1