diff --git a/umn/source/_static/images/en-us_image_0000001223473845.png b/umn/source/_static/images/en-us_image_0000001082048529.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001223473845.png rename to umn/source/_static/images/en-us_image_0000001082048529.png diff --git a/umn/source/_static/images/en-us_image_0000001088110417.png b/umn/source/_static/images/en-us_image_0000001088110417.png deleted file mode 100644 index b138cac..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001088110417.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001113962636.png b/umn/source/_static/images/en-us_image_0000001113962636.png new file mode 100644 index 0000000..7c35ae0 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001113962636.png differ diff --git a/umn/source/_static/images/en-us_image_0000001126243447.png b/umn/source/_static/images/en-us_image_0000001126243447.png deleted file mode 100644 index 19d17be..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001126243447.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001144208440.png b/umn/source/_static/images/en-us_image_0000001144208440.png deleted file mode 100644 index f6280ad..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001144208440.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001144342236.png b/umn/source/_static/images/en-us_image_0000001144342236.png deleted file mode 100644 index 07d911c..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001144342236.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001144342238.png b/umn/source/_static/images/en-us_image_0000001144342238.png deleted file mode 100644 index b8cd51d..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001144342238.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001144578756.png b/umn/source/_static/images/en-us_image_0000001144578756.png deleted file mode 100644 index 4eda4be..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001144578756.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001144738550.png b/umn/source/_static/images/en-us_image_0000001144738550.png deleted file mode 100644 index 9a9b444..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001144738550.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001144779784.png b/umn/source/_static/images/en-us_image_0000001144779784.png deleted file mode 100644 index a4bbbd0..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001144779784.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001145535931.png b/umn/source/_static/images/en-us_image_0000001145545261.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001145535931.png rename to umn/source/_static/images/en-us_image_0000001145545261.png diff --git a/umn/source/_static/images/en-us_image_0000001148989534.png b/umn/source/_static/images/en-us_image_0000001148989534.png deleted file mode 100644 index f64927d..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001148989534.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001150420952.png b/umn/source/_static/images/en-us_image_0000001150420952.png deleted file mode 100644 index 9bca8f6..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001150420952.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001152953258.png b/umn/source/_static/images/en-us_image_0000001152953258.png deleted file mode 100644 index b19a824..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001152953258.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001160642447.png b/umn/source/_static/images/en-us_image_0000001160642447.png new file mode 100644 index 0000000..1be0aa0 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001160642447.png differ diff --git a/umn/source/_static/images/en-us_image_0000001160731158.png b/umn/source/_static/images/en-us_image_0000001160731158.png deleted file mode 100644 index 3e03cd3..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001160731158.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001172076961.png b/umn/source/_static/images/en-us_image_0000001172076961.png deleted file mode 100644 index 378328a..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001172076961.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001172392670.png b/umn/source/_static/images/en-us_image_0000001172392670.png new file mode 100644 index 0000000..05cb59e Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001172392670.png differ diff --git a/umn/source/_static/images/en-us_image_0000001176255102.png b/umn/source/_static/images/en-us_image_0000001176255102.png deleted file mode 100644 index c32d9be..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001176255102.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001176818150.png b/umn/source/_static/images/en-us_image_0000001176818150.png new file mode 100644 index 0000000..efdf9ff Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001176818150.png differ diff --git a/umn/source/_static/images/en-us_image_0000001178034114.png b/umn/source/_static/images/en-us_image_0000001178034114.png deleted file mode 100644 index d7b0553..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001178034114.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001178034116.png b/umn/source/_static/images/en-us_image_0000001178034116.png deleted file mode 100644 index e6a687b..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001178034116.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001178192666.png b/umn/source/_static/images/en-us_image_0000001178192666.png deleted file mode 100644 index d000a3f..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001178192666.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001178352604.png b/umn/source/_static/images/en-us_image_0000001178352604.png deleted file mode 100644 index 5bfbc6c..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001178352604.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190048341.png b/umn/source/_static/images/en-us_image_0000001190048341.png deleted file mode 100644 index 57a44dd..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190048341.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190168507.png b/umn/source/_static/images/en-us_image_0000001190168507.png deleted file mode 100644 index 5b3a3e0..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190168507.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190302085.png b/umn/source/_static/images/en-us_image_0000001190302085.png deleted file mode 100644 index dce5dbc..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190302085.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190302087.png b/umn/source/_static/images/en-us_image_0000001190302087.png deleted file mode 100644 index d70fb1a..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190302087.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190302089.png b/umn/source/_static/images/en-us_image_0000001190302089.png deleted file mode 100644 index 42c594e..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190302089.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190302091.png b/umn/source/_static/images/en-us_image_0000001190302091.png deleted file mode 100644 index cbdee42..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190302091.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190302095.png b/umn/source/_static/images/en-us_image_0000001190302095.png deleted file mode 100644 index 4be70fa..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190302095.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190302097.png b/umn/source/_static/images/en-us_image_0000001190302097.png deleted file mode 100644 index 689fd9c..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190302097.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190538605.png b/umn/source/_static/images/en-us_image_0000001190538605.png deleted file mode 100644 index 4d6df5d..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190538605.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001190658439.png b/umn/source/_static/images/en-us_image_0000001190658439.png deleted file mode 100644 index 7df14a7..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001190658439.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001192028618.png b/umn/source/_static/images/en-us_image_0000001192028618.png new file mode 100644 index 0000000..67590f4 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001192028618.png differ diff --git a/umn/source/_static/images/en-us_image_0000001195057213.png b/umn/source/_static/images/en-us_image_0000001195057213.png deleted file mode 100644 index c2958d9..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001195057213.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001198867835.png b/umn/source/_static/images/en-us_image_0000001198867835.png deleted file mode 100644 index 101fab4..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001198867835.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001198980979.png b/umn/source/_static/images/en-us_image_0000001198980979.png deleted file mode 100644 index 3432242..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001198980979.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001180446397.png b/umn/source/_static/images/en-us_image_0000001199021278.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001180446397.png rename to umn/source/_static/images/en-us_image_0000001199021278.png diff --git a/umn/source/_static/images/en-us_image_0000001098645539.png b/umn/source/_static/images/en-us_image_0000001199021298.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001098645539.png rename to umn/source/_static/images/en-us_image_0000001199021298.png diff --git a/umn/source/_static/images/en-us_image_0186273271.png b/umn/source/_static/images/en-us_image_0000001199021308.png similarity index 100% rename from umn/source/_static/images/en-us_image_0186273271.png rename to umn/source/_static/images/en-us_image_0000001199021308.png diff --git a/umn/source/_static/images/en-us_image_0000001243407853.png b/umn/source/_static/images/en-us_image_0000001199021320.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001243407853.png rename to umn/source/_static/images/en-us_image_0000001199021320.png diff --git a/umn/source/_static/images/en-us_image_0000001093275701.png b/umn/source/_static/images/en-us_image_0000001199021334.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001093275701.png rename to umn/source/_static/images/en-us_image_0000001199021334.png diff --git a/umn/source/_static/images/en-us_image_0000001190859184.png b/umn/source/_static/images/en-us_image_0000001199181228.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001190859184.png rename to umn/source/_static/images/en-us_image_0000001199181228.png diff --git a/umn/source/_static/images/en-us_image_0000001192723194.png b/umn/source/_static/images/en-us_image_0000001199181230.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001192723194.png rename to umn/source/_static/images/en-us_image_0000001199181230.png diff --git a/umn/source/_static/images/en-us_image_0000001409700093.png b/umn/source/_static/images/en-us_image_0000001199181232.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001409700093.png rename to umn/source/_static/images/en-us_image_0000001199181232.png diff --git a/umn/source/_static/images/en-us_image_0000001168537057.png b/umn/source/_static/images/en-us_image_0000001199181266.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001168537057.png rename to umn/source/_static/images/en-us_image_0000001199181266.png diff --git a/umn/source/_static/images/en-us_image_0000001190538599.png b/umn/source/_static/images/en-us_image_0000001199181298.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001190538599.png rename to umn/source/_static/images/en-us_image_0000001199181298.png diff --git a/umn/source/_static/images/en-us_image_0000001159292060.png b/umn/source/_static/images/en-us_image_0000001199181334.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001159292060.png rename to umn/source/_static/images/en-us_image_0000001199181334.png diff --git a/umn/source/_static/images/en-us_image_0000001231949185.png b/umn/source/_static/images/en-us_image_0000001199181336.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001231949185.png rename to umn/source/_static/images/en-us_image_0000001199181336.png diff --git a/umn/source/_static/images/en-us_image_0000001116237931.png b/umn/source/_static/images/en-us_image_0000001199181338.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001116237931.png rename to umn/source/_static/images/en-us_image_0000001199181338.png diff --git a/umn/source/_static/images/en-us_image_0295359661.png b/umn/source/_static/images/en-us_image_0000001199181340.png similarity index 100% rename from umn/source/_static/images/en-us_image_0295359661.png rename to umn/source/_static/images/en-us_image_0000001199181340.png diff --git a/umn/source/_static/images/en-us_image_0144049227.png b/umn/source/_static/images/en-us_image_0000001199341250.png similarity index 100% rename from umn/source/_static/images/en-us_image_0144049227.png rename to umn/source/_static/images/en-us_image_0000001199341250.png diff --git a/umn/source/_static/images/en-us_image_0000001199341268.png b/umn/source/_static/images/en-us_image_0000001199341268.png new file mode 100644 index 0000000..672ccf8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001199341268.png differ diff --git a/umn/source/_static/images/en-us_image_0000001199341330.png b/umn/source/_static/images/en-us_image_0000001199341330.png new file mode 100644 index 0000000..5ddb8e3 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001199341330.png differ diff --git a/umn/source/_static/images/en-us_image_0000001192723190.png b/umn/source/_static/images/en-us_image_0000001199501200.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001192723190.png rename to umn/source/_static/images/en-us_image_0000001199501200.png diff --git a/umn/source/_static/images/en-us_image_0000001163847995.png b/umn/source/_static/images/en-us_image_0000001199501230.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001163847995.png rename to umn/source/_static/images/en-us_image_0000001199501230.png diff --git a/umn/source/_static/images/en-us_image_0000001199501262.png b/umn/source/_static/images/en-us_image_0000001199501262.png new file mode 100644 index 0000000..e3a75af Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001199501262.png differ diff --git a/umn/source/_static/images/en-us_image_0000001409860177.png b/umn/source/_static/images/en-us_image_0000001199501276.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001409860177.png rename to umn/source/_static/images/en-us_image_0000001199501276.png diff --git a/umn/source/_static/images/en-us_image_0000001199848585.png b/umn/source/_static/images/en-us_image_0000001199501290.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001199848585.png rename to umn/source/_static/images/en-us_image_0000001199501290.png diff --git a/umn/source/_static/images/en-us_image_0000001199757520.png b/umn/source/_static/images/en-us_image_0000001199757520.png new file mode 100644 index 0000000..0987110 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001199757520.png differ diff --git a/umn/source/_static/images/en-us_image_0258503428.png b/umn/source/_static/images/en-us_image_0000001201381906.png similarity index 100% rename from umn/source/_static/images/en-us_image_0258503428.png rename to umn/source/_static/images/en-us_image_0000001201381906.png diff --git a/umn/source/_static/images/en-us_image_0276664213.png b/umn/source/_static/images/en-us_image_0000001201823500.png similarity index 100% rename from umn/source/_static/images/en-us_image_0276664213.png rename to umn/source/_static/images/en-us_image_0000001201823500.png diff --git a/umn/source/_static/images/en-us_image_0000001202101148.png b/umn/source/_static/images/en-us_image_0000001202101148.png new file mode 100644 index 0000000..42fccb7 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001202101148.png differ diff --git a/umn/source/_static/images/en-us_image_0276664792.png b/umn/source/_static/images/en-us_image_0000001202103502.png similarity index 100% rename from umn/source/_static/images/en-us_image_0276664792.png rename to umn/source/_static/images/en-us_image_0000001202103502.png diff --git a/umn/source/_static/images/en-us_image_0000001203031716.png b/umn/source/_static/images/en-us_image_0000001203031716.png new file mode 100644 index 0000000..7cdfc47 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001203031716.png differ diff --git a/umn/source/_static/images/en-us_image_0000001359820608.png b/umn/source/_static/images/en-us_image_0000001203385342.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001359820608.png rename to umn/source/_static/images/en-us_image_0000001203385342.png diff --git a/umn/source/_static/images/en-us_image_0000001204449561.png b/umn/source/_static/images/en-us_image_0000001204449561.png deleted file mode 100644 index bf526e2..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001204449561.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001205757902.png b/umn/source/_static/images/en-us_image_0000001205757902.png new file mode 100644 index 0000000..36a1a48 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001205757902.png differ diff --git a/umn/source/_static/images/en-us_image_0000001206876656.png b/umn/source/_static/images/en-us_image_0000001206876656.png new file mode 100644 index 0000000..303124e Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001206876656.png differ diff --git a/umn/source/_static/images/en-us_image_0000001207511384.png b/umn/source/_static/images/en-us_image_0000001207511384.png new file mode 100644 index 0000000..5a0e760 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001207511384.png differ diff --git a/umn/source/_static/images/en-us_image_0000001217183707.png b/umn/source/_static/images/en-us_image_0000001217183707.png new file mode 100644 index 0000000..aa7b279 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001217183707.png differ diff --git a/umn/source/_static/images/en-us_image_0000001218074121.png b/umn/source/_static/images/en-us_image_0000001218074121.png new file mode 100644 index 0000000..97a22d8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001218074121.png differ diff --git a/umn/source/_static/images/en-us_image_0000001221007635.png b/umn/source/_static/images/en-us_image_0000001221007635.png deleted file mode 100644 index 2907131..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001221007635.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001221376671.png b/umn/source/_static/images/en-us_image_0000001221376671.png deleted file mode 100644 index ba5937d..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001221376671.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001221501677.png b/umn/source/_static/images/en-us_image_0000001221501677.png new file mode 100644 index 0000000..f1403c1 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001221501677.png differ diff --git a/umn/source/_static/images/en-us_image_0000001221820189.png b/umn/source/_static/images/en-us_image_0000001221820189.png new file mode 100644 index 0000000..2af028b Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001221820189.png differ diff --git a/umn/source/_static/images/en-us_image_0000001222591781.png b/umn/source/_static/images/en-us_image_0000001222591781.png deleted file mode 100644 index 866483a..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001222591781.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001223152415.png b/umn/source/_static/images/en-us_image_0000001223152415.png deleted file mode 100644 index a0a3b16..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001223152415.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001223152417.png b/umn/source/_static/images/en-us_image_0000001223152417.png deleted file mode 100644 index 64a42b7..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001223152417.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001223393893.png b/umn/source/_static/images/en-us_image_0000001223393893.png deleted file mode 100644 index 3f5e34e..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001223393893.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001225747980.png b/umn/source/_static/images/en-us_image_0000001225747980.png new file mode 100644 index 0000000..e36aa54 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001225747980.png differ diff --git a/umn/source/_static/images/en-us_image_0000001226818003.png b/umn/source/_static/images/en-us_image_0000001226818003.png new file mode 100644 index 0000000..c586bbf Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001226818003.png differ diff --git a/umn/source/_static/images/en-us_image_0000001227977765.png b/umn/source/_static/images/en-us_image_0000001227977765.png deleted file mode 100644 index 10acfee..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001227977765.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001229793402.png b/umn/source/_static/images/en-us_image_0000001229793402.png deleted file mode 100644 index 794961c..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001229793402.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001229794946.png b/umn/source/_static/images/en-us_image_0000001229794946.png deleted file mode 100644 index ca497cf..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001229794946.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001236263298.png b/umn/source/_static/images/en-us_image_0000001236263298.png deleted file mode 100644 index 405ed94..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001236263298.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001236562704.png b/umn/source/_static/images/en-us_image_0000001236562704.png new file mode 100644 index 0000000..d8f8266 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001236562704.png differ diff --git a/umn/source/_static/images/en-us_image_0000001236582394.png b/umn/source/_static/images/en-us_image_0000001236582394.png deleted file mode 100644 index 0e0debd..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001236582394.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001236723668.png b/umn/source/_static/images/en-us_image_0000001236723668.png new file mode 100644 index 0000000..4f4611d Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001236723668.png differ diff --git a/umn/source/_static/images/en-us_image_0000001238163131.png b/umn/source/_static/images/en-us_image_0000001238163131.png deleted file mode 100644 index edccd54..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001238163131.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001238489436.png b/umn/source/_static/images/en-us_image_0000001238489436.png new file mode 100644 index 0000000..22b800a Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001238489436.png differ diff --git a/umn/source/_static/images/en-us_image_0000001238830246.png b/umn/source/_static/images/en-us_image_0000001238830246.png new file mode 100644 index 0000000..9d1f035 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001238830246.png differ diff --git a/umn/source/_static/images/en-us_image_0000001238903330.png b/umn/source/_static/images/en-us_image_0000001238903330.png new file mode 100644 index 0000000..82ffe7e Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001238903330.png differ diff --git a/umn/source/_static/images/en-us_image_0000001238003081.png b/umn/source/_static/images/en-us_image_0000001243981115.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001238003081.png rename to umn/source/_static/images/en-us_image_0000001243981115.png diff --git a/umn/source/_static/images/en-us_image_0000001117575950.png b/umn/source/_static/images/en-us_image_0000001243981117.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001117575950.png rename to umn/source/_static/images/en-us_image_0000001243981117.png diff --git a/umn/source/_static/images/en-us_image_0000001144779790.png b/umn/source/_static/images/en-us_image_0000001243981141.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001144779790.png rename to umn/source/_static/images/en-us_image_0000001243981141.png diff --git a/umn/source/_static/images/en-us_image_0000001098403383.png b/umn/source/_static/images/en-us_image_0000001243981147.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001098403383.png rename to umn/source/_static/images/en-us_image_0000001243981147.png diff --git a/umn/source/_static/images/en-us_image_0000001243981177.png b/umn/source/_static/images/en-us_image_0000001243981177.png new file mode 100644 index 0000000..d1794e7 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001243981177.png differ diff --git a/umn/source/_static/images/en-us_image_0276664171.png b/umn/source/_static/images/en-us_image_0000001243981181.png similarity index 100% rename from umn/source/_static/images/en-us_image_0276664171.png rename to umn/source/_static/images/en-us_image_0000001243981181.png diff --git a/umn/source/_static/images/en-us_image_0000001144502022.png b/umn/source/_static/images/en-us_image_0000001243981203.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001144502022.png rename to umn/source/_static/images/en-us_image_0000001243981203.png diff --git a/umn/source/_static/images/en-us_image_0000001086743939.png b/umn/source/_static/images/en-us_image_0000001244101107.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001086743939.png rename to umn/source/_static/images/en-us_image_0000001244101107.png diff --git a/umn/source/_static/images/en-us_image_0000001134406294.png b/umn/source/_static/images/en-us_image_0000001244101121.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001134406294.png rename to umn/source/_static/images/en-us_image_0000001244101121.png diff --git a/umn/source/_static/images/en-us_image_0000001159118361.png b/umn/source/_static/images/en-us_image_0000001244101223.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001159118361.png rename to umn/source/_static/images/en-us_image_0000001244101223.png diff --git a/umn/source/_static/images/en-us_image_0000001244141105.png b/umn/source/_static/images/en-us_image_0000001244141105.png new file mode 100644 index 0000000..8567ed8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001244141105.png differ diff --git a/umn/source/_static/images/en-us_image_0000001144620002.png b/umn/source/_static/images/en-us_image_0000001244141139.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001144620002.png rename to umn/source/_static/images/en-us_image_0000001244141139.png diff --git a/umn/source/_static/images/en-us_image_0144054048.gif b/umn/source/_static/images/en-us_image_0000001244141141.gif similarity index 100% rename from umn/source/_static/images/en-us_image_0144054048.gif rename to umn/source/_static/images/en-us_image_0000001244141141.gif diff --git a/umn/source/_static/images/en-us_image_0000001163928763.png b/umn/source/_static/images/en-us_image_0000001244141181.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001163928763.png rename to umn/source/_static/images/en-us_image_0000001244141181.png diff --git a/umn/source/_static/images/en-us_image_0000001409580465.png b/umn/source/_static/images/en-us_image_0000001244141191.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001409580465.png rename to umn/source/_static/images/en-us_image_0000001244141191.png diff --git a/umn/source/_static/images/en-us_image_0000001198861255.png b/umn/source/_static/images/en-us_image_0000001244141217.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001198861255.png rename to umn/source/_static/images/en-us_image_0000001244141217.png diff --git a/umn/source/_static/images/en-us_image_0000001244261055.png b/umn/source/_static/images/en-us_image_0000001244261055.png new file mode 100644 index 0000000..2512244 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001244261055.png differ diff --git a/umn/source/_static/images/en-us_image_0000001360140132.png b/umn/source/_static/images/en-us_image_0000001244261069.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001360140132.png rename to umn/source/_static/images/en-us_image_0000001244261069.png diff --git a/umn/source/_static/images/en-us_image_0000001142984374.png b/umn/source/_static/images/en-us_image_0000001244261071.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001142984374.png rename to umn/source/_static/images/en-us_image_0000001244261071.png diff --git a/umn/source/_static/images/en-us_image_0000001120226646.png b/umn/source/_static/images/en-us_image_0000001244261073.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001120226646.png rename to umn/source/_static/images/en-us_image_0000001244261073.png diff --git a/umn/source/_static/images/en-us_image_0254985211.png b/umn/source/_static/images/en-us_image_0000001244261103.png similarity index 100% rename from umn/source/_static/images/en-us_image_0254985211.png rename to umn/source/_static/images/en-us_image_0000001244261103.png diff --git a/umn/source/_static/images/en-us_image_0000001144342232.png b/umn/source/_static/images/en-us_image_0000001244261119.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001144342232.png rename to umn/source/_static/images/en-us_image_0000001244261119.png diff --git a/umn/source/_static/images/en-us_image_0254986677.png b/umn/source/_static/images/en-us_image_0000001244261161.png similarity index 100% rename from umn/source/_static/images/en-us_image_0254986677.png rename to umn/source/_static/images/en-us_image_0000001244261161.png diff --git a/umn/source/_static/images/en-us_image_0000001244261167.png b/umn/source/_static/images/en-us_image_0000001244261167.png new file mode 100644 index 0000000..f1ed744 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001244261167.png differ diff --git a/umn/source/_static/images/en-us_image_0000001160748146.png b/umn/source/_static/images/en-us_image_0000001244261169.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001160748146.png rename to umn/source/_static/images/en-us_image_0000001244261169.png diff --git a/umn/source/_static/images/en-us_image_0000001159831938.png b/umn/source/_static/images/en-us_image_0000001244261171.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001159831938.png rename to umn/source/_static/images/en-us_image_0000001244261171.png diff --git a/umn/source/_static/images/en-us_image_0000001153101092.png b/umn/source/_static/images/en-us_image_0000001244261173.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001153101092.png rename to umn/source/_static/images/en-us_image_0000001244261173.png diff --git a/umn/source/_static/images/en-us_image_0000001244997085.png b/umn/source/_static/images/en-us_image_0000001244997085.png new file mode 100644 index 0000000..ff8087b Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001244997085.png differ diff --git a/umn/source/_static/images/en-us_image_0000001247802971.png b/umn/source/_static/images/en-us_image_0000001247802971.png new file mode 100644 index 0000000..fcdc776 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001247802971.png differ diff --git a/umn/source/_static/images/en-us_image_0276664178.png b/umn/source/_static/images/en-us_image_0000001248663503.png similarity index 100% rename from umn/source/_static/images/en-us_image_0276664178.png rename to umn/source/_static/images/en-us_image_0000001248663503.png diff --git a/umn/source/_static/images/en-us_image_0276664570.png b/umn/source/_static/images/en-us_image_0000001249023453.png similarity index 100% rename from umn/source/_static/images/en-us_image_0276664570.png rename to umn/source/_static/images/en-us_image_0000001249023453.png diff --git a/umn/source/_static/images/en-us_image_0000001249073211.png b/umn/source/_static/images/en-us_image_0000001249073211.png new file mode 100644 index 0000000..87446d8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001249073211.png differ diff --git a/umn/source/_static/images/en-us_image_0000001249958645.png b/umn/source/_static/images/en-us_image_0000001249958645.png new file mode 100644 index 0000000..36a1a48 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001249958645.png differ diff --git a/umn/source/_static/images/en-us_image_0000001251716033.png b/umn/source/_static/images/en-us_image_0000001251716033.png new file mode 100644 index 0000000..92f9830 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001251716033.png differ diff --git a/umn/source/_static/images/en-us_image_0000001256348238.jpg b/umn/source/_static/images/en-us_image_0000001256348238.jpg new file mode 100644 index 0000000..7747408 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001256348238.jpg differ diff --git a/umn/source/_static/images/en-us_image_0000001274316069.png b/umn/source/_static/images/en-us_image_0000001274316069.png deleted file mode 100644 index f86a5ec..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001274316069.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001274882416.png b/umn/source/_static/images/en-us_image_0000001274882416.png new file mode 100644 index 0000000..935bdf7 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001274882416.png differ diff --git a/umn/source/_static/images/en-us_image_0000001276433425.png b/umn/source/_static/images/en-us_image_0000001276433425.png new file mode 100644 index 0000000..6a1cd90 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001276433425.png differ diff --git a/umn/source/_static/images/en-us_image_0000001280171657.png b/umn/source/_static/images/en-us_image_0000001280171657.png deleted file mode 100644 index c3426e3..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001280171657.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001280181541.png b/umn/source/_static/images/en-us_image_0000001280181541.png deleted file mode 100644 index 6e12ed4..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001280181541.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001280421317.png b/umn/source/_static/images/en-us_image_0000001280421317.png deleted file mode 100644 index 108dd89..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001280421317.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001280466745.png b/umn/source/_static/images/en-us_image_0000001280466745.png deleted file mode 100644 index 0492b6b..0000000 Binary files a/umn/source/_static/images/en-us_image_0000001280466745.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001283301301.png b/umn/source/_static/images/en-us_image_0000001283301301.png new file mode 100644 index 0000000..770a9fc Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001283301301.png differ diff --git a/umn/source/_static/images/en-us_image_0000001283343269.png b/umn/source/_static/images/en-us_image_0000001283343269.png new file mode 100644 index 0000000..4c17d7c Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001283343269.png differ diff --git a/umn/source/_static/images/en-us_image_0000001290111529.png b/umn/source/_static/images/en-us_image_0000001290111529.png new file mode 100644 index 0000000..226fcf7 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001290111529.png differ diff --git a/umn/source/_static/images/en-us_image_0000001291567729.png b/umn/source/_static/images/en-us_image_0000001291567729.png new file mode 100644 index 0000000..b48bf29 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001291567729.png differ diff --git a/umn/source/_static/images/en-us_image_0000001325364477.png b/umn/source/_static/images/en-us_image_0000001325364477.png new file mode 100644 index 0000000..b181bea Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001325364477.png differ diff --git a/umn/source/_static/images/en-us_image_0000001325377749.png b/umn/source/_static/images/en-us_image_0000001325377749.png new file mode 100644 index 0000000..6cab297 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001325377749.png differ diff --git a/umn/source/_static/images/en-us_image_0000001283755568.png b/umn/source/_static/images/en-us_image_0000001336475537.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001283755568.png rename to umn/source/_static/images/en-us_image_0000001336475537.png diff --git a/umn/source/_static/images/en-us_image_0000001352539924.png b/umn/source/_static/images/en-us_image_0000001352539924.png new file mode 100644 index 0000000..5ae1c1d Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001352539924.png differ diff --git a/umn/source/_static/images/en-us_image_0000001360670117.png b/umn/source/_static/images/en-us_image_0000001360670117.png new file mode 100644 index 0000000..603c946 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001360670117.png differ diff --git a/umn/source/_static/images/en-us_image_0000001378942548.png b/umn/source/_static/images/en-us_image_0000001378942548.png new file mode 100644 index 0000000..a004b89 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001378942548.png differ diff --git a/umn/source/_static/images/en-us_image_0000001392259910.png b/umn/source/_static/images/en-us_image_0000001392259910.png new file mode 100644 index 0000000..1dbc838 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001392259910.png differ diff --git a/umn/source/_static/images/en-us_image_0000001392280374.png b/umn/source/_static/images/en-us_image_0000001392280374.png new file mode 100644 index 0000000..c569d0f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001392280374.png differ diff --git a/umn/source/_static/images/en-us_image_0000001392318380.png b/umn/source/_static/images/en-us_image_0000001392318380.png new file mode 100644 index 0000000..8e6d050 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001392318380.png differ diff --git a/umn/source/_static/images/en-us_image_0000001397733101.png b/umn/source/_static/images/en-us_image_0000001397733101.png new file mode 100644 index 0000000..ece3f19 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001397733101.png differ diff --git a/umn/source/_static/images/en-us_image_0000001402494682.png b/umn/source/_static/images/en-us_image_0000001402494682.png new file mode 100644 index 0000000..cf48188 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001402494682.png differ diff --git a/umn/source/_static/images/en-us_image_0165888686.png b/umn/source/_static/images/en-us_image_0000001408895746.png similarity index 100% rename from umn/source/_static/images/en-us_image_0165888686.png rename to umn/source/_static/images/en-us_image_0000001408895746.png diff --git a/umn/source/_static/images/en-us_image_0000001460905374.png b/umn/source/_static/images/en-us_image_0000001460905374.png new file mode 100644 index 0000000..50d2b8f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001460905374.png differ diff --git a/umn/source/_static/images/en-us_image_0000001461224886.png b/umn/source/_static/images/en-us_image_0000001461224886.png new file mode 100644 index 0000000..fd35a0f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001461224886.png differ diff --git a/umn/source/_static/images/en-us_image_0000001359980148.png b/umn/source/_static/images/en-us_image_0000001464878016.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001359980148.png rename to umn/source/_static/images/en-us_image_0000001464878016.png diff --git a/umn/source/_static/images/en-us_image_0000001360140128.png b/umn/source/_static/images/en-us_image_0000001465197524.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001360140128.png rename to umn/source/_static/images/en-us_image_0000001465197524.png diff --git a/umn/source/_static/images/en-us_image_0000001480191270.png b/umn/source/_static/images/en-us_image_0000001480191270.png new file mode 100644 index 0000000..290b94f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001480191270.png differ diff --git a/umn/source/_static/images/en-us_image_0000001482541956.png b/umn/source/_static/images/en-us_image_0000001482541956.png new file mode 100644 index 0000000..ca9934b Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001482541956.png differ diff --git a/umn/source/_static/images/en-us_image_0000001482546084.png b/umn/source/_static/images/en-us_image_0000001482546084.png new file mode 100644 index 0000000..b2f9f12 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001482546084.png differ diff --git a/umn/source/_static/images/en-us_image_0000001482701968.png b/umn/source/_static/images/en-us_image_0000001482701968.png new file mode 100644 index 0000000..9b7e01f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001482701968.png differ diff --git a/umn/source/_static/images/en-us_image_0000001482796460.png b/umn/source/_static/images/en-us_image_0000001482796460.png new file mode 100644 index 0000000..60b03ef Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001482796460.png differ diff --git a/umn/source/_static/images/en-us_image_0000001409700089.png b/umn/source/_static/images/en-us_image_0000001515838557.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001409700089.png rename to umn/source/_static/images/en-us_image_0000001515838557.png diff --git a/umn/source/_static/images/en-us_image_0000001409740389.png b/umn/source/_static/images/en-us_image_0000001515917789.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001409740389.png rename to umn/source/_static/images/en-us_image_0000001515917789.png diff --git a/umn/source/_static/images/en-us_image_0000001528627005.png b/umn/source/_static/images/en-us_image_0000001528627005.png new file mode 100644 index 0000000..d74a10f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001528627005.png differ diff --git a/umn/source/_static/images/en-us_image_0000001531373685.png b/umn/source/_static/images/en-us_image_0000001531373685.png new file mode 100644 index 0000000..2375222 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001531373685.png differ diff --git a/umn/source/_static/images/en-us_image_0000001531533045.png b/umn/source/_static/images/en-us_image_0000001531533045.png new file mode 100644 index 0000000..f963c97 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001531533045.png differ diff --git a/umn/source/_static/images/en-us_image_0000001531533921.png b/umn/source/_static/images/en-us_image_0000001531533921.png new file mode 100644 index 0000000..da5aed3 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001531533921.png differ diff --git a/umn/source/_static/images/en-us_image_0000001533181077.png b/umn/source/_static/images/en-us_image_0000001533181077.png new file mode 100644 index 0000000..47324ea Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001533181077.png differ diff --git a/umn/source/_static/images/en-us_image_0000001533585325.png b/umn/source/_static/images/en-us_image_0000001533585325.png new file mode 100644 index 0000000..c18b799 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001533585325.png differ diff --git a/umn/source/_static/images/en-us_image_0000001533586881.png b/umn/source/_static/images/en-us_image_0000001533586881.png new file mode 100644 index 0000000..92f9830 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0000001533586881.png differ diff --git a/umn/source/_static/images/en-us_image_0121749065.png b/umn/source/_static/images/en-us_image_0121749065.png deleted file mode 100644 index 7fab2b4..0000000 Binary files a/umn/source/_static/images/en-us_image_0121749065.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0144042759.png b/umn/source/_static/images/en-us_image_0144042759.png deleted file mode 100644 index 8df3e3b..0000000 Binary files a/umn/source/_static/images/en-us_image_0144042759.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0144045351.png b/umn/source/_static/images/en-us_image_0144045351.png deleted file mode 100644 index 4f6eaf1..0000000 Binary files a/umn/source/_static/images/en-us_image_0144045351.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0165899095.png b/umn/source/_static/images/en-us_image_0165899095.png deleted file mode 100644 index 4be5db1..0000000 Binary files a/umn/source/_static/images/en-us_image_0165899095.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0165899282.png b/umn/source/_static/images/en-us_image_0165899282.png deleted file mode 100644 index 8e539c6..0000000 Binary files a/umn/source/_static/images/en-us_image_0165899282.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0181616313.png b/umn/source/_static/images/en-us_image_0181616313.png deleted file mode 100644 index bfcccfe..0000000 Binary files a/umn/source/_static/images/en-us_image_0181616313.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0181616314.png b/umn/source/_static/images/en-us_image_0181616314.png deleted file mode 100644 index bd21029..0000000 Binary files a/umn/source/_static/images/en-us_image_0181616314.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0183134473.png b/umn/source/_static/images/en-us_image_0183134473.png deleted file mode 100644 index c293cf9..0000000 Binary files a/umn/source/_static/images/en-us_image_0183134473.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0183134479.png b/umn/source/_static/images/en-us_image_0183134479.png deleted file mode 100644 index c293cf9..0000000 Binary files a/umn/source/_static/images/en-us_image_0183134479.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0183134608.png b/umn/source/_static/images/en-us_image_0183134608.png deleted file mode 100644 index c293cf9..0000000 Binary files a/umn/source/_static/images/en-us_image_0183134608.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0183674977.png b/umn/source/_static/images/en-us_image_0183674977.png deleted file mode 100644 index 1fe5ee3..0000000 Binary files a/umn/source/_static/images/en-us_image_0183674977.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0195434915.png b/umn/source/_static/images/en-us_image_0195434915.png deleted file mode 100644 index 4f6eaf1..0000000 Binary files a/umn/source/_static/images/en-us_image_0195434915.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0198873490.png b/umn/source/_static/images/en-us_image_0198873490.png deleted file mode 100644 index 577d034..0000000 Binary files a/umn/source/_static/images/en-us_image_0198873490.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0198876479.png b/umn/source/_static/images/en-us_image_0198876479.png deleted file mode 100644 index 577d034..0000000 Binary files a/umn/source/_static/images/en-us_image_0198876479.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0214003838.png b/umn/source/_static/images/en-us_image_0214003838.png deleted file mode 100644 index 15ee848..0000000 Binary files a/umn/source/_static/images/en-us_image_0214003838.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0220702939.png b/umn/source/_static/images/en-us_image_0220702939.png deleted file mode 100644 index 8ea2b58..0000000 Binary files a/umn/source/_static/images/en-us_image_0220702939.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0220765374.png b/umn/source/_static/images/en-us_image_0220765374.png deleted file mode 100644 index e6d9c02..0000000 Binary files a/umn/source/_static/images/en-us_image_0220765374.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0250508826.png b/umn/source/_static/images/en-us_image_0250508826.png deleted file mode 100644 index 229ce20..0000000 Binary files a/umn/source/_static/images/en-us_image_0250508826.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0259714782.png b/umn/source/_static/images/en-us_image_0259714782.png deleted file mode 100644 index e69e52c..0000000 Binary files a/umn/source/_static/images/en-us_image_0259714782.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0259716601.png b/umn/source/_static/images/en-us_image_0259716601.png deleted file mode 100644 index 81ed369..0000000 Binary files a/umn/source/_static/images/en-us_image_0259716601.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0259814716.png b/umn/source/_static/images/en-us_image_0259814716.png deleted file mode 100644 index 81ed369..0000000 Binary files a/umn/source/_static/images/en-us_image_0259814716.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0259814717.png b/umn/source/_static/images/en-us_image_0259814717.png deleted file mode 100644 index e69e52c..0000000 Binary files a/umn/source/_static/images/en-us_image_0259814717.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0000001223152421.png b/umn/source/_static/images/en-us_image_0261818822.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001223152421.png rename to umn/source/_static/images/en-us_image_0261818822.png diff --git a/umn/source/_static/images/en-us_image_0000001178034110.png b/umn/source/_static/images/en-us_image_0261818824.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001178034110.png rename to umn/source/_static/images/en-us_image_0261818824.png diff --git a/umn/source/_static/images/en-us_image_0261818867.png b/umn/source/_static/images/en-us_image_0261818867.png new file mode 100644 index 0000000..99f7849 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0261818867.png differ diff --git a/umn/source/_static/images/en-us_image_0000001178034108.png b/umn/source/_static/images/en-us_image_0261818875.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001178034108.png rename to umn/source/_static/images/en-us_image_0261818875.png diff --git a/umn/source/_static/images/en-us_image_0000001178192670.png b/umn/source/_static/images/en-us_image_0261818885.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001178192670.png rename to umn/source/_static/images/en-us_image_0261818885.png diff --git a/umn/source/_static/images/en-us_image_0000001223393899.png b/umn/source/_static/images/en-us_image_0261818886.png similarity index 100% rename from umn/source/_static/images/en-us_image_0000001223393899.png rename to umn/source/_static/images/en-us_image_0261818886.png diff --git a/umn/source/_static/images/en-us_image_0261818893.png b/umn/source/_static/images/en-us_image_0261818893.png new file mode 100644 index 0000000..e1b6ec8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0261818893.png differ diff --git a/umn/source/_static/images/en-us_image_0261818896.png b/umn/source/_static/images/en-us_image_0261818896.png new file mode 100644 index 0000000..e1b6ec8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0261818896.png differ diff --git a/umn/source/_static/images/en-us_image_0261818899.png b/umn/source/_static/images/en-us_image_0261818899.png new file mode 100644 index 0000000..e1b6ec8 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0261818899.png differ diff --git a/umn/source/_static/images/en-us_image_0261820020.png b/umn/source/_static/images/en-us_image_0261820020.png new file mode 100644 index 0000000..b063957 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0261820020.png differ diff --git a/umn/source/_static/images/en-us_image_0268523694.png b/umn/source/_static/images/en-us_image_0268523694.png new file mode 100644 index 0000000..718c8fa Binary files /dev/null and b/umn/source/_static/images/en-us_image_0268523694.png differ diff --git a/umn/source/_static/images/en-us_image_0269288708.png b/umn/source/_static/images/en-us_image_0269288708.png deleted file mode 100644 index 63c26fe..0000000 Binary files a/umn/source/_static/images/en-us_image_0269288708.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0273156799.png b/umn/source/_static/images/en-us_image_0273156799.png deleted file mode 100644 index f6a5599..0000000 Binary files a/umn/source/_static/images/en-us_image_0273156799.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0275445543.png b/umn/source/_static/images/en-us_image_0275445543.png new file mode 100644 index 0000000..1909444 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0275445543.png differ diff --git a/umn/source/_static/images/en-us_image_0275445566.png b/umn/source/_static/images/en-us_image_0275445566.png new file mode 100644 index 0000000..1909444 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0275445566.png differ diff --git a/umn/source/_static/images/en-us_image_0275452681.png b/umn/source/_static/images/en-us_image_0275452681.png new file mode 100644 index 0000000..1909444 Binary files /dev/null and b/umn/source/_static/images/en-us_image_0275452681.png differ diff --git a/umn/source/_static/images/en-us_image_0278498565.png b/umn/source/_static/images/en-us_image_0278498565.png new file mode 100644 index 0000000..2756b2f Binary files /dev/null and b/umn/source/_static/images/en-us_image_0278498565.png differ diff --git a/umn/source/_static/images/en-us_image_0298565473.png b/umn/source/_static/images/en-us_image_0298565473.png deleted file mode 100644 index 68d37b1..0000000 Binary files a/umn/source/_static/images/en-us_image_0298565473.png and /dev/null differ diff --git a/umn/source/_static/images/en-us_image_0300973777.png b/umn/source/_static/images/en-us_image_0300973777.png deleted file mode 100644 index 1fe5ee3..0000000 Binary files a/umn/source/_static/images/en-us_image_0300973777.png and /dev/null differ diff --git a/umn/source/add-ons/autoscaler.rst b/umn/source/add-ons/autoscaler.rst index c22caea..edb2c9d 100644 --- a/umn/source/add-ons/autoscaler.rst +++ b/umn/source/add-ons/autoscaler.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0154.html +:original_name: cce_10_0154.html -.. _cce_01_0154: +.. _cce_10_0154: autoscaler ========== @@ -12,7 +12,7 @@ Autoscaler is an important Kubernetes controller. It supports microservice scali When the CPU or memory usage of a microservice is too high, horizontal pod autoscaling is triggered to add pods to reduce the load. These pods can be automatically reduced when the load is low, allowing the microservice to run as efficiently as possible. -CCE simplifies the creation, upgrade, and manual scaling of Kubernetes clusters, in which traffic loads change over time. To balance resource usage and workload performance of nodes, Kubernetes introduces the autoscaler add-on to automatically resize a cluster based on the resource usage required for workloads deployed in the cluster. For details, see :ref:`Creating a Node Scaling Policy `. +CCE simplifies the creation, upgrade, and manual scaling of Kubernetes clusters, in which traffic loads change over time. To balance resource usage and workload performance of nodes, Kubernetes introduces the autoscaler add-on to automatically resize a cluster based on the resource usage required for workloads deployed in the cluster. For details, see :ref:`Creating a Node Scaling Policy `. Open source community: https://github.com/kubernetes/autoscaler @@ -23,16 +23,20 @@ autoscaler controls auto scale-out and scale-in. - **Auto scale-out** - If pods in a cluster cannot be scheduled due to insufficient worker nodes, cluster scaling is triggered to add nodes. The nodes to be added have the same specification as configured for the node pool to which the nodes belong. For details, see :ref:`Creating a Node Scaling Policy `. + You can choose either of the following methods: - The add-on follows the "No Less, No More" policy. For example, if three cores are required for creating a pod and the system supports four-core and eight-core nodes, autoscaler will preferentially create a four-core node. - - .. note:: + - If pods in a cluster cannot be scheduled due to insufficient worker nodes, cluster scaling is triggered to add nodes. The nodes to be added have the same specification as configured for the node pool to which the nodes belong. Auto scale-out will be performed when: - Node resources are insufficient. - - No node affinity policy is set in the pod scheduling configuration. That is, if a node has been configured as an affinity node for pods, no node will not be automatically added when pods cannot be scheduled. For details about how to configure the node affinity policy, see :ref:`Node Affinity `. + - No node affinity policy is set in the pod scheduling configuration. That is, if a node has been configured as an affinity node for pods, no node will not be automatically added when pods cannot be scheduled. For details about how to configure the node affinity policy, see :ref:`Scheduling Policy (Affinity/Anti-affinity) `. + + - When the cluster meets the node scaling policy, cluster scale-out is also triggered. For details, see :ref:`Creating a Node Scaling Policy `. + + .. note:: + + The add-on follows the "No Less, No More" policy. For example, if three cores are required for creating a pod and the system supports four-core and eight-core nodes, autoscaler will preferentially create a four-core node. - **Auto scale-in** @@ -49,126 +53,96 @@ Notes and Constraints - Only clusters of v1.9.10-r2 and later support autoscaler. - Ensure that there are sufficient resources for installing the add-on. - -.. _cce_01_0154__section15573161754711: +- The default node pool does not support auto scaling. For details, see :ref:`Description of DefaultPool `. Installing the Add-on --------------------- -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Marketplace** tab page, click **Install Add-on** under **autoscaler**. +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Add-ons** in the navigation pane, locate **autoscaler** on the right, and click **Install**. +#. Configure add-on installation parameters. -#. On the **Install Add-on** page, select the cluster and the add-on version, and click **Next: Configuration**. + .. table:: **Table 1** Specifications configuration -#. Configure add-on installation parameters listed in :ref:`Table 1 `. + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+========================================================================================================================================================================================================================================================================================================================================================+ + | Add-on Specifications | The add-on can be deployed in the following specifications: | + | | | + | | .. note:: | + | | | + | | When the autoscaler add-on is deployed in HA or customized mode, anti-affinity policies exist between add-on instances and the add-on instances are deployed on different nodes. Therefore, the number of available nodes in the cluster must be greater than or equal to the number of add-on instances to ensure high availability of the add-on. | + | | | + | | - **Single**: The add-on is deployed with only one pod. | + | | - **HA50**: The add-on is deployed with two pods, serving a cluster with 50 nodes and ensuring high availability. | + | | - **HA200**: The add-on is deployed with two pods, serving a cluster with 50 nodes and ensuring high availability. Each pod uses more resources than those of the **HA50** specification. | + | | - **Custom**: You can customize the number of pods and specifications as required. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - .. _cce_01_0154__table1582194517429: + .. table:: **Table 2** Parameter configuration - .. table:: **Table 1** Basic settings - - +-------------------------+------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Add-on Version | Description | - +=========================+====================================+==========================================================================================================================================================================================================================================================================================================================================================+ - | Add-on Specifications | Available in all versions | The add-on can be deployed in the following specifications: | - | | | | - | | | - **Single**: The add-on is deployed with only one pod. | - | | | - **HA50**: The add-on is deployed with two pods, serving a cluster with 50 nodes and ensuring high availability. | - | | | - **HA200**: The add-on is deployed with two pods, serving a cluster with 50 nodes and ensuring high availability. Each pod uses more resources than those of the **HA50** specification. | - | | | - **Custom**: You can customize the number of pods and specifications as required. | - +-------------------------+------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Instances | Available in all versions | Number of pods that will be created to match the selected add-on specifications. The number cannot be modified. | - +-------------------------+------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container | Available in all versions | CPU and memory quotas of the container allowed for the selected add-on specifications. The quotas cannot be modified. | - +-------------------------+------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Login Mode | Available only in certain versions | Select a login mode for the worker nodes to be added during auto scale-up. | - | | | | - | | | If you select **Key pair**: | - | | | | - | | | **Key pair**: Select an existing key pair or create a new one for identity authentication during remote login to the added nodes. | - +-------------------------+------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Auto Scale-In | Available in all versions | **Off**: Auto scale-down is not allowed. Only auto scale-up is allowed. | - | | | | - | | | **On**: Enable auto scale-in. The scale-in policy is valid for node pools in the cluster with auto scaling enabled. | - | | | | - | | | - **Idle Time (min)**: Time for which a node should be unneeded before it is eligible for scale-down. Default value: 10 minutes. | - | | | - **Resource Usage**: If the percentage of both CPU and memory usage on a node is below this threshold, auto scale-down will be triggered to delete the node from the cluster. The default value is 0.5, which means 50%. | - | | | - **Scale-in Cooldown After Scale-out**: The time after scale-up that the scale-down evaluation will resume. Default value: 10 minutes. | - | | | | - | | | .. note:: | - | | | | - | | | If both auto scale-out and scale-in exist in a cluster, you are advised to set **Scale-in Cooldown After Scale-out** to 0 minutes. This can prevent the node scale-in from being blocked due to continuous scale-out of some node pools or retries upon a scale-out failure, resulting in unexpected waste of node resources. | - | | | | - | | | - **Scale-in Cooldown After Node Deletion**: The time after node deletion that the scale-down evaluation will resume. Default value: 10 minutes. | - | | | - **Scale-in Cooldown After Failure**: The time after a scale-down failure that the scale-down evaluation will resume. Default value: 3 minutes. For details about the impact and relationship between the scale-in cooling intervals configured in the node pool and autoscaler, see :ref:`Scale-in Cooling Interval `. | - | | | - **Max empty bulk delete**: The maximum number of empty nodes that can be deleted at the same time. Default value: 10. | - | | | - **Node Recheck Timeout**: The timeout before autoscaler checks again the node that could not be previously removed. Default value: 5 minutes. | - +-------------------------+------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Node Pool Configuration | Available only in certain versions | Configuration of the default node pool. A node pool is a group of compute nodes with the same node type (VM or BMS), specifications, and labels. When a cluster needs to be scaled up, autoscaler will automatically add nodes from node pools to the cluster. If no custom node pool is available, autoscaler will use the default node pool. | - | | | | - | | | Click **Add Node Pool Configuration** and set the following parameters: | - | | | | - | | | - **AZ**: A physical region where resources use independent power supplies and networks. AZs are physically isolated but interconnected through the internal network. | - | | | | - | | | - **OS**: OS of the nodes to be created. | - | | | | - | | | - **Taints**: No taints are added by default. | - | | | | - | | | Taints allow nodes to repel a set of pods. You can add a maximum of 10 taints for each node pool. Each taint contains the following parameters: | - | | | | - | | | - **Key**: A key must contain 1 to 63 characters starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | - | | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | - | | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | - | | | | - | | | .. important:: | - | | | | - | | | NOTICE: | - | | | | - | | | - If taints are used, you must configure tolerations in the YAML files of pods. Otherwise, scale-up may fail or pods cannot be scheduled onto the added nodes. | - | | | - Taints cannot be modified after configuration. Incorrect taints may cause a scale-up failure or prevent pods from being scheduled onto the added nodes. | - | | | | - | | | - **Resource Tags**: Resource tags can be added to classify resources. | - | | | | - | | | .. note:: | - | | | | - | | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | - | | | | - | | | - **Specifications**: CPU and memory of the added nodes. | - +-------------------------+------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - To configure more add-on parameters, click **Advanced Settings** at the bottom of this page. - - .. table:: **Table 2** Advanced settings - - +-------------------+------------------------------------+----------------------------------------------------------------------------------------------------------+ - | Parameter | Add-on Version | Description | - +===================+====================================+==========================================================================================================+ - | Total Nodes | Available in all versions | Maximum number of nodes that can be managed by the cluster, within which cluster scale-out is performed. | - +-------------------+------------------------------------+----------------------------------------------------------------------------------------------------------+ - | Total Cores | Available in all versions | Maximum sum of CPU cores of all nodes in a cluster, within which cluster scale-out is performed. | - +-------------------+------------------------------------+----------------------------------------------------------------------------------------------------------+ - | Total Memory (GB) | Available in all versions | Maximum sum of memory of all nodes in a cluster, within which cluster scale-out is performed. | - +-------------------+------------------------------------+----------------------------------------------------------------------------------------------------------+ - | Auto Scale-Out | Available only in certain versions | **Triggered when there are pods unscheduled**: Selected by default. | - +-------------------+------------------------------------+----------------------------------------------------------------------------------------------------------+ + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+========================================================================================================================================================================================================================================================================================================================================================================+ + | Scaling | You can select the following options as required: | + | | | + | | - **Nodes are automatically added (from the node pool) when pods in the cluster cannot be scheduled.** | + | | | + | | That is, when a pod is in **Pending** state, automatic scale-out is performed. If a node has been configured as an affinity node for pods, no node will not be automatically added when pods cannot be scheduled. Generally, an HPA policy works with such scaling. For details, see :ref:`Using HPA and CA for Auto Scaling of Workloads and Nodes `. | + | | | + | | If this parameter is not selected, scaling can be performed only through :ref:`node scaling policies `. | + | | | + | | - Auto node scale-in | + | | | + | | - **Node Idle Time (min)**: Time for which a node should be unneeded before it is eligible for scale-down. Default value: 10 minutes. | + | | | + | | - **Scale-in Threshold**: If the percentage of both requested CPU and memory on a node is below this threshold, auto scale-down will be triggered to delete the node from the cluster. The default value is 0.5, which means 50%. | + | | | + | | - **Stabilization Window (s)** | + | | | + | | How long after a scale-out that a scale-in evaluation resumes. Default value: 10 minutes. | + | | | + | | .. note:: | + | | | + | | If both auto scale-out and scale-in exist in a cluster, you are advised to set **How long after a scale-out that a scale-in evaluation resumes** to 0 minutes. This can prevent the node scale-in from being blocked due to continuous scale-out of some node pools or retries upon a scale-out failure, resulting in unexpected waste of node resources. | + | | | + | | How long after the node deletion that a scale-in evaluation resumes. Default value: 10 minutes. | + | | | + | | How long after a scale-in failure that a scale-in evaluation resumes. Default value: 3 minutes. For details about the impact and relationship between the scale-in cooling intervals configured in the node pool and autoscaler, see :ref:`Description of the Scale-In Cool-Down Period `. | + | | | + | | - **Max. Nodes for Batch Deletion**: Maximum number of empty nodes that can be deleted at the same time. Default value: 10. | + | | | + | | This feature applies only to idle nodes. Idle nodes can be concurrently scaled in. Nodes that are not idle can only be scaled in one by one. | + | | | + | | .. note:: | + | | | + | | During node scale-in, if the pod on the node does not need to be evicted (such as the pods of DaemonSet), the node is idle. Otherwise, the node is not idle. | + | | | + | | - **Check Interval**: Interval for checking again a node that could not be removed before. Default value: 5 minutes. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Total Nodes | Maximum number of nodes that can be managed by the cluster, within which cluster scale-out is performed. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Total CPUs | Maximum sum of CPU cores of all nodes in a cluster, within which cluster scale-out is performed. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Total Memory (GB) | Maximum sum of memory of all nodes in a cluster, within which cluster scale-out is performed. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. When the configuration is complete, click **Install**. - After the add-on is installed, click **Go Back to Previous Page**. On the **Add-on Instance** tab page, select the corresponding cluster to view the running instance. This indicates that the add-on has been installed on each node in the cluster. +.. _cce_10_0154__section59676731017: -Upgrading the Add-on --------------------- +Description of the Scale-In Cool-Down Period +-------------------------------------------- -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, click **Upgrade** under **autoscaler**. +Scale-in cooling intervals can be configured in the node pool settings and the autoscaler add-on settings. - .. note:: +**Scale-in cooling interval configured in a node pool** - - If the **Upgrade** button is unavailable, the current add-on is already up-to-date and no upgrade is required. - - If the **Upgrade** button is available, click **Upgrade** to upgrade the add-on. - - During the upgrade, the coredns add-on of the original version on cluster nodes will be discarded, and the add-on of the target version will be installed. +This interval indicates the period during which nodes added to the current node pool after a scale-out operation cannot be deleted. This interval takes effect at the node pool level. -#. In the dialog box displayed, set parameters and upgrade the add-on. For details about the parameters, see the parameter description in :ref:`Installing the Add-on `. +**Scale-in cooling interval configured in the autoscaler add-on** -Uninstalling the Add-on ------------------------ +The interval after a scale-out indicates the period during which the entire cluster cannot be scaled in after the autoscaler add-on triggers scale-out (due to the unschedulable pods, metrics, and scaling policies). This interval takes effect at the cluster level. -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, select the target cluster and click **Uninstall** under **autoscaler**. -#. In the dialog box displayed, click **Yes** to uninstall the add-on. +The interval after a node is deleted indicates the period during which the cluster cannot be scaled in after the autoscaler add-on triggers scale-in. This interval takes effect at the cluster level. + +The interval after a failed scale-in indicates the period during which the cluster cannot be scaled in after the autoscaler add-on triggers scale-in. This interval takes effect at the cluster level. diff --git a/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst b/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst index 1b1fc16..812cbf8 100644 --- a/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst +++ b/umn/source/add-ons/coredns_system_resource_add-on_mandatory.rst @@ -1,8 +1,8 @@ -:original_name: cce_01_0129.html +:original_name: cce_10_0129.html -.. _cce_01_0129: +.. _cce_10_0129: -coredns (System Resource Add-on, Mandatory) +coredns (System Resource Add-On, Mandatory) =========================================== Introduction @@ -10,7 +10,7 @@ Introduction The coredns add-on is a DNS server that provides domain name resolution services for Kubernetes clusters. coredns chains plug-ins to provide additional features. -coredns is an open-source software and has been a part of CNCF. It provides a means for cloud services to discover each other in cloud-native deployments. Each of the plug-ins chained by coredns provides a particular DNS function. You can integrate coredns with only the plug-ins you need to make it fast, efficient, and flexible. When used in a Kubernetes cluster, coredns can automatically discover services in the cluster and provide domain name resolution for these services. By working with a cloud DNS server, coredns can resolve external domain names for workloads in a cluster. +coredns is an open-source software and has been a part of CNCF. It provides a means for cloud services to discover each other in cloud-native deployments. Each of the plug-ins chained by coredns provides a particular DNS function. You can integrate coredns with only the plug-ins you need to make it fast, efficient, and flexible. When used in a Kubernetes cluster, coredns can automatically discover services in the cluster and provide domain name resolution for these services. By working with DNS server, coredns can resolve external domain names for workloads in a cluster. **coredns is a system resource add-on. It is installed by default when a cluster of Kubernetes v1.11 or later is created.** @@ -20,137 +20,150 @@ CoreDNS official website: https://coredns.io/ Open source community: https://github.com/coredns/coredns +.. note:: + + For details, see :ref:`DNS `. + Notes and Constraints --------------------- -When CoreDNS is running properly or being upgraded, ensure that the number of available nodes is greater than or equal to the number of CoreDNS instances and all CoreDNS instances are running. Otherwise, the upgrade will fail. +When coredns is running properly or being upgraded, ensure that the number of available nodes is greater than or equal to the number of coredns instances and all coredns instances are running. Otherwise, the upgrade will fail. Installing the Add-on --------------------- This add-on has been installed by default. If it is uninstalled due to some reasons, you can reinstall it by performing the following steps: -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Marketplace** tab page, click **Install Add-on** under **coredns**. +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Add-ons** in the navigation pane, locate **coredns** on the right, and click **Install**. -#. On the **Install Add-on** page, select the cluster and the add-on version, and click **Next: Configuration**. - -#. In the **Configuration** step, set the following parameters: +#. On the **Install Add-on** page, select the add-on specifications and set related parameters. .. table:: **Table 1** coredns add-on parameters - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +=======================+=========================================================================================================================================================================================================================================================================================================================================+ - | Add-on Specifications | Concurrent domain name resolution ability. Select add-on specifications that best fit your needs. | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Instances | Number of pods that will be created to match the selected add-on specifications. The number cannot be modified. | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container | CPU and memory quotas of the container allowed for the selected add-on specifications. The quotas cannot be modified. | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Notes | Add-on precautions. Read the precautions before you proceed with the step. | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | stub domain | A domain name server for a user-defined domain name. The format is a key-value pair. The key is a suffix of DNS domain name, and the value is one or more DNS IP addresses. For example, **acme.local -- 1.2.3.4,6.7.8.9** means that DNS requests with the **.acme.local** suffix are forwarded to a DNS listening at 1.2.3.4,6.7.8.9. | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ + | Add-on Specifications | Concurrent domain name resolution ability. Select add-on specifications that best fit your needs. | + | | | + | | If you select **Custom qps**, the domain name resolution QPS provided by CoreDNS is positively correlated with the CPU consumption. Adjust the number of pods and container CPU/memory quotas as required. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pods | Number of pods that will be created to match the selected add-on specifications. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Containers | CPU and memory quotas of the container allowed for the selected add-on specifications. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameters | - **parameterSyncStrategy**: indicates whether to configure consistency check when an add-on is upgraded. | + | | | + | | - **ensureConsistent**: indicates that the configuration consistency check is enabled. If the configuration recorded in the cluster is inconsistent with the actual configuration, the add-on cannot be upgraded. | + | | - **force**: indicates that the configuration consistency check is ignored during an upgrade. Ensure that the current effective configuration is the same as the original configuration. After the add-on is upgraded, restore the value of **parameterSyncStrategy** to **ensureConsistent** and enable the configuration consistency check again. | + | | | + | | - **stub_domains**: A domain name server for a user-defined domain name. The format is a key-value pair. The key is a suffix of DNS domain name, and the value is one or more DNS IP addresses. | + | | | + | | - **upstream_nameservers**: IP address of the upstream DNS server. | + | | | + | | - **servers**: The servers configuration is available since coredns 1.23.1. You can customize the servers configuration. For details, see `dns-custom-nameservers `__. **plugins** indicates the configuration of each component in coredns (https://coredns.io/manual/plugins/). You are advised to retain the default configurations in common scenarios to prevent CoreDNS from being unavailable due to configuration errors. Each plugin component contains **name**, **parameters** (optional), and **configBlock** (optional). The format of the generated Corefile is as follows: | + | | | + | | $name $parameters { | + | | | + | | $configBlock | + | | | + | | } | + | | | + | | :ref:`Table 2 ` describes common plugins. | + | | | + | | Example: | + | | | + | | .. code-block:: | + | | | + | | { | + | | "servers": [ | + | | { | + | | "plugins": [ | + | | { | + | | "name": "bind", | + | | "parameters": "{$POD_IP}" | + | | }, | + | | { | + | | "name": "cache", | + | | "parameters": 30 | + | | }, | + | | { | + | | "name": "errors" | + | | }, | + | | { | + | | "name": "health", | + | | "parameters": "{$POD_IP}:8080" | + | | }, | + | | { | + | | "configBlock": "pods insecure\nfallthrough in-addr.arpa ip6.arpa", | + | | "name": "kubernetes", | + | | "parameters": "cluster.local in-addr.arpa ip6.arpa" | + | | }, | + | | { | + | | "name": "loadbalance", | + | | "parameters": "round_robin" | + | | }, | + | | { | + | | "name": "prometheus", | + | | "parameters": "{$POD_IP}:9153" | + | | }, | + | | { | + | | "configBlock": "policy random", | + | | "name": "forward", | + | | "parameters": ". /etc/resolv.conf" | + | | }, | + | | { | + | | "name": "reload" | + | | }, | + | | { | + | | "name": "log" | + | | } | + | | ], | + | | "port": 5353, | + | | "zones": [ | + | | { | + | | "zone": "." | + | | } | + | | ] | + | | } | + | | ], | + | | "stub_domains": { | + | | "acme.local": [ | + | | "1.2.3.4", | + | | "6.7.8.9" | + | | ] | + | | }, | + | | "upstream_nameservers": ["8.8.8.8", "8.8.4.4"] | + | | } | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + .. _cce_10_0129__table1420814384015: + + .. table:: **Table 2** Default plugin configuration of the active zone of coredns + + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | plugin Name | Description | + +=============+======================================================================================================================================================================================+ + | bind | Host IP address listened by coredns. You are advised to retain the default value **{$POD_IP}**. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | cache | DNS cache is enabled. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | errors | Errors are logged to stdout. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | health | Health check configuration. The current listening IP address is {$POD_IP}:8080. Retain the default value. Otherwise, the coredns health check fails and coredns restarts repeatedly. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes | CoreDNS Kubernetes plug-in, which provides the service parsing capability in a cluster. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | loadbalance | Round-robin DNS load balancer that randomizes the order of A, AAAA, and MX records in the answer. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | prometheus | Port for obtaining coredns metrics. The default zone listening IP address is {$\ *POD_IP*}:9153. Retain the default value. Otherwise, CloudScope cannot collect coredns metrics. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | forward | Any queries that are not within the cluster domain of Kubernetes will be forwarded to predefined resolvers (/etc/resolv.conf). | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | reload | The changed Corefile can be automatically reloaded. After editing the ConfigMap, wait for two minutes for the modification to take effect. | + +-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. After the preceding configurations are complete, click **Install**. - After the add-on is installed, click **Go Back to Previous Page**. On the **Add-on Instance** tab page, select the corresponding cluster to view the running instance. This indicates that the add-on has been installed on each node in the cluster. - -Configuring the Stub Domain for CoreDNS ---------------------------------------- - -Cluster administrators can modify the ConfigMap for the CoreDNS Corefile to change how service discovery works. They can configure stub domains for CoreDNS using the proxy plug-in. - -Assume that a cluster administrator has a Consul DNS server located at 10.150.0.1 and all Consul domain names have the suffix **.consul.local**. - -To configure this Consul DNS server in CoreDNS, run the following command to edit the CoreDNS ConfigMap: - -**kubectl edit configmap coredns -n kube-system** - -Example configuration: - -.. code-block:: - - consul.local:5353 { - errors - cache 30 - proxy . 10.150.0.1 - } - -**In clusters of v1.15.11 and later, the modified ConfigMap is as follows:** - -.. code-block:: - - apiVersion: v1 - metadata: - name: coredns - namespace: kube-system - selfLink: /api/v1/namespaces/kube-system/configmaps/coredns - uid: 00cb8f29-62d7-4df8-a769-0a16237903c1 - resourceVersion: '2074614' - creationTimestamp: '2021-04-07T03:52:42Z' - labels: - app: coredns - k8s-app: coredns - kubernetes.io/cluster-service: 'true' - kubernetes.io/name: CoreDNS - release: cceaddon-coredns - data: - Corefile: |- - .:5353 { - bind {$POD_IP} - cache 30 - errors - health {$POD_IP}:8080 - kubernetes cluster.local in-addr.arpa ip6.arpa { - pods insecure - upstream /etc/resolv.conf - fallthrough in-addr.arpa ip6.arpa - } - loadbalance round_robin - prometheus {$POD_IP}:9153 - forward . /etc/resolv.conf - reload - } - - consul.local:5353 { - errors - cache 30 - proxy . 10.150.0.1 - } - -**In clusters earlier than v1.15.11, the modified ConfigMap is as follows:** - -.. code-block:: - - apiVersion: v1 - data: - Corefile: |- - .:5353 { - cache 30 - errors - health - kubernetes cluster.local in-addr.arpa ip6.arpa { - pods insecure - upstream /etc/resolv.conf - fallthrough in-addr.arpa ip6.arpa - } - loadbalance round_robin - prometheus 0.0.0.0:9153 - proxy . /etc/resolv.conf - reload - } - - consul.local:5353 { - errors - cache 30 - proxy . 10.150.0.1 - } - kind: ConfigMap - metadata: - name: coredns - namespace: kube-system - How Does Domain Name Resolution Work in Kubernetes? --------------------------------------------------- @@ -181,40 +194,7 @@ DNS policies can be set on a per-pod basis. Currently, Kubernetes supports four - Names that do not match the suffix (for example, **widget.com**): The request is forwarded to the upstream DNS. -.. figure:: /_static/images/en-us_image_0186273271.png +.. figure:: /_static/images/en-us_image_0000001199021308.png :alt: **Figure 1** Routing **Figure 1** Routing - -Upgrading the Add-on --------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, click **Upgrade** under **coredns**. - - .. note:: - - - If the **Upgrade** button is unavailable, the current add-on is already up-to-date and no upgrade is required. - - During the upgrade, the previous configurations are lost and need to be specified again. - - When the upgrade is complete, the original coredns version on cluster nodes will be replaced by the latest version. If an exception occurs during the upgrade, uninstall the add-on and then re-install it. - -#. On the **Basic Information** page, select the add-on version and click **Next**. - -#. Configure the parameters listed in :ref:`Table 2 `. After the configuration is complete, click **Upgrade** to upgrade the coredns add-on. - - .. _cce_01_0129__table1410658238: - - .. table:: **Table 2** Parameters for installing coredns - - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +=======================+=========================================================================================================================================================================================================================================================================================================================================+ - | Add-on Specifications | Concurrent domain name resolution ability. Select add-on specifications that best fit your needs. | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | stub domain | A domain name server for a user-defined domain name. The format is a key-value pair. The key is a suffix of DNS domain name, and the value is one or more DNS IP addresses. For example, **acme.local -- 1.2.3.4,6.7.8.9** means that DNS requests with the **.acme.local** suffix are forwarded to a DNS listening at 1.2.3.4,6.7.8.9. | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -Uninstalling the Add-on ------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, click **Uninstall** under **coredns**. -#. In the dialog box displayed, click **Yes** to uninstall the add-on. diff --git a/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst b/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst index 11cf87a..cef0210 100644 --- a/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst +++ b/umn/source/add-ons/everest_system_resource_add-on_mandatory.rst @@ -1,52 +1,68 @@ -:original_name: cce_01_0066.html +:original_name: cce_10_0066.html -.. _cce_01_0066: +.. _cce_10_0066: -everest (System Resource Add-on, Mandatory) +everest (System Resource Add-On, Mandatory) =========================================== Introduction ------------ -Everest is a cloud-native container storage system. Based on Container Storage Interface (CSI), clusters of Kubernetes v1.15 or later can interconnect with cloud storage services such as EVS, OBS, SFS, and SFS Turbo. +Everest is a cloud native container storage system. Based on the Container Storage Interface (CSI), clusters of Kubernetes v1.15.6 or later obtain access to cloud storage services. **everest is a system resource add-on. It is installed by default when a cluster of Kubernetes v1.15 or later is created.** Notes and Constraints --------------------- -- If your cluster is upgraded from v1.13 to v1.15, :ref:`storage-driver ` is replaced by everest (v1.1.6 or later) for container storage. The takeover does not affect the original storage functions. For details about CSI and FlexVolume, see :ref:`Differences Between CSI and FlexVolume Plug-ins `. +- If your cluster is upgraded from v1.13 to v1.15, :ref:`storage-driver ` is replaced by everest (v1.1.6 or later) for container storage. The takeover does not affect the original storage functions. - In version 1.2.0 of the everest add-on, **key authentication** is optimized when OBS is used. After the everest add-on is upgraded from a version earlier than 1.2.0, you need to restart all workloads that use OBS in the cluster. Otherwise, workloads may not be able to use OBS. -- By default, this add-on is installed in **clusters of v1.15 and later**. For clusters of v1.13 and earlier, the :ref:`storage-driver ` add-on is installed by default. +- By default, this add-on is installed in **clusters of v1.15 and later**. For clusters of v1.13 and earlier, the :ref:`storage-driver ` add-on is installed by default. Installing the Add-on --------------------- This add-on has been installed by default. If it is uninstalled due to some reasons, you can reinstall it by performing the following steps: -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Marketplace** tab page, click **Install Add-on** under **everest**. +#. Log in to the CCE console and access the cluster console. Choose **Add-ons** in the navigation pane, locate **everest** on the right, and click **Install**. -#. On the **Install Add-on** page, select the cluster and the add-on version, and click **Next: Configuration**. +#. Select **Standalone**, **HA**, or **Custom** for **Add-on Specifications**. -#. Select **Single** or **HA** for **Add-on Specifications**, and click **Install**. + The everest add-on contains the following containers. You can adjust the specifications as required. - After the add-on is installed, click **Go Back to Previous Page**. On the **Add-on Instance** tab page, select the corresponding cluster to view the running instance. This indicates that the add-on has been installed on each node in the cluster. + - **everest-csi-controller**: A Deployment workload. This container is responsible for creating, deleting, snapshotting, expanding, attaching, and detaching volumes. If the cluster version is 1.19 or later and the add-on version is 1.2.\ *x*, the pod of the everest-csi-driver component also has an everest-localvolume-manager container by default. This container manages the creation of LVM storage pools and local PVs on the node. -Upgrading the Add-on --------------------- + .. note:: -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, click **Upgrade** under **everest**. + If you select **Custom**, the recommended **everest-csi-controller** memory configuration is as follows: - .. note:: + - If the number of pods and PVCs is less than 2000, set the memory upper limit to 600 MiB. + - If the number of pods and PVCs is less than 5000, set the memory upper limit to 1 GiB. - - If the **Upgrade** button is unavailable, the current add-on is already up-to-date and no upgrade is required. - - When the upgrade is complete, the original everest version on cluster nodes will be replaced by the latest version. + - **everest-csi-driver**: A DaemonSet workload. This container is responsible for mounting and unmounting PVs and resizing file systems. If the add-on version is 1.2.\ *x* and the region where the cluster is located supports node-attacher, the pod of the everest-csi-driver component also contains an everest-node-attacher container. This container is responsible for distributed EVS attaching. This configuration item is available in some regions. -#. On the **Basic Information** page, select the add-on version and click **Next**. -#. Select **Single** or **HA** for **Add-on Specifications**, and click **Upgrade**. + .. note:: -Uninstalling the Add-on ------------------------ + If you select **Custom**, it is recommended that the **everest-csi-driver** memory limit be greater than or equal to 300 MiB. If the value is too small, the add-on container cannot be started and the add-on is unavailable. -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, click **Uninstall** under **everest**. -#. In the dialog box displayed, click **Yes** to uninstall the add-on. +#. Set related parameters. + + In everest 1.2.26 or later, the performance of attaching a large number of EVS volumes is optimized. The following three parameters are provided: + + - **csi_attacher_worker_threads**: number of workers that can concurrently mount EVS volumes. The default value is **60**. + - **csi_attacher_detach_worker_threads**: number of workers that can concurrently unmount EVS volumes. The default value is **60**. + - **volume_attaching_flow_ctrl**: maximum number of EVS volumes that can be mounted by the everest add-on within one minute. The default value is **0**, indicating that the EVS volume mounting performance is determined by the underlying storage resources. + + The preceding three parameters are associated with each other and are constrained by the underlying storage resources in the region where the cluster is located. If you want to mount a large number of volumes (more than 500 EVS volumes per minute), you can contact the customer service personnel and configure the parameters under their guidance to prevent the everest add-on from running abnormally due to improper parameter settings. + + Other parameters + + - **cluster_id**: cluster ID + - **default_vpc_id**: ID of the VPC to which the data warehouse cluster belongs + - **disable_auto_mount_secret**: indicates whether the default AK/SK can be used when an object bucket or parallel file system is mounted. The default value is **false**. + - **enable_node_attacher**: indicates whether to enable the attacher on the agent to process the `VolumeAttachment `__. + - **flow_control**: This parameter is left blank by default. + - **over_subscription**: overcommitment ratio of the local storage pool (**local_storage**). The default value is **80**. If the size of the local storage pool is 100 GB, you can overcommit 180 GB. + - **project_id**: ID of the project to which the cluster belongs. + +#. Click **Install**. diff --git a/umn/source/add-ons/gpu-beta.rst b/umn/source/add-ons/gpu-beta.rst index d4b92a1..75ac615 100644 --- a/umn/source/add-ons/gpu-beta.rst +++ b/umn/source/add-ons/gpu-beta.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0141.html +:original_name: cce_10_0141.html -.. _cce_01_0141: +.. _cce_10_0141: gpu-beta ======== @@ -8,37 +8,31 @@ gpu-beta Introduction ------------ -gpu-beta is a device management add-on that supports GPUs in containers. It supports only NVIDIA Tesla drivers. +gpu-beta is a device management add-on that supports GPUs in containers. If GPU nodes are used in the cluster, the gpu-beta add-on must be installed. Notes and Constraints --------------------- -- This add-on is available only in certain regions. -- This add-on can be installed only in CCE clusters of v1.11 or later. -- If GPU nodes are used in the cluster, the gpu-beta add-on must be installed. - The driver to be downloaded must be a **.run** file. -- Only Tesla drivers are supported, not GRID drivers. - -.. important:: - - - If the download link is a public network address, for example, **https://us.download.nvidia.com/tesla/396.37/NVIDIA-Linux-x86_64-396.37.run**, bind an EIP to each GPU node. For details about how to obtain the driver link, see :ref:`Obtaining the Driver Link from Public Network `. - - If the download link is an OBS URL, you do not need to bind an EIP to GPU nodes. - - Ensure that the NVIDIA driver version matches the GPU node. - - After the driver version is changed, restart the node for the change to take effect. +- Only NVIDIA Tesla drivers are supported, not GRID drivers. +- When installing or reinstalling the add-on, ensure that the driver download address is correct and accessible. CCE does not verify the address validity. +- The gpu-beta add-on only enables you to download the driver and execute the installation script. The add-on status only indicates that how the add-on is running, not whether the driver is successfully installed. Installing the Add-on --------------------- -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Marketplace** tab page, click **Install Add-on** under **gpu-beta**. +#. Log in to the CCE console and access the cluster console. Choose **Add-ons** in the navigation pane, locate **gpu-beta** on the right, and click **Install**. +#. Configure the driver link. -#. On the **Install Add-on** page, select the cluster and the add-on version, and click **Next: Configuration**. + .. important:: -#. In the **Configuration** step, enter the link to download the NVIDIA driver. + - If the download link is a public network address, for example, **https://us.download.nvidia.com/tesla/470.103.01/NVIDIA-Linux-x86_64-470.103.01.run**, bind an EIP to each GPU node. For details about how to obtain the driver link, see :ref:`Obtaining the Driver Link from Public Network `. + - If the download link is an OBS URL, you do not need to bind an EIP to GPU nodes. + - Ensure that the NVIDIA driver version matches the GPU node. + - After the driver version is changed, restart the node for the change to take effect. #. Click **Install**. - After the add-on is installed, click **Go Back to Previous Page**. On the **Add-on Instance** tab page, select the corresponding cluster to view the running instance. This indicates that the add-on has been installed on each GPU node in the cluster. - Verifying the Add-on -------------------- @@ -60,7 +54,7 @@ If GPU information is returned, the device is available and the add-on is succes |image1| -.. _cce_01_0141__section95451728192112: +.. _cce_10_0141__section95451728192112: Obtaining the Driver Link from Public Network --------------------------------------------- @@ -70,45 +64,35 @@ Obtaining the Driver Link from Public Network 3. Visit https://www.nvidia.com/Download/Find.aspx?lang=en. -4. Select the driver information on the **NVIDIA Driver Downloads** page, as shown in :ref:`Figure 1 `. **Operating System** must be **Linux 64-bit**. +4. Select the driver information on the **NVIDIA Driver Downloads** page, as shown in :ref:`Figure 1 `. **Operating System** must be **Linux 64-bit**. - .. _cce_01_0141__fig11696366517: + .. _cce_10_0141__fig11696366517: - .. figure:: /_static/images/en-us_image_0000001280466745.png + .. figure:: /_static/images/en-us_image_0000001531533921.png :alt: **Figure 1** Setting parameters **Figure 1** Setting parameters -5. After confirming the driver information, click **SEARCH**. A page is displayed, showing the driver information, as shown in :ref:`Figure 2 `. Click **DOWNLOAD**. +5. After confirming the driver information, click **SEARCH**. A page is displayed, showing the driver information, as shown in :ref:`Figure 2 `. Click **DOWNLOAD**. - .. _cce_01_0141__fig7873421145213: + .. _cce_10_0141__fig7873421145213: - .. figure:: /_static/images/en-us_image_0181616313.png + .. figure:: /_static/images/en-us_image_0000001531373685.png :alt: **Figure 2** Driver information **Figure 2** Driver information 6. Obtain the driver link in either of the following ways: - - Method 1: As shown in :ref:`Figure 3 `, find *url=/tesla/396.37/NVIDIA-Linux-x86_64-396.37.run* in the browser address box. Then, supplement it to obtain the driver link https://us.download.nvidia.com/tesla/396.37/NVIDIA-Linux-x86_64-396.37.run. By using this method, you must bind an EIP to each GPU node. + - Method 1: As shown in :ref:`Figure 3 `, find *url=/tesla/470.103.01/NVIDIA-Linux-x86_64-470.103.01.run* in the browser address box. Then, supplement it to obtain the driver link https://us.download.nvidia.com/tesla/470.103.01/NVIDIA-Linux-x86_64-470.103.01.run. By using this method, you must bind an EIP to each GPU node. - - Method 2: As shown in :ref:`Figure 3 `, click **AGREE & DOWNLOAD** to download the driver. Then, upload the driver to OBS and record the OBS URL. By using this method, you do not need to bind an EIP to GPU nodes. + - Method 2: As shown in :ref:`Figure 3 `, click **AGREE & DOWNLOAD** to download the driver. Then, upload the driver to OBS and record the OBS URL. By using this method, you do not need to bind an EIP to GPU nodes. - .. _cce_01_0141__fig5901194614534: + .. _cce_10_0141__fig5901194614534: - .. figure:: /_static/images/en-us_image_0181616314.png + .. figure:: /_static/images/en-us_image_0000001531533045.png :alt: **Figure 3** Obtaining the link **Figure 3** Obtaining the link -Uninstalling the Add-on ------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, select the cluster and click **Uninstall** under **gpu-beta**. -#. In the dialog box displayed, click **Yes** to uninstall the add-on. - - .. note:: - - The driver will not be uninstalled during gpu-beta add-on uninstall. If the driver is reinstalled, you must restart all GPU nodes. - .. |image1| image:: /_static/images/en-us_image_0000001238225460.png diff --git a/umn/source/add-ons/index.rst b/umn/source/add-ons/index.rst index ef6de48..e622c8b 100644 --- a/umn/source/add-ons/index.rst +++ b/umn/source/add-ons/index.rst @@ -1,17 +1,19 @@ -:original_name: cce_01_0064.html +:original_name: cce_10_0064.html -.. _cce_01_0064: +.. _cce_10_0064: Add-ons ======= -- :ref:`Overview ` -- :ref:`coredns (System Resource Add-on, Mandatory) ` -- :ref:`storage-driver (System Resource Add-on, Mandatory) ` -- :ref:`everest (System Resource Add-on, Mandatory) ` -- :ref:`autoscaler ` -- :ref:`metrics-server ` -- :ref:`gpu-beta ` +- :ref:`Overview ` +- :ref:`coredns (System Resource Add-On, Mandatory) ` +- :ref:`storage-driver (System Resource Add-On, Discarded) ` +- :ref:`everest (System Resource Add-On, Mandatory) ` +- :ref:`npd ` +- :ref:`autoscaler ` +- :ref:`metrics-server ` +- :ref:`gpu-beta ` +- :ref:`volcano ` .. toctree:: :maxdepth: 1 @@ -19,8 +21,10 @@ Add-ons overview coredns_system_resource_add-on_mandatory - storage-driver_system_resource_add-on_mandatory + storage-driver_system_resource_add-on_discarded everest_system_resource_add-on_mandatory + npd autoscaler metrics-server gpu-beta + volcano diff --git a/umn/source/add-ons/metrics-server.rst b/umn/source/add-ons/metrics-server.rst index e7dfabb..e3af12a 100644 --- a/umn/source/add-ons/metrics-server.rst +++ b/umn/source/add-ons/metrics-server.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0205.html +:original_name: cce_10_0205.html -.. _cce_01_0205: +.. _cce_10_0205: metrics-server ============== @@ -9,43 +9,12 @@ From version 1.8 onwards, Kubernetes provides resource usage metrics, such as th metrics-server is an aggregator for monitoring data of core cluster resources. You can quickly install this add-on on the CCE console. -After metrics-server is installed, you can create an HPA policy on the **Workload Scaling** tab page of the **Auto Scaling** page. For details, see :ref:`Creating an HPA Policy for Workload Auto Scaling `. +After metrics-server is installed, you can create an HPA policy on the **Workload Scaling** tab page of the **Auto Scaling** page. For details, see :ref:`Creating an HPA Policy for Workload Auto Scaling `. The official community project and documentation are available at https://github.com/kubernetes-sigs/metrics-server. -Notes and Constraints ---------------------- - -This add-on can be installed only in CCE clusters of v1.13 or later. - -.. _cce_01_0205__section1962241123816: - Installing the Add-on --------------------- -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Marketplace** tab page, click **Install Add-on** under **metrics-server**. - -#. On the **Install Add-on** page, select the cluster and the add-on version, and click **Next: Configuration**. - +#. Log in to the CCE console and access the cluster console. Choose **Add-ons** in the navigation pane, locate **metrics-server** on the right, and click **Install**. #. Select **Single** or **HA** for **Add-on Specifications**, and click **Install**. - - After the add-on is installed, click **Go Back to Previous Page**. On the **Add-on Instance** tab page, select the corresponding cluster to view the running instance. This indicates that the add-on has been installed on each node in the cluster. - -Upgrading the Add-on --------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, click **Upgrade** under **metrics-server**. - - .. note:: - - - If the **Upgrade** button is not available, the current add-on is already up-to-date and no upgrade is required. - - During the upgrade, the metrics-server add-on of the original version on cluster nodes will be discarded, and the add-on of the target version will be installed. - -#. On the **Basic Information** page, select the add-on version and click **Next**. -#. Set the parameters by referring to the parameter description in :ref:`Installing the Add-on ` and click **Upgrade**. - -Uninstalling the Add-on ------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, click **Uninstall** under **metrics-server**. -#. In the dialog box displayed, click **Yes** to uninstall the add-on. diff --git a/umn/source/add-ons/npd.rst b/umn/source/add-ons/npd.rst new file mode 100644 index 0000000..10d233e --- /dev/null +++ b/umn/source/add-ons/npd.rst @@ -0,0 +1,384 @@ +:original_name: cce_10_0132.html + +.. _cce_10_0132: + +npd +=== + +Introduction +------------ + +node-problem-detector (npd for short) is an add-on that monitors abnormal events of cluster nodes and connects to a third-party monitoring platform. It is a daemon running on each node. It collects node issues from different daemons and reports them to the API server. The npd add-on can run as a DaemonSet or a daemon. + +For more information, see `node-problem-detector `__. + +Notes and Constraints +--------------------- + +- When using this add-on, do not format or partition node disks. +- Each npd process occupies 30 mCPU and 100 MB memory. + +Permission Description +---------------------- + +To monitor kernel logs, the npd add-on needs to read the host **/dev/kmsg**. Therefore, the privileged mode must be enabled. For details, see `privileged `__. + +In addition, CCE mitigates risks according to the least privilege principle. Only the following privileges are available for npd running: + +- cap_dac_read_search: permission to access **/run/log/journal**. +- cap_sys_admin: permission to access **/dev/kmsg**. + +Installing the Add-on +--------------------- + +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Add-ons** in the navigation pane, locate **npd** on the right, and click **Install**. + +#. On the **Install Add-on** page, select the add-on specifications and set related parameters. + + - **Pods**: Set the number of pods based on service requirements. + - **Containers**: Select a proper container quota based on service requirements. + +#. Set the parameters according to the following table and click **Install**. + + Only 1.16.0 and later versions support the configurations. + + **npc.enable**: indicates whether to enable :ref:`Node-problem-controller `. + +npd Check Items +--------------- + +.. note:: + + Check items are supported only in 1.16.0 and later versions. + +Check items cover events and statuses. + +- Event-related + + For event-related check items, when a problem occurs, npd reports an event to the API server. The event type can be **Normal** (normal event) or **Warning** (abnormal event). + + .. table:: **Table 1** Event-related check items + + +-----------------------+-------------------------------------------------------+-----------------------+ + | Check Item | Function | Description | + +=======================+=======================================================+=======================+ + | OOMKilling | Check whether OOM events occur and are reported. | Warning event | + +-----------------------+-------------------------------------------------------+-----------------------+ + | TaskHung | Check whether taskHung events occur and are reported. | Warning event | + +-----------------------+-------------------------------------------------------+-----------------------+ + | KernelOops | Check kernel nil pointer panic errors. | Warning event | + +-----------------------+-------------------------------------------------------+-----------------------+ + | ConntrackFull | Check whether the conntrack table is full. | Warning event | + | | | | + | | | Interval: 30 seconds | + | | | | + | | | Threshold: 80% | + +-----------------------+-------------------------------------------------------+-----------------------+ + +- Status-related + + For status-related check items, when a problem occurs, npd reports an event to the API server and changes the node status synchronously. This function can be used together with :ref:`Node-problem-controller fault isolation ` to isolate nodes. + + **If the check period is not specified in the following check items, the default period is 30 seconds.** + + .. table:: **Table 2** Application and OS check items + + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Check Item | Function | Description | + +===========================+===============================================================================================================================================================+============================================================================================================================================================+ + | FrequentKubeletRestart | Check whether kubelet restarts frequently by listening to journald logs. | - Interval: 5 minutes | + | | | | + | | | - Backtracking: 10 minutes | + | | | | + | | | - Threshold: 10 times | + | | | | + | | | If the system restarts for 10 times within the backtracking period, it indicates that the system restarts frequently and a fault alarm is generated. | + | | | | + | | | - Listening object: logs in the **/run/log/journal** directory | + | | | | + | | | .. note:: | + | | | | + | | | The Ubuntu OS does not support the preceding check items due to incompatible log formats. | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | FrequentDockerRestart | Check whether Docker restarts frequently by listening to journald logs. | | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | FrequentContainerdRestart | Check whether containerd restarts frequently by listening to journald logs. | | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | CRIProblem | Check the CRI component status. | Check object: Docker or containerd | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | KUBELETProblem | Check the kubelet status. | None | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | NTPProblem | Check the NTP and Chrony service status. | Threshold of the clock offset: 8000 ms | + | | | | + | | Check whether the node clock offsets. | | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | PIDProblem | Check whether PIDs are sufficient. | - Threshold: 90% | + | | | - Usage: nr_threads in /proc/loadavg | + | | | - Maximum value: smaller value between **/proc/sys/kernel/pid_max** and **/proc/sys/kernel/threads-max**. | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | FDProblem | Check whether file handles are sufficient. | - Threshold: 90% | + | | | - Usage: the first value in **/proc/sys/fs/file-nr** | + | | | - Maximum value: the third value in **/proc/sys/fs/file-nr** | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | MemoryProblem | Check whether the overall node memory is sufficient. | - Threshold: 90% | + | | | - Usage: **MemTotal-MemAvailable** in **/proc/meminfo** | + | | | - Maximum value: **MemTotal** in **/proc/meminfo** | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ResolvConfFileProblem | Check whether the ResolvConf file is lost. | Object: **/etc/resolv.conf** | + | | | | + | | Check whether the ResolvConf file is normal. | | + | | | | + | | Exception definition: No upstream domain name resolution server (nameserver) is included. | | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ProcessD | Check whether there is a process D on the node. | Source: | + | | | | + | | | - /proc/{PID}/stat | + | | | - Alternately, you can run **ps aux**. | + | | | | + | | | Exception scenario: ProcessD ignores the resident processes (heartbeat and update) that are in the D state that the SDI driver on the BMS node depends on. | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ProcessZ | Check whether the node has processes in Z state. | | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ScheduledEvent | Check whether host plan events exist on the node. | Source: | + | | | | + | | Typical scenario: The host is faulty, for example, the fan is damaged or the disk has bad sectors. As a result, cold and live migration is triggered for VMs. | - http://169.254.169.254/meta-data/latest/events/scheduled | + | | | | + | | | This check item is an Alpha feature and is disabled by default. | + +---------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + .. table:: **Table 3** Network connection check items + + +------------------+------------------------------------------------------+-------------+ + | Check Item | Function | Description | + +==================+======================================================+=============+ + | CNIProblem | Check whether the CNI component is running properly. | None | + +------------------+------------------------------------------------------+-------------+ + | KUBEPROXYProblem | Check whether kube-proxy is running properly. | None | + +------------------+------------------------------------------------------+-------------+ + + .. table:: **Table 4** Storage check items + + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Check Item | Function | Description | + +================================+====================================================================================================================================================================================================================================================================================================================================================================================================+====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ + | ReadonlyFilesystem | Check whether the **Remount root filesystem read-only** error occurs in the system kernel by listening to the kernel logs. | Listening object: **/dev/kmsg** | + | | | | + | | Typical scenario: A user detaches a data disk from a node by mistake on the ECS, and applications continuously write data to the mount point of the data disk. As a result, an I/O error occurs in the kernel and the disk is reattached as a read-only disk. | Matching rule: **Remounting filesystem read-only** | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | DiskReadonly | Check whether the system disk, Docker disk, and kubelet disk are read-only. | Detection paths: | + | | | | + | | | - /mnt/paas/kubernetes/kubelet/ | + | | | - /var/lib/docker/ | + | | | - /var/lib/containerd/ | + | | | - /var/paas/sys/log/cceaddon-npd/ | + | | | | + | | | The temporary file **npd-disk-write-ping** is generated in the detection path. | + | | | | + | | | Currently, additional data disks are not supported. | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | DiskProblem | Check the usage of the system disk, Docker disk, and kubelet disk. | - Threshold: 80% | + | | | | + | | | - Source: | + | | | | + | | | .. code-block:: | + | | | | + | | | df -h | + | | | | + | | | Currently, additional data disks are not supported. | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | EmptyDirVolumeGroupStatusError | Check whether the ephemeral volume group on the node is normal. | - Detection period: 60s | + | | | | + | | Impact: The pod that depends on the storage pool cannot write data to the temporary volume. The temporary volume is remounted as a read-only file system by the kernel due to an I/O error. | - Source: | + | | | | + | | Typical scenario: When creating a node, a user configures two data disks as a temporary volume storage pool. The user deletes some data disks by mistake. As a result, the storage pool becomes abnormal. | .. code-block:: | + | | | | + | | | vgs -o vg_name, vg_attr | + | | | | + | | | - Principle: Check whether the VG (storage pool) is in the P state. If yes, some PVs (data disks) are lost. | + | | | | + | | | - Joint scheduling: The scheduler can automatically identify an abnormal node and prevent pods that depend on the storage pool from being scheduled to the node. | + | | | | + | | | - Exception scenario: The npd add-on cannot detect the loss of all PVs (data disks), resulting in the loss of VGs (storage pools). In this case, kubelet automatically isolates the node, detects the loss of VGs (storage pools), and updates the corresponding resources in **nodestatus.allocatable** to **0**. This prevents pods that depend on the storage pool from being scheduled to the node. The damage of a single PV cannot be detected. In this case, the ReadonlyFilesystem detection is abnormal. | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | LocalPvVolumeGroupStatusError | Check the PV group on the node. | | + | | | | + | | Impact: Pods that depend on the storage pool cannot write data to the persistent volume. The persistent volume is remounted as a read-only file system by the kernel due to an I/O error. | | + | | | | + | | Typical scenario: When creating a node, a user configures two data disks as a persistent volume storage pool. Some data disks are deleted by mistake. | | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | MountPointProblem | Check the mount point on the node. | Alternatively, you can run the following command: | + | | | | + | | Exception definition: You cannot access the mount point by running the **cd** command. | .. code-block:: | + | | | | + | | Typical scenario: Network File System (NFS), for example, obsfs and s3fs is mounted to a node. When the connection is abnormal due to network or peer NFS server exceptions, all processes that access the mount point are suspended. For example, during a cluster upgrade, a kubelet is restarted, and all mount points are scanned. If the abnormal mount point is detected, the upgrade fails. | for dir in `df -h | grep -v "Mounted on" | awk "{print \\$NF}"`;do cd $dir; done && echo "ok" | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | DiskHung | Check whether I/O faults occur on the disk of the node. | - Check object: all data disks | + | | | | + | | Definition of I/O faults: The system does not respond to disk I/O requests, and some processes are in the D state. | - Source: | + | | | | + | | Typical Scenario: Disks cannot respond due to abnormal OS hard disk drivers or severe faults on the underlying network. | /proc/diskstat | + | | | | + | | | Alternatively, you can run the following command: | + | | | | + | | | .. code-block:: | + | | | | + | | | iostat -xmt 1 | + | | | | + | | | - Threshold: | + | | | | + | | | - Average usage. The value of ioutil is greater than or equal to 0.99. | + | | | - Average I/O queue length. avgqu-sz >=1 | + | | | - Average I/O transfer volume, iops (w/s) + ioth (wMB/s) < = 1 | + | | | | + | | | .. note:: | + | | | | + | | | In some OSs, no data changes during I/O. In this case, calculate the CPU I/O time usage. The value of iowait is greater than 0.8. | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | DiskSlow | Check whether slow I/O occurs on the disk of the node. | - Check object: all data disks | + | | | | + | | Definition of slow I/O: The average response time exceeds the threshold. | - Source: | + | | | | + | | Typical scenario: EVS disks have slow I/Os due to network fluctuation. | /proc/diskstat | + | | | | + | | | Alternatively, you can run the following command: | + | | | | + | | | .. code-block:: | + | | | | + | | | iostat -xmt 1 | + | | | | + | | | - Threshold: | + | | | | + | | | Average I/O latency: await > = 5000 ms | + | | | | + | | | .. note:: | + | | | | + | | | If I/O requests are not responded and the **await** data is not updated. In this case, this check item is invalid. | + +--------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + The kubelet component has the following default check items, which have bugs or defects. You can fix them by upgrading the cluster or using npd. + + .. table:: **Table 5** Default kubelet check items + + +-----------------------+------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Check Item | Function | Description | + +=======================+========================================================================+==========================================================================================================================================================================================================================================================================================================================+ + | PIDPressure | Check whether PIDs are sufficient. | - Interval: 10 seconds | + | | | - Threshold: 90% | + | | | - Defect: In community version 1.23.1 and earlier versions, this check item becomes invalid when over 65535 PIDs are used. For details, see `issue 107107 `__. In community version 1.24 and earlier versions, thread-max is not considered in this check item. | + +-----------------------+------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | MemoryPressure | Check whether the allocable memory for the containers is sufficient. | - Interval: 10 seconds | + | | | - Threshold: Max. 100 MiB | + | | | - Allocable = Total memory of a node - Reserved memory of a node | + | | | - Defect: This check item checks only the memory consumed by containers, and does not consider that consumed by other elements on the node. | + +-----------------------+------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | DiskPressure | Check the disk usage and inodes usage of the kubelet and Docker disks. | Interval: 10 seconds | + | | | | + | | | Threshold: 90% | + +-----------------------+------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +.. _cce_10_0132__section1471610580474: + +Node-problem-controller Fault Isolation +--------------------------------------- + +.. note:: + + Fault isolation is supported only by add-ons of 1.16.0 and later versions. + + When installing the npd add-on, set **npc.enable** to **true** to deploy dual Node-problem-controller (NPC). You can deploy NPC as single-instance but such NPC does not ensure high availability. + + By default, if multiple nodes become faulty, NPC adds taints to only one node. You can set **npc.maxTaintedNode** to increase the threshold. When the fault is rectified, NPC is not running and taints remain. You need to manually clear the taints or start NPC. + +The open source NPD plug-in provides fault detection but not fault isolation. CCE enhances the node-problem-controller (NPC) based on the open source NPD. This component is implemented based on the Kubernetes `node controller `__. For faults reported by NPD, NPC automatically adds taints to nodes for node fault isolation. + +You can modify **add-onnpc.customConditionToTaint** according to the following table to configure fault isolation rules. + +.. table:: **Table 6** Parameters + + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | Default | + +================================+=========================================================+=========================================================================================================================================+ + | npc.enable | Whether to enable NPC | true | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | npc.customCondtionToTaint | Fault isolation rules | See :ref:`Table 7 `. | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | npc.customConditionToTaint[i] | Fault isolation rule items | N/A | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | npc.customConditionToTaint[i]. | Fault status | true | + | | | | + | condition.status | | | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | npc.customConditionToTaint[i]. | Fault type | N/A | + | | | | + | condition.type | | | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | npc.customConditionToTaint[i]. | Whether to enable the fault isolation rule. | false | + | | | | + | enable | | | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | npc.customConditionToTaint[i]. | Fault isolation effect | NoSchedule | + | | | | + | .taint.effect | NoSchedule, PreferNoSchedule, or NoExecute | Value options: **NoSchedule**, **PreferNoSchedule**, and **NoExecute** | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | npc. maxTaintedNode | Number of nodes in a cluster that can be tainted by NPC | 1 | + | | | | + | | The int format and percentage format are supported. | Values: | + | | | | + | | | - The value is in int format and ranges from 1 to infinity. | + | | | - The value ranges from 1% to 100%, in percentage. The minimum value of this parameter multiplied by the number of cluster nodes is 1. | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + | Npc.affinity | Node affinity of the controller | N/A | + +--------------------------------+---------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------+ + +.. _cce_10_0132__table147438134911: + +.. table:: **Table 7** Fault isolation rule configuration + + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | Fault | Fault Details | Taint | + +===========================+=====================================================================+======================================+ + | DiskReadonly | Disk read-only | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | DiskProblem | The disk space is insufficient, and key logical disks are detached. | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | FrequentKubeletRestart | kubelet restarts frequently. | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | FrequentDockerRestart | Docker restarts frequently. | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | FrequentContainerdRestart | containerd restarts frequently. | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | KUBEPROXYProblem | kube-proxy is abnormal. | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | PIDProblem | Insufficient PIDs | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | FDProblem | Insufficient file handles | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + | MemoryProblem | Insufficient node memory | **NoSchedule**: No new pods allowed. | + +---------------------------+---------------------------------------------------------------------+--------------------------------------+ + +Collecting Prometheus Metrics +----------------------------- + +The NPD daemon pod exposes Prometheus metric data on port 19901. By default, the NPD pod is added with the annotation **metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"prometheus","path":"/metrics","port":"19901","names":""}]'**. You can build a Prometheus collector to identify and obtain NPD metrics from **http://{{NpdPodIP}}:{{NpdPodPort}}/metrics**. + +.. note:: + + If the npd add-on version is earlier than 1.16.5, the exposed port of Prometheus metrics is **20257**. + +Currently, the metric data includes **problem_counter** and **problem_gauge**, as shown below. + +.. code-block:: + + # HELP problem_counter Number of times a specific type of problem have occurred. + # TYPE problem_counter counter + problem_counter{reason="DockerHung"} 0 + problem_counter{reason="DockerStart"} 0 + problem_counter{reason="EmptyDirVolumeGroupStatusError"} 0 + ... + # HELP problem_gauge Whether a specific type of problem is affecting the node or not. + # TYPE problem_gauge gauge + problem_gauge{reason="CNIIsDown",type="CNIProblem"} 0 + problem_gauge{reason="CNIIsUp",type="CNIProblem"} 0 + problem_gauge{reason="CRIIsDown",type="CRIProblem"} 0 + problem_gauge{reason="CRIIsUp",type="CRIProblem"} 0 + .. diff --git a/umn/source/add-ons/overview.rst b/umn/source/add-ons/overview.rst index 2d92756..ce588ac 100644 --- a/umn/source/add-ons/overview.rst +++ b/umn/source/add-ons/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0277.html +:original_name: cce_10_0277.html -.. _cce_01_0277: +.. _cce_10_0277: Overview ======== @@ -9,18 +9,22 @@ CCE provides multiple types of add-ons to extend cluster functions and meet feat .. table:: **Table 1** Add-on list - +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add-on Name | Introduction | - +=====================================+==================================================================================================================================================================================+ - | :ref:`coredns ` | The coredns add-on is a DNS server that provides domain name resolution services for Kubernetes clusters. coredns chains plug-ins to provide additional features. | - +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`storage-driver ` | storage-driver is a FlexVolume driver used to support IaaS storage services such as EVS, SFS, and OBS. | - +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`everest ` | Everest is a cloud native container storage system. Based on CSI, clusters of Kubernetes v1.15.6 and later can connect to storage services such as EVS, OBS, SFS, and SFS Turbo. | - +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`autoscaler ` | The autoscaler add-on resizes a cluster based on pod scheduling status and resource usage. | - +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`metrics-server ` | metrics-server is an aggregator for monitoring data of core cluster resources. | - +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`gpu-beta ` | gpu-beta is a device management add-on that supports GPUs in containers. It supports only NVIDIA drivers. | - +-------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Add-on Name | Introduction | + +=========================================================================+=================================================================================================================================================================================================================================================================================================================================+ + | :ref:`coredns (System Resource Add-On, Mandatory) ` | The coredns add-on is a DNS server that provides domain name resolution services for Kubernetes clusters. coredns chains plug-ins to provide additional features. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`storage-driver (System Resource Add-On, Discarded) ` | storage-driver is a FlexVolume driver used to support IaaS storage services such as EVS, SFS, and OBS. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`everest (System Resource Add-On, Mandatory) ` | Everest is a cloud native container storage system. Based on the Container Storage Interface (CSI), clusters of Kubernetes v1.15.6 or later obtain access to cloud storage services. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`npd ` | node-problem-detector (npd for short) is an add-on that monitors abnormal events of cluster nodes and connects to a third-party monitoring platform. It is a daemon running on each node. It collects node issues from different daemons and reports them to the API server. The npd add-on can run as a DaemonSet or a daemon. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`autoscaler ` | The autoscaler add-on resizes a cluster based on pod scheduling status and resource usage. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`metrics-server ` | metrics-server is an aggregator for monitoring data of core cluster resources. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`gpu-beta ` | gpu-beta is a device management add-on that supports GPUs in containers. It supports only NVIDIA drivers. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`volcano ` | Volcano provides general-purpose, high-performance computing capabilities, such as job scheduling, heterogeneous chip management, and job running management, serving end users through computing frameworks for different industries, such as AI, big data, gene sequencing, and rendering. | + +-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/add-ons/storage-driver_system_resource_add-on_discarded.rst b/umn/source/add-ons/storage-driver_system_resource_add-on_discarded.rst new file mode 100644 index 0000000..5f212e1 --- /dev/null +++ b/umn/source/add-ons/storage-driver_system_resource_add-on_discarded.rst @@ -0,0 +1,34 @@ +:original_name: cce_10_0127.html + +.. _cce_10_0127: + +storage-driver (System Resource Add-On, Discarded) +================================================== + +Introduction +------------ + +storage-driver functions as a standard Kubernetes FlexVolume plug-in to allow containers to use EVS, SFS, OBS, and SFS Turbo storage resources. By installing and upgrading storage-driver, you can quickly install and update cloud storage capabilities. + +**storage-driver is a system resource add-on. It is installed by default when a cluster of Kubernetes v1.13 or earlier is created.** + +Notes and Constraints +--------------------- + +- For clusters created in CCE, Kubernetes v1.15.11 is a transitional version in which the FlexVolume plug-in (storage-driver) is compatible with the CSI plug-in (:ref:`everest `). Clusters of v1.17 and later versions do not support FlexVolume anymore. You need to use the everest add-on. +- The FlexVolume plug-in will be maintained by Kubernetes developers, but new functionality will only be added to CSI. You are advised not to create storage that connects to the FlexVolume plug-in (storage-driver) in CCE anymore. Otherwise, the storage resources may not function normally. +- This add-on can be installed only in **clusters of v1.13 or earlier**. By default, the :ref:`everest ` add-on is installed when clusters of v1.15 or later are created. + + .. note:: + + **In a cluster of v1.13 or earlier**, when an upgrade or bug fix is available for storage functionalities, you only need to install or upgrade the storage-driver add-on. Upgrading the cluster or creating a cluster is not required. + +Installing the Add-on +--------------------- + +This add-on has been installed by default. If it is uninstalled due to some reasons, you can reinstall it by performing the following steps: + +If storage-driver is not installed in a cluster, perform the following steps to install it: + +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Add-ons** in the navigation pane, locate **storage-driver** on the right, and click **Install**. +#. Click **Install** to install the add-on. Note that the storage-driver has no configurable parameters and can be directly installed. diff --git a/umn/source/add-ons/storage-driver_system_resource_add-on_mandatory.rst b/umn/source/add-ons/storage-driver_system_resource_add-on_mandatory.rst deleted file mode 100644 index d21b65b..0000000 --- a/umn/source/add-ons/storage-driver_system_resource_add-on_mandatory.rst +++ /dev/null @@ -1,58 +0,0 @@ -:original_name: cce_01_0127.html - -.. _cce_01_0127: - -storage-driver (System Resource Add-on, Mandatory) -================================================== - -Introduction ------------- - -storage-driver functions as a standard Kubernetes FlexVolume plug-in to allow containers to use IaaS storage resources. By installing and upgrading storage-driver, you can quickly install and update cloud storage capabilities. - -**storage-driver is a system resource add-on. It is installed by default when a cluster of Kubernetes v1.13 or earlier is created.** - -Notes and Constraints ---------------------- - -- For clusters created in CCE, Kubernetes v1.15.11 is a transitional version in which the FlexVolume plug-in (storage-driver) is compatible with the CSI plug-in (:ref:`everest `). Clusters of v1.17 and later versions do not support FlexVolume any more. You need to use the everest add-on. For details about CSI and FlexVolume, see :ref:`Differences Between CSI and FlexVolume Plug-ins `. -- The FlexVolume plug-in will be maintained by Kubernetes developers, but new functionality will only be added to CSI. You are advised not to create storage that connects to the FlexVolume plug-in (storage-driver) in CCE any more. Otherwise, the storage resources may not function normally. -- This add-on can be installed only in **clusters of v1.13 or earlier**. By default, the :ref:`everest ` add-on is installed when clusters of v1.15 or later are created. - - .. note:: - - **In a cluster of v1.13 or earlier**, when an upgrade or bug fix is available for storage functionalities, you only need to install or upgrade the storage-driver add-on. Upgrading the cluster or creating a cluster is not required. - -Installing the Add-on ---------------------- - -This add-on has been installed by default. If it is uninstalled due to some reasons, you can reinstall it by performing the following steps: - -If storage-driver is not installed in a cluster, perform the following steps to install it: - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Marketplace** tab page, click **Install Add-on** under **storage-driver**. - -#. On the **Install Add-on** page, select the cluster and the add-on version, and click **Next: Configuration**. - -#. Click **Install** to install the add-on. Note that the storage-driver has no configurable parameters and can be directly installed. - - After the add-on is installed, click **Go Back to Previous Page**. On the **Add-on Instance** tab page, select the corresponding cluster to view the running instance. This indicates that the add-on has been installed on each node in the cluster. - -Upgrading the Add-on --------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, select the target cluster and click **Upgrade** under **storage-driver**. - - .. note:: - - - If the **Upgrade** button is unavailable, the current add-on is already up-to-date and no upgrade is required. - - When the upgrade is complete, the original storage-driver version on cluster nodes will be replaced by the latest version. - -#. On the **Basic Information** page, select the add-on version and click **Next**. -#. Click **Upgrade** to upgrade the storage-driver add-on. Note that the storage-driver has no configurable parameters and can be directly upgraded. - -Uninstalling the Add-on ------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Add-ons**. On the **Add-on Instance** tab page, select the target cluster and click **Uninstall** under **storage-driver**. -#. In the dialog box displayed, click **Yes** to uninstall the add-on. diff --git a/umn/source/add-ons/volcano.rst b/umn/source/add-ons/volcano.rst new file mode 100644 index 0000000..34772c3 --- /dev/null +++ b/umn/source/add-ons/volcano.rst @@ -0,0 +1,517 @@ +:original_name: cce_10_0193.html + +.. _cce_10_0193: + +volcano +======= + +Introduction +------------ + +Volcano is a batch processing platform based on Kubernetes. It provides a series of features required by machine learning, deep learning, bioinformatics, genomics, and other big data applications, as a powerful supplement to Kubernetes capabilities. + +Volcano provides general-purpose, high-performance computing capabilities, such as job scheduling engine, heterogeneous chip management, and job running management, serving end users through computing frameworks for different industries, such as AI, big data, gene sequencing, and rendering. (Volcano has been open-sourced in GitHub.) + +Volcano provides job scheduling, job management, and queue management for computing applications. Its main features are as follows: + +- Diverse computing frameworks, such as TensorFlow, MPI, and Spark, can run on Kubernetes in containers. Common APIs for batch computing jobs through CRD, various plug-ins, and advanced job lifecycle management are provided. +- Advanced scheduling capabilities are provided for batch computing and high-performance computing scenarios, including group scheduling, preemptive priority scheduling, packing, resource reservation, and task topology. +- Queues can be effectively managed for scheduling jobs. Complex job scheduling capabilities such as queue priority and multi-level queues are supported. + +Open source community: https://github.com/volcano-sh/volcano + +Installing the Add-on +--------------------- + +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Add-ons** in the navigation pane, locate **volcano** on the right, and click **Install**. + +#. Select **Standalone**, **Custom**, or **HA** for **Add-on Specifications**. + + If you select **Custom**, the recommended values of **volcano-controller** and **volcano-scheduler** are as follows: + + - If the number of nodes is less than 100, retain the default configuration. That is, the CPU request value is **500m**, and the limit value is **2000m**. The memory request value is **500Mi**, and the limit value is **2000Mi**. + - If the number of nodes is greater than 100, increase the CPU request value by **500m** and the memory request value by **1000Mi** each time 100 nodes (10000 pods) are added. You are advised to increase the CPU limit value by **1500m** and the memory limit by **1000Mi**. + + .. table:: **Table 1** Recommended values for volcano-controller and volcano-scheduler + + +--------------------+----------------+--------------+--------------------+------------------+ + | Number of Node/Pod | CPU Request(m) | CPU Limit(m) | Memory Request(Mi) | Memory Limit(Mi) | + +====================+================+==============+====================+==================+ + | 50/5k | 500 | 2000 | 500 | 2000 | + +--------------------+----------------+--------------+--------------------+------------------+ + | 100/1w | 1000 | 2500 | 1500 | 2500 | + +--------------------+----------------+--------------+--------------------+------------------+ + | 200/2w | 1500 | 3000 | 2500 | 3500 | + +--------------------+----------------+--------------+--------------------+------------------+ + | 300/3w | 2000 | 3500 | 3500 | 4500 | + +--------------------+----------------+--------------+--------------------+------------------+ + | 400/4w | 2500 | 4000 | 4500 | 5500 | + +--------------------+----------------+--------------+--------------------+------------------+ + +#. Parameters of the volcano default scheduler. For details, see :ref:`Table 2 `. + + .. code-block:: + + ca_cert: '' + default_scheduler_conf: + actions: 'allocate, backfill' + tiers: + - plugins: + - name: 'priority' + - name: 'gang' + - name: 'conformance' + - plugins: + - name: 'drf' + - name: 'predicates' + - name: 'nodeorder' + - plugins: + - name: 'cce-gpu-topology-predicate' + - name: 'cce-gpu-topology-priority' + - name: 'cce-gpu' + - plugins: + - name: 'nodelocalvolume' + - name: 'nodeemptydirvolume' + - name: 'nodeCSIscheduling' + - name: 'networkresource' + server_cert: '' + server_key: '' + + .. _cce_10_0193__table562185146: + + .. table:: **Table 2** Volcano Plugins + + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | Add-on | Function | Description | Demonstration | + +============================+=============================================================================================================================================================================================================================+==========================================================================================================================+=============================================================+ + | binpack | The binpack plugin schedules pods to nodes with high resource utilization to reduce resource fragments. | - **binpack.weight**: Weight of the binpack plugin. | .. code-block:: | + | | | - **binpack.cpu**: Ratio of CPU resources to all resources. Defaults to **1**. | | + | | | - **binpack.memory**: Ratio of memory resources to all resources. Defaults to **1**. | - plugins: | + | | | - binpack.resources: | - name: binpack | + | | | | arguments: | + | | | | binpack.weight: 10 | + | | | | binpack.cpu: 1 | + | | | | binpack.memory: 1 | + | | | | binpack.resources: nvidia.com/gpu, example.com/foo | + | | | | binpack.resources.nvidia.com/gpu: 2 | + | | | | binpack.resources.example.com/foo: 3 | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | conformance | The conformance plugin considers that the tasks in namespace **kube-system** have a higher priority. These tasks will not be preempted. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | gang | The gang plugin considers a group of pods as a whole to allocate resources. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | priority | The priority plugin schedules pods based on the custom workload priority. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | overcommit | Resources in a cluster are scheduled after being accumulated in a certain multiple to improve the workload enqueuing efficiency. If all workloads are Deployments, remove this plugin or set the raising factor to **2.0**. | **overcommit-factor**: Raising factor. Defaults to **1.2**. | .. code-block:: | + | | | | | + | | | | - plugins: | + | | | | - name: overcommit | + | | | | arguments: | + | | | | overcommit-factor: 2.0 | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | drf | The DRF plugin schedules resources based on the container group Domaint Resource. The smallest Domaint Resource would be selected for priority scheduling. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | predicates | Determines whether a task is bound to a node by using a series of evaluation algorithms, such as node/pod affinity, taint tolerance, node port repetition, volume limits, and volume zone matching. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | nodeorder | The nodeorder plugin scores all nodes for a task by using a series of scoring algorithms. | - **nodeaffinity.weight**: Pods are scheduled based on the node affinity. Defaults to **1**. | .. code-block:: | + | | | - **podaffinity.weight**: Pods are scheduled based on the pod affinity. Defaults to **1**. | | + | | | - **leastrequested.weight**: Pods are scheduled to the node with the least resources. Defaults to **1**. | - plugins: | + | | | - **balancedresource.weight**: Pods are scheduled to the node with balanced resource. Defaults to **1**. | - name: nodeorder | + | | | - **mostrequested.weight**: Pods are scheduled to the node with the most requested resources. Defaults to **0**. | arguments: | + | | | - **tainttoleration.weight**: Pods are scheduled to the node with a high taint tolerance. Defaults to **1**. | leastrequested.weight: 1 | + | | | - **imagelocality.weight**: Pods are scheduled to the node where the required images exist. Defaults to **1**. | mostrequested.weight: 0 | + | | | - **selectorspread.weight**: Pods are evenly scheduled to different nodes. Defaults to **0**. | nodeaffinity.weight: 1 | + | | | - **volumebinding.weight**: Pods are scheduled to the node with the local PV delayed binding policy. Defaults to **1**. | podaffinity.weight: 1 | + | | | - **podtopologyspread.weight**: Pods are scheduled based on the pod topology. Defaults to **2**. | balancedresource.weight: 1 | + | | | | tainttoleration.weight: 1 | + | | | | imagelocality.weight: 1 | + | | | | volumebinding.weight: 1 | + | | | | podtopologyspread.weight: 2 | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | cce-gpu-topology-predicate | GPU-topology scheduling preselection algorithm | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | cce-gpu-topology-priority | GPU-topology scheduling priority algorithm | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | cce-gpu | Works with the gpu add-on of CCE to support GPU resource allocation and decimal GPU configuration. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | numaaware | NUMA topology scheduling | weight: Weight of the numa-aware plugin. | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | networkresource | The ENI requirement node can be preselected and filtered. The parameters are transferred by CCE and do not need to be manually configured. | NetworkType: Network type (eni or vpc-router). | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | nodelocalvolume | The nodelocalvolume plugin filters out nodes that do not meet local volume requirements can be filtered out. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | nodeemptydirvolume | The nodeemptydirvolume plugin filters out nodes that do not meet the emptyDir requirements. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + | nodeCSIscheduling | The nodeCSIscheduling plugin filters out nodes that have the everest component exception. | ``-`` | ``-`` | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------+ + +#. Click **Install**. + +Modifying the volcano-scheduler Configuration Using the Console +--------------------------------------------------------------- + +Volcano allows you to configure the scheduler during installation, upgrade, and editing. The configuration will be synchronized to volcano-scheduler-configmap. + +This section describes how to configure the volcano scheduler. + +.. note:: + + Only Volcano of v1.7.1 and later support this function. On the new plug-in page, options such as **plugins.eas_service** and **resource_exporter_enable** are replaced by **default_scheduler_conf**. + +Log in to the CCE console and access the cluster console. Choose **Add-ons** in the navigation pane. On the right of the page, locate **volcano** and click **Install** or **Upgrade**. In the **Parameters** area, configure the volcano scheduler parameters. + +- Using **resource_exporter**: + + .. code-block:: + + { + "ca_cert": "", + "default_scheduler_conf": { + "actions": "allocate, backfill", + "tiers": [ + { + "plugins": [ + { + "name": "priority" + }, + { + "name": "gang" + }, + { + "name": "conformance" + } + ] + }, + { + "plugins": [ + { + "name": "drf" + }, + { + "name": "predicates" + }, + { + "name": "nodeorder" + } + ] + }, + { + "plugins": [ + { + "name": "cce-gpu-topology-predicate" + }, + { + "name": "cce-gpu-topology-priority" + }, + { + "name": "cce-gpu" + }, + { + "name": "numa-aware" # add this also enable resource_exporter + } + ] + }, + { + "plugins": [ + { + "name": "nodelocalvolume" + }, + { + "name": "nodeemptydirvolume" + }, + { + "name": "nodeCSIscheduling" + }, + { + "name": "networkresource" + } + ] + } + ] + }, + "server_cert": "", + "server_key": "" + } + + After this function is enabled, you can use the functions of the numa-aware plug-in and resource_exporter at the same time. + +- Using **eas_service**: + + .. code-block:: + + { + "ca_cert": "", + "default_scheduler_conf": { + "actions": "allocate, backfill", + "tiers": [ + { + "plugins": [ + { + "name": "priority" + }, + { + "name": "gang" + }, + { + "name": "conformance" + } + ] + }, + { + "plugins": [ + { + "name": "drf" + }, + { + "name": "predicates" + }, + { + "name": "nodeorder" + } + ] + }, + { + "plugins": [ + { + "name": "cce-gpu-topology-predicate" + }, + { + "name": "cce-gpu-topology-priority" + }, + { + "name": "cce-gpu" + }, + { + "name": "eas", + "custom": { + "availability_zone_id": "", + "driver_id": "", + "endpoint": "", + "flavor_id": "", + "network_type": "", + "network_virtual_subnet_id": "", + "pool_id": "", + "project_id": "", + "secret_name": "eas-service-secret" + } + } + ] + }, + { + "plugins": [ + { + "name": "nodelocalvolume" + }, + { + "name": "nodeemptydirvolume" + }, + { + "name": "nodeCSIscheduling" + }, + { + "name": "networkresource" + } + ] + } + ] + }, + "server_cert": "", + "server_key": "" + } + +- Using **ief**: + + .. code-block:: + + { + "ca_cert": "", + "default_scheduler_conf": { + "actions": "allocate, backfill", + "tiers": [ + { + "plugins": [ + { + "name": "priority" + }, + { + "name": "gang" + }, + { + "name": "conformance" + } + ] + }, + { + "plugins": [ + { + "name": "drf" + }, + { + "name": "predicates" + }, + { + "name": "nodeorder" + } + ] + }, + { + "plugins": [ + { + "name": "cce-gpu-topology-predicate" + }, + { + "name": "cce-gpu-topology-priority" + }, + { + "name": "cce-gpu" + }, + { + "name": "ief", + "enableBestNode": true + } + ] + }, + { + "plugins": [ + { + "name": "nodelocalvolume" + }, + { + "name": "nodeemptydirvolume" + }, + { + "name": "nodeCSIscheduling" + }, + { + "name": "networkresource" + } + ] + } + ] + }, + "server_cert": "", + "server_key": "" + } + +Retaining the Original volcano-scheduler-configmap Configuration +---------------------------------------------------------------- + +If you want to use the original configuration after the plug-in is upgraded, perform the following steps: + +#. Check and back up the original volcano-scheduler-configmap configuration. + + Example: + + .. code-block:: + + # kubectl edit cm volcano-scheduler-configmap -n kube-system + apiVersion: v1 + data: + default-scheduler.conf: |- + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: gang + - name: conformance + - plugins: + - name: drf + - name: predicates + - name: nodeorder + - name: binpack + arguments: + binpack.cpu: 100 + binpack.weight: 10 + binpack.resources: nvidia.com/gpu + binpack.resources.nvidia.com/gpu: 10000 + - plugins: + - name: cce-gpu-topology-predicate + - name: cce-gpu-topology-priority + - name: cce-gpu + - plugins: + - name: nodelocalvolume + - name: nodeemptydirvolume + - name: nodeCSIscheduling + - name: networkresource + +#. Enter the customized content in the **Parameters** on the console. + + .. code-block:: + + { + "ca_cert": "", + "default_scheduler_conf": { + "actions": "enqueue, allocate, backfill", + "tiers": [ + { + "plugins": [ + { + "name": "priority" + }, + { + "name": "gang" + }, + { + "name": "conformance" + } + ] + }, + { + "plugins": [ + { + "name": "drf" + }, + { + "name": "predicates" + }, + { + "name": "nodeorder" + }, + { + "name": "binpack", + "arguments": { + "binpack.cpu": 100, + "binpack.weight": 10, + "binpack.resources": "nvidia.com/gpu", + "binpack.resources.nvidia.com/gpu": 10000 + } + } + ] + }, + { + "plugins": [ + { + "name": "cce-gpu-topology-predicate" + }, + { + "name": "cce-gpu-topology-priority" + }, + { + "name": "cce-gpu" + } + ] + }, + { + "plugins": [ + { + "name": "nodelocalvolume" + }, + { + "name": "nodeemptydirvolume" + }, + { + "name": "nodeCSIscheduling" + }, + { + "name": "networkresource" + } + ] + } + ] + }, + "server_cert": "", + "server_key": "" + } + + .. note:: + + When this function is used, the original content in volcano-scheduler-configmap will be overwritten. Therefore, you must check whether volcano-scheduler-configmap has been modified during the upgrade. If yes, synchronize the modification to the upgrade page. diff --git a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/index.rst b/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/index.rst deleted file mode 100644 index 6ef0345..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -:original_name: cce_01_0231.html - -.. _cce_01_0231: - -Custom Scheduling Policies -========================== - -- :ref:`Node Affinity ` -- :ref:`Workload Affinity ` -- :ref:`Workload Anti-Affinity ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - node_affinity - workload_affinity - workload_anti-affinity diff --git a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/node_affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/node_affinity.rst deleted file mode 100644 index c122712..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/node_affinity.rst +++ /dev/null @@ -1,102 +0,0 @@ -:original_name: cce_01_0232.html - -.. _cce_01_0232: - -Node Affinity -============= - -Using the CCE Console ---------------------- - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. - -#. Click a workload name in the Deployment or StatefulSet list. On the displayed workload details page, click the **Scheduling Policies** tab and then click **Add Custom Scheduling Policy**. - -#. In the **Node Affinity** area, you can specify node labels to meet required or preferred rules in scheduling. - - .. table:: **Table 1** Node affinity settings - - +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===========+======================================================================================================================================================================================================================================================================================================================================================+ - | Required | It specifies a rule that must be met in scheduling. It corresponds to **requiredDuringSchedulingIgnoredDuringExecution** in Kubernetes. You can click **Add Rule** to add multiple required rules. A pod will be scheduled on a node that meets any of the rules configured. | - +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Preferred | It specifies a preference in scheduling. It corresponds to **preferredDuringSchedulingIgnoredDuringExecution** in Kubernetes. You can click **Add Rule** to add multiple preferred rules. The scheduler will try to enforce the rules but will not guarantee. If the scheduler cannot satisfy any one of the rules, the pod will still be scheduled. | - +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Set a rule according to the following table. You can click **Add Selector** to configure multiple selectors for a rule. - - .. table:: **Table 2** Selector settings - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================================================================================+ - | Weight | - This parameter is unavailable for a required rule. | - | | - Set the weight of a preferred rule. A higher weight indicates a higher priority. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Label | Node label. You can use the default label or customize a label. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Operator | The following relations are supported: **In**, **NotIn**, **Exists**, **DoesNotExist**, **Gt**, and **Lt** | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Value | Tag value. | - | | | - | | Operators **In** and **NotIn** allow one or more label values. Values are separated with colons (;). Operators **Exists** and **DoesNotExist** are used to determine whether a label exists, and do not require a label value. If you set the operator to **Gt** or **Lt** for a label, the label value must be greater than or less than a certain integer. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Operation | You can click **Delete** to delete a selector. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Selector | A selector corresponds to **matchExpressions** in Kubernetes. You can click **Add Selector** to add multiple selectors for a scheduling rule. The rule is applied in scheduling only when all its selectors are satisfied. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - - .. figure:: /_static/images/en-us_image_0000001190658439.png - :alt: **Figure 1** Node affinity scheduling policy - - **Figure 1** Node affinity scheduling policy - -Using kubectl -------------- - -This section uses Nginx as an example to describe how to configure node affinity. - -**Prerequisites** - -A workload that uses the nginx container image has been deployed on a node. - -**Procedure** - -Set **Label** to **kubernetes.io/hostname**, add affinity nodes, and set the operator to **In**. Then, click **OK**. - -YAML file of the workload with node affinity: - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - namespace: default - spec: - replicas: 2 - selector: - matchLabels: - app: nginx - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: kubernetes.io/hostname - operator: In - values: - - 192.168.6.174 diff --git a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/workload_affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/workload_affinity.rst deleted file mode 100644 index 3b5b5b6..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/workload_affinity.rst +++ /dev/null @@ -1,122 +0,0 @@ -:original_name: cce_01_0233.html - -.. _cce_01_0233: - -Workload Affinity -================= - -Using the CCE Console ---------------------- - -Workload affinity determines the pods as which the target workload will be deployed in the same topology domain. - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. - -#. Click a workload name in the Deployment or StatefulSet list. On the displayed workload details page, click the **Scheduling Policies** tab and then click **Add Custom Scheduling Policy**. - -#. In the **Pod Affinity** area, set the namespace, topology key, and the label requirements to be met. - - There are two types of pod affinity rules: **Required** (hard rule) and **Preferred** (soft rule). The label operators include **In**, **NotIn**, **Exists**, and **DoesNotExist**. - - .. table:: **Table 1** Pod affinity settings - - +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===========+======================================================================================================================================================================================================================================================================================================================================================+ - | Required | It specifies a rule that must be met in scheduling. It corresponds to **requiredDuringSchedulingIgnoredDuringExecution** in Kubernetes. You can click **Add Rule** to add multiple required rules. Ensure that all the labels specified in the rules must be in the same workload. Each rule requires a namespace and topology key. | - +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Preferred | It specifies a preference in scheduling. It corresponds to **preferredDuringSchedulingIgnoredDuringExecution** in Kubernetes. You can click **Add Rule** to add multiple preferred rules. The scheduler will try to enforce the rules but will not guarantee. If the scheduler cannot satisfy any one of the rules, the pod will still be scheduled. | - +-----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Set a rule according to the following table. You can click **Add Selector** to configure multiple selectors for a rule. - - .. table:: **Table 2** Selector settings - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+================================================================================================================================================================================================================================+ - | Weight | - This parameter is unavailable for a required rule. | - | | - Set the weight of a preferred rule. A higher weight indicates a higher priority. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | By default, the namespace of the current pod is used. You can also use another namespace. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Topology Key | Key of the worker node label that the system uses to denote a topology domain in which scheduling can be performed. Default and custom node labels can be used. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Label | Label of the workload. You can customize the label name. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Operator | The following relations are supported: **In**, **NotIn**, **Exists**, and **DoesNotExist** | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Value | Tag value. | - | | | - | | Operators **In** and **NotIn** allow one or more label values. Values are separated with colons (;). Operators **Exists** and **DoesNotExist** are used to determine whether a label exists, and do not require a label value. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Operation | You can click **Delete** to delete a selector. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Selector | A selector corresponds to **matchExpressions** in Kubernetes. You can click **Add Selector** to add multiple selectors for a scheduling rule. The rule is applied in scheduling only when all its selectors are satisfied. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - - .. figure:: /_static/images/en-us_image_0000001144578756.png - :alt: **Figure 1** Pod affinity scheduling policy - - **Figure 1** Pod affinity scheduling policy - -Using kubectl -------------- - -This section uses Nginx as an example to describe how to configure pod affinity. - -**Prerequisites** - -A workload that uses the nginx container image has been deployed on a node. - -**Procedure** - -Set **Namespace** to **default** and **Topology Key** to the built-in node label **kubernetes.io/hostname**, which means that the scheduling scope is a node. Set labels **app** and **type** and their value to **redis** and **database**, respectively. Set **Operator** to **In** and click **OK**. - -The YAML of the workload with pod affinity is as follows: - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - namespace: default - spec: - replicas: 2 - selector: - matchLabels: - app: nginx - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - nodeAffinity: {} - podAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchExpressions: - - key: app - operator: In - values: - - redis - - key: type - operator: In - values: - - database - namespaces: - - default - topologyKey: kubernetes.io/hostname - -.. important:: - - In this example, only when a candidate workload (for example, workload A) with both labels **app=redis** and **type=database** is found can the workload Nginx be successfully scheduled to the node of the candidate workload. diff --git a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/workload_anti-affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/workload_anti-affinity.rst deleted file mode 100644 index 5593edf..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/custom_scheduling_policies/workload_anti-affinity.rst +++ /dev/null @@ -1,115 +0,0 @@ -:original_name: cce_01_0234.html - -.. _cce_01_0234: - -Workload Anti-Affinity -====================== - -Using the CCE Console ---------------------- - -Workload anti-affinity determines the pods from which the target workload will be deployed in a different topology domain. - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. - -#. Click a workload name in the Deployment or StatefulSet list. On the displayed workload details page, click the **Scheduling Policies** tab and then click **Add Custom Scheduling Policy**. - -#. In the **Pod Anti-Affinity** area, set the namespace, topology key, and the label requirements to be met. - - There are two types of pod anti-affinity rules: **Required** (hard rule) and **Preferred** (soft rule), and the label operators include **In**, **NotIn**, **Exists**, and **DoesNotExist**. - - .. table:: **Table 1** Workload anti-affinity settings - - +-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===========+================================================================================================================================================================================================================================================================================================================================+ - | Required | It specifies a rule that must be met in scheduling. It corresponds to **requiredDuringSchedulingIgnoredDuringExecution** in Kubernetes. You can add multiple required rules. Ensure that all the labels specified in the rules must be in the same workload. Each rule requires a namespace and topology key. | - +-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Preferred | It specifies a preference in scheduling. It corresponds to **preferredDuringSchedulingIgnoredDuringExecution** in Kubernetes. You can add multiple preferred rules. The scheduler will try to enforce the rules but will not guarantee. If the scheduler cannot satisfy any one of the rules, the pod will still be scheduled. | - +-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Set a rule according to the following table. You can click **Add Selector** to configure multiple selectors for a rule. - - .. table:: **Table 2** Selector settings - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+================================================================================================================================================================================================================================+ - | Weight | - This parameter is unavailable for a required rule. | - | | - Set the weight of a preferred rule. A higher weight indicates a higher priority. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | By default, the namespace of the current pod is used. You can also use another namespace. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Topology Key | Key of the worker node label that the system uses to denote a topology domain in which scheduling can be performed. Default and custom node labels can be used. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Label | Label of the workload. You can customize the label name. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Operator | The following relations are supported: **In**, **NotIn**, **Exists**, and **DoesNotExist** | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Value | Tag value. | - | | | - | | Operators **In** and **NotIn** allow one or more label values. Values are separated with colons (;). Operators **Exists** and **DoesNotExist** are used to determine whether a label exists, and do not require a label value. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Operation | You can click **Delete** to delete a selector. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Selector | A selector corresponds to **matchExpressions** in Kubernetes. You can click **Add Selector** to add multiple selectors for a scheduling rule. The rule is applied in scheduling only when all its selectors are satisfied. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - - .. figure:: /_static/images/en-us_image_0000001144738550.png - :alt: **Figure 1** Pod anti-affinity scheduling policy - - **Figure 1** Pod anti-affinity scheduling policy - -Using kubectl -------------- - -This section uses Nginx as an example to describe how to configure pod anti-affinity. - -**Prerequisites** - -A workload that uses the nginx container image has been deployed on a node. - -**Procedure** - -Set **Namespace** to **default** and **Topology Key** to the built-in node label **kubernetes.io/hostname**, which means that the scheduling scope is a node. Set the label **app** and its value to **redis**. Set **Operator** to **In** and click **OK**. - -The YAML of the workload with pod anti-affinity: - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - namespace: default - spec: - replicas: 2 - selector: - matchLabels: - app: nginx - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - nodeAffinity: {} - podAffinity: {} - podAntiAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - - labelSelector: - matchExpressions: - - key: app - operator: In - values: - - redis - namespaces: - - default - topologyKey: kubernetes.io/hostname diff --git a/umn/source/affinity_and_anti-affinity_scheduling/index.rst b/umn/source/affinity_and_anti-affinity_scheduling/index.rst deleted file mode 100644 index 04d1b74..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -:original_name: cce_01_0149.html - -.. _cce_01_0149: - -Affinity and Anti-Affinity Scheduling -===================================== - -- :ref:`Scheduling Policy Overview ` -- :ref:`Custom Scheduling Policies ` -- :ref:`Simple Scheduling Policies ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - scheduling_policy_overview - custom_scheduling_policies/index - simple_scheduling_policies/index diff --git a/umn/source/affinity_and_anti-affinity_scheduling/scheduling_policy_overview.rst b/umn/source/affinity_and_anti-affinity_scheduling/scheduling_policy_overview.rst deleted file mode 100644 index f514284..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/scheduling_policy_overview.rst +++ /dev/null @@ -1,67 +0,0 @@ -:original_name: cce_01_0051.html - -.. _cce_01_0051: - -Scheduling Policy Overview -========================== - -Custom Scheduling Policies --------------------------- - -You can configure node affinity, workload affinity, and workload anti-affinity in custom scheduling policies. - -- :ref:`Node Affinity ` -- :ref:`Workload Affinity ` -- :ref:`Workload Anti-Affinity ` - -.. note:: - - Custom scheduling policies depend on node labels and pod labels. You can use default labels or customize labels as required. - -Simple Scheduling Policies --------------------------- - -A simple scheduling policy allows you to configure affinity between workloads and AZs, between workloads and nodes, and between workloads. - -- **Workload-AZ affinity**: Multiple AZ-based scheduling policies (including affinity and anti-affinity policies) can be configured. However, scheduling is performed as long as one of the scheduling policies is met. - - - **Affinity between workloads and AZs**: :ref:`Workload-AZ Affinity ` - - **Anti-affinity between workloads and AZs**: :ref:`Workload-AZ Anti-Affinity ` - -- **Workload-node affinity**: Multiple node-based scheduling policies (including affinity and anti-affinity scheduling) can be configured. However, scheduling is performed as long as one of the scheduling policies is met. For example, if a cluster contains nodes A, B, and C and two scheduling policies are set (one policy defines node A as an affinity node and the other policy defines node B as an anti-affinity node), then the workload can be scheduled to any node other than B. - - - **Affinity between workloads and nodes**: :ref:`Workload-Node Affinity ` - - **Anti-affinity between workloads and nodes**: :ref:`Workload-Node Anti-Affinity ` - -- **Workload-workload affinity**: Multiple workload-based scheduling policies can be configured, but the labels in these policies must belong to the same workload. - - - **Affinity between workloads**: For details, see :ref:`Workload-Workload Affinity `. You can deploy workloads on the same node to reduce consumption of network resources. - - :ref:`Figure 1 ` shows an example of affinity deployment, in which all workloads are deployed on the same node. - - .. _cce_01_0051__fig3017424713: - - .. figure:: /_static/images/en-us_image_0165899095.png - :alt: **Figure 1** Affinity between workloads - - **Figure 1** Affinity between workloads - - - **Anti-affinity between workloads**: For details, see :ref:`Workload-Workload Anti-Affinity `. Constraining multiple instances of the same workload from being deployed on the same node reduces the impact of system breakdowns. Anti-affinity deployment is also recommended for workloads that may interfere with each other. - - :ref:`Figure 2 ` shows an example of anti-affinity deployment, in which four workloads are deployed on four different nodes. - - .. _cce_01_0051__fig1505421971: - - .. figure:: /_static/images/en-us_image_0165899282.png - :alt: **Figure 2** Anti-affinity between workloads - - **Figure 2** Anti-affinity between workloads - -.. important:: - - When setting workload-workload affinity and workload-node affinity, ensure that the affinity relationships do not contradict each other; otherwise, workload deployment will fail. - - For example, Workload 3 will fail to be deployed when the following conditions are met: - - - Anti-affinity is configured for Workload 1 and Workload 2. Workload 1 is deployed on **Node A** and Workload 2 is deployed on **Node B**. - - Affinity is configured between Workload 2 and Workload 3, but the target node on which Workload 3 is to be deployed is **Node C** or **Node A**. diff --git a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/index.rst b/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/index.rst deleted file mode 100644 index 31fea94..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/index.rst +++ /dev/null @@ -1,24 +0,0 @@ -:original_name: cce_01_0230.html - -.. _cce_01_0230: - -Simple Scheduling Policies -========================== - -- :ref:`Workload-AZ Affinity ` -- :ref:`Workload-AZ Anti-Affinity ` -- :ref:`Workload-Node Affinity ` -- :ref:`Workload-Node Anti-Affinity ` -- :ref:`Workload-Workload Affinity ` -- :ref:`Workload-Workload Anti-Affinity ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - workload-az_affinity - workload-az_anti-affinity - workload-node_affinity - workload-node_anti-affinity - workload-workload_affinity - workload-workload_anti-affinity diff --git a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-az_affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-az_affinity.rst deleted file mode 100644 index 58375f5..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-az_affinity.rst +++ /dev/null @@ -1,77 +0,0 @@ -:original_name: cce_01_0228.html - -.. _cce_01_0228: - -Workload-AZ Affinity -==================== - -Using the CCE Console ---------------------- - -#. When :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `, in the **Scheduling Policies** area on the **Configure Advanced Settings** page, click |image1| next to **Workload-AZ Affinity and Anti-affinity** > **Affinity with AZs**. - -#. Select the AZ in which you want to deploy the workload. - - The created workload will be deployed in the selected AZ. - -.. _cce_01_0228__section4201420133117: - -Using kubectl -------------- - -This section uses an Nginx workload as an example to describe how to create a workload using kubectl. - -**Prerequisites** - -The ECS where the kubectl client runs has been connected to your cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -**Procedure** - -When :ref:`using kubectl to create a Deployment ` or :ref:`using kubectl to create a StatefulSet `, configure workload-AZ affinity. The following is an example YAML file for workload-AZ affinity. - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: az-in-deployment - spec: - replicas: 1 - selector: - matchLabels: - app: az-in-deployment - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: az-in-deployment - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: failure-domain.beta.kubernetes.io/zone #node's label key - operator: In - values: - - az1 #node's key value - -Setting the Object Type After Creating a Workload -------------------------------------------------- - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. -#. Click the name of the workload for which you will add a scheduling policy. On the workload details page, choose **Scheduling Policies** > **Add Simple Scheduling Policy** > **Add Affinity Object**. -#. Set **Object Type** to **Availability Zone**, and select the AZ in which the workload is eligible to be deployed. The workload will be deployed in the selected AZ. - - .. note:: - - This method can be used to add, edit, or delete scheduling policies. - -.. |image1| image:: /_static/images/en-us_image_0198873490.png diff --git a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-az_anti-affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-az_anti-affinity.rst deleted file mode 100644 index ba0ceb3..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-az_anti-affinity.rst +++ /dev/null @@ -1,77 +0,0 @@ -:original_name: cce_01_0229.html - -.. _cce_01_0229: - -Workload-AZ Anti-Affinity -========================= - -Using the CCE Console ---------------------- - -#. When :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `, in the **Scheduling Policies** area on the **Configure Advanced Settings** page, click |image1| next to **Workload-AZ Affinity and Anti-affinity** > **Anti-affinity with AZs**. - -#. Select an AZ in which the workload is ineligible to be deployed. - - The created workload is not deployed on the selected AZ. - -.. _cce_01_0229__section102822029173111: - -Using kubectl -------------- - -This section uses Nginx as an example to describe how to create a workload using kubectl. - -**Prerequisites** - -The ECS where the kubectl client runs has been connected to your cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -**Procedure** - -When :ref:`using kubectl to create a Deployment ` or :ref:`using kubectl to create a StatefulSet `, configure workload-AZ anti-affinity. The following is an example YAML file for workload-AZ anti-affinity. - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - spec: - replicas: 1 - selector: - matchLabels: - app: nginx - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: failure-domain.beta.kubernetes.io/zone #node's label key - operator: NotIn - values: - - az1 #node's key value - -Setting the Object Type After Creating a Workload -------------------------------------------------- - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. -#. Click the name of the workload for which you will add a scheduling policy. On the workload details page, choose **Scheduling Policies** > **Add Simple Scheduling Policy** > **Add Anti-affinity Object**. -#. Set **Object Type** to **Availability Zone** and select the AZ in which the workload is ineligible to be deployed. The workload will be constrained from being deployed in the selected AZ. - - .. note:: - - This method can be used to add, edit, or delete scheduling policies. - -.. |image1| image:: /_static/images/en-us_image_0198876479.png diff --git a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-node_affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-node_affinity.rst deleted file mode 100644 index ec4be42..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-node_affinity.rst +++ /dev/null @@ -1,75 +0,0 @@ -:original_name: cce_01_0225.html - -.. _cce_01_0225: - -Workload-Node Affinity -====================== - -Using the CCE Console ---------------------- - -#. When :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `, in the **Scheduling Policies** area on the **Configure Advanced Settings** page, choose **Workload-Node Affinity and Anti-affinity** > **Affinity with Nodes** > **Add**. - -#. Select the node on which you want to deploy the workload, and click **OK**. - - If you select multiple nodes, the system automatically chooses one of them during workload deployment. - -.. _cce_01_0225__section711574271117: - -Using kubectl -------------- - -This section uses an Nginx workload as an example to describe how to create a workload using kubectl. - -**Prerequisites** - -The ECS where the kubectl client runs has been connected to your cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -**Procedure** - -When :ref:`using kubectl to create a Deployment ` or :ref:`using kubectl to create a StatefulSet `, configure workload-node affinity. The following is an example YAML file for workload-node affinity. - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - spec: - replicas: 1 - selector: - matchLabels: - app: nginx - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: nodeName #node's label key - operator: In - values: - - test-node-1 #node's label value - -Setting the Object Type After Creating a Workload -------------------------------------------------- - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. -#. Click the name of the workload for which you will add a scheduling policy. On the workload details page, choose **Scheduling Policies** > **Add Simple Scheduling Policy** > **Add Affinity Object**. -#. Set **Object Type** to **Node** and select the node where the workload is to be deployed. The workload will be deployed on the selected node. - - .. note:: - - This method can be used to add, edit, or delete scheduling policies. diff --git a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-node_anti-affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-node_anti-affinity.rst deleted file mode 100644 index 09eda98..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-node_anti-affinity.rst +++ /dev/null @@ -1,75 +0,0 @@ -:original_name: cce_01_0226.html - -.. _cce_01_0226: - -Workload-Node Anti-Affinity -=========================== - -Using the CCE Console ---------------------- - -#. When :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `, in the **Scheduling Policies** area on the **Configure Advanced Settings** page, choose **Workload-Node Affinity and Anti-affinity** > **Anti-affinity with Nodes** > **Add**. - -#. Select the node on which the workload is ineligible to be deployed, and click **OK**. - - If you select multiple nodes, the workload will not be deployed on these nodes. - -.. _cce_01_0226__section1361482522712: - -Using kubectl -------------- - -This section uses Nginx as an example to describe how to create a workload using kubectl. - -**Prerequisites** - -The ECS where the kubectl client runs has been connected to your cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -**Procedure** - -When :ref:`using kubectl to create a Deployment ` or :ref:`using kubectl to create a StatefulSet `, configure workload-node affinity. The following is an example YAML file for workload-node anti-affinity. - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - spec: - replicas: 1 - selector: - matchLabels: - app: nginx - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - nodeAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: nodeName #node's label key - operator: NotIn #Indicates that the workload will not be deployed on the node. - values: - - test-node-1 #node's label value - -Setting the Object Type After Creating a Workload -------------------------------------------------- - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. -#. Click the name of the workload for which you will add a scheduling policy. On the workload details page, choose **Scheduling Policies** > **Add Simple Scheduling Policy** > **Add Anti-affinity Object**. -#. Set **Object Type** to **Node** and select the node on which the workload is ineligible to be deployed. The workload will be constrained from being deployed on the selected node. - - .. note:: - - This method can be used to add, edit, or delete scheduling policies. diff --git a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-workload_affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-workload_affinity.rst deleted file mode 100644 index 47ed9a6..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-workload_affinity.rst +++ /dev/null @@ -1,75 +0,0 @@ -:original_name: cce_01_0220.html - -.. _cce_01_0220: - -Workload-Workload Affinity -========================== - -Using the CCE Console ---------------------- - -#. When :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `, in the **Scheduling Policies** area on the **Configure Advanced Settings** page, choose **Inter-Pod Affinity and Anti-affinity** > **Affinity with Pods** > **Add**. - -#. Select the workloads that will be co-located with the current workload on the same node, and click **OK**. - - The workload to be created will be deployed on the same node as the selected affinity workloads. - -.. _cce_01_0220__section5140193643912: - -Using kubectl -------------- - -This section uses Nginx as an example to describe how to create a workload using kubectl. - -**Prerequisites** - -The ECS where the kubectl client runs has been connected to your cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -**Procedure** - -When :ref:`using kubectl to create a Deployment ` or :ref:`using kubectl to create a StatefulSet `, configure workload-workload affinity. The following is an example YAML file for workload-workload affinity. - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - spec: - replicas: 1 - selector: - matchLabels: - app: nginx - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - podAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: app #workload's label key - operator: In - values: - - test #workload's label value - -Setting the Object Type After Creating a Workload -------------------------------------------------- - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. -#. Click the name of the workload for which you will add a scheduling policy. On the workload details page, choose **Scheduling Policies** > **Add Simple Scheduling Policy** > **Add Affinity Object**. -#. Set **Object Type** to **Workload** and select the workloads to be deployed on the same node as the created workload. The created workload and the selected workloads will be deployed on the same node. - - .. note:: - - This method can be used to add, edit, or delete scheduling policies. diff --git a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-workload_anti-affinity.rst b/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-workload_anti-affinity.rst deleted file mode 100644 index 4f5990b..0000000 --- a/umn/source/affinity_and_anti-affinity_scheduling/simple_scheduling_policies/workload-workload_anti-affinity.rst +++ /dev/null @@ -1,75 +0,0 @@ -:original_name: cce_01_0227.html - -.. _cce_01_0227: - -Workload-Workload Anti-Affinity -=============================== - -Using the CCE Console ---------------------- - -#. When :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `, in the **Scheduling Policies** area on the **Configure Advanced Settings** page, choose **Inter-Pod Affinity and Anti-affinity** > **Anti-affinity with Pods** > **Add**. - -#. Select the workloads to which you want to deploy the target workload on a different node, and click **OK**. - - The workload to be created and the selected workloads will be deployed on different nodes. - -.. _cce_01_0227__section1894310152317: - -Using kubectl -------------- - -This section uses Nginx as an example to describe how to create a workload using kubectl. - -**Prerequisites** - -The ECS where the kubectl client runs has been connected to your cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -**Procedure** - -When :ref:`using kubectl to create a Deployment ` or :ref:`using kubectl to create a StatefulSet `, configure workload-workload anti-affinity. The following is an example YAML file for workload-workload anti-affinity. - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - spec: - replicas: 1 - selector: - matchLabels: - app: nginx - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - imagePullPolicy: Always - name: nginx - imagePullSecrets: - - name: default-secret - affinity: - podAffinity: - requiredDuringSchedulingIgnoredDuringExecution: - nodeSelectorTerms: - - matchExpressions: - - key: app #workload's label key - operator: NotIn - values: - - test #workload's label value - -Setting the Object Type After Creating a Workload -------------------------------------------------- - -#. Log in to the CCE console and choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** in the navigation pane. -#. Click the name of the workload for which you will add a scheduling policy. On the workload details page, choose **Scheduling Policies** > **Add Simple Scheduling Policy** > **Add Anti-affinity Object**. -#. Set **Object Type** to **Workload** and select the workloads to be deployed on a different node from the created workload. The created workload and the selected workloads will be deployed on different nodes. - - .. note:: - - This method can be used to add, edit, or delete scheduling policies. diff --git a/umn/source/auto_scaling/index.rst b/umn/source/auto_scaling/index.rst index 622df56..8715284 100644 --- a/umn/source/auto_scaling/index.rst +++ b/umn/source/auto_scaling/index.rst @@ -1,13 +1,14 @@ -:original_name: cce_01_0207.html +:original_name: cce_10_0207.html -.. _cce_01_0207: +.. _cce_10_0207: Auto Scaling ============ -- :ref:`Overview ` -- :ref:`Scaling a Workload ` -- :ref:`Scaling a Cluster/Node ` +- :ref:`Overview ` +- :ref:`Scaling a Workload ` +- :ref:`Scaling a Node ` +- :ref:`Using HPA and CA for Auto Scaling of Workloads and Nodes ` .. toctree:: :maxdepth: 1 @@ -15,4 +16,5 @@ Auto Scaling overview scaling_a_workload/index - scaling_a_cluster_node/index + scaling_a_node/index + using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes diff --git a/umn/source/auto_scaling/overview.rst b/umn/source/auto_scaling/overview.rst index 36a11c4..b10fff2 100644 --- a/umn/source/auto_scaling/overview.rst +++ b/umn/source/auto_scaling/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0279.html +:original_name: cce_10_0279.html -.. _cce_01_0279: +.. _cce_10_0279: Overview ======== @@ -12,17 +12,17 @@ Context More and more applications are developed based on Kubernetes. It becomes increasingly important to quickly scale out applications on Kubernetes to cope with service peaks and to scale in applications during off-peak hours to save resources and reduce costs. -In a Kubernetes cluster, auto scaling involves pods and nodes. A pod is an application instance. Each pod contains one or more containers and runs on a node (VM). If a cluster does not have sufficient nodes to run new pods, you need to add nodes to the cluster to ensure service running. +In a Kubernetes cluster, auto scaling involves pods and nodes. A pod is an application instance. Each pod contains one or more containers and runs on a node (VM or bare-metal server). If a cluster does not have sufficient nodes to run new pods, you need to add nodes to the cluster to ensure service running. In CCE, auto scaling is used for online services, large-scale computing and training, deep learning GPU or shared GPU training and inference, periodic load changes, and many other scenarios. Auto Scaling in CCE ------------------- -**CCE supportsauto scaling for workloads and nodes.** +**CCE supports auto scaling for workloads and nodes.** -- :ref:`Workload scaling `\ **:** Auto scaling at the scheduling layer to change the scheduling capacity of workloads. For example, you can use the HPA, a scaling component at the scheduling layer, to adjust the number of replicas of an application. Adjusting the number of replicas changes the scheduling capacity occupied by the current workload, thereby enabling scaling at the scheduling layer. -- :ref:`Node scaling `\ **:** Auto scaling at the resource layer. When the planned cluster nodes cannot allow workload scheduling, ECS resources are provided to support scheduling. +- :ref:`Workload scaling `\ **:** Auto scaling at the scheduling layer to change the scheduling capacity of workloads. For example, you can use the HPA, a scaling component at the scheduling layer, to adjust the number of replicas of an application. Adjusting the number of replicas changes the scheduling capacity occupied by the current workload, thereby enabling scaling at the scheduling layer. +- :ref:`Node scaling `\ **:** Auto scaling at the resource layer. When the planned cluster nodes cannot allow workload scheduling, ECS resources are provided to support scheduling. Components ---------- @@ -34,7 +34,7 @@ Components +------+-------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+ | Type | Component Name | Component Description | Reference | +======+=====================================+====================================================================================================================================================================================+=======================================================================+ - | HPA | :ref:`metrics-server ` | A built-in component of Kubernetes, which enables horizontal scaling of pods. It adds the application-level cooldown time window and scaling threshold functions based on the HPA. | :ref:`Creating an HPA Policy for Workload Auto Scaling ` | + | HPA | :ref:`metrics-server ` | A built-in component of Kubernetes, which enables horizontal scaling of pods. It adds the application-level cooldown time window and scaling threshold functions based on the HPA. | :ref:`Creating an HPA Policy for Workload Auto Scaling ` | +------+-------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------+ **Node scaling components are described as follows:** @@ -44,5 +44,5 @@ Components +---------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+-----------------------------------------------------+ | Component Name | Component Description | Application Scenario | Reference | +=================================+===============================================================================================================================================+=========================================================================================+=====================================================+ - | :ref:`autoscaler ` | An open-source Kubernetes component for horizontal scaling of nodes, which is optimized in terms of scheduling and auto scaling capabilities. | Online services, deep learning, and large-scale computing with limited resource budgets | :ref:`Creating a Node Scaling Policy ` | + | :ref:`autoscaler ` | An open source Kubernetes component for horizontal scaling of nodes, which is optimized in terms of scheduling and auto scaling capabilities. | Online services, deep learning, and large-scale computing with limited resource budgets | :ref:`Creating a Node Scaling Policy ` | +---------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------+-----------------------------------------------------+ diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/index.rst b/umn/source/auto_scaling/scaling_a_cluster_node/index.rst deleted file mode 100644 index 8e60f71..0000000 --- a/umn/source/auto_scaling/scaling_a_cluster_node/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -:original_name: cce_01_0291.html - -.. _cce_01_0291: - -Scaling a Cluster/Node -====================== - -- :ref:`Node Scaling Mechanisms ` -- :ref:`Creating a Node Scaling Policy ` -- :ref:`Managing Node Scaling Policies ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - node_scaling_mechanisms - creating_a_node_scaling_policy - managing_node_scaling_policies diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/managing_node_scaling_policies.rst b/umn/source/auto_scaling/scaling_a_cluster_node/managing_node_scaling_policies.rst deleted file mode 100644 index 523f982..0000000 --- a/umn/source/auto_scaling/scaling_a_cluster_node/managing_node_scaling_policies.rst +++ /dev/null @@ -1,54 +0,0 @@ -:original_name: cce_01_0063.html - -.. _cce_01_0063: - -Managing Node Scaling Policies -============================== - -Scenario --------- - -After a node scaling policy is created, you can delete, edit, disable, enable, or clone the policy. - -Viewing a Node Scaling Policy ------------------------------ - -You can view the associated node pool, rules, and scaling history of a node scaling policy and rectify faults according to the error information displayed. - -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Node Scaling** tab page, click |image1| in front of the policy to be viewed. -#. In the expanded area, the **Associated Node Pool**, **Execution Rules**, and **Scaling Records** tab pages are displayed. If the policy is abnormal, locate and rectify the fault based on the error information. - - .. note:: - - You can also enable or disable auto scaling in **Node Pools**. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**, and click **Edit** in the upper right corner of the node pool to be operated. In the **Edit Node Pool** dialog box displayed, you can enable **Autoscaler** and set the limits of the number of nodes. - -Deleting a Node Scaling Policy ------------------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Node Scaling** tab page, click **Delete** in the **Operation** column of the policy to be deleted. -#. In the **Delete Node Policy** dialog box displayed, confirm whether to delete the policy. -#. Enter **DELETE** in the text box. -#. Click **OK** to delete the policy. - -Editing a Node Scaling Policy ------------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Node Scaling** tab page, click **Edit** in the **Operation** column of the policy. -#. On the **Edit Node Scaling Policy** page displayed, modify policy parameter values listed in :ref:`Table 1 `. -#. After the configuration is complete, click **OK**. - -Cloning a Node Scaling Policy ------------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Node Scaling** tab page, click **More** > **Clone** in the **Operation** column of the policy. -#. On the **Create Node Scaling Policy** page displayed, certain parameters have been cloned. Add or modify other policy parameters based on service requirements. -#. Click **Create Now** to clone the policy. The cloned policy is displayed in the policy list on the **Node Scaling** tab page. - -Enabling or Disabling a Node Scaling Policy -------------------------------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Node Scaling** tab page, click **More** > **Disable** or **Enable** in the **Operation** column of the policy. -#. In the dialog box displayed, confirm whether to disable or enable the node policy. -#. Click **Yes**. The policy status is displayed in the node scaling list. - -.. |image1| image:: /_static/images/en-us_image_0254986677.png diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/creating_a_node_scaling_policy.rst b/umn/source/auto_scaling/scaling_a_node/creating_a_node_scaling_policy.rst similarity index 72% rename from umn/source/auto_scaling/scaling_a_cluster_node/creating_a_node_scaling_policy.rst rename to umn/source/auto_scaling/scaling_a_node/creating_a_node_scaling_policy.rst index 3444f6e..aa2a7fc 100644 --- a/umn/source/auto_scaling/scaling_a_cluster_node/creating_a_node_scaling_policy.rst +++ b/umn/source/auto_scaling/scaling_a_node/creating_a_node_scaling_policy.rst @@ -1,11 +1,11 @@ -:original_name: cce_01_0209.html +:original_name: cce_10_0209.html -.. _cce_01_0209: +.. _cce_10_0209: Creating a Node Scaling Policy ============================== -CCE provides auto scaling through the :ref:`autoscaler ` add-on. Nodes with different specifications can be automatically added across AZs on demand. +CCE provides auto scaling through the :ref:`autoscaler ` add-on. Nodes with different specifications can be automatically added across AZs on demand. If a node scaling policy and the configuration in the autoscaler add-on take effect at the same time, for example, there are pods that cannot be scheduled and the value of a metric reaches the threshold at the same time, scale-out is performed first for the unschedulable pods. @@ -15,28 +15,27 @@ If a node scaling policy and the configuration in the autoscaler add-on take eff Prerequisites ------------- -Before using the node scaling function, you must install the :ref:`autoscaler ` add-on of v1.13.8 or later. +Before using the node scaling function, you must install the :ref:`autoscaler ` add-on of v1.13.8 or later. + +Notes and Constraints +--------------------- + +- Auto scaling policies apply to node pools. When the number of nodes in a node pool is 0 and the scaling policy is based on CPU or memory usage, node scaling is not triggered. Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Node Scaling** tab page, click **Create Node Scaling Policy**. -#. In the **Check Add-ons** step: +#. Log in to the CCE console and access the cluster console. +#. Choose **Node Scaling** in the navigation pane. - - If |image1| is displayed next to the add-on name, click **Install**, set add-on parameters as required, and click **Install** to install the add-on. - - If |image2| is displayed next to the add-on name, the add-on has been installed. + - If **Uninstalled** is displayed next to the add-on name, click **Install**, set add-on parameters as required, and click **Install** to install the add-on. + - If **Installed** is displayed next to the add-on name, the add-on has been installed. -#. After the required add-ons have been installed, click **Next: Policy configuration**. - - .. note:: - - If the add-ons have been installed, after you click **Create Node Scaling Policy**, you will directly land on the second step to configure the policy. The first step (checking the add-ons) has been completed almost instantly. - -#. On the **Create Node Scaling Policy** page, set the following policy parameters. +#. Click **Create Node Scaling Policy** in the upper right corner and set the parameters as follows: - **Policy Name**: name of the policy to be created, which can be customized. - - **Associated Node Pool**: Click **Add Node Pool** and select the node pool to be associated. You can associate multiple node pools to use the same scaling policy. + - **Associated Node Pools**: Select the node pool to be associated. You can associate multiple node pools to use the same scaling policy. .. note:: @@ -48,51 +47,49 @@ Procedure d. If the VM specifications of multiple node pools are the same but the node pools are deployed in different AZs, the system randomly selects a node pool to trigger scaling. e. If the resources of the preferred node pool are insufficient, the system automatically selects next node pool based on the priority. - For details about the node pool priority, see :ref:`Autoscaler `. + For details about the node pool priority, see :ref:`Creating a Node Pool `. - - **Execution Rules**: Click **Add Rule**. In the dialog box displayed, set the following parameters: + - **Rules**: Click **Add Rule**. In the dialog box displayed, set the following parameters: - **Name**: Enter a rule name. + **Rule Name**: Enter a rule name. - **Type**: You can select **Metric-based** or **Periodic**. The differences between the two types are as follows: + **Rule Type**: You can select **Metric-based** or **Periodic**. The differences between the two types are as follows: - **Metric-based**: - - **Condition**: Select **CPU allocation** or **Memory allocation** and enter a value. The value must be greater than the scale-in percentage configured in the autoscaler add-on. + **Condition**: Select **CPU allocation rate** or **Memory allocation rate** and enter a value. The value must be greater than the scale-in percentage configured in the autoscaler add-on. - .. note:: + .. note:: - - Resource allocation (%) = Resources requested by pods in the node pool/Resources allocatable to pods in the node pool + - Resource allocation (%) = Resources requested by pods in the node pool/Resources allocatable to pods in the node pool - - **If multiple rules meet the conditions, the rules are executed in either of the following modes:** + - **If multiple rules meet the conditions, the rules are executed in either of the following modes:** - If rules based on the **CPU allocation rate** and **memory allocation rate** are configured and two or more rules meet the scale-out conditions, the rule that will add the most nodes will be executed. + If rules based on the **CPU allocation rate** and **memory allocation rate** are configured and two or more rules meet the scale-out conditions, the rule that will add the most nodes will be executed. - If a rule based on the **CPU allocation rate** and **a periodic rule** are configured and they both meet the scale-out conditions, one of them will be executed randomly. The rule executed first (rule A) changes the node pool to the scaling state. As a result, the other rule (rule B) cannot be executed. After rule A is executed and the node pool status becomes normal, rule B will not be executed. + If a rule based on the **CPU allocation rate** and **a periodic rule** are configured and they both meet the scale-out conditions, one of them will be executed randomly. The rule executed first (rule A) changes the node pool to the scaling state. As a result, the other rule (rule B) cannot be executed. After rule A is executed and the node pool status becomes normal, rule B will not be executed. - - If rules based on the **CPU allocation rate** and **memory allocation rate** are configured, the policy detection period varies with the processing logic of each loop of the autoscaler add-on. Scale-out is triggered once the conditions are met, but it is constrained by other factors such as the cool-down interval and node pool status. - - - **Action**: Set an action to be performed when the trigger condition is met. + - If rules based on the **CPU allocation rate** and **memory allocation rate** are configured, the policy detection period varies with the processing logic of each loop of the autoscaler add-on. Scale-out is triggered once the conditions are met, but it is constrained by other factors such as the cool-down interval and node pool status. - **Periodic**: - - **Triggered At**: You can select a specific time point every day, every week, every month, or every year. - - **Action**: Set an action to be performed when the **Triggered At** value is reached. + **Trigger Time**: You can select a specific time point every day, every week, every month, or every year. + + **Action**: Set an action to be performed when the trigger condition is met. You can click **Add Rule** again to add more node scaling policies. You can add a maximum of one CPU usage-based rule and one memory usage-based rule. The total number of rules cannot exceed 10. -#. After the configuration is complete, click **Create**. If the system displays a message indicating that the request to create a node scaling policy is submitted successfully, click **Back to Node Scaling Policy List**. -#. On the **Node Scaling** tab page, you can view the created node scaling policy. +#. Click **OK**. Constraints on Scale-in ----------------------- -CCE cannot trigger scale-in by using node scaling policies. You can set a scale-in policy when installing the :ref:`autoscaler add-on `. +You can set node scale-in policies only when installing the :ref:`autoscaler add-on `. Node scale-in can be triggered only by the resource allocation rate. When CPU and memory allocation rates in a cluster are lower than the specified thresholds (set when the autoscaler add-on is installed or modified), scale-in is triggered for nodes in the node pool (this function can be disabled). -Example YAML File ------------------ +Example YAML +------------ The following is a YAML example of a node scaling policy: @@ -135,7 +132,7 @@ The following is a YAML example of a node scaling policy: targetNodepoolIds: - 7d48eca7-3419-11ea-bc29-0255ac1001a8 -.. _cce_01_0209__table18763092201: +.. _cce_10_0209__table18763092201: .. table:: **Table 1** Key parameters @@ -176,6 +173,3 @@ The following is a YAML example of a node scaling policy: +---------------------------------------------+---------+---------------------------------------------------------------------------------------------------------------------+ | spec.targetNodepoolIds[x] | String | ID of the node pool associated with the scaling policy. | +---------------------------------------------+---------+---------------------------------------------------------------------------------------------------------------------+ - -.. |image1| image:: /_static/images/en-us_image_0259814716.png -.. |image2| image:: /_static/images/en-us_image_0259814717.png diff --git a/umn/source/auto_scaling/scaling_a_node/index.rst b/umn/source/auto_scaling/scaling_a_node/index.rst new file mode 100644 index 0000000..220efbb --- /dev/null +++ b/umn/source/auto_scaling/scaling_a_node/index.rst @@ -0,0 +1,18 @@ +:original_name: cce_10_0291.html + +.. _cce_10_0291: + +Scaling a Node +============== + +- :ref:`Node Scaling Mechanisms ` +- :ref:`Creating a Node Scaling Policy ` +- :ref:`Managing Node Scaling Policies ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + node_scaling_mechanisms + creating_a_node_scaling_policy + managing_node_scaling_policies diff --git a/umn/source/auto_scaling/scaling_a_node/managing_node_scaling_policies.rst b/umn/source/auto_scaling/scaling_a_node/managing_node_scaling_policies.rst new file mode 100644 index 0000000..2c49103 --- /dev/null +++ b/umn/source/auto_scaling/scaling_a_node/managing_node_scaling_policies.rst @@ -0,0 +1,61 @@ +:original_name: cce_10_0063.html + +.. _cce_10_0063: + +Managing Node Scaling Policies +============================== + +Scenario +-------- + +After a node scaling policy is created, you can delete, edit, disable, enable, or clone the policy. + +Viewing a Node Scaling Policy +----------------------------- + +You can view the associated node pool, rules, and scaling history of a node scaling policy and rectify faults according to the error information displayed. + +#. Log in to the CCE console and access the cluster console. +#. Choose **Node Scaling** in the navigation pane and click |image1| in front of the policy to be viewed. +#. In the expanded area, the **Associated Node Pools**, **Rules**, and **Scaling History** tab pages are displayed. If the policy is abnormal, locate and rectify the fault based on the error information. + + .. note:: + + You can also disable or enable auto scaling on the **Node Pools** page. + + a. Log in to the CCE console and access the cluster console. + b. In the navigation pane, choose **Nodes** and switch to the **Node Pools** tab page. + c. Click **Edit** of the node pool to be operated. In the **Edit Node Pool** dialog box that is displayed, set the limits of the number of nodes. + +Deleting a Node Scaling Policy +------------------------------ + +#. Log in to the CCE console and access the cluster console. +#. Choose **Node Scaling** in the navigation pane and choose **More** > **Delete** next to the policy to be deleted. +#. In the **Delete Node Scaling Policy** dialog box displayed, confirm whether to delete the policy. +#. Click **Yes** to delete the policy. + +Editing a Node Scaling Policy +----------------------------- + +#. Log in to the CCE console and access the cluster console. +#. Choose **Node Scaling** in the navigation pane and click **Edit** in the **Operation** column of the policy to be edited. +#. On the **Edit Node Scaling Policy** page displayed, modify policy parameter values listed in :ref:`Table 1 `. +#. After the configuration is complete, click **OK**. + +Cloning a Node Scaling Policy +----------------------------- + +#. Log in to the CCE console and access the cluster console. +#. Choose **Node Scaling** in the navigation pane and choose **More** > **Clone** next to the policy to be cloned. +#. On the **Clone Node Scaling Policy** page displayed, certain parameters have been cloned. Add or modify other policy parameters based on service requirements. +#. Click **OK**. + +Enabling or Disabling a Node Scaling Policy +------------------------------------------- + +#. Log in to the CCE console and access the cluster console. +#. Choose **Node Scaling** in the navigation pane and click **Disable** in the **Operation** column of the policy to be disabled. If the policy is in the disabled state, click **Enable** in the **Operation** column of the policy. +#. In the dialog box displayed, confirm whether to disable or enable the node policy. + +.. |image1| image:: /_static/images/en-us_image_0000001244261161.png diff --git a/umn/source/auto_scaling/scaling_a_cluster_node/node_scaling_mechanisms.rst b/umn/source/auto_scaling/scaling_a_node/node_scaling_mechanisms.rst similarity index 92% rename from umn/source/auto_scaling/scaling_a_cluster_node/node_scaling_mechanisms.rst rename to umn/source/auto_scaling/scaling_a_node/node_scaling_mechanisms.rst index eb6cf01..d117a1a 100644 --- a/umn/source/auto_scaling/scaling_a_cluster_node/node_scaling_mechanisms.rst +++ b/umn/source/auto_scaling/scaling_a_node/node_scaling_mechanisms.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0296.html +:original_name: cce_10_0296.html -.. _cce_01_0296: +.. _cce_10_0296: Node Scaling Mechanisms ======================= @@ -12,7 +12,7 @@ Kubernetes HPA is designed for pods. However, if the cluster resources are insuf Prerequisites ------------- -Before using the node scaling function, you must install the :ref:`autoscaler ` add-on of v1.13.8 or later. +Before using the node scaling function, you must install the :ref:`autoscaler ` add-on of v1.13.8 or later. How autoscaler Works -------------------- @@ -35,11 +35,11 @@ However, a node cannot be removed from a cluster if the following pods exist: autoscaler Architecture ----------------------- -:ref:`Figure 1 ` shows the autoscaler architecture and its core modules: +:ref:`Figure 1 ` shows the autoscaler architecture and its core modules: -.. _cce_01_0296__fig114831750115719: +.. _cce_10_0296__fig114831750115719: -.. figure:: /_static/images/en-us_image_0000001199848585.png +.. figure:: /_static/images/en-us_image_0000001199501290.png :alt: **Figure 1** autoscaler architecture **Figure 1** autoscaler architecture diff --git a/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst b/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst index ee93f25..9bff10e 100644 --- a/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst +++ b/umn/source/auto_scaling/scaling_a_workload/creating_an_hpa_policy_for_workload_auto_scaling.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0208.html +:original_name: cce_10_0208.html -.. _cce_01_0208: +.. _cce_10_0208: Creating an HPA Policy for Workload Auto Scaling ================================================ @@ -10,15 +10,13 @@ Horizontal Pod Autoscaling (HPA) in Kubernetes implements horizontal scaling of Prerequisites ------------- -The :ref:`metrics-server ` add-on has been installed. This add-on collects public metrics of kubelet in Kubernetes clusters, including the CPU usage and memory usage. +To use HPA policies, you need to install an add-on that can provide the metrics API, such as metrics-server and prometheus. Notes and Constraints --------------------- - HPA policies can be created only for clusters of v1.13 or later. -- Only one policy can be created for each workload. That is, if you have created an HPA policy, you cannot create other HPA policies for the workload. You can delete the created HPA policy and create a new one. - - For clusters earlier than v1.19.10, if an HPA policy is used to scale out a workload with EVS volumes mounted, the existing pods cannot be read or written when a new pod is scheduled to another node. For clusters of v1.19.10 and later, if an HPA policy is used to scale out a workload with EVS volume mounted, a new pod cannot be started because EVS disks cannot be attached. @@ -26,74 +24,79 @@ Notes and Constraints Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Workload Scaling** tab page, click **Create HPA Policy**. +#. Log in to the CCE console and access the cluster console. -#. In the **Check Add-ons** step: +#. In the navigation pane, choose **Workload Scaling**. Then click **Create HPA Policy** in the upper right corner. - - If |image1| is displayed next to the add-on name, click **Install**, set add-on parameters as required, and click **Install** to install the add-on. - - If |image2| is displayed next to the add-on name, the add-on has been installed. +#. Set policy parameters. -#. After the required add-ons have been installed, click **Next: Policy configuration**. - - .. note:: - - If the add-ons have been installed, after you click **Create HPA Policy**, you will directly land on the second step to configure the policy. The first step (checking the add-ons) has been completed almost instantly. - -#. Set policy parameters by referring to :ref:`Table 1 `. - - .. _cce_01_0208__table8638121213265: + .. _cce_10_0208__table8638121213265: .. table:: **Table 1** HPA policy parameters - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===========================================================================================================================================================================================================================+ - | Policy Name | Name of the policy to be created. Set this parameter as required. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Name | Cluster to which the workload belongs. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace to which the workload belongs. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Associated Workload | Workload with which the HPA policy is associated. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Pod Range | Minimum and maximum numbers of pods. | - | | | - | | When a policy is triggered, the workload pods are scaled within this range. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cooldown Period | Interval between a scale-in and a scale-out. The unit is minute. **The interval cannot be shorter than 1 minute.** | - | | | - | | **This parameter is available only for clusters of v1.15 and later. It is not supported in clusters of v1.13 or earlier.** | - | | | - | | This parameter indicates the interval between consecutive scaling operations. The cooldown period ensures that a scaling operation is initiated only when the previous one is completed and the system is running stably. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Rules | Policy rules can be based on system metrics. | - | | | - | | **System metrics** | - | | | - | | - **Metric**: You can select **CPU usage** or **Memory usage**. | - | | | - | | .. note:: | - | | | - | | Usage = CPUs or memory used by pods/Requested CPUs or memory. | - | | | - | | - **Expected Value**: Enter the expected average resource usage. | - | | | - | | This parameter indicates the expected value of the selected metric. The number of new pods required (rounded up) = Current metric value/Expected value x Number of current pods | - | | | - | | - **Threshold**: Enter the scaling thresholds. | - | | | - | | If the metric value is greater than the scale-in threshold and less than the scale-out threshold, no scaling is triggered. **This parameter is supported only in clusters of v1.15 or later.** | - | | | - | | You can click **Add Rule** again to add more scaling policies. | - | | | - | | .. note:: | - | | | - | | When calculating the number of pods to be added or reduced, the HPA policy uses the maximum metrics values in the last 5 minutes. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +==============================================================+=========================================================================================================================================================================================================================================================================================================+ + | Policy Name | Name of the policy to be created. Set this parameter as required. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Namespace | Namespace to which the workload belongs. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Associated Workload | Workload with which the HPA policy is associated. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pod Range | Minimum and maximum numbers of pods. | + | | | + | | When a policy is triggered, the workload pods are scaled within this range. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Cooldown Period | Interval between a scale-in and a scale-out. The unit is minute. **The interval cannot be shorter than 1 minute.** | + | | | + | | **This parameter is supported only from clusters of v1.15 to v1.23.** | + | | | + | | This parameter indicates the interval between consecutive scaling operations. The cooldown period ensures that a scaling operation is initiated only when the previous one is completed and the system is running stably. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Scaling Behavior | **This parameter is supported only in clusters of v1.25 or later.** | + | | | + | | - **Default**: Scales workloads using the Kubernetes default behavior. For details, see `Default Behavior `__. | + | | - **Custom**: Scales workloads using custom policies such as stabilization window, steps, and priorities. Unspecified parameters use the values recommended by Kubernetes. | + | | | + | | - **Disable scale-out/scale-in**: Select whether to disable scale-out or scale-in. | + | | - **Stabilization Window**: A period during which CCE continuously checks whether the metrics used for scaling keep fluctuating. CCE triggers scaling if the desired state is not maintained for the entire window. This window restricts the unwanted flapping of pod count due to metric changes. | + | | - **Step**: specifies the scaling step. You can set the number or percentage of pods to be scaled in or out within a specified period. If there are multiple policies, you can select the policy that maximizes or minimizes the number of pods. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | System Policy | - **Metric**: You can select **CPU usage** or **Memory usage**. | + | | | + | | .. note:: | + | | | + | | Usage = CPUs or memory used by pods/Requested CPUs or memory. | + | | | + | | - **Desired Value**: Enter the desired average resource usage. | + | | | + | | This parameter indicates the desired value of the selected metric. Number of pods to be scaled (rounded up) = (Current metric value/Desired value) x Number of current pods | + | | | + | | .. note:: | + | | | + | | When calculating the number of pods to be added or reduced, the HPA policy uses the maximum number of pods in the last 5 minutes. | + | | | + | | - **Tolerance Range**: Scaling is not triggered when the metric value is within the tolerance range. The desired value must be within the tolerance range. | + | | | + | | If the metric value is greater than the scale-in threshold and less than the scale-out threshold, no scaling is triggered. **This parameter is supported only in clusters of v1.15 or later.** | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Custom Policy (supported only in clusters of v1.15 or later) | .. note:: | + | | | + | | Before setting a custom policy, you need to install an add-on that supports custom metric collection in the cluster, for example, prometheus add-on. | + | | | + | | - **Metric Name**: name of the custom metric. You can select a name as prompted. | + | | | + | | For details, see :ref:`Custom Monitoring `. | + | | | + | | - **Metric Source**: Select an object type from the drop-down list. You can select **Pod**. | + | | | + | | - **Desired Value**: the average metric value of all pods. Number of pods to be scaled (rounded up) = (Current metric value/Desired value) x Number of current pods | + | | | + | | .. note:: | + | | | + | | When calculating the number of pods to be added or reduced, the HPA policy uses the maximum number of pods in the last 5 minutes. | + | | | + | | - **Tolerance Range**: Scaling is not triggered when the metric value is within the tolerance range. The desired value must be within the tolerance range. | + +--------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -#. After the configuration is complete, click **Create**. If the system displays a message indicating that the request to create workload policy \**\* is successfully submitted, click **Back to Workload Scaling**. - -#. On the **Workload Scaling** tab page, you can view the newly created HPA policy. - -.. |image1| image:: /_static/images/en-us_image_0259716601.png -.. |image2| image:: /_static/images/en-us_image_0259714782.png +#. Click **Create**. diff --git a/umn/source/auto_scaling/scaling_a_workload/index.rst b/umn/source/auto_scaling/scaling_a_workload/index.rst index f469471..b6508fa 100644 --- a/umn/source/auto_scaling/scaling_a_workload/index.rst +++ b/umn/source/auto_scaling/scaling_a_workload/index.rst @@ -1,14 +1,13 @@ -:original_name: cce_01_0293.html +:original_name: cce_10_0293.html -.. _cce_01_0293: +.. _cce_10_0293: Scaling a Workload ================== -- :ref:`Workload Scaling Mechanisms ` -- :ref:`Creating an HPA Policy for Workload Auto Scaling ` -- :ref:`Managing Workload Scaling Policies ` -- :ref:`Switching from AOM to HPA for Auto Scaling ` +- :ref:`Workload Scaling Mechanisms ` +- :ref:`Creating an HPA Policy for Workload Auto Scaling ` +- :ref:`Managing Workload Scaling Policies ` .. toctree:: :maxdepth: 1 @@ -17,4 +16,3 @@ Scaling a Workload workload_scaling_mechanisms creating_an_hpa_policy_for_workload_auto_scaling managing_workload_scaling_policies - switching_from_aom_to_hpa_for_auto_scaling diff --git a/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst b/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst index 0acbde9..d38baef 100644 --- a/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst +++ b/umn/source/auto_scaling/scaling_a_workload/managing_workload_scaling_policies.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0083.html +:original_name: cce_10_0083.html -.. _cce_01_0083: +.. _cce_10_0083: Managing Workload Scaling Policies ================================== @@ -15,12 +15,17 @@ Checking an HPA Policy You can view the rules, status, and events of an HPA policy and handle exceptions based on the error information displayed. -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Workload Scaling** tab page, click |image1| in front of the target policy. +#. Log in to the CCE console and access the cluster console. +#. In the navigation pane, choose **Workload Scaling**. On the **HPA Policies** tab page, click |image1| next to the target HPA policy. #. In the expanded area, you can view the **Rules**, **Status**, and **Events** tab pages. If the policy is abnormal, locate and rectify the fault based on the error information. .. note:: - You can also view the created HPA policy on the workload details page. Log in to the CCE console, choose **Workloads > Deployments** or **Workloads > StatefulSets** in the navigation pane, and choose **More** > **Scaling** in the **Operation** column. On the workload details page, click the **Scaling** tab. You can see the **Auto Scaling-HPA** pane, as well as the HPA policy you have configured on the **Auto Scaling** page. + You can also view the created HPA policy on the workload details page. + + a. Log in to the CCE console and access the cluster console. + b. In the navigation pane, choose **Workloads**. Click the workload name to view its details. + c. On the workload details page, swich to the **Auto Scaling** tab page to view the HPA policies. You can also view the scaling policies you configured in **Workload Scaling**. .. table:: **Table 1** Event types and names @@ -59,29 +64,24 @@ Updating an HPA Policy An HPA policy is used as an example. -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Workload Scaling** tab page, click **Update** in the **Operation** column of the policy to be updated. -#. On the **Update HPA Policy** page displayed, set the policy parameters listed in :ref:`Table 1 `. +#. Log in to the CCE console and access the cluster console. +#. In the navigation pane, choose **Workload Scaling**. Click **Update** in the **Operation** column of the target policy. +#. On the **Update HPA Policy** page displayed, set the policy parameters listed in :ref:`Table 1 `. #. Click **Update**. -Cloning an HPA Policy ---------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Workload Scaling** tab page, click **Clone** in the **Operation** column of the target policy. -#. For example, for an HPA policy, on the **Create HPA Policy** page, you can view that parameters such as **Pod Range**, **Cooldown Period**, and **Rules** have been cloned. Add or modify other policy parameters as needed. -#. Click **Create** to complete policy cloning. On the **Workload Scaling** tab page, you can view the cloned policy in the policy list. - Editing the YAML File (HPA Policy) ---------------------------------- -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Workload Scaling** tab page, choose **More** > **Edit YAML** in the **Operation** column of the target policy. +#. Log in to the CCE console and access the cluster console. +#. In the navigation pane, choose **Workload Scaling**. Click **More > Edit YAML** in the **Operation** column of the target HPA policy. #. In the **Edit YAML** dialog box displayed, edit or download the YAML file. #. Click the close button in the upper right corner. Deleting an HPA Policy ---------------------- -#. Log in to the CCE console. In the navigation pane, choose **Auto Scaling**. On the **Workload Scaling** tab page, choose **More** > **Delete** in the **Operation** column of the target policy. -#. In the **Delete HPA Policy** dialog box displayed, confirm whether to delete the HPA policy. -#. Click **Yes** to delete the policy. +#. Log in to the CCE console and access the cluster console. +#. In the navigation pane, choose **Workload Scaling**. Click **Delete** in the **Operation** column of the target policy. +#. In the dialog box displayed, click **Yes**. -.. |image1| image:: /_static/images/en-us_image_0254985211.png +.. |image1| image:: /_static/images/en-us_image_0000001244261103.png diff --git a/umn/source/auto_scaling/scaling_a_workload/switching_from_aom_to_hpa_for_auto_scaling.rst b/umn/source/auto_scaling/scaling_a_workload/switching_from_aom_to_hpa_for_auto_scaling.rst deleted file mode 100644 index 8e664bb..0000000 --- a/umn/source/auto_scaling/scaling_a_workload/switching_from_aom_to_hpa_for_auto_scaling.rst +++ /dev/null @@ -1,14 +0,0 @@ -:original_name: cce_01_0395.html - -.. _cce_01_0395: - -Switching from AOM to HPA for Auto Scaling -========================================== - -CCE clusters of v1.15 or earlier support workload scaling based on AOM monitoring data. This function is no longer supported in CCE clusters of v1.17 or later. - -If you have configured auto scaling based on AOM, you can switch to :ref:`HPA ` policies after your cluster is upgraded to v1.17. Note the following differences during the switchover: - -- In AOM-based auto scaling, resource usage rate is calculated based on the limit of a workload, from 0% to 100%. - -For example, if the memory request of a workload is 2 GB and the memory limit is 16 GB, a scale-out is triggered as long as the memory utilization reaches 50% of the limit (8 GB) in AOM-based auto scaling. In HPA-based scaling, you need to set the memory usage rate to 400% (16 x 50%/2) to trigger the same scaling. diff --git a/umn/source/auto_scaling/scaling_a_workload/workload_scaling_mechanisms.rst b/umn/source/auto_scaling/scaling_a_workload/workload_scaling_mechanisms.rst index 29fe78c..352265b 100644 --- a/umn/source/auto_scaling/scaling_a_workload/workload_scaling_mechanisms.rst +++ b/umn/source/auto_scaling/scaling_a_workload/workload_scaling_mechanisms.rst @@ -1,14 +1,10 @@ -:original_name: cce_01_0290.html +:original_name: cce_10_0290.html -.. _cce_01_0290: +.. _cce_10_0290: Workload Scaling Mechanisms =========================== -.. note:: - - **Scaling policy priority**: If you do not manually adjust the number of pods, auto scaling policies will take effect for resource scheduling. If :ref:`manual scaling ` is triggered, auto scaling policies will be temporarily invalid. - How HPA Works ------------- diff --git a/umn/source/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst b/umn/source/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst new file mode 100644 index 0000000..5acb210 --- /dev/null +++ b/umn/source/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst @@ -0,0 +1,385 @@ +:original_name: cce_10_0300.html + +.. _cce_10_0300: + +Using HPA and CA for Auto Scaling of Workloads and Nodes +======================================================== + +Scenario +-------- + +The best way to handle surging traffic is to automatically adjust the number of machines based on the traffic volume or resource usage, which is called scaling. + +In CCE, the resources that can be used by containers are fixed during application deployment. Therefore, in auto scaling, pods are scaled first. The node resource usage increases only after the number of pods increases. Then, nodes can be scaled based on the node resource usage. How to configure auto scaling in CCE? + +Solution +-------- + +Two major auto scaling policies are HPA (Horizontal Pod Autoscaling) and CA (Cluster AutoScaling). HPA is for workload auto scaling and CA is for node auto scaling. + +HPA and CA work with each other. HPA requires sufficient cluster resources for successful scaling. When the cluster resources are insufficient, CA is needed to add nodes. If HPA reduces workloads, the cluster will have a large number of idle resources. In this case, CA needs to release nodes to avoid resource waste. + +As shown in :ref:`Figure 1 `, HPA performs scale-out based on the monitoring metrics. When cluster resources are insufficient, newly created pods are in Pending state. CA then checks these pending pods and selects the most appropriate node pool based on the configured scaling policy to scale out the node pool. + +.. _cce_10_0300__cce_bestpractice_00282_fig6540132372015: + +.. figure:: /_static/images/en-us_image_0000001290111529.png + :alt: **Figure 1** HPA and CA working flows + + **Figure 1** HPA and CA working flows + +Using HPA and CA can easily implement auto scaling in most scenarios. In addition, the scaling process of nodes and pods can be easily observed. + +This section uses an example to describe the auto scaling process using HPA and CA policies together. + +Preparations +------------ + +#. Create a cluster with one node. The node should have 2 cores of CPU and 4 GB of memory, or a higher specification, as well as an EIP to allow external access. If no EIP is bound to the node during node creation, you can manually bind one on the ECS console after creating the node. +#. Install add-ons for the cluster. + + - autoscaler: node scaling add-on + - metrics-server: an aggregator of resource usage data in a Kubernetes cluster. It can collect measurement data of major Kubernetes resources, such as pods, nodes, containers, and Services. + +#. Log in to the cluster node and run a computing-intensive application. When a user sends a request, the result needs to be calculated before being returned to the user. + + a. Create a PHP file named **index.php** to calculate the square root of the request for 1,000,000 times before returning **OK!**. + + .. code-block:: + + vi index.php + + Example file content: + + .. code-block:: + + + + b. Compile a Dockerfile to build an image. + + .. code-block:: + + vi Dockerfile + + Example Dockerfile: + + .. code-block:: + + FROM php:5-apache + COPY index.php /var/www/html/index.php + RUN chmod a+rx index.php + + c. Run the following command to build an image named **hpa-example** with the tag **latest**. + + .. code-block:: + + docker build -t hpa-example:latest . + + d. .. _cce_10_0300__cce_bestpractice_00282_li108181514125: + + (Optional) Log in to the SWR console, choose **Organization Management** in the navigation pane, and click **Create Organization** in the upper right corner to create an organization. + + Skip this step if you already have an organization. + + e. .. _cce_10_0300__cce_bestpractice_00282_li187221141362: + + In the navigation pane, choose **My Images** and then click **Upload Through Client**. On the page displayed, click **Generate a temporary login command** and click |image1| to copy the command. + + f. Run the login command copied in the previous step on the cluster node. If the login is successful, the message "Login Succeeded" is displayed. + + g. Tag the hpa-example image. + + **docker tag** **[Image name 1:Tag 1]** **[Image repository address]/[Organization name]/[Image name 2:Tag 2]** + + - **[Image name 1:Tag 1]**: name and tag of the local image to be uploaded. + - **[Image repository address]**: The domain name at the end of the login command in :ref:`5 ` is the image repository address, which can be obtained on the SWR console. + - **[Organization name]**: name of the organization created in :ref:`4 `. + - **[Image name 2:Tag 2]**: desired image name and tag to be displayed on the SWR console. + + Example: + + **docker tag hpa-example:latest swr.eu-de.otc.t-systems.com/group/hpa-example:latest** + + h. Push the image to the image repository. + + **docker push** **[Image repository address]/[Organization name]/[Image name 2:Tag 2]** + + Example: + + **docker push swr.eu-de.otc.t-systems.com/group/hpa-example:latest** + + The following information will be returned upon a successful push: + + .. code-block:: + + 6d6b9812c8ae: Pushed + ... + fe4c16cbf7a4: Pushed + latest: digest: sha256:eb7e3bbd*** size: ** + + To view the pushed image, go to the SWR console and refresh the **My Images** page. + +Creating a Node Pool and a Node Scaling Policy +---------------------------------------------- + +#. Log in to the CCE console, access the created cluster, click **Nodes** on the left, click the **Node Pools** tab, and click **Create Node Pool** in the upper right corner. + +#. Set node pool parameters, add a node with 2 vCPUs and 4 GB memory, and enable auto scaling. + + - **Nodes**: Set it to **1**, indicating that one node is created by default when a node pool is created. + - **Auto Scaling**: Enable the option, meaning that nodes will be automatically created or deleted in the node pool based on the cluster loads. + - **Max. Nodes**: Set it to **5**, indicating the maximum number of nodes in a node pool. + - **Specifications**: 2 vCPUs \| 4 GiB + + Retain the defaults for other parameters. For details, see `Creating a Node Pool `__. + +#. Click **Add-ons** on the left of the cluster console, click **Edit** under the autoscaler add-on, modify the add-on configuration, enable **Auto node scale-in**, and configure scale-in parameters. For example, trigger scale-in when the node resource utilization is less than 50%. + + |image2| + + After the preceding configurations, scale-out is performed based on the pending status of the pod and scale-in is triggered when the node resource utilization decreases. + +#. Click **Node Scaling** on the left of the cluster console and click **Create Node Scaling Policy** in the upper right corner. Node scaling policies added here trigger scale-out based on the CPU/memory allocation rate or periodically. + + As shown in the following figure, when the cluster CPU allocation rate is greater than 70%, one node will be added. A node scaling policy needs to be associated with a node pool. Multiple node pools can be associated. When you need to scale nodes, node with proper specifications will be added or reduced from the node pool based on the minimum waste principle. For details, see `Creating a Node Scaling Policy `__. + + |image3| + +Creating a Workload +------------------- + +Use the hpa-example image to create a Deployment with one replica. The image path is related to the organization uploaded to the SWR repository and needs to be replaced with the actual value. + +.. code-block:: + + kind: Deployment + apiVersion: apps/v1 + metadata: + name: hpa-example + spec: + replicas: 1 + selector: + matchLabels: + app: hpa-example + template: + metadata: + labels: + app: hpa-example + spec: + containers: + - name: container-1 + image: 'hpa-example:latest ' # Replace it with the address of the image you uploaded to SWR. + resources: + limits: # The value of limits must be the same as that of requests to prevent flapping during scaling. + cpu: 500m + memory: 200Mi + requests: + cpu: 500m + memory: 200Mi + imagePullSecrets: + - name: default-secret + +Then, create a NodePort Service for the workload so that the workload can be accessed from external networks. + +.. code-block:: + + kind: Service + apiVersion: v1 + metadata: + name: hpa-example + spec: + ports: + - name: cce-service-0 + protocol: TCP + port: 80 + targetPort: 80 + nodePort: 31144 + selector: + app: hpa-example + type: NodePort + +Creating an HPA Policy +---------------------- + +Create an HPA policy. As shown below, the policy is associated with the hpa-example workload, and the target CPU usage is 50%. + +There are two other annotations. One annotation defines the CPU thresholds, indicating that scaling is not performed when the CPU usage is between 30% and 70% to prevent impact caused by slight fluctuation. The other is the scaling time window, indicating that after the policy is successfully executed, a scaling operation will not be triggered again in this cooling interval to prevent impact caused by short-term fluctuation. + +.. code-block:: + + apiVersion: autoscaling/v2 + kind: HorizontalPodAutoscaler + metadata: + name: hpa-policy + annotations: + extendedhpa.metrics: '[{"type":"Resource","name":"cpu","targetType":"Utilization","targetRange":{"low":"30","high":"70"}}]' + extendedhpa.option: '{"downscaleWindow":"5m","upscaleWindow":"3m"}' + spec: + scaleTargetRef: + kind: Deployment + name: hpa-example + apiVersion: apps/v1 + minReplicas: 1 + maxReplicas: 100 + metrics: + - type: Resource + resource: + name: cpu + targetAverageUtilization: 50 + +Set the parameters as follows if you are using the console. + +|image4| + +Observing the Auto Scaling Process +---------------------------------- + +#. Check the cluster node status. In the following example, there are two nodes. + + .. code-block:: + + # kubectl get node + NAME STATUS ROLES AGE VERSION + 192.168.0.183 Ready 2m20s v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.26 Ready 55m v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + + Check the HPA policy. The CPU usage of the target workload is 0%. + + .. code-block:: + + # kubectl get hpa hpa-policy + NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE + hpa-policy Deployment/hpa-example 0%/50% 1 100 1 4m + +#. Run the following command to access the workload. In the following command, {ip:port} indicates the access address of the workload, which can be queried on the workload details page. + + **while true;do wget -q -O- http://**\ *{ip:port}*\ **; done** + + .. note:: + + If no EIP is displayed, the cluster node has not been assigned any EIP. You need to create one, bind it to the node, and synchronize node data. . + + Observe the scaling process of the workload. + + .. code-block:: + + # kubectl get hpa hpa-policy --watch + NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE + hpa-policy Deployment/hpa-example 0%/50% 1 100 1 4m + hpa-policy Deployment/hpa-example 190%/50% 1 100 1 4m23s + hpa-policy Deployment/hpa-example 190%/50% 1 100 4 4m31s + hpa-policy Deployment/hpa-example 200%/50% 1 100 4 5m16s + hpa-policy Deployment/hpa-example 200%/50% 1 100 4 6m16s + hpa-policy Deployment/hpa-example 85%/50% 1 100 4 7m16s + hpa-policy Deployment/hpa-example 81%/50% 1 100 4 8m16s + hpa-policy Deployment/hpa-example 81%/50% 1 100 7 8m31s + hpa-policy Deployment/hpa-example 57%/50% 1 100 7 9m16s + hpa-policy Deployment/hpa-example 51%/50% 1 100 7 10m + hpa-policy Deployment/hpa-example 58%/50% 1 100 7 11m + + You can see that the CPU usage of the workload is 190% at 4m23s, which exceeds the target value. In this case, scaling is triggered to expand the workload to four replicas/pods. In the subsequent several minutes, the CPU usage does not decrease until 7m16s. This is because the new pods may not be successfully created. The possible cause is that resources are insufficient and the pods are in Pending state. During this period, nodes are added. + + At 7m16s, the CPU usage decreases, indicating that the pods are successfully created and start to bear traffic. The CPU usage decreases to 81% at 8m, still greater than the target value (50%) and the high threshold (70%). Therefore, 7 pods are added at 9m16s, and the CPU usage decreases to 51%, which is within the range of 30% to 70%. From then on, the number of pods remains 7. + + In the following output, you can see the workload scaling process and the time when the HPA policy takes effect. + + .. code-block:: + + # kubectl describe deploy hpa-example + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal ScalingReplicaSet 25m deployment-controller Scaled up replica set hpa-example-79dd795485 to 1 + Normal ScalingReplicaSet 20m deployment-controller Scaled up replica set hpa-example-79dd795485 to 4 + Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set hpa-example-79dd795485 to 7 + # kubectl describe hpa hpa-policy + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal SuccessfulRescale 20m horizontal-pod-autoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target + Normal SuccessfulRescale 16m horizontal-pod-autoscaler New size: 7; reason: cpu resource utilization (percentage of request) above target + + Check the number of nodes. The following output shows that two nodes are added. + + .. code-block:: + + # kubectl get node + NAME STATUS ROLES AGE VERSION + 192.168.0.120 Ready 3m5s v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.136 Ready 6m58s v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.183 Ready 18m v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.26 Ready 71m v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + + You can also view the scaling history on the console. For example, the CA policy is executed once when the CPU allocation rate in the cluster is greater than 70%, and the number of nodes in the node pool is increased from 2 to 3. The new node is automatically added by autoscaler based on the pending state of pods in the initial phase of HPA. + + The node scaling process is as follows: + + a. After the number of pods changes to 4, the pods are in Pending state due to insufficient resources. As a result, the default scale-out policy of the autoscaler add-on is triggered, and the number of nodes is increased by one. + b. The second node scale-out is triggered because the CPU allocation rate in the cluster is greater than 70%. As a result, the number of nodes is increased by one, which is recorded in the scaling history on the console. Scaling based on the allocation rate ensures that the cluster has sufficient resources. + +#. Stop accessing the workload and check the number of pods. + + .. code-block:: + + # kubectl get hpa hpa-policy --watch + NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE + hpa-policy Deployment/hpa-example 50%/50% 1 100 7 12m + hpa-policy Deployment/hpa-example 21%/50% 1 100 7 13m + hpa-policy Deployment/hpa-example 0%/50% 1 100 7 14m + hpa-policy Deployment/hpa-example 0%/50% 1 100 7 18m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 18m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 23m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 23m + hpa-policy Deployment/hpa-example 0%/50% 1 100 1 23m + + You can see that the CPU usage is 21% at 13m. The number of pods is reduced to 3 at 18m, and then reduced to 1 at 23m. + + In the following output, you can see the workload scaling process and the time when the HPA policy takes effect. + + .. code-block:: + + # kubectl describe deploy hpa-example + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal ScalingReplicaSet 25m deployment-controller Scaled up replica set hpa-example-79dd795485 to 1 + Normal ScalingReplicaSet 20m deployment-controller Scaled up replica set hpa-example-79dd795485 to 4 + Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set hpa-example-79dd795485 to 7 + Normal ScalingReplicaSet 6m28s deployment-controller Scaled down replica set hpa-example-79dd795485 to 3 + Normal ScalingReplicaSet 72s deployment-controller Scaled down replica set hpa-example-79dd795485 to 1 + # kubectl describe hpa hpa-policy + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal SuccessfulRescale 20m horizontal-pod-autoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target + Normal SuccessfulRescale 16m horizontal-pod-autoscaler New size: 7; reason: cpu resource utilization (percentage of request) above target + Normal SuccessfulRescale 6m45s horizontal-pod-autoscaler New size: 3; reason: All metrics below target + Normal SuccessfulRescale 90s horizontal-pod-autoscaler New size: 1; reason: All metrics below target + + You can also view the HPA policy execution history on the console. Wait until the one node is reduced. + + The reason why the other two nodes in the node pool are not reduced is that they both have pods in the kube-system namespace (and these pods are not created by DaemonSets). For details, see `Node Scaling Mechanisms `__. + +Summary +------- + +Using HPA and CA can easily implement auto scaling in most scenarios. In addition, the scaling process of nodes and pods can be easily observed. + +.. |image1| image:: /_static/images/en-us_image_0000001360670117.png +.. |image2| image:: /_static/images/en-us_image_0000001533181077.png +.. |image3| image:: /_static/images/en-us_image_0000001482541956.png +.. |image4| image:: /_static/images/en-us_image_0000001482701968.png diff --git a/umn/source/best_practice/auto_scaling/index.rst b/umn/source/best_practice/auto_scaling/index.rst new file mode 100644 index 0000000..eff15db --- /dev/null +++ b/umn/source/best_practice/auto_scaling/index.rst @@ -0,0 +1,14 @@ +:original_name: cce_bestpractice_0090.html + +.. _cce_bestpractice_0090: + +Auto Scaling +============ + +- :ref:`Using HPA and CA for Auto Scaling of Workloads and Nodes ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes diff --git a/umn/source/best_practice/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst b/umn/source/best_practice/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst new file mode 100644 index 0000000..1904a80 --- /dev/null +++ b/umn/source/best_practice/auto_scaling/using_hpa_and_ca_for_auto_scaling_of_workloads_and_nodes.rst @@ -0,0 +1,385 @@ +:original_name: cce_bestpractice_00282.html + +.. _cce_bestpractice_00282: + +Using HPA and CA for Auto Scaling of Workloads and Nodes +======================================================== + +Scenario +-------- + +The best way to handle surging traffic is to automatically adjust the number of machines based on the traffic volume or resource usage, which is called scaling. + +In CCE, the resources that can be used by containers are fixed during application deployment. Therefore, in auto scaling, pods are scaled first. The node resource usage increases only after the number of pods increases. Then, nodes can be scaled based on the node resource usage. How to configure auto scaling in CCE? + +Solution +-------- + +Two major auto scaling policies are HPA (Horizontal Pod Autoscaling) and CA (Cluster AutoScaling). HPA is for workload auto scaling and CA is for node auto scaling. + +HPA and CA work with each other. HPA requires sufficient cluster resources for successful scaling. When the cluster resources are insufficient, CA is needed to add nodes. If HPA reduces workloads, the cluster will have a large number of idle resources. In this case, CA needs to release nodes to avoid resource waste. + +As shown in :ref:`Figure 1 `, HPA performs scale-out based on the monitoring metrics. When cluster resources are insufficient, newly created pods are in Pending state. CA then checks these pending pods and selects the most appropriate node pool based on the configured scaling policy to scale out the node pool. + +.. _cce_bestpractice_00282__fig6540132372015: + +.. figure:: /_static/images/en-us_image_0000001290111529.png + :alt: **Figure 1** HPA and CA working flows + + **Figure 1** HPA and CA working flows + +Using HPA and CA can easily implement auto scaling in most scenarios. In addition, the scaling process of nodes and pods can be easily observed. + +This section uses an example to describe the auto scaling process using HPA and CA policies together. + +Preparations +------------ + +#. Create a cluster with one node. The node should have 2 cores of CPU and 4 GB of memory, or a higher specification, as well as an EIP to allow external access. If no EIP is bound to the node during node creation, you can manually bind one on the ECS console after creating the node. +#. Install add-ons for the cluster. + + - autoscaler: node scaling add-on + - metrics-server: an aggregator of resource usage data in a Kubernetes cluster. It can collect measurement data of major Kubernetes resources, such as pods, nodes, containers, and Services. + +#. Log in to the cluster node and run a computing-intensive application. When a user sends a request, the result needs to be calculated before being returned to the user. + + a. Create a PHP file named **index.php** to calculate the square root of the request for 1,000,000 times before returning **OK!**. + + .. code-block:: + + vi index.php + + Example file content: + + .. code-block:: + + + + b. Compile a Dockerfile to build an image. + + .. code-block:: + + vi Dockerfile + + Example Dockerfile: + + .. code-block:: + + FROM php:5-apache + COPY index.php /var/www/html/index.php + RUN chmod a+rx index.php + + c. Run the following command to build an image named **hpa-example** with the tag **latest**. + + .. code-block:: + + docker build -t hpa-example:latest . + + d. .. _cce_bestpractice_00282__li108181514125: + + (Optional) Log in to the SWR console, choose **Organization Management** in the navigation pane, and click **Create Organization** in the upper right corner to create an organization. + + Skip this step if you already have an organization. + + e. .. _cce_bestpractice_00282__li187221141362: + + In the navigation pane, choose **My Images** and then click **Upload Through Client**. On the page displayed, click **Generate a temporary login command** and click |image1| to copy the command. + + f. Run the login command copied in the previous step on the cluster node. If the login is successful, the message "Login Succeeded" is displayed. + + g. Tag the hpa-example image. + + **docker tag** **[Image name 1:Tag 1]** **[Image repository address]/[Organization name]/[Image name 2:Tag 2]** + + - **[Image name 1:Tag 1]**: name and tag of the local image to be uploaded. + - **[Image repository address]**: The domain name at the end of the login command in :ref:`5 ` is the image repository address, which can be obtained on the SWR console. + - **[Organization name]**: name of the organization created in :ref:`4 `. + - **[Image name 2:Tag 2]**: desired image name and tag to be displayed on the SWR console. + + Example: + + **docker tag hpa-example:latest swr.eu-de.otc.t-systems.com/group/hpa-example:latest** + + h. Push the image to the image repository. + + **docker push** **[Image repository address]/[Organization name]/[Image name 2:Tag 2]** + + Example: + + **docker push swr.eu-de.otc.t-systems.com/group/hpa-example:latest** + + The following information will be returned upon a successful push: + + .. code-block:: + + 6d6b9812c8ae: Pushed + ... + fe4c16cbf7a4: Pushed + latest: digest: sha256:eb7e3bbd*** size: ** + + To view the pushed image, go to the SWR console and refresh the **My Images** page. + +Creating a Node Pool and a Node Scaling Policy +---------------------------------------------- + +#. Log in to the CCE console, access the created cluster, click **Nodes** on the left, click the **Node Pools** tab, and click **Create Node Pool** in the upper right corner. + +#. Set node pool parameters, add a node with 2 vCPUs and 4 GB memory, and enable auto scaling. + + - **Nodes**: Set it to **1**, indicating that one node is created by default when a node pool is created. + - **Auto Scaling**: Enable the option, meaning that nodes will be automatically created or deleted in the node pool based on the cluster loads. + - **Max. Nodes**: Set it to **5**, indicating the maximum number of nodes in a node pool. + - **Specifications**: 2 vCPUs \| 4 GiB + + Retain the defaults for other parameters. For details, see `Creating a Node Pool `__. + +#. Click **Add-ons** on the left of the cluster console, click **Edit** under the autoscaler add-on, modify the add-on configuration, enable **Auto node scale-in**, and configure scale-in parameters. For example, trigger scale-in when the node resource utilization is less than 50%. + + |image2| + + After the preceding configurations, scale-out is performed based on the pending status of the pod and scale-in is triggered when the node resource utilization decreases. + +#. Click **Node Scaling** on the left of the cluster console and click **Create Node Scaling Policy** in the upper right corner. Node scaling policies added here trigger scale-out based on the CPU/memory allocation rate or periodically. + + As shown in the following figure, when the cluster CPU allocation rate is greater than 70%, one node will be added. A node scaling policy needs to be associated with a node pool. Multiple node pools can be associated. When you need to scale nodes, node with proper specifications will be added or reduced from the node pool based on the minimum waste principle. For details, see `Creating a Node Scaling Policy `__. + + |image3| + +Creating a Workload +------------------- + +Use the hpa-example image to create a Deployment with one replica. The image path is related to the organization uploaded to the SWR repository and needs to be replaced with the actual value. + +.. code-block:: + + kind: Deployment + apiVersion: apps/v1 + metadata: + name: hpa-example + spec: + replicas: 1 + selector: + matchLabels: + app: hpa-example + template: + metadata: + labels: + app: hpa-example + spec: + containers: + - name: container-1 + image: 'hpa-example:latest ' # Replace it with the address of the image you uploaded to SWR. + resources: + limits: # The value of limits must be the same as that of requests to prevent flapping during scaling. + cpu: 500m + memory: 200Mi + requests: + cpu: 500m + memory: 200Mi + imagePullSecrets: + - name: default-secret + +Then, create a NodePort Service for the workload so that the workload can be accessed from external networks. + +.. code-block:: + + kind: Service + apiVersion: v1 + metadata: + name: hpa-example + spec: + ports: + - name: cce-service-0 + protocol: TCP + port: 80 + targetPort: 80 + nodePort: 31144 + selector: + app: hpa-example + type: NodePort + +Creating an HPA Policy +---------------------- + +Create an HPA policy. As shown below, the policy is associated with the hpa-example workload, and the target CPU usage is 50%. + +There are two other annotations. One annotation defines the CPU thresholds, indicating that scaling is not performed when the CPU usage is between 30% and 70% to prevent impact caused by slight fluctuation. The other is the scaling time window, indicating that after the policy is successfully executed, a scaling operation will not be triggered again in this cooling interval to prevent impact caused by short-term fluctuation. + +.. code-block:: + + apiVersion: autoscaling/v2 + kind: HorizontalPodAutoscaler + metadata: + name: hpa-policy + annotations: + extendedhpa.metrics: '[{"type":"Resource","name":"cpu","targetType":"Utilization","targetRange":{"low":"30","high":"70"}}]' + extendedhpa.option: '{"downscaleWindow":"5m","upscaleWindow":"3m"}' + spec: + scaleTargetRef: + kind: Deployment + name: hpa-example + apiVersion: apps/v1 + minReplicas: 1 + maxReplicas: 100 + metrics: + - type: Resource + resource: + name: cpu + targetAverageUtilization: 50 + +Set the parameters as follows if you are using the console. + +|image4| + +Observing the Auto Scaling Process +---------------------------------- + +#. Check the cluster node status. In the following example, there are two nodes. + + .. code-block:: + + # kubectl get node + NAME STATUS ROLES AGE VERSION + 192.168.0.183 Ready 2m20s v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.26 Ready 55m v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + + Check the HPA policy. The CPU usage of the target workload is 0%. + + .. code-block:: + + # kubectl get hpa hpa-policy + NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE + hpa-policy Deployment/hpa-example 0%/50% 1 100 1 4m + +#. Run the following command to access the workload. In the following command, {ip:port} indicates the access address of the workload, which can be queried on the workload details page. + + **while true;do wget -q -O- http://**\ *{ip:port}*\ **; done** + + .. note:: + + If no EIP is displayed, the cluster node has not been assigned any EIP. You need to create one, bind it to the node, and synchronize node data. . + + Observe the scaling process of the workload. + + .. code-block:: + + # kubectl get hpa hpa-policy --watch + NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE + hpa-policy Deployment/hpa-example 0%/50% 1 100 1 4m + hpa-policy Deployment/hpa-example 190%/50% 1 100 1 4m23s + hpa-policy Deployment/hpa-example 190%/50% 1 100 4 4m31s + hpa-policy Deployment/hpa-example 200%/50% 1 100 4 5m16s + hpa-policy Deployment/hpa-example 200%/50% 1 100 4 6m16s + hpa-policy Deployment/hpa-example 85%/50% 1 100 4 7m16s + hpa-policy Deployment/hpa-example 81%/50% 1 100 4 8m16s + hpa-policy Deployment/hpa-example 81%/50% 1 100 7 8m31s + hpa-policy Deployment/hpa-example 57%/50% 1 100 7 9m16s + hpa-policy Deployment/hpa-example 51%/50% 1 100 7 10m + hpa-policy Deployment/hpa-example 58%/50% 1 100 7 11m + + You can see that the CPU usage of the workload is 190% at 4m23s, which exceeds the target value. In this case, scaling is triggered to expand the workload to four replicas/pods. In the subsequent several minutes, the CPU usage does not decrease until 7m16s. This is because the new pods may not be successfully created. The possible cause is that resources are insufficient and the pods are in Pending state. During this period, nodes are added. + + At 7m16s, the CPU usage decreases, indicating that the pods are successfully created and start to bear traffic. The CPU usage decreases to 81% at 8m, still greater than the target value (50%) and the high threshold (70%). Therefore, 7 pods are added at 9m16s, and the CPU usage decreases to 51%, which is within the range of 30% to 70%. From then on, the number of pods remains 7. + + In the following output, you can see the workload scaling process and the time when the HPA policy takes effect. + + .. code-block:: + + # kubectl describe deploy hpa-example + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal ScalingReplicaSet 25m deployment-controller Scaled up replica set hpa-example-79dd795485 to 1 + Normal ScalingReplicaSet 20m deployment-controller Scaled up replica set hpa-example-79dd795485 to 4 + Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set hpa-example-79dd795485 to 7 + # kubectl describe hpa hpa-policy + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal SuccessfulRescale 20m horizontal-pod-autoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target + Normal SuccessfulRescale 16m horizontal-pod-autoscaler New size: 7; reason: cpu resource utilization (percentage of request) above target + + Check the number of nodes. The following output shows that two nodes are added. + + .. code-block:: + + # kubectl get node + NAME STATUS ROLES AGE VERSION + 192.168.0.120 Ready 3m5s v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.136 Ready 6m58s v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.183 Ready 18m v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + 192.168.0.26 Ready 71m v1.17.9-r0-CCE21.1.1.3.B001-17.36.8 + + You can also view the scaling history on the console. For example, the CA policy is executed once when the CPU allocation rate in the cluster is greater than 70%, and the number of nodes in the node pool is increased from 2 to 3. The new node is automatically added by autoscaler based on the pending state of pods in the initial phase of HPA. + + The node scaling process is as follows: + + a. After the number of pods changes to 4, the pods are in Pending state due to insufficient resources. As a result, the default scale-out policy of the autoscaler add-on is triggered, and the number of nodes is increased by one. + b. The second node scale-out is triggered because the CPU allocation rate in the cluster is greater than 70%. As a result, the number of nodes is increased by one, which is recorded in the scaling history on the console. Scaling based on the allocation rate ensures that the cluster has sufficient resources. + +#. Stop accessing the workload and check the number of pods. + + .. code-block:: + + # kubectl get hpa hpa-policy --watch + NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE + hpa-policy Deployment/hpa-example 50%/50% 1 100 7 12m + hpa-policy Deployment/hpa-example 21%/50% 1 100 7 13m + hpa-policy Deployment/hpa-example 0%/50% 1 100 7 14m + hpa-policy Deployment/hpa-example 0%/50% 1 100 7 18m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 18m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 19m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 23m + hpa-policy Deployment/hpa-example 0%/50% 1 100 3 23m + hpa-policy Deployment/hpa-example 0%/50% 1 100 1 23m + + You can see that the CPU usage is 21% at 13m. The number of pods is reduced to 3 at 18m, and then reduced to 1 at 23m. + + In the following output, you can see the workload scaling process and the time when the HPA policy takes effect. + + .. code-block:: + + # kubectl describe deploy hpa-example + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal ScalingReplicaSet 25m deployment-controller Scaled up replica set hpa-example-79dd795485 to 1 + Normal ScalingReplicaSet 20m deployment-controller Scaled up replica set hpa-example-79dd795485 to 4 + Normal ScalingReplicaSet 16m deployment-controller Scaled up replica set hpa-example-79dd795485 to 7 + Normal ScalingReplicaSet 6m28s deployment-controller Scaled down replica set hpa-example-79dd795485 to 3 + Normal ScalingReplicaSet 72s deployment-controller Scaled down replica set hpa-example-79dd795485 to 1 + # kubectl describe hpa hpa-policy + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal SuccessfulRescale 20m horizontal-pod-autoscaler New size: 4; reason: cpu resource utilization (percentage of request) above target + Normal SuccessfulRescale 16m horizontal-pod-autoscaler New size: 7; reason: cpu resource utilization (percentage of request) above target + Normal SuccessfulRescale 6m45s horizontal-pod-autoscaler New size: 3; reason: All metrics below target + Normal SuccessfulRescale 90s horizontal-pod-autoscaler New size: 1; reason: All metrics below target + + You can also view the HPA policy execution history on the console. Wait until the one node is reduced. + + The reason why the other two nodes in the node pool are not reduced is that they both have pods in the kube-system namespace (and these pods are not created by DaemonSets). For details, see `Node Scaling Mechanisms `__. + +Summary +------- + +Using HPA and CA can easily implement auto scaling in most scenarios. In addition, the scaling process of nodes and pods can be easily observed. + +.. |image1| image:: /_static/images/en-us_image_0000001360670117.png +.. |image2| image:: /_static/images/en-us_image_0000001533181077.png +.. |image3| image:: /_static/images/en-us_image_0000001482541956.png +.. |image4| image:: /_static/images/en-us_image_0000001482701968.png diff --git a/umn/source/reference/checklist_for_migrating_containerized_applications_to_the_cloud.rst b/umn/source/best_practice/checklist_for_deploying_containerized_applications_in_the_cloud.rst similarity index 55% rename from umn/source/reference/checklist_for_migrating_containerized_applications_to_the_cloud.rst rename to umn/source/best_practice/checklist_for_deploying_containerized_applications_in_the_cloud.rst index 36fa0f0..45ac85f 100644 --- a/umn/source/reference/checklist_for_migrating_containerized_applications_to_the_cloud.rst +++ b/umn/source/best_practice/checklist_for_deploying_containerized_applications_in_the_cloud.rst @@ -1,66 +1,66 @@ -:original_name: cce_faq_00006.html +:original_name: cce_bestpractice_00006.html -.. _cce_faq_00006: +.. _cce_bestpractice_00006: -Checklist for Migrating Containerized Applications to the Cloud +Checklist for Deploying Containerized Applications in the Cloud =============================================================== Overview -------- -Cloud Container Engine (CCE) provides highly scalable, high-performance, enterprise-class Kubernetes clusters and supports Docker containers. With CCE, you can easily deploy, manage, and scale out containerized applications. - -This checklist describes the system availability, data reliability, and O&M reliability of migrating containerized applications to the cloud. It contains check items, impact, FAQs, and examples, helping you migrate services to CCE and avoid application exceptions or cluster reconstruction caused by improper use. +Security, efficiency, stability, and availability are common requirements on all cloud services. To meet these requirements, the system availability, data reliability, and O&M stability must be perfectly coordinated. This checklist describes the check items for deploying containerized applications on the cloud to help you efficiently migrate services to CCE, reducing potential cluster or application exceptions caused by improper use. Check Items ----------- .. table:: **Table 1** System availability - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Category | Check Item | Type | Impact | - +==========+==============================================================================================================================================================================================================================+==================+=========================================================================================================================================================================================================================================================================================================+ - | Cluster | When creating a cluster, set **High Availability** to **Yes**. | Reliability | A cluster with **High Availability** set to **No** is a non-HA cluster with only one master. If the master node is faulty, the entire cluster will be unavailable. Therefore, you are advised to create an HA cluster in the production environment. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Before creating a cluster, determine the container network model that is suitable to the service scenario. | Network planning | Different container network models apply to different scenarios. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Before creating a cluster, plan the subnet CIDR block and container network CIDR block properly. | Network planning | If the range of the subnet and container network CIDR blocks is not properly set, the number of available nodes in the cluster will be less than the number of nodes supported by the cluster. Network planning has different constraints on different container network models. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Before creating a cluster, properly plan CIDR blocks for the related Direct Connect, peering connection, container network, service network, and subnet to avoid IP address conflicts. | Network planning | If CIDR blocks are not properly set and IP address conflicts occur, service access will be affected. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Workload | When creating a workload, set the upper and lower limits of CPU and memory resources. | Deployment | If the upper and lower resource limits are not set for an application, a resource leak of this application will make resources unavailable for other applications deployed on the same node. In addition, applications that do not have upper and lower resource limits cannot be accurately monitored. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | When creating an application, set the number of pods to more than two and set the scheduling policy based on service requirements. | Reliability | A single-pod application will be faulty if the node or pod is faulty. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Properly set affinity and anti-affinity. | Reliability | If affinity and anti-affinity are both configured for an application that provides Services externally, Services may fail to be accessed after the application is upgraded or restarted. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | When creating a workload, set the health check policy, that is, set the workload liveness probe and the readiness probe. | Reliability | If the two probes are not set, pods cannot detect service exceptions or automatically restart the service to restore it. This results in a situation where the pod status is normal but the service in the pod is abnormal. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | When creating a workload, set the pre-stop processing command (**Lifecycle** > **Pre-Stop**) to ensure that the services running in the pods can be completed in advance in the case of application upgrade or pod deletion. | Reliability | If the pre-stop processing command is not configured, the pod will be directly killed and services will be interrupted during application upgrade. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | When creating a Service, select an access mode based on service requirements. Currently, the following types of access modes are supported: intra-cluster access, intra-VPC access, and external access. | Deployment | If the access mode is not properly set, internal and external access may be in disorder and resources may be wasted. | - +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Category | Check Item | Type | Impact | + +==========+==============================================================================================================================================================================================================================+==================+============================================================================================================================================================================================================================================================================================+ + | Cluster | Before creating a cluster, properly plan the node network and container network based on service requirements to allow subsequent service expansion. | Network planning | If the subnet or container CIDR block where the cluster resides is small, the number of available nodes supported by the cluster may be less than required. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Before creating a cluster, properly plan CIDR blocks for the related Direct Connect, peering connection, container network, service network, and subnet to avoid IP address conflicts. | Network planning | If CIDR blocks are not properly set and IP address conflicts occur, service access will be affected. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | When a cluster is created, the default security group is automatically created and bound to the cluster. You can set custom security group rules based on service requirements. | Deployment | Security groups are key to security isolation. Improper security policy configuration may cause security risks and service connectivity problems. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Enable the multi-master node mode, and set the number of master nodes to **3** when creating a cluster. | Reliability | After the multi-master node mode is enabled, three master nodes will be created. If a master node is faulty, the cluster can still be available without affecting service functions. In commercial scenarios, it is advised to enable the multi-master node mode. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | When creating a cluster, select a proper network model, such as container tunnel network or VPC network. | Deployment | After a cluster is created, the network model cannot be changed. Exercise caution when selecting a network model. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Workload | When creating a workload, you need to set the CPU and memory limits to improve service robustness. | Deployment | When multiple applications are deployed on the same node, if the upper and lower resource limits are not set for an application, resource leakage occurs. As a result, resources cannot be allocated to other applications, and the application monitoring information will be inaccurate. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | When creating a workload, you can set probes for container health check, including **liveness probe** and **readiness probe**. | Reliability | If the health check function is not configured, a pod cannot detect service exceptions or automatically restart the service to restore it. This results in a situation where the pod status is normal but the service in the pod is abnormal. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | When creating a workload, select a proper access mode (Service). Currently, the following types of Services are supported: ClusterIP, NodePort, and LoadBalancer. | Deployment | Improper Service configuration may cause logic confusion for internal and external access and resource waste. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | When creating a workload, do not set the number of replicas for a single pod. Set a proper node scheduling policy based on your service requirements. | Reliability | For example, if the number of replicas of a single pod is set, the service will be abnormal when the node or pod is abnormal. To ensure that your pods can be successfully scheduled, ensure that the node has idle resources for container scheduling after you set the scheduling rule. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Properly set affinity and anti-affinity. | Reliability | If affinity and anti-affinity are both configured for an application that provides Services externally, Services may fail to be accessed after the application is upgraded or restarted. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | When creating a workload, set the pre-stop processing command (**Lifecycle** > **Pre-Stop**) to ensure that the services running in the pods can be completed in advance in the case of application upgrade or pod deletion. | Reliability | If the pre-stop processing command is not configured, the pod will be directly killed and services will be interrupted during application upgrade. | + +----------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. table:: **Table 2** Data reliability - +----------------------------+-------------------------------------------------------------------+-------------+----------------------------------------------------------------------------------------+ - | Category | Check Item | Type | Impact | - +============================+===================================================================+=============+========================================================================================+ - | Container data persistency | Store application data in the cloud, rather than on a local disk. | Reliability | If a node is faulty and cannot be restored, data on the local disk cannot be restored. | - +----------------------------+-------------------------------------------------------------------+-------------+----------------------------------------------------------------------------------------+ - | Backup | Back up application data. | Reliability | Data cannot be restored after being lost. | - +----------------------------+-------------------------------------------------------------------+-------------+----------------------------------------------------------------------------------------+ + +----------------------------+-----------------------------------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Category | Check Item | Type | Impact | + +============================+=================================================================+=============+================================================================================================================================================================================+ + | Container data persistency | Select a proper data volume type based on service requirements. | Reliability | When a node is faulty and cannot be recovered, data in the local disk cannot be recovered. Therefore, you are advised to use cloud storage volumes to ensure data reliability. | + +----------------------------+-----------------------------------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Backup | Back up application data. | Reliability | Data cannot be restored after being lost. | + +----------------------------+-----------------------------------------------------------------+-------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. table:: **Table 3** O&M reliability - +---------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Category | Check Item | Type | Impact | - +===============+================================================================================================================================================+============+====================================================================================================================================================================================================+ - | Project | The quotas of ECS, VPC, subnet, EIP, and EVS resources must meet customer requirements. | Deployment | If the quota is insufficient, resources will fail to be created. Specifically, users who have configured automatic capacity expansion must have sufficient resource quotas. | - +---------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Do not install private software or modify OS configurations on a cluster node. | Deployment | If private software is installed on a cluster node or OS configurations are modified, exceptions may occur on Kubernetes components on the node, making it unavailable for application deployment. | - +---------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Do not modify information about resources created by CCE, such as security groups and EVS disks. Resources created by CCE are labeled **cce**. | Deployment | CCE cluster functions may be abnormal. | - +---------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Proactive O&M | Configure alarm monitoring on AOM for the applications you deployed in CCE clusters. | Monitoring | If alarm monitoring is not configured, you cannot receive alarms when applications are faulty and need to manually locate the faults. | - +---------------+------------------------------------------------------------------------------------------------------------------------------------------------+------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Category | Check Item | Type | Impact | + +===============+=====================================================================================================================================================================================================================================================================================+============+========================================================================================================================================================================================================+ + | Project | The quotas of ECS, VPC, subnet, EIP, and EVS resources must meet customer requirements. | Deployment | If the quota is insufficient, resources will fail to be created. Specifically, users who have configured auto scaling must have sufficient resource quotas. | + +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | You are not advised to modify kernel parameters, system configurations, cluster core component versions, security groups, and ELB-related parameters on cluster nodes, or install software that has not been verified. | Deployment | Exceptions may occur on CCE clusters or Kubernetes components on the node, making the node unavailable for application deployment. | + +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Do not modify information about resources created by CCE, such as security groups and EVS disks. Resources created by CCE are labeled **cce**. | Deployment | CCE cluster functions may be abnormal. | + +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Proactive O&M | CCE provides multi-dimensional monitoring and alarm reporting functions, and supports basic resource monitoring based on fine-grained metrics by interconnecting with Application Operations Management (AOM). Alarms allow users to locate and rectify faults as soon as possible. | Monitoring | If the alarms are not configured, the standard of container cluster performance cannot be established. When an exception occurs, you cannot receive alarms and will need to manually locate the fault. | + +---------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/nodes/adding_a_second_data_disk_to_a_node_in_a_cce_cluster.rst b/umn/source/best_practice/cluster/adding_a_second_data_disk_to_a_node_in_a_cce_cluster.rst similarity index 98% rename from umn/source/nodes/adding_a_second_data_disk_to_a_node_in_a_cce_cluster.rst rename to umn/source/best_practice/cluster/adding_a_second_data_disk_to_a_node_in_a_cce_cluster.rst index 5106e62..a92488a 100644 --- a/umn/source/nodes/adding_a_second_data_disk_to_a_node_in_a_cce_cluster.rst +++ b/umn/source/best_practice/cluster/adding_a_second_data_disk_to_a_node_in_a_cce_cluster.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0344.html +:original_name: cce_bestpractice_00190.html -.. _cce_01_0344: +.. _cce_bestpractice_00190: Adding a Second Data Disk to a Node in a CCE Cluster ==================================================== diff --git a/umn/source/best_practice/cluster/connecting_to_multiple_clusters_using_kubectl.rst b/umn/source/best_practice/cluster/connecting_to_multiple_clusters_using_kubectl.rst new file mode 100644 index 0000000..d701eb0 --- /dev/null +++ b/umn/source/best_practice/cluster/connecting_to_multiple_clusters_using_kubectl.rst @@ -0,0 +1,313 @@ +:original_name: cce_bestpractice_00254.html + +.. _cce_bestpractice_00254: + +Connecting to Multiple Clusters Using kubectl +============================================= + +Painpoint +--------- + +When you have multiple CCE clusters, you may find it difficult to efficiently connect to all of them. + +Solution +-------- + +This section describes how to configure access to multiple clusters by modifying **kubeconfig.json**. The file describes multiple clusters, users, and contexts. To access different clusters, run the **kubectl config use-context** command to switch between contexts. + + +.. figure:: /_static/images/en-us_image_0261820020.png + :alt: **Figure 1** Using kubectl to connect to multiple clusters + + **Figure 1** Using kubectl to connect to multiple clusters + +Prerequisites +------------- + +kubectl can access multiple clusters. + +Introduction to kubeconfig.json +------------------------------- + +kubeconfig.json is the configuration file of kubectl. You can download it on the cluster details page. + +|image1| + +The content of kubeconfig.json is as follows: + +.. code-block:: + + { + "kind": "Config", + "apiVersion": "v1", + "preferences": {}, + "clusters": [{ + "name": "internalCluster", + "cluster": { + "server": "https://192.168.0.85:5443", + "certificate-authority-data": "LS0tLS1CRUULIE..." + } + }, { + "name": "externalCluster", + "cluster": { + "server": "https://xxx.xxx.xxx.xxx:5443", + "insecure-skip-tls-verify": true + } + }], + "users": [{ + "name": "user", + "user": { + "client-certificate-data": "LS0tLS1CRUdJTiBDRVJ...", + "client-key-data": "LS0tLS1CRUdJTiBS..." + } + }], + "contexts": [{ + "name": "internal", + "context": { + "cluster": "internalCluster", + "user": "user" + } + }, { + "name": "external", + "context": { + "cluster": "externalCluster", + "user": "user" + } + }], + "current-context": "external" + } + +It mainly consists of three sections. + +- **clusters**: describes the cluster information, mainly the access address of the cluster. +- **users**: describes information about the users who access the cluster. It includes the **client-certificate-data** and **client-key-data** certificate files. +- **contexts**: describes the configuration contexts. You switch between contexts to access different clusters. A context is associated with **user** and **cluster**, that is, it defines which user accesses which cluster. + +The preceding kubeconfig.json defines the private network address and public network address of the cluster as two clusters with two different contexts. You can switch the context to use different addresses to access the cluster. + +Configuring Access to Multiple Clusters +--------------------------------------- + +The following steps walk you through the procedure of configuring access to two clusters by modifying kubeconfig.json. + +This example configures only the public network access to the clusters. If you want to access multiple clusters over private networks, retain the **clusters** field and ensure that the clusters can be accessed over private networks. Its configuration is similar to that described in this example. + +#. Download kubeconfig.json of the two clusters and delete the lines related to private network access, as shown in the following figure. + + - Cluster A: + + .. code-block:: + + { + "kind": "Config", + "apiVersion": "v1", + "preferences": {}, + "clusters": [ { + "name": "externalCluster", + "cluster": { + "server": "https://119.xxx.xxx.xxx:5443", + "insecure-skip-tls-verify": true + } + }], + "users": [{ + "name": "user", + "user": { + "client-certificate-data": "LS0tLS1CRUdJTxM...", + "client-key-data": "LS0tLS1CRUdJTiB...." + } + }], + "contexts": [{ + "name": "external", + "context": { + "cluster": "externalCluster", + "user": "user" + } + }], + "current-context": "external" + } + + - Cluster B: + + .. code-block:: + + { + "kind": "Config", + "apiVersion": "v1", + "preferences": {}, + "clusters": [ { + "name": "externalCluster", + "cluster": { + "server": "https://124.xxx.xxx.xxx:5443", + "insecure-skip-tls-verify": true + } + }], + "users": [{ + "name": "user", + "user": { + "client-certificate-data": "LS0tLS1CRUdJTxM...", + "client-key-data": "LS0rTUideUdJTiB...." + } + }], + "contexts": [{ + "name": "external", + "context": { + "cluster": "externalCluster", + "user": "user" + } + }], + "current-context": "external" + } + + The preceding files have the same structure except that the **client-certificate-data** and **client-key-data** fields of **user** and the **clusters.cluster.server** field are different. + +#. Modify the **name** field as follows: + + - Cluster A: + + .. code-block:: + + { + "kind": "Config", + "apiVersion": "v1", + "preferences": {}, + "clusters": [ { + "name": "Cluster-A", + "cluster": { + "server": "https://119.xxx.xxx.xxx:5443", + "insecure-skip-tls-verify": true + } + }], + "users": [{ + "name": "Cluster-A-user", + "user": { + "client-certificate-data": "LS0tLS1CRUdJTxM...", + "client-key-data": "LS0tLS1CRUdJTiB...." + } + }], + "contexts": [{ + "name": "Cluster-A-Context", + "context": { + "cluster": "Cluster-A", + "user": "Cluster-A-user" + } + }], + "current-context": "Cluster-A-Context" + } + + - Cluster B: + + .. code-block:: + + { + "kind": "Config", + "apiVersion": "v1", + "preferences": {}, + "clusters": [ { + "name": "Cluster-B", + "cluster": { + "server": "https://124.xxx.xxx.xxx:5443", + "insecure-skip-tls-verify": true + } + }], + "users": [{ + "name": "Cluster-B-user", + "user": { + "client-certificate-data": "LS0tLS1CRUdJTxM...", + "client-key-data": "LS0rTUideUdJTiB...." + } + }], + "contexts": [{ + "name": "Cluster-B-Context", + "context": { + "cluster": "Cluster-B", + "user": "Cluster-B-user" + } + }], + "current-context": "Cluster-B-Context" + } + +#. Combine these two files. + + The file structure remains unchanged. Combine the contents of **clusters**, **users**, and **contexts** as follows: + + .. code-block:: + + { + "kind": "Config", + "apiVersion": "v1", + "preferences": {}, + "clusters": [ { + "name": "Cluster-A", + "cluster": { + "server": "https://119.xxx.xxx.xxx:5443", + "insecure-skip-tls-verify": true + } + }, + { + "name": "Cluster-B", + "cluster": { + "server": "https://124.xxx.xxx.xxx:5443", + "insecure-skip-tls-verify": true + } + }], + "users": [{ + "name": "Cluster-A-user", + "user": { + "client-certificate-data": "LS0tLS1CRUdJTxM...", + "client-key-data": "LS0tLS1CRUdJTiB...." + } + }, + { + "name": "Cluster-B-user", + "user": { + "client-certificate-data": "LS0tLS1CRUdJTxM...", + "client-key-data": "LS0rTUideUdJTiB...." + } + }], + "contexts": [{ + "name": "Cluster-A-Context", + "context": { + "cluster": "Cluster-A", + "user": "Cluster-A-user" + } + }, + { + "name": "Cluster-B-Context", + "context": { + "cluster": "Cluster-B", + "user": "Cluster-B-user" + } + }], + "current-context": "Cluster-A-Context" + } + +Verification +------------ + +Run the following commands to copy the file to the kubectl configuration path: + +**mkdir -p $HOME/.kube** + +**mv -f kubeconfig.json $HOME/.kube/config** + +Run the kubectl commands to check whether the two clusters can be connected. + +.. code-block:: + + # kubectl config use-context Cluster-A-Context + Switched to context "Cluster-A-Context". + # kubectl cluster-info + Kubernetes control plane is running at https://119.xxx.xxx.xxx:5443 + CoreDNS is running at https://119.xxx.xxx.xxx:5443/api/v1/namespaces/kube-system/services/coredns:dns/proxy + + To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. + + # kubectl config use-context Cluster-B-Context + Switched to context "Cluster-B-Context". + # kubectl cluster-info + Kubernetes control plane is running at https://124.xxx.xxx.xxx:5443 + CoreDNS is running at https://124.xxx.xxx.xxx:5443/api/v1/namespaces/kube-system/services/coredns:dns/proxy + + To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'. + +.. |image1| image:: /_static/images/en-us_image_0000001274882416.png diff --git a/umn/source/best_practice/cluster/index.rst b/umn/source/best_practice/cluster/index.rst new file mode 100644 index 0000000..4d07a38 --- /dev/null +++ b/umn/source/best_practice/cluster/index.rst @@ -0,0 +1,16 @@ +:original_name: cce_bestpractice_0050.html + +.. _cce_bestpractice_0050: + +Cluster +======= + +- :ref:`Connecting to Multiple Clusters Using kubectl ` +- :ref:`Adding a Second Data Disk to a Node in a CCE Cluster ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + connecting_to_multiple_clusters_using_kubectl + adding_a_second_data_disk_to_a_node_in_a_cce_cluster diff --git a/umn/source/best_practice/container/configuring_core_dumps.rst b/umn/source/best_practice/container/configuring_core_dumps.rst new file mode 100644 index 0000000..74c28dc --- /dev/null +++ b/umn/source/best_practice/container/configuring_core_dumps.rst @@ -0,0 +1,78 @@ +:original_name: cce_bestpractice_0325.html + +.. _cce_bestpractice_0325: + +Configuring Core Dumps +====================== + +Challenges +---------- + +Linux allows you to create a core dump file if an application crashes, which contains the data the application had in memory at the time of the crash. You can analyze the file to locate the fault. + +Generally, when a service application crashes, its container exits and is reclaimed and destroyed. Therefore, container core files need to be permanently stored on the host or cloud storage. This topic describes how to configure container core dumps. + +Enabling Core Dump on a Node +---------------------------- + +Log in to the node, run the following command to enable core dump, and set the path and format for storing core files: + +**echo "/tmp/cores/core.%h.%e.%p.%t" > /proc/sys/kernel/core_pattern** + +Parameters: + +- **%h**: host name (or pod name). You are advised to configure this parameter. +- **%e**: program file name. You are advised to configure this parameter. +- **%p**: (optional) process ID. +- **%t**: (optional) time of the core dump. + +You can also configure a pre-installation or post-installation script to automatically run this command when creating a node. + +Permanently Storing Core Dumps +------------------------------ + +A core file can be stored in your host (using a hostPath volume) or cloud storage (using a PVC). The following is an example YAML file for using a hostPath volume. + +.. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + name: coredump + spec: + volumes: + - name: coredump-path + hostPath: + path: /home/coredump + containers: + - name: ubuntu + image: ubuntu:12.04 + command: ["/bin/sleep","3600"] + volumeMounts: + - mountPath: /tmp/cores + name: coredump-path + +Create a pod using kubectl. + +**kubectl create -f pod.yaml** + +Verification +------------ + +After the pod is created, access the container and trigger a segmentation fault of the current shell terminal. + +.. code-block:: + + $ kubectl get pod + NAME READY STATUS RESTARTS AGE + coredump 1/1 Running 0 56s + $ kubectl exec -it coredump -- /bin/bash + root@coredump:/# kill -s SIGSEGV $$ + command terminated with exit code 139 + +Log in to the node and check whether a core file is generated in the **/home/coredump** directory. The following example indicates that a core file is generated. + +.. code-block:: + + # ls /home/coredump + core.coredump.bash.18.1650438992 diff --git a/umn/source/best_practice/container/index.rst b/umn/source/best_practice/container/index.rst new file mode 100644 index 0000000..936b016 --- /dev/null +++ b/umn/source/best_practice/container/index.rst @@ -0,0 +1,22 @@ +:original_name: cce_bestpractice_0051.html + +.. _cce_bestpractice_0051: + +Container +========= + +- :ref:`Properly Allocating Container Computing Resources ` +- :ref:`Modifying Kernel Parameters Using a Privileged Container ` +- :ref:`Initializing a Container ` +- :ref:`Using hostAliases to Configure /etc/hosts in a Pod ` +- :ref:`Configuring Core Dumps ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + properly_allocating_container_computing_resources + modifying_kernel_parameters_using_a_privileged_container + initializing_a_container + using_hostaliases_to_configure_etc_hosts_in_a_pod + configuring_core_dumps diff --git a/umn/source/best_practice/container/initializing_a_container.rst b/umn/source/best_practice/container/initializing_a_container.rst new file mode 100644 index 0000000..cd62c30 --- /dev/null +++ b/umn/source/best_practice/container/initializing_a_container.rst @@ -0,0 +1,90 @@ +:original_name: cce_bestpractice_00228.html + +.. _cce_bestpractice_00228: + +Initializing a Container +======================== + +Concepts +-------- + +Before containers running applications are started, one or some init containers are started first. If there are multiple init containers, they will be started in the defined sequence. The application containers are started only after all init containers run to completion and exit. Storage volumes in a pod are shared. Therefore, the data generated in the init containers can be used by the application containers. + +Init containers can be used in multiple Kubernetes resources, such as Deployments, DaemonSets, and jobs. They perform initialization before application containers are started. + +Scenario +-------- + +Before deploying a service, you can use an init container to make preparations before the pod where the service is running is deployed. After the preparations are complete, the init container runs to completion and exit, and the container to be deployed will be started. + +- **Scenario 1: Wait for other modules to be ready.** For example, an application contains two containerized services: web server and database. The web server service needs to access the database service. However, when the application is started, the database service may have not been started. Therefore, web server may fail to access database. To solve this problem, you can use an init container in the pod where web server is running to check whether database is ready. The init container runs to completion only when database is accessible. Then, web server is started and initiates a formal access request to database. +- **Scenario 2: Initialize the configuration.** For example, the init container can check all existing member nodes in the cluster and prepare the cluster configuration information for the application container. After the application container is started, it can be added to the cluster using the configuration information. +- **Other scenarios**: For example, register a pod with a central database and download application dependencies. + +For details, see `Init Containers `__. + +Procedure +--------- + +#. Edit the YAML file of the init container workload. + + **vi deployment.yaml** + + An example YAML file is provided as follows: + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: mysql + spec: + replicas: 1 + selector: + matchLabels: + name: mysql + template: + metadata: + labels: + name: mysql + spec: + initContainers: + - name: getresource + image: busybox + command: ['sleep 20'] + containers: + - name: mysql + image: percona:5.7.22 + imagePullPolicy: Always + ports: + - containerPort: 3306 + resources: + limits: + memory: "500Mi" + cpu: "500m" + requests: + memory: "500Mi" + cpu: "250m" + env: + - name: MYSQL_ROOT_PASSWORD + value: "mysql" + +#. Create an init container workload. + + **kubectl create -f deployment.yaml** + + Information similar to the following is displayed: + + .. code-block:: + + deployment.apps/mysql created + +#. Query the created Docker container on the node where the workload is running. + + **docker ps -a|grep mysql** + + The init container will exit after it runs to completion. The query result **Exited (0)** shows the exit status of the init container. + + |image1| + +.. |image1| image:: /_static/images/en-us_image_0261818867.png diff --git a/umn/source/best_practice/container/modifying_kernel_parameters_using_a_privileged_container.rst b/umn/source/best_practice/container/modifying_kernel_parameters_using_a_privileged_container.rst new file mode 100644 index 0000000..8876287 --- /dev/null +++ b/umn/source/best_practice/container/modifying_kernel_parameters_using_a_privileged_container.rst @@ -0,0 +1,116 @@ +:original_name: cce_bestpractice_00227.html + +.. _cce_bestpractice_00227: + +Modifying Kernel Parameters Using a Privileged Container +======================================================== + +Prerequisites +------------- + +To access a Kubernetes cluster from a client, you can use the Kubernetes command line tool kubectl. + +Procedure +--------- + +#. Create a DaemonSet in the background, select the Nginx image, enable the Privileged Container, configure the lifecycle, and add the **hostNetwork** field (value: **true**). + + a. Create a DaemonSet file. + + **vi daemonSet.yaml** + + An example YAML file is provided as follows: + + .. important:: + + The **spec.spec.containers.lifecycle** field indicates the command that will be run after the container is started. + + .. code-block:: + + kind: DaemonSet + apiVersion: apps/v1 + metadata: + name: daemonset-test + labels: + name: daemonset-test + spec: + selector: + matchLabels: + name: daemonset-test + template: + metadata: + labels: + name: daemonset-test + spec: + hostNetwork: true + containers: + - name: daemonset-test + image: nginx:alpine-perl + command: + - "/bin/sh" + args: + - "-c" + - while :; do time=$(date);done + imagePullPolicy: IfNotPresent + lifecycle: + postStart: + exec: + command: + - sysctl + - "-w" + - net.ipv4.tcp_tw_reuse=1 + securityContext: + privileged: true + imagePullSecrets: + - name: default-secret + + b. Create a DaemonSet. + + **kubectl create -f daemonSet.yaml** + +#. Check whether the DaemonSet is successfully created. + + **kubectl get daemonset** *DaemonSet name* + + In this example, run the following command: + + **kubectl get daemonset** daemonset-test + + Information similar to the following is displayed: + + .. code-block:: + + NAME DESIRED CURRENT READY UP-T0-DATE AVAILABLE NODE SELECTOR AGE + daemonset-test 2 2 2 2 2 2h + +#. Query the container ID of DaemonSet on the node. + + **docker ps -a|grep** *DaemonSet name* + + In this example, run the following command: + + **docker ps -a|grep** daemonset-test + + Information similar to the following is displayed: + + .. code-block:: + + 897b99faa9ce 3e094d5696c1 "/bin/sh -c while..." 31 minutes ago Up 30 minutes ault_fa7cc313-4ac1-11e9-a716-fa163e0aalba_0 + +#. Access the container. + + **docker exec -it** *containerid* **/bin/sh** + + In this example, run the following command: + + **docker exec -it** *897b99faa9ce* **/bin/sh** + +#. Check whether the configured command is executed after the container is started. + + **sysctl -a \|grep net.ipv4.tcp_tw_reuse** + + If the following information is displayed, the system parameters are modified successfully: + + .. code-block:: + + net.ipv4.tcp_tw_reuse=1 diff --git a/umn/source/best_practice/container/properly_allocating_container_computing_resources.rst b/umn/source/best_practice/container/properly_allocating_container_computing_resources.rst new file mode 100644 index 0000000..d1ffb3f --- /dev/null +++ b/umn/source/best_practice/container/properly_allocating_container_computing_resources.rst @@ -0,0 +1,125 @@ +:original_name: cce_bestpractice_00002.html + +.. _cce_bestpractice_00002: + +Properly Allocating Container Computing Resources +================================================= + +If a node has sufficient memory resources, a container on this node can use more memory resources than requested, but no more than limited. If the memory allocated to a container exceeds the upper limit, the container is stopped first. If the container continuously uses memory resources more than limited, the container is terminated. If a stopped container is allowed to be restarted, kubelet will restart it, but other types of run errors will occur. + +Scenario 1 +---------- + +The node's memory has reached the memory limit reserved for the node. As a result, OOM killer is triggered. + +**Solution** + +You can either scale up the node or migrate the pods on the node to other nodes. + +Scenario 2 +---------- + +The upper limit of resources configured for the pod is too small. When the actual usage exceeds the limit, OOM killer is triggered. + +**Solution** + +Set a higher upper limit for the workload. + +Example +------- + +A pod will be created and allocated memory that exceeds the limit. As shown in the following configuration file of the pod, the pod requests 50 MB memory and the memory limit is set to 100 MB. + +Example YAML file (memory-request-limit-2.yaml): + +.. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + name: memory-demo-2 + spec: + containers: + - name: memory-demo-2-ctr + image: vish/stress + resources: + requests: + memory: 50Mi + limits: + memory: "100Mi" + args: + - -mem-total + - 250Mi + - -mem-alloc-size + - 10Mi + - -mem-alloc-sleep + - 1s + +The **args** parameters indicate that the container attempts to request 250 MB memory, which exceeds the pod's upper limit (100 MB). + +Creating a pod: + +.. code-block:: + + kubectl create -f https://k8s.io/docs/tasks/configure-pod-container/memory-request-limit-2.yaml --namespace=mem-example + +Viewing the details about the pod: + +.. code-block:: + + kubectl get pod memory-demo-2 --namespace=mem-example + +In this stage, the container may be running or be killed. If the container is not killed, repeat the previous command until the container is killed. + +.. code-block:: + + NAME READY STATUS RESTARTS AGE + memory-demo-2 0/1 OOMKilled 1 24s + +Viewing detailed information about the container: + +.. code-block:: + + kubectl get pod memory-demo-2 --output=yaml --namespace=mem-example + +This output indicates that the container is killed because the memory limit is exceeded. + +.. code-block:: + + lastState: + terminated: + containerID: docker://7aae52677a4542917c23b10fb56fcb2434c2e8427bc956065183c1879cc0dbd2 + exitCode: 137 + finishedAt: 2020-02-20T17:35:12Z + reason: OOMKilled + startedAt: null + +In this example, the container can be automatically restarted. Therefore, kubelet will start it again. You can run the following command several times to see how the container is killed and started: + +.. code-block:: + + kubectl get pod memory-demo-2 --namespace=mem-example + +The preceding command output indicates how the container is killed and started back and forth: + +.. code-block:: + + $ kubectl get pod memory-demo-2 --namespace=mem-example + NAME READY STATUS RESTARTS AGE + memory-demo-2 0/1 OOMKilled 1 37s + $ kubectl get pod memory-demo-2 --namespace=mem-example + NAME READY STATUS RESTARTS AGE + memory-demo-2 1/1 Running 2 40s + +Viewing the historical information of the pod: + +.. code-block:: + + kubectl describe pod memory-demo-2 --namespace=mem-example + +The following command output indicates that the pod is repeatedly killed and started. + +.. code-block:: + + ... Normal Created Created container with id 66a3a20aa7980e61be4922780bf9d24d1a1d8b7395c09861225b0eba1b1f8511 + ... Warning BackOff Back-off restarting failed container diff --git a/umn/source/best_practice/container/using_hostaliases_to_configure_etc_hosts_in_a_pod.rst b/umn/source/best_practice/container/using_hostaliases_to_configure_etc_hosts_in_a_pod.rst new file mode 100644 index 0000000..f7f1718 --- /dev/null +++ b/umn/source/best_practice/container/using_hostaliases_to_configure_etc_hosts_in_a_pod.rst @@ -0,0 +1,121 @@ +:original_name: cce_bestpractice_00226.html + +.. _cce_bestpractice_00226: + +Using hostAliases to Configure /etc/hosts in a Pod +================================================== + +Scenario +-------- + +If DNS or other related settings are inappropriate, you can use **hostAliases** to overwrite the resolution of the host name at the pod level when adding entries to the **/etc/hosts** file of the pod. + +Procedure +--------- + +#. Use kubectl to connect to the cluster. + +#. Create the **hostaliases-pod.yaml** file. + + **vi hostaliases-pod.yaml** + + The field in bold in the YAML file indicates the image name and tag. You can replace the example value as required. + + .. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + name: hostaliases-pod + spec: + hostAliases: + - ip: 127.0.0.1 + hostnames: + - foo.local + - bar.local + - ip: 10.1.2.3 + hostnames: + - foo.remote + - bar.remote + containers: + - name: cat-hosts + image: tomcat:9-jre11-slim + lifecycle: + postStart: + exec: + command: + - cat + - /etc/hosts + imagePullSecrets: + - name: default-secret + + .. table:: **Table 1** pod field description + + +------------+--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory/Optional | Description | + +============+====================+============================================================================================================================================================+ + | apiVersion | Mandatory | API version number | + +------------+--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kind | Mandatory | Type of the object to be created | + +------------+--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | metadata | Mandatory | Metadata definition of a resource object | + +------------+--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | name | Mandatory | Name of a pod | + +------------+--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | spec | Mandatory | Detailed description of the pod. For details, see :ref:`Table 2 `. | + +------------+--------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + .. _cce_bestpractice_00226__en-us_topic_0226102200_en-us_topic_0179003345_table33531919193: + + .. table:: **Table 2** spec field description + + +-------------+--------------------+----------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory/Optional | Description | + +=============+====================+============================================================================================================================+ + | hostAliases | Mandatory | Host alias | + +-------------+--------------------+----------------------------------------------------------------------------------------------------------------------------+ + | containers | Mandatory | For details, see :ref:`Table 3 `. | + +-------------+--------------------+----------------------------------------------------------------------------------------------------------------------------+ + + .. _cce_bestpractice_00226__en-us_topic_0226102200_en-us_topic_0179003345_table196127172016: + + .. table:: **Table 3** containers field description + + ========= ================== ==================== + Parameter Mandatory/Optional Description + ========= ================== ==================== + name Mandatory Container name + image Mandatory Container image name + lifecycle Optional Lifecycle + ========= ================== ==================== + +#. Create a pod. + + **kubectl create -f hostaliases-pod.yaml** + + If information similar to the following is displayed, the pod is created. + + .. code-block:: + + pod/hostaliases-pod created + +#. Query the pod status. + + **kubectl get pod hostaliases-pod** + + If the pod is in the **Running** state, the pod is successfully created. + + .. code-block:: + + NAME READY STATUS RESTARTS AGE + hostaliases-pod 1/1 Running 0 16m + +#. Check whether the **hostAliases** functions properly. + + **docker ps \|grep hostaliases-pod** + + **docker exec -ti Container ID /bin/sh** + + |image1| + +.. |image1| image:: /_static/images/en-us_image_0278498565.png diff --git a/umn/source/best_practice/devops/index.rst b/umn/source/best_practice/devops/index.rst new file mode 100644 index 0000000..f84d128 --- /dev/null +++ b/umn/source/best_practice/devops/index.rst @@ -0,0 +1,14 @@ +:original_name: cce_bestpractice_0322.html + +.. _cce_bestpractice_0322: + +DevOps +====== + +- :ref:`Interconnecting GitLab with SWR and CCE for CI/CD ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + interconnecting_gitlab_with_swr_and_cce_for_ci_cd diff --git a/umn/source/best_practice/devops/interconnecting_gitlab_with_swr_and_cce_for_ci_cd.rst b/umn/source/best_practice/devops/interconnecting_gitlab_with_swr_and_cce_for_ci_cd.rst new file mode 100644 index 0000000..4a1c088 --- /dev/null +++ b/umn/source/best_practice/devops/interconnecting_gitlab_with_swr_and_cce_for_ci_cd.rst @@ -0,0 +1,191 @@ +:original_name: cce_bestpractice_0324.html + +.. _cce_bestpractice_0324: + +Interconnecting GitLab with SWR and CCE for CI/CD +================================================= + +Challenges +---------- + +GitLab is an open-source version management system developed with Ruby on Rails for Git project repository management. It supports web-based access to public and private projects. Similar to GitHub, GitLab allows you to browse source code, manage bugs and comments, and control team member access to repositories. You will find it very easy to view committed versions and file history database. Team members can communicate with each other using the built-in chat program (Wall). + +GitLab provides powerful CI/CD functions and is widely used in software development. + + +.. figure:: /_static/images/en-us_image_0000001291567729.png + :alt: **Figure 1** GitLab CI/CD process + + **Figure 1** GitLab CI/CD process + +This section describes how to interconnect GitLab with SWR and CCE for CI/CD. + +Preparations +------------ + +#. Create a CCE cluster and a node and bind an EIP to the node for downloading an image during GitLab Runner installation. +#. Download and configure kubectl to connect to the cluster. +#. `Install Helm 3 `__. + +Installing GitLab Runner +------------------------ + +Log in to `GitLab `__, choose **Settings** > **CI/CD** in the project view, click **Expand** next to **Runners**, and search for the GitLab Runner registration URL and token. + +|image1| + +Create the **values.yaml** file and fill in the following information: + +.. code-block:: + + # Registration URL + gitlabUrl: https://gitlab.com/ + # Registration token + runnerRegistrationToken: "GR13489411dKVzmTyaywEDTF_1QXb" + rbac: + create: true + runners: + privileged: true + +Create a GitLab namespace. + +.. code-block:: + + kubectl create namespace gitlab + +Install GitLab Runner using Helm. + +.. code-block:: + + helm repo add gitlab https://charts.gitlab.io + helm install --namespace gitlab gitlab-runner -f values.yaml gitlab/gitlab-runner + +After the installation, you can query the workload of gitlab-runner on the CCE console and view the connection information in GitLab later. + +|image2| + +Creating an Application +----------------------- + +Place the application to be created in the GitLab project repository. This section takes Nginx modification as an example. For details, visit https://gitlab.com/c8147/cidemo/-/tree/main. + +The following files are included: + +- **.gitlab-ci.yml**: Gitlab CI file, which will be described in detail in :ref:`Creating a Pipeline `. +- **Dockerfile**: used to build Docker images. +- **index.html**: used to replace the index page of Nginx. +- **k8s.yaml**: used to deploy the Nginx app. A Deployment named **nginx-test** and a Service named **nginx-test** will be created. + +The preceding files are only examples. You can replace or modify them accordingly. + +Configuring Global Variables +---------------------------- + +When using pipelines, you need to build an image, upload it to SWR, and run kubectl commands to deploy the image in the cluster. Before performing these operations, you must log in to SWR and obtain the credential for connecting to the cluster. You can define the information as variables in GitLab. + +Log in to `GitLab `__, choose **Settings** > **CI/CD** in the project view, and click **Expand** next to **Variables** to add variables. + +|image3| + +- **kube_config** + + **kubeconfig.json** file used for kubectl command authentication. Run the following command on the host where kubectl is configured to convert the file to the Base64 format: + + **echo $(cat ~/.kube/config \| base64) \| tr -d " "** + + The command output is the content of **kubeconfig.json**. + +- **project**: project name. + + Log in to the management console, click your username in the upper right corner, and click **My Credentials**. In the **Projects** area on the **API Credentials** page, check the name of the project in your current region. + +- **swr_ak**: access key. + + Log in to the management console, click your username in the upper right corner, and click **My Credentials**. In the navigation pane on the left, choose **Access Keys**. Click **Create Access Key**, enter the description, and click **OK**. In the displayed **Information** dialog box, click **Download**. After the certificate is downloaded, obtain the AK and SK information from the **credentials** file. + +- **swr_sk**: secret key for logging in to SWR. + + Run the following command to obtain the key pair. Replace *$AK* and *$SK* with the AK and SK obtained in the preceding steps. + + **printf "$AK" \| openssl dgst -binary -sha256 -hmac "$SK" \| od -An -vtx1 \| sed 's/[ \\n]//g' \| sed 'N;s/\\n//'** + + The command output displays the login key pair. + +.. _cce_bestpractice_0324__section171541431101910: + +Creating a Pipeline +------------------- + +Log in to `Gitlab `__ and add the **.gitlab-ci.yml** file to **Repository**. + +|image4| + +The content is as follows: + +.. code-block:: + + # Define pipeline stages, including package, build, and deploy. + stages: + - package + - build + - deploy + # If no image is specified in each stage, the default image docker:latest is used. + image: docker:latest + # In the package stage, only printing is performed. + package: + stage: package + script: + - echo "package" + # In the build stage, the Docker-in-Docker mode is used. + build: + stage: build + # Define environment variables for the build stage. + variables: + DOCKER_HOST: tcp://docker:2375 + # Define the image for running Docker-in-Docker. + services: + - docker:18.09-dind + script: + - echo "build" + # Log in to SWR. + - docker login -u $project@$swr_ak -p $swr_sk swr.eu-de.otc.t-systems.com + # Build an image. k8s-dev is the organization name in SWR. Replace it to the actual name. + - docker build -t swr.eu-de.otc.t-systems.com/k8s-dev/nginx:$CI_PIPELINE_ID . + # Push the image to SWR. + - docker push swr.eu-de.otc.t-systems.com/k8s-dev/nginx:$CI_PIPELINE_ID + deploy: + # Use the kubectl image. + image: + name: bitnami/kubectl:latest + entrypoint: [""] + stage: deploy + script: + # Configure the kubeconfig file. + - echo $kube_config |base64 -d > $KUBECONFIG + # Replace the image in the k8s.yaml file. + - sed -i "s//swr.eu-de.otc.t-systems.com\/k8s-dev\/nginx:$CI_PIPELINE_ID/g" k8s.yaml + - cat k8s.yaml + # Deploy an application. + - kubectl apply -f k8s.yaml + +After the **.gitlab-ci.yml** file is saved, the pipeline is started immediately. You can view the pipeline execution status in GitLab. + +|image5| + +Verifying Deployment +-------------------- + +After the pipeline is deployed, locate the **nginx-test** Service on the CCE console, query its access address, and run the **curl** command to access the Service. + +.. code-block:: + + # curl xxx.xxx.xxx.xxx:31111 + Hello Gitlab! + +If the preceding information is displayed, the deployment is correct. + +.. |image1| image:: /_static/images/en-us_image_0000001238489436.png +.. |image2| image:: /_static/images/en-us_image_0000001283301301.png +.. |image3| image:: /_static/images/en-us_image_0000001238903330.png +.. |image4| image:: /_static/images/en-us_image_0000001238830246.png +.. |image5| image:: /_static/images/en-us_image_0000001283343269.png diff --git a/umn/source/best_practice/disaster_recovery/implementing_high_availability_for_containers_in_cce.rst b/umn/source/best_practice/disaster_recovery/implementing_high_availability_for_containers_in_cce.rst new file mode 100644 index 0000000..8b7cd14 --- /dev/null +++ b/umn/source/best_practice/disaster_recovery/implementing_high_availability_for_containers_in_cce.rst @@ -0,0 +1,134 @@ +:original_name: cce_bestpractice_00220.html + +.. _cce_bestpractice_00220: + +Implementing High Availability for Containers in CCE +==================================================== + +Basic Principles +---------------- + +To achieve high availability for your CCE containers, you can do as follows: + +#. Deploy three master nodes for the cluster. +#. When nodes are deployed across AZs, set custom scheduling policies based on site requirements to maximize resource utilization. +#. Create multiple node pools in different AZs and use them for node scaling. +#. Set the number of pods to be greater than 2 when creating a workload. +#. Set pod affinity rules to distribute pods to different AZs and nodes. + +Procedure +--------- + +Assume that there are four nodes in a cluster distributed in the following AZs: + +.. code-block:: + + $ kubectl get node -L topology.kubernetes.io/zone,kubernetes.io/hostname + NAME STATUS ROLES AGE VERSION ZONE HOSTNAME + 192.168.5.112 Ready 42m v1.21.7-r0-CCE21.11.1.B007 eu-de-01 192.168.5.112 + 192.168.5.179 Ready 42m v1.21.7-r0-CCE21.11.1.B007 eu-de-01 192.168.5.179 + 192.168.5.252 Ready 37m v1.21.7-r0-CCE21.11.1.B007 eu-de-02 192.168.5.252 + 192.168.5.8 Ready 33h v1.21.7-r0-CCE21.11.1.B007 eu-de-03 192.168.5.8 + +Create workloads according to the following two podAntiAffinity rules: + +- The first one is the pod anti-affinity in an AZ. Set the parameters as follows: + + - **weight**: A larger weight value indicates a higher priority. In this example, set it to **50**. + - **topologyKey**: a default or custom key for the node label that the system uses to denote a topology domain. A topology key determines the scope where the pod should be scheduled to. In this example, set this parameter to **topology.kubernetes.io/zone**, which is the label for identifying the AZ where the node is located. + - **labelSelector**: Select the label of the workload to realize the anti-affinity between this container and the workload. + +- The second one is the pod anti-affinity in the node host name. Set the parameters as follows: + + - **weight**: Set it to **50**. + - **topologyKey**: Set it to **kubernetes.io/hostname**. + - **labelSelector**: Select the label of the pod, which is anti-affinity with the pod. + +.. code-block:: + + kind: Deployment + apiVersion: apps/v1 + metadata: + name: nginx + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: container-0 + image: nginx:alpine + resources: + limits: + cpu: 250m + memory: 512Mi + requests: + cpu: 250m + memory: 512Mi + affinity: + podAntiAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 50 + podAffinityTerm: + labelSelector: # Select the label of the workload to realize the anti-affinity between this container and the workload. + matchExpressions: + - key: app + operator: In + values: + - nginx + namespaces: + - default + topologyKey: topology.kubernetes.io/zone # It takes effect in the same AZ. + - weight: 50 + podAffinityTerm: + labelSelector: # Select the label of the workload to realize the anti-affinity between this container and the workload. + matchExpressions: + - key: app + operator: In + values: + - nginx + namespaces: + - default + topologyKey: kubernetes.io/hostname # It takes effect on the node. + imagePullSecrets: + - name: default-secret + +Create a workload and view the node where the pod is located. + +.. code-block:: + + $ kubectl get pod -owide + NAME READY STATUS RESTARTS AGE IP NODE + nginx-6fffd8d664-dpwbk 1/1 Running 0 17s 10.0.0.132 192.168.5.112 + nginx-6fffd8d664-qhclc 1/1 Running 0 17s 10.0.1.133 192.168.5.252 + +Increase the number of pods to 3. The pod is scheduled to another node, and the three nodes are in three different AZs. + +.. code-block:: + + $ kubectl scale --replicas=3 deploy/nginx + deployment.apps/nginx scaled + $ kubectl get pod -owide + NAME READY STATUS RESTARTS AGE IP NODE + nginx-6fffd8d664-8t7rv 1/1 Running 0 3s 10.0.0.9 192.168.5.8 + nginx-6fffd8d664-dpwbk 1/1 Running 0 2m45s 10.0.0.132 192.168.5.112 + nginx-6fffd8d664-qhclc 1/1 Running 0 2m45s 10.0.1.133 192.168.5.252 + +Increase the number of pods to 4. The pod is scheduled to the last node. With podAntiAffinity rules, pods can be evenly distributed to AZs and nodes. + +.. code-block:: + + $ kubectl scale --replicas=4 deploy/nginx + deployment.apps/nginx scaled + $ kubectl get pod -owide + NAME READY STATUS RESTARTS AGE IP NODE + nginx-6fffd8d664-8t7rv 1/1 Running 0 2m30s 10.0.0.9 192.168.5.8 + nginx-6fffd8d664-dpwbk 1/1 Running 0 5m12s 10.0.0.132 192.168.5.112 + nginx-6fffd8d664-h796b 1/1 Running 0 78s 10.0.1.5 192.168.5.179 + nginx-6fffd8d664-qhclc 1/1 Running 0 5m12s 10.0.1.133 192.168.5.252 diff --git a/umn/source/best_practice/disaster_recovery/index.rst b/umn/source/best_practice/disaster_recovery/index.rst new file mode 100644 index 0000000..536b351 --- /dev/null +++ b/umn/source/best_practice/disaster_recovery/index.rst @@ -0,0 +1,14 @@ +:original_name: cce_bestpractice_0323.html + +.. _cce_bestpractice_0323: + +Disaster Recovery +================= + +- :ref:`Implementing High Availability for Containers in CCE ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + implementing_high_availability_for_containers_in_cce diff --git a/umn/source/best_practice/index.rst b/umn/source/best_practice/index.rst new file mode 100644 index 0000000..5b5e2de --- /dev/null +++ b/umn/source/best_practice/index.rst @@ -0,0 +1,32 @@ +:original_name: cce_bestpractice.html + +.. _cce_bestpractice: + +Best Practice +============= + +- :ref:`Checklist for Deploying Containerized Applications in the Cloud ` +- :ref:`Migration ` +- :ref:`DevOps ` +- :ref:`Disaster Recovery ` +- :ref:`Security ` +- :ref:`Auto Scaling ` +- :ref:`Cluster ` +- :ref:`Networking ` +- :ref:`Storage ` +- :ref:`Container ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + checklist_for_deploying_containerized_applications_in_the_cloud + migration/index + devops/index + disaster_recovery/index + security/index + auto_scaling/index + cluster/index + networking/index + storage/index + container/index diff --git a/umn/source/best_practice/migration/index.rst b/umn/source/best_practice/migration/index.rst new file mode 100644 index 0000000..011cb32 --- /dev/null +++ b/umn/source/best_practice/migration/index.rst @@ -0,0 +1,14 @@ +:original_name: cce_bestpractice_00237.html + +.. _cce_bestpractice_00237: + +Migration +========= + +- :ref:`Migrating On-premises Kubernetes Clusters to CCE ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + migrating_on-premises_kubernetes_clusters_to_cce/index diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/index.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/index.rst new file mode 100644 index 0000000..67c6583 --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/index.rst @@ -0,0 +1,28 @@ +:original_name: cce_bestpractice_0306.html + +.. _cce_bestpractice_0306: + +Migrating On-premises Kubernetes Clusters to CCE +================================================ + +- :ref:`Solution Overview ` +- :ref:`Planning Resources for the Target Cluster ` +- :ref:`Migrating Resources Outside a Cluster ` +- :ref:`Installing the Migration Tool ` +- :ref:`Migrating Resources in a Cluster ` +- :ref:`Updating Resources Accordingly ` +- :ref:`Performing Additional Tasks ` +- :ref:`Troubleshooting ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + solution_overview + planning_resources_for_the_target_cluster + migrating_resources_outside_a_cluster + installing_the_migration_tool + migrating_resources_in_a_cluster + updating_resources_accordingly + performing_additional_tasks + troubleshooting diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/installing_the_migration_tool.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/installing_the_migration_tool.rst new file mode 100644 index 0000000..ea94f39 --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/installing_the_migration_tool.rst @@ -0,0 +1,198 @@ +:original_name: cce_bestpractice_0310.html + +.. _cce_bestpractice_0310: + +Installing the Migration Tool +============================= + +Velero is an open-source backup and migration tool for Kubernetes clusters. It integrates the persistent volume (PV) data backup capability of the Restic tool and can be used to back up Kubernetes resource objects (such as Deployments, jobs, Services, and ConfigMaps) in the source cluster. Data in the PV mounted to the pod is backed up and uploaded to the object storage. When a disaster occurs or migration is required, the target cluster can use Velero to obtain the corresponding backup data from OBS and restore cluster resources as required. + +According to :ref:`Migration Solution `, you need to prepare temporary object storage to store backup files before the migration. Velero supports OSB or `MinIO `__ as the object storage. OBS requires sufficient storage space for storing backup files. You can estimate the storage space based on your cluster scale and data volume. You are advised to use OBS for backup. For details about how to deploy Velero, see :ref:`Installing Velero `. + +Prerequisites +------------- + +- The Kubernetes version of the source on-premises cluster must be 1.10 or later, and the cluster can use DNS and Internet services properly. +- If you use OBS to store backup files, you need to obtain the AK/SK of a user who has the right to operate OBS. For details about how to obtain the AK/SK, see `Access Keys `__. +- If you use MinIO to store backup files, bind an EIP to the server where MinIO is installed and enable the API and console port of MinIO in the security group. +- The target CCE cluster has been created. +- The source cluster and target cluster must each have at least one idle node. It is recommended that the node specifications be 4 vCPUs and 8 GB memory or higher. + +Installing MinIO +---------------- + +MinIO is an open-source, high-performance object storage tool compatible with the S3 API protocol. If MinIO is used to store backup files for cluster migration, you need a temporary server to deploy MinIO and provide services for external systems. If you use OBS to store backup files, skip this section and go to :ref:`Installing Velero `. + +MinIO can be installed in any of the following locations: + +- Temporary ECS outside the cluster + + If the MinIO server is installed outside the cluster, backup files will not be affected when a catastrophic fault occurs in the cluster. + +- Idle nodes in the cluster + + You can remotely log in to a node to install the MinIO server or install MinIO in a container. For details, see the official Velero documentation at https://velero.io/docs/v1.7/contributions/minio/#set-up-server. + + .. important:: + + For example, to install MinIO in a container, run the following command: + + - The storage type in the YAML file provided by Velero is **emptyDir**. You are advised to change the storage type to **HostPath** or **Local**. Otherwise, backup files will be permanently lost after the container is restarted. + - Ensure that the MinIO service is accessible externally. Otherwise, backup files cannot be downloaded outside the cluster. You can change the Service type to NodePort or use other types of public network access Services. + +Regardless of which deployment method is used, the server where MinIO is installed must have sufficient storage space, an EIP must be bound to the server, and the MinIO service port must be enabled in the security group. Otherwise, backup files cannot be uploaded or downloaded. + +In this example, MinIO is installed on a temporary ECS outside the cluster. + +#. Download MinIO. + + .. code-block:: + + mkdir /opt/minio + mkdir /opt/miniodata + cd /opt/minio + wget https://dl.minio.io/server/minio/release/linux-amd64/minio + chmod +x minio + +#. .. _cce_bestpractice_0310__li126129251432: + + Set the username and password of MinIO. + + The username and password set using this method are temporary environment variables and must be reset after the service is restarted. Otherwise, the default root credential **minioadmin:minioadmin** will be used to create the service. + + .. code-block:: + + export MINIO_ROOT_USER=minio + export MINIO_ROOT_PASSWORD=minio123 + +#. Create a service. In the command, **/opt/miniodata/** indicates the local disk path for MinIO to store data. + + The default API port of MinIO is 9000, and the console port is randomly generated. You can use the **--console-address** parameter to specify a console port. + + .. code-block:: + + ./minio server /opt/miniodata/ --console-address ":30840" & + + .. note:: + + Enable the API and console ports in the firewall and security group on the server where MinIO is to be installed. Otherwise, access to the object bucket will fail. + +#. Use a browser to access http://{*EIP of the node where MinIO resides*}:30840. The MinIO console page is displayed. + +.. _cce_bestpractice_0310__section138392220432: + +Installing Velero +----------------- + +Go to the OBS console or MinIO console and create a bucket named **velero** to store backup files. You can custom the bucket name, which must be used when installing Velero. Otherwise, the bucket cannot be accessed and the backup fails. For details, see :ref:`4 `. + +.. important:: + + - Velero instances need to be installed and deployed in both the **source and target clusters**. The installation procedures are the same, which are used for backup and restoration, respectively. + - The master node of a CCE cluster does not provide a port for remote login. You can install Velero using kubectl. + - If there are a large number of resources to back up, you are advised to adjust the CPU and memory resources of Velero and Restic to 1 vCPU and 1 GB memory or higher. For details, see :ref:`Backup Tool Resources Are Insufficient `. + - The object storage bucket for storing backup files must be **empty**. + +Download the latest, stable binary file from https://github.com/vmware-tanzu/velero/releases. This section uses Velero 1.7.0 as an example. The installation process in the source cluster is the same as that in the target cluster. + +#. Download the binary file of Velero 1.7.0. + + .. code-block:: + + wget https://github.com/vmware-tanzu/velero/releases/download/v1.7.0/velero-v1.7.0-linux-amd64.tar.gz + +#. Install the Velero client. + + .. code-block:: + + tar -xvf velero-v1.7.0-linux-amd64.tar.gz + cp ./velero-v1.7.0-linux-amd64/velero /usr/local/bin + +#. .. _cce_bestpractice_0310__li197871715322: + + Create the access key file **credentials-velero** for the backup object storage. + + .. code-block:: + + vim credentials-velero + + Replace the AK/SK in the file based on the site requirements. If OBS is used, obtain the AK/SK by referring to . If MinIO is used, the AK and SK are the username and password created in :ref:`2 `. + + .. code-block:: + + [default] + aws_access_key_id = {AK} + aws_secret_access_key = {SK} + +#. .. _cce_bestpractice_0310__li1722825643415: + + Deploy the Velero server. Change the value of **--bucket** to the name of the created object storage bucket. In this example, the bucket name is **velero**. For more information about custom installation parameters, see `Customize Velero Install `__. + + .. code-block:: + + velero install \ + --provider aws \ + --plugins velero/velero-plugin-for-aws:v1.2.1 \ + --bucket velero \ + --secret-file ./credentials-velero \ + --use-restic \ + --use-volume-snapshots=false \ + --backup-location-config region=eu-de,s3ForcePathStyle="true",s3Url=http://obs.eu-de.otc.t-systems.com + + .. table:: **Table 1** Installation parameters of Velero + + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+====================================================================================================================================================================================================================================================================+ + | --provider | Vendor who provides the plug-in. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | --plugins | API component compatible with AWS S3. Both OBS and MinIO support the S3 protocol. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | --bucket | Name of the object storage bucket for storing backup files. The bucket must be created in advance. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | --secret-file | Secret file for accessing the object storage, that is, the **credentials-velero** file created in :ref:`3 `. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | --use-restic | Whether to use Restic to support PV data backup. You are advised to enable this function. Otherwise, storage volume resources cannot be backed up. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | --use-volume-snapshots | Whether to create the VolumeSnapshotLocation object for PV snapshot, which requires support from the snapshot program. Set this parameter to **false**. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | --backup-location-config | OBS bucket configurations, including region, s3ForcePathStyle, and s3Url. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | region | Region to which object storage bucket belongs. | + | | | + | | - If OBS is used, set this parameter according to your region, for example, **eu-de**. | + | | - If MinIO is used, set this parameter to **minio**. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | s3ForcePathStyle | The value **true** indicates that the S3 file path format is used. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | s3Url | API access address of the object storage bucket. | + | | | + | | - If OBS is used, set this parameter to **http://obs.{region}.otc.t-systems.com** (*region* indicates the region where the object storage bucket is located). For example, if the region is eu-de, the parameter value is **http://obs.eu-de.otc.t-systems.com**. | + | | - If MinIO is used, set this parameter to **http://{EIP of the node where minio is located}:9000**. The value of this parameter is determined based on the IP address and port of the node where MinIO is installed. | + | | | + | | .. note:: | + | | | + | | - The access port in s3Url must be set to the API port of MinIO instead of the console port. The default API port of MinIO is 9000. | + | | - To access MinIO installed outside the cluster, you need to enter the public IP address of MinIO. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. By default, a namespace named **velero** is created for the Velero instance. Run the following command to view the pod status: + + .. code-block:: + + $ kubectl get pod -n velero + NAME READY STATUS RESTARTS AGE + restic-rn29c 1/1 Running 0 16s + velero-c9ddd56-tkzpk 1/1 Running 0 16s + + .. note:: + + To prevent memory insufficiency during backup in the actual production environment, you are advised to change the CPU and memory allocated to Restic and Velero by referring to :ref:`Backup Tool Resources Are Insufficient `. + +#. Check the interconnection between Velero and the object storage and ensure that the status is **Available**. + + .. code-block:: + + $ velero backup-location get + NAME PROVIDER BUCKET/PREFIX PHASE LAST VALIDATED ACCESS MODE DEFAULT + default aws velero Available 2021-10-22 15:21:12 +0800 CST ReadWrite true diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_in_a_cluster.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_in_a_cluster.rst new file mode 100644 index 0000000..37da7e9 --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_in_a_cluster.rst @@ -0,0 +1,159 @@ +:original_name: cce_bestpractice_0311.html + +.. _cce_bestpractice_0311: + +Migrating Resources in a Cluster +================================ + +Scenario +-------- + +WordPress is used as an example to describe how to migrate an application from an on-premises Kubernetes cluster to a CCE cluster. The WordPress application consists of the WordPress and MySQL components, which are containerized. The two components are bound to two local storage volumes of the Local type respectively and provide external access through the NodePort Service. + +Before the migration, use a browser to access the WordPress site, create a site named **Migrate to CCE**, and publish an article to verify the integrity of PV data after the migration. The article published in WordPress will be stored in the **wp_posts** table of the MySQL database. If the migration is successful, all contents in the database will be migrated to the new cluster. You can verify the PV data migration based on the migration result. + +Prerequisites +------------- + +- Before the migration, clear the abnormal pod resources in the source cluster. If the pod is in the abnormal state and has a PVC mounted, the PVC is in the pending state after the cluster is migrated. +- Ensure that the cluster on the CCE side does not have the same resources as the cluster to be migrated because Velero does not restore the same resources by default. +- To ensure that container image images can be properly pulled after cluster migration, migrate the images to SWR. +- CCE does not support EVS disks of the **ReadWriteMany** type. If resources of this type exist in the source cluster, change the storage type to **ReadWriteOnce**. +- Velero integrates the Restic tool to back up and restore storage volumes. Currently, the storage volumes of the HostPath type are not supported. For details, see `Restic Restrictions `__. If you need to back up storage volumes of this type, replace the hostPath volumes with local volumes by referring to :ref:`Storage Volumes of the HostPath Type Cannot Be Backed Up `. If a backup task involves storage of the HostPath type, the storage volumes of this type will be automatically skipped and a warning message will be generated. This will not cause a backup failure. + +.. _cce_bestpractice_0311__section750718193288: + +Backing Up Applications in the Source Cluster +--------------------------------------------- + +#. .. _cce_bestpractice_0311__li686918502812: + + (Optional) If you need to back up the data of a specified storage volume in the pod, add an annotation to the pod. The annotation template is as follows: + + .. code-block:: + + kubectl -n annotate backup.velero.io/backup-volumes=,,... + + - ****: namespace where the pod is located. + - ****: pod name. + - ****: name of the persistent volume mounted to the pod. You can run the **describe** statement to query the pod information. The **Volume** field indicates the names of all persistent volumes attached to the pod. + + Add annotations to the pods of WordPress and MySQL. The pod names are **wordpress-758fbf6fc7-s7fsr** and **mysql-5ffdfbc498-c45lh**. As the pods are in the default namespace **default**, the **-n ** parameter can be omitted. + + .. code-block:: + + kubectl annotate pod/wordpress-758fbf6fc7-s7fsr backup.velero.io/backup-volumes=wp-storage + kubectl annotate pod/mysql-5ffdfbc498-c45lh backup.velero.io/backup-volumes=mysql-storage + +#. Back up the application. During the backup, you can specify resources based on parameters. If no parameter is added, the entire cluster resources are backed up by default. For details about the parameters, see `Resource filtering `__. + + - **--default-volumes-to-restic**: indicates that the Restic tool is used to back up all storage volumes mounted to the pod. Storage volumes of the HostPath type are not supported. If this parameter is not specified, the storage volume specified by annotation in :ref:`1 ` is backed up by default. This parameter is available only when **--use-restic** is specified during :ref:`Velero installation `. + + .. code-block:: + + velero backup create --default-volumes-to-restic + + - **--include-namespaces**: backs up resources in a specified namespace. + + .. code-block:: + + velero backup create --include-namespaces + + - **--include-resources**: backs up the specified resources. + + .. code-block:: + + velero backup create --include-resources deployments + + - **--selector**: backs up resources that match the selector. + + .. code-block:: + + velero backup create --selector = + + In this section, resources in the namespace **default** are backed up. **wordpress-backup** is the backup name. You need to specify the same backup name when restoring applications. Example: + + .. code-block:: + + velero backup create wordpress-backup --include-namespaces default --default-volumes-to-restic + + If the following information is displayed, the backup task is successfully created: + + .. code-block:: + + Backup request "wordpress-backup" submitted successfully. Run `velero backup describe wordpress-backup` or `velero backup logs wordpress-backup` for more details. + +#. Check the backup status. + + .. code-block:: + + velero backup get + + Information similar to the following is displayed: + + .. code-block:: + + NAME STATUS ERRORS WARNINGS CREATED EXPIRES STORAGE LOCATION SELECTOR + wordpress-backup Completed 0 0 2021-10-14 15:32:07 +0800 CST 29d default + + In addition, you can go to the object bucket to view the backup files. The backups path is the application resource backup path, and the restic path is the PV data backup path. + + |image1| + +.. _cce_bestpractice_0311__section482103142819: + +Restoring Applications in the Target Cluster +-------------------------------------------- + +The storage infrastructure of an on-premises cluster is different from that of a cloud cluster. After the cluster is migrated, PVs cannot be mounted to pods. Therefore, during the migration, you need to update the storage class of the target cluster to shield the differences of underlying storage interfaces between the two clusters when creating a workload and request storage resources of the corresponding type. For details, see :ref:`Updating the Storage Class `. + +#. Use kubectl to connect to the CCE cluster. Create a storage class with the same name as that of the source cluster. + + In this example, the storage class name of the source cluster is **local** and the storage type is local disk. Local disks completely depend on the node availability. The data DR performance is poor. When the node is unavailable, the existing storage data is affected. Therefore, EVS volumes are used as storage resources in CCE clusters, and SAS disks are used as backend storage media. + + .. note:: + + - When an application containing PV data is restored in a CCE cluster, the defined storage class dynamically creates and mounts storage resources (such as EVS volumes) based on the PVC. + - The storage resources of the cluster can be changed as required, not limited to EVS volumes. To mount other types of storage, such as file storage and object storage, see :ref:`Updating the Storage Class `. + + YAML file of the migrated cluster: + + .. code-block:: + + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: local + provisioner: kubernetes.io/no-provisioner + volumeBindingMode: WaitForFirstConsumer + + The following is an example of the YAML file of the migration cluster: + + .. code-block:: + + allowVolumeExpansion: true + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: local + selfLink: /apis/storage.k8s.io/v1/storageclasses/csi-disk + parameters: + csi.storage.k8s.io/csi-driver-name: disk.csi.everest.io + csi.storage.k8s.io/fstype: ext4 + everest.io/disk-volume-type: SAS + everest.io/passthrough: "true" + provisioner: everest-csi-provisioner + reclaimPolicy: Delete + volumeBindingMode: Immediate + +#. Use the Velero tool to create a restore and specify a backup named **wordpress-backup** to restore the WordPress application to the CCE cluster. + + .. code-block:: + + velero restore create --from-backup wordpress-backup + + You can run the **velero restore get** statement to view the application restoration status. + +#. After the restoration is complete, check whether the application is running properly. If other adaptation problems may occur, rectify the fault by following the procedure described in :ref:`Updating Resources Accordingly `. + +.. |image1| image:: /_static/images/en-us_image_0000001480191270.png diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_outside_a_cluster.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_outside_a_cluster.rst new file mode 100644 index 0000000..e4403c0 --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/migrating_resources_outside_a_cluster.rst @@ -0,0 +1,57 @@ +:original_name: cce_bestpractice_0309.html + +.. _cce_bestpractice_0309: + +Migrating Resources Outside a Cluster +===================================== + +If your migration does not involve resources outside a cluster listed in :ref:`Table 1 ` or you do not need to use other services to update resources after the migration, skip this section. + +Migrating Container Images +-------------------------- + +To ensure that container images can be properly pulled after cluster migration and improve container deployment efficiency, you are advised to migrate private images to SoftWare Repository for Container (SWR). CCE works with SWR to provide a pipeline for automated container delivery. Images are pulled in parallel, which greatly improves container delivery efficiency. + +You need to manually migrate container images. + +#. Remotely log in to any node in the source cluster and run the **docker pull** command to pull all images to the local host. + +#. Log in to the SWR console, click **Login Command** in the upper right corner of the page, and copy the command. + +#. Run the copied login command on the node. + + The message "Login Succeeded" will be displayed upon a successful login. + +#. Add tags to all local images. + + .. code-block:: + + docker tag [Image name 1:tag 1] [Image repository address]/[Organization name]/[Image name 2:tag 2] + + - *[Image name 1*:*tag 1]*: name and tag of the local image to be pulled. + - *[Image repository address]*: You can query the image repository address on the SWR console. + - *[Organization name]*: Enter the name of the organization you created on the SWR console. + - *[Image name 2*:*tag 2]*: image name and tag displayed on the SWR console. + + **Example** + + .. code-block:: + + docker tag nginx:v1 swr.eu-de.otc.t-systems.com/cloud-develop/mynginx:v1 + +#. Run the **docker push** command to upload all local container image files to SWR. + + .. code-block:: + + docker push [Image repository address]/[Organization name]/[Image name 2:tag 2] + + **Example** + + .. code-block:: + + docker push swr.eu-de.otc.t-systems.com/cloud-develop/mynginx:v1 + +Migrating Databases and Storage (On-Demand) +------------------------------------------- + +You can determine whether to use **Relational Database Service (RDS)** and **Object Storage Service (OBS)** based on your production requirements. After the migration is complete, you need to reconfigure the database and storage for applications in the target CCE cluster. diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/performing_additional_tasks.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/performing_additional_tasks.rst new file mode 100644 index 0000000..57ce0ad --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/performing_additional_tasks.rst @@ -0,0 +1,34 @@ +:original_name: cce_bestpractice_0313.html + +.. _cce_bestpractice_0313: + +Performing Additional Tasks +=========================== + +Verifying Application Functions +------------------------------- + +Cluster migration involves full migration of application data, which may cause intra-application adaptation problems. In this example, after the cluster is migrated, the redirection link of the article published in WordPress is still the original domain name. If you click the article title, you will be redirected to the application in the source cluster. Therefore, you need to search for the original domain name in WordPress and replace it with the new domain name, change the values of **site_url** and primary URL in the database. For details, see `Changing The Site URL `__. + +Access the new address of the WordPress application. If the article published before the migration is displayed, the data of the persistent volume is successfully restored. + +|image1| + +Switching Live Traffic to the Target Cluster +-------------------------------------------- + +O&M personnel switch DNS to direct live traffic to the target cluster. + +- DNS traffic switching: Adjust the DNS configuration to switch traffic. +- Client traffic switching: Upgrade the client code or update the configuration to switch traffic. + +Bringing the Source Cluster Offline +----------------------------------- + +After confirming that the service on the target cluster is normal, bring the source cluster offline and delete the backup files. + +- Verify that the service on the target cluster is running properly. +- Bring the source cluster offline. +- Delete backup files. + +.. |image1| image:: /_static/images/en-us_image_0000001217183707.png diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/planning_resources_for_the_target_cluster.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/planning_resources_for_the_target_cluster.rst new file mode 100644 index 0000000..21126fc --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/planning_resources_for_the_target_cluster.rst @@ -0,0 +1,48 @@ +:original_name: cce_bestpractice_0308.html + +.. _cce_bestpractice_0308: + +Planning Resources for the Target Cluster +========================================= + +CCE allows you to customize cluster resources to meet various service requirements. :ref:`Table 1 ` lists the key performance parameters of a cluster and provides the planned values. You can set the parameters based on your service requirements. It is recommended that the performance configuration be the same as that of the source cluster. + +.. important:: + + After a cluster is created, the resource parameters marked with asterisks (``*``) in :ref:`Table 1 ` cannot be modified. + +.. _cce_bestpractice_0308__table1841815113913: + +.. table:: **Table 1** CCE cluster planning + + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ + | Resource | Key Performance Parameter | Description | Example Value | + +=================+===========================================================+==============================================================================================================================================================================================================================================================================================================================================================================================+=================================================================+ + | Cluster | **\***\ Cluster Type | - **CCE cluster**: supports VM nodes. You can run your containers in a secure and stable container runtime environment based on a high-performance network model. | CCE cluster | + | | | - **CCE Turbo cluster**: runs on a cloud native infrastructure that features software-hardware synergy to support passthrough networking, high security and reliability, intelligent scheduling, and BMS nodes. | | + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ + | | **\***\ Network Model | - **VPC network**: The container network uses VPC routing to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the route quota in a VPC network. | VPC network | + | | | - **Tunnel network**: The container network is an overlay tunnel network on top of a VPC network and uses the VXLAN technology. This network model is applicable when there is no high requirements on performance. | | + | | | - **Cloud Native Network 2.0**: The container network deeply integrates the elastic network interface (ENI) capability of VPC, uses the VPC CIDR block to allocate container addresses, and supports passthrough networking to containers through a load balancer. | | + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ + | | **\***\ Number of master nodes | - **3**: Three master nodes will be created to deliver better DR performance. If one master node is faulty, the cluster can still be available without affecting service functions. | 3 | + | | | - **1**: A single master node will be created. This mode is not recommended in commercial scenarios. | | + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ + | Node | OS | - EulerOS | EulerOS | + | | | - CentOS | | + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ + | | Node Specifications (vary depending on the actual region) | - **General-purpose**: provides a balance of computing, memory, and network resources. It is a good choice for many applications. General-purpose nodes can be used for web servers, workload development, workload testing, and small-scale databases. | General-purpose (node specifications: 4 vCPUs and 8 GiB memory) | + | | | - **Memory-optimized**: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive. | | + | | | - **GPU-accelerated**: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. GPU-accelerated nodes can be added only to clusters of v1.11 or later. | | + | | | - **General computing-plus**: provides stable performance and exclusive resources to enterprise-class workloads with high and stable computing performance. | | + | | | - **Disk-intensive**: supports local disk storage and provides high networking performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads. | | + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ + | | System Disk | - **Common I/O**: The backend storage media is SATA disks. | High I/O | + | | | - **High I/O**: The backend storage media is SAS disks. | | + | | | - **Ultra-high I/O**: The backend storage media is SSD disks. | | + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ + | | Storage Type | - **EVS volumes**: Mount an EVS volume to a container path. When containers are migrated, the attached EVS volumes are migrated accordingly. This storage mode is suitable for data that needs to be permanently stored. | EVS volumes | + | | | - **SFS volumes**: Create SFS volumes and mount them to a container path. The file system volumes created by the underlying SFS service can also be used. SFS volumes are applicable to persistent storage for frequent read/write in multiple workload scenarios, including media processing, content management, big data analysis, and workload analysis. | | + | | | - **OBS volumes**: Create OBS volumes and mount them to a container path. OBS volumes are applicable to scenarios such as cloud workload, data analysis, content analysis, and hotspot objects. | | + | | | - **SFS Turbo volumes**: Create SFS Turbo volumes and mount them to a container path. SFS Turbo volumes are fast, on-demand, and scalable, which makes them suitable for DevOps, containerized microservices, and enterprise office applications. | | + +-----------------+-----------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------+ diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/solution_overview.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/solution_overview.rst new file mode 100644 index 0000000..eb667e6 --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/solution_overview.rst @@ -0,0 +1,109 @@ +:original_name: cce_bestpractice_0307.html + +.. _cce_bestpractice_0307: + +Solution Overview +================= + +Scenario +-------- + +Containers are growing in popularity and Kubernetes simplifies containerized deployment. Many companies choose to build their own Kubernetes clusters. However, the O&M workload of on-premises clusters is heavy, and O&M personnel need to configure the management systems and monitoring solutions by themselves. This increases the labor costs while decreasing the efficiency. + +In terms of performance, an on-premises cluster has poor scalability due to its fixed specifications. Auto scaling cannot be implemented in case of traffic surges, which may easily result in the insufficient or waste of cluster resources. In addition, an on-premises cluster is usually deployed on a single node without considering disaster recovery risks. Once a fault occurs, the entire cluster cannot be used, which may cause serious production incident. + +Now you can address the preceding challenges by using CCE, a service that allows easy cluster management and flexible scaling, integrated with application service mesh and Helm charts to simplify cluster O&M and reduce operations costs. CCE is easy to use and delivers high performance, security, reliability, openness, and compatibility. This section describes the solution and procedure for migrating on-premises clusters to CCE. + +.. _cce_bestpractice_0307__section96147345128: + +Migration Solution +------------------ + +This section describes a cluster migration solution, which applies to the following types of clusters: + +- Kubernetes clusters built in local IDCs +- On-premises clusters built using multiple ECSs +- Cluster services provided by other cloud service providers + +Before the migration, you need to analyze all resources in the source clusters and then determine the migration solution. Resources that can be migrated include resources inside and outside the clusters, as listed in the following table. + +.. _cce_bestpractice_0307__table1126932541820: + +.. table:: **Table 1** Resources that can be migrated + + +-----------------------------+----------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Category | Migration Object | Remarks | + +=============================+========================================================================================+================================================================================================================================================================================================================================================================================================================================================+ + | Resources inside a cluster | All objects in a cluster, including pods, jobs, Services, Deployments, and ConfigMaps. | You are not advised to migrate the resources in the **velero** and **kube-system** namespaces. | + | | | | + | | | - **velero**: Resources in this namespace are created by the migration tool and do not need to be migrated. | + | | | - **kube-system**: Resources in this namespace are system resources. If this namespace of the source cluster contains resources created by users, migrate the resources on demand. | + | | | | + | | | .. caution:: | + | | | | + | | | CAUTION: | + | | | If you are migrating or backing up cluster resources in CCE, for example, from a namespace to another, do not back up Secret **paas.elb**. It is because secret **paas.elb** is periodically updated. After the backup is complete, the secret may become invalid when it is restored. As a result, network storage functions are affected. | + +-----------------------------+----------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | PersistentVolumes (PVs) mounted to containers | Due to restrictions of the Restic tool, migration is not supported for the hostPath storage volume. For details about how to solve the problem, see :ref:`Storage Volumes of the HostPath Type Cannot Be Backed Up `. | + +-----------------------------+----------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Resources outside a cluster | On-premises image repository | Resources can be migrated to SoftWare Repository for Container (SWR). | + +-----------------------------+----------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Non-containerized database | Resources can be migrated to Relational Database Service (RDS). | + +-----------------------------+----------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Non-local storage, such as object storage | Resources can be migrated to Object Storage Service (OBS). | + +-----------------------------+----------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +:ref:`Figure 1 ` shows the migration process. You can migrate resources outside a cluster as required. + +.. _cce_bestpractice_0307__fig203631140201419: + +.. figure:: /_static/images/en-us_image_0000001172392670.png + :alt: **Figure 1** Migration solution diagram + + **Figure 1** Migration solution diagram + +Migration Process +----------------- + +|image1| + +The cluster migration process is as follows: + +#. **Plan resources for the target cluster.** + + For details about the differences between CCE clusters and on-premises clusters, see **Key Performance Parameter** in :ref:`Planning Resources for the Target Cluster `. Plan resources as required and ensure that the performance configuration of the target cluster is the same as that of the source cluster. + +#. **Migrate resources outside a cluster.** + + If you need to migrate resources outside the cluster, see :ref:`Migrating Resources Outside a Cluster `. + +#. **Install the migration tool.** + + After resources outside a cluster are migrated, you can use a migration tool to back up and restore application configurations in the source and target clusters. For details about how to install the tool, see :ref:`Installing the Migration Tool `. + +#. **Migrate resources in the cluster.** + + Use Velero to back up resources in the source cluster to OBS and restore the resources in the target cluster. For details, see :ref:`Migrating Resources in a Cluster `. + + - :ref:`Backing Up Applications in the Source Cluster ` + + To back up resources, use the Velero tool to create a backup object in the original cluster, query and back up cluster data and resources, package the data, and upload the package to the object storage that is compatible with the S3 protocol. Cluster resources are stored in the JSON format. + + - :ref:`Restoring Applications in the Target Cluster ` + + During restoration in the target cluster, Velero specifies the temporary object bucket that stores the backup data, downloads the backup data to the new cluster, and redeploys resources based on the JSON file. + +#. **Update resources accordingly.** + + After the migration, cluster resources may fail to be deployed. You need to update the faulty resources. The possible adaptation problems are as follows: + + - :ref:`Updating Images ` + - :ref:`Updating Services ` + - :ref:`Updating the Storage Class ` + - :ref:`Updating Databases ` + +#. **Perform additional tasks.** + + After cluster resources are properly deployed, verify application functions after the migration and switch service traffic to the target cluster. After confirming that all services are running properly, bring the source cluster offline. + +.. |image1| image:: /_static/images/en-us_image_0000001218074121.png diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/troubleshooting.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/troubleshooting.rst new file mode 100644 index 0000000..864f9c6 --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/troubleshooting.rst @@ -0,0 +1,118 @@ +:original_name: cce_bestpractice_0314.html + +.. _cce_bestpractice_0314: + +Troubleshooting +=============== + +.. _cce_bestpractice_0314__section11197194820367: + +Storage Volumes of the HostPath Type Cannot Be Backed Up +-------------------------------------------------------- + +Both HostPath and Local volumes are local storage volumes. However, the Restic tool integrated in Velero cannot back up the PVs of the HostPath type and supports only the Local type. Therefore, you need to replace the storage volumes of the HostPath type with the Local type in the source cluster. + +.. note:: + + It is recommended that Local volumes be used in Kubernetes v1.10 or later and can only be statically created. For details, see `local `__. + +#. Create a storage class for the Local volume. + + Example YAML: + + .. code-block:: + + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: local + provisioner: kubernetes.io/no-provisioner + volumeBindingMode: WaitForFirstConsumer + +#. Change the **hostPath** field to the **local** field, specify the original local disk path of the host machine, and add the **nodeAffinity** field. + + Example YAML: + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolume + metadata: + name: mysql-pv + labels: + app: mysql + spec: + accessModes: + - ReadWriteOnce + capacity: + storage: 5Gi + storageClassName: local # Storage class created in the previous step + persistentVolumeReclaimPolicy: Delete + local: + path: "/mnt/data" # Path of the attached local disk + nodeAffinity: + required: + nodeSelectorTerms: + - matchExpressions: + - key: kubernetes.io/hostname + operator: Exists + +#. Run the following commands to verify the creation result: + + .. code-block:: + + kubectl get pv + + Information similar to the following is displayed: + + .. code-block:: + + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + mysql-pv 5Gi RWO Delete Available local 3s + +.. _cce_bestpractice_0314__section321054511332: + +Backup Tool Resources Are Insufficient +-------------------------------------- + +In the production environment, if there are many backup resources, for example, the default resource size of the backup tool is used, the resources may be insufficient. In this case, perform the following steps to adjust the CPU and memory size allocated to the Velero and Restic: + +**Before installing Velero:** + +You can specify the size of resources used by Velero and Restic when :ref:`installing Velero `. + +The following is an example of installation parameters: + +.. code-block:: + + velero install \ + --velero-pod-cpu-request 500m \ + --velero-pod-mem-request 1Gi \ + --velero-pod-cpu-limit 1000m \ + --velero-pod-mem-limit 1Gi \ + --use-restic \ + --restic-pod-cpu-request 500m \ + --restic-pod-mem-request 1Gi \ + --restic-pod-cpu-limit 1000m \ + --restic-pod-mem-limit 1Gi + +**After Velero is installed:** + +#. Edit the YAML files of the Velero and Restic workloads in the **velero** namespace. + + .. code-block:: + + kubectl edit deploy velero -n velero + kubectl edit deploy restic -n velero + +#. Modify the resource size under the **resources** field. The modification is the same for the Velero and Restic workloads, as shown in the following: + + .. code-block:: + + resources: + limits: + cpu: "1" + memory: 1Gi + requests: + cpu: 500m + memory: 1Gi diff --git a/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/updating_resources_accordingly.rst b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/updating_resources_accordingly.rst new file mode 100644 index 0000000..0824c54 --- /dev/null +++ b/umn/source/best_practice/migration/migrating_on-premises_kubernetes_clusters_to_cce/updating_resources_accordingly.rst @@ -0,0 +1,203 @@ +:original_name: cce_bestpractice_0312.html + +.. _cce_bestpractice_0312: + +Updating Resources Accordingly +============================== + +.. _cce_bestpractice_0312__section7125750134820: + +Updating Images +--------------- + +The WordPress and MySQL images used in this example can be pulled from SWR. Therefore, the image pull failure (ErrImagePull) will not occur. If the application to be migrated is created from a private image, perform the following steps to update the image: + +#. Migrate the image resources to SWR. For details, see `Uploading an Image Through a Container Engine Client `__. + +#. Log in to the SWR console and obtain the image path used after the migration. + + The image path is in the following format: + + .. code-block:: + + 'swr.{Region}.otc.t-systems.com/{Organization name}/{Image name}:{Tag}' + +#. Run the following command to modify the workload and replace the **image** field in the YAML file with the image path: + + .. code-block:: + + kubectl edit deploy wordpress + +#. Check the running status of the workload. + +.. _cce_bestpractice_0312__section41282507482: + +Updating Services +----------------- + +After the cluster is migrated, the Service of the source cluster may fail to take effect. You can perform the following steps to update the Service. If ingresses are configured in the source cluster, you need to connect the new cluster to ELB again after the migration. For details, see `Using kubectl to Create an ELB Ingress `__. + +#. Connect to the cluster using kubectl. + +#. Edit the YAML file of the corresponding Service to change the Service type and port number. + + .. code-block:: + + kubectl edit svc wordpress + + To update load balancer resources, you need to connect to ELB again. Add the annotations by following the procedure described in `LoadBalancer `__. + + .. code-block:: + + annotations: + kubernetes.io/elb.class: union # Shared load balancer + kubernetes.io/elb.id: 9d06a39d-xxxx-xxxx-xxxx-c204397498a3 # Load balancer ID, which can be queried on the ELB console. + kubernetes.io/elb.subnet-id: f86ba71c-xxxx-xxxx-xxxx-39c8a7d4bb36 # ID of the cluster where the subnet resides + kubernetes.io/session-affinity-mode: SOURCE_IP # Enable the sticky session based on the source IP address. + +#. Use a browser to check whether the Service is available. + +.. _cce_bestpractice_0312__section746195321414: + +Updating the Storage Class +-------------------------- + +As the storage infrastructures of clusters may be different, storage volumes cannot be mounted to the target cluster. You can use either of the following methods to update the volumes: + +.. important:: + + Both update methods can be performed only before the application is restored in the target cluster. Otherwise, PV data resources may fail to be restored. In this case, use the Velero to restore applications after the storage class update is complete. For details, see :ref:`Restoring Applications in the Target Cluster `. + +**Method 1: Creating a ConfigMap mapping** + +#. Create a ConfigMap in the CCE cluster and map the storage class used by the source cluster to the default storage class of the CCE cluster. + + .. code-block:: + + apiVersion: v1 + kind: ConfigMap + metadata: + name: change-storageclass-plugin-config + namespace: velero + labels: + app.kubernetes.io/name: velero + velero.io/plugin-config: "true" + velero.io/change-storage-class: RestoreItemAction + data: + {Storage class name01 in the source cluster}: {Storage class name01 in the target cluster} + {Storage class name02 in the source cluster}: {Storage class name02 in the target cluster} + +#. Run the following command to apply the ConfigMap configuration: + + .. code-block:: + + $ kubectl create -f change-storage-class.yaml + configmap/change-storageclass-plugin-config created + +**Method 2: Creating a storage class with the same name** + +#. Run the following command to query the default storage class supported by CCE: + + .. code-block:: + + kubectl get sc + + Information similar to the following is displayed: + + .. code-block:: + + NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE + csi-disk everest-csi-provisioner Delete Immediate true 3d23h + csi-disk-topology everest-csi-provisioner Delete WaitForFirstConsumer true 3d23h + csi-nas everest-csi-provisioner Delete Immediate true 3d23h + csi-obs everest-csi-provisioner Delete Immediate false 3d23h + csi-sfsturbo everest-csi-provisioner Delete Immediate true 3d23h + + .. table:: **Table 1** Storage classes + + ================= ======================== + Storage Class Storage Resource + ================= ======================== + csi-disk EVS + csi-disk-topology EVS with delayed binding + csi-nas SFS + csi-obs OBS + csi-sfsturbo SFS Turbo + ================= ======================== + +#. Run the following command to export the required storage class details in YAML format: + + .. code-block:: + + kubectl get sc -o=yaml + +#. Copy the YAML file and create a new storage class. + + Change the storage class name to the name used in the source cluster to call basic storage resources of the cloud. + + The YAML file of csi-obs is used as an example. Delete the unnecessary information in italic under the **metadata** field and modify the information in bold. You are advised not to modify other parameters. + + .. code-block:: + + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + creationTimestamp: "2021-10-18T06:41:36Z" + name: # Use the name of the storage class used in the source cluster. + resourceVersion: "747" + selfLink: /apis/storage.k8s.io/v1/storageclasses/csi-obs + uid: 4dbbe557-ddd1-4ce8-bb7b-7fa15459aac7 + parameters: + csi.storage.k8s.io/csi-driver-name: obs.csi.everest.io + csi.storage.k8s.io/fstype: obsfs + everest.io/obs-volume-type: STANDARD + provisioner: everest-csi-provisioner + reclaimPolicy: Delete + volumeBindingMode: Immediate + + .. note:: + + - SFS Turbo file systems cannot be directly created using StorageClass. You need to go to the SFS Turbo console to create SFS Turbo file systems that belong to the same VPC subnet and have inbound ports (111, 445, 2049, 2051, 2052, and 20048) enabled in the security group. + - CCE does not support EVS disks of the ReadWriteMany type. If resources of this type exist in the source cluster, change the storage type to **ReadWriteOnce**. + +#. Restore the cluster application by referring to :ref:`Restoring Applications in the Target Cluster ` and check whether the PVC is successfully created. + + .. code-block:: + + kubectl get pvc + + In the command output, the **VOLUME** column indicates the name of the PV automatically created using the storage class. + + .. code-block:: + + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE + pvc Bound pvc-4c8e655a-1dbc-4897-ae6c-446b502f5e77 5Gi RWX local 13s + +.. _cce_bestpractice_0312__section728213614323: + +Updating Databases +------------------ + +In this example, the database is a local MySQL database and does not need to be reconfigured after the migration. + +.. note:: + + - If the RDS instance is in the same VPC as the CCE cluster, it can be accessed using the private IP address. Otherwise, it can only be accessed only through public networks by binding an EIP. You are advised to use the private network access mode for high security and good RDS performance. + - Ensure that the inbound rule of the security group to which RDS belongs has been enabled for the cluster. Otherwise, the connection will fail. + +#. Log in to the RDS console and obtain the private IP address and port number of the DB instance on the **Basic Information** page. + +#. Run the following command to modify the WordPress workload: + + .. code-block:: + + kubectl edit deploy wordpress + + Set the environment variables in the **env** field. + + - **WORDPRESS_DB_HOST**: address and port number used for accessing the database, that is, the internal network address and port number obtained in the previous step. + - **WORDPRESS_DB_USERU**: username for accessing the database. + - **WORDPRESS_DB_PASSWORD**: password for accessing the database. + - **WORDPRESS_DB_NAME**: name of the database to be connected. + +#. Check whether the RDS database is properly connected. diff --git a/umn/source/best_practice/networking/implementing_sticky_session_through_load_balancing.rst b/umn/source/best_practice/networking/implementing_sticky_session_through_load_balancing.rst new file mode 100644 index 0000000..4196a6b --- /dev/null +++ b/umn/source/best_practice/networking/implementing_sticky_session_through_load_balancing.rst @@ -0,0 +1,189 @@ +:original_name: cce_bestpractice_00231.html + +.. _cce_bestpractice_00231: + +Implementing Sticky Session Through Load Balancing +================================================== + +Concepts +-------- + +Session persistence is one of the most common while complex problems in load balancing. + +Session persistence is also called sticky sessions. After the sticky session function is enabled, requests from the same client are distributed to the same backend ECS by the load balancer for better continuity. + +In load balancing and sticky session, connection and session are two key concepts. When only load balancing is concerned, session and connection refer to the same thing. + +Simply put, if a user needs to log in, it can be regarded as a session; otherwise, a connection. + +The sticky session mechanism fundamentally conflicts with the basic functions of load balancing. A load balancer forwards requests from clients to multiple backend servers to avoid overload on a single server. However, sticky session requires that some requests be forwarded to the same server for processing. Therefore, you need to select a proper sticky session mechanism based on the application environment. + +Layer-4 Load Balancing (Service) +-------------------------------- + +In layer-4 load balancing, source IP address-based sticky session (Hash routing based on the client IP address) can be enabled. To enable source IP address-based sticky session on Services, the following conditions must be met: + +#. **Service Affinity** of the Service is set to **Node level** (that is, the value of the **externalTrafficPolicy** field of the Service is **Local**). + +#. Enable the source IP address-based sticky session in the load balancing configuration of the Service. + + .. code-block:: + + apiVersion: v1 + kind: Service + metadata: + name: svc-example + namespace: default + annotations: + kubernetes.io/elb.class: union + kubernetes.io/elb.id: 56dcc1b4-8810-480c-940a-a44f7736f0dc + kubernetes.io/elb.lb-algorithm: ROUND_ROBIN + kubernetes.io/elb.session-affinity-mode: SOURCE_IP + spec: + selector: + app: nginx + externalTrafficPolicy: Local + ports: + - name: cce-service-0 + targetPort: 80 + nodePort: 32633 + port: 80 + protocol: TCP + type: LoadBalancer + +#. Anti-affinity is enabled for the backend application corresponding to the Service. + +Layer-7 Load Balancing (Ingress) +-------------------------------- + +In layer-7 load balancing, sticky session based on HTTP cookies and app cookies can be enabled. To enable such sticky session, the following conditions must be met: + +#. The application (workload) corresponding to the ingress is enabled with workload anti-affinity. +#. Node affinity is enabled for the Service corresponding to the ingress. + +**Procedure** + +#. Create a Nginx workload. + + Set the number of pods to 3 and set the podAntiAffinity. + + .. code-block:: + + kind: Deployment + apiVersion: apps/v1 + metadata: + name: nginx + namespace: default + spec: + replicas: 3 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: container-0 + image: 'nginx:perl' + resources: + limits: + cpu: 250m + memory: 512Mi + requests: + cpu: 250m + memory: 512Mi + imagePullSecrets: + - name: default-secret + affinity: + podAntiAffinity: # Pod anti-affinity. + requiredDuringSchedulingIgnoredDuringExecution: + - labelSelector: + matchExpressions: + - key: app + operator: In + values: + - nginx + topologyKey: kubernetes.io/hostname + +#. Creating a NodePort Service + + Configure the sticky session in a Service. An ingress can connect to multiple Services, and each Service can have different sticky sessions. + + .. code-block:: + + apiVersion: v1 + kind: Service + metadata: + name: nginx + namespace: default + annotations: + kubernetes.io/elb.lb-algorithm: ROUND_ROBIN + kubernetes.io/elb.session-affinity-mode: HTTP_COOKIE # HTTP cookie type. + kubernetes.io/elb.session-affinity-option: '{"persistence_timeout":"1440"}' # Session stickiness duration, in minutes. The value ranges from 1 to 1440. + spec: + selector: + app: nginx + ports: + - name: cce-service-0 + protocol: TCP + port: 80 + targetPort: 80 + nodePort: 32633 # Node port number. + type: NodePort + externalTrafficPolicy: Local # Node-level forwarding. + + You can also select **APP_COOKIE**. + + .. code-block:: + + apiVersion: v1 + kind: Service + metadata: + name: nginx + namespace: default + annotations: + kubernetes.io/elb.lb-algorithm: ROUND_ROBIN + kubernetes.io/elb.session-affinity-mode: APP_COOKIE # Select APP_COOKIE. + kubernetes.io/elb.session-affinity-option: '{"app_cookie_name":"test"}' # Application cookie name. + ... + +#. Create an ingress and associate it with a Service. The following example describes how to automatically create a shared load balancer. For details about how to specify other types of load balancers, see `Using kubectl to Create an ELB Ingress `__. + + .. code-block:: + + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: ingress-test + namespace: default + annotations: + kubernetes.io/elb.class: union + kubernetes.io/elb.port: '80' + kubernetes.io/elb.autocreate: + '{ + "type":"public", + "bandwidth_name":"cce-bandwidth-test", + "bandwidth_chargemode":"traffic", + "bandwidth_size":1, + "bandwidth_sharetype":"PER", + "eip_type":"5_bgp" + }' + spec: + rules: + - host: 'www.example.com' + http: + paths: + - path: '/' + backend: + service: + name: nginx #Service name + port: + number: 80 + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce + +#. Log in to the ELB console, access the load balancer details page, and check whether the sticky session feature is enabled. diff --git a/umn/source/best_practice/networking/index.rst b/umn/source/best_practice/networking/index.rst new file mode 100644 index 0000000..a6560a3 --- /dev/null +++ b/umn/source/best_practice/networking/index.rst @@ -0,0 +1,20 @@ +:original_name: cce_bestpractice_0052.html + +.. _cce_bestpractice_0052: + +Networking +========== + +- :ref:`Planning CIDR Blocks for a Cluster ` +- :ref:`Selecting a Network Model ` +- :ref:`Implementing Sticky Session Through Load Balancing ` +- :ref:`Obtaining the Client Source IP Address for a Container ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + planning_cidr_blocks_for_a_cluster + selecting_a_network_model + implementing_sticky_session_through_load_balancing + obtaining_the_client_source_ip_address_for_a_container diff --git a/umn/source/best_practice/networking/obtaining_the_client_source_ip_address_for_a_container.rst b/umn/source/best_practice/networking/obtaining_the_client_source_ip_address_for_a_container.rst new file mode 100644 index 0000000..8c90f70 --- /dev/null +++ b/umn/source/best_practice/networking/obtaining_the_client_source_ip_address_for_a_container.rst @@ -0,0 +1,78 @@ +:original_name: cce_bestpractice_00035.html + +.. _cce_bestpractice_00035: + +Obtaining the Client Source IP Address for a Container +====================================================== + +Background +---------- + +There may be different types of proxy servers between a client and a container server. How can a container obtain the real source IP address of the client? This section describes several scenarios you may encounter. + +Principles +---------- + +|image1| + +**Layer-7 forwarding:** + +Ingress: If this access mode is used, the client source IP address is saved in the **X-Forwarded-For** HTTP header field by default. No other configuration is required. + +- ELB ingress: A self-developed ingress to implement layer-7 network access between the internet and intranet (in the same VPC) based on ELB. If the backend Service type is **NodePort**, set **Service Affinity** to **Node level**. + +**Layer-4 forwarding:** + +- LoadBalancer: Use ELB to achieve load balancing. You can manually enable the **Obtain Client IP Address** option for TCP and UDP listeners of shared load balancers. By default, the **Obtain Client IP Address** option is enabled for TCP and UDP listeners of dedicated load balancers. You do not need to manually enable it. +- NodePort: In this access mode, the container port is mapped to the node port. If cluster-level affinity is configured, access requests will be forwarded through the node and the client source IP address cannot be obtained. If node-level affinity is configured, access requests are not forwarded and the client source IP address can be obtained. + +Ingress +------- + +Configure the application server and obtain the IP address of a client from the HTTP header. + +The real IP address is placed in the **X-Forwarded-For** HTTP header field by the load balancer in the following format: + +.. code-block:: + + X-Forwarded-For: IP address of the client,Proxy server 1-IP address,Proxy server 2-IP address,... + +If you use this method, the first IP address obtained is the IP address of the client. + +For details, see `How Can I Obtain the IP Address of a Client? `__ + +.. note:: + + - When adding an ingress, if the backend service is of the NodePort type, set **Service Affinity** to **Node level**, that is, set **spec.externalTrafficPolicy** to **Local**. For details, see :ref:`NodePort `. + +LoadBalancer +------------ + +For a LoadBalancer Service, different types of clusters obtain source IP addresses in different scenarios. In some scenarios, source IP addresses cannot be obtained currently. + +**VPC and Container Tunnel Network Models** + +To obtain source IP addresses, perform the following steps: + +#. When creating a LoadBalancer Service on the CCE console, set **Service Affinity** to **Node level** instead of **Cluster level**. +#. Go to the ELB console and enable the function of obtaining the client IP address of the listener corresponding to the load balancer. **Transparent transmission of source IP addresses is enabled for dedicated load balancers by default. You do not need to manually enable this function.** + + a. Log in to the ELB console. + b. Click |image2| in the upper left corner to select the desired region and project. + c. Click **Service List**. Under **Networking**, click **Elastic Load Balance**. + d. On the **Load Balancers** page, click the name of the load balancer. + e. Click **Listeners**. + f. To add a listener, click **Add Listener**. + g. To modify a listener, locate the listener and click |image3| on the right of its name. + h. Enable **Obtain Client IP Address**. + +.. _cce_bestpractice_00035__section6340152911914: + +NodePort +-------- + +Set the service affinity of a NodePort Service to **Node level** instead of **Cluster level**. That is, set **spec.externalTrafficPolicy** of the Service to **Local**. + +.. |image1| image:: /_static/images/en-us_image_0000001176818150.png +.. |image2| image:: /_static/images/en-us_image_0000001221501677.png +.. |image3| image:: /_static/images/en-us_image_0000001221820189.png diff --git a/umn/source/best_practice/networking/planning_cidr_blocks_for_a_cluster.rst b/umn/source/best_practice/networking/planning_cidr_blocks_for_a_cluster.rst new file mode 100644 index 0000000..77090e8 --- /dev/null +++ b/umn/source/best_practice/networking/planning_cidr_blocks_for_a_cluster.rst @@ -0,0 +1,180 @@ +:original_name: cce_bestpractice_00004.html + +.. _cce_bestpractice_00004: + +Planning CIDR Blocks for a Cluster +================================== + +Before creating a cluster on CCE, determine the number of VPCs, number of subnets, container CIDR blocks, and Services for access based on service requirements. + +This topic describes the addresses in a CCE cluster in a VPC and how to plan CIDR blocks. + +Notes and Constraints +--------------------- + +To access a CCE cluster through a VPN, ensure that the VPN does not conflict with the VPC CIDR block where the cluster resides and the container CIDR block. + +Basic Concepts +-------------- + +- **VPC CIDR Block** + + Virtual Private Cloud (VPC) enables you to provision logically isolated, configurable, and manageable virtual networks for cloud servers, cloud containers, and cloud databases. You have complete control over your virtual network, including selecting your own CIDR block, creating subnets, and configuring security groups. You can also assign EIPs and allocate bandwidth in your VPC for secure and easy access to your business system. + +- **Subnet CIDR Block** + + A subnet is a network that manages ECS network planes. It supports IP address management and DNS. The IP addresses of all ECSs in a subnet belong to the subnet. + + + .. figure:: /_static/images/en-us_image_0261818822.png + :alt: **Figure 1** VPC CIDR block architecture + + **Figure 1** VPC CIDR block architecture + + By default, ECSs in all subnets of the same VPC can communicate with one another, while ECSs in different VPCs cannot communicate with each other. + + You can create a peering connection on VPC to enable ECSs in different VPCs to communicate with each other. + +- **Container (Pod) CIDR Block** + + Pod is a Kubernetes concept. Each pod has an IP address. + + When creating a cluster on CCE, you can specify the pod (container) CIDR block, which cannot overlap with the subnet CIDR block. For example, if the subnet CIDR block is 192.168.0.0/16, the container CIDR block cannot be 192.168.0.0/18 or 192.168.1.0/18, because these addresses are included in 192.168.0.0/16. + +- **Container Subnet** (Only for CCE Turbo Clusters) + + In a CCE Turbo cluster, a container is assigned an IP address from the CIDR block of a VPC. The container subnet can overlap with the subnet CIDR block. Note that the subnet you select determines the maximum number of pods in the cluster. After a cluster is created, you can only add container subnets but cannot delete them. + +- **Service CIDR Block** + + Service is also a Kubernetes concept. Each Service has an address. When creating a cluster on CCE, you can specify the Service CIDR block. Similarly, the Service CIDR block cannot overlap with the subnet CIDR block or the container CIDR block. The Service CIDR block can be used only within a cluster. + +Single-VPC Single-Cluster Scenarios +----------------------------------- + +**CCE Clusters**: include clusters in VPC network model and container tunnel network model. :ref:`Figure 2 ` shows the CIDR block planning of a cluster. + +- VPC CIDR Block: specifies the VPC CIDR block where the cluster resides. The size of this CIDR block affects the maximum number of nodes that can be created in the cluster. +- Subnet CIDR Block: specifies the subnet CIDR block where the node in the cluster resides. The subnet CIDR block is included in the VPC CIDR block. Different nodes in the same cluster can be allocated to different subnet CIDR blocks. +- Container CIDR Block: cannot overlap with the subnet CIDR block. +- Service CIDR Block: cannot overlap with the subnet CIDR block or the container CIDR block. + +.. _cce_bestpractice_00004__en-us_topic_0099587154_fig15791152874920: + +.. figure:: /_static/images/en-us_image_0000001392318380.png + :alt: **Figure 2** Network CIDR block planning in the single-VPC single-cluster scenario (CCE cluster) + + **Figure 2** Network CIDR block planning in the single-VPC single-cluster scenario (CCE cluster) + +:ref:`Figure 3 ` shows the CIDR block planning for a **CCE Turbo cluster** (cloud native network 2.0). + +- VPC CIDR Block: specifies the VPC CIDR block where the cluster resides. The size of this CIDR block affects the maximum number of nodes that can be created in the cluster. +- Subnet CIDR Block: specifies the subnet CIDR block where the node in the cluster resides. The subnet CIDR block is included in the VPC CIDR block. Different nodes in the same cluster can be allocated to different subnet CIDR blocks. +- Container Subnet CIDR Block: The container subnet is included in the VPC CIDR block and can overlap with the subnet CIDR block or even be the same as the subnet CIDR block. Note that the container subnet size determines the maximum number of containers in the cluster because IP addresses in the VPC are directly allocated to containers. After a cluster is created, you can only add container subnets but cannot delete them. You are advised to set a larger IP address segment for the container subnet to prevent insufficient container IP addresses. +- Service CIDR Block: cannot overlap with the subnet CIDR block or the container CIDR block. + +.. _cce_bestpractice_00004__fig19746213285: + +.. figure:: /_static/images/en-us_image_0000001392280374.png + :alt: **Figure 3** CIDR block planning in the single-VPC single-cluster scenario (CCE Turbo cluster) + + **Figure 3** CIDR block planning in the single-VPC single-cluster scenario (CCE Turbo cluster) + +**Single-VPC Multi-Cluster Scenarios** +-------------------------------------- + +**VPC network model** + +Pod packets are forwarded through VPC routes. CCE automatically configures a routing table on the VPC routes to each container CIDR block. The network scale is limited by the VPC route table. :ref:`Figure 4 ` shows the CIDR block planning of the cluster. + +- VPC CIDR Block: specifies the VPC CIDR block where the cluster resides. The size of this CIDR block affects the maximum number of nodes that can be created in the cluster. +- Subnet CIDR Block: The subnet CIDR block in each cluster cannot overlap with the container CIDR block. +- Container CIDR Block: If multiple VPC network model clusters exist in a single VPC, the container CIDR blocks of all clusters cannot overlap because the clusters use the same routing table. In this case, CCE clusters are partially interconnected. A pod of a cluster can directly access the pods of another cluster, but cannot access the Services of the cluster. +- Service CIDR Block: can be used only in clusters. Therefore, the service CIDR blocks of different clusters can overlap, but cannot overlap with the subnet CIDR block and container CIDR block of the cluster to which the clusters belong. + +.. _cce_bestpractice_00004__en-us_topic_0099587154_fig69527530400: + +.. figure:: /_static/images/en-us_image_0261818824.png + :alt: **Figure 4** VPC network - multi-cluster scenario + + **Figure 4** VPC network - multi-cluster scenario + +**Tunnel Network** + +Though at some cost of performance, the tunnel encapsulation enables higher interoperability and compatibility with advanced features (such as network policy-based isolation), meeting the requirements of most applications. :ref:`Figure 5 ` shows the CIDR block planning of the cluster. + +- VPC CIDR Block: specifies the VPC CIDR block where the cluster resides. The size of this CIDR block affects the maximum number of nodes that can be created in the cluster. +- Subnet CIDR Block: The subnet CIDR block in each cluster cannot overlap with the container CIDR block. +- Container CIDR Block: The container CIDR blocks of all clusters can overlap. In this case, pods in different clusters cannot be directly accessed using IP addresses. It is recommended that ELB be used for the cross-cluster access between containers. +- Service CIDR Block: can be used only in clusters. Therefore, the service CIDR blocks of different clusters can overlap, but cannot overlap with the subnet CIDR block and container CIDR block of the cluster to which the clusters belong. + +.. _cce_bestpractice_00004__en-us_topic_0099587154_fig8672112184219: + +.. figure:: /_static/images/en-us_image_0261818885.png + :alt: **Figure 5** Tunnel network - multi-cluster scenario + + **Figure 5** Tunnel network - multi-cluster scenario + +**Cloud native network 2.0 network model** (CCE Turbo cluster) + +In this mode, container IP addresses are allocated from the VPC CIDR block. ELB passthrough networking is supported to direct access requests to containers. Security groups and multiple types of VPC networks can be bound to deliver high performance. + +- VPC CIDR Block: specifies the VPC CIDR block where the cluster resides. In a CCE Turbo cluster, the CIDR block size affects the total number of nodes and containers that can be created in the cluster. +- Subnet CIDR Block: There is no special restriction on the subnet CIDR blocks in CCE Turbo clusters. +- Container Subnet: The CIDR block of the container subnet is included in the VPC CIDR block. Container subnets in different clusters can overlap with each other or overlap with the subnet CIDR block. However, you are advised to stagger the container CIDR blocks of different clusters and ensure that the container subnet CIDR blocks have sufficient IP addresses. In this case, pods in different clusters can directly access each other through IP addresses. +- Service CIDR Block: can be used only in clusters. Therefore, the service CIDR blocks of different clusters can overlap, but cannot overlap with the subnet CIDR block or container CIDR block. + + +.. figure:: /_static/images/en-us_image_0000001392259910.png + :alt: **Figure 6** Cloud native network 2.0 network model - multi-cluster scenario + + **Figure 6** Cloud native network 2.0 network model - multi-cluster scenario + +**Coexistence of Clusters in Multi-Network** + +When a VPC contains clusters created with different network models, comply with the following rules when creating a cluster: + +- VPC CIDR Block: In this scenario, all clusters are located in the same VPC CIDR block. Ensure that there are sufficient available IP addresses in the VPC. +- Subnet CIDR Block: Ensure that the subnet CIDR block does not overlap with the container CIDR block. Even in some scenarios (for example, coexistence with CCE Turbo clusters), the subnet CIDR block can overlap with the container (subnet) CIDR block. However, this is not recommended. +- Container CIDR Block: Ensure that the container CIDR blocks of clusters in **VPC network model** do not overlap. +- Service CIDR Block: The service CIDR blocks of all clusters can overlap, but cannot overlap with the subnet CIDR block and container CIDR block of the cluster. + +Cross-VPC Cluster Interconnection +--------------------------------- + +When two VPC networks are interconnected, you can configure the packets to be sent to the peer VPC in the route table. + +In the VPC network model, after creating a peering connection, you need to add routes for the peering connection to enable communication between the two VPCs. + + +.. figure:: /_static/images/en-us_image_0261818886.png + :alt: **Figure 7** VPC Network - VPC interconnection scenario + + **Figure 7** VPC Network - VPC interconnection scenario + +When creating a VPC peering connection between containers across VPCs, pay attention to the following points: + +- The VPC to which the clusters belong must not overlap. In each cluster, the subnet CIDR block cannot overlap with the container CIDR block. +- The container CIDR blocks of clusters cannot overlap, but the Service CIDR blocks can. +- You need to add not only the peer VPC CIDR block but also the peer container CIDR block to the VPC routing tables at both ends. Note that this operation must be performed in the VPC route tables of the clusters. + +In the tunnel network model, after creating a peering connection, you need to add routes for the peering connection to enable communication between the two VPCs. + + +.. figure:: /_static/images/en-us_image_0000001082048529.png + :alt: **Figure 8** Tunnel network - VPC interconnection scenario + + **Figure 8** Tunnel network - VPC interconnection scenario + +Pay attention to the following: + +- The VPC of the clusters must not overlap. +- The container CIDR blocks of all clusters can overlap, so do the Service CIDR blocks. +- Add the peer subnet CIDR block to the route table of the VPC peering connection. + +In **Cloud Native Network 2.0** mode, after creating a VPC peering connection, you only need to add routes for the VPC peering connection to enable communication between the two VPCs. Ensure that the VPC of the clusters does not overlap. + +**VPC-IDC Scenarios** +--------------------- + +Similar to the VPC interconnection scenario, some CIDR blocks in the VPC are routed to the IDC. The pod IP addresses of CCE clusters cannot overlap with the addresses within these CIDR blocks. To access the pod IP addresses in the cluster in the IDC, you need to configure the route table to the private line VBR on the IDC. diff --git a/umn/source/best_practice/networking/selecting_a_network_model.rst b/umn/source/best_practice/networking/selecting_a_network_model.rst new file mode 100644 index 0000000..b48f95c --- /dev/null +++ b/umn/source/best_practice/networking/selecting_a_network_model.rst @@ -0,0 +1,70 @@ +:original_name: cce_bestpractice_00162.html + +.. _cce_bestpractice_00162: + +Selecting a Network Model +========================= + +CCE uses self-proprietary, high-performance container networking add-ons to support the tunnel network, Cloud Native Network 2.0, and VPC network models. + +.. caution:: + + After a cluster is created, the network model cannot be changed. Exercise caution when selecting a network model. + +- **Tunnel network**: The container network is an overlay tunnel network on top of a VPC network and uses the VXLAN technology. This network model is applicable when there is no high requirements on performance. VXLAN encapsulates Ethernet packets as UDP packets for tunnel transmission. Though at some cost of performance, the tunnel encapsulation enables higher interoperability and compatibility with advanced features (such as network policy-based isolation), meeting the requirements of most applications. + + + .. figure:: /_static/images/en-us_image_0000001145545261.png + :alt: **Figure 1** Container tunnel network + + **Figure 1** Container tunnel network + +- **VPC network**: The container network uses VPC routing to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the route quota in a VPC network. Each node is assigned a CIDR block of a fixed size. VPC networks are free from tunnel encapsulation overhead and outperform container tunnel networks. In addition, as VPC routing includes routes to node IP addresses and container network segment, container pods in the cluster can be directly accessed from outside the cluster. + + + .. figure:: /_static/images/en-us_image_0261818875.png + :alt: **Figure 2** VPC network + + **Figure 2** VPC network + +- **Cloud Native Network 2.0**: The container network deeply integrates the elastic network interface (ENI) capability of VPC, uses the VPC CIDR block to allocate container addresses, and supports passthrough networking to containers through a load balancer. + + + .. figure:: /_static/images/en-us_image_0000001352539924.png + :alt: **Figure 3** Cloud Native Network 2.0 + + **Figure 3** Cloud Native Network 2.0 + +The following table lists the differences between the network models. + +.. table:: **Table 1** Networking model comparison + + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | Dimension | Tunnel Network | VPC Network | Cloud Native Network 2.0 | + +========================+===================================================================================================================================+======================================================================================================================================================+========================================================================================================+ + | Core technology | OVS | IPvlan and VPC route | VPC ENI/sub-ENI | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | Applicable Clusters | CCE cluster | CCE cluster | CCE Turbo cluster | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | Network isolation | Kubernetes native NetworkPolicy for pods | No | Pods support security group isolation. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | Passthrough networking | No | No | Yes | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | IP address management | - The container CIDR block is allocated separately. | - The container CIDR block is allocated separately. | The container CIDR block is divided from the VPC subnet and does not need to be allocated separately. | + | | - CIDR blocks are divided by node and can be dynamically allocated (CIDR blocks can be dynamically added after being allocated.) | - CIDR blocks are divided by node and statically allocated (the CIDR block cannot be changed after a node is created). | | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | Performance | Performance loss due to VXLAN encapsulation | No tunnel encapsulation. Cross-node packets are forwarded through VPC routers, delivering performance equivalent to that of the host network. | The container network is integrated with the VPC network, eliminating performance loss. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | Networking scale | A maximum of 2,000 nodes are supported. | By default, 200 nodes are supported. | A maximum of 2,000 nodes are supported. | + | | | | | + | | | Each time a node is added to the cluster, a route is added to the VPC routing table. Therefore, the cluster scale is limited by the VPC route table. | | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + | Scenario | - Common container services | - Scenarios that have high requirements on network latency and bandwidth | - Scenarios that have high requirements on network latency, bandwidth, and performance | + | | - Scenarios that do not have high requirements on network latency and bandwidth | - Containers communicate with VMs using a microservice registration framework, such as Dubbo and CSE. | - Containers communicate with VMs using a microservice registration framework, such as Dubbo and CSE. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------+ + +.. important:: + + #. The scale of a cluster that uses the VPC network model is limited by the custom routes of the VPC. Therefore, you need to estimate the number of required nodes before creating a cluster. + #. The scale of a cluster that uses the Cloud Native Network 2.0 model depends on the size of the VPC subnet CIDR block selected for the network attachment definition. Before creating a cluster, evaluate the scale of your cluster. + #. By default, VPC routing network supports direct communication between containers and hosts in the same VPC. If a peering connection policy is configured between the VPC and another VPC, the containers can directly communicate with hosts on the peer VPC. In addition, in hybrid networking scenarios such as Direct Connect and VPN, communication between containers and hosts on the peer end can also be achieved with proper planning. diff --git a/umn/source/best_practice/security/cluster_security.rst b/umn/source/best_practice/security/cluster_security.rst new file mode 100644 index 0000000..ea7b068 --- /dev/null +++ b/umn/source/best_practice/security/cluster_security.rst @@ -0,0 +1,173 @@ +:original_name: cce_bestpractice_0317.html + +.. _cce_bestpractice_0317: + +Cluster Security +================ + +For security purposes, you are advised to configure a cluster as follows. + +Using the CCE Cluster of the Latest Version +------------------------------------------- + +Kubernetes releases a major version in about four months. CCE follows the same frequency as Kubernetes to release major versions. To be specific, a new CCE version is released about three months after a new Kubernetes version is released in the community. For example, Kubernetes v1.19 was released in September 2020 and CCE v1.19 was released in March 2021. + +The latest cluster version has known vulnerabilities fixed or provides a more comprehensive security protection mechanism. You are advised to select the latest cluster version when creating a cluster. Before a cluster version is deprecated and removed, upgrade your cluster to a supported version. + +Disabling the Automatic Token Mounting Function of the Default Service Account +------------------------------------------------------------------------------ + +By default, Kubernetes associates the default service account with every pod. That is, the token is mounted to a container. The container can use this token to pass the authentication by the kube-apiserver and kubelet components. In a cluster with RBAC disabled, the service account who owns the token has the control permissions for the entire cluster. In a cluster with RBAC enabled, the permissions of the service account who owns the token depends on the roles associated by the administrator. The service account's token is generally used by workloads that need to access kube-apiserver, such as coredns, autoscaler, and prometheus. For workloads that do not need to access kube-apiserver, you are advised to disable the automatic association between the service account and token. + +Two methods are available: + +- Method 1: Set the **automountServiceAccountToken** field of the service account to **false**. After the configuration is complete, newly created workloads will not be associated with the default service account by default. Set this field for each namespace as required. + + .. code-block:: + + apiVersion: v1 + kind: ServiceAccount + metadata: + name: default + automountServiceAccountToken: false + ... + + When a workload needs to be associated with a service account, explicitly set the **automountServiceAccountToken** field to **true** in the YAML file of the workload. + + .. code-block:: + + ... + spec: + template: + spec: + serviceAccountName: default + automountServiceAccountToken: true + ... + +- Method 2: Explicitly disable the function of automatically associating with service accounts for workloads. + + .. code-block:: + + ... + spec: + template: + spec: + automountServiceAccountToken: false + ... + +Configuring Proper Cluster Access Permissions for Users +------------------------------------------------------- + +CCE allows you to create multiple IAM users. Your account can create different user groups, assign different access permissions to different user groups, and add users to the user groups with corresponding permissions when creating IAM users. In this way, users can control permissions on different regions and assign read-only permissions. Your account can also assign namespace-level permissions for users or user groups. To ensure security, it is advised that minimum user access permissions are assigned. + +If you need to create multiple IAM users, configure the permissions of the IAM users and namespaces properly. + +Configuring Resource Quotas for Cluster Namespaces +-------------------------------------------------- + +CCE provides resource quota management, which allows users to limit the total amount of resources that can be allocated to each namespace. These resources include CPU, memory, storage volumes, pods, Services, Deployments, and StatefulSets. Proper configuration can prevent excessive resources created in a namespace from affecting the stability of the entire cluster. + +Configuring LimitRange for Containers in a Namespace +---------------------------------------------------- + +With resource quotas, cluster administrators can restrict the use and creation of resources by namespace. In a namespace, a pod or container can use the maximum CPU and memory resources defined by the resource quota of the namespace. In this case, a pod or container may monopolize all available resources in the namespace. You are advised to configure LimitRange to restrict resource allocation within the namespace. The LimitRange parameter has the following restrictions: + +- Limits the minimum and maximum resource usage of each pod or container in a namespace. + + For example, create the maximum and minimum CPU usage limits for a pod in a namespace as follows: + + cpu-constraints.yaml + + .. code-block:: + + apiVersion: v1 + kind: LimitRange + metadata: + name: cpu-min-max-demo-lr + spec: + limits: + - max: + cpu: "800m" + min: + cpu: "200m" + type: Container + + Then, run **kubectl -n** ** **create -f** *cpu-constraints.yaml* to complete the creation. If the default CPU usage is not specified for the container, the platform automatically configures the default CPU usage. That is, the default configuration is automatically added after the container is created. + + .. code-block:: + + ... + spec: + limits: + - default: + cpu: 800m + defaultRequest: + cpu: 800m + max: + cpu: 800m + min: + cpu: 200m + type: Container + +- Limits the maximum and minimum storage space that each PersistentVolumeClaim can apply for in a namespace. + + storagelimit.yaml + + .. code-block:: + + apiVersion: v1 + kind: LimitRange + metadata: + name: storagelimit + spec: + limits: + - type: PersistentVolumeClaim + max: + storage: 2Gi + min: + storage: 1Gi + + Then, run **kubectl -n** ** **create -f** *storagelimit.yaml* to complete the creation. + +Configuring Network Isolation in a Cluster +------------------------------------------ + +- Container tunnel network + + If networks need to be isolated between namespaces in a cluster or between workloads in the same namespace, you can configure network policies to isolate the networks. + +- Cloud Native Network 2.0 + + In the Cloud Native Network 2.0 model, you can configure security groups to isolate networks between pods. For details, see `SecurityGroups `__. + +- VPC network + + Network isolation is not supported. + +Enabling the Webhook Authentication Mode with kubelet +----------------------------------------------------- + +.. important:: + + CCE clusters of v1.15.6-r1 or earlier are involved, whereas versions later than v1.15.6-r1 are not. + + Upgrade the CCE cluster version to 1.13 or 1.15 and enable the RBAC capability for the cluster. If the version is 1.13 or later, no upgrade is required. + +When creating a node, you can enable the kubelet authentication mode by injecting the **postinstall** file (by setting the kubelet startup parameter **--authorization-node=Webhook**). + +#. Run the following command to create clusterrolebinding: + + **kubectl create clusterrolebinding kube-apiserver-kubelet-admin --clusterrole=system:kubelet-api-admin --user=system:kube-apiserver** + +#. For an existing node, log in to the node, change **authorization mode** in **/var/paas/kubernetes/kubelet/kubelet_config.yaml** on the node to **Webhook**, and restart kubelet. + + **sed -i s/AlwaysAllow/Webhook/g /var/paas/kubernetes/kubelet/kubelet_config.yaml; systemctl restart kubelet** + +#. For a new node, add the following command to the post-installation script to change the kubelet permission mode: + + **sed -i s/AlwaysAllow/Webhook/g /var/paas/kubernetes/kubelet/kubelet_config.yaml; systemctl restart kubelet** + +Uninstalling web-terminal After Use +----------------------------------- + +The web-terminal add-on can be used to manage CCE clusters. Keep the login password secure and uninstall the add-on when it is no longer needed. diff --git a/umn/source/best_practice/security/container_security.rst b/umn/source/best_practice/security/container_security.rst new file mode 100644 index 0000000..e16075e --- /dev/null +++ b/umn/source/best_practice/security/container_security.rst @@ -0,0 +1,131 @@ +:original_name: cce_bestpractice_0319.html + +.. _cce_bestpractice_0319: + +Container Security +================== + +Controlling the Pod Scheduling Scope +------------------------------------ + +The nodeSelector or nodeAffinity is used to limit the range of nodes to which applications can be scheduled, preventing the entire cluster from being threatened due to the exceptions of a single application. + +Suggestions on Container Security Configuration +----------------------------------------------- + +- Set the computing resource limits (**request** and **limit**) of a container. This prevents the container from occupying too many resources and affecting the stability of the host and other containers on the same node. +- Unless necessary, do not mount sensitive host directories to containers, such as **/**, **/boot**, **/dev**, **/etc**, **/lib**, **/proc**, **/sys**, and **/usr**. +- Do not run the sshd process in containers unless necessary. +- Unless necessary, it is not recommended that containers and hosts share the network namespace. +- Unless necessary, it is not recommended that containers and hosts share the process namespace. +- Unless necessary, it is not recommended that containers and hosts share the IPC namespace. +- Unless necessary, it is not recommended that containers and hosts share the UTS namespace. +- Unless necessary, do not mount the sock file of Docker to any container. + +Container Permission Access Control +----------------------------------- + +When using a containerized application, comply with the minimum privilege principle and properly set securityContext of Deployments or StatefulSets. + +- Configure runAsUser to specify a non-root user to run a container. + +- Configure privileged to prevent containers being used in scenarios where privilege is not required. + +- Configure capabilities to accurately control the privileged access permission of containers. + +- Configure allowPrivilegeEscalation to disable privilege escape in scenarios where privilege escalation is not required for container processes. + +- Configure seccomp to restrict the container syscalls. For details, see `Restrict a Container's Syscalls with seccomp `__ in the official Kubernetes documentation. + +- Configure ReadOnlyRootFilesystem to protect the root file system of a container. + + Example YAML for a Deployment: + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: security-context-example + namespace: security-example + spec: + replicas: 1 + selector: + matchLabels: + app: security-context-example + label: security-context-example + strategy: + rollingUpdate: + maxSurge: 25% + maxUnavailable: 25% + type: RollingUpdate + template: + metadata: + annotations: + seccomp.security.alpha.kubernetes.io/pod: runtime/default + labels: + app: security-context-example + label: security-context-example + spec: + containers: + - image: ... + imagePullPolicy: Always + name: security-context-example + securityContext: + allowPrivilegeEscalation: false + readOnlyRootFilesystem: true + runAsUser: 1000 + capabilities: + add: + - NET_BIND_SERVICE + drop: + - all + volumeMounts: + - mountPath: /etc/localtime + name: localtime + readOnly: true + - mountPath: /opt/write-file-dir + name: tmpfs-example-001 + securityContext: + seccompProfile: + type: RuntimeDefault + volumes: + - hostPath: + path: /etc/localtime + type: "" + name: localtime + - emptyDir: {} + name: tmpfs-example-001 + +Restricting the Access of Containers to the Management Plane +------------------------------------------------------------ + +If application containers on a node do not need to access Kubernetes, you can perform the following operations to disable containers from accessing kube-apiserver: + +#. Query the container CIDR block and private API server address. + + On the **Clusters** page of the CCE console, click the name of the cluster to find the information on the details page. + +#. Log in to each node in the CCE cluster as user **root** and run the following command: + + - VPC network: + + .. code-block:: + + iptables -I OUTPUT -s {container_cidr} -d {Private API server IP} -j REJECT + + - Container tunnel network: + + .. code-block:: + + iptables -I FORWARD -s {container_cidr} -d {Private API server IP} -j REJECT + + *{container_cidr}* indicates the container network of the cluster, for example, 10.0.0.0/16, and *{master_ip}* indicates the IP address of the master node. + + To ensure configuration persistence, you are advised to write the command to the **/etc/rc.local** script. + +#. Run the following command in the container to access kube-apiserver and check whether the request is intercepted: + + .. code-block:: + + curl -k https://{Private API server IP}:5443 diff --git a/umn/source/best_practice/security/index.rst b/umn/source/best_practice/security/index.rst new file mode 100644 index 0000000..ee55f6b --- /dev/null +++ b/umn/source/best_practice/security/index.rst @@ -0,0 +1,20 @@ +:original_name: cce_bestpractice_0315.html + +.. _cce_bestpractice_0315: + +Security +======== + +- :ref:`Cluster Security ` +- :ref:`Node Security ` +- :ref:`Container Security ` +- :ref:`Secret Security ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + cluster_security + node_security + container_security + secret_security diff --git a/umn/source/best_practice/security/node_security.rst b/umn/source/best_practice/security/node_security.rst new file mode 100644 index 0000000..ae947b2 --- /dev/null +++ b/umn/source/best_practice/security/node_security.rst @@ -0,0 +1,89 @@ +:original_name: cce_bestpractice_0318.html + +.. _cce_bestpractice_0318: + +Node Security +============= + +Preventing Nodes from Being Exposed to Public Networks +------------------------------------------------------ + +- Do not bind an EIP to a node unless necessary to reduce the attack surface. +- If an EIP must be used, properly configure the firewall or security group rules to restrict access of unnecessary ports and IP addresses. + +You may have configured the **kubeconfig.json** file on a node in your cluster. kubectl can use the certificate and private key in this file to control the entire cluster. You are advised to delete unnecessary files from the **/root/.kube** directory on the node to prevent malicious use. + +rm -rf /root/.kube + +Hardening VPC Security Group Rules +---------------------------------- + +CCE is a universal container platform. Its default security group rules apply to common scenarios. Based on security requirements, you can harden the security group rules set for CCE clusters on the **Security Groups** page of **Network Console**. + +Hardening Nodes on Demand +------------------------- + +CCE cluster nodes use the default settings of open source OSs. After a node is created, you need to perform security hardening according to your service requirements. + +In CCE, you can perform hardening as follows: + +- Use the post-installation script after the node is created. For details, see the description about **Post-installation Script** in **Advanced Settings** when creating a node. This script is user-defined. + +Forbidding Containers to Obtain Host Machine Metadata +----------------------------------------------------- + +If a single CCE cluster is shared by multiple users to deploy containers, containers cannot access the management address (169.254.169.254) of OpenStack, preventing containers from obtaining metadata of host machines. + +For details about how to restore the metadata, see the "Notes" section in `Obtaining Metadata `__. + +.. warning:: + + This solution may affect the password change on the ECS console. Therefore, you must verify the solution before rectifying the fault. + +#. Obtain the network model and container CIDR of the cluster. + + On the **Clusters** page of the CCE console, view the network model and container CIDR of the cluster. + + |image1| + +#. Prevent the container from obtaining host metadata. + + - VPC network + + a. Log in to each node in the CCE cluster as user **root** and run the following command: + + .. code-block:: + + iptables -I OUTPUT -s {container_cidr} -d 169.254.169.254 -j REJECT + + *{container_cidr}* indicates the container CIDR of the cluster, for example, 10.0.0.0/16. + + To ensure configuration persistence, you are advised to write the command to the **/etc/rc.local** script. + + b. Run the following commands in the container to access the **userdata** and **metadata** interfaces of OpenStack and check whether the request is intercepted: + + .. code-block:: + + curl 169.254.169.254/openstack/latest/meta_data.json + curl 169.254.169.254/openstack/latest/user_data + + - Container tunnel network + + a. Log in to each node in the CCE cluster as user **root** and run the following command: + + .. code-block:: + + iptables -I FORWARD -s {container_cidr} -d 169.254.169.254 -j REJECT + + *{container_cidr}* indicates the container CIDR of the cluster, for example, 10.0.0.0/16. + + To ensure configuration persistence, you are advised to write the command to the **/etc/rc.local** script. + + b. Run the following commands in the container to access the **userdata** and **metadata** interfaces of OpenStack and check whether the request is intercepted: + + .. code-block:: + + curl 169.254.169.254/openstack/latest/meta_data.json + curl 169.254.169.254/openstack/latest/user_data + +.. |image1| image:: /_static/images/en-us_image_0000001226818003.png diff --git a/umn/source/best_practice/security/secret_security.rst b/umn/source/best_practice/security/secret_security.rst new file mode 100644 index 0000000..cd48f03 --- /dev/null +++ b/umn/source/best_practice/security/secret_security.rst @@ -0,0 +1,122 @@ +:original_name: cce_bestpractice_0320.html + +.. _cce_bestpractice_0320: + +Secret Security +=============== + +Currently, CCE has configured static encryption for secret resources. The secrets created by users will be encrypted and stored in etcd of the CCE cluster. Secrets can be used in two modes: environment variable and file mounting. No matter which mode is used, CCE still transfers the configured data to users. Therefore, it is recommended that: + +#. Do not record sensitive information in logs. + +#. For the secret that uses the file mounting mode, the default file permission mapped in the container is 0644. Configure stricter permissions for the file. For example: + + .. code-block:: + + apiversion: v1 + kind: Pod + metadata: + name: mypod + spec: + containers: + - name: mypod + image: redis + volumeMounts: + - name: foo + mountPath: "/etc/foo" + volumes: + - name: foo + secret: + secretName: mysecret + defaultMode: 256 + + In **defaultMode: 256**, **256** is a decimal number, which corresponds to the octal number **0400**. + +#. When the file mounting mode is used, configure the secret file name to hide the file in the container. + + .. code-block:: + + apiVersion: v1 + kind: Secret + metadata: + name: dotfile-secret + data: + .secret-file: dmFsdWUtMg0KDQo= + --- + apiVersion: v1 + kind: Pod + metadata; + name: secret-dotfiles-pod + spec: + volumes: + - name: secret-volume + secret: + secretName: dotfile-secret + containers: + - name: dotfile-test-container + image: k8s.gcr.io/busybox + command: + - ls + - "-1" + - "/etc/secret-volume" + volumeMounts: + - name: secret-volume + readOnly: true + mountPath: "/etc/secret-volume" + + In this way, **.secret-file** cannot be viewed by running the **ls -l** command in the **/etc/secret-volume/** directory, but can be viewed by running the **ls -al** command. + +#. Encrypt sensitive information before creating a secret and decrypt the information when using it. + +Using a Bound ServiceAccount Token to Access a Cluster +------------------------------------------------------ + +The secret-based ServiceAccount token does not support expiration time or auto update. In addition, after the mounting pod is deleted, the token is still stored in the secret. Token leakage may incur security risks. A bound ServiceAccount token is recommended for CCE clusters of version 1.23 or later. In this mode, the expiration time can be set and is the same as the pod lifecycle, reducing token leakage risks. Example: + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: security-token-example + namespace: security-example + spec: + replicas: 1 + selector: + matchLabels: + app: security-token-example + label: security-token-example + template: + metadata: + annotations: + seccomp.security.alpha.kubernetes.io/pod: runtime/default + labels: + app: security-token-example + label: security-token-example + spec: + serviceAccountName: test-sa + containers: + - image: ... + imagePullPolicy: Always + name: security-token-example + volumes: + - name: test-projected + projected: + defaultMode: 420 + sources: + - serviceAccountToken: + expirationSeconds: 1800 + path: token + - configMap: + items: + - key: ca.crt + path: ca.crt + name: kube-root-ca.crt + - downwardAPI: + items: + - fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + path: namespace + +For details, visit https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/. diff --git a/umn/source/best_practice/storage/custom_storage_classes.rst b/umn/source/best_practice/storage/custom_storage_classes.rst new file mode 100644 index 0000000..b3079ee --- /dev/null +++ b/umn/source/best_practice/storage/custom_storage_classes.rst @@ -0,0 +1,326 @@ +:original_name: cce_bestpractice_00281_0.html + +.. _cce_bestpractice_00281_0: + +Custom Storage Classes +====================== + +Challenges +---------- + +When using storage resources in CCE, the most common method is to specify **storageClassName** to define the type of storage resources to be created when creating a PVC. The following configuration shows how to use a PVC to apply for an SAS (high I/O) EVS disk (block storage). + +.. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: pvc-evs-example + namespace: default + annotations: + everest.io/disk-volume-type: SAS + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + storageClassName: csi-disk + +If you need to specify the EVS disk type, you can set the **everest.io/disk-volume-type** field. The value **SAS** is used as an example here, indicating the high I/O EVS disk type. Or you can choose **SATA** (common I/O) and **SSD** (ultra-high I/O). + +This configuration method may not work if you want to: + +- Set **storageClassName** only, which is simpler than specifying the EVS disk type by using **everest.io/disk-volume-type**. +- Avoid modifying YAML files or Helm charts. Some users switch from self-built or other Kubernetes services to CCE and have written YAML files of many applications. In these YAML files, different types of storage resources are specified by different StorageClassNames. When using CCE, they need to modify a large number of YAML files or Helm charts to use storage resources, which is labor-consuming and error-prone. +- Set the default **storageClassName** for all applications to use the default storage class. In this way, you can create storage resources of the default type without needing to specify **storageClassName** in the YAML file. + +Solution +-------- + +This section describes how to set a custom storage class in CCE and how to set the default storage class. You can specify different types of storage resources by setting **storageClassName**. + +- For the first scenario, you can define custom storageClassNames for SAS and SSD EVS disks. For example, define a storage class named **csi-disk-sas** for creating SAS disks. The following figure shows the differences before and after you use a custom storage class. + + |image1| + +- For the second scenario, you can define a storage class with the same name as that in the existing YAML file without needing to modify **storageClassName** in the YAML file. + +- For the third scenario, you can set the default storage class as described below to create storage resources without specifying **storageClassName** in YAML files. + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: pvc-evs-example + namespace: default + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + +Storage Classes in CCE +---------------------- + +Run the following command to query the supported storage classes. + +.. code-block:: + + # kubectl get sc + NAME PROVISIONER AGE + csi-disk everest-csi-provisioner 17d # Storage class for EVS disks + csi-disk-topology everest-csi-provisioner 17d # Storage class for EVS disks with delayed association + csi-nas everest-csi-provisioner 17d # Storage class for SFS file systems + csi-obs everest-csi-provisioner 17d # Storage Class for OBS buckets + csi-sfsturbo everest-csi-provisioner 17d # Storage class for SFS Turbo file systems + +Check the details of **csi-disk**. You can see that the type of the disk created by **csi-disk** is SAS by default. + +.. code-block:: + + # kubectl get sc csi-disk -oyaml + allowVolumeExpansion: true + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + creationTimestamp: "2021-03-17T02:10:32Z" + name: csi-disk + resourceVersion: "760" + selfLink: /apis/storage.k8s.io/v1/storageclasses/csi-disk + uid: 4db97b6c-853b-443d-b0dc-41cdcb8140f2 + parameters: + csi.storage.k8s.io/csi-driver-name: disk.csi.everest.io + csi.storage.k8s.io/fstype: ext4 + everest.io/disk-volume-type: SAS + everest.io/passthrough: "true" + provisioner: everest-csi-provisioner + reclaimPolicy: Delete + volumeBindingMode: Immediate + + +Custom Storage Classes +---------------------- + +You can customize a high I/O storage class in a YAML file. For example, the name **csi-disk-sas** indicates that the disk type is SAS (high I/O). + +.. code-block:: + + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: csi-disk-sas # Name of the high I/O storage class, which can be customized. + parameters: + csi.storage.k8s.io/csi-driver-name: disk.csi.everest.io + csi.storage.k8s.io/fstype: ext4 + everest.io/disk-volume-type: SAS # High I/O EVS disk type, which cannot be customized. + everest.io/passthrough: "true" + provisioner: everest-csi-provisioner + reclaimPolicy: Delete + volumeBindingMode: Immediate + allowVolumeExpansion: true # true indicates that capacity expansion is allowed. + +For an ultra-high I/O storage class, you can set the class name to **csi-disk-ssd** to create SSD EVS disk (ultra-high I/O). + +.. code-block:: + + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: csi-disk-ssd # Name of the ultra-high I/O storage class, which can be customized. + parameters: + csi.storage.k8s.io/csi-driver-name: disk.csi.everest.io + csi.storage.k8s.io/fstype: ext4 + everest.io/disk-volume-type: SSD # Ultra-high I/O EVS disk type, which cannot be customized. + everest.io/passthrough: "true" + provisioner: everest-csi-provisioner + reclaimPolicy: Delete + volumeBindingMode: Immediate + allowVolumeExpansion: true + +**reclaimPolicy**: indicates the recycling policies of the underlying cloud storage. The value can be **Delete** or **Retain**. + +- **Delete**: When a PVC is deleted, both the PV and the EVS disk are deleted. +- **Retain**: When a PVC is deleted, the PV and underlying storage resources are not deleted. Instead, you must manually delete these resources. After that, the PV resource is in the **Released** state and cannot be bound to the PVC again. + +.. note:: + + The reclamation policy set here has no impact on the SFS Turbo storage. Therefore, the yearly/monthly SFS Turbo resources will not be reclaimed when the cluster or PVC is deleted. + +If high data security is required, you are advised to select **Retain** to prevent data from being deleted by mistake. + +After the definition is complete, run the **kubectl create** commands to create storage resources. + +.. code-block:: + + # kubectl create -f sas.yaml + storageclass.storage.k8s.io/csi-disk-sas created + # kubectl create -f ssd.yaml + storageclass.storage.k8s.io/csi-disk-ssd created + +Query the storage class again. Two more types of storage classes are displayed in the command output, as shown below. + +.. code-block:: + + # kubectl get sc + NAME PROVISIONER AGE + csi-disk everest-csi-provisioner 17d + csi-disk-sas everest-csi-provisioner 2m28s + csi-disk-ssd everest-csi-provisioner 16s + csi-disk-topology everest-csi-provisioner 17d + csi-nas everest-csi-provisioner 17d + csi-obs everest-csi-provisioner 17d + csi-sfsturbo everest-csi-provisioner 17d + +Other types of storage resources can be defined in the similar way. You can use kubectl to obtain the YAML file and modify it as required. + +- File storage + + .. code-block:: + + # kubectl get sc csi-nas -oyaml + kind: StorageClass + apiVersion: storage.k8s.io/v1 + metadata: + name: csi-nas + provisioner: everest-csi-provisioner + parameters: + csi.storage.k8s.io/csi-driver-name: nas.csi.everest.io + csi.storage.k8s.io/fstype: nfs + everest.io/share-access-level: rw + everest.io/share-access-to: 5e3864c6-e78d-4d00-b6fd-de09d432c632 # ID of the VPC to which the cluster belongs + everest.io/share-is-public: 'false' + everest.io/zone: xxxxx # AZ + reclaimPolicy: Delete + allowVolumeExpansion: true + volumeBindingMode: Immediate + +- Object storage + + .. code-block:: + + # kubectl get sc csi-obs -oyaml + kind: StorageClass + apiVersion: storage.k8s.io/v1 + metadata: + name: csi-obs + provisioner: everest-csi-provisioner + parameters: + csi.storage.k8s.io/csi-driver-name: obs.csi.everest.io + csi.storage.k8s.io/fstype: s3fs # Object storage type. s3fs indicates an object bucket, and obsfs indicates a parallel file system. + everest.io/obs-volume-type: STANDARD # Storage class of the OBS bucket + reclaimPolicy: Delete + volumeBindingMode: Immediate + +Setting a Default Storage Class +------------------------------- + +You can specify a storage class as the default class. In this way, if you do not specify **storageClassName** when creating a PVC, the PVC is created using the default storage class. + +For example, to specify **csi-disk-ssd** as the default storage class, edit your YAML file as follows: + +.. code-block:: + + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: csi-disk-ssd + annotations: + storageclass.kubernetes.io/is-default-class: "true" # Specifies the default storage class in a cluster. A cluster can have only one default storage class. + parameters: + csi.storage.k8s.io/csi-driver-name: disk.csi.everest.io + csi.storage.k8s.io/fstype: ext4 + everest.io/disk-volume-type: SSD + everest.io/passthrough: "true" + provisioner: everest-csi-provisioner + reclaimPolicy: Delete + volumeBindingMode: Immediate + allowVolumeExpansion: true + +Delete the created csi-disk-ssd disk, run the **kubectl create** command to create a csi-disk-ssd disk again, and then query the storage class. The following information is displayed. + +.. code-block:: + + # kubectl delete sc csi-disk-ssd + storageclass.storage.k8s.io "csi-disk-ssd" deleted + # kubectl create -f ssd.yaml + storageclass.storage.k8s.io/csi-disk-ssd created + # kubectl get sc + NAME PROVISIONER AGE + csi-disk everest-csi-provisioner 17d + csi-disk-sas everest-csi-provisioner 114m + csi-disk-ssd (default) everest-csi-provisioner 9s + csi-disk-topology everest-csi-provisioner 17d + csi-nas everest-csi-provisioner 17d + csi-obs everest-csi-provisioner 17d + csi-sfsturbo everest-csi-provisioner 17d + +Verification +------------ + +- Use **csi-disk-sas** to create a PVC. + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: sas-disk + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + storageClassName: csi-disk-sas + + Create a storage class and view its details. As shown below, the object can be created and the value of **STORAGECLASS** is **csi-disk-sas**. + + .. code-block:: + + # kubectl create -f sas-disk.yaml + persistentvolumeclaim/sas-disk created + # kubectl get pvc + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE + sas-disk Bound pvc-6e2f37f9-7346-4419-82f7-b42e79f7964c 10Gi RWO csi-disk-sas 24s + # kubectl get pv + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-6e2f37f9-7346-4419-82f7-b42e79f7964c 10Gi RWO Delete Bound default/sas-disk csi-disk-sas 30s + + View the PVC details on the CCE console. On the PV details page, you can see that the disk type is high I/O. + +- If **storageClassName** is not specified, the default configuration is used, as shown below. + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: ssd-disk + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + + Create and view the storage resource. You can see that the storage class of PVC ssd-disk is csi-disk-ssd, indicating that csi-disk-ssd is used by default. + + .. code-block:: + + # kubectl create -f ssd-disk.yaml + persistentvolumeclaim/ssd-disk created + # kubectl get pvc + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE + sas-disk Bound pvc-6e2f37f9-7346-4419-82f7-b42e79f7964c 10Gi RWO csi-disk-sas 16m + ssd-disk Bound pvc-4d2b059c-0d6c-44af-9994-f74d01c78731 10Gi RWO csi-disk-ssd 10s + # kubectl get pv + NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE + pvc-4d2b059c-0d6c-44af-9994-f74d01c78731 10Gi RWO Delete Bound default/ssd-disk csi-disk-ssd 15s + pvc-6e2f37f9-7346-4419-82f7-b42e79f7964c 10Gi RWO Delete Bound default/sas-disk csi-disk-sas 17m + + View the PVC details on the CCE console. On the PV details page, you can see that the disk type is ultra-high I/O. + +.. |image1| image:: /_static/images/en-us_image_0000001102275444.png diff --git a/umn/source/best_practice/storage/dynamically_creating_and_mounting_subdirectories_of_an_sfs_turbo_file_system.rst b/umn/source/best_practice/storage/dynamically_creating_and_mounting_subdirectories_of_an_sfs_turbo_file_system.rst new file mode 100644 index 0000000..46a70e6 --- /dev/null +++ b/umn/source/best_practice/storage/dynamically_creating_and_mounting_subdirectories_of_an_sfs_turbo_file_system.rst @@ -0,0 +1,251 @@ +:original_name: cce_bestpractice_00253_0.html + +.. _cce_bestpractice_00253_0: + +Dynamically Creating and Mounting Subdirectories of an SFS Turbo File System +============================================================================ + +Background +---------- + +The minimum capacity of an SFS Turbo file system is 500 GB, and the SFS Turbo file system cannot be billed by usage. By default, the root directory of an SFS Turbo file system is mounted to a container which, in most case, does not require such a large capacity. + +The everest add-on allows you to dynamically create subdirectories in an SFS Turbo file system and mount these subdirectories to containers. In this way, an SFS Turbo file system can be shared by multiple containers to increase storage efficiency. + +Notes and Constraints +--------------------- + +- Only clusters of v1.15 and later are supported. +- The cluster must use the everest add-on of version 1.1.13 or later. +- Kata containers are not supported. +- A maximum of 10 PVCs can be created concurrently at a time by using the subdirectory function. + +Creating an SFS Turbo Volume of the subpath Type +------------------------------------------------ + +.. caution:: + + The CCE console has not yet supported the operations related to this feature, such as expanding, disassociating, and deleting subPath volumes. + +#. Import an SFS Turbo file system that is located in the same VPC and subnet as the cluster. + +#. Create a StorageClass YAML file, for example, **sfsturbo-sc-test.yaml**. + + Configuration example: + + .. code-block:: + + apiVersion: storage.k8s.io/v1 + allowVolumeExpansion: true + kind: StorageClass + metadata: + name: sfsturbo-sc-test + mountOptions: + - lock + parameters: + csi.storage.k8s.io/csi-driver-name: sfsturbo.csi.everest.io + csi.storage.k8s.io/fstype: nfs + everest.io/archive-on-delete: "true" + everest.io/share-access-to: 7ca2dba2-1234-1234-1234-626371a8fb3a + everest.io/share-expand-type: bandwidth + everest.io/share-export-location: 192.168.1.1:/sfsturbo/ + everest.io/share-source: sfs-turbo + everest.io/share-volume-type: STANDARD + everest.io/volume-as: subpath + everest.io/volume-id: 0d773f2e-1234-1234-1234-de6a35074696 + provisioner: everest-csi-provisioner + reclaimPolicy: Delete + volumeBindingMode: Immediate + + In this example: + + - **name**: name of the StorageClass. + - **mountOptions**: mount options. This field is optional. + + - In versions later than everest 1.1.13 and earlier than everest 1.2.8, only the **nolock** parameter can be configured. By default, the **nolock** parameter is used for the mount operation and does not need to be configured. If **nolock** is set to **false**, the **lock** field is used. + + - Starting from everest 1.2.8, more parameters are supported. The default parameter configurations are shown below. For details, see `Setting Mount Options `__. **Do not set nolock to true. Otherwise, the mount operation fails.** + + .. code-block:: + + mountOptions: + - vers=3 + - timeo=600 + - nolock + - hard + + - **everest.io/volume-as**: Set this parameter to **subpath**. + - **everest.io/share-access-to**: This parameter is optional. In subpath mode, set this parameter to the ID of the VPC where the SFS Turbo file system is located. + - **everest.io/share-expand-type**: This parameter is optional. If the type of the SFS Turbo file system is SFS Turbo Standard - Enhanced or SFS Turbo Performance - Enhanced, set this parameter to **bandwidth**. + - **everest.io/share-export-location**: root directory to be mounted. It consists of the SFS Turbo shared path and sub-directory. The shared path can be queried on the SFS Turbo console. The sub-directory is user-defined. The PVCs created by the StorageClass are located in the sub-directory. + - **everest.io/share-volume-type**: This parameter is optional. It specifies the SFS Turbo file system type. The value can be **STANDARD** or **PERFORMANCE**. For enhanced types, this parameter must be used together with **everest.io/share-expand-type** (whose value should be **bandwidth**). + - **everest.io/zone**: This parameter is optional. Set it to the AZ where the SFS Turbo file system is located. + - **everest.io/volume-id**: ID of the SFS Turbo volume. You can query the volume ID on the SFS Turbo page. + - **everest.io/archive-on-delete**: If this parameter is set to **true** and the recycling policy is set to **Delete**, the original PV file will be archived when the PVC is deleted. The archive directory is named in the format of *archived-$PV name.timestamp*. If this parameter is set to **false**, the SFS Turbo sub-directory corresponding to the PV will be deleted. The default value is **true**. + +3. Run the **kubectl create -f sfsturbo-sc-test.yaml** command to create a StorageClass. + +4. Create a PVC YAML file named **sfs-turbo-test.yaml**. + + Configuration example: + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: sfs-turbo-test + namespace: default + spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 50Gi + storageClassName: sfsturbo-sc-test + volumeMode: Filesystem + + In this example: + + - **name**: name of the PVC. + - **storageClassName**: name of the StorageClass created in the previous step. + - **storage**: In the subpath mode, this parameter is invalid. The storage capacity is limited by the total capacity of the SFS Turbo file system. If the total capacity of the SFS Turbo file system is insufficient, expand the capacity on the SFS Turbo page in a timely manner. + +5. Run the **kubectl create -f sfs-turbo-test.yaml** command to create a PVC. + +.. note:: + + It is meaningless to conduct capacity expansion on an SFS Turbo volume created in the subpath mode. This operation does not expand the capacity of the SFS Turbo file system. You need to ensure that the total capacity of the SFS Turbo file system is not used up. + +Creating a Deployment and Mounting an Existing Volume +----------------------------------------------------- + +#. Create a Deployment YAML file named **deployment-test.yaml**. + + Configuration example: + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: test-turbo-subpath-example + namespace: default + generation: 1 + labels: + appgroup: '' + spec: + replicas: 1 + selector: + matchLabels: + app: test-turbo-subpath-example + template: + metadata: + labels: + app: test-turbo-subpath-example + spec: + containers: + - image: nginx:latest + name: container-0 + volumeMounts: + - mountPath: /tmp + name: pvc-sfs-turbo-example + restartPolicy: Always + imagePullSecrets: + - name: default-secret + volumes: + - name: pvc-sfs-turbo-example + persistentVolumeClaim: + claimName: sfs-turbo-test + + In this example: + + - **name**: name of the Deployment. + - **image**: image used by the Deployment. + - **mountPath**: mount path of the container. In this example, the volume is mounted to the **/tmp** directory. + - **claimName**: name of an existing PVC. + +2. Run the **kubectl create -f deployment-test.yaml** command to create a Deployment. + +Creating a StatefulSet That Uses a Volume Dynamically Created in subpath Mode +----------------------------------------------------------------------------- + +#. Create a StatefulSet YAML file named **statefulset-test.yaml**. + + Configuration example: + + .. code-block:: + + apiVersion: apps/v1 + kind: StatefulSet + metadata: + name: test-turbo-subpath + namespace: default + generation: 1 + labels: + appgroup: '' + spec: + replicas: 2 + selector: + matchLabels: + app: test-turbo-subpath + template: + metadata: + labels: + app: test-turbo-subpath + annotations: + metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"","path":"","port":"","names":""}]' + pod.alpha.kubernetes.io/initialized: 'true' + spec: + containers: + - name: container-0 + image: 'nginx:latest' + env: + - name: PAAS_APP_NAME + value: deploy-sfs-nfs-rw-in + - name: PAAS_NAMESPACE + value: default + - name: PAAS_PROJECT_ID + value: 8190a2a1692c46f284585c56fc0e2fb9 + resources: {} + volumeMounts: + - name: sfs-turbo-160024548582479676 + mountPath: /tmp + terminationMessagePath: /dev/termination-log + terminationMessagePolicy: File + imagePullPolicy: IfNotPresent + restartPolicy: Always + terminationGracePeriodSeconds: 30 + dnsPolicy: ClusterFirst + securityContext: {} + imagePullSecrets: + - name: default-secret + affinity: {} + schedulerName: default-scheduler + volumeClaimTemplates: + - metadata: + name: sfs-turbo-160024548582479676 + namespace: default + annotations: {} + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + storageClassName: sfsturbo-sc-test + serviceName: wwww + podManagementPolicy: OrderedReady + updateStrategy: + type: RollingUpdate + revisionHistoryLimit: 10 + + In this example: + + - **name**: name of the StatefulSet. + - **image**: image used by the StatefulSet. + - **mountPath**: mount path of the container. In this example, the volume is mounted to the **/tmp** directory. + - **spec.template.spec.containers.volumeMounts.name** and **spec.volumeClaimTemplates.metadata.name** must be consistent because they have a mapping relationship. + - **storageClassName**: name of the created StorageClass. + +2. Run the **kubectl create -f statefulset-test.yaml** command to create a StatefulSet. diff --git a/umn/source/best_practice/storage/expanding_node_disk_capacity.rst b/umn/source/best_practice/storage/expanding_node_disk_capacity.rst new file mode 100644 index 0000000..c18b43d --- /dev/null +++ b/umn/source/best_practice/storage/expanding_node_disk_capacity.rst @@ -0,0 +1,88 @@ +:original_name: cce_bestpractice_00198.html + +.. _cce_bestpractice_00198: + +Expanding Node Disk Capacity +============================ + +System Disk +----------- + +#. Expand the capacity of the system disk on the EVS console. +#. Restart the node on the ECS console. +#. Log in to the CCE console and click the cluster. In the navigation pane, choose **Nodes**. Click **More** > **Sync Server Data** at the row containing the target node. + +Node Data Disk (Dedicated for Docker) +------------------------------------- + +#. Expand the capacity of the data disk on the EVS console. + +#. Log in to the CCE console and click the cluster. In the navigation pane, choose **Nodes**. Click **More** > **Sync Server Data** at the row containing the target node. + +#. Log in to the target node. + +#. Run the **lsblk** command to check the block device information of the node. + + A data disk is divided depending on the container storage **Rootfs**: + + - Overlayfs: No independent thin pool is allocated. Image data is stored in the **dockersys** disk. + + .. code-block:: + + # lsblk + NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT + sda 8:0 0 50G 0 disk + └─sda1 8:1 0 50G 0 part / + sdb 8:16 0 200G 0 disk + ├─vgpaas-dockersys 253:0 0 90G 0 lvm /var/lib/docker # Space used by Docker. + └─vgpaas-kubernetes 253:1 0 10G 0 lvm /mnt/paas/kubernetes/kubelet # Space used by Kubernetes. + + Run the following commands on the node to add the new disk capacity to the **dockersys** disk: + + .. code-block:: + + pvresize /dev/sdb + lvextend -l+100%FREE -n vgpaas/dockersys + resize2fs /dev/vgpaas/dockersys + + - Devicemapper: A thin pool is allocated to store image data. + + .. code-block:: + + # lsblk + NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT + sda 8:0 0 50G 0 disk + └─sda1 8:1 0 50G 0 part / + sdb 8:16 0 200G 0 disk + ├─vgpaas-dockersys 253:0 0 18G 0 lvm /var/lib/docker + ├─vgpaas-thinpool_tmeta 253:1 0 3G 0 lvm + │ └─vgpaas-thinpool 253:3 0 67G 0 lvm # Thin pool space. + │ ... + ├─vgpaas-thinpool_tdata 253:2 0 67G 0 lvm + │ └─vgpaas-thinpool 253:3 0 67G 0 lvm + │ ... + └─vgpaas-kubernetes 253:4 0 10G 0 lvm /mnt/paas/kubernetes/kubelet + + Run the following commands on the node to add the new disk capacity to the **thinpool** disk: + + .. code-block:: + + pvresize /dev/sdb + lvextend -l+100%FREE -n vgpaas/thinpool + +Node Data Disk (Kubernetes) +--------------------------- + +#. Expand the capacity of the data disk on the EVS console. + +#. Log in to the CCE console and click the cluster. In the navigation pane, choose **Nodes**. Click **More** > **Sync Server Data** at the row containing the target node. + +#. Log in to the target node. + +#. Run the following commands on the node to add the new disk capacity to the Kubernetes disk: + + .. code-block:: + + pvresize /dev/sdb + lvextend -l+100%FREE -n vgpaas/kubernetes + resize2fs /dev/vgpaas/kubernetes diff --git a/umn/source/reference/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst b/umn/source/best_practice/storage/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst similarity index 91% rename from umn/source/reference/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst rename to umn/source/best_practice/storage/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst index 06f32a0..ea004c8 100644 --- a/umn/source/reference/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst +++ b/umn/source/best_practice/storage/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst @@ -5,7 +5,7 @@ How Do I Change the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest? ============================================================================================ -For clusters of v1.15.11-r1 and later, the CSI everest add-on has taken over all functions of the fuxi FlexVolume driver (the storage-driver add-on) for container storage management. In versions later than 1.17.9-r0, the fuxi FlexVolume driver (storage-driver) is no longer supported. +In clusters later than v1.15.11-r1, CSI (the everest add-on) has taken over all functions of fuxi FlexVolume (the storage-driver add-on) for managing container storage. You are advised to use CSI Everest. To migrate your storage volumes, create a static PV to associate with the original underlying storage, and then create a PVC to associate with this static PV. When you upgrade your application, mount the new PVC to the original mounting path to migrate the storage volumes. @@ -18,8 +18,6 @@ Procedure #. (Optional) Back up data to prevent data loss in case of exceptions. -#. Run kubectl commands. - #. .. _cce_bestpractice_0107__li1219802032512: Configure a YAML file of the PV in the CSI format according to the PV in the FlexVolume format and associate the PV with the existing storage. @@ -39,7 +37,7 @@ Procedure metadata: labels: failure-domain.beta.kubernetes.io/region: eu-de - failure-domain.beta.kubernetes.io/zone: eu-de-01 + failure-domain.beta.kubernetes.io/zone: annotations: pv.kubernetes.io/provisioned-by: everest-csi-provisioner name: pv-evs-example @@ -53,7 +51,7 @@ Procedure fsType: ext4 volumeAttributes: everest.io/disk-mode: SCSI - everest.io/disk-volume-type: SATA + everest.io/disk-volume-type: SAS storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner volumeHandle: 0992dbda-6340-470e-a74e-4f0db288ed82 persistentVolumeReclaimPolicy: Delete @@ -63,27 +61,27 @@ Procedure .. table:: **Table 1** EVS volume configuration parameters - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +==========================================+==========================================================================================================================================================================================================================================+ - | failure-domain.beta.kubernetes.io/region | Region where the EVS disk is located. Use the same value as that of the FlexVolume PV. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/zone | AZ where the EVS disk is located. Use the same value as that of the FlexVolume PV. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | name | Name of the PV, which must be unique in the cluster. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | storage | EVS volume capacity in the unit of Gi. Use the value of **spec.capacity.storage** of the FlexVolume PV. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | driver | Storage driver used to attach the volume. Set the driver to **disk.csi.everest.io** for the EVS volume. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeHandle | Volume ID of the EVS disk. Use the value of **spec.flexVolume.options.volumeID** of the FlexVolume PV. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | everest.io/disk-mode | EVS disk mode. Use the value of **spec.flexVolume.options.disk-mode** of the FlexVolume PV. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | everest.io/disk-volume-type | EVS disk type. Currently, high I/O (SAS), ultra-high I/O (SSD), and common I/O (SATA) are supported. Use the value of **kubernetes.io/volumetype** in the storage class corresponding to **spec.storageClassName** of the FlexVolume PV. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | storageClassName | Name of the Kubernetes storage class associated with the storage volume. Set this field to **csi-disk** for EVS disks. | - +------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +==========================================+====================================================================================================================================================+ + | failure-domain.beta.kubernetes.io/region | Region where the EVS disk is located. Use the same value as that of the FlexVolume PV. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/zone | AZ where the EVS disk is located. Use the same value as that of the FlexVolume PV. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | name | Name of the PV, which must be unique in the cluster. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | storage | EVS volume capacity in the unit of Gi. Use the value of **spec.capacity.storage** of the FlexVolume PV. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | driver | Storage driver used to attach the volume. Set the driver to **disk.csi.everest.io** for the EVS volume. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | volumeHandle | Volume ID of the EVS disk. Use the value of **spec.flexVolume.options.volumeID** of the FlexVolume PV. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | everest.io/disk-mode | EVS disk mode. Use the value of **spec.flexVolume.options.disk-mode** of the FlexVolume PV. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | everest.io/disk-volume-type | EVS disk type. Use the value of **kubernetes.io/volumetype** in the storage class corresponding to **spec.storageClassName** of the FlexVolume PV. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ + | storageClassName | Name of the Kubernetes storage class associated with the storage volume. Set this field to **csi-disk** for EVS disks. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------+ Configuration example of **a PV for an SFS volume**: @@ -104,7 +102,7 @@ Procedure driver: nas.csi.everest.io fsType: nfs volumeAttributes: - everest.io/share-export-location: sfs-nas01.Endpoint:/share-436304e8 + everest.io/share-export-location: # Shared path of the file storage storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner volumeHandle: 682f00bb-ace0-41d8-9b3e-913c9aa6b695 persistentVolumeReclaimPolicy: Delete @@ -165,7 +163,7 @@ Procedure +============================+===========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ | name | Name of the PV, which must be unique in the cluster. | +----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | storage | Storage capacity in the unit of Gi. Set this parameter to the fixed value **1Gi**. | + | storage | Storage capacity, in the unit of Gi. Set this parameter to the fixed value **1Gi**. | +----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | driver | Storage driver used to attach the volume. Set the driver to **obs.csi.everest.io** for the OBS volume. | +----------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -227,7 +225,7 @@ Procedure #. .. _cce_bestpractice_0107__li1710710385418: - Configure a YAML file of the PVC in the CSI format according to the PVC in the FlexVolume format and associate the PVC with the PV created in :ref:`3 `. + Configure a YAML file of the PVC in the CSI format according to the PVC in the FlexVolume format and associate the PVC with the PV created in :ref:`2 `. To be specific, run the following commands to configure the pvc-example.yaml file, which is used to create a PVC. @@ -244,9 +242,9 @@ Procedure metadata: labels: failure-domain.beta.kubernetes.io/region: eu-de - failure-domain.beta.kubernetes.io/zone: eu-de-01 + failure-domain.beta.kubernetes.io/zone: annotations: - everest.io/disk-volume-type: SATA + everest.io/disk-volume-type: SAS volume.beta.kubernetes.io/storage-provisioner: everest-csi-provisioner name: pvc-evs-example namespace: default @@ -270,7 +268,7 @@ Procedure +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | failure-domain.beta.kubernetes.io/zone | AZ where the EVS disk is deployed. Use the same value as that of the FlexVolume PVC. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | everest.io/disk-volume-type | Storage class of the EVS disk. The value can be **SAS**, **SSD**, or **SATA**. Set this parameter to the same value as that of the PV created in :ref:`3 `. | + | everest.io/disk-volume-type | Storage class of the EVS disk. The value can be **SAS** or **SSD**. Set this parameter to the same value as that of the PV created in :ref:`2 `. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | name | PVC name, which must be unique in the namespace. The value must be unique in the namespace. (If the PVC is dynamically created by a stateful application, the value of this parameter must be the same as the name of the FlexVolume PVC.) | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -278,7 +276,7 @@ Procedure +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage | Requested capacity of the PVC, which must be the same as the storage size of the existing PV. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`3 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`2 `. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storageClassName | Name of the Kubernetes storage class. Set this field to **csi-disk** for EVS disks. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -318,7 +316,7 @@ Procedure +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storageClassName | Set this field to **csi-nas**. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`3 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`2 `. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Configuration example of **a PVC for an OBS volume**: @@ -350,7 +348,7 @@ Procedure +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +============================+============================================================================================================================================================================================================================================+ - | everest.io/obs-volume-type | OBS volume type, which can be **STANDARD** (standard bucket) and **WARM** (infrequent access bucket). Set this parameter to the same value as that of the PV created in :ref:`3 `. | + | everest.io/obs-volume-type | OBS volume type, which can be **STANDARD** (standard bucket) and **WARM** (infrequent access bucket). Set this parameter to the same value as that of the PV created in :ref:`2 `. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | csi.storage.k8s.io/fstype | File type, which can be **obsfs** or **s3fs**. The value must be the same as that of **fsType** of the static OBS volume PV. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -358,11 +356,11 @@ Procedure +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | namespace | Namespace to which the PVC belongs. Use the same value as that of the FlexVolume PVC. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | storage | Storage capacity in the unit of Gi. Set this parameter to the fixed value **1Gi**. | + | storage | Storage capacity, in the unit of Gi. Set this parameter to the fixed value **1Gi**. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storageClassName | Name of the Kubernetes storage class. Set this field to **csi-obs**. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`3 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`2 `. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Configuration example of **a PVC for an SFS Turbo volume**: @@ -400,7 +398,7 @@ Procedure +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage | Storage capacity, in the unit of Gi. The value must be the same as the storage size of the existing PV. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`3 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`2 `. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. .. _cce_bestpractice_0107__li487255772614: @@ -417,7 +415,7 @@ Procedure .. note:: - Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`3 ` and :ref:`4 `. + Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. b. Go to the CCE console. On the workload upgrade page, click **Upgrade** > **Advanced Settings** > **Data Storage** > **Cloud Storage**. @@ -437,7 +435,7 @@ Procedure .. note:: - Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`3 ` and :ref:`4 `. + Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. b. Run the **kubectl edit** command to edit the StatefulSet and use the newly created PVC. @@ -475,7 +473,7 @@ Procedure .. note:: - Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`3 ` and :ref:`4 `. + Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. e. Change the number of pods back to the original value and wait until the pods are running. @@ -511,7 +509,7 @@ Procedure namespace: default creationTimestamp: null annotations: - everest.io/disk-volume-type: SATA + everest.io/disk-volume-type: SAS spec: accessModes: - ReadWriteOnce @@ -520,7 +518,7 @@ Procedure storage: 10Gi storageClassName: csi-disk - The parameter value must be the same as the PVC of the EVS volume created in :ref:`4 `. + The parameter value must be the same as the PVC of the EVS volume created in :ref:`3 `. Configuration example of **volumeClaimTemplates for an SFS volume**: @@ -537,9 +535,9 @@ Procedure resources: requests: storage: 10Gi - storageClassName: csi-na + storageClassName: csi-nas - The parameter value must be the same as the PVC of the SFS volume created in :ref:`4 `. + The parameter value must be the same as the PVC of the SFS volume created in :ref:`3 `. Configuration example of **volumeClaimTemplates for an OBS volume**: @@ -561,7 +559,7 @@ Procedure storage: 1Gi storageClassName: csi-obs - The parameter value must be the same as the PVC of the OBS volume created in :ref:`4 `. + The parameter value must be the same as the PVC of the OBS volume created in :ref:`3 `. - Delete the StatefulSet. @@ -578,7 +576,7 @@ Procedure .. note:: - If a rollback is required, perform :ref:`5 `. Select the PVC in FlexVolume format and upgrade the application. + If a rollback is required, perform :ref:`4 `. Select the PVC in FlexVolume format and upgrade the application. #. Uninstall the PVC in the FlexVolume format. @@ -588,6 +586,10 @@ Procedure .. caution:: - Before deleting a PV, change the persistentVolumeReclaimPolicy policy of the PV to **Retain**. Otherwise, the underlying storage will be reclaimed after the PV is deleted. + Before deleting a PV, change the persistentVolumeReclaimPolicy of the PV to **Retain**. Otherwise, the underlying storage will be reclaimed after the PV is deleted. -.. |image1| image:: /_static/images/en-us_image_0000001178352604.png + If the cluster has been upgraded before the storage migration, PVs may fail to be deleted. You can remove the PV protection field **finalizers** to delete PVs. + + kubectl patch pv {pv_name} -p '{"metadata":{"finalizers":null}}' + +.. |image1| image:: /_static/images/en-us_image_0000001097062729.png diff --git a/umn/source/best_practice/storage/index.rst b/umn/source/best_practice/storage/index.rst new file mode 100644 index 0000000..1c955f5 --- /dev/null +++ b/umn/source/best_practice/storage/index.rst @@ -0,0 +1,24 @@ +:original_name: cce_bestpractice_0053.html + +.. _cce_bestpractice_0053: + +Storage +======= + +- :ref:`Expanding Node Disk Capacity ` +- :ref:`Mounting an Object Storage Bucket of a Third-Party Tenant ` +- :ref:`Dynamically Creating and Mounting Subdirectories of an SFS Turbo File System ` +- :ref:`How Do I Change the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest? ` +- :ref:`Custom Storage Classes ` +- :ref:`Realizing Automatic Topology for EVS Disks When Nodes Are Deployed Across AZs (csi-disk-topology) ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + expanding_node_disk_capacity + mounting_an_object_storage_bucket_of_a_third-party_tenant + dynamically_creating_and_mounting_subdirectories_of_an_sfs_turbo_file_system + how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest + custom_storage_classes + realizing_automatic_topology_for_evs_disks_when_nodes_are_deployed_across_azs_csi-disk-topology diff --git a/umn/source/best_practice/storage/mounting_an_object_storage_bucket_of_a_third-party_tenant.rst b/umn/source/best_practice/storage/mounting_an_object_storage_bucket_of_a_third-party_tenant.rst new file mode 100644 index 0000000..9c6ce2a --- /dev/null +++ b/umn/source/best_practice/storage/mounting_an_object_storage_bucket_of_a_third-party_tenant.rst @@ -0,0 +1,195 @@ +:original_name: cce_bestpractice_00199.html + +.. _cce_bestpractice_00199: + +Mounting an Object Storage Bucket of a Third-Party Tenant +========================================================= + +This section describes how to mount OBS buckets and OBS parallel file systems (preferred) of third-party tenants. + +Scenario +-------- + +The CCE cluster of a SaaS service provider needs to be mounted with the OBS bucket of a third-party tenant, as shown in :ref:`Figure 1 `. + +.. _cce_bestpractice_00199__fig1315433183918: + +.. figure:: /_static/images/en-us_image_0268523694.png + :alt: **Figure 1** Mounting an OBS bucket of a third-party tenant + + **Figure 1** Mounting an OBS bucket of a third-party tenant + +#. :ref:`The third-party tenant authorizes the SaaS service provider to access the OBS buckets or parallel file systems ` by setting the bucket policy and bucket ACL. +#. :ref:`The SaaS service provider statically imports the OBS buckets and parallel file systems of the third-party tenant `. +#. The SaaS service provider processes the service and writes the processing result (result file or result data) back to the OBS bucket of the third-party tenant. + +Precautions +----------- + +- Only parallel file systems and OBS buckets of third-party tenants in the same region can be mounted. +- Only clusters where the everest add-on of v1.1.11 or later has been installed (the cluster version must be v1.15 or later) can be mounted with OBS buckets of third-party tenants. +- The service platform of the SaaS service provider needs to manage the lifecycle of the third-party bucket PVs. When a PVC is deleted separately, the PV is not deleted. Instead, it will be retained. To do so, you need to call the native Kubernetes APIs to create and delete static PVs. + +.. _cce_bestpractice_00199__section193471249193310: + +Authorizing the SaaS Service Provider to Access the OBS Buckets +--------------------------------------------------------------- + +The following uses an OBS bucket as an example to describe how to set a bucket policy and bucket ACL to authorize the SaaS service provider. The configuration for an OBS parallel file system is the same. + +#. Log in to the OBS console. In the navigation pane, choose **Buckets**. +#. In the bucket list, click a bucket name to access the **Overview** page. + +3. In the navigation pane, choose **Permissions** > **Bucket Policy**. On the displayed page, click **Create** to create a bucket policy. + + Set the parameters as shown in the following figure. + + + .. figure:: /_static/images/en-us_image_0000001325377749.png + :alt: **Figure 2** Creating a bucket policy + + **Figure 2** Creating a bucket policy + + - **Policy Mode**: Select **Customized**. + - **Effect**: Select **Allow**. + - **Principal**: Select **Other account**, and enter the account ID and user ID. The bucket policy takes effect for the specified users. + - **Resources**: Select the resources that can be operated. + - **Actions**: Select the actions that can be operated. + +4. In the navigation pane, choose **Permissions** > **Bucket ACLs**. In the right pane, click **Add**.Enter the account ID or account name of the authorized user, select **Read** and **Write** for **Access to Bucket**, select **Read** and **Write** for **Access to ACL**, and click **OK**. + +.. _cce_bestpractice_00199__en-us_topic_0196817407_section155006183017: + +Statically Importing OBS Buckets and Parallel File Systems +---------------------------------------------------------- + +- **Static PV of an OBS bucket:** + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolume + metadata: + name: objbucket #Replace the name with the actual PV name of the bucket. + annotations: + pv.kubernetes.io/provisioned-by: everest-csi-provisioner + spec: + accessModes: + - ReadWriteMany + capacity: + storage: 1Gi + mountOptions: + - default_acl=bucket-owner-full-control #New OBS mounting parameters + csi: + driver: obs.csi.everest.io + fsType: obsfs + volumeAttributes: + everest.io/obs-volume-type: STANDARD + everest.io/region: eu-de #Set it to the ID of the current region. + storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner + volumeHandle: objbucket #Replace the name with the actual bucket name of the third-party tenant. + persistentVolumeReclaimPolicy: Retain #This parameter must be set to Retain to ensure that the bucket will not be deleted when a PV is deleted. + storageClassName: csi-obs-mountoption #You can associate a new custom OBS storage class or the built-in csi-obs of the cluster. + + - **mountOptions**: This field contains the new OBS mounting parameters that allow the bucket owner to have full access to the data in the bucket. This field solves the problem that the bucket owner cannot read the data written into a mounted third-party bucket. If the object storage of a third-party tenant is mounted, **default_acl** must be set to **bucket-owner-full-control**. For details about other values of **default_acl**, see `Bucket ACLs and Object ACLs `__. + - **persistentVolumeReclaimPolicy**: When the object storage of a third-party tenant is mounted, this field must be set to **Retain**. In this way, the OBS bucket will not be deleted when a PV is deleted. The service platform of the SaaS service provider needs to manage the lifecycle of the third-party bucket PVs. When a PVC is deleted separately, the PV is not deleted. Instead, it will be retained. To do so, you need to call the native Kubernetes APIs to create and delete static PVs. + - **storageClassName**: You can associate a new custom OBS storage class (:ref:`click here `) or the built-in csi-obs of the cluster. + + **PVC of a bound OBS bucket:** + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + annotations: + csi.storage.k8s.io/fstype: obsfs + everest.io/obs-volume-type: STANDARD + volume.beta.kubernetes.io/storage-provisioner: everest-csi-provisioner + name: objbucketpvc #Replace the name with the actual PVC name of the bucket. + namespace: default + spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 1Gi + storageClassName: csi-obs-mountoption #The value must be the same as the storage class associated with the bound PV. + volumeName: objbucket #Replace the name with the actual PV name of the bucket to be bound. + +- **Static PV of an OBS parallel file system:** + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolume + metadata: + name: obsfscheck #Replace the name with the actual PV name of the parallel file system. + annotations: + pv.kubernetes.io/provisioned-by: everest-csi-provisioner + spec: + accessModes: + - ReadWriteMany + capacity: + storage: 1Gi + mountOptions: + - default_acl=bucket-owner-full-control #New OBS mounting parameters + csi: + driver: obs.csi.everest.io + fsType: obsfs + volumeAttributes: + everest.io/obs-volume-type: STANDARD + everest.io/region: eu-de + storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner + volumeHandle: obsfscheck #Replace the name with the actual name of the parallel file system of the third-party tenant. + persistentVolumeReclaimPolicy: Retain #This parameter must be set to Retain to ensure that the bucket will not be deleted when a PV is deleted. + storageClassName: csi-obs-mountoption #You can associate a new custom OBS storage class or the built-in csi-obs of the cluster. + + - **mountOptions**: This field contains the new OBS mounting parameters that allow the bucket owner to have full access to the data in the bucket. This field solves the problem that the bucket owner cannot read the data written into a mounted third-party bucket. If the object storage of a third-party tenant is mounted, **default_acl** must be set to **bucket-owner-full-control**. For details about other values of **default_acl**, see `Bucket ACLs and Object ACLs `__. + - **persistentVolumeReclaimPolicy**: When the object storage of a third-party tenant is mounted, this field must be set to **Retain**. In this way, the OBS bucket will not be deleted when a PV is deleted. The service platform of the SaaS service provider needs to manage the lifecycle of the third-party bucket PVs. When a PVC is deleted separately, the PV is not deleted. Instead, it will be retained. To do so, you need to call the native Kubernetes APIs to create and delete static PVs. + - **storageClassName**: You can associate a new custom OBS storage class (:ref:`click here `) or the built-in csi-obs of the cluster. + + PVC of a bound OBS parallel file system: + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + annotations: + csi.storage.k8s.io/fstype: obsfs + everest.io/obs-volume-type: STANDARD + volume.beta.kubernetes.io/storage-provisioner: everest-csi-provisioner + name: obsfscheckpvc #Replace the name with the actual PVC name of the parallel file system. + namespace: default + spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 1Gi + storageClassName: csi-obs-mountoption #The value must be the same as the storage class associated with the bound PV. + volumeName: obsfscheck #Replace the name with the actual PV name of the parallel file system. + +- .. _cce_bestpractice_00199__li1235812419467: + + **(Optional) Creating a custom OBS storage class to associate with a static PV:** + + .. code-block:: + + apiVersion: storage.k8s.io/v1 + kind: StorageClass + metadata: + name: csi-obs-mountoption + mountOptions: + - default_acl=bucket-owner-full-control + parameters: + csi.storage.k8s.io/csi-driver-name: obs.csi.everest.io + csi.storage.k8s.io/fstype: obsfs + everest.io/obs-volume-type: STANDARD + provisioner: everest-csi-provisioner + reclaimPolicy: Retain + volumeBindingMode: Immediate + + - **csi.storage.k8s.io/fstype**: File type. The value can be **obsfs** or **s3fs**. If the value is **s3fs**, an OBS bucket is created and mounted using s3fs. If the value is **obsfs**, an OBS parallel file system is created and mounted using obsfs. + - **reclaimPolicy**: Reclaim policy of a PV. The value will be set in **PV.spec.persistentVolumeReclaimPolicy** dynamically created based on the new PVC associated with the storage class. If the value is **Delete**, the external OBS bucket and the PV will be deleted when the PVC is deleted. If the value is **Retain**, the PV and external storage are retained when the PVC is deleted. In this case, you need to clear the PV separately. In the scenario where an imported third-party bucket is associated, the storage class is used only for associating static PVs (with this field set to **Retain**). Dynamic creation is not involved. diff --git a/umn/source/best_practice/storage/realizing_automatic_topology_for_evs_disks_when_nodes_are_deployed_across_azs_csi-disk-topology.rst b/umn/source/best_practice/storage/realizing_automatic_topology_for_evs_disks_when_nodes_are_deployed_across_azs_csi-disk-topology.rst new file mode 100644 index 0000000..43bf91f --- /dev/null +++ b/umn/source/best_practice/storage/realizing_automatic_topology_for_evs_disks_when_nodes_are_deployed_across_azs_csi-disk-topology.rst @@ -0,0 +1,324 @@ +:original_name: cce_bestpractice_00284.html + +.. _cce_bestpractice_00284: + +Realizing Automatic Topology for EVS Disks When Nodes Are Deployed Across AZs (csi-disk-topology) +================================================================================================= + +Challenges +---------- + +EVS disks cannot be attached across AZs. For example, EVS disks in AZ 1 cannot be attached to nodes in AZ 2. + +If the storage class csi-disk is used for StatefulSets, when a StatefulSet is scheduled, a PVC and a PV are created immediately (an EVS disk is created along with the PV), and then the PVC is bound to the PV. + +However, when the cluster nodes are located in multiple AZs, the EVS disk created by the PVC and the node to which the pods are scheduled may be in different AZs. As a result, the pods fail to be scheduled. + +|image1| + +Solution +-------- + +CCE provides a storage class named **csi-disk-topology**. When you use this storage class to create a PVC, no PV will be created in pace with the PVC. Instead, the PV is created in the AZ of the node where the pod will be scheduled. An EVS disk is then created in the same AZ to ensure that the EVS disk can be attached and the pod can be successfully scheduled. + +csi-disk-topology postpones the binding between a PVC and a PV for a while. + +|image2| + +Failed Pod Scheduling Due to csi-disk Used in Cross-AZ Node Deployment +---------------------------------------------------------------------- + +Create a cluster with three nodes in different AZs. + +Use the csi-disk storage class to create a StatefulSet and check whether the workload is successfully created. + +.. code-block:: + + apiVersion: apps/v1 + kind: StatefulSet + metadata: + name: nginx + spec: + serviceName: nginx # Name of the headless Service + replicas: 4 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: container-0 + image: nginx:alpine + resources: + limits: + cpu: 600m + memory: 200Mi + requests: + cpu: 600m + memory: 200Mi + volumeMounts: # Storage mounted to the pod + - name: data + mountPath: /usr/share/nginx/html # Mount the storage to /usr/share/nginx/html. + imagePullSecrets: + - name: default-secret + volumeClaimTemplates: + - metadata: + name: data + annotations: + everest.io/disk-volume-type: SAS + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: csi-disk + +The StatefulSet uses the following headless Service. + +.. code-block:: + + apiVersion: v1 + kind: Service # Object type (Service) + metadata: + name: nginx + labels: + app: nginx + spec: + ports: + - name: nginx # Name of the port for communication between pods + port: 80 # Port number for communication between pods + selector: + app: nginx # Select the pod whose label is app:nginx. + clusterIP: None # Set this parameter to None, indicating the headless Service. + +After the creation, check the PVC and pod status. In the following output, the PVC has been created and bound successfully, and a pod is in the Pending state. + +.. code-block:: + + # kubectl get pvc -owide + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE + data-nginx-0 Bound pvc-04e25985-fc93-4254-92a1-1085ce19d31e 1Gi RWO csi-disk 64s Filesystem + data-nginx-1 Bound pvc-0ae6336b-a2ea-4ddc-8f63-cfc5f9efe189 1Gi RWO csi-disk 47s Filesystem + data-nginx-2 Bound pvc-aa46f452-cc5b-4dbd-825a-da68c858720d 1Gi RWO csi-disk 30s Filesystem + data-nginx-3 Bound pvc-3d60e532-ff31-42df-9e78-015cacb18a0b 1Gi RWO csi-disk 14s Filesystem + + # kubectl get pod -owide + NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES + nginx-0 1/1 Running 0 2m25s 172.16.0.12 192.168.0.121 + nginx-1 1/1 Running 0 2m8s 172.16.0.136 192.168.0.211 + nginx-2 1/1 Running 0 111s 172.16.1.7 192.168.0.240 + nginx-3 0/1 Pending 0 95s + +The event information of the pod shows that the scheduling fails due to no available node. Two nodes (in AZ 1 and AZ 2) do not have sufficient CPUs, and the created EVS disk is not in the AZ where the third node (in AZ 3) is located. As a result, the pod cannot use the EVS disk. + +.. code-block:: + + # kubectl describe pod nginx-3 + Name: nginx-3 + ... + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Warning FailedScheduling 111s default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims. + Warning FailedScheduling 111s default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims. + Warning FailedScheduling 28s default-scheduler 0/3 nodes are available: 1 node(s) had volume node affinity conflict, 2 Insufficient cpu. + +Check the AZ where the EVS disk created from the PVC is located. It is found that data-nginx-3 is in AZ 1. In this case, the node in AZ 1 has no resources, and only the node in AZ 3 has CPU resources. As a result, the scheduling fails. Therefore, there should be a delay between creating the PVC and binding the PV. + +Storage Class for Delayed Binding +--------------------------------- + +If you check the cluster storage class, you can see that the binding mode of csi-disk-topology is **WaitForFirstConsumer**, indicating that a PV is created and bound when a pod uses the PVC. That is, the PV and the underlying storage resources are created based on the pod information. + +.. code-block:: + + # kubectl get storageclass + NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE + csi-disk everest-csi-provisioner Delete Immediate true 156m + csi-disk-topology everest-csi-provisioner Delete WaitForFirstConsumer true 156m + csi-nas everest-csi-provisioner Delete Immediate true 156m + csi-obs everest-csi-provisioner Delete Immediate false 156m + +**VOLUMEBINDINGMODE** is displayed if your cluster is v1.19. It is not displayed in clusters of v1.17 or v1.15. + +You can also view the binding mode in the csi-disk-topology details. + +.. code-block:: + + # kubectl describe sc csi-disk-topology + Name: csi-disk-topology + IsDefaultClass: No + Annotations: + Provisioner: everest-csi-provisioner + Parameters: csi.storage.k8s.io/csi-driver-name=disk.csi.everest.io,csi.storage.k8s.io/fstype=ext4,everest.io/disk-volume-type=SAS,everest.io/passthrough=true + AllowVolumeExpansion: True + MountOptions: + ReclaimPolicy: Delete + VolumeBindingMode: WaitForFirstConsumer + Events: + +Create PVCs of the csi-disk and csi-disk-topology classes. Observe the differences between these two types of PVCs. + +- csi-disk + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: disk + annotations: + everest.io/disk-volume-type: SAS + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + storageClassName: csi-disk # StorageClass + +- csi-disk-topology + + .. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: topology + annotations: + everest.io/disk-volume-type: SAS + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 10Gi + storageClassName: csi-disk-topology # StorageClass + +View the PVC details. As shown below, the csi-disk PVC is in Bound state and the csi-disk-topology PVC is in Pending state. + +.. code-block:: + + # kubectl create -f pvc1.yaml + persistentvolumeclaim/disk created + # kubectl create -f pvc2.yaml + persistentvolumeclaim/topology created + # kubectl get pvc + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE + disk Bound pvc-88d96508-d246-422e-91f0-8caf414001fc 10Gi RWO csi-disk 18s + topology Pending csi-disk-topology 2s + +View details about the csi-disk-topology PVC. You can see that "waiting for first consumer to be created before binding" is displayed in the event, indicating that the PVC is bound after the consumer (pod) is created. + +.. code-block:: + + # kubectl describe pvc topology + Name: topology + Namespace: default + StorageClass: csi-disk-topology + Status: Pending + Volume: + Labels: + Annotations: everest.io/disk-volume-type: SAS + Finalizers: [kubernetes.io/pvc-protection] + Capacity: + Access Modes: + VolumeMode: Filesystem + Used By: + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal WaitForFirstConsumer 5s (x3 over 30s) persistentvolume-controller waiting for first consumer to be created before binding + +Create a workload that uses the PVC. Set the PVC name to **topology**. + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx-deployment + spec: + selector: + matchLabels: + app: nginx + replicas: 1 + template: + metadata: + labels: + app: nginx + spec: + containers: + - image: nginx:alpine + name: container-0 + volumeMounts: + - mountPath: /tmp # Mount path + name: topology-example + restartPolicy: Always + volumes: + - name: topology-example + persistentVolumeClaim: + claimName: topology # PVC name + +After the PVC is created, check the PVC details. You can see that the PVC is bound successfully. + +.. code-block:: + + # kubectl describe pvc topology + Name: topology + Namespace: default + StorageClass: csi-disk-topology + Status: Bound + .... + Used By: nginx-deployment-fcd9fd98b-x6tbs + Events: + Type Reason Age From Message + ---- ------ ---- ---- ------- + Normal WaitForFirstConsumer 84s (x26 over 7m34s) persistentvolume-controller waiting for first consumer to be created before binding + Normal Provisioning 54s everest-csi-provisioner_everest-csi-controller-7965dc48c4-5k799_2a6b513e-f01f-4e77-af21-6d7f8d4dbc98 External provisioner is provisioning volume for claim "default/topology" + Normal ProvisioningSucceeded 52s everest-csi-provisioner_everest-csi-controller-7965dc48c4-5k799_2a6b513e-f01f-4e77-af21-6d7f8d4dbc98 Successfully provisioned volume pvc-9a89ea12-4708-4c71-8ec5-97981da032c9 + +Using csi-disk-topology in Cross-AZ Node Deployment +--------------------------------------------------- + +The following uses csi-disk-topology to create a StatefulSet with the same configurations used in the preceding example. + +.. code-block:: + + volumeClaimTemplates: + - metadata: + name: data + annotations: + everest.io/disk-volume-type: SAS + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + storageClassName: csi-disk-topology + +After the creation, check the PVC and pod status. As shown in the following output, the PVC and pod can be created successfully. The nginx-3 pod is created on the node in AZ 3. + +.. code-block:: + + # kubectl get pvc -owide + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE VOLUMEMODE + data-nginx-0 Bound pvc-43802cec-cf78-4876-bcca-e041618f2470 1Gi RWO csi-disk-topology 55s Filesystem + data-nginx-1 Bound pvc-fc942a73-45d3-476b-95d4-1eb94bf19f1f 1Gi RWO csi-disk-topology 39s Filesystem + data-nginx-2 Bound pvc-d219f4b7-e7cb-4832-a3ae-01ad689e364e 1Gi RWO csi-disk-topology 22s Filesystem + data-nginx-3 Bound pvc-b54a61e1-1c0f-42b1-9951-410ebd326a4d 1Gi RWO csi-disk-topology 9s Filesystem + + # kubectl get pod -owide + NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES + nginx-0 1/1 Running 0 65s 172.16.1.8 192.168.0.240 + nginx-1 1/1 Running 0 49s 172.16.0.13 192.168.0.121 + nginx-2 1/1 Running 0 32s 172.16.0.137 192.168.0.211 + nginx-3 1/1 Running 0 19s 172.16.1.9 192.168.0.240 + +.. |image1| image:: /_static/images/en-us_image_0000001113962636.png +.. |image2| image:: /_static/images/en-us_image_0000001160642447.png diff --git a/umn/source/change_history.rst b/umn/source/change_history.rst index 07d2267..572e0ab 100644 --- a/umn/source/change_history.rst +++ b/umn/source/change_history.rst @@ -10,37 +10,49 @@ Change History +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Released On | What's New | +===================================+=======================================================================================================================================================================================================================================+ + | 2023-02-10 | - Supported the creation of clusters of v1.25. | + | | - Added :ref:`Configuring Pod Security Admission `. | + | | - Added :ref:`Vulnerability Fixing Policies `. | + | | - Updated :ref:`Using kubectl to Create an ELB Ingress `. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | 2022-12-20 | - Updated :ref:`OS Patch Notes for Cluster Nodes `. | + | | - Added :ref:`volcano `. | + | | - Added :ref:`Service Account Token Security Improvement `. | + | | - Definition of new permission management roles: CCE ReadOnlyAccess, CCE Administrator, CCE FullAccess. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | 2022-11-21 | Added :ref:`Best Practice `. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2022-08-27 | EulerOS 2.9 is supported. For details, see :ref:`OS Patch Notes for Cluster Nodes `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | 2022-07-13 | Supported egress rules. For details, see :ref:`Network Policies `. | + | 2022-07-13 | Supported egress rules. For details, see :ref:`Network Policies `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2022-05-24 | - Supported the creation of clusters of v1.23. | | | - Allowed cluster upgrade from v1.21 to v1.23. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2022-05-16 | Added :ref:`Linux Polkit Privilege Escalation Vulnerability (CVE-2021-4034) `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | 2022-04-14 | Allowed cluster upgrade from v1.19 to v1.21. For details, see :ref:`Performing In-place Upgrade `. | + | 2022-04-14 | Allowed cluster upgrade from v1.19 to v1.21. For details, see :ref:`Performing In-place Upgrade `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2022-03-24 | - Supported the creation of clusters of v1.21. | - | | - Two-way authentication is supported for domain name access. For details, see :ref:`Two-Way Authentication for Domain Names `. | - | | - The Docker storage mode of nodes running CentOS 7 in CCE clusters is changed from Device Mapper to OverlayFS. For details, see :ref:`Node Overview `. | + | | - Two-way authentication is supported for domain name access. For details, see :ref:`Two-Way Authentication for Domain Names `. | + | | - The Docker storage mode of nodes running CentOS 7 in CCE clusters is changed from Device Mapper to OverlayFS. For details, see :ref:`Node Overview `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2022-02-17 | Supported the creation of CCE Turbo Cluster. | | | | - | | - Added :ref:`CCE Turbo Clusters and CCE Clusters `. | - | | - Added :ref:`Creating a CCE Turbo Cluster `. | - | | - Added :ref:`Creating a Node in a CCE Turbo Cluster `. | - | | - Added :ref:`ENI LoadBalancer `. | - | | - Added :ref:`SecurityGroups `. | + | | - Added :ref:`CCE Turbo Clusters and CCE Clusters `. | + | | - Added :ref:`Creating a CCE Turbo Cluster `. | + | | - Added :ref:`Creating a Node in a CCE Turbo Cluster `. | + | | - Added ENI LoadBalancer. | + | | - Added :ref:`SecurityGroups `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | 2021-12-14 | The validity period of the certificate of cluster can be configured. For details, see :ref:`Obtaining a Cluster Certificate `. | + | 2021-12-14 | The validity period of the certificate of cluster can be configured. For details, see :ref:`Obtaining a Cluster Certificate `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | 2021-11-30 | - Added :ref:`Removing a Node `. | - | | - Added :ref:`Configuring Node Scheduling (Tainting) `. | + | 2021-11-30 | - Added :ref:`Removing a Node `. | + | | - Added :ref:`Configuring Node Scheduling (Tainting) `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2021-11-15 | - Supported the creation of clusters of v1.19.10. | - | | - SFS and OBS mount options can be configured. For details, see :ref:`Setting Mount Options `. | - | | - Custom keys are supported for OBS. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. | + | | - SFS and OBS mount options can be configured. For details, see :ref:`Setting Mount Options `. | + | | - Custom keys are supported for OBS. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2021-06-23 | - Updated autoscaler. | | | | @@ -53,10 +65,10 @@ Change History | | - Deleted the description of Open source images. | | | - Deleted the description of DNAT. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | 2021-01-30 | - Updated :ref:`Creating a CCE Cluster `. | - | | - Updated :ref:`Upgrading a Cluster `. | - | | - Updated :ref:`Managing a Node Pool `. | - | | - Updated :ref:`Ingress `. | + | 2021-01-30 | - Updated :ref:`Creating a CCE Cluster `. | + | | - Updated :ref:`Upgrading a Cluster `. | + | | - Updated :ref:`Managing a Node Pool `. | + | | - Updated :ref:`Ingress `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2020-11-02 | Allowed cluster upgrade from v1.15 to v1.17. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -66,7 +78,7 @@ Change History +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2020-03-25 | Supported clusters of v1.15.6. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | 2020-02-21 | Updated :ref:`Namespaces `. | + | 2020-02-21 | Updated :ref:`Namespaces `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2019-10-30 | - Added the gpu-beta add-on. | | | - Revised descriptions to indicate support for creating Kubernetes clusters 1.13.10. | @@ -83,7 +95,7 @@ Change History | 2019-09-03 | Revised descriptions according to the suggestions raised in UAT. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2019-07-30 | - Allows users to modify Maximum Number of Unavailable Pods after creating an application. | - | | - Allows users to add pod scheduling policies after creating an application. For details, see :ref:`Affinity and Anti-Affinity Scheduling `. | + | | - Allows users to add pod scheduling policies after creating an application. For details, see :ref:`Affinity and Anti-Affinity Scheduling `. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2019-07-29 | Revised descriptions according to the suggestions raised in UAT. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -154,7 +166,6 @@ Change History +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2018-09-15 | - Added a step to the procedure of adding a node to a BMS cluster. For details, see 3.4-Adding Existing Nodes to a BMS Cluster. | | | - Deleted the EVS and ELB related constraints. For details, see 3.4-Constraints. | - | | - Added the description of DeH in 3.7-Table Parameters for creating a node. | +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | 2018-09-05 | - Only manual scaling is supported by stateful applications. For details, see 4.10-Manual Scaling. | | | - Added the procedure for creating BMS clusters. | diff --git a/umn/source/charts_helm/my_charts/preparing_a_chart.rst b/umn/source/charts/deploying_an_application_from_a_chart.rst similarity index 57% rename from umn/source/charts_helm/my_charts/preparing_a_chart.rst rename to umn/source/charts/deploying_an_application_from_a_chart.rst index e0c3437..2d9716e 100644 --- a/umn/source/charts_helm/my_charts/preparing_a_chart.rst +++ b/umn/source/charts/deploying_an_application_from_a_chart.rst @@ -1,69 +1,23 @@ -:original_name: cce_01_0144.html +:original_name: cce_10_0146.html -.. _cce_01_0144: +.. _cce_10_0146: -Preparing a Chart -================= +Deploying an Application from a Chart +===================================== -You can prepare a chart using one of the following methods: +On the CCE console, you can upload a Helm chart package, deploy it, and manage the deployed pods. -- :ref:`Customizing a Chart ` -- :ref:`Using a Kubernetes Official Chart ` +Notes and Constraints +--------------------- -.. _cce_01_0144__s84a75de063eb4fb29387e64d133b0da6: - -Customizing a Chart -------------------- - -#. Customize the content of a chart as required. - - For details about how to create a chart, see https://helm.sh/docs/chart_template_guide/getting_started/. - -#. Set the chart directory structure and name the chart based on the requirements defined in :ref:`Chart Specifications `. - -.. _cce_01_0144__s5f9699b10586401d81cfebd947cf416f: - -Using a Kubernetes Official Chart ---------------------------------- - -#. .. _cce_01_0144__l6d35ccf85da74660b802f524cc9e3095: - - Visit https://artifacthub.io/ to obtain the required chart. - -#. Log in to a Linux host. - -#. Upload the chart obtained in :ref:`1 `. - -#. Run the following command to compress the chart. - - - If the Helm client is not installed on the Linux host, run the following command: - - **tar pzcf {name}-{version}.tgz {name}/** - - In the preceding command, - - **{name}** indicates the actual chart name. - - **{version}** indicates the actual chart version. - - .. important:: - - The values of {name} and {version} must be the same as the values of name and version in the **Chart.yaml** file in the chart. - - - If the Helm client is installed on the Linux host, run the following command: - - **helm package {name}/** - - In the preceding command, replace **{name}** with the actual chart name. - -#. Set the chart directory structure and name the chart based on the requirements defined in :ref:`Chart Specifications `. - -.. _cce_01_0144__s8af9336c49a44399865c7a0f3149d789: +- The number of charts that can be uploaded by a single user is limited. The value displayed on the console of each region is the allowed quantity. +- A chart with multiple versions consumes the same amount of portion of chart quota. +- Users with chart operation permissions can perform multiple operations on clusters. Therefore, exercise caution when assigning users the chart lifecycle management permissions, including uploading charts and creating, deleting, and updating chart releases. Chart Specifications -------------------- -This section uses the redis chart as an example to illustrate the chart specifications. +The Redis workload is used as an example to illustrate the chart specifications. - **Naming Requirement** @@ -91,9 +45,9 @@ This section uses the redis chart as an example to illustrate the chart specific Chart.yaml .helmignore - As listed in :ref:`Table 1 `, the parameters marked with \* are mandatory. + As listed in :ref:`Table 1 `, the parameters marked with \* are mandatory. - .. _cce_01_0144__tb7d789a3467e4fe9b4385a51f3460321: + .. _cce_10_0146__tb7d789a3467e4fe9b4385a51f3460321: .. table:: **Table 1** Parameters in the directory structure of a chart @@ -119,6 +73,80 @@ This section uses the redis chart as an example to illustrate the chart specific | | - Information about chart installation and configuration. | +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | \* Chart.yaml | Basic information about the chart. | + | | | + | | Note: Helm v3 bumps the apiVersion from v1 to v2. | +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | .helmignore | Files or data that does not need to read templates during workload installation. | +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +Uploading a Chart +----------------- + +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Charts** in the navigation pane and click **Upload Chart** in the upper right corner. +#. Click **Select File**, select the chart to be uploaded, and click **Upload**. + + .. note:: + + When you upload a chart, the naming rule of the OBS bucket is changed from cce-charts-{*region*}-{**domain_name**} to cce-charts-{*region*}-{**domain_id**}. In the old naming rule, the system converts the **domain_name** value into a Base64 string and uses the first 63 characters. If you cannot find the chart in the OBS bucket with the new name, search for the bucket with the old name. + +Creating a Release +------------------ + +#. Log in to the CCE console, click the cluster name, and access the cluster console. In the navigation pane, choose **Charts**. + +#. On the **My Charts** tab page, click **Install** of the target chart. + +#. Set workload installation parameters by referring to :ref:`Table 2 `. + + .. _cce_10_0146__t26bc1c499f114b5185e5edcf61e44d95: + + .. table:: **Table 2** Installation parameters + + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+===========================================================================================================================================================================================+ + | Instance | Unique name of the chart release. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Namespace | Namespace to which the workload will be deployed. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Select Version | Version of a chart. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Configuration File | You can import and replace the **values.yaml** file or directly edit the chart parameters online. | + | | | + | | .. note:: | + | | | + | | An imported **values.yaml** file must comply with YAML specifications, that is, KEY:VALUE format. The fields in the file are not restricted. | + | | | + | | The key value of the imported values.yaml must be the same as that of the selected chart package. Otherwise, the values.yaml does not take effect. That is, the key cannot be changed. | + | | | + | | a. Click **Select File**. | + | | b. Select the corresponding **values.yaml** file and click **Open**. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. Click **Install**. + + On the **Releases** tab page, you can view the installation status of the release. + +Upgrading a Chart-based Workload +-------------------------------- + +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Charts** in the navigation pane and click the **Releases** tab. +#. Click **Upgrade** in the row where the desired workload resides and set the parameters for the workload. +#. Select a chart version for **Chart Version**. +#. Follow the prompts to modify the chart parameters. Click **Upgrade**, and then click **Submit**. +#. Click **Back to Release List**. If the chart status changes to **Upgrade successful**, the workload is successfully upgraded. + +Rolling Back a Chart-based Workload +----------------------------------- + +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Charts** in the navigation pane and click the **Releases** tab. + +#. Click **More** > **Roll Back** for the workload to be rolled back, select the workload version, and click **Roll back** **to this version**. + + In the workload list, if the status is **Rollback successful**, the workload is rolled back successfully. + +Uninstalling a Chart-based Workload +----------------------------------- + +#. Log in to the CCE console, click the cluster name, and access the cluster console. Choose **Charts** in the navigation pane and click the **Releases** tab. +#. Click **More** > **Uninstall** next to the release to be uninstalled, and click **Yes**. Exercise caution when performing this operation because releases cannot be restored after being uninstalled. diff --git a/umn/source/charts/index.rst b/umn/source/charts/index.rst new file mode 100644 index 0000000..83d69ba --- /dev/null +++ b/umn/source/charts/index.rst @@ -0,0 +1,16 @@ +:original_name: cce_10_0019.html + +.. _cce_10_0019: + +Charts +====== + +- :ref:`Overview ` +- :ref:`Deploying an Application from a Chart ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + overview + deploying_an_application_from_a_chart diff --git a/umn/source/charts/overview.rst b/umn/source/charts/overview.rst new file mode 100644 index 0000000..d848ec9 --- /dev/null +++ b/umn/source/charts/overview.rst @@ -0,0 +1,36 @@ +:original_name: cce_10_0191.html + +.. _cce_10_0191: + +Overview +======== + +CCE provides a console for managing Helm charts, helping you easily deploy applications using the charts and manage applications on the console. + +Helm +---- + +`Helm `__ is a package manager for Kubernetes and manages charts. A Helm chart is a series of YAML files used to encapsulate native Kubernetes applications. When deploying an application, you can customize some metadata of the application for easy application distribution. Application releasers can use Helm to package applications, manage application dependencies and application versions, and release applications to the software repository. After using Helm, users do not need to compile complex application deployment files. They can easily search for, install, upgrade, roll back, and uninstall applications on Kubernetes. + +The relationship between Helm and Kubernetes is as follows: + +- Helm <-> Kubernetes +- Apt <-> Ubuntu +- Yum <-> CentOS +- Pip <-> Python + +The following figure shows the solution architecture: + +|image1| + +Helm can help application orchestration for Kubernetes: + +- Manages, edits, and updates a large number of Kubernetes configuration files. +- Deploys a complex Kubernetes application that contains a large number of configuration files. +- Shares and reuses Kubernetes configurations and applications. +- Supports multiple environments with parameter-based configuration templates. +- Manages the release of applications, including rolling back the application, finding differences (using the **diff** command), and viewing the release history. +- Controls phases in a deployment cycle. +- Tests and verifies the released version. + +.. |image1| image:: /_static/images/en-us_image_0000001325364477.png diff --git a/umn/source/charts_helm/index.rst b/umn/source/charts_helm/index.rst deleted file mode 100644 index 4a6fa2c..0000000 --- a/umn/source/charts_helm/index.rst +++ /dev/null @@ -1,14 +0,0 @@ -:original_name: cce_01_0019.html - -.. _cce_01_0019: - -Charts (Helm) -============= - -- :ref:`My Charts ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - my_charts/index diff --git a/umn/source/charts_helm/my_charts/creating_a_workload_from_a_chart.rst b/umn/source/charts_helm/my_charts/creating_a_workload_from_a_chart.rst deleted file mode 100644 index 3e8ad2a..0000000 --- a/umn/source/charts_helm/my_charts/creating_a_workload_from_a_chart.rst +++ /dev/null @@ -1,72 +0,0 @@ -:original_name: cce_01_0146.html - -.. _cce_01_0146: - -Creating a Workload from a Chart -================================ - -Creating a Chart-based Workload -------------------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Charts** > **Uploaded Chart**. - -#. In the list of uploaded charts, click **Install**. - -#. Set the installation parameters listed in :ref:`Table 1 `. The parameters marked with an asterisk (*) are mandatory. - - .. _cce_01_0146__t26bc1c499f114b5185e5edcf61e44d95: - - .. table:: **Table 1** Installation parameters - - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===========================================================================================================================================================================================+ - | \* Release Name | Unique name of the chart release. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Chart Version | Chart version by default. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Cluster | Cluster where the workload will be deployed. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Namespace | Namespace to which the workload will be deployed. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Advanced Settings | You can import and replace the **values.yaml** file or directly edit the chart parameters online. | - | | | - | | .. note:: | - | | | - | | An imported **values.yaml** file must comply with YAML specifications, that is, KEY:VALUE format. The fields in the file are not restricted. | - | | | - | | The key value of the imported values.yaml must be the same as that of the selected chart package. Otherwise, the values.yaml does not take effect. That is, the key cannot be changed. | - | | | - | | a. Click **Import Configuration File**. | - | | b. Select the corresponding **values.yaml** file and click **Open**. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. After the configuration is complete, click **Next**. - -#. Confirm the configuration and click **Submit**. - -#. Click **Back to Release List** to view the running status of the chart-based workload (also called release), or click **View Release Details** to view details about the release. - -Upgrading a Chart-based Workload --------------------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Charts** > **Uploaded Charts**. Click the **Template Instances** tab. -#. Click **Upgrade** in the row where the desired workload resides and set the parameters for the workload. -#. Select a chart version for **Chart Version**. -#. Follow the prompts to modify the chart parameters. Click **Upgrade**, and then click **Submit**. -#. Click **Back to Release List**. If the chart status changes to **Upgrade successful**, the workload is successfully upgraded. - -Rolling Back a Chart-based Workload ------------------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Charts** > **Uploaded Charts**. Click the **Template Instances** tab. - -#. Click **More** > **Roll Back** for the workload to be rolled back, select the workload version, and click **Roll back** **to this version**. - - In the workload list, if the status is **Rollback successful**, the workload is rolled back successfully. - -Uninstalling a Chart-based Workload ------------------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Charts** > **Uploaded Charts**. Click the **Template Instances** tab. -#. Click **More** > **Uninstall** next to the release to be uninstalled, and click **Yes**. Exercise caution when performing this operation because releases cannot be restored after being uninstalled. diff --git a/umn/source/charts_helm/my_charts/index.rst b/umn/source/charts_helm/my_charts/index.rst deleted file mode 100644 index 8e41a39..0000000 --- a/umn/source/charts_helm/my_charts/index.rst +++ /dev/null @@ -1,20 +0,0 @@ -:original_name: cce_01_0143.html - -.. _cce_01_0143: - -My Charts -========= - -- :ref:`Overview ` -- :ref:`Preparing a Chart ` -- :ref:`Uploading a Chart ` -- :ref:`Creating a Workload from a Chart ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - overview - preparing_a_chart - uploading_a_chart - creating_a_workload_from_a_chart diff --git a/umn/source/charts_helm/my_charts/overview.rst b/umn/source/charts_helm/my_charts/overview.rst deleted file mode 100644 index ceda815..0000000 --- a/umn/source/charts_helm/my_charts/overview.rst +++ /dev/null @@ -1,24 +0,0 @@ -:original_name: cce_01_0191.html - -.. _cce_01_0191: - -Overview -======== - -CCE uses Helm, a Kubernetes package manager, to simplify deployment and management of packages (also called charts). A chart is a collection of files that describe a related set of Kubernetes resources. The use of charts handles all the complexity in Kubernetes resource installation and management, making it possible to achieve unified resource scheduling and management. - -.. note:: - - Helm is a tool for packaging Kubernetes applications. For more information, see `Helm documentation `__. - -Custom charts simplify workload deployment. - -This section describes how to create a workload using a custom chart. You can use multiple methods to create an orchestration chart on the CCE console. - -Notes and Constraints ---------------------- - -- The number of charts that can be uploaded by a single user is limited. The value displayed on the console of each region is the allowed quantity. -- CCE uses Helm v2.12. If you use Helm v3 or later to manage CCE, compatibility problems may occur. -- A chart with multiple versions consumes the same amount of portion of chart quota. -- Users with chart operation permissions can perform multiple operations on clusters. Therefore, exercise caution when assigning users the chart lifecycle management permissions, including uploading charts and creating, deleting, and updating chart releases. diff --git a/umn/source/charts_helm/my_charts/uploading_a_chart.rst b/umn/source/charts_helm/my_charts/uploading_a_chart.rst deleted file mode 100644 index dc49772..0000000 --- a/umn/source/charts_helm/my_charts/uploading_a_chart.rst +++ /dev/null @@ -1,44 +0,0 @@ -:original_name: cce_01_0145.html - -.. _cce_01_0145: - -Uploading a Chart -================= - -Scenario --------- - -Upload a chart to **Charts** > **Uploaded Charts** for subsequent workload creation. - -Procedure ---------- - -#. Log in to the CCE console. In the navigation pane, choose **Charts** > **Uploaded Charts** and click **Upload Chart**. -#. Click **Select File**, select the chart to be uploaded, and click **Upload**. - - .. note:: - - When you upload a chart, the naming rule of the OBS bucket is changed from cce-charts-{*region*}-{**domain_name**} to cce-charts-{*region*}-{**domain_id**}. In the old naming rule, the system converts the **domain_name** value into a Base64 string and uses the first 63 characters. If you cannot find the chart in the OBS bucket with the new name, search for the bucket with the old name. - -Related Operations ------------------- - -After a chart is created, you can perform operations listed in :ref:`Table 1 ` on the **Uploaded Charts** page. - -.. _cce_01_0145__t84ae87674877489b975382f30a71dfab: - -.. table:: **Table 1** Related operations - - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==============================================================================================================================================+ - | Installing a chart | Click **Install Chart** to install the chart for creating workloads. For details, see :ref:`Creating a Workload from a Chart `. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ - | Updating a chart | The chart content will be updated while the chart version remains unchanged. The procedure is similar to that of uploading a chart. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ - | Downloading a chart | Click **More** > **Download** to download the chart to the local host. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ - | Deleting a chart | Click **More** > **Delete** to delete the installed chart. | - | | | - | | Deleted charts cannot be restored. Exercise caution when performing this operation. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/cloud_trace_service_cts/cce_operations_supported_by_cts.rst b/umn/source/cloud_trace_service_cts/cce_operations_supported_by_cts.rst index c76adaf..c266914 100644 --- a/umn/source/cloud_trace_service_cts/cce_operations_supported_by_cts.rst +++ b/umn/source/cloud_trace_service_cts/cce_operations_supported_by_cts.rst @@ -1,14 +1,10 @@ -:original_name: cce_01_0025.html +:original_name: cce_10_0025.html -.. _cce_01_0025: +.. _cce_10_0025: CCE Operations Supported by CTS =============================== -.. important:: - - CTS is available only in certain regions. - .. table:: **Table 1** CCE operations supported by CTS +--------------------------------------------------------+---------------------+------------------------------+ diff --git a/umn/source/cloud_trace_service_cts/index.rst b/umn/source/cloud_trace_service_cts/index.rst index 0a8c0d2..7149ada 100644 --- a/umn/source/cloud_trace_service_cts/index.rst +++ b/umn/source/cloud_trace_service_cts/index.rst @@ -1,12 +1,12 @@ -:original_name: cce_01_0024.html +:original_name: cce_10_0024.html -.. _cce_01_0024: +.. _cce_10_0024: Cloud Trace Service (CTS) ========================= -- :ref:`CCE Operations Supported by CTS ` -- :ref:`Querying CTS Logs ` +- :ref:`CCE Operations Supported by CTS ` +- :ref:`Querying CTS Logs ` .. toctree:: :maxdepth: 1 diff --git a/umn/source/cloud_trace_service_cts/querying_cts_logs.rst b/umn/source/cloud_trace_service_cts/querying_cts_logs.rst index fdc622b..3c0459c 100644 --- a/umn/source/cloud_trace_service_cts/querying_cts_logs.rst +++ b/umn/source/cloud_trace_service_cts/querying_cts_logs.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0026.html +:original_name: cce_10_0026.html -.. _cce_01_0026: +.. _cce_10_0026: Querying CTS Logs ================= @@ -42,7 +42,7 @@ Procedure #. Click |image2| on the left of a trace to expand its details, as shown below. - .. figure:: /_static/images/en-us_image_0000001144779790.png + .. figure:: /_static/images/en-us_image_0000001243981141.png :alt: **Figure 1** Expanding trace details **Figure 1** Expanding trace details @@ -50,10 +50,10 @@ Procedure #. Click **View Trace** in the **Operation** column. The trace details are displayed. - .. figure:: /_static/images/en-us_image_0000001144620002.png + .. figure:: /_static/images/en-us_image_0000001244141139.png :alt: **Figure 2** Viewing event details **Figure 2** Viewing event details -.. |image1| image:: /_static/images/en-us_image_0144054048.gif -.. |image2| image:: /_static/images/en-us_image_0144049227.png +.. |image1| image:: /_static/images/en-us_image_0000001244141141.gif +.. |image2| image:: /_static/images/en-us_image_0000001199341250.png diff --git a/umn/source/clusters/cce_turbo_clusters_and_cce_clusters.rst b/umn/source/clusters/cce_turbo_clusters_and_cce_clusters.rst deleted file mode 100644 index 4519afe..0000000 --- a/umn/source/clusters/cce_turbo_clusters_and_cce_clusters.rst +++ /dev/null @@ -1,43 +0,0 @@ -:original_name: cce_01_0342.html - -.. _cce_01_0342: - -CCE Turbo Clusters and CCE Clusters -=================================== - -Comparison Between CCE Turbo Clusters and CCE Clusters ------------------------------------------------------- - -The following table lists the differences between CCE Turbo clusters and CCE clusters: - -.. table:: **Table 1** Cluster types - - +-----------------+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | Dimensions | Sub-dimension | CCE Turbo Cluster | CCE Cluster | - +=================+=============================+==================================================================================================================================+======================================================================================================================================+ - | Cluster | Positioning | Next-generation container cluster for Cloud Native 2.0 with accelerated computing, networking, and scheduling | Standard cluster for common commercial use | - +-----------------+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | | Node type | Hybrid deployment of VMs and bare-metal servers | Hybrid deployment of VMs | - +-----------------+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | Network | Network model | **Cloud Native Network 2.0**: applies to large-scale and high-performance scenarios. | **Cloud-native network 1.0** for scenarios that do not require high performance or involve large-scale deployment. | - | | | | | - | | | Networking scale: 2000 nodes | - Tunnel network model | - | | | | - VPC network model | - +-----------------+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | | Network performance | The VPC network and container network are flattened into one, achieving zero performance loss. | The VPC network is overlaid with the container network, causing certain performance loss. | - +-----------------+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | | Container network isolation | Pods can be directly associated with security groups to configure isolation policies for resources inside and outside a cluster. | - Tunnel network model: Network isolation policies are supported for intra-cluster communication (by configuring network policies). | - | | | | - VPC network model: Isolation is not supported. | - +-----------------+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | Security | Isolation | - Bare-metal server: You can select secure containers for VM-level isolation. | Common containers are deployed and isolated by Cgroups. | - | | | - VM: Common containers are deployed. | | - +-----------------+-----------------------------+----------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - -QingTian Architecture ---------------------- - -|image1| - -The QingTian architecture consists of data plane (software-hardware synergy) and management plane (Alkaid Smart Cloud Brain). The data plane innovates in five dimensions: simplified data center, diversified computing power, QingTian cards, ultra-fast engines, and simplified virtualization, to fully offload and accelerate compute, storage, networking, and security components. VMs, bare metal servers, and containers can run together. As a distributed operating system, the Alkaid Smart Cloud Brain focuses on the cloud, AI, and 5G, and provide all-domain scheduling to achieve cloud-edge-device collaboration and governance. - -.. |image1| image:: /_static/images/en-us_image_0000001212924318.png diff --git a/umn/source/clusters/changing_cluster_scale.rst b/umn/source/clusters/changing_cluster_scale.rst new file mode 100644 index 0000000..5521e82 --- /dev/null +++ b/umn/source/clusters/changing_cluster_scale.rst @@ -0,0 +1,36 @@ +:original_name: cce_10_0403.html + +.. _cce_10_0403: + +Changing Cluster Scale +====================== + +Scenario +-------- + +CCE allows you to change the number of nodes managed in a cluster. + +Notes and Constraints +--------------------- + +- This function is supported for clusters of v1.15 and later versions. +- Starting from v1.15.11, the number of nodes in a cluster can be changed to 2000. The number of nodes in a single master node cannot be changed to 1000 or more. +- Currently, a cluster can only be scaled out to a larger specification, but cannot be scaled in. +- During the specifications change, master nodes will be powered off and on, and the cluster cannot run properly. Perform the change during off-peak hours. +- Changing the cluster scale does not affect the services running in the cluster. However, the control plane (master nodes) will be interrupted for a short period of time. You are advised not to perform any other operations (such as creating workloads) during the change. +- Change failures will trigger a cluster rollback to the normal state. If the rollback fails, submit a service ticket. + +Procedure +--------- + +#. Log in to the CCE console. In the navigation pane, choose **Clusters**. + +#. Click |image1| next to the cluster whose specifications need to be changed. + +#. On the page displayed, select a new flavor for **Target Flavor** as required. + +#. Click **OK**. + + You can click **Operation Records** in the upper left corner to view the cluster change history. The status changes from **Executing** to **Successful**, indicating that the cluster specifications are successfully changed. + +.. |image1| image:: /_static/images/en-us_image_0000001236723668.png diff --git a/umn/source/clusters/cluster_overview.rst b/umn/source/clusters/cluster_overview/basic_cluster_information.rst similarity index 50% rename from umn/source/clusters/cluster_overview.rst rename to umn/source/clusters/cluster_overview/basic_cluster_information.rst index f742ea5..f15add4 100644 --- a/umn/source/clusters/cluster_overview.rst +++ b/umn/source/clusters/cluster_overview/basic_cluster_information.rst @@ -1,20 +1,20 @@ -:original_name: cce_01_0002.html +:original_name: cce_10_0430.html -.. _cce_01_0002: +.. _cce_10_0430: -Cluster Overview -================ +Basic Cluster Information +========================= -`Kubernetes `__ is a containerized application software system that can be easily deployed and managed. It facilitates container scheduling and orchestration. +`Kubernetes `__ allows you to easily deploy and manage containerized application and facilitates container scheduling and orchestration. -For application developers, Kubernetes can be regarded as a cluster operating system. Kubernetes provides functions such as service discovery, scaling, load balancing, self-healing, and even leader election, freeing developers from infrastructure-related configurations. +For developers, Kubernetes is a cluster operating system. Kubernetes provides service discovery, scaling, load balancing, self-healing, and even leader election, freeing developers from infrastructure-related configurations. -When using Kubernetes, it is like you run a large number of servers as one on which your applications run. Regardless of the number of servers in a Kubernetes cluster, the method for deploying applications in Kubernetes is always the same. +When using Kubernetes, it is like you run a large number of servers as one and the method for deploying applications in Kubernetes is always the same. Kubernetes Cluster Architecture ------------------------------- -A Kubernetes cluster consists of master nodes (masters) and worker nodes (nodes). Applications are deployed on worker nodes, and you can specify the nodes for deployment. +A Kubernetes cluster consists of master nodes (Masters) and worker nodes (Nodes). Applications are deployed on worker nodes, and you can specify the nodes for deployment. The following figure shows the architecture of a Kubernetes cluster. @@ -33,7 +33,7 @@ A master node is the machine where the control plane components run, including A - Scheduler: schedules containers to nodes based on various conditions (such as available resources and node affinity). - etcd: serves as a distributed data storage component that stores cluster configuration information. -In the production environment, multiple master nodes are deployed to ensure cluster high availability. For example, you can deploy three master nodes for your CCE cluster. +In a production environment, multiple master nodes are deployed to ensure high cluster availability. For example, you can deploy three master nodes for your CCE cluster. **Worker node** @@ -43,12 +43,12 @@ A worker node is a compute node in a cluster, that is, a node running containeri - kube-proxy: serves as an access proxy between application components. - Container runtime: functions as the software for running containers. You can download images to build your container runtime, such as Docker. -Number of Master Nodes and Cluster Scale ----------------------------------------- +Master Nodes and Cluster Scale +------------------------------ -When you create a cluster on CCE, the number of master nodes can be set to 1 or 3. Three master nodes can be deployed to create a cluster in HA mode. +When you create a cluster on CCE, you can have one or three master nodes. Three master nodes can create a cluster in HA mode. -The master node specifications determine the number of nodes that can be managed by a cluster. When creating a cluster, you can select the cluster management scale, for example, 50 or 200 nodes. +The master node specifications decide the number of nodes that can be managed by a cluster. You can select the cluster management scale, for example, 50 or 200 nodes. Cluster Network --------------- @@ -58,42 +58,12 @@ From the perspective of the network, all nodes in a cluster are located in a VPC A cluster network can be divided into three network types: - Node network: IP addresses are assigned to nodes in a cluster. -- Container network: IP addresses are assigned to containers in a cluster for communication between them. Currently, multiple container network models are supported, and each model has its own working mechanism. -- Service network: A service is a Kubernetes object used to access containers. Each Service has a fixed IP address. +- Container network: IP addresses are assigned to containers in a cluster for communication. Currently, multiple container network models are supported, and each model has its own working mechanism. +- Service network: A Service is a Kubernetes object used to access containers. Each Service has a fixed IP address. -When you create a cluster, select a proper CIDR block for each network to ensure that the CIDR blocks do not conflict with each other and each CIDR block has sufficient available IP addresses. **After a cluster is created, the container network model cannot be modified.** Plan the container network model properly before creating a cluster. +When you create a cluster, select a proper CIDR block for each network. Ensure that the CIDR blocks do not conflict with each other and have sufficient available IP addresses. **You cannot change the container network model after the cluster is created.** Plan the container network model properly in advance. -You are advised to learn about the cluster network and container network models before creating a cluster. For details, see :ref:`Overview `. - -Cluster Security Groups ------------------------ - -When a cluster is created, the following security groups are created to ensure cluster security: - -- *Cluster name*-cce-control-*Random number*: security group of the master node. - - Observe the following principles when configuring security groups: - - - The source IP addresses defined in the security group rules must be permitted. - - **4789** (required only for clusters using the container tunnel network model): used for network access between containers. - - **5443** and **5444**: ports to which kube-apiserver of the master node listens. These ports must permit requests from VPC and container CIDR blocks. - - **9443**: used by the network add-on of a worker node to access the master node. - - **8445**: used by the storage add-on of a worker node to access the master node. - -- *Cluster name*-cce-node-*Random number*: security group of a worker node. - - Observe the following principles when configuring security groups: - - - The source IP addresses defined in the security group rules must be permitted. - - **4789** (required only for clusters using the container tunnel network model): used for network access between containers. - - **10250**: used by the master node to access the kubelet component of a worker node (for example, run the kubectl exec {pod} command). - - **30000**\ ``-``\ **32767**: external access port (Nodeport) of a node. These ports need be specified when you create a Service. These ports must permit requests from VPC, container, and ELB CIDR blocks. - -After a cluster is created, you can view the created security group on the VPC console. - -.. warning:: - - Do not delete the security groups and related rules automatically configured during cluster creation. Otherwise, the cluster will exhibit unexpected behavior. +You are advised to learn about the cluster network and container network models before creating a cluster. For details, see :ref:`Container Network Models `. Cluster Lifecycle ----------------- @@ -105,7 +75,7 @@ Cluster Lifecycle +=============+===================================================================+ | Creating | A cluster is being created and is requesting for cloud resources. | +-------------+-------------------------------------------------------------------+ - | Normal | A cluster is running properly. | + | Running | A cluster is running properly. | +-------------+-------------------------------------------------------------------+ | Scaling-out | A node is being added to a cluster. | +-------------+-------------------------------------------------------------------+ @@ -121,9 +91,3 @@ Cluster Lifecycle +-------------+-------------------------------------------------------------------+ | Deleting | A cluster is being deleted. | +-------------+-------------------------------------------------------------------+ - - -.. figure:: /_static/images/en-us_image_0000001160731158.png - :alt: **Figure 2** Cluster status transition - - **Figure 2** Cluster status transition diff --git a/umn/source/clusters/cluster_overview/cce_turbo_clusters_and_cce_clusters.rst b/umn/source/clusters/cluster_overview/cce_turbo_clusters_and_cce_clusters.rst new file mode 100644 index 0000000..3e752a3 --- /dev/null +++ b/umn/source/clusters/cluster_overview/cce_turbo_clusters_and_cce_clusters.rst @@ -0,0 +1,43 @@ +:original_name: cce_10_0342.html + +.. _cce_10_0342: + +CCE Turbo Clusters and CCE Clusters +=================================== + +Comparison Between CCE Turbo Clusters and CCE Clusters +------------------------------------------------------ + +The following table lists the differences between CCE Turbo clusters and CCE clusters: + +.. table:: **Table 1** Cluster types + + +-----------------+-----------------------------+--------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+ + | Dimension | Sub-dimension | CCE Turbo Cluster | CCE Cluster | + +=================+=============================+================================================================================================================================+========================================================================================+ + | Cluster | Positioning | Next-gen container cluster, with accelerated computing, networking, and scheduling. Designed for Cloud Native 2.0 | Standard cluster for common commercial use | + +-----------------+-----------------------------+--------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+ + | | Node type | Hybrid deployment of VMs and bare-metal servers | Hybrid deployment of VMs | + +-----------------+-----------------------------+--------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+ + | Networking | Model | **Cloud Native Network 2.0**: applies to large-scale and high-performance scenarios. | **Cloud-native network 1.0**: applies to common, smaller-scale scenarios. | + | | | | | + | | | Max networking scale: 2,000 nodes | - Tunnel network model | + | | | | - VPC network model | + +-----------------+-----------------------------+--------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+ + | | Performance | Flattens the VPC network and container network into one. No performance loss. | Overlays the VPC network with the container network, causing certain performance loss. | + +-----------------+-----------------------------+--------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+ + | | Container network isolation | Associates pods with security groups. Unifies security isolation in and out the cluster via security groups' network policies. | - Tunnel network model: supports network policies for intra-cluster communications. | + | | | | - VPC network model: supports no isolation. | + +-----------------+-----------------------------+--------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+ + | Security | Isolation | - Physical machine: runs Kata containers, allowing VM-level isolation. | Runs common containers, isolated by cgroups. | + | | | - VM: runs common containers. | | + +-----------------+-----------------------------+--------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------+ + +QingTian Architecture +--------------------- + +|image1| + +The QingTian architecture consists of data plane (software-hardware synergy) and management plane (Alkaid Smart Cloud Brain). The data plane innovates in five dimensions: simplified data center, diversified computing power, QingTian cards, ultra-fast engines, and simplified virtualization, to fully offload and accelerate compute, storage, networking, and security components. VMs, bare metal servers, and containers can run together. As a distributed operating system, the Alkaid Smart Cloud Brain focuses on the cloud, AI, and 5G, and provide all-domain scheduling to achieve cloud-edge-device collaboration and governance. + +.. |image1| image:: /_static/images/en-us_image_0000001212924318.png diff --git a/umn/source/clusters/cluster_overview/comparing_iptables_and_ipvs.rst b/umn/source/clusters/cluster_overview/comparing_iptables_and_ipvs.rst new file mode 100644 index 0000000..65b82de --- /dev/null +++ b/umn/source/clusters/cluster_overview/comparing_iptables_and_ipvs.rst @@ -0,0 +1,42 @@ +:original_name: cce_10_0349.html + +.. _cce_10_0349: + +Comparing iptables and IPVS +=========================== + +kube-proxy is a key component of a Kubernetes cluster. It is responsible for load balancing and forwarding between a Service and its backend pod. + +CCE supports two forwarding modes: iptables and IPVS. + +- IPVS allows higher throughput and faster forwarding. This mode applies to scenarios where the cluster scale is large or the number of Services is large. +- iptables is the traditional kube-proxy mode. This mode applies to the scenario where the number of Services is small or a large number of short connections are concurrently sent on the client. + +Notes and Constraints +--------------------- + +In a cluster using the IPVS proxy mode, if the ingress and Service use the same ELB load balancer, the ingress cannot be accessed from the nodes and containers in the cluster because kube-proxy mounts the LoadBalancer Service address to the ipvs-0 bridge. This bridge intercepts the traffic of the load balancer connected to the ingress. You are advised to use different ELB load balancers for the ingress and Service. + +iptables +-------- + +iptables is a Linux kernel function that provides a large amount of data packet processing and filtering capabilities. It allows flexible sequences of rules to be attached to various hooks in the packet processing pipeline. When iptables is used, kube-proxy implements NAT and load balancing in the NAT pre-routing hook. + +kube-proxy is an O(n) algorithm, in which *n* increases with the cluster scale. The cluster scale refers to the number of Services and backend pods. + +IPVS +---- + +IP Virtual Server (IPVS) is constructed on top of Netfilter and implements transport-layer load balancing as part of the Linux kernel. IPVS can direct requests for TCP- and UDP-based services to the real servers, and make services of the real servers appear as virtual services on a single IP address. + +In the IPVS mode, kube-proxy uses IPVS load balancing instead of iptables. IPVS is designed to balance loads for a large number of Services. It has a set of optimized APIs and uses optimized search algorithms instead of simply searching for rules from a list. + +The complexity of the connection process of IPVS-based kube-proxy is O(1). In other words, in most cases, the connection processing efficiency is irrelevant to the cluster scale. + +IPVS involves multiple load balancing algorithms, such as round-robin, shortest expected delay, least connections, and various hashing methods. However, iptables has only one algorithm for random selection. + +Compared with iptables, IPVS has the following advantages: + +#. Provides better scalability and performance for large clusters. +#. Supports better load balancing algorithms than iptables. +#. Supports functions including server health check and connection retries. diff --git a/umn/source/clusters/cluster_overview/index.rst b/umn/source/clusters/cluster_overview/index.rst new file mode 100644 index 0000000..89108f2 --- /dev/null +++ b/umn/source/clusters/cluster_overview/index.rst @@ -0,0 +1,20 @@ +:original_name: cce_10_0002.html + +.. _cce_10_0002: + +Cluster Overview +================ + +- :ref:`Basic Cluster Information ` +- :ref:`CCE Turbo Clusters and CCE Clusters ` +- :ref:`Comparing iptables and IPVS ` +- :ref:`Release Notes ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + basic_cluster_information + cce_turbo_clusters_and_cce_clusters + comparing_iptables_and_ipvs + release_notes/index diff --git a/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.17_release_notes.rst b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.17_release_notes.rst new file mode 100644 index 0000000..ee6e2ec --- /dev/null +++ b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.17_release_notes.rst @@ -0,0 +1,42 @@ +:original_name: cce_10_0471.html + +.. _cce_10_0471: + +CCE Kubernetes 1.17 Release Notes +================================= + +CCE has passed the Certified Kubernetes Conformance Program and is a certified Kubernetes offering. This section describes the updates in CCE Kubernetes 1.17. + +Resource Changes and Deprecations +--------------------------------- + +- All resources in the **apps/v1beta1** and **apps/v1beta2** API versions are no longer served. Migrate to use the **apps/v1** API version. +- DaemonSets, Deployments, and ReplicaSets in the **extensions/v1beta1** API version are no longer served. You can use the **apps/v1** API version. +- NetworkPolicies in the **extensions/v1beta1** API version are no longer served. Migrate to use the **networking.k8s.io/v1** API version. +- PodSecurityPolicies in the **extensions/v1beta1** API version are no longer served. Migrate to use the **policy/v1beta1** API version. +- Ingresses in the **extensions/v1beta1** API version will no longer be served in v1.20. Migrate to use the **networking.k8s.io/v1beta1** API version. +- PriorityClass in the **scheduling.k8s.io/v1beta1** and **scheduling.k8s.io/v1alpha1** API versions is no longer served in v1.17. Migrate to use the **scheduling.k8s.io/v1** API version. +- The **event series.state** field in the **events.k8s.io/v1beta1** API version has been deprecated and will be removed from v1.18. +- **CustomResourceDefinition** in the **apiextensions.k8s.io/v1beta1** API version has been deprecated and will no longer be served in v1.19. Use the **apiextensions.k8s.io/v1** API version. +- **MutatingWebhookConfiguration** and **ValidatingWebhookConfiguration** in the **admissionregistration.k8s.io/v1beta1** API version have been deprecated and will no longer be served in v1.19. You can use the **admissionregistration.k8s.io/v1** API version. +- The **rbac.authorization.k8s.io/v1alpha1** and **rbac.authorization.k8s.io/v1beta1** API versions have been deprecated and will no longer be served in v1.20. Use the **rbac.authorization.k8s.io/v1** API version. +- The **CSINode** object of **storage.k8s.io/v1beta1** has been deprecated and will be removed in later versions. + +Other Deprecations and Removals +------------------------------- + +- **OutOfDisk node condition** is removed in favor of **DiskPressure**. +- The **scheduler.alpha.kubernetes.io/critical-pod** annotation is removed in favor of **priorityClassName**. +- **beta.kubernetes.io/os** and **beta.kubernetes.io/arch** have been deprecated in v1.14 and will be removed in v1.18. +- Do not use **--node-labels** to set labels prefixed with **kubernetes.io** and **k8s.io**. The **kubernetes.io/availablezone** label in earlier versions is removed in v1.17 and changed to **failure-domain.beta.kubernetes.io/zone**. +- The **beta.kubernetes.io/instance-type** is deprecated in favor of **node.kubernetes.io/instance-type**. +- Remove the **{kubelet_root_dir}/plugins** path. +- Remove the built-in cluster roles **system:csi-external-provisioner** and **system:csi-external-attacher**. + +References +---------- + +For more details about the performance comparison and function evolution between Kubernetes 1.17 and other versions, see the following documents: + +- `Kubernetes v1.17.0 Release Notes `__ +- `Kubernetes v1.16.0 Release Notes `__ diff --git a/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.19_release_notes.rst b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.19_release_notes.rst new file mode 100644 index 0000000..9720144 --- /dev/null +++ b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.19_release_notes.rst @@ -0,0 +1,91 @@ +:original_name: cce_10_0470.html + +.. _cce_10_0470: + +CCE Kubernetes 1.19 Release Notes +================================= + +CCE has passed the Certified Kubernetes Conformance Program and is a certified Kubernetes offering. This section describes the updates in CCE Kubernetes 1.19. + +Resource Changes and Deprecations +--------------------------------- + +**Kubernetes 1.19 Release Notes** + +- vSphere in-tree volumes can be migrated to vSphere CSI drivers. The in-tree vSphere Volume plugin is no longer used and will be deleted in later versions. +- **apiextensions.k8s.io/v1beta1** has been deprecated. You are advised to use **apiextensions.k8s.io/v1**. +- **apiregistration.k8s.io/v1beta1** has been deprecated. You are advised to use **apiregistration.k8s.io/v1**. +- **authentication.k8s.io/v1beta1** and **authorization.k8s.io/v1beta1** have been deprecated and will be removed from Kubernetes 1.22. You are advised to use **authentication.k8s.io/v1** and **authorization.k8s.io/v1**. +- **autoscaling/v2beta1** has been deprecated. You are advised to use **autoscaling/v2beta2**. +- **coordination.k8s.io/v1beta1** has been deprecated in Kubernetes 1.19 and will be removed from version 1.22. You are advised to use **coordination.k8s.io/v1**. +- kube-apiserver: The **componentstatus** API has been deprecated. +- kubeadm: The **kubeadm config view** command has been deprecated and will be deleted in later versions. Use **kubectl get cm -o yaml -n kube-system kubeadm-config** to directly obtain the kubeadm configuration. +- kubeadm: The **kubeadm alpha kubelet config enable-dynamic** command has been deprecated. +- kubeadm: The **--use-api** flag in the **kubeadm alpha certs renew** command has been deprecated. +- Kubernetes no longer supports **hyperkube** image creation. +- The **--export** flag is removed from the **kubectl get** command. +- The alpha feature **ResourceLimitsPriorityFunction** has been deleted. +- **storage.k8s.io/v1beta1** has been deprecated. You are advised to use **storage.k8s.io/v1**. + +**Kubernetes 1.18 Release Notes** + +- kube-apiserver + + - All resources in the **apps/v1beta1** and **apps/v1beta2** API versions are no longer served. You can use the **apps/v1** API version. + - DaemonSets, Deployments, and ReplicaSets in the **extensions/v1beta1** API version are no longer served. You can use the **apps/v1** API version. + - NetworkPolicies in the **extensions/v1beta1** API version are no longer served. You can use the **networking.k8s.io/v1** API version. + - PodSecurityPolicies in the **extensions/v1beta1** API version are no longer served. Migrate to use the **policy/v1beta1** API version. + +- kubelet + + - **--redirect-container-streaming** is not recommended and will be deprecated in v1.20. + - The resource measurement endpoint **/metrics/resource/v1alpha1** and all measurement standards under this endpoint have been deprecated. Use the measurement standards under the endpoint **/metrics/resource** instead: + + - scrape_error --> scrape_error + - node_cpu_usage_seconds_total --> node_cpu_usage_seconds + - node_memory_working_set_bytes --> node_memory_working_set_bytes + - container_cpu_usage_seconds_total --> container_cpu_usage_seconds + - container_memory_working_set_bytes --> container_memory_working_set_bytes + - scrape_error --> scrape_error + + - In future releases, kubelet will no longer create the target directory **CSI NodePublishVolume** according to the CSI specifications. You may need to update the CSI driver accordingly to correctly create and process the target path. + +- kube-proxy + + - You are not advised to use the **--healthz-port** and **--metrics-port** flags. Use **--healthz-bind-address** and **--metrics-bind-address** instead. + - The **EndpointSliceProxying** function option is added to control the use of EndpointSlices in kube-proxy. This function is disabled by default. + +- kubeadm + + - The **--kubelet-version** flag of **kubeadm upgrade node** has been deprecated and will be deleted in later versions. + - The **--use-api** flag in the **kubeadm alpha certs renew** command has been deprecated. + - kube-dns has been deprecated and will no longer be supported in future versions. + - The ClusterStatus structure in the kubeadm-config ConfigMap has been deprecated and will be deleted in later versions. + +- kubectl + + - You are not advised to use boolean and unset values for **--dry-run**. **server|client|none** is used in the new version. + - **--server-dry-run** has been deprecated for **kubectl apply** and replaced by **--dry-run=server**. + +- add-ons + +The cluster-monitoring add-on is deleted. + +- kube-scheduler + + - The **scheduling_duration_seconds** metric has been deprecated. + - The **scheduling_algorithm_predicate_evaluation_seconds** and **scheduling_algorithm_priority_evaluation_seconds counters** metrics are no longer used and are replaced by **framework_extension_point_duration_seconds[extension_point="Filter"]** and **framework_extension_point_duration_seconds[extension_point="Score"]**. + - The scheduler policy AlwaysCheckAllPredictes has been deprecated. + +- Other changes + + - The k8s.io/node-api component is no longer updated. Instead, you can use the **RuntimeClass** type in **k8s.io/api** and the generated clients in **k8s.io/client-go**. + - The **client** label has been deleted from **apiserver_request_total**. + +References +---------- + +For more details about the performance comparison and function evolution between Kubernetes 1.19 and other versions, see the following documents: + +- `Kubernetes v1.19.0 Release Notes `__ +- `Kubernetes v1.18.0 Release Notes `__ diff --git a/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.21_release_notes.rst b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.21_release_notes.rst new file mode 100644 index 0000000..b646a8e --- /dev/null +++ b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.21_release_notes.rst @@ -0,0 +1,36 @@ +:original_name: cce_10_0469.html + +.. _cce_10_0469: + +CCE Kubernetes 1.21 Release Notes +================================= + +CCE has passed the Certified Kubernetes Conformance Program and is a certified Kubernetes offering. This section describes the updates in CCE Kubernetes 1.21. + +Resource Changes and Deprecations +--------------------------------- + +**Kubernetes 1.21 Release Notes** + +- CronJob is now in the stable state, and the version number changes to batch/v1. +- The immutable Secret and ConfigMap have now been upgraded to the stable state. A new immutable field is added to these objects to reject changes. The rejection protects clusters from accidental updates that may cause application outages. As these resources are immutable, kubelet does not monitor or poll for changes. This reduces the load of kube-apiserver and improves scalability and performance of your clusters. For more information, see `Immutable ConfigMaps `__. +- Graceful node shutdown has been upgraded to the test state. With this update, kubelet can detect that a node is shut down and gracefully terminate the pods on the node. Prior to this update, when the node was shut down, its pod did not follow the expected termination lifecycle, which caused workload problems. Now kubelet can use systemd to detect the systems that are about to be shut down and notify the running pods to terminate them gracefully. +- For a pod with multiple containers, you can use **kubectl.kubernetes.io/** to pre-select containers. +- PodSecurityPolicy is deprecated. For details, see https://kubernetes.io/blog/2021/04/06/podsecuritypolicy-deprecation-past-present-and-future/. +- The `BoundServiceAccountTokenVolume `__ feature has entered the beta test. This feature improves the token security of the service account and changes the method of mounting tokens to pods. Kubernetes clusters of v1.21 and later enable this approach by default. + +**Kubernetes 1.20 Release Notes** + +- The API priority and fairness have reached the test state and are enabled by default. This allows kube-apiserver to classify incoming requests by priority. For more information, see `API Priority and Fairness `__. +- The bug of **exec probe timeouts** is fixed. Before this bug is fixed, the exec probe does not consider the **timeoutSeconds** field. Instead, the probe will run indefinitely, even beyond its configured deadline. It will stop until the result is returned. Now, if no value is specified, the default value is used, that is, one second. If the detection time exceeds one second, the application health check may fail. Update the **timeoutSeconds** field for the applications that use this feature during the upgrade. The repair provided by the newly introduced ExecProbeTimeout feature gating enables the cluster operator to restore the previous behavior, but this behavior will be locked and removed in later versions. +- RuntimeClass enters the stable state. RuntimeClass provides a mechanism to support multiple runtimes in a cluster and expose information about the container runtime to the control plane. +- kubectl debugging has reached the test state. kubectl debugging provides support for common debugging workflows. +- Dockershim was marked as deprecated in Kubernetes 1.20. Currently, you can continue to use Docker in the cluster. This change is irrelevant to the container image used by clusters. You can still use Docker to build your images. For more information, see `Dockershim Deprecation FAQ `__. + +References +---------- + +For more details about the performance comparison and function evolution between Kubernetes 1.21 and other versions, see the following documents: + +- `Kubernetes v1.21 Release Notes `__ +- `Kubernetes v1.20 Release Notes `__ diff --git a/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.23_release_notes.rst b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.23_release_notes.rst new file mode 100644 index 0000000..e476d02 --- /dev/null +++ b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.23_release_notes.rst @@ -0,0 +1,39 @@ +:original_name: cce_10_0468.html + +.. _cce_10_0468: + +CCE Kubernetes 1.23 Release Notes +================================= + +CCE has passed the Certified Kubernetes Conformance Program and is a certified Kubernetes offering. This section describes the updates in CCE Kubernetes 1.23. + +Resource Changes and Deprecations +--------------------------------- + +**Changes in CCE 1.23** + +- The web-terminal add-on is no longer supported. Use CloudShell or kubectl instead. + +**Kubernetes 1.23 Release Notes** + +- FlexVolume is deprecated. Use CSI. +- HorizontalPodAutoscaler v2 is promoted to GA, and HorizontalPodAutoscaler API v2 is gradually stable in version 1.23. The HorizontalPodAutoscaler v2beta2 API is not recommended. Use the v2 API. +- `PodSecurity `__ moves to beta, replacing the deprecated PodSecurityPolicy. PodSecurity is an admission controller that enforces pod security standards on pods in the namespace based on specific namespace labels that set the enforcement level. PodSecurity is enabled by default in version 1.23. + +**Kubernetes 1.22 Release Notes** + +- Ingresses no longer support networking.k8s.io/v1beta1 and extensions/v1beta1 APIs. If you use the API of an earlier version to manage ingresses, an application cannot be exposed to external services. Use networking.k8s.io/v1. +- CustomResourceDefinitions no longer support the apiextensions.k8s.io/v1beta1 API. If you use the API of an earlier version to create a CRD, the creation will fail, which affects the controller that reconciles this CRD. Use apiextensions.k8s.io/v1. +- ClusterRoles, ClusterRoleBindings, Roles, and RoleBindings no longer support the rbac.authorization.k8s.io/v1beta1 API. If you use the API of an earlier version to manage RBAC resources, application permissions control is affected and even cannot work in the cluster. Use rbac.authorization.k8s.io/v1. +- The Kubernetes release cycle is changed from four releases a year to three releases a year. +- StatefulSets support **minReadySeconds**. +- During scale-in, pods are randomly selected and deleted based on the pod UID by default (LogarithmicScaleDown). This feature enhances the randomness of the pods to be deleted and alleviates the problems caused by pod topology spread constraints. For more information, see `KEP-2185 `__ and `issue 96748 `__. +- The `BoundServiceAccountTokenVolume `__ feature is stable. This feature improves the token security of the service account and changes the method of mounting tokens to pods. Kubernetes clusters of v1.21 and later enable this approach by default. + +References +---------- + +For more details about the performance comparison and function evolution between Kubernetes 1.23 and other versions, see the following documents: + +- `Kubernetes v1.23 Release Notes `__ +- `Kubernetes v1.22 Release Notes `__ diff --git a/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.25_release_notes.rst b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.25_release_notes.rst new file mode 100644 index 0000000..28b686b --- /dev/null +++ b/umn/source/clusters/cluster_overview/release_notes/cce_kubernetes_1.25_release_notes.rst @@ -0,0 +1,38 @@ +:original_name: cce_10_0467.html + +.. _cce_10_0467: + +CCE Kubernetes 1.25 Release Notes +================================= + +CCE has passed the Certified Kubernetes Conformance Program and is a certified Kubernetes offering. This section describes the updates in CCE Kubernetes 1.25. + +Resource Changes and Deprecations +--------------------------------- + +**Kubernetes 1.25 Release Notes** + +- PodSecurityPolicy is replaced by Pod Security Admission. For details about the migration, see `Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission Controller `__. +- SeccompDefault is in Beta. To enable this feature, you need to add the startup parameter **--seccomp-default=true** to kubelet. In this way, seccomp is set to **RuntimeDefault** by default, improving the system security. Clusters of v1.25 no longer use **seccomp.security.alpha.kubernetes.io/pod** and **container.seccomp.security.alpha.kubernetes.io/annotation** to use seccomp. Replace them with the **securityContext.seccompProfile** field in the pod or container. For details, see `Configure a Security Context for a Pod or Container `__. + + .. note:: + + After the feature is enabled, the system calls required by the application may be restricted by the runtime. Ensure that the debugging is performed in the test environment and the application is not affected. + +- EndPort in Network Policy is stable. This feature is incorporated in version 1.21. EndPort is added to NetworkPolicy for you to specify a port range. +- Since clusters of v1.25, Kubernetes does not support certificate authentication generated using the SHA1WithRSA or ECDSAWithSHA1 algorithm. You are advised to use the SHA256 algorithm. + +**Kubernetes 1.24 Release Notes** + +- Beta APIs are disabled by default. When some long-term beta APIs are removed from Kubernetes, 90% cluster administrators do not care about the beta APIs. Beta features are not recommended in the production environment. However, due to the default enabling policy, these APIs are enabled in the production environment, incurring risks. Therefore, in v1.24 and later versions, beta APIs are disabled by default except for the enabled beta APIs. +- The LegacyServiceAccountTokenNoAutoGeneration feature is in beta state. By default, this feature is enabled and no more secret token will be automatically generated for the service account. If you want to use a token that never expires, you need to create a secret and mount it. For details, see `Service account token secrets `__. +- **service.alpha.kubernetes.io/tolerate-unready-endpoints** is replaced by **Service.spec.publishNotReadyAddresses**. +- The **Service.Spec.LoadBalancerIP** tag is deprecated and may be removed in later versions. Use a customized annotation. + +References +---------- + +For more details about the performance comparison and function evolution between Kubernetes 1.25 and other versions, see the following documents: + +- `Kubernetes v1.25 Release Notes `__ +- `Kubernetes v1.24 Release Notes `__ diff --git a/umn/source/clusters/cluster_overview/release_notes/index.rst b/umn/source/clusters/cluster_overview/release_notes/index.rst new file mode 100644 index 0000000..b1553ac --- /dev/null +++ b/umn/source/clusters/cluster_overview/release_notes/index.rst @@ -0,0 +1,22 @@ +:original_name: cce_10_0068.html + +.. _cce_10_0068: + +Release Notes +============= + +- :ref:`CCE Kubernetes 1.25 Release Notes ` +- :ref:`CCE Kubernetes 1.23 Release Notes ` +- :ref:`CCE Kubernetes 1.21 Release Notes ` +- :ref:`CCE Kubernetes 1.19 Release Notes ` +- :ref:`CCE Kubernetes 1.17 Release Notes ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + cce_kubernetes_1.25_release_notes + cce_kubernetes_1.23_release_notes + cce_kubernetes_1.21_release_notes + cce_kubernetes_1.19_release_notes + cce_kubernetes_1.17_release_notes diff --git a/umn/source/clusters/cluster_parameters/index.rst b/umn/source/clusters/cluster_parameters/index.rst deleted file mode 100644 index db5a457..0000000 --- a/umn/source/clusters/cluster_parameters/index.rst +++ /dev/null @@ -1,14 +0,0 @@ -:original_name: cce_01_0347.html - -.. _cce_01_0347: - -Cluster Parameters -================== - -- :ref:`Maximum Number of Pods That Can Be Created on a Node ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - maximum_number_of_pods_that_can_be_created_on_a_node diff --git a/umn/source/clusters/controlling_cluster_permissions.rst b/umn/source/clusters/controlling_cluster_permissions.rst deleted file mode 100644 index 036e0e0..0000000 --- a/umn/source/clusters/controlling_cluster_permissions.rst +++ /dev/null @@ -1,90 +0,0 @@ -:original_name: cce_01_0085.html - -.. _cce_01_0085: - -Controlling Cluster Permissions -=============================== - -Scenario --------- - -This section describes how to control permissions on resources in a cluster, for example, allow user A to read and write application data in a namespace, and user B to only read resource data in a cluster. - -Procedure ---------- - -#. If you need to perform permission control on the cluster, select **Enhanced authentication** for **Authentication Mode** during cluster creation, upload your own **CA certificate**, **client certificate**, and **client certificate private key** (for details about how to create a certificate, see `Certificates `__), and select **I have confirmed that the uploaded certificates are valid**. For details, see :ref:`Table 1 `. - - .. caution:: - - - Upload a file **smaller than 1 MB**. The CA certificate and client certificate can be in **.crt** or **.cer** format. The private key of the client certificate can only be uploaded **unencrypted**. - - The validity period of the client certificate must be longer than five years. - - The uploaded CA certificate is used for both the authentication proxy and the kube-apiserver aggregation layer configuration. **If the certificate is invalid, the cluster cannot be created**. - -#. Create a role using kubectl. - - The following example shows how to create a **role** and allow the role to read all pods in the default namespace. For details about the parameters, see the `official Kubernetes documentation `__. - - .. code-block:: - - kind: Role - apiVersion: rbac.authorization.k8s.io/v1 - metadata: - namespace: default - name: pod-reader - rules: - - apiGroups: [""] - resources: ["pods"] - verbs: ["get", "watch", "list"] - -#. Bind the role to a user by using kubectl. - - In the following example, the **RoleBinding** assigns the role of **pod-reader** in the default namespace to user **jane**. This policy allows user **jane** to read all pods in the default namespace. For details about the parameters, see the `official Kubernetes documentation `__. - - .. code-block:: - - kind: RoleBinding - apiVersion: rbac.authorization.k8s.io/v1 - metadata: - name: read-pods - namespace: default - subjects: - - kind: User - name: jane #User name - apiGroup: rbac.authorization.k8s.io - roleRef: - kind: Role - name: pod-reader #Name of the role that is created - apiGroup: rbac.authorization.k8s.io - -#. After a role is created and bound to a user, call a Kubernetes API by initiating an API request message where headers carry user information and the certificate uploaded during cluster creation. For example, to call the pod query API, run the following command: - - **curl -k -H "X-Remote-User: jane" --cacert /root/tls-ca.crt --key /root/tls.key --cert /root/tls.crt https://**\ *192.168.23.5:5443*\ **/api/v1/namespaces/default/pods** - - If **200** is returned, user **jane** is authorized to read pods in the cluster's default namespace. If **403** is returned, user **jane** is not authorized to read pods in the cluster's default namespace. - - .. note:: - - To prevent the command execution failure, upload the certificate to the **/root** directory in advance. - - The parameter descriptions are as follows: - - - **X-Remote-User: jane**: The request header is fixed at **X-Remote-User**, and **jane** is the username. - - - **tls-ca.crt**: CA root certificate uploaded during cluster creation. - - - **tls.crt**: client certificate that matches the CA root certificate uploaded during cluster creation. - - - **tls.key**: client key corresponding to the CA root certificate uploaded during cluster creation. - - - **192.168.23.5:5443**: address for connecting to the cluster. To obtain the address, perform the following steps: - - Log in to the CCE console. In the navigation pane, choose **Resource Management > Clusters**. Click the name of the cluster to be connected and obtain the IP address and port number from **Internal API Server Address** on the cluster details page. - - - .. figure:: /_static/images/en-us_image_0000001144208440.png - :alt: **Figure 1** Obtaining the access address - - **Figure 1** Obtaining the access address - - In addition, the **X-Remote-Group** header field, that is, the user group name, is supported. During role binding, a role can be bound to a group and carry user group information when you access the cluster. diff --git a/umn/source/clusters/creating_a_cce_cluster.rst b/umn/source/clusters/creating_a_cce_cluster.rst index 337f097..34da41e 100644 --- a/umn/source/clusters/creating_a_cce_cluster.rst +++ b/umn/source/clusters/creating_a_cce_cluster.rst @@ -1,13 +1,13 @@ -:original_name: cce_01_0028.html +:original_name: cce_10_0028.html -.. _cce_01_0028: +.. _cce_10_0028: Creating a CCE Cluster ====================== On the CCE console, you can easily create Kubernetes clusters. Kubernetes can manage container clusters at scale. A cluster manages a group of node resources. -In CCE, you can create a CCE cluster to manage VMs as nodes. By using high-performance network models, hybrid clusters provide a multi-scenario, secure, and stable runtime environment for containers. +In CCE, you can create a CCE cluster to manage VMs. By using high-performance network models, hybrid clusters provide a multi-scenario, secure, and stable runtime environment for containers. Notes and Constraints --------------------- @@ -16,377 +16,86 @@ Notes and Constraints - You can create a maximum of 50 clusters in a single region. - After a cluster is created, the following items cannot be changed: - - Number of master nodes in the cluster. - - AZ of a master node. - - Network configuration of the cluster, such as the VPC, subnet, container CIDR block, Service CIDR block, and kube-proxy (forwarding) settings. - - Network model. For example, change the **tunnel network** to the **VPC network**. + - Cluster type + - Number of master nodes in the cluster + - AZ of a master node + - Network configuration of the cluster, such as the VPC, subnet, container CIDR block, Service CIDR block, and kube-proxy (forwarding) settings + - Network model. For example, change **Tunnel network** to **VPC network**. Procedure --------- -#. Log in to the CCE console. On the **Dashboard** page, click **Create Cluster**. Alternatively, choose **Resource Management** > **Clusters** in the navigation pane and click **Create** next to **CCE Cluster**. +#. Log in to the CCE console. Choose **Clusters**. On the displayed page, click **Create** next to **CCE cluster**. -#. Set cluster parameters by referring to :ref:`Table 1 `. Pay attention to the parameters marked with an asterisk (*). +#. Set cluster parameters. - .. _cce_01_0028__table8638121213265: + **Basic Settings** - .. table:: **Table 1** Parameters for creating a cluster + - **Cluster Name** - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==========================================================================================================================================================================================================================================================================================================================================================================================================================+ - | Region | Select a region near you to ensure the lowest latency possible. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Cluster Name | Name of the new cluster, which cannot be changed after the cluster is created. | - | | | - | | A cluster name contains 4 to 128 characters starting with a letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Version | Kubernetes community baseline version. The latest version is recommended. | - | | | - | | If a **Beta** version is available, you can use it for trial. However, it is not recommended for commercial use. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Management Scale | Maximum number of worker nodes that can be managed by the master nodes of the current cluster. You can select 50 nodes, 200 nodes, or 1,000 nodes for your cluster, or 2,000 nodes if you are buying a cluster of v1.15.11 or later. | - | | | - | | If you select **1000 nodes**, the master nodes of the cluster can manage a maximum of 1000 worker nodes. The configuration fee varies depending on the specifications of master nodes for different management scales. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Number of master nodes | **3**: Three master nodes will be created to make the cluster highly available. If a master node is faulty, the cluster can still be available without affecting service functions. Click **Change**. In the dialog box displayed, you can configure the following parameters: | - | | | - | | **Disaster recovery level** | - | | | - | | - **AZ**: Master nodes are deployed in different AZs for disaster recovery. | - | | - **Fault domain**: Master nodes are deployed in different failure domains in the same AZ for disaster recovery. This option is displayed only when the environment supports failure domains. | - | | - **Host computer**: Master nodes are deployed on different hosts in the same AZ for disaster recovery. | - | | - **Customize**: You can select different locations to deploy different master nodes. In the fault domain mode, master nodes must be in the same AZ. | - | | | - | | **1**: Only one master node is created in the cluster, which cannot ensure SLA for the cluster. Single-master clusters (non-HA clusters) are not recommended for commercial scenarios. Click **Change**. In the **AZ Settings** dialog box, select an AZ for the master node. | - | | | - | | .. note:: | - | | | - | | - You are advised to create multiple master nodes to improve the cluster DR capability in commercial scenarios. | - | | - The multi-master mode cannot be changed after the cluster is created. A single-master cluster cannot be upgraded to a multi-master cluster. For a single-master cluster, if a master node is faulty, services will be affected. | - | | - To ensure reliability, the multi-master mode is enabled by default for a cluster with 1,000 or more nodes. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*VPC | VPC where the cluster is located. The value cannot be changed after the cluster is created. | - | | | - | | A VPC provides a secure and logically isolated network environment. | - | | | - | | If no VPC is available, click **Create a VPC** to create a VPC. After the VPC is created, click the refresh icon. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Subnet | Subnet where the node VM runs. The value cannot be changed after the cluster is created. | - | | | - | | A subnet provides dedicated network resources that are logically isolated from other networks for network security. | - | | | - | | If no subnet is available, click **Create Subnet** to create a subnet. After the subnet is created, click the refresh icon. For details about the relationship between VPCs, subnets, and clusters, see :ref:`Cluster Overview `. | - | | | - | | During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name. | - | | | - | | **The selected subnet cannot be changed after the cluster is created.** | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Network Model | After a cluster is created, the network model cannot be changed. Exercise caution when selecting a network model. For details about how to select a network model, see :ref:`Overview `. | - | | | - | | **VPC network** | - | | | - | | In this network model, each node occupies one VPC route. The number of VPC routes supported by the current region and the number of container IP addresses that can be allocated to each node (that is, the maximum number of pods that can be created) are displayed on the console. | - | | | - | | - The container network uses VPC routes to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. However, each node occupies one VPC route, and the maximum number of nodes allowed in a cluster depends on the VPC route quota. | - | | - Each node is assigned a CIDR block of a fixed size. VPC networks are free from packet encapsulation overheads and outperform container tunnel networks. In addition, as VPC routing includes routes to node IP addresses and the container CIDR block, container pods in the cluster can be directly accessed from outside the cluster. | - | | | - | | .. note:: | - | | | - | | - In the VPC network model, extended CIDR blocks and network policies are not supported. | - | | - When creating multiple clusters using the VPC network model in one VPC, select a CIDR block for each cluster that does not overlap with the VPC address or other container CIDR blocks. | - | | | - | | **Tunnel network** | - | | | - | | - The container network is an overlay tunnel network on top of a VPC network and uses the VXLAN technology. This network model is applicable when there is no high requirements on performance. | - | | - VXLAN encapsulates Ethernet packets as UDP packets for tunnel transmission. Though at some cost of performance, the tunnel encapsulation enables higher interoperability and compatibility with advanced features (such as network policy-based isolation), meeting the requirements of most applications. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Network Segment | An IP address range that can be allocated to container pods. After the cluster is created, the value cannot be changed. | - | | | - | | - If **Automatically select** is deselected, enter a CIDR block manually. If the CIDR block you specify conflicts with a subnet CIDR block, the system prompts you to select another CIDR block. The recommended CIDR blocks are 10.0.0.0/8-18, 172.16.0.0/16-18, and 192.168.0.0/16-18. | - | | | - | | **If different clusters share a container CIDR block, an IP address conflict will occur and access to applications may fail.** | - | | | - | | - If **Automatically select** is selected, the system automatically assigns a CIDR block that does not conflict with any subnet CIDR block. | - | | | - | | The mask of the container CIDR block must be appropriate. It determines the number of available nodes in a cluster. A too small mask value will cause the cluster to soon fall short of nodes. After the mask is set, the estimated maximum number of containers supported by the current CIDR block will be displayed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Service Network Segment | An IP address range that can be allocated to Kubernetes Services. After the cluster is created, the value cannot be changed. The Service CIDR block cannot conflict with the created route. If they conflict, select another CIDR block. | - | | | - | | - **Default**: The default CIDR block 10.247.0.0/16 will be used. | - | | - **Custom**: Manually set a CIDR block and mask based on service requirements. The mask determines the maximum number of Service IP addresses available in the cluster. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Authorization Mode | **RBAC** is selected by default and cannot be deselected. | - | | | - | | After RBAC is enabled, IAM users access resources in the cluster according to fine-grained permissions policies. For details, see :ref:`Namespace Permissions (Kubernetes RBAC-based) `. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Authentication Mode | The authentication mechanism controls user permission on resources in a cluster. | - | | | - | | The X.509-based authentication mode is enabled by default. X.509 is a commonly used certificate format. | - | | | - | | If you want to perform permission control on the cluster, select **Enhanced authentication**. The cluster will identify users based on the header of the request for authentication. | - | | | - | | You need to upload your own **CA certificate**, **client certificate**, and **client certificate private key** (for details about how to create a certificate, see `Certificates `__), and select **I have confirmed that the uploaded certificates are valid**. | - | | | - | | .. caution:: | - | | | - | | CAUTION: | - | | | - | | - Upload a file **smaller than 1 MB**. The CA certificate and client certificate can be in **.crt** or **.cer** format. The private key of the client certificate can only be uploaded **unencrypted**. | - | | - The validity period of the client certificate must be longer than five years. | - | | - The uploaded CA certificate is used for both the authentication proxy and the kube-apiserver aggregation layer configuration. **If the certificate is invalid, the cluster cannot be created**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Description | Optional. Enter the description of the new container cluster. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Advanced Settings | Click **Advanced Settings** to expand the details page. The following functions are supported (unsupported functions in current AZs are hidden): | - | | | - | | **Service Forwarding Mode** | - | | | - | | - **iptables**: Traditional kube-proxy uses iptables rules to implement Service load balancing. In this mode, too many iptables rules will be generated when many Services are deployed. In addition, non-incremental updates will cause a latency and even obvious performance issues in the case of heavy service traffic. | - | | | - | | - **ipvs**: optimized kube-proxy mode to achieve higher throughput and faster speed, ideal for large-sized clusters. This mode supports incremental updates and can keep connections uninterrupted during Service updates. | - | | | - | | In this mode, when the ingress and Service use the same ELB instance, the ingress cannot be accessed from the nodes and containers in the cluster. | - | | | - | | .. note:: | - | | | - | | - ipvs provides better scalability and performance for large clusters. | - | | - Compared with iptables, ipvs supports more complex load balancing algorithms such as least load first (LLF) and weighted least connections (WLC). | - | | - ipvs supports server health checking and connection retries. | - | | | - | | **CPU Policy** | - | | | - | | This parameter is displayed only for clusters of v1.13.10-r0 and later. | - | | | - | | - **On**: Exclusive CPU cores can be allocated to workload pods. Select **On** if your workload is sensitive to latency in CPU cache and scheduling. | - | | - **Off**: Exclusive CPU cores will not be allocated to workload pods. Select **Off** if you want a large pool of shareable CPU cores. | - | | | - | | For details about CPU management policies, see `Feature Highlight: CPU Manager `__. | - | | | - | | After **CPU Policy** is enabled, workloads cannot be started or created on nodes after the node specifications are changed. | - | | | - | | **Open EIP** | - | | | - | | An independent public IP address that is reachable from public networks. Select an EIP that has not been bound to any node. A cluster's EIP is preset in the cluster's certificate. Do no delete the EIP after the cluster has been created. Otherwise, two-way authentication will fail. | - | | | - | | - **Do not configure**: The cluster's master node will not have an EIP. | - | | - **Configure now**: If no EIP is available for selection, create one. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Cluster Version**: Select the Kubernetes version used by the cluster. -#. Click **Next: Create Node** and set the following parameters. + - **Cluster Scale**: maximum number of nodes that can be managed by the cluster. - - **Create Node** + - **HA**: distribution mode of master nodes. By default, master nodes are randomly distributed in different AZs to improve DR capabilities. - - **Create now**: Create a node when creating a cluster. Currently, only VM nodes are supported. If a node fails to be created, the cluster will be rolled back. - - **Create later**: No node will be created. Only an empty cluster will be created. + You can also expand advanced settings and customize the master node distribution mode. The following two modes are supported: - - **Current Region**: geographic location of the nodes to be created. + - **Random**: Master nodes are created in different AZs for DR. + - **Custom**: You can determine the location of each master node. - - **AZ**: Set this parameter based on the site requirements. An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. + - **Host**: Master nodes are created on different hosts in the same AZ. + - **Custom**: You can determine the location of each master node. - You are advised to deploy worker nodes in different AZs after the cluster is created to make your workloads more reliable. When creating a cluster, you can deploy nodes only in one AZ. + **Network Settings** - - **Node Type** + The cluster network settings cover nodes, containers, and Services. For details about the cluster networking and container network models, see :ref:`Overview `. - - **VM node**: A VM node will be created in the cluster. + - **Network Model**: CCE clusters support **VPC network** and **tunnel network** models. For details, see :ref:`VPC Network ` and :ref:`Container Tunnel Network `. + - **VPC**: Select the VPC to which the cluster belongs. If no VPC is available, click **Create VPC** to create one. The VPC cannot be changed after creation. + - **Master Node Subnet**: Select the subnet where the master node is deployed. If no subnet is available, click **Create Subnet** to create one. The subnet cannot be changed after creation. + - **Container CIDR Block**: Set the CIDR block used by containers. + - **Service CIDR Block**: CIDR block for Services used by containers in the same cluster to access each other. The value determines the maximum number of Services you can create. The value cannot be changed after creation. - - **Node Name**: Enter a node name. A node name contains 1 to 56 characters starting with a lowercase letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed. + **Advanced Settings** - - **Specifications**: Select node specifications that best fit your business needs. + - **Request Forwarding**: The IPVS and iptables modes are supported. For details, see :ref:`Comparing iptables and IPVS `. + - **CPU Manager**: For details, see :ref:`Binding CPU Cores `. + - **Certificate Authentication**: - - **General-purpose**: provides a balance of computing, memory, and network resources. It is a good choice for many applications, such as web servers, workload development, workload testing, and small-scale databases. - - **Memory-optimized**: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive. - - **GPU-accelerated**: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. **GPU-accelerated nodes can be created only in clusters of v1.11 or later**. GPU-accelerated nodes are available only in certain regions. - - **General computing-plus**: provides stable performance and exclusive resources to enterprise-class workloads with high and stable computing performance. - - **Disk-intensive**: supports :ref:`local disk storage ` and provides high network performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads. + - **Default**: The X509-based authentication mode is enabled by default. X509 is a commonly used certificate format. - To ensure node stability, CCE automatically reserves some resources to run necessary system components. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. + - **Custom:** The cluster can identify users based on the header in the request body for authentication. - - **OS**: Select an OS for the node to be created. + You need to upload your **CA root certificate**, **client certificate**, and **private key** of the client certificate. - Reinstalling the OS or modifying OS configurations could make the node unavailable. Exercise caution when performing these operations. + .. caution:: - - **System Disk**: Set the system disk space of the worker node. The value ranges from 40GB to 1024 GB. The default value is 40GB. + - Upload a file **smaller than 1 MB**. The CA certificate and client certificate can be in **.crt** or **.cer** format. The private key of the client certificate can only be uploaded **unencrypted**. + - The validity period of the client certificate must be longer than five years. + - The uploaded CA certificate is used for both the authentication proxy and the kube-apiserver aggregation layer configuration. **If the certificate is invalid, the cluster cannot be created**. + - Starting from v1.25, Kubernetes no longer supports certificate authentication generated using the SHA1WithRSA or ECDSAWithSHA1 algorithm. You are advised to use the SHA256 algorithm. - By default, system disks support Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) EVS disks. + - **Description**: The value can contain a maximum of 200 English characters. - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. **This function is available only in certain regions.** +#. Click **Next: Add-on Configuration**. - - **Encryption** is not selected by default. - - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. + By default, :ref:`cordens ` and :ref:`everest ` add-ons are installed. - - .. _cce_01_0028__li12223421320: + **Service log** - **Data Disk**: Set the data disk space of the worker node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. The EVS disk types provided for the data disk are the same as those for the system disk. + - **ICAgent**: - .. caution:: + A log collector provided by Application Operations Management (AOM), reporting logs to AOM and Log Tank Service (LTS) according to the log collection rules you configured. - If the data disk is uninstalled or damaged, the Docker service becomes abnormal and the node becomes unavailable. You are advised not to delete the data disk. + You can collect stdout logs as required. - - **LVM**: If this option is selected, CCE data disks are managed by the Logical Volume Manager (LVM). On this condition, you can adjust the disk space allocation for different resources. This option is selected for the first disk by default and cannot be unselected. You can choose to enable or disable LVM for new data disks. +#. After the parameters are specified, click **Next: Confirm**. The cluster resource list is displayed. Confirm the information and click **Submit**. - - This option is selected by default, indicating that LVM management is enabled. - - You can deselect the check box to disable LVM management. - - .. caution:: - - - Disk space of the data disks managed by LVM will be allocated according to the ratio you set. - - When creating a node in a cluster of v1.13.10 or later, if LVM is not selected for a data disk, follow instructions in :ref:`Adding a Second Data Disk to a Node in a CCE Cluster ` to fill in the pre-installation script and format the data disk. Otherwise, the data disk will still be managed by LVM. - - When creating a node in a cluster earlier than v1.13.10, you must format the data disks that are not managed by LVM. Otherwise, either these data disks or the first data disk will be managed by LVM. - - - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. - - **This function is supported only for clusters of v1.13.10 or later in certain regions,** and is not displayed for clusters of v1.13.10 or earlier. - - - **Encryption** is not selected by default. - - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. - - - **Add Data Disk**: Currently, a maximum of two data disks can be attached to a node. After the node is created, you can go to the ECS console to attach more data disks. This function is available only to clusters of certain versions. - - - **Data disk space allocation**: Click |image1| to specify the resource ratio for **Kubernetes Space** and **User Space**. Disk space of the data disks managed by LVM will be allocated according to the ratio you set. This function is available only to clusters of certain versions. - - - **Kubernetes Space**: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs. - - The Docker space cannot be less than 10%, and the space size cannot be less than 60 GB. The kubelet space cannot be less than 10%. - - The Docker space size is determined by your service requirements. For details, see :ref:`Data Disk Space Allocation `. - - - **User Space**: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted. - - .. note:: - - Note that the mount path cannot be **/**, **/home/paas**, **/var/paas**, **/var/lib**, **/var/script**, **/var/log**, **/mnt/paas**, or **/opt/cloud**, and cannot conflict with the system directories (such as **bin**, **lib**, **home**, **root**, **boot**, **dev**, **etc**, **lost+found**, **mnt**, **proc**, **sbin**, **srv**, **tmp**, **var**, **media**, **opt**, **selinux**, **sys**, and **usr**). Otherwise, the system or node installation will fail. - - **If the cluster version is v1.13.10-r0 or later and the node specification is Disk-intensive, the following options are displayed for data disks:** - - - **EVS**: Parameters are the same as those when the node type is not Disk-intensive. For details, see :ref:`Data Disk ` above. - - - **Local disk**: Local disks may break down and do not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks. - - Local disk parameters are as follows: - - - **Disk Mode**: If the node type is **disk-intensive**, the supported disk mode is HDD. - - **Read/Write Mode**: When multiple local disks exist, you can set the read/write mode. The serial and sequential modes are supported. **Sequential** indicates that data is read and written in linear mode. When a disk is used up, the next disk is used. **Serial** indicates that data is read and written in striping mode, allowing multiple local disks to be read and written at the same time. - - **Kubernetes Space**: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs. - - **User Space**: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted. - - .. important:: - - - The ratio of disk space allocated to the Kubernetes space and user space must be equal to 100% in total. You can click |image2| to refresh the data after you have modified the ratio. - - By default, disks run in the direct-lvm mode. If data disks are removed, the loop-lvm mode will be used and this will impair system stability. - - - **VPC**: A VPC where the current cluster is located. This parameter cannot be changed and is displayed only for clusters of v1.13.10-r0 or later. - - - **Subnet**: A subnet improves network security by providing exclusive network resources that are isolated from other networks. You can select any subnet in the cluster VPC. Cluster nodes can belong to different subnets. - - During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name. - - - **EIP**: an independent public IP address. If the nodes to be created require public network access, select **Automatically assign** or **Use existing**. - - An EIP bound to the node allows public network access. EIP bandwidth can be modified at any time. An ECS without a bound EIP cannot access the Internet or be accessed by public networks. - - - **Do not use**: A node without an EIP cannot be accessed from public networks. It can be used only as a cloud server for deploying services or clusters on a private network. - - - **Automatically assign**: An EIP with specified configurations is automatically assigned to each node. If the number of EIPs is smaller than the number of nodes, the EIPs are randomly bound to the nodes. - - Configure the EIP specifications, billing factor, bandwidth type, and bandwidth size as required. When creating an ECS, ensure that the elastic IP address quota is sufficient. - - - **Use existing**: Existing EIPs are assigned to the nodes to be created. - - .. note:: - - By default, VPC's SNAT feature is disabled for CCE. If SNAT is enabled, you do not need to use EIPs to access public networks. For details about SNAT, see :ref:`Custom Policies `. - - - **Login Mode**: - - - **Key pair**: Select the key pair used to log in to the node. You can select a shared key. - - A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. - - .. important:: - - When creating a node using a key pair, IAM users can select only the key pairs created by their own, regardless of whether these users are in the same group. For example, user B cannot use the key pair created by user A to create a node, and the key pair is not displayed in the drop-down list on the CCE console. - - - **Advanced ECS Settings** (optional): Click |image3| to show advanced ECS settings. - - - **ECS Group**: An ECS group logically groups ECSs. The ECSs in the same ECS group comply with the same policy associated with the ECS group. - - - **Anti-affinity**: ECSs in an ECS group are deployed on different physical hosts to improve service reliability. - - Select an existing ECS group, or click **Create ECS Group** to create one. After the ECS group is created, click the refresh button. - - - **Resource Tags**: By adding tags to resources, you can classify resources. - - You can create predefined tags in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency. - - CCE will automatically create the "CCE-Dynamic-Provisioning-Node=node id" tag. A maximum of 5 tags can be added. - - - **Agency**: An agency is created by a tenant administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. To authorize an ECS or BMS to call cloud services, select **Cloud service** as the agency type, click **Select**, and then select **ECS BMS**. - - - **Pre-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. The script is usually used to format data disks. - - - **Post-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed after Kubernetes software is installed and will not affect the installation. The script is usually used to modify Docker parameters. - - - **Subnet IP Address**: Select **Automatically assign IP address** (recommended) or **Manually assigning IP addresses**. - - - **Advanced Kubernetes Settings**: (Optional) Click |image4| to show advanced cluster settings. - - - **Max Pods**: maximum number of pods that can be created on a node, including the system's default pods. If the cluster uses the **VPC network model**, the maximum value is determined by the number of IP addresses that can be allocated to containers on each node. - - This limit prevents the node from being overloaded by managing too many pods. For details, see :ref:`Maximum Number of Pods That Can Be Created on a Node `. - - - **Maximum Data Space per Container**: maximum data space that can be used by a container. The value ranges from 10 GB to 500 GB. If the value of this field is larger than the data disk space allocated to Docker resources, the latter will override the value specified here. Typically, 90% of the data disk space is allocated to Docker resources. This parameter is displayed only for clusters of v1.13.10-r0 and later. - - - **Nodes**: The value cannot exceed the management scale you select when configuring cluster parameters. Set this parameter based on service requirements and the remaining quota displayed on the page. Click |image5| to view the factors that affect the number of nodes to be added (depending on the factor with the minimum value). - -#. Click **Next: Install Add-on**, and select the add-ons to be installed in the **Install Add-on** step. - - System resource add-ons must be installed. Advanced functional add-ons are optional. - - You can also install all add-ons after the cluster is created. To do so, choose **Add-ons** in the navigation pane of the CCE console and select the add-on you will install. For details, see :ref:`Add-ons `. - -#. Click **Next: Confirm**. Read the **product instructions** and select **I am aware of the above limitations**. Confirm the configured parameters, specifications, and fees. - -#. Click **Submit**. - - It takes about 6 to 10 minutes to create a cluster. You can click **Back to Cluster List** to perform other operations on the cluster or click **Go to Cluster Events** to view the cluster details. If the cluster status is Available, the cluster is successfully created. + It takes about 6 to 10 minutes to create a cluster. You can click **Back to Cluster List** to perform other operations on the cluster or click **Go to Cluster Events** to view the cluster details. Related Operations ------------------ -- After creating a cluster, you can use the Kubernetes command line (CLI) tool kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. -- Add more nodes to the cluster. For details, see :ref:`Creating a Node `. -- Log in to a node. For details, see :ref:`Logging In to a Node `. - -- Create a namespace. You can create multiple namespaces in a cluster and organize resources in the cluster into different namespaces. These namespaces serve as logical groups and can be managed separately. For more information about how to create a namespace for a cluster, see :ref:`Namespaces `. -- Create a workload. Once the cluster is created, you can use an image to create an application that can be accessed from public networks. For details, see :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `. -- Click the cluster name to view cluster details. - - .. table:: **Table 2** Cluster details - - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Tab | Description | - +===================================+=======================================================================================================================================================================================================================================+ - | Cluster Details | View the details and operating status of the cluster. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Monitoring | You can view the CPU and memory allocation rates of all nodes in the cluster (that is, the maximum allocated amount), as well as the CPU usage, memory usage, and specifications of the master node(s). | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Events | - View cluster events on the **Events** tab page. | - | | - Set search criteria. For example, you can set the time segment or enter an event name to view corresponding events. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Auto Scaling | You can configure auto scaling to add or reduce worker nodes in a cluster to meet service requirements. For details, see :ref:`Setting Cluster Auto Scaling `. | - | | | - | | Clusters of v1.17 do not support auto scaling using AOM. You can use node pools for auto scaling. For details, see :ref:`Node Pool Overview `. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubectl | To access a Kubernetes cluster from a PC, you need to use the Kubernetes command line tool `kubectl `__. For details, see :ref:`Connecting to a Cluster Using kubectl `. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -.. |image1| image:: /_static/images/en-us_image_0273156799.png -.. |image2| image:: /_static/images/en-us_image_0220702939.png -.. |image3| image:: /_static/images/en-us_image_0183134608.png -.. |image4| image:: /_static/images/en-us_image_0183134479.png -.. |image5| image:: /_static/images/en-us_image_0250508826.png +- After creating a cluster, you can use the Kubernetes command line (CLI) tool kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +- Add nodes to the cluster. For details, see :ref:`Creating a Node `. diff --git a/umn/source/clusters/creating_a_cce_turbo_cluster.rst b/umn/source/clusters/creating_a_cce_turbo_cluster.rst index d73aeaf..79cd235 100644 --- a/umn/source/clusters/creating_a_cce_turbo_cluster.rst +++ b/umn/source/clusters/creating_a_cce_turbo_cluster.rst @@ -1,168 +1,107 @@ -:original_name: cce_01_0298.html +:original_name: cce_10_0298.html -.. _cce_01_0298: +.. _cce_10_0298: Creating a CCE Turbo Cluster ============================ CCE Turbo clusters run on a cloud native infrastructure that features software-hardware synergy to support passthrough networking, high security and reliability, and intelligent scheduling. -CCE Turbo clusters are paired with the Cloud Native Network 2.0 model for large-scale, high-performance container deployment. Containers are assigned IP addresses from the VPC CIDR block. Containers and nodes can belong to different subnets. Access requests from external networks in a VPC can be directly routed to container IP addresses, which greatly improves networking performance. **It is recommended** that you go through :ref:`Cloud Native Network 2.0 ` to understand the features and network planning of each CIDR block of Cloud Native Network 2.0. +CCE Turbo clusters are paired with the Cloud Native Network 2.0 model for large-scale, high-performance container deployment. Containers are assigned IP addresses from the VPC CIDR block. Containers and nodes can belong to different subnets. Access requests from external networks in a VPC can be directly routed to container IP addresses, which greatly improves networking performance. **It is recommended** that you go through :ref:`Cloud Native Network 2.0 ` to understand the features and network planning of each CIDR block of Cloud Native Network 2.0. Notes and Constraints --------------------- - During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name. - You can create a maximum of 50 clusters in a single region. -- CCE Turbo clusters support only Cloud Native Network 2.0. For details about this network model, see :ref:`Cloud Native Network 2.0 `. -- Nodes in a CCE Turbo cluster must be the models developed on the QingTian architecture that features software-hardware synergy. -- CCE Turbo clusters are available only in certain regions. +- CCE Turbo clusters support only Cloud Native Network 2.0. For details about this network model, see :ref:`Cloud Native Network 2.0 `. +- After a cluster is created, the following items cannot be changed: + + - Cluster type + - Number of master nodes in the cluster + - AZ of a master node + - Network configuration of the cluster, such as the VPC, subnet, container CIDR block, Service CIDR block, and kube-proxy (forwarding) settings. + - Network model. For example, change **Tunnel network** to **VPC network**. Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. Click **Create** next to **CCE Turbo Cluster**. +#. Log in to the CCE console. Choose **Clusters**. On the displayed page, click **Create** next to **CCE Turbo cluster**. +#. Specify cluster parameters. - .. figure:: /_static/images/en-us_image_0000001150420952.png - :alt: **Figure 1** Creating a CCE Turbo cluster + **Basic Settings** - **Figure 1** Creating a CCE Turbo cluster + - **Cluster Name** -#. On the page displayed, set the following parameters: + - **Cluster Version**: Select the Kubernetes version used by the cluster. - **Basic configuration** + - **Cluster Scale**: Select the maximum number of nodes that can be managed by the cluster. After the creation is complete, only scale-out is supported, but not scale-in. - Specify the basic cluster configuration. + - **HA**: distribution mode of master nodes. By default, master nodes are randomly distributed in different AZs to improve DR capabilities. - .. table:: **Table 1** Basic parameters for creating a cluster + You can also expand advanced settings and customize the master node distribution mode. The following modes are supported: - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+====================================================================================================================================================================+ - | Cluster Name | Name of the cluster to be created. The cluster name must be unique under the same account and cannot be changed after the cluster is created. | - | | | - | | A cluster name contains 4 to 128 characters, starting with a letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Version | Version of Kubernetes to use for the cluster. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Management Scale | Maximum number of worker nodes that can be managed by the master nodes of the cluster. You can select 200 nodes, 1,000 nodes, or 2,000 nodes for your cluster. | - | | | - | | Master node specifications change with the cluster management scale you choose, and you will be charged accordingly. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Host**: Master nodes are created on different hosts in the same AZ. + - **Custom**: You can determine the location of each master node. - **Networking configuration** + **Network Settings** - Select the CIDR blocks used by nodes and containers in the cluster. If IP resources in the CIDR blocks are insufficient, nodes and containers cannot be created. + The cluster network settings cover nodes, containers, and Services. For details about the cluster networking and container network models, see :ref:`Overview `. - .. table:: **Table 2** Networking parameters - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==========================================================================================================================================================================================================================================================================================================================================================================================================================+ - | Network Model | **Cloud Native Network 2.0**: This network model deeply integrates the native elastic network interfaces (ENIs) of VPC, uses the VPC CIDR block to allocate container addresses, and supports direct traffic distribution to containers through a load balancer to deliver high performance. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | VPC | Select the VPC used by nodes and containers in the cluster. The VPC cannot be changed after the cluster is created. | - | | | - | | A VPC provides a secure and logically isolated network environment. | - | | | - | | If no VPC is available, create one on the **VPC console**. After the VPC is created, click the refresh icon. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Node Subnet | This parameter is available after you select a VPC. | - | | | - | | The subnet you select is used by nodes in the cluster and determines the maximum number of nodes in the cluster. This subnet will be the default subnet where your nodes are created. When creating a node, you can select other subnets in the same VPC. | - | | | - | | A node subnet provides dedicated network resources that are logically isolated from other networks for higher security. | - | | | - | | If no node subnet is available, click **Create Subnet** to create a subnet. After the subnet is created, click the refresh icon. For details about the relationship between VPCs, subnets, and clusters, see :ref:`Cluster Overview `. | - | | | - | | During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name. | - | | | - | | **The selected subnet cannot be changed after the cluster is created.** | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Pod Subnet | This parameter is available after you select a VPC. | - | | | - | | The subnet you select is used by pods in the cluster and determines the maximum number of pods in the cluster. The subnet cannot be changed after the cluster is created. | - | | | - | | IP addresses used by pods will be allocated from this subnet. | - | | | - | | .. note:: | - | | | - | | If the pod subnet is the same as the node subnet, pods and nodes share the remaining IP addresses in the subnet. As a result, pods or nodes may fail to be created due to insufficient IP addresses. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Network Model**: CCE Turbo clusters support only **Cloud Native Network 2.0**. For details, see :ref:`Cloud Native Network 2.0 `. + - **VPC**: Select the VPC to which the cluster belongs. If no VPC is available, click **Create VPC** to create one. The value cannot be changed after creation. + - **Master Node Subnet**: Select the subnet where the master node is deployed. If no subnet is available, click **Create Subnet** to create one. A master node requires at least four IP addresses, which cannot be changed after creation. + - **Pod Subnet**: Select the subnet where the container is located. If no subnet is available, click **Create Subnet** to create one. The pod subnet determines the maximum number of containers in the cluster. You can add pod subnets after creating the cluster. + - **Service CIDR Block**: CIDR block for :ref:`Services ` used by containers in the same cluster to access each other. The value determines the maximum number of Services you can create. The value cannot be changed after creation. **Advanced Settings** - Configure enhanced capabilities for your CCE Turbo cluster. + - **Request Forwarding**: The IPVS and iptables modes are supported. For details, see :ref:`Comparing iptables and IPVS `. - .. table:: **Table 3** Networking parameters + - **CPU Manager**: For details, see :ref:`Binding CPU Cores `. - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+=========================================================================================================================================================================================================================================================================================================+ - | Service Network Segment | An IP range from which IP addresses are allocated to Kubernetes Services. After the cluster is created, the CIDR block cannot be changed. The Service CIDR block cannot conflict with the created routes. If they conflict, select another CIDR block. | - | | | - | | The default value is **10.247.0.0/16**. You can change the CIDR block and mask according to your service requirements. The mask determines the maximum number of Service IP addresses available in the cluster. | - | | | - | | After you set the mask, the console will provide an estimated maximum number of Services you can create in this CIDR block. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kube-proxy Mode | Load balancing between Services and their backend pods. The value cannot be changed after the cluster is created. | - | | | - | | - **IPVS**: optimized kube-proxy mode to achieve higher throughput and faster speed, ideal for large-sized clusters. This mode supports incremental updates and can keep connections uninterrupted during Service updates. | - | | | - | | In this mode, when the ingress and Service use the same ELB instance, the ingress cannot be accessed from the nodes and containers in the cluster. | - | | | - | | - **iptables**: Use iptables rules to implement Service load balancing. In this mode, too many iptables rules will be generated when many Services are deployed. In addition, non-incremental updates will cause a latency and even tangible performance issues in the case of service traffic spikes. | - | | | - | | .. note:: | - | | | - | | - IPVS provides better scalability and performance for large clusters. | - | | - Compared with iptables, IPVS supports more complex load balancing algorithms such as least load first (LLF) and weighted least connections (WLC). | - | | - IPVS supports server health check and connection retries. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | CPU Policy | - **On**: Exclusive CPU cores can be allocated to workload pods. Select **On** if your workload is sensitive to latency in CPU cache and scheduling. | - | | - **Off**: Exclusive CPU cores will not be allocated to workload pods. Select **Off** if you want a large pool of shareable CPU cores. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Resource Tag**: -#. Click **Next: Confirm** to review the configurations and change them if required. + You can add resource tags to classify resources. -#. Click **Submit**. + You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and resource migration efficiency. + + - **Certificate Authentication**: + + - **Default**: The X509-based authentication mode is enabled by default. X509 is a commonly used certificate format. + + - **Custom:** The cluster can identify users based on the header in the request body for authentication. + + You need to upload your **CA root certificate**, **client certificate**, and **private key** of the client certificate. + + .. caution:: + + - Upload a file **smaller than 1 MB**. The CA certificate and client certificate can be in **.crt** or **.cer** format. The private key of the client certificate can only be uploaded **unencrypted**. + - The validity period of the client certificate must be longer than five years. + - The uploaded CA certificate is used for both the authentication proxy and the kube-apiserver aggregation layer configuration. **If the certificate is invalid, the cluster cannot be created**. + - Starting from v1.25, Kubernetes no longer supports certificate authentication generated using the SHA1WithRSA or ECDSAWithSHA1 algorithm. You are advised to use the SHA256 algorithm. + + - **Description**: The value can contain a maximum of 200 English characters. + +#. Click **Next: Add-on Configuration**. + + By default, :ref:`cordens ` and :ref:`everest ` add-ons are installed. + + **Service log** + + - **ICAgent**: + + A log collector provided by Application Operations Management (AOM), reporting logs to AOM and Log Tank Service (LTS) according to the log collection rules you configured. + + You can collect stdout logs as required. + +#. After configuring the parameters, click **Next: Confirm**. It takes about 6 to 10 minutes to create a cluster. You can click **Back to Cluster List** to perform other operations on the cluster or click **Go to Cluster Events** to view the cluster details. -#. If the cluster status is **Available**, the CCE Turbo cluster is successfully created, and **Turbo** is displayed next to the cluster name. - Related Operations ------------------ -- Using kubectl to connect to the cluster: :ref:`Connecting to a Cluster Using kubectl ` -- Logging in to the node: :ref:`Logging In to a Node ` - -- Creating a namespace: You can create multiple namespaces in a cluster and organize resources in the cluster into different namespaces. These namespaces serve as logical groups and can be managed separately. For details about how to create a namespace for a cluster, see :ref:`Namespaces `. -- Creating a workload: Once the cluster is created, you can use an image to create an application that can be accessed from public networks. For details, see :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `. -- Viewing cluster details: Click the cluster name to view cluster details. - - .. table:: **Table 4** Details about the created cluster - - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Tab | Description | - +===================================+====================================================================================================================================================================================================================================================+ - | Basic Information | You can view the details and running status of the cluster. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Monitoring | You can view the CPU and memory allocation rates of all nodes in the cluster (that is, the maximum allocated amount), as well as the CPU usage, memory usage, and specifications of the master node(s). | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Events | - View cluster events. | - | | - Set search criteria, such as the event name or the time segment during which an event is generated, to filter events. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Auto Scaling | You can configure auto scaling to add or reduce worker nodes in a cluster to meet service requirements. For details, see :ref:`Setting Cluster Auto Scaling `. | - | | | - | | Clusters of v1.17 do not support auto scaling using AOM. You can use node pools for auto scaling. For details, see :ref:`Node Pool Overview `. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubectl | To access a Kubernetes cluster from a PC, you need to use the Kubernetes command line tool `kubectl `__. For details, see :ref:`Connecting to a Cluster Using kubectl `. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Resource Tags | Resource tags can be added to classify resources. | - | | | - | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and resource migration efficiency. | - | | | - | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *Node ID*" tag. A maximum of 5 tags can be added. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +- Using kubectl to connect to the cluster: :ref:`Connecting to a Cluster Using kubectl ` +- Add nodes to the cluster. For details, see :ref:`Creating a Node `. diff --git a/umn/source/clusters/index.rst b/umn/source/clusters/index.rst index d2b2f5c..cf95826 100644 --- a/umn/source/clusters/index.rst +++ b/umn/source/clusters/index.rst @@ -1,34 +1,28 @@ -:original_name: cce_01_0027.html +:original_name: cce_10_0091.html -.. _cce_01_0027: +.. _cce_10_0091: Clusters ======== -- :ref:`Cluster Overview ` -- :ref:`CCE Turbo Clusters and CCE Clusters ` -- :ref:`Creating a CCE Turbo Cluster ` -- :ref:`Creating a CCE Cluster ` -- :ref:`Using kubectl to Run a Cluster ` -- :ref:`Setting Cluster Auto Scaling ` -- :ref:`Upgrading a Cluster ` -- :ref:`Managing a Cluster ` -- :ref:`Obtaining a Cluster Certificate ` -- :ref:`Controlling Cluster Permissions ` -- :ref:`Cluster Parameters ` +- :ref:`Cluster Overview ` +- :ref:`Creating a CCE Turbo Cluster ` +- :ref:`Creating a CCE Cluster ` +- :ref:`Using kubectl to Run a Cluster ` +- :ref:`Upgrading a Cluster ` +- :ref:`Managing a Cluster ` +- :ref:`Obtaining a Cluster Certificate ` +- :ref:`Changing Cluster Scale ` .. toctree:: :maxdepth: 1 :hidden: - cluster_overview - cce_turbo_clusters_and_cce_clusters + cluster_overview/index creating_a_cce_turbo_cluster creating_a_cce_cluster using_kubectl_to_run_a_cluster/index - setting_cluster_auto_scaling upgrading_a_cluster/index managing_a_cluster/index obtaining_a_cluster_certificate - controlling_cluster_permissions - cluster_parameters/index + changing_cluster_scale diff --git a/umn/source/clusters/managing_a_cluster/cluster_overload_control.rst b/umn/source/clusters/managing_a_cluster/cluster_overload_control.rst new file mode 100644 index 0000000..e3b1094 --- /dev/null +++ b/umn/source/clusters/managing_a_cluster/cluster_overload_control.rst @@ -0,0 +1,36 @@ +:original_name: cce_10_0602.html + +.. _cce_10_0602: + +Cluster Overload Control +======================== + +Scenario +-------- + +If overload control is enabled, concurrent requests are dynamically controlled based on the resource pressure of master nodes to keep them and the cluster available. + +Notes and Constraints +--------------------- + +The cluster version must be 1.23 or later. + +Enabling Overload Control +------------------------- + +**Method 1: Enabling it when creating a cluster** + +When creating a cluster of v1.23 or later, you can enable overload control during the cluster creation. + +**Method 2: Enabling it in an existing cluster** + +#. Log in to the CCE console and go to an existing cluster whose version is v1.23 or later. +#. On the cluster information page, view the master node information. If overload control is not enabled, a message is displayed. You can click **Enable** to enable the function. + +Disabling Cluster Overload Control +---------------------------------- + +#. Log in to the CCE console and go to an existing cluster whose version is v1.23 or later. +#. On the **Cluster Information** page, click **Manage** in the upper right corner. +#. Set **support-overload** to **false** under **kube-apiserver**. +#. Click **OK**. diff --git a/umn/source/clusters/managing_a_cluster/configuring_kubernetes_parameters.rst b/umn/source/clusters/managing_a_cluster/configuring_kubernetes_parameters.rst deleted file mode 100644 index 44fd21c..0000000 --- a/umn/source/clusters/managing_a_cluster/configuring_kubernetes_parameters.rst +++ /dev/null @@ -1,104 +0,0 @@ -:original_name: cce_01_0213.html - -.. _cce_01_0213: - -Configuring Kubernetes Parameters -================================= - -Scenario --------- - -CCE clusters allow you to manage Kubernetes parameters, through which you can let core components work under your very requirements. - -Notes and Constraints ---------------------- - -This function is supported only in clusters of **v1.15 and later**. It is not displayed for versions earlier than v1.15. - -Procedure ---------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. -#. Choose **More** > **Configuration**. -#. On the **Configuration** page on the right, change the values of the following Kubernetes parameters: - - .. table:: **Table 1** Kubernetes parameters - - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | Component | Parameter | Description | Value | - +=========================+========================================+==========================================================================================================================================+=====================+ - | kube-apiserver | default-not-ready-toleration-seconds | notReady tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | default-unreachable-toleration-seconds | unreachable tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | max-mutating-requests-inflight | Maximum number of concurrent mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 1000 | - | | | | | - | | | The value **0** indicates no limitation. | | - | | | | | - | | | Manual configuration is no longer supported since cluster version 1.21. The value is automatically specified based on the cluster scale. | | - | | | | | - | | | - **200** for clusters with 50 or 200 nodes | | - | | | - **500** for clusters with 1000 nodes | | - | | | - **1000** for clusters with 2000 nodes | | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | max-requests-inflight | Maximum number of concurrent non-mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 2000 | - | | | | | - | | | The value **0** indicates no limitation. | | - | | | | | - | | | Manual configuration is no longer supported since cluster version 1.21. The value is automatically specified based on the cluster scale. | | - | | | | | - | | | - **400** for clusters with 50 or 200 nodes | | - | | | - **1000** for clusters with 1000 nodes | | - | | | - **2000** for clusters with 2000 nodes | | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | service-node-port-range | Range of node port numbers. | Default: | - | | | | | - | | | | 30000-32767 | - | | | | | - | | | | Options: | - | | | | | - | | | | min>20105 | - | | | | | - | | | | max<32768 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | kube-controller-manager | concurrent-deployment-syncs | Number of Deployments that are allowed to synchronize concurrently. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-endpoint-syncs | Number of endpoints that are allowed to synchronize concurrently. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-gc-syncs | Number of garbage collector workers that are allowed to synchronize concurrently. | Default: 20 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-job-syncs | Number of jobs that can be synchronized at the same time. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-namespace-syncs | Number of namespaces that are allowed to synchronize concurrently. | Default: 10 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-replicaset-syncs | Number of ReplicaSets that are allowed to synchronize concurrently. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-resource-quota-syncs | Number of resource quotas that are allowed to synchronize concurrently. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-service-syncs | Number of Services that are allowed to synchronize concurrently. | Default: 10 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-serviceaccount-token-syncs | Number of service account tokens that are allowed to synchronize concurrently. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent-ttl-after-finished-syncs | Number of TTL-after-finished controller workers that are allowed to synchronize concurrently. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | concurrent_rc_syncs | Number of replication controllers that are allowed to synchronize concurrently. | Default: 5 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | horizontal-pod-autoscaler-sync-period | How often HPA audits metrics in a cluster. | Default: 15 seconds | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | Default: 100 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | kube-api-burst | Burst to use while talking with kube-apiserver. | Default: 100 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | kube-scheduler | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | Default: 100 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - | | kube-api-burst | Burst to use while talking with kube-apiserver. | Default: 100 | - +-------------------------+----------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+---------------------+ - -#. Click **OK**. - -References ----------- - -- `kube-apiserver `__ -- `kube-controller-manager `__ -- `kube-scheduler `__ diff --git a/umn/source/clusters/managing_a_cluster/deleting_a_cluster.rst b/umn/source/clusters/managing_a_cluster/deleting_a_cluster.rst index 95c76f2..100f20a 100644 --- a/umn/source/clusters/managing_a_cluster/deleting_a_cluster.rst +++ b/umn/source/clusters/managing_a_cluster/deleting_a_cluster.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0212.html +:original_name: cce_10_0212.html -.. _cce_01_0212: +.. _cce_10_0212: Deleting a Cluster ================== @@ -28,18 +28,25 @@ Precautions Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. +#. Log in to the CCE console. In the navigation pane, choose **Clusters**. -#. Choose **More** > **Delete**. +#. Click |image1| next to the cluster to be deleted. -#. Delete the cluster. +#. In the displayed **Delete Cluster** dialog box, select the resources to be released. + - Delete cloud storage resources attached to workloads in the cluster. - .. figure:: /_static/images/en-us_image_0000001190168507.png - :alt: **Figure 1** Deleting a cluster + .. note:: - **Figure 1** Deleting a cluster + Before you delete the PVCs and volumes, pay attention to the following rules: + + - The underlying storage resources are deleted according to the reclaim policy you defined. + - If there are a large number of files (more than 1,000) in the OBS bucket, manually clear the files and then delete the cluster. + + - Delete networking resources, such as load balancers in a cluster. (Only automatically created load balancers can be deleted.) #. Click **Yes** to start deleting the cluster. The delete operation takes 1 to 3 minutes to complete. + +.. |image1| image:: /_static/images/en-us_image_0000001244997085.png diff --git a/umn/source/clusters/managing_a_cluster/hibernating_and_waking_up_a_cluster.rst b/umn/source/clusters/managing_a_cluster/hibernating_and_waking_up_a_cluster.rst index 3b4ba85..bc87150 100644 --- a/umn/source/clusters/managing_a_cluster/hibernating_and_waking_up_a_cluster.rst +++ b/umn/source/clusters/managing_a_cluster/hibernating_and_waking_up_a_cluster.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0214.html +:original_name: cce_10_0214.html -.. _cce_01_0214: +.. _cce_10_0214: Hibernating and Waking Up a Cluster =================================== @@ -8,33 +8,30 @@ Hibernating and Waking Up a Cluster Scenario -------- -If you do not need to use a cluster temporarily, you are advised to hibernate the cluster to save cluster management costs. +If you do not need to use a cluster temporarily, you are advised to hibernate the cluster. After a cluster is hibernated, resources such as workloads cannot be created or managed in the cluster. A hibernated cluster can be quickly woken up and used normally. +Notes and Constraints +--------------------- + +During cluster wakeup, the master node may fail to be started due to insufficient resources. As a result, the cluster fails to be woken up. Wait for a while and wake up the cluster again. + Hibernating a Cluster --------------------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. -#. Choose **More** > **Hibernate** for the target cluster. +#. Log in to the CCE console. In the navigation pane, choose **Clusters**. +#. Click |image1| next to the cluster to be hibernated. #. In the dialog box displayed, check the precautions and click **Yes**. Wait until the cluster is hibernated. - .. important:: - - - After a cluster is hibernated, resources, such as worker nodes (ECSs), bound EIPs, and bandwidth, are still billed based on their own billing modes. To shut down nodes, select **Stop all nodes in the cluster** in the dialog box or see :ref:`Stopping a Node `. - -#. When the cluster status changes from **Hibernating** to **Hibernation**, the cluster is hibernated. - Waking Up a Cluster ------------------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. -#. Choose **More** > **Wake**. -#. In the dialog box displayed, click **Yes** and wait until the cluster is woken up. -#. When the cluster status changes from **Waking** to **Available**, the cluster is woken up. +#. Log in to the CCE console. In the navigation pane, choose **Clusters**. +#. Click |image2| next to the cluster to be woken up. +#. When the cluster status changes from **Waking** to **Running**, the cluster is woken up. It takes about 3 to 5 minutes to wake up the cluster. - .. note:: - - After the cluster is woken up, billing will be resumed for the resources on the master node. +.. |image1| image:: /_static/images/en-us_image_0000001236562704.png +.. |image2| image:: /_static/images/en-us_image_0000001225747980.png diff --git a/umn/source/clusters/managing_a_cluster/index.rst b/umn/source/clusters/managing_a_cluster/index.rst index 5da3ae6..e4b3ac6 100644 --- a/umn/source/clusters/managing_a_cluster/index.rst +++ b/umn/source/clusters/managing_a_cluster/index.rst @@ -1,18 +1,20 @@ -:original_name: cce_01_0031.html +:original_name: cce_10_0031.html -.. _cce_01_0031: +.. _cce_10_0031: Managing a Cluster ================== -- :ref:`Deleting a Cluster ` -- :ref:`Hibernating and Waking Up a Cluster ` -- :ref:`Configuring Kubernetes Parameters ` +- :ref:`Managing Cluster Components ` +- :ref:`Deleting a Cluster ` +- :ref:`Hibernating and Waking Up a Cluster ` +- :ref:`Cluster Overload Control ` .. toctree:: :maxdepth: 1 :hidden: + managing_cluster_components deleting_a_cluster hibernating_and_waking_up_a_cluster - configuring_kubernetes_parameters + cluster_overload_control diff --git a/umn/source/clusters/managing_a_cluster/managing_cluster_components.rst b/umn/source/clusters/managing_a_cluster/managing_cluster_components.rst new file mode 100644 index 0000000..f6fc814 --- /dev/null +++ b/umn/source/clusters/managing_a_cluster/managing_cluster_components.rst @@ -0,0 +1,156 @@ +:original_name: cce_10_0213.html + +.. _cce_10_0213: + +Managing Cluster Components +=========================== + +Scenario +-------- + +CCE allows you to manage cluster parameters, through which you can let core components work under your very requirements. + +Notes and Constraints +--------------------- + +This function is supported only in clusters of **v1.15 and later**. It is not displayed for versions earlier than v1.15. + +Procedure +--------- + +#. Log in to the CCE console. In the navigation pane, choose **Clusters**. +#. Click |image1| next to the target cluster. +#. On the **Manage Component** page on the right, change the values of the following Kubernetes parameters: + + .. table:: **Table 1** Extended controller parameters + + +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ + | Parameter | Description | Value | + +=======================+======================================================================================================================================+=======================+ + | enable-resource-quota | Whether to automatically create a resource quota object when creating a namespace. | Default: false | + | | | | + | | - **false**: no auto creation | | + | | - **true**: auto creation enabled For details about the resource quota defaults, see :ref:`Setting a Resource Quota `. | | + +-----------------------+--------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ + + .. table:: **Table 2** kube-apiserver parameters + + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | Parameter | Description | Value | + +========================================+=============================================================================================================================================================================+=========================================+ + | default-not-ready-toleration-seconds | notReady tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300s | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | default-unreachable-toleration-seconds | unreachable tolerance time, in seconds. NoExecute that is added by default to every pod that does not already have such a toleration. | Default: 300s | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | max-mutating-requests-inflight | Maximum number of concurrent mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 1000 | + | | | | + | | The value **0** indicates no limitation. | | + | | | | + | | Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. | | + | | | | + | | - **200** for clusters with 50 or 200 nodes | | + | | - **500** for clusters with 1,000 nodes | | + | | - **1000** for clusters with 2,000 nodes | | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | max-requests-inflight | Maximum number of concurrent non-mutating requests. When the value of this parameter is exceeded, the server rejects requests. | Default: 2000 | + | | | | + | | The value **0** indicates no limitation. | | + | | | | + | | Manual configuration is no longer supported since cluster v1.21. The value is automatically specified based on the cluster scale. | | + | | | | + | | - **400** for clusters with 50 or 200 nodes | | + | | - **1000** for clusters with 1,000 nodes | | + | | - **2000** for clusters with 2,000 nodes | | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | service-node-port-range | Range of node port numbers. | Default: | + | | | | + | | | 30000-32767 | + | | | | + | | | Options: | + | | | | + | | | min>20105 | + | | | | + | | | max<32768 | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + | support-overload | Cluster overload control. If enabled, concurrent requests are dynamically controlled based on the resource pressure of master nodes to keep them and the cluster available. | - false: Overload control is disabled. | + | | | - true: Overload control is enabled. | + +----------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------+ + + .. table:: **Table 3** kube-controller-manager parameters + + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | Parameter | Description | Value | + +=======================================+=======================================================================================================================+=======================+ + | concurrent-deployment-syncs | Number of Deployments that are allowed to synchronize concurrently. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-endpoint-syncs | Number of endpoints that are allowed to synchronize concurrently. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-gc-syncs | Number of garbage collector workers that are allowed to synchronize concurrently. | Default: 20 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-job-syncs | Number of jobs that can be synchronized at the same time. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-namespace-syncs | Number of namespaces that are allowed to synchronize concurrently. | Default: 10 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-replicaset-syncs | Number of ReplicaSets that are allowed to synchronize concurrently. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-resource-quota-syncs | Number of resource quotas that are allowed to synchronize concurrently. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-service-syncs | Number of Services that are allowed to synchronize concurrently. | Default: 10 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-serviceaccount-token-syncs | Number of service account tokens that are allowed to synchronize concurrently. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent-ttl-after-finished-syncs | Number of TTL-after-finished controller workers that are allowed to synchronize concurrently. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | concurrent_rc_syncs | Number of replication controllers that are allowed to synchronize concurrently. | Default: 5 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | horizontal-pod-autoscaler-sync-period | How often HPA audits metrics in a cluster. | Default: 15 seconds | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | Default: 100 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | kube-api-burst | Burst to use while talking with kube-apiserver. | Default: 100 | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + | terminated-pod-gc-threshold | Number of terminated pods that can exist before the terminated pod garbage collector starts deleting terminated pods. | Default: 1000 | + | | | | + | | If <= 0, the terminated pod garbage collector is disabled. | | + +---------------------------------------+-----------------------------------------------------------------------------------------------------------------------+-----------------------+ + + .. table:: **Table 4** kube-scheduler parameters + + +----------------+------------------------------------------------------------------+--------------+ + | Parameter | Description | Value | + +================+==================================================================+==============+ + | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | Default: 100 | + +----------------+------------------------------------------------------------------+--------------+ + | kube-api-burst | Burst to use while talking with kube-apiserver. | Default: 100 | + +----------------+------------------------------------------------------------------+--------------+ + + .. table:: **Table 5** eni parameters (supported only by CCE Turbo clusters) + + +----------------------------+----------------------------------------------------------------------------------------------+-----------------------+ + | Parameter | Description | Value | + +============================+==============================================================================================+=======================+ + | nic-minimum-target | Minimum number of ENIs bound to a node at the cluster level | Default: 10 | + +----------------------------+----------------------------------------------------------------------------------------------+-----------------------+ + | nic-maximum-target | Maximum number of ENIs pre-bound to a node at the cluster level | Default: 0 | + +----------------------------+----------------------------------------------------------------------------------------------+-----------------------+ + | nic-warm-target | Number of ENIs pre-bound to a node at the cluster level | Default: 2 | + +----------------------------+----------------------------------------------------------------------------------------------+-----------------------+ + | nic-max-above-warm-target | Reclaim number of ENIs pre-bound to a node at the cluster level | Default: 2 | + +----------------------------+----------------------------------------------------------------------------------------------+-----------------------+ + | prebound-subeni-percentage | Low threshold of the number of bound ENIs:High threshold of the number of bound ENIs | Default: 0:0 | + | | | | + | | .. note:: | | + | | | | + | | This parameter is discarded. Use the other four dynamic preheating parameters of the ENI. | | + +----------------------------+----------------------------------------------------------------------------------------------+-----------------------+ + +#. Click **OK**. + +References +---------- + +- `kube-apiserver `__ +- `kube-controller-manager `__ +- `kube-scheduler `__ + +.. |image1| image:: /_static/images/en-us_image_0000001199757520.png diff --git a/umn/source/clusters/obtaining_a_cluster_certificate.rst b/umn/source/clusters/obtaining_a_cluster_certificate.rst index f792f7f..2bc42d4 100644 --- a/umn/source/clusters/obtaining_a_cluster_certificate.rst +++ b/umn/source/clusters/obtaining_a_cluster_certificate.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0175.html +:original_name: cce_10_0175.html -.. _cce_01_0175: +.. _cce_10_0175: Obtaining a Cluster Certificate =============================== @@ -8,19 +8,19 @@ Obtaining a Cluster Certificate Scenario -------- -Before accessing cluster resources through open-source Kubernetes APIs, obtain the cluster's certificate. +This section describes how to obtain the cluster certificate from the console and use it to access Kubernetes clusters. Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. +#. Log in to the CCE console and access the cluster console. -#. In the card view of the target cluster, choose **More** > **Download X.509 Certificate**. +#. Choose **Cluster Information** from the navigation pane and click **Download** next to **Authentication Mode** in the **Connection Information** area. #. In the **Download X.509 Certificate** dialog box displayed, select the certificate expiration time and download the X.509 certificate of the cluster as prompted. - .. figure:: /_static/images/en-us_image_0000001190859184.png + .. figure:: /_static/images/en-us_image_0000001199181228.png :alt: **Figure 1** Downloading a certificate **Figure 1** Downloading a certificate diff --git a/umn/source/clusters/setting_cluster_auto_scaling.rst b/umn/source/clusters/setting_cluster_auto_scaling.rst deleted file mode 100644 index 5de8c5a..0000000 --- a/umn/source/clusters/setting_cluster_auto_scaling.rst +++ /dev/null @@ -1,109 +0,0 @@ -:original_name: cce_01_0157.html - -.. _cce_01_0157: - -Setting Cluster Auto Scaling -============================ - -Scenario --------- - -The Cluster Auto Scaling feature allows CCE to automatically scale out a cluster (adding worker nodes to a cluster) according to custom policies when workloads cannot be scheduled into the cluster due to insufficient cluster resources. - -Notes and Constraints ---------------------- - -- Currently, master nodes cannot be automatically added to or removed from clusters. -- If both auto scale-in and auto scale-out are required, use the autoscaler add-on. For details, see :ref:`autoscaler `. -- Clusters of v1.17 do not support auto scaling using AOM. You can use node pools for auto scaling. For details, see :ref:`Node Pool Overview `. - -Automatic Cluster Scale-out ---------------------------- - -#. Log in to the CCE console. Choose **Resource Management** > **Clusters** in the navigation pane. In the card view of the cluster to be scaled, choose **More** > **Auto Scaling**. - -#. Click the **Scale-out Settings** tab and then **Edit**. Set the maximum number of nodes, minimum number of nodes, cooldown period, and node configuration. - - .. table:: **Table 1** Scale-out settings - - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==================================================================================================================================================================================================================================+ - | Cooldown Period | Interval between consecutive scale-out operations, in the unit of second. The cooldown period ensures that a scale-out operation is initiated only when previous scaling operation is finished and the system is running stably. | - | | | - | | The value ranges from 60 to 3600, in seconds. The default value is 900. If the cooling interval is less than 900 seconds (15 minutes), the auto scaling may not work well, because creating a node may take 2 to 10 minutes. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Maximum Nodes | Maximum number of nodes to which the cluster can scale out. | - | | | - | | 1 <= Maximum Nodes < cluster node quota | - | | | - | | .. note:: | - | | | - | | The cluster node quota depends on the cluster size (maximum number of nodes that can be managed by a cluster) and the node quota of the account. The cluster node quota used here is the smaller of the two. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Node Configuration | If scale-out is required after the scale-out policy is executed, the system creates a node. | - | | | - | | a. Click **Set** and set the node parameters. For details about how to set the node parameters, see :ref:`Creating a Node `. | - | | b. After the parameters are configured, click **Submit**. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. After confirming the scale-out configuration and node parameters, click **OK**. - -#. Set the scale-out policy for the cluster. Click the **Scale-out Policies** tab and click **Add Policy**. - - - **Policy Name**: Enter a policy name, for example, **policy01**. - - **Policy Type**: Currently, the following types of auto scale-out policies are supported: - - - :ref:`Metric-based policy `: Scale-out is performed based on the CPU or memory settings. - - .. _cce_01_0157__table23209107191540: - - .. table:: **Table 2** Parameters for adding a metric-based policy - - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================+ - | \*Metric | Select **Allocated CPU** or **Allocated Memory**. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Trigger Condition | Set a condition for triggering a scale-out policy, that is, when the average CPU or memory allocation value is greater than or less than a specified percentage. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Monitoring Window | Size of the data aggregation window. Select a value from the drop-down list. | - | | | - | | If you select **15min**, the selected metric is measured every 15 minutes. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Threshold Crossings | Number of consecutive times that the threshold is reached within the monitoring window. The calculation cycle is fixed at one minute. If you set this parameter to **3**, the configured action will be triggered when the metrics meet the specified threshold for three consecutive times. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Action | Action executed after a policy is triggered. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - - :ref:`Scheduled policy `: Scale-out is performed at a specified time. - - .. _cce_01_0157__table62540231191540: - - .. table:: **Table 3** Parameters for adding a scheduled policy - - ============== ============================================ - Parameter Description - ============== ============================================ - \*Policy Type Set this parameter to **Scheduled policy**. - \*Trigger Time Time at which the policy is triggered. - \*Action Action executed after a policy is triggered. - ============== ============================================ - - - :ref:`Periodic policy `: Scale-out can be performed by day, week, or month. - - .. _cce_01_0157__table60088509191540: - - .. table:: **Table 4** Parameters for adding a periodic policy - - ============= ============================================ - Parameter Description - ============= ============================================ - \*Policy Type Set the parameter to **Periodic policy**. - \*Time Range Specify the time for triggering the policy. - \*Action Action executed after a policy is triggered. - ============= ============================================ - -#. Click **OK**. - - After the auto scale-out is completed, choose **Resource Management** > **Nodes** in the navigation pane. On the node list, you can view the worker nodes added during cluster auto scaling. diff --git a/umn/source/clusters/upgrading_a_cluster/before_you_start.rst b/umn/source/clusters/upgrading_a_cluster/before_you_start.rst index 39305d4..4e31c57 100644 --- a/umn/source/clusters/upgrading_a_cluster/before_you_start.rst +++ b/umn/source/clusters/upgrading_a_cluster/before_you_start.rst @@ -1,34 +1,36 @@ -:original_name: cce_01_0302.html +:original_name: cce_10_0302.html -.. _cce_01_0302: +.. _cce_10_0302: Before You Start ================ -Before the upgrade, you can check whether your cluster can be upgraded and which versions are available on the CCE console. For details, see :ref:`Overview `. +Before the upgrade, you can check whether your cluster can be upgraded and which versions are available on the CCE console. For details, see :ref:`Upgrade Overview `. Precautions ----------- - **Upgraded clusters cannot be rolled back. Therefore, perform the upgrade during off-peak hours to minimize the impact on your services.** -- Do not shut down or restart nodes during cluster upgrade. Otherwise, the upgrade fails. -- Before upgrading a cluster, disable auto scaling policies to prevent node scaling during the upgrade. Otherwise, the upgrade fails. +- Do not **shut down, restart, or delete nodes** during cluster upgrade. Otherwise, the upgrade fails. +- Before upgrading a cluster, **disable auto scaling policies** to prevent node scaling during the upgrade. Otherwise, the upgrade fails. - If you locally modify the configuration of a cluster node, the cluster upgrade may fail or the configuration may be lost after the upgrade. Therefore, modify the configurations on the CCE console (cluster or node pool list page) so that they will be automatically inherited during the upgrade. - During the cluster upgrade, the running workload services will not be interrupted, but access to the API server will be temporarily interrupted. - Before upgrading the cluster, check whether the cluster is healthy. - To ensure data security, you are advised to back up data before upgrading the cluster. During the upgrade, you are not advised to perform any operations on the cluster. -- CCE 1.17 and later versions do not support workload scaling using the AOM service. Before and after the upgrade, switch scaling policies by referring to :ref:`Switching from AOM to HPA for Auto Scaling `. +- During the cluster upgrade, the **node.kubernetes.io/upgrade** taint (the effect is **NoSchedule**) is added to the node. After the cluster upgrade is complete, the taint is removed. Do not add taints with the same key name on the node. Even if the taints have different effects, they may be deleted by the system by mistake after the upgrade. Notes and Constraints --------------------- - Currently, only CCE clusters consisting of VM nodes can be upgraded. +- After the cluster is upgraded, if the containerd vulnerability of the container engine is fixed in :ref:`Cluster Version Release Notes `, you need to manually restart containerd for the upgrade to take effect. The same applies to the existing pods. + - If initContainer or Istio is used in the in-place upgrade of a cluster of v1.15, pay attention to the following restrictions: - In kubelet 1.16 and later versions, `QoS classes `__ are different from those in earlier versions. In kubelet 1.15 and earlier versions, only containers in **spec.containers** are counted. In kubelet 1.16 and later versions, containers in both **spec.containers** and **spec.initContainers** are counted. The QoS class of a pod will change after the upgrade. As a result, the container in the pod restarts. You are advised to modify the QoS class of the service container before the upgrade to avoid this problem. For details, see :ref:`Table 1 `. + In kubelet 1.16 and later versions, `QoS classes `__ are different from those in earlier versions. In kubelet 1.15 and earlier versions, only containers in **spec.containers** are counted. In kubelet 1.16 and later versions, containers in both **spec.containers** and **spec.initContainers** are counted. The QoS class of a pod will change after the upgrade. As a result, the container in the pod restarts. You are advised to modify the QoS class of the service container before the upgrade to avoid this problem. For details, see :ref:`Table 1 `. - .. _cce_01_0302__table10713231143911: + .. _cce_10_0302__table10713231143911: .. table:: **Table 1** QoS class changes before and after the upgrade @@ -54,86 +56,10 @@ Notes and Constraints | Burstable | Guaranteed | Burstable | Yes | +----------------------------------------------------------+---------------------------------------------------------+-------------------------------------------------------------------+-----------------+ -Performing Pre-upgrade Check ----------------------------- - -Before upgrading a cluster, check the health status of the cluster and nodes and ensure that they are available. - -**Method 1: Use the console.** - -On the CCE console, click **Resource Management** in the navigation pane, and click **Clusters** and **Nodes** separately to check whether the cluster and nodes are normal. - -**Method 2: Run kubectl commands.** - -#. Run the following command to verify that all cluster modules are in the Healthy state: - - **kubectl get cs** - - Information similar to the following is displayed: - - .. code-block:: - - NAME STATUS MESSAGE ERROR - scheduler Healthy ok - controller-manager Healthy ok - etcd-0 Healthy {"health": "true"} - etcd-1 Healthy {"health": "true"} - etcd-2 Healthy {"health": "true"} - - .. note:: - - In the command output, the value of **STATUS** must be **Healthy** for all items. - -#. Run the following command to verify that all nodes are in the Ready state: - - **kubectl get nodes** - - .. note:: - - All nodes must be in the **Ready** state. - - .. code-block:: - - NAME STATUS ROLES AGE VERSION - xxx.xxx.xx.xx Ready 38d v1.9.7-r1 - xxx.xxx.xx.xx Ready 38d v1.9.7-r1 - xxx.xxx.xx.xx Ready 38d v1.9.7-r1 - -Pre-upgrade Checklist ---------------------- - -Before upgrading a cluster, follow the pre-upgrade checklist to identify risks and problems in advance. - -.. table:: **Table 2** Cluster upgrade check items - - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Module | Item | - +============+===============================================================================================================================================================================================================================================================================================================================================+ - | Cluster | Check whether the node IP addresses (including EIPs) of the current cluster are used in other configurations or whitelists. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Perform the pre-upgrade check. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Workload | Record the number and status of workloads for comparison after the upgrade. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | For the databases you use (such as Direct Connect, Redis, and MongoDB), you need to consider the changes in their whitelists, routes, or security group policies in advance. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage | Record the storage status to check whether storage resources are lost after the upgrade. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Networking | Check and back up the load balancing services and ingresses. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | If Direct Connect is used, check whether the upgrade causes changes in the IP addresses of nodes or pods where services are deployed. To handle changes, you need to enable routes on Direct Connect in advance. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add-on | When Kubernetes 1.9 is upgraded to 1.11, the kube-dns of the cluster is uninstalled and replaced with CoreDNS. Back up the DNS address configured in kube-dns so that you can use it in CoreDNS when the domain name resolution is abnormal. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | O&M | Private configurations: Check whether data plane passwords, certificates, and environment variables are configured for nodes or containers in the cluster before the upgrade. If a container is restarted (for example, the node is abnormal and the pod is re-scheduled), the configurations will be lost and your service will be abnormal. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | | Check and back up kernel parameters or system configurations. | - +------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - Upgrade Backup -------------- -Currently, there are two backup modes for cluster upgrade: +How to back up a node: - etcd database backup: CCE automatically backs up the etcd database during the cluster upgrade. - Master node backup (recommended, **manual confirmation required**): On the upgrade confirmation page, click **Backup** to back up the entire master node of the cluster. The backup process uses the Cloud Backup and Recovery (CBR) service and takes about 20 minutes. If there are many cloud backup tasks at the current site, the backup time may be prolonged. diff --git a/umn/source/clusters/upgrading_a_cluster/cce_kubernetes_release_notes.rst b/umn/source/clusters/upgrading_a_cluster/cce_kubernetes_release_notes.rst deleted file mode 100644 index 52d6dfc..0000000 --- a/umn/source/clusters/upgrading_a_cluster/cce_kubernetes_release_notes.rst +++ /dev/null @@ -1,60 +0,0 @@ -:original_name: cce_01_0068.html - -.. _cce_01_0068: - -CCE Kubernetes Release Notes -============================ - -CCE has passed the Certified Kubernetes Conformance Program and is a certified Kubernetes offering. To enable interoperability from one Kubernetes installation to the next, you must upgrade your Kubernetes clusters before the maintenance period ends. - -After the latest Kubernetes version is released, CCE will provide you the changes in this version. For details, see :ref:`Table 1 `. - -.. _cce_01_0068__table826812711586: - -.. table:: **Table 1** Cluster version differences - - +-----------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Source Version | Target Version | Description | - +=======================+=======================+==============================================================================================================================================================================================================================================================================================================================================+ - | v1.19 | v1.21 | - Changelog from v1.19 to v1.21 | - | | | | - | | | Changelog from v1.20 to v1.21: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md | - | | | | - | | | Changelog from v1.19 to v1.20: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.20.md | - +-----------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.17 | v1.19 | - Changelog from v1.17 to v1.19 | - | | | | - | | | Changelog from v1.18 to v1.19: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md | - | | | | - | | | Changelog from v1.17 to v1.18: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.18.md | - +-----------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.15 | v1.17 | - Changelog from v1.15 to v1.17 | - | | | | - | | | Changelog from v1.16 to v1.17: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.17.md | - | | | | - | | | Changelog from v1.15 to v1.16: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.16.md | - +-----------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.13 | v1.15 | - Changelog from v1.13 to v1.15 | - | | | | - | | | Changelog from v1.14 to v1.15: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.15.md | - | | | | - | | | Changelog from v1.13 to v1.14: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.14.md | - | | | | - | | | - After a cluster is upgraded from v1.13 to v1.15, the FlexVolume plug-in (storage-driver) is taken over by the CSI plug-in (everest v1.1.6 or later) for container storage. This takeover brings in no function changes, however, you are advised not to create FlexVolume storage resources any more, which will not work in the cluster. | - +-----------------------+-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/clusters/upgrading_a_cluster/index.rst b/umn/source/clusters/upgrading_a_cluster/index.rst index 2ddf35c..abb1088 100644 --- a/umn/source/clusters/upgrading_a_cluster/index.rst +++ b/umn/source/clusters/upgrading_a_cluster/index.rst @@ -1,24 +1,22 @@ -:original_name: cce_01_0215.html +:original_name: cce_10_0215.html -.. _cce_01_0215: +.. _cce_10_0215: Upgrading a Cluster =================== -- :ref:`Overview ` -- :ref:`Before You Start ` -- :ref:`Performing Replace/Rolling Upgrade (v1.13 and Earlier) ` -- :ref:`Performing In-place Upgrade (v1.15 and Later) ` -- :ref:`Migrating Services Across Clusters of Different Versions ` -- :ref:`CCE Kubernetes Release Notes ` +- :ref:`Upgrade Overview ` +- :ref:`Before You Start ` +- :ref:`Performing Replace/Rolling Upgrade ` +- :ref:`Performing In-place Upgrade ` +- :ref:`Migrating Services Across Clusters of Different Versions ` .. toctree:: :maxdepth: 1 :hidden: - overview + upgrade_overview before_you_start - performing_replace_rolling_upgrade_v1.13_and_earlier - performing_in-place_upgrade_v1.15_and_later + performing_replace_rolling_upgrade + performing_in-place_upgrade migrating_services_across_clusters_of_different_versions - cce_kubernetes_release_notes diff --git a/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst b/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst index 030d13e..fba3587 100644 --- a/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst +++ b/umn/source/clusters/upgrading_a_cluster/migrating_services_across_clusters_of_different_versions.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0210.html +:original_name: cce_10_0210.html -.. _cce_01_0210: +.. _cce_10_0210: Migrating Services Across Clusters of Different Versions ======================================================== @@ -38,15 +38,15 @@ Procedure #. **Create a CCE cluster.** - Create a cluster with the same specifications and configurations as the cluster of the earlier version. For details, see :ref:`Creating a CCE Cluster `. + Create a cluster with the same specifications and configurations as the cluster of the earlier version. For details, see :ref:`Creating a CCE Cluster `. #. **Add a node.** - Add nodes with the same specifications and manual configuration items. For details, see :ref:`Creating a Node `. + Add nodes with the same specifications and manual configuration items. For details, see :ref:`Creating a Node `. #. **Create a storage volume in the new cluster.** - Use an existing storage volume to create a PVC in the new cluster. The PVC name remains unchanged. For details, see :ref:`PersistentVolumeClaims (PVCs) `. + Use an existing storage volume to create a PVC in the new cluster. The PVC name remains unchanged. For details, see :ref:`PersistentVolumeClaims (PVCs) `. .. note:: @@ -54,11 +54,11 @@ Procedure #. **Create a workload in the new cluster.** - The workload name and specifications remain unchanged. For details about how to create a workload, see :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `. For details about how to mount a storage volume to the workload, see :ref:`Creating a Pod Mounted with an EVS Volume `. + The workload name and specifications remain unchanged. For details about how to create a workload, see :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `. For details about how to attach a storage volume to the workload, see :ref:`Creating a Deployment Mounted with an EVS Volume `. #. **Create a Service in the new cluster.** - The Service name and specifications remain unchanged. For details about how to create a Service, see :ref:`Services `. + The Service name and specifications remain unchanged. For details about how to create a Service, see :ref:`Services `. #. **Commission services.** @@ -66,4 +66,4 @@ Procedure #. **Delete the old cluster.** - When all functions of the new cluster are stable, delete the old cluster. For details about how to delete a cluster, see :ref:`Deleting a Cluster `. + When all functions of the new cluster are stable, delete the old cluster. For details about how to delete a cluster, see :ref:`Deleting a Cluster `. diff --git a/umn/source/clusters/upgrading_a_cluster/overview.rst b/umn/source/clusters/upgrading_a_cluster/overview.rst deleted file mode 100644 index 825d3bd..0000000 --- a/umn/source/clusters/upgrading_a_cluster/overview.rst +++ /dev/null @@ -1,123 +0,0 @@ -:original_name: cce_01_0197.html - -.. _cce_01_0197: - -Overview -======== - -To enable interoperability from one Kubernetes installation to the next, you must upgrade your Kubernetes clusters before the maintenance period ends. - -After the latest Kubernetes version is available in CCE, CCE will describe the changes in this version. - -You can use the CCE console to upgrade the Kubernetes version of a cluster. - -An upgrade flag will be displayed on the cluster card view if there is a new version for the cluster to upgrade. - -**How to check:** - -Choose **Resource Management** > **Clusters** and check whether there is an upgrade flag in the upper right corner of the cluster card view. If yes, the cluster can be upgraded. - - -.. figure:: /_static/images/en-us_image_0000001190048341.png - :alt: **Figure 1** Cluster with the upgrade flag - - **Figure 1** Cluster with the upgrade flag - -Cluster Upgrade ---------------- - -The following table describes the target version to which each cluster version can be upgraded, the supported upgrade modes, and upgrade impacts. - -.. table:: **Table 1** Cluster upgrade paths and impacts - - +-----------------+---------------------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Source Version | Target Version | Upgrade Modes | Impacts | - +=================+===================================================+==================+============================================================================================================================================================================================================================================================+ - | v1.21 | v1.23 | In-place upgrade | You need to identify the differences between versions. | - +-----------------+---------------------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.19 | v1.21 | In-place upgrade | You need to identify the differences between versions. | - +-----------------+---------------------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.17 | v1.19 | In-place upgrade | You need to identify the differences between versions. | - | | | | | - | v1.15 | | | | - +-----------------+---------------------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.13 | v1.15 | Rolling upgrade | - The **proxy** configuration item in the coredns add-on configuration is not supported and needs to be replaced with **forward**. | - | | | | - The storage add-on is changed from storage-driver to everest. | - | | | Replace upgrade | | - +-----------------+---------------------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.11 | 1.15 | Replace upgrade | - The cluster signature certificate mechanism is changed. As a result, the original cluster certificate becomes invalid. You need to obtain the certificate or kubeconfig file again after the cluster is upgraded. | - | | | | - RBAC is enabled for clusters of Kubernetes v1.13 by default. Applications need to adapt to RBAC. | - | v1.9 | | | - After the cluster is upgraded from v1.9 to v1.15, kube-dns in the cluster will be replaced with CoreDNS. Before the upgrade, you need to back up the kube-dns configuration. After the upgrade, you need to reconfigure kube-dns in the coredns add-on. | - +-----------------+---------------------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.9 | Latest version that can be created on the console | Migration | You need to identify the differences between versions. | - | | | | | - | v1.7 | | | | - +-----------------+---------------------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -Upgrade Modes -------------- - -CCE provides the following upgrade modes based on the cluster version and deployment site. The upgrade processes are the same for master nodes. The differences between the upgrade modes of worker nodes are described as follows: - -.. table:: **Table 2** Differences between upgrade modes and their advantages and disadvantages - - +----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Upgrade Mode | Method | Advantage | Disadvantage | - +======================+===========================================================================================================================================================================================================================================================================================================================================================================================================================================+=========================================================================+===========================================================================================================================================================================================================+ - | **In-place upgrade** | Kubernetes components, network components, and CCE management components are upgraded on the node. During the upgrade, service pods and networks are not affected. The **SchedulingDisabled** label will be added to all existing nodes. After the upgrade is complete, you can properly use existing nodes. | You do not need to migrate services, ensuring service continuity. | In-place upgrade does not upgrade the OS of a node. If you want to upgrade the OS, clear the corresponding node after the node upgrade is complete and reset the node to upgrade the OS to a new version. | - +----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Rolling upgrade** | Only the Kubernetes components and certain network components are upgraded on the node. The **SchedulingDisabled** label will be added to all existing nodes to ensure that the running applications are not affected. **After the upgrade is complete, you need to manually create nodes and gradually release the old nodes**, thereby migrating your applications to the new nodes. In this mode, you can control the upgrade process. | Services are not interrupted. | ``-`` | - +----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Replace upgrade** | The latest worker node image is used to reset the node OS. | This is the fastest upgrade mode and requires few manual interventions. | Data or configurations on the node will be lost, and services will be interrupted for a period of time. | - +----------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -.. _cce_01_0197__section16738338445: - -Cluster Upgrade Between Major Versions --------------------------------------- - -.. table:: **Table 3** Changelog between minor versions - - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ - | Source Version | Target Version | Description | - +=======================+=======================+=====================================================================================+ - | v1.21 | v1.23 | - Changelog from v1.21 to v1.23 | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.23.md | - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ - | v1.19 | v1.21 | - Changelog from v1.19 to v1.21 | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.21.md | - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ - | v1.17 | v1.19 | - Changelog from v1.17 to v1.19 | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.19.md | - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ - | v1.15 | v1.17 | - Changelog from v1.15 to v1.17 | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.17.md | - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ - | v1.13 | v1.15 | - Changelog from v1.9 to v1.15 | - | | | | - | | | Changelog from v1.13 to v1.15: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.15.md | - | | | | - | | | Changelog from v1.11 to v1.13: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.13.md | - | | | | - | | | Changelog from v1.10 to v1.11: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.11.md | - | | | | - | | | Changelog from v1.9 to v1.10: | - | | | | - | | | https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.10.md | - | | | | - | | | - Replacement of cluster kube-dns by core-dns | - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ - | v1.11 | | | - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ - | v1.9 | | | - +-----------------------+-----------------------+-------------------------------------------------------------------------------------+ diff --git a/umn/source/clusters/upgrading_a_cluster/performing_in-place_upgrade.rst b/umn/source/clusters/upgrading_a_cluster/performing_in-place_upgrade.rst new file mode 100644 index 0000000..f4c648a --- /dev/null +++ b/umn/source/clusters/upgrading_a_cluster/performing_in-place_upgrade.rst @@ -0,0 +1,79 @@ +:original_name: cce_10_0301.html + +.. _cce_10_0301: + +Performing In-place Upgrade +=========================== + +Scenario +-------- + +You can upgrade your clusters to a newer version on the CCE console. + +Before the upgrade, learn about the target version to which each CCE cluster can be upgraded in what ways, and the upgrade impacts. For details, see :ref:`Upgrade Overview ` and :ref:`Before You Start `. + +Description +----------- + +- An in-place upgrade updates the Kubernetes components on cluster nodes, without changing their OS version. +- Data plane nodes are upgraded in batches. By default, they are prioritized based on their CPU, memory, and `PodDisruptionBudgets (PDBs) `__. You can also set the priorities according to your service requirements. + +Precautions +----------- + +- During the cluster upgrade, the system will automatically upgrade add-ons to a version compatible with the target cluster version. Do not uninstall or reinstall add-ons during the cluster upgrade. +- Before the upgrade, ensure that all add-ons are running. If an add-on fails to be upgraded, rectify the fault and try again. +- During the upgrade, CCE checks the add-on running status. Some add-ons (such as coredns) require at least two nodes to run normally. In this case, at least two nodes must be available for the upgrade. + +For more information, see :ref:`Before You Start `. + +Procedure +--------- + +This section describes how to upgrade a CCE cluster of v1.15 or later. For other versions, see :ref:`Performing Replace/Rolling Upgrade `. + +#. Log in to the CCE console and click the cluster name to access the cluster. + +#. In the navigation pane, choose **Cluster Upgrade**. You can view the new version available for upgrade on the right. + + Check the version information, last update/upgrade time, available upgrade version, and upgrade history of the current cluster. + + The cluster upgrade goes through pre-upgrade check, add-on upgrade/uninstallation, master node upgrade, worker node upgrade, and post-upgrade processing. + + .. note:: + + - If your cluster version is up-to-date, the **Upgrade** button is grayed out. + - If your cluster status is abnormal or there are abnormal add-ons, the **Upgrade** button is dimmed. Perform a check by referring to :ref:`Before You Start `. + +#. Click **Upgrade** or **Install Patch** on the right. Set the upgrade parameters. + + - **New Version**: Select the Kubernetes version to which the cluster can be upgraded. + + - (Optional) **Cluster Backup**: Determine whether to back up the entire master node. This backup mode is recommended. + + A manual confirmation is required for backing up the entire master node. The backup process uses the Cloud Backup and Recovery (CBR) service and takes about 20 minutes. If there are many cloud backup tasks at the current site, the backup time may be prolonged. You are advised to back up the master node. + + - **Add-on Upgrade Configuration**: Add-ons that have been installed in your cluster are listed. During the cluster upgrade, the system automatically upgrades the add-ons to be compatible with the target cluster version. You can click **Set** to re-define the add-on parameters. + + .. note:: + + If a red dot |image1| is displayed on the right of an add-on, the add-on is incompatible with the target cluster version. During the upgrade, the add-on will be uninstalled and then re-installed. Ensure that the add-on parameters are correctly configured. + + - **Node Upgrade Configuration**: Before setting the node upgrade priority, you need to select a node pool. Nodes and node pools will be upgraded according to the priorities you specify. You can set the maximum number of nodes to be upgraded in a batch, or set priorities for nodes to be upgraded. If you do not set this parameter, the system will determine the nodes to upgrade in batches based on specific conditions. + + - **Add Upgrade Priority**: Add upgrade priorities for node pools. + - **Add Node Priority**: After adding a node pool priority, you can set the upgrade sequence of nodes in the node pool. The system upgrades nodes in the sequence you specify. If you skip this setting, the system upgrades nodes based on the default policy. + +#. Read the upgrade instructions carefully, and select **I have read and agree to Upgrade Precautions**. Click **Upgrade**. + +#. After you click **Upgrade**, the cluster upgrade starts. You can view the upgrade process in the lower part of the page. + + During the upgrade, you can click **Suspend** on the right to suspend the cluster upgrade. To continue the upgrade, click **Continue**. + + If an upgrade failure message is displayed during the cluster upgrade, rectify the fault as prompted and try again. + +#. When the upgrade progress reaches 100%, the cluster is upgraded. The version information will be properly displayed, and no upgrade is required. + +#. After the upgrade is complete, verify the cluster Kubernetes version on the **Clusters** page. + +.. |image1| image:: /_static/images/en-us_image_0000001244101223.png diff --git a/umn/source/clusters/upgrading_a_cluster/performing_in-place_upgrade_v1.15_and_later.rst b/umn/source/clusters/upgrading_a_cluster/performing_in-place_upgrade_v1.15_and_later.rst deleted file mode 100644 index cd9c070..0000000 --- a/umn/source/clusters/upgrading_a_cluster/performing_in-place_upgrade_v1.15_and_later.rst +++ /dev/null @@ -1,125 +0,0 @@ -:original_name: cce_01_0301.html - -.. _cce_01_0301: - -Performing In-place Upgrade (v1.15 and Later) -============================================= - -Scenario --------- - -On the CCE console, You can perform an in-place cluster upgrade to use new cluster features. - -Before the upgrade, learn about the target version to which each CCE cluster can be upgraded in what ways, and the upgrade impacts. For details, see :ref:`Overview ` and :ref:`Before You Start `. - -Description ------------ - -- An in-place upgrade updates the Kubernetes components on cluster nodes, without changing their OS version. -- Data plane nodes are upgraded in batches. By default, they are prioritized based on their CPU, memory, and `PodDisruptionBudgets (PDBs) `__. You can also set the priorities according to your service requirements. - -Precautions ------------ - -- During the cluster upgrade, the system will automatically upgrade add-ons to a version compatible with the target cluster version. Do not uninstall or reinstall add-ons during the cluster upgrade. -- Before the upgrade, ensure that all add-ons are running. If an add-on fails to be upgraded, rectify the fault and try again. -- During the upgrade, CCE checks the add-on running status. Some add-ons (such as coredns) require at least two nodes to run normally. In this case, at least two nodes must be available for the upgrade. - -For more information, see :ref:`Before You Start `. - -Procedure ---------- - -This section describes how to upgrade a CCE cluster of v1.15 or later. For other versions, see :ref:`Performing Replace/Rolling Upgrade (v1.13 and Earlier) `. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. In the cluster list, view the cluster version. - -#. Click **More** for the cluster you want to upgrade, and select **Upgrade** from the drop-down menu. - - - .. figure:: /_static/images/en-us_image_0000001229793402.png - :alt: **Figure 1** Upgrading a cluster - - **Figure 1** Upgrading a cluster - - .. note:: - - - If your cluster version is up-to-date, the **Upgrade** button is grayed out. - - If the cluster status is **Unavailable**, the upgrade flag in the upper right corner of the cluster card view will be grayed out. Check the cluster status by referring to :ref:`Before You Start `. - -#. (Optional) On the cluster upgrade confirmation page, click **Backup** to back up the entire master node. This backup mode is recommended. - - A manual confirmation is required for backing up the entire master node. The backup process uses the Cloud Backup and Recovery (CBR) service and takes about 20 minutes. If there are many cloud backup tasks at the current site, the backup time may be prolonged. You are advised to back up the master node. - - - .. figure:: /_static/images/en-us_image_0000001280171657.png - :alt: **Figure 2** Determining whether to back up the entire master node - - **Figure 2** Determining whether to back up the entire master node - -#. Check the version information, last update/upgrade time, available upgrade version, and upgrade history of the current cluster. - - The cluster upgrade goes through pre-upgrade check, add-on upgrade/uninstallation, master node upgrade, worker node upgrade, and post-upgrade processing. - - - .. figure:: /_static/images/en-us_image_0000001274316069.png - :alt: **Figure 3** Cluster upgrade page - - **Figure 3** Cluster upgrade page - -#. Click **Upgrade** on the right. Set the upgrade parameters. - - - **Available Versions**: Select v1.19 in this example. - - **Cluster Backup**: A manual confirmation is required for backing up the entire master node. The backup process uses the Cloud Backup and Recovery (CBR) service and takes about 20 minutes. If there are many cloud backup tasks at the current site, the backup time may be prolonged. - - **Add-on Upgrade Configuration**: Add-ons that have been installed in your cluster are listed. During the cluster upgrade, the system automatically upgrades the add-ons to be compatible with the target cluster version. You can click **Set** to re-define the add-on parameters. - - .. note:: - - If a red dot |image1| is displayed on the right of an add-on, the add-on is incompatible with the target cluster version. During the upgrade, the add-on will be uninstalled and then re-installed. Ensure that the add-on parameters are correctly configured. - - - **Node Upgrade Configuration**: Before setting the node upgrade priority, you need to select a node pool. Nodes and node pools will be upgraded according to the priorities you specify. You can set the maximum number of nodes to be upgraded in batch, or set priorities for nodes to be upgraded. If you do not set this parameter, the system will determine the nodes to upgrade in batches based on specific conditions. - - - **Add Upgrade Priority**: Add upgrade priorities for node pools. - - **Add Node Priority**: After adding a node pool priority, you can set the upgrade sequence of nodes in the node pool. The system upgrades nodes in the sequence you specify. If you skip this setting, the system upgrades nodes based on the default policy. - - - .. figure:: /_static/images/en-us_image_0000001229794946.png - :alt: **Figure 4** Configuring upgrade parameters - - **Figure 4** Configuring upgrade parameters - -#. Read the upgrade instructions carefully, and select **I have read the upgrade instructions**. Click **Upgrade**. - - - .. figure:: /_static/images/en-us_image_0000001280421317.png - :alt: **Figure 5** Final step before upgrade - - **Figure 5** Final step before upgrade - -#. After you click **Upgrade**, the cluster upgrade starts. You can view the upgrade process in the lower part of the page. - - During the upgrade, you can click **Suspend** on the right to suspend the cluster upgrade. To continue the upgrade, click **Continue**. - - - .. figure:: /_static/images/en-us_image_0000001280181541.png - :alt: **Figure 6** Cluster upgrade in process - - **Figure 6** Cluster upgrade in process - -#. When the upgrade progress reaches 100%, the cluster is upgraded. The version information will be properly displayed, and no upgrade is required. - - - .. figure:: /_static/images/en-us_image_0000001236582394.png - :alt: **Figure 7** Upgrade completed - - **Figure 7** Upgrade completed - -#. After the upgrade is complete, verify the cluster Kubernetes version on the **Clusters** page. - - - .. figure:: /_static/images/en-us_image_0000001236263298.png - :alt: **Figure 8** Verifying the upgrade success - - **Figure 8** Verifying the upgrade success - -.. |image1| image:: /_static/images/en-us_image_0000001159118361.png diff --git a/umn/source/clusters/upgrading_a_cluster/performing_replace_rolling_upgrade_v1.13_and_earlier.rst b/umn/source/clusters/upgrading_a_cluster/performing_replace_rolling_upgrade.rst similarity index 69% rename from umn/source/clusters/upgrading_a_cluster/performing_replace_rolling_upgrade_v1.13_and_earlier.rst rename to umn/source/clusters/upgrading_a_cluster/performing_replace_rolling_upgrade.rst index 1fe097d..a8c4c29 100644 --- a/umn/source/clusters/upgrading_a_cluster/performing_replace_rolling_upgrade_v1.13_and_earlier.rst +++ b/umn/source/clusters/upgrading_a_cluster/performing_replace_rolling_upgrade.rst @@ -1,25 +1,25 @@ -:original_name: cce_01_0120.html +:original_name: cce_10_0120.html -.. _cce_01_0120: +.. _cce_10_0120: -Performing Replace/Rolling Upgrade (v1.13 and Earlier) -====================================================== +Performing Replace/Rolling Upgrade +================================== Scenario -------- You can upgrade your clusters to a newer Kubernetes version on the CCE console. -Before the upgrade, learn about the target version to which each CCE cluster can be upgraded in what ways, and the upgrade impacts. For details, see :ref:`Overview ` and :ref:`Before You Start `. +Before the upgrade, learn about the target version to which each CCE cluster can be upgraded in what ways, and the upgrade impacts. For details, see :ref:`Upgrade Overview ` and :ref:`Before You Start `. Precautions ----------- -- If the coredns add-on needs to be upgraded during the cluster upgrade, ensure that the number of nodes is greater than or equal to the number of coredns instances and all coredns instances are running. Otherwise, the upgrade will fail. Before upgrading a cluster of v1.11 or v1.13, you need to upgrade the coredns add-on to the latest version available for the cluster. +- If the coredns add-on needs to be upgraded during the cluster upgrade, ensure that the number of nodes is greater than or equal to the number of coredns instances and all coredns instances are running. Otherwise, the upgrade will fail. Before upgrading a cluster of v1.13, you need to upgrade the coredns add-on to the latest version available for the cluster. - When a cluster of v1.11 or earlier is upgraded to v1.13, the impacts on the cluster are as follows: - All cluster nodes will be restarted as their OSs are upgraded, which affects application running. - - The cluster signing certificate mechanism is changed. As a result, the original cluster certificate becomes invalid. You need to obtain the certificate or kubeconfig file again after the cluster is upgraded. + - The cluster signature certificate mechanism is changed. As a result, the original cluster certificate becomes invalid. You need to obtain the certificate or kubeconfig file again after the cluster is upgraded. - During the upgrade from one release of v1.13 to a later release of v1.13, applications in the cluster are interrupted for a short period of time only during the upgrade of network components. - During the upgrade from Kubernetes 1.9 to 1.11, the kube-dns of the cluster will be uninstalled and replaced with CoreDNS, which may cause loss of the cascading DNS configuration in the kube-dns or temporary interruption of the DNS service. Back up the DNS address configured in the kube-dns so you can configure the domain name in the CoreDNS again when domain name resolution is abnormal. @@ -27,14 +27,14 @@ Precautions Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Clusters**. In the cluster list, check the cluster version. +#. Log in to the CCE console and click the cluster name to access the cluster. -#. Click **More** for the cluster you want to upgrade, and select **Upgrade** from the drop-down menu. +#. In the navigation pane, choose **Cluster Upgrade**. You can view the new version available for upgrade on the right. Click **Upgrade**. .. note:: - If your cluster version is up-to-date, the **Upgrade** button is grayed out. - - If the cluster status is **Unavailable**, the upgrade flag in the upper right corner of the cluster card view will be grayed out. Check the cluster status by referring to :ref:`Before You Start `. + - If your cluster status is abnormal or there are abnormal add-ons, the **Upgrade** button is dimmed. Perform a check by referring to :ref:`Before You Start `. #. In the displayed **Pre-upgrade Check** dialog box, click **Check Now**. @@ -42,39 +42,41 @@ Procedure #. When the status of the pre-upgrade check is **Completed**, click **Upgrade**. -#. On the cluster upgrade page, review or configure basic information by referring to :ref:`Table 1 `. +#. On the cluster upgrade page, review or configure basic information by referring to :ref:`Table 1 `. - .. _cce_01_0120__table924319911495: + .. _cce_10_0120__table924319911495: .. table:: **Table 1** Basic information - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===================================================================================================================================================================================================================+ - | Cluster Name | Review the name of the cluster to be upgraded. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Current Version | Review the version of the cluster to be upgraded. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Target Version | Review the target version after the upgrade. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Node Upgrade Policy | **Replace** (replace upgrade): Worker nodes will be reset. Their OSs will be reinstalled, and data on the system and data disks will be cleared. Exercise caution when performing this operation. | - | | | - | | .. note:: | - | | | - | | - The lifecycle management function of the nodes and workloads in the cluster is unavailable. | - | | - APIs cannot be called temporarily. | - | | - Running workloads will be interrupted because nodes are reset during the upgrade. | - | | - Data in the system and data disks on the worker nodes will be cleared. Back up important data before resetting the nodes. | - | | - Data disks without LVM mounted to worker nodes need to be mounted again after the upgrade, and data on the disks will not be lost during the upgrade. | - | | - The EVS disk quota must be greater than 0. | - | | - The container IP addresses change, but the communication between containers is not affected. | - | | - Custom labels on the worker nodes will be cleared. | - | | - It takes about 20 minutes to upgrade a master node and about 30 to 120 minutes to upgrade worker nodes (about 3 minutes for each worker node), depending on the number of worker nodes and upgrade batches. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Login Mode | - **Key pair**: Select the key pair used to log in to the node. You can select a shared key. | - | | | - | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+===================================================================================================================================================================================================+ + | Cluster Name | Review the name of the cluster to be upgraded. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Current Version | Review the version of the cluster to be upgraded. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Target Version | Review the target version after the upgrade. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Node Upgrade Policy | **Replace** (replace upgrade): Worker nodes will be reset. Their OSs will be reinstalled, and data on the system and data disks will be cleared. Exercise caution when performing this operation. | + | | | + | | .. note:: | + | | | + | | - The lifecycle management function of the nodes and workloads in the cluster is unavailable. | + | | - APIs cannot be called temporarily. | + | | - Running workloads will be interrupted because nodes are reset during the upgrade. | + | | - Data in the system and data disks on the worker nodes will be cleared. Back up important data before resetting the nodes. | + | | - Data disks without LVM mounted to worker nodes need to be mounted again after the upgrade, and data on the disks will not be lost during the upgrade. | + | | - The EVS disk quota must be greater than 0. | + | | - The container IP addresses change, but the communication between containers is not affected. | + | | - Custom labels on the worker nodes will be cleared. | + | | - It takes about 12 minutes to complete the cluster upgrade. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Login Mode | **Key Pair** | + | | | + | | Select the key pair used to log in to the node. You can select a shared key. | + | | | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Click **Next**. In the dialog box displayed, click **OK**. diff --git a/umn/source/clusters/upgrading_a_cluster/upgrade_overview.rst b/umn/source/clusters/upgrading_a_cluster/upgrade_overview.rst new file mode 100644 index 0000000..6dcb5f3 --- /dev/null +++ b/umn/source/clusters/upgrading_a_cluster/upgrade_overview.rst @@ -0,0 +1,123 @@ +:original_name: cce_10_0197.html + +.. _cce_10_0197: + +Upgrade Overview +================ + +To enable interoperability from one Kubernetes installation to the next, you must upgrade your Kubernetes clusters before the maintenance period ends. + +After the latest Kubernetes version is available in CCE, CCE will describe the changes in this version. + +You can use the CCE console to upgrade the Kubernetes version of a cluster. + +An upgrade flag will be displayed on the cluster card view if there is a new version for the cluster to upgrade. + +**How to check:** + +Log in to the CCE console and check whether the message "New version available" is displayed in the lower left corner of the cluster. If yes, the cluster can be upgraded. If no, the cluster cannot be upgraded. + + +.. figure:: /_static/images/en-us_image_0000001482796460.png + :alt: **Figure 1** Cluster with the upgrade flag + + **Figure 1** Cluster with the upgrade flag + +.. _cce_10_0197__section19981121648: + +Cluster Upgrade +--------------- + +The following table describes the target version to which each cluster version can be upgraded, the supported upgrade modes, and upgrade impacts. + +.. table:: **Table 1** Cluster upgrade paths and impacts + + +-----------------+-----------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Source Version | Target Version | Upgrade Modes | Impacts | + +=================+=================+==================+================================================================================================================================================================+ + | v1.19 | v1.21 | In-place upgrade | You need to learn about the differences between versions. For details, see :ref:`Precautions for Major Version Upgrade `. | + +-----------------+-----------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | v1.17 | v1.19 | In-place upgrade | You need to learn about the differences between versions. For details, see :ref:`Precautions for Major Version Upgrade `. | + | | | | | + | v1.15 | | | | + +-----------------+-----------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | v1.13 | v1.15 | Rolling upgrade | - **proxy** in the coredns add-on cannot be configured and needs to be replaced with **forward**. | + | | | | - The storage add-on is changed from storage-driver to everest. | + | | | Replace upgrade | | + +-----------------+-----------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +Upgrade Modes +------------- + +The upgrade processes are the same for master nodes. The differences between the upgrade modes of worker nodes are described as follows: + +.. table:: **Table 2** Differences between upgrade modes and their advantages and disadvantages + + +----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Upgrade Mode | Method | Advantage | Disadvantage | + +======================+==============================================================================================================================================================================================================================================================================================================+=========================================================================+=============================================================================================================================================================================================================================================+ + | **In-place upgrade** | Kubernetes components, network components, and CCE management components are upgraded on the node. During the upgrade, service pods and networks are not affected. The **SchedulingDisabled** label will be added to all existing nodes. After the upgrade is complete, you can properly use existing nodes. | You do not need to migrate services, ensuring service continuity. | In-place upgrade does not upgrade the OS of a node. If you want to upgrade the OS, clear the corresponding node data after the node upgrade is complete and reset the node to upgrade the OS to a new version. | + +----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Rolling upgrade** | Only the Kubernetes components and certain network components are upgraded on the node. The **SchedulingDisabled** label will be added to all existing nodes to ensure that the running applications are not affected. | Services are not interrupted. | - **After the upgrade is complete, you need to manually create nodes and gradually release the old nodes.** The new nodes are billed additionally. After services are migrated to the new nodes, the old nodes can be deleted. | + | | | | | + | | .. important:: | | - After the rolling upgrade is complete, if you want to continue the upgrade to a later version, you need to reset the old nodes first. Otherwise, the pre-upgrade check cannot be passed. Services may be interrupted during the upgrade. | + | | | | | + | | NOTICE: | | | + | | | | | + | | - **After the upgrade is complete, you need to manually create nodes and gradually release the old nodes**, thereby migrating your applications to the new nodes. In this mode, you can control the upgrade process. | | | + +----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Replace upgrade** | The latest worker node image is used to reset the node OS. | This is the fastest upgrade mode and requires few manual interventions. | Data or configurations on the node will be lost, and services will be interrupted for a period of time. | + +----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +.. _cce_10_0197__section191131551162610: + +Precautions for Major Version Upgrade +------------------------------------- + ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Upgrade Path | Difference | Self-Check | ++=======================+==============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ +| v1.19 to v1.21 | The bug of **exec probe timeouts** is fixed in Kubernetes 1.21. Before this bug fix, the exec probe does not consider the **timeoutSeconds** field. Instead, the probe will run indefinitely, even beyond its configured deadline. It will stop until the result is returned. If this field is not specified, the default value **1** is used. This field takes effect after the upgrade. If the probe runs over 1 second, the application health check may fail and the application may restart frequently. | Before the upgrade, check whether the timeout is properly set for the exec probe. | ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| | kube-apiserver of CCE 1.19 or later requires that the Subject Alternative Names (SANs) field be configured for the certificate of your webhook server. Otherwise, kube-apiserver fails to call the webhook server after the upgrade, and containers cannot be started properly. | Before the upgrade, check whether the SAN field is configured in the certificate of your webhook server. | +| | | | +| | Root cause: X.509 `CommonName `__ is discarded in Go 1.15. kube-apiserver of CCE 1.19 is compiled using Go 1.15. If your webhook certificate does not have SANs, kube-apiserver does not process the **CommonName** field of the X.509 certificate as the host name by default. As a result, the authentication fails. | - If you do not have your own webhook server, you can skip this check. | +| | | - If the field is not set, you are advised to use the SAN field to specify the IP address and domain name supported by the certificate. | ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| v1.15 to v1.19 | The control plane of CCE 1.19 is incompatible with Kubelet 1.15. If the master node fails to be upgraded or the node to be upgraded restarts after the master node is successfully upgraded, there is a high probability that the node is in the **NotReady** status. | #. In normal cases, this scenario is not triggered. | +| | | #. After the master node is upgraded, do not suspend the upgrade. Upgrade the node quickly. | +| | There is a high probability that kubelet restarts on the node that fails to be upgraded, triggering the node registration process. The default registration labels of kubelet 1.15 (**failure-domain.beta.kubernetes.io/is-baremetal** and **kubernetes.io/availablezone**) are regarded as an invalid label by kube-apiserver 1.19. | #. If a node fails to be upgraded and cannot be restored, evict applications on the node as soon as possible. Contact technical support and skip the node upgrade. After the upgrade is complete, reset the node. | +| | | | +| | The valid labels in v1.19 are **node.kubernetes.io/baremetal** and **failure-domain.beta.kubernetes.io/zone**. | | ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| | In CCE 1.15 and 1.19 clusters, the Docker storage driver file system is switched from XFS to Ext4. As a result, the import package sequence in the pods of the upgraded Java application may be abnormal, causing pod exceptions. | Before the upgrade, check the Docker configuration file **/etc/docker/daemon.json** on the node. Check whether the value of **dm.fs** is **xfs**. | +| | | | +| | | - If the value is **ext4** or the storage driver is Overlay, you can skip the next steps. | +| | | - If the value is **xfs**, you are advised to deploy applications in the cluster of the new version in advance to test whether the applications are compatible with the new cluster version. | +| | | | +| | | .. code-block:: | +| | | | +| | | { | +| | | "storage-driver": "devicemapper", | +| | | "storage-opts": [ | +| | | "dm.thinpooldev=/dev/mapper/vgpaas-thinpool", | +| | | "dm.use_deferred_removal=true", | +| | | "dm.fs=xfs", | +| | | "dm.use_deferred_deletion=true" | +| | | ] | +| | | } | ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| | kube-apiserver of CCE 1.19 or later requires that the Subject Alternative Names (SANs) field be configured for the certificate of your webhook server. Otherwise, kube-apiserver fails to call the webhook server after the upgrade, and containers cannot be started properly. | Before the upgrade, check whether the SAN field is configured in the certificate of your webhook server. | +| | | | +| | Root cause: X.509 `CommonName `__ is discarded in Go 1.15. kube-apiserver of CCE 1.19 is compiled using Go 1.15. The **CommonName** field is processed as the host name. As a result, the authentication fails. | - If you do not have your own webhook server, you can skip this check. | +| | | - If the field is not set, you are advised to use the SAN field to specify the IP address and domain name supported by the certificate. | +| | | | +| | | .. important:: | +| | | | +| | | NOTICE: | +| | | To mitigate the impact of version differences on cluster upgrade, CCE performs special processing during the upgrade from 1.15 to 1.19 and still supports certificates without SANs. However, no special processing is required for subsequent upgrades. You are advised to rectify your certificate as soon as possible. | ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| | In clusters of v1.17.17 and later, CCE automatically creates pod security policies (PSPs) for you, which restrict the creation of pods with unsafe configurations, for example, pods for which **net.core.somaxconn** under a sysctl is configured in the security context. | After an upgrade, you can allow insecure system configurations as required. For details, see :ref:`Configuring a Pod Security Policy `. | ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| v1.13 to v1.15 | After a VPC network cluster is upgraded, the master node occupies an extra CIDR block due to the upgrade of network components. If no container CIDR block is available for the new node, the pod scheduled to the node cannot run. | Generally, this problem occurs when the nodes in the cluster are about to fully occupy the container CIDR block. For example, the container CIDR block is 10.0.0.0/16, the number of available IP addresses is 65,536, and the VPC network is allocated a CIDR block with the fixed size (using the mask to determine the maximum number of container IP addresses allocated to each node). If the upper limit is 128, the cluster supports a maximum of 512 (65536/128) nodes, including the three master nodes. After the cluster is upgraded, each of the three master nodes occupies one CIDR block. As a result, 506 nodes are supported. | ++-----------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst index 7b4a4de..6a3e620 100644 --- a/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst +++ b/umn/source/clusters/using_kubectl_to_run_a_cluster/common_kubectl_commands.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0139.html +:original_name: cce_10_0139.html -.. _cce_01_0139: +.. _cce_10_0139: Common kubectl Commands ======================= @@ -88,7 +88,7 @@ The **expose** command exposes a resource as a new Kubernetes service. Possible .. note:: - The example command creates a service of NodePort type for the deployment with the name specified in **deployname**. The service will serve on port 81 specified in **-port** and connect to the containers on port 80 specified in **-target-port**. More specifically, the service is reachable at :, and containers are reachable at :. + In the preceding command, **--port** indicates the port exposed by the Service, **--type** indicates the Service type, and **--target-port** indicates the port of the pod backing the Service. Visiting *ClusterIP*:*Port* allows you to access the applications in the cluster. **run** @@ -325,7 +325,7 @@ To listen on ports 5000 and 6000 locally, forwarding data to/from ports 5000 and .. code-block:: - kubectl port -forward podname 5000:6000 + kubectl port-forward podname 5000:6000 **proxy\*** diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/connecting_to_a_cluster_using_kubectl.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/connecting_to_a_cluster_using_kubectl.rst index b792257..0be6fa1 100644 --- a/umn/source/clusters/using_kubectl_to_run_a_cluster/connecting_to_a_cluster_using_kubectl.rst +++ b/umn/source/clusters/using_kubectl_to_run_a_cluster/connecting_to_a_cluster_using_kubectl.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0107.html +:original_name: cce_10_0107.html -.. _cce_01_0107: +.. _cce_10_0107: Connecting to a Cluster Using kubectl ===================================== @@ -15,52 +15,94 @@ Permission Description When you access a cluster using kubectl, CCE uses the **kubeconfig.json** file generated on the cluster for authentication. This file contains user information, based on which CCE determines which Kubernetes resources can be accessed by kubectl. The permissions recorded in a **kubeconfig.json** file vary from user to user. -For details about user permissions, see :ref:`Cluster Permissions (IAM-based) and Namespace Permissions (Kubernetes RBAC-based) `. +For details about user permissions, see :ref:`Cluster Permissions (IAM-based) and Namespace Permissions (Kubernetes RBAC-based) `. Using kubectl ------------- -**Background** - -To connect a client to a Kubernetes cluster, you can use kubectl. For details, see `Install Tools `__. - -**Prerequisites** +To connect to a Kubernetes cluster from a PC, you can use kubectl, a Kubernetes command line tool. You can log in to the CCE console, click the name of the cluster to be connected, and view the access address and kubectl connection procedure on the cluster details page. CCE allows you to access a cluster through a **VPC network** or a **public network**. -- VPC internal access: Clusters in the same VPC can access each other. -- Public network access: You need to prepare an ECS that can connect to a public network. +- **Intra-VPC access**: The client that accesses the cluster must be in the same VPC as the cluster. +- **Public access**:The client that accesses the cluster must be able to access public networks and the cluster has been bound with a public network IP. -.. important:: + .. important:: - If public network access is used, the kube-apiserver of the cluster will be exposed to the public network and may be attacked. You are advised to configure Advanced Anti-DDoS for the EIP of the node where the kube-apiserver is located. + To bind a public IP (EIP) to the cluster, go to the cluster details page and click **Bind** next to **EIP** in the **Connection Information** pane. In a cluster with an EIP bound, kube-apiserver will be exposed to public networks and may be attacked. You are advised to configure Advanced Anti-DDoS (AAD) for the EIP of the node where kube-apiserver resides. -**Downloading kubectl** +Download kubectl and the configuration file. Copy the file to your client, and configure kubectl. After the configuration is complete, you can access your Kubernetes clusters. Procedure: -You need to download kubectl and configuration file, copy the file to your client, and configure kubectl. After the configuration is complete, you can use kubectl to access your Kubernetes clusters. +#. .. _cce_10_0107__li194691356201712: -On the `Kubernetes release `__ page, click the corresponding link based on the cluster version, click **Client Binaries**, and download the corresponding platform software package. + Download kubectl. + + On the `Kubernetes release `__ page, click the corresponding link based on the cluster version, click **Client Binaries**, and download the corresponding platform software package. Alternatively, you can install kubectl with curl following the guide in `Install Tools `__. -.. figure:: /_static/images/en-us_image_0000001283755568.png - :alt: **Figure 1** Downloading kubectl + .. figure:: /_static/images/en-us_image_0000001336475537.png + :alt: **Figure 1** Downloading kubectl - **Figure 1** Downloading kubectl + **Figure 1** Downloading kubectl -**Installing and configuring kubectl** +#. .. _cce_10_0107__li34691156151712: -#. Log in to the CCE console, click **Resource Management** > **Clusters**, and choose **Command Line Tool** > **Kubectl** under the cluster to be connected. -#. On the **Kubectl** tab page of the cluster details page, connect to the cluster as prompted. + Obtain the kubectl configuration file (kubeconfig). + + On the **Connection Information** pane on the cluster details page, click **Learn more** next to **kubectl**. On the window displayed, download the configuration file. .. note:: - - You can download the kubectl configuration file (**kubeconfig.json**) on the **kubectl** tab page. This file is used for user cluster authentication. If the file is leaked, your clusters may be attacked. - - If two-way authentication is enabled for the current cluster and an EIP has been bound to the cluster, when the authentication fails (x509: certificate is valid), you need to bind the EIP and download the **kubeconfig.json** file again. - - By default, two-way authentication is disabled for domain names in the current cluster. You can run the **kubectl config use-context externalTLSVerify** command to enable two-way authentication. For details, see :ref:`Two-Way Authentication for Domain Names `. For a cluster that has been bound to an EIP, if the authentication fails (x509: certificate is valid) when two-way authentication is used, you need to bind the EIP again and download **kubeconfig.json** again. + - The kubectl configuration file **kubeconfig.json** is used for cluster authentication. If the file is leaked, your clusters may be attacked. + - By default, two-way authentication is disabled for domain names in the current cluster. You can run the **kubectl config use-context externalTLSVerify** command to enable two-way authentication. For details, see :ref:`Two-Way Authentication for Domain Names `. For a cluster that has been bound to an EIP, if the authentication fails (x509: certificate is valid) when two-way authentication is used, you need to bind the EIP again and download **kubeconfig.json** again. - The Kubernetes permissions assigned by the configuration file downloaded by IAM users are the same as those assigned to the IAM users on the CCE console. - If the KUBECONFIG environment variable is configured in the Linux OS, kubectl preferentially loads the KUBECONFIG environment variable instead of **$home/.kube/config**. -.. _cce_01_0107__section1559919152711: +#. Configure kubectl. + + Install and configure kubectl (A Linux OS is used as an example). + + a. Copy the kubectl downloaded in :ref:`1 ` and the configuration file downloaded in :ref:`2 ` to the **/home** directory on your client. + + b. Log in to your client and configure kubectl. If you have installed kubectl, skip this step. + + .. code-block:: + + cd /home + chmod +x kubectl + mv -f kubectl /usr/local/bin + + c. Log in to your client and configure the kubeconfig file. + + .. code-block:: + + cd /home + mkdir -p $HOME/.kube + mv -f kubeconfig.json $HOME/.kube/config + + d. Switch the kubectl access mode based on service scenarios. + + - Run this command to enable intra-VPC access: + + .. code-block:: + + kubectl config use-context internal + + - Run this command to enable public access (EIP required): + + .. code-block:: + + kubectl config use-context external + + - Run this command to enable public access and two-way authentication (EIP required): + + .. code-block:: + + kubectl config use-context externalTLSVerify + + For details about the cluster two-way authentication, see :ref:`Two-Way Authentication for Domain Names `. + +.. _cce_10_0107__section1559919152711: Two-Way Authentication for Domain Names --------------------------------------- @@ -71,15 +113,15 @@ Currently, CCE supports two-way authentication for domain names. - When an EIP is bound to or unbound from a cluster, or a custom domain name is configured or updated, the cluster server certificate will be added the latest cluster access address (including the EIP bound to the cluster and all custom domain names configured for the cluster). -- Asynchronous cluster synchronization takes about 5 to 10 minutes. +- Asynchronous cluster synchronization takes about 5 to 10 minutes. You can view the synchronization result in **Synchronize Certificate** in **Operation Records**. - For a cluster that has been bound to an EIP, if the authentication fails (x509: certificate is valid) when two-way authentication is used, you need to bind the EIP again and download **kubeconfig.json** again. -- If the domain name two-way authentication is not supported, **kubeconfig.json** contains the **"insecure-skip-tls-verify": true** field, as shown in :ref:`Figure 2 `. To use two-way authentication, you can download the **kubeconfig.json** file again and enable two-way authentication for the domain names. +- If the domain name two-way authentication is not supported, **kubeconfig.json** contains the **"insecure-skip-tls-verify": true** field, as shown in :ref:`Figure 2 `. To use two-way authentication, you can download the **kubeconfig.json** file again and enable two-way authentication for the domain names. - .. _cce_01_0107__fig1941342411: + .. _cce_10_0107__fig1941342411: - .. figure:: /_static/images/en-us_image_0000001243407853.png + .. figure:: /_static/images/en-us_image_0000001199021320.png :alt: **Figure 2** Two-way authentication disabled for domain names **Figure 2** Two-way authentication disabled for domain names @@ -91,4 +133,4 @@ When you use kubectl to create or query Kubernetes resources, the following outp # kubectl get deploy Error from server (Forbidden): deployments.apps is forbidden: User "0c97ac3cb280f4d91fa7c0096739e1f8" cannot list resource "deployments" in API group "apps" in the namespace "default" -The cause is that the user does not have the permissions to operate the Kubernetes resources. For details about how to assign permissions, see :ref:`Namespace Permissions (Kubernetes RBAC-based) `. +The cause is that the user does not have the permissions to operate the Kubernetes resources. For details about how to assign permissions, see :ref:`Namespace Permissions (Kubernetes RBAC-based) `. diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/customizing_a_cluster_certificate_san.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/customizing_a_cluster_certificate_san.rst new file mode 100644 index 0000000..d0e4f39 --- /dev/null +++ b/umn/source/clusters/using_kubectl_to_run_a_cluster/customizing_a_cluster_certificate_san.rst @@ -0,0 +1,42 @@ +:original_name: cce_10_0367.html + +.. _cce_10_0367: + +Customizing a Cluster Certificate SAN +===================================== + +Scenario +-------- + +A **Subject Alternative Name (SAN)** can be signed in to a cluster server certificate. A SAN is usually used by the client to verify the server validity in TLS handshakes. Specifically, the validity check includes whether the server certificate is issued by a CA trusted by the client and whether the SAN in the certificate matches the IP address or DNS domain name that the client actually accesses. + +If the client cannot directly access the private IP or EIP of the cluster, you can sign the IP address or DNS domain name that can be directly accessed by the client into the cluster server certificate to enable two-way authentication on the client, which improves security. Typical use cases include DNAT access and domain name access. + +Notes and Constraints +--------------------- + +This feature is available only to clusters of v1.19 and later. + +Customizing a SAN +----------------- + +#. Log in to the CCE console. +#. Click the target cluster in the cluster list to go to the cluster details page. +#. In the **Connection Information** area, click |image1| next to **Custom SAN**. In the dialog box displayed, add the IP address or domain name and click **Save**. + + .. note:: + + 1. This operation will restart kube-apiserver and update the **kubeconfig.json** file for a short period of time. Do not perform operations on the cluster during this period. + + 2. A maximum of 128 domain names or IP addresses, separated by commas (,), are allowed. + + 3. If a custom domain name needs to be bound to an EIP, ensure that an EIP has been configured. + +Typical Domain Name Access Scenarios +------------------------------------ + +- Add the response domain name mapping when specifying the DNS domain name address in the host domain name configuration on the client, or configuring **/etc/hosts** on the client host. +- Use domain name access in the intranet. DNS allows you to configure mappings between cluster EIPs and custom domain names. After an EIP is updated, you can continue to use two-way authentication and the domain name to access the cluster without downloading the **kubeconfig.json** file again. +- Add A records on a self-built DNS server. + +.. |image1| image:: /_static/images/en-us_image_0000001199341268.png diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst index 8eef3c2..1e4e5b5 100644 --- a/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst +++ b/umn/source/clusters/using_kubectl_to_run_a_cluster/index.rst @@ -1,18 +1,18 @@ -:original_name: cce_01_0140.html +:original_name: cce_10_0140.html -.. _cce_01_0140: +.. _cce_10_0140: Using kubectl to Run a Cluster ============================== -- :ref:`Connecting to a Cluster Using kubectl ` -- :ref:`Common kubectl Commands ` -- :ref:`kubectl Usage Guide ` +- :ref:`Connecting to a Cluster Using kubectl ` +- :ref:`Customizing a Cluster Certificate SAN ` +- :ref:`Common kubectl Commands ` .. toctree:: :maxdepth: 1 :hidden: connecting_to_a_cluster_using_kubectl + customizing_a_cluster_certificate_san common_kubectl_commands - kubectl_usage_guide diff --git a/umn/source/clusters/using_kubectl_to_run_a_cluster/kubectl_usage_guide.rst b/umn/source/clusters/using_kubectl_to_run_a_cluster/kubectl_usage_guide.rst deleted file mode 100644 index 68bca3c..0000000 --- a/umn/source/clusters/using_kubectl_to_run_a_cluster/kubectl_usage_guide.rst +++ /dev/null @@ -1,62 +0,0 @@ -:original_name: cce_01_0023.html - -.. _cce_01_0023: - -kubectl Usage Guide -=================== - -Before running kubectl commands, you should have the kubectl development skills and understand the kubectl operations. For details, see `Kubernetes API `__ and `kubectl CLI `__. - -Go to the `Kubernetes release page `__ to download kubectl corresponding to the cluster version or a later version. - -Cluster Connection ------------------- - -- :ref:`Connecting to a Kubernetes cluster using kubectl ` - -Workload Creation ------------------ - -- :ref:`Creating a Deployment using kubectl ` -- :ref:`Creating a StatefulSet using kubectl ` - -Workload Affinity/Anti-affinity Scheduling ------------------------------------------- - -- :ref:`Example YAML for workload-node affinity ` -- :ref:`Example YAML for workload-node anti-affinity ` -- :ref:`Example YAML for workload-workload affinity ` -- :ref:`Example YAML for workload-workload anti-affinity ` -- :ref:`Example YAML for workload-AZ affinity ` -- :ref:`Example YAML for workload-AZ anti-affinity ` - -Workload Access Mode Settings ------------------------------ - -- :ref:`Implementing intra-cluster access using kubectl ` -- :ref:`Implementing node access using kubectl ` -- :ref:`Implementing Layer 4 load balancing using kubectl ` -- :ref:`Implementing Layer 7 load balancing using kubectl ` - -Advanced Workload Settings --------------------------- - -- :ref:`Example YAML for setting the container lifecycle ` - -Job Management --------------- - -- :ref:`Creating a job using kubectl ` -- :ref:`Creating a cron job using kubectl ` - -Configuration Center --------------------- - -- :ref:`Creating a ConfigMap using kubectl ` -- :ref:`Creating a secret using kubectl ` - -Storage Management ------------------- - -- :ref:`Creating a PV using kubectl ` -- :ref:`Creating a PVC using kubectl ` diff --git a/umn/source/configuration_center/cluster_secrets.rst b/umn/source/configuration_center/cluster_secrets.rst index 4fc80a9..e062cd3 100644 --- a/umn/source/configuration_center/cluster_secrets.rst +++ b/umn/source/configuration_center/cluster_secrets.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0388.html +:original_name: cce_10_0388.html -.. _cce_01_0388: +.. _cce_10_0388: Cluster Secrets =============== @@ -11,10 +11,10 @@ By default, CCE creates the following secrets in each namespace: - paas.elb - default-token-*xxxxx* (*xxxxx* is a random number.) -|image1| - The functions of these secrets are described as follows. +.. _cce_10_0388__section11760122012591: + default-secret -------------- @@ -86,5 +86,3 @@ By default, Kubernetes creates a service account named **default** for each name Mountable secrets: default-token-vssmw Tokens: default-token-vssmw Events: - -.. |image1| image:: /_static/images/en-us_image_0000001227977765.png diff --git a/umn/source/configuration_center/creating_a_configmap.rst b/umn/source/configuration_center/creating_a_configmap.rst index 13202bc..0124ef5 100644 --- a/umn/source/configuration_center/creating_a_configmap.rst +++ b/umn/source/configuration_center/creating_a_configmap.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0152.html +:original_name: cce_10_0152.html -.. _cce_01_0152: +.. _cce_10_0152: Creating a ConfigMap ==================== @@ -10,7 +10,7 @@ Scenario A ConfigMap is a type of resource that stores configuration information required by a workload. Its content is user-defined. After creating ConfigMaps, you can use them as files or environment variables in a containerized workload. -ConfigMaps allow you to decouple configuration files from container images to enhance the portability of containerized workloads. +ConfigMaps allow you to decouple configuration files from container images to enhance the portability of workloads. Benefits of ConfigMaps: @@ -18,74 +18,39 @@ Benefits of ConfigMaps: - Deploy workloads in different environments. Multiple versions are supported for configuration files so that you can update and roll back workloads easily. - Quickly import configurations in the form of files to containers. -Prerequisites -------------- - -Cluster and node resources have been created. For more information, see :ref:`Creating a CCE Cluster `. If you have available clusters and node resources, skip this operation. - Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Configuration Center** > **ConfigMaps**. Click **Create ConfigMap**. +#. Log in to the CCE console and access the cluster console. -#. You can create a ConfigMap directly or based on YAML. If you create a ConfigMap based on YAML, go to :ref:`4 `. +#. Choose **ConfigMaps and Secrets** in the navigation pane and click **Create ConfigMap** in the upper right corner. -#. Method 1: Create a ConfigMap directly. +#. Set parameters. - Set the parameters by referring to :ref:`Table 1 `. - - .. _cce_01_0152__table16321825732: + .. _cce_10_0152__table16321825732: .. table:: **Table 1** Parameters for creating a ConfigMap - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+=================================================================================================================================================================================+ - | Name | Name of a ConfigMap, which must be unique in a namespace. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster | Cluster that will use the ConfigMap you create. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace to which the ConfigMap belongs. If you do not specify this parameter, the value **default** is used by default. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Description | Description of the ConfigMap. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Data | The workload configuration data can be used in a container or used to store the configuration data. **Key** indicates a file name. **Value** indicates the content in the file. | - | | | - | | a. Click **Add Data**. | - | | b. Set **Key** and **Value**. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Labels | Labels are attached to objects such as workloads, nodes, and Services in key-value pairs. | - | | | - | | Labels define the identifiable attributes of these objects and are used to manage and select the objects. | - | | | - | | a. Click **Add Label**. | - | | b. Set **Key** and **Value**. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+===========================================================================================================================+ + | Name | Name of a ConfigMap, which must be unique in a namespace. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + | Namespace | Namespace to which the ConfigMap belongs. If you do not specify this parameter, the value **default** is used by default. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + | Description | Description of the ConfigMap. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + | Data | Data of a ConfigMap, in the key-value pair format. | + | | | + | | Click |image1| to add data. The value can be in string, JSON, or YAML format. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------+ + | Label | Label of the ConfigMap. Enter a key-value pair and click **Add**. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------+ -#. .. _cce_01_0152__li2731182712159: - - Method 2: Create a ConfigMap based on YAML. - - .. note:: - - To create ConfigMaps by uploading a file, ensure that the resource description file has been created. CCE supports files in YAML format. For more information, see :ref:`ConfigMap Requirements `. - - Click **Create YAML** on the right of the page. - - - Method 1: Import the orchestration file. - - Click **Add File** to import the file in YAML format. The orchestration content can be directly displayed. - - - Method 2: Directly orchestrate the content. - - In the orchestration content area, enter the content of the YAML file. - -#. After the configuration is complete, click **Create**. +#. After the configuration is complete, click **OK**. The new ConfigMap is displayed in the ConfigMap list. -.. _cce_01_0152__section66903416102: - ConfigMap Requirements ---------------------- @@ -103,12 +68,10 @@ The file name is **configmap.yaml** and the following shows a configuration exam data-1: value-1 data-2: value-2 -.. _cce_01_0152__section639712716372: - Creating a ConfigMap Using kubectl ---------------------------------- -#. Configure the **kubectl** command to connect an ECS to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Configure the **kubectl** command to connect an ECS to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create and edit the **cce-configmap.yaml** file. @@ -139,22 +102,24 @@ Creating a ConfigMap Using kubectl Related Operations ------------------ -After creating a configuration item, you can update or delete it as described in :ref:`Table 2 `. +After creating a configuration item, you can update or delete it as described in :ref:`Table 2 `. -.. _cce_01_0152__table1619535674020: +.. _cce_10_0152__table1619535674020: .. table:: **Table 2** Related operations - +-----------------------------------+--------------------------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==============================================================================================================+ - | Viewing a YAML file | Click **View YAML** next to the ConfigMap name to view the YAML file corresponding to the current ConfigMap. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------+ - | Updating a ConfigMap | #. Select the name of the ConfigMap to be updated and click **Update**. | - | | #. Modify the secret data. For more information, see :ref:`Table 1 `. | - | | #. Click **Update**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------+ - | Deleting a ConfigMap | Select the configuration you want to delete and click **Delete**. | - | | | - | | Follow the prompts to delete the ConfigMap. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + | Operation | Description | + +===================================+======================================================================================================+ + | Editing a YAML file | Click **Edit YAML** in the row where the target ConfigMap resides to edit its YAML file. | + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + | Updating a ConfigMap | #. Select the name of the ConfigMap to be updated and click **Update**. | + | | #. Modify the secret data. For more information, see :ref:`Table 1 `. | + | | #. Click **OK**. | + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + | Deleting a ConfigMap | Select the configuration you want to delete and click **Delete**. | + | | | + | | Follow the prompts to delete the ConfigMap. | + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + +.. |image1| image:: /_static/images/en-us_image_0000001205757902.png diff --git a/umn/source/configuration_center/creating_a_secret.rst b/umn/source/configuration_center/creating_a_secret.rst index 63a47d6..877105a 100644 --- a/umn/source/configuration_center/creating_a_secret.rst +++ b/umn/source/configuration_center/creating_a_secret.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0153.html +:original_name: cce_10_0153.html -.. _cce_01_0153: +.. _cce_10_0153: Creating a Secret ================= @@ -10,91 +10,53 @@ Scenario A secret is a type of resource that holds sensitive data, such as authentication and key information. Its content is user-defined. After creating secrets, you can use them as files or environment variables in a containerized workload. -Prerequisites -------------- - -Cluster and node resources have been created. For more information, see :ref:`Creating a CCE Cluster `. If you have available clusters and node resources, skip this operation. - Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Configuration Center** > **Secrets**. Click **Create Secret**. +#. Log in to the CCE console and access the cluster console. -#. You can create a secret directly or based on YAML. If you want to create a secret based on YAML, go to :ref:`4 `. +#. Choose **ConfigMaps and Secrets** in the navigation pane, click the **Secrets** tab, and click **Create Secret** in the upper right corner. -#. Method 1: Create a secret directly. +#. Set parameters. - Set the basic information by referring to :ref:`Table 1 `. - - .. _cce_01_0153__table16321825732: + .. _cce_10_0153__table16321825732: .. table:: **Table 1** Parameters for creating a secret - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==================================================================================================================================================================================+ - | Name | Name of the secret you create, which must be unique. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster | Cluster that will use the secret you create. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace to which the secret belongs. If you do not specify this parameter, the value **default** is used by default. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Description | Description of a secret. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Type | Type of the secret you create. | - | | | - | | - Opaque: common secret. | - | | - kubernetes.io/dockerconfigjson: a secret that stores the authentication information required for pulling images from a private repository. | - | | - IngressTLS: a secret that stores the certificate required by ingresses (layer-7 load balancing Services). | - | | - Other: another type of secret, which is specified manually. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Secret Data | Workload secret data can be used in containers. | - | | | - | | - If the secret is of the Opaque type: | - | | | - | | a. Click **Add Data**. | - | | b. Enter the key and value. The value must be based on the Base64 coding method. For details about the method, see :ref:`Base64 Encoding `. | - | | | - | | - If the secret type is kubernetes.io/dockerconfigjson, enter the account and password of the private image repository. | - | | - If the secret type is IngressTLS, upload the certificate file and private key file. | - | | | - | | .. note:: | - | | | - | | - A certificate is a self-signed or CA-signed credential used for identity authentication. | - | | - A certificate request is a request for a signature with a private key. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Secret Label | Labels are attached to objects such as workloads, nodes, and Services in key-value pairs. | - | | | - | | Labels define the identifiable attributes of these objects and are used to manage and select the objects. | - | | | - | | a. Click **Add Label**. | - | | b. Enter the key and value. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+===============================================================================================================================================+ + | Name | Name of the secret you create, which must be unique. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ + | Namespace | Namespace to which the secret belongs. If you do not specify this parameter, the value **default** is used by default. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ + | Description | Description of a secret. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ + | Type | Type of the secret you create. | + | | | + | | - Opaque: common secret. | + | | - kubernetes.io/dockerconfigjson: a secret that stores the authentication information required for pulling images from a private repository. | + | | - IngressTLS: a secret that stores the certificate required by ingresses (layer-7 load balancing Services). | + | | - Other: another type of secret, which is specified manually. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ + | Secret Data | Workload secret data can be used in containers. | + | | | + | | - If **Secret Type** is **Opaque**, click |image1|. In the dialog box displayed, enter a key-value pair and select **Auto Base64 Encoding**. | + | | - If the secret type is kubernetes.io/dockerconfigjson, enter the account and password of the private image repository. | + | | - If the secret type is IngressTLS, upload the certificate file and private key file. | + | | | + | | .. note:: | + | | | + | | - A certificate is a self-signed or CA-signed credential used for identity authentication. | + | | - A certificate request is a request for a signature with a private key. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ + | Secret Label | Label of the secret. Enter a key-value pair and click **Add**. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------+ -#. .. _cce_01_0153__li69121840101813: - - Method 2: Create a secret based on the YAML file. - - .. note:: - - To create a resource by uploading a file, ensure that the resource description file has been created. CCE supports files in JSON or YAML format. For more information, see :ref:`Secret Resource File Configuration `. - - You can import or directly write the file content in YAML or JSON format. - - - Method 1: Import the orchestration file. - - Click **Add File** to import the file in YAML or JSON format. The orchestration content can be directly displayed. - - - Method 2: Directly orchestrate the content. - - In the orchestration content area, enter the content of the YAML or JSON file. - -#. After the configuration is complete, click **Create**. +#. After the configuration is complete, click **OK**. The new secret is displayed in the key list. -.. _cce_01_0153__section187197531454: - Secret Resource File Configuration ---------------------------------- @@ -104,7 +66,7 @@ For example, you can retrieve the username and password for a workload through a - YAML format - The **secret.yaml** file is defined as shown below. The value must be based on the Base64 coding method. For details about the method, see :ref:`Base64 Encoding `. + The **secret.yaml** file is defined as shown below. The value must be based on the Base64 coding method. For details about the method, see :ref:`Base64 Encoding `. .. code-block:: @@ -118,12 +80,10 @@ For example, you can retrieve the username and password for a workload through a password: ****** #The value must be encoded using Base64. type: Opaque #You are advised not to change this parameter value. -.. _cce_01_0153__section821112149514: - Creating a Secret Using kubectl ------------------------------- -#. According to :ref:`Connecting to a Cluster Using kubectl `, configure the **kubectl** command to connect an ECS to the cluster. +#. According to :ref:`Connecting to a Cluster Using kubectl `, configure the **kubectl** command to connect an ECS to the cluster. #. Create and edit the Base64-encoded **cce-secret.yaml** file. @@ -156,42 +116,44 @@ Creating a Secret Using kubectl Related Operations ------------------ -After creating a secret, you can update or delete it as described in :ref:`Table 2 `. +After creating a secret, you can update or delete it as described in :ref:`Table 2 `. .. note:: The secret list contains system secret resources that can be queried only. The system secret resources cannot be updated or deleted. -.. _cce_01_0153__table555785274319: +.. _cce_10_0153__table555785274319: .. table:: **Table 2** Related Operations - +-----------------------------------+--------------------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+========================================================================================================+ - | Viewing a YAML file | Click **View YAML** next to the secret name to view the YAML file corresponding to the current secret. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------+ - | Updating a secret | #. Select the name of the secret to be updated and click **Update**. | - | | #. Modify the secret data. For more information, see :ref:`Table 1 `. | - | | #. Click **Update**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------+ - | Deleting a secret | Select the secret you want to delete and click **Delete**. | - | | | - | | Follow the prompts to delete the secret. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------+ - | Deleting secrets in batches | #. Select the secrets to be deleted. | - | | #. Click **Delete** above the secret list. | - | | #. Follow the prompts to delete the secrets. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------+ + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + | Operation | Description | + +===================================+======================================================================================================+ + | Editing a YAML file | Click **Edit YAML** in the row where the target secret resides to edit its YAML file. | + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + | Updating a secret | #. Select the name of the secret to be updated and click **Update**. | + | | #. Modify the secret data. For more information, see :ref:`Table 1 `. | + | | #. Click **OK**. | + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + | Deleting a secret | Select the secret you want to delete and click **Delete**. | + | | | + | | Follow the prompts to delete the secret. | + +-----------------------------------+------------------------------------------------------------------------------------------------------+ + | Deleting secrets in batches | #. Select the secrets to be deleted. | + | | #. Click **Delete** above the secret list. | + | | #. Follow the prompts to delete the secrets. | + +-----------------------------------+------------------------------------------------------------------------------------------------------+ -.. _cce_01_0153__section175000605919: +.. _cce_10_0153__section175000605919: Base64 Encoding --------------- -To encrypt a character string using Base64, run the **echo -n to-be-encoded content \| base64** command. The following is an example. +To Base64-encode a string, run the **echo -n content to be encoded \| base64** command. The following is an example: .. code-block:: root@ubuntu:~# echo -n "content to be encoded" | base64 ****** + +.. |image1| image:: /_static/images/en-us_image_0000001249958645.png diff --git a/umn/source/configuration_center/index.rst b/umn/source/configuration_center/index.rst index 5ca083a..7690d2a 100644 --- a/umn/source/configuration_center/index.rst +++ b/umn/source/configuration_center/index.rst @@ -1,15 +1,15 @@ -:original_name: cce_01_0045.html +:original_name: cce_10_0045.html -.. _cce_01_0045: +.. _cce_10_0045: Configuration Center ==================== -- :ref:`Creating a ConfigMap ` -- :ref:`Using a ConfigMap ` -- :ref:`Creating a Secret ` -- :ref:`Using a Secret ` -- :ref:`Cluster Secrets ` +- :ref:`Creating a ConfigMap ` +- :ref:`Using a ConfigMap ` +- :ref:`Creating a Secret ` +- :ref:`Using a Secret ` +- :ref:`Cluster Secrets ` .. toctree:: :maxdepth: 1 diff --git a/umn/source/configuration_center/using_a_configmap.rst b/umn/source/configuration_center/using_a_configmap.rst index 3c5cc62..0a3664b 100644 --- a/umn/source/configuration_center/using_a_configmap.rst +++ b/umn/source/configuration_center/using_a_configmap.rst @@ -1,13 +1,13 @@ -:original_name: cce_01_0015.html +:original_name: cce_10_0015.html -.. _cce_01_0015: +.. _cce_10_0015: Using a ConfigMap ================= -- :ref:`Setting Workload Environment Variables ` -- :ref:`Setting Command Line Parameters ` -- :ref:`Attaching a ConfigMap to the Workload Data Volume ` +- :ref:`Setting Workload Environment Variables ` +- :ref:`Setting Command Line Parameters ` +- :ref:`Attaching a ConfigMap to the Workload Data Volume ` The following example shows how to use a ConfigMap. @@ -25,7 +25,7 @@ The following example shows how to use a ConfigMap. When a ConfigMap is used in a pod, the pod and ConfigMap must be in the same cluster and namespace. -.. _cce_01_0015__section1737733192813: +.. _cce_10_0015__section1737733192813: Setting Workload Environment Variables -------------------------------------- @@ -85,7 +85,7 @@ To add all data in a ConfigMap to environment variables, use the **envFrom** par name: cce-configmap restartPolicy: Never -.. _cce_01_0015__section17930105710189: +.. _cce_10_0015__section17930105710189: Setting Command Line Parameters ------------------------------- @@ -122,7 +122,7 @@ After the pod runs, the following information is displayed: Hello CCE -.. _cce_01_0015__section1490261161916: +.. _cce_10_0015__section1490261161916: Attaching a ConfigMap to the Workload Data Volume ------------------------------------------------- @@ -156,4 +156,4 @@ After the pod is run, the **SPECIAL_LEVEL** and **SPECIAL_TYPE** files are gener SPECIAL_TYPE SPECIAL_LEVEL -To mount a ConfigMap to a data volume, you can also perform operations on the CCE console. When creating a workload, set advanced settings for the container, choose **Data Storage** > **Local Volume**, click **Add Local Volume**, and select **ConfigMap**. For details, see :ref:`ConfigMap `. +To mount a ConfigMap to a data volume, you can also perform operations on the CCE console. When creating a workload, set advanced settings for the container, choose **Data Storage** > **Local Volume**, click **Add Local Volume**, and select **ConfigMap**. For details, see :ref:`ConfigMap `. diff --git a/umn/source/configuration_center/using_a_secret.rst b/umn/source/configuration_center/using_a_secret.rst index a93d7ac..9945789 100644 --- a/umn/source/configuration_center/using_a_secret.rst +++ b/umn/source/configuration_center/using_a_secret.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0016.html +:original_name: cce_10_0016.html -.. _cce_01_0016: +.. _cce_10_0016: Using a Secret ============== @@ -12,8 +12,8 @@ Using a Secret - Do not operate secrets under kube-system. - Do not operate default-secret and paas.elb in any of the namespaces. The default-secret is used to pull the private image of SWR, and the paas.elb is used to connect the service in the namespace to the ELB service. -- :ref:`Configuring the Data Volume of a Pod ` -- :ref:`Setting Environment Variables of a Pod ` +- :ref:`Configuring the Data Volume of a Pod ` +- :ref:`Setting Environment Variables of a Pod ` The following example shows how to use a secret. @@ -32,7 +32,7 @@ The following example shows how to use a secret. When a secret is used in a pod, the pod and secret must be in the same cluster and namespace. -.. _cce_01_0016__section472505211214: +.. _cce_10_0016__section472505211214: Configuring the Data Volume of a Pod ------------------------------------ @@ -82,9 +82,9 @@ In addition, you can specify the directory and permission to access a secret. Th path: my-group/my-username mode: 511 -To mount a secret to a data volume, you can also perform operations on the CCE console. When creating a workload, set advanced settings for the container, choose **Data Storage > Local Volume**, click **Add Local Volume**, and select **Secret**. For details, see :ref:`Secret `. +To mount a secret to a data volume, you can also perform operations on the CCE console. When creating a workload, set advanced settings for the container, choose **Data Storage > Local Volume**, click **Add Local Volume**, and select **Secret**. For details, see :ref:`Secret `. -.. _cce_01_0016__section207271352141216: +.. _cce_10_0016__section207271352141216: Setting Environment Variables of a Pod -------------------------------------- diff --git a/umn/source/index.rst b/umn/source/index.rst index 9f74074..5ab5adf 100644 --- a/umn/source/index.rst +++ b/umn/source/index.rst @@ -13,18 +13,19 @@ Cloud Container Engine - User Guide nodes/index node_pools/index workloads/index - affinity_and_anti-affinity_scheduling/index networking/index - storage_csi/index - storage_flexvolume/index - monitoring_and_logs/index + storage/index + monitoring_and_alarm/index + logging/index namespaces/index configuration_center/index - charts_helm/index - add-ons/index auto_scaling/index + add-ons/index + charts/index permissions_management/index cloud_trace_service_cts/index + storage_flexvolume/index reference/index + best_practice/index migrating_data_from_cce_1.0_to_cce_2.0/index change_history diff --git a/umn/source/instruction.rst b/umn/source/instruction.rst index dc69715..880b4c2 100644 --- a/umn/source/instruction.rst +++ b/umn/source/instruction.rst @@ -18,7 +18,7 @@ Complete the following tasks to get started with CCE. **Figure 1** Procedure for getting started with CCE -#. **Authorize an IAM user to use CCE.** +#. :ref:`Charts (Helm) `\ Authorize an IAM user to use CCE. The accounts have the permission to use CCE. However, IAM users created by the accounts do not have the permission. You need to manually assign the permission to IAM users. @@ -30,12 +30,12 @@ Complete the following tasks to get started with CCE. Select existing images/chart, or create new images/chart. - - For details on how to create a workload from images, see `Workloads `__. + - For details on how to create a workload from images, see :ref:`Workloads `. - For details on how to create a workload from a chart, see `Charts (Helm) `__. #. **View workload status and logs. Upgrade, scale, and monitor the workload.** - For details, see `Managing Workloads and Jobs `__. + For details, see :ref:`Managing Workloads and Jobs `. FAQs ---- @@ -46,7 +46,7 @@ FAQs #. **Is CCE suitable for users who have little experience in building images?** - Yes. You can select images from **Third-party Images**, and **Shared Images** pages on the CCE console. The **My Images** page displays only the images created by you. For details, see `Workloads `__. + Yes. You can select images from **Third-party Images**, and **Shared Images** pages on the CCE console. The **My Images** page displays only the images created by you. For details, see :ref:`Workloads `. #. **How do I create a workload using CCE?** @@ -54,7 +54,7 @@ FAQs #. **How do I create a workload accessible to public networks?** - CCE provides different types of Services for workload access in diverse scenarios. Currently, CCE provides two access types to expose a workload to public networks: NodePort and LoadBalancer. For details, see `Networking `__. + CCE provides different types of Services for workload access in diverse scenarios. Currently, CCE provides two access types to expose a workload to public networks: NodePort and LoadBalancer. For details, see :ref:`Networking `. #. **How can I allow multiple workloads in the same cluster to access each other?** @@ -64,4 +64,4 @@ FAQs Example: - Assume that workload A needs to access workload B in the same cluster. Then, you can create a `ClusterIP `__ Service for workload B. After the ClusterIP Service is created, workload B is reachable at ..svc.cluster.local:. + Assume that workload A needs to access workload B in the same cluster. Then, you can create a :ref:`ClusterIP ` Service for workload B. After the ClusterIP Service is created, workload B is reachable at ..svc.cluster.local:. diff --git a/umn/source/logging/index.rst b/umn/source/logging/index.rst new file mode 100644 index 0000000..7c22d1c --- /dev/null +++ b/umn/source/logging/index.rst @@ -0,0 +1,16 @@ +:original_name: cce_10_0553.html + +.. _cce_10_0553: + +Logging +======= + +- :ref:`Log Management Overview ` +- :ref:`Using ICAgent to Collect Container Logs ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + log_management_overview + using_icagent_to_collect_container_logs diff --git a/umn/source/logging/log_management_overview.rst b/umn/source/logging/log_management_overview.rst new file mode 100644 index 0000000..1ba037d --- /dev/null +++ b/umn/source/logging/log_management_overview.rst @@ -0,0 +1,19 @@ +:original_name: cce_10_0557.html + +.. _cce_10_0557: + +Log Management Overview +======================= + +CCE allows you to configure policies for collecting, managing, and analyzing workload logs periodically to prevent logs from being over-sized. + +- Using ICAgent: + + By default, the ICAgent collects container standard outputs (stdout logs). No configuration required. + + You can also configure the path for storing container logs when creating a workload so that the ICAgent collects logs from this path. + + You can select either of the following modes for container logs: + + - hostPath: A host path is mounted to the specified container path (mount path). In the node host path, you can view the container logs output into the mount path. + - emptyDir: A temporary path of the node is mounted to the specified path (mount path). Log data that exists in the temporary path but is not reported by the collector to AOM will disappear after the pod is deleted. diff --git a/umn/source/monitoring_and_logs/container_logs.rst b/umn/source/logging/using_icagent_to_collect_container_logs.rst similarity index 73% rename from umn/source/monitoring_and_logs/container_logs.rst rename to umn/source/logging/using_icagent_to_collect_container_logs.rst index a99fdc3..407b7a5 100644 --- a/umn/source/monitoring_and_logs/container_logs.rst +++ b/umn/source/logging/using_icagent_to_collect_container_logs.rst @@ -1,40 +1,28 @@ -:original_name: cce_01_0018.html +:original_name: cce_10_0018.html -.. _cce_01_0018: +.. _cce_10_0018: -Container Logs -============== +Using ICAgent to Collect Container Logs +======================================= -Scenario --------- +CCE works with AOM to collect workload logs. When creating a node, CCE installs the ICAgent for you (the DaemonSet named **icagent** in the kube-system namespace of the cluster). After the ICAgent collects workload logs and reports them to AOM, you can view workload logs on the CCE or AOM console. -CCE allows you to configure policies for collecting, managing, and analyzing workload logs periodically to prevent logs from being over-sized. - -CCE works with AOM to collect workload logs. When a node is created, the ICAgent (the DaemonSet named **icagent** in the kube-system namespace of the cluster) of AOM is installed by default. After the ICAgent collects workload logs and reports them to AOM, you can view workload logs on the CCE or AOM console. - -- By default, the ICAgent collects the standard outputs of containers. You do not need to perform any configuration. - -- You can also configure the path for storing container logs when creating a workload so that the ICAgent collects logs from this path. - - You can select either of the following modes for container logs: - - - HostPath: The host path is mounted to the specified container path (mount path). In the node host path, you can view the container logs output into the mount path. - - EmptyDir: The temporary path of the node is mounted to the specified path (mount path). Log data that exists in the temporary path but is not reported by the collector to AOM will disappear after the pod is deleted. - -Precautions ------------ +Notes and Constraints +--------------------- The ICAgent only collects **\*.log**, **\*.trace**, and **\*.out** text log files. -Setting the Path for Storing Container Logs -------------------------------------------- +Using ICAgent to Collect Logs +----------------------------- -#. When creating a :ref:`workload ` on the CCE console, add a container and expand **Log Policies**. +#. When :ref:`creating a workload `, set logging for the container. -#. In the **Log Policies** area, click **Add Log Policy**. Configure parameters in the log policy. The following uses Nginx as an example. +#. Click |image1| to add a log policy. + + The following uses Nginx as an example. Log policies vary depending on workloads. - .. figure:: /_static/images/en-us_image_0000001190538599.png + .. figure:: /_static/images/en-us_image_0000001199181298.png :alt: **Figure 1** Adding a log policy **Figure 1** Adding a log policy @@ -43,67 +31,58 @@ Setting the Path for Storing Container Logs .. table:: **Table 1** Configuring log policies - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+=======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ - | Storage Type | - **Host Path**: In HostPath mode, the host path is mounted to the specified container path (mount path). In the node host path, you can view the container logs output into the mount path. | - | | - **Container Path**: In EmptyDir mode, the temporary path of the node is mounted to the specified path (mount path). Log data that exists in the temporary path but is not reported by the collector to AOM will disappear after the pod is deleted. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Add Container Path** | | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Host Path | Enter the host path, for example, **/var/paas/sys/log/nginx**. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Path | Container path (for example, **/tmp**) to which the storage resources will be mounted. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | | - | | - Do not mount storage to a system directory such as **/** or **/var/run**; this action may cause a container error to occur. You are advised to mount the container to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | - When the container is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | - AOM collects only the first 20 log files that have been modified recently. It collects files from 2 levels of subdirectories by default. | - | | - AOM only collects **.log**, **.trace**, and **.out** text log files in mounting paths. | - | | - For details about how to set permissions for mount points in a container, see `Configure a Security Context for a Pod or Container `__. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Extended Host Path | This parameter is mandatory only if **Storage Type** is set to **Host Path**. | - | | | - | | Extended host paths contain pod IDs or container names to distinguish different containers into which the host path is mounted. | - | | | - | | A level-3 directory is added to the original volume directory/subdirectory. You can easily obtain the files output by a single Pod. | - | | | - | | - **None**: No extended path is configured. | - | | - **PodUID**: ID of a pod. | - | | - **PodName**: name of a pod. | - | | - **PodUID/ContainerName**: ID of a pod or name of a container. | - | | - **PodName/ContainerName**: name of a pod or container. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Log Dumping | Log dump refers to rolling log files on a local host. | - | | | - | | - **Enabled**: AOM scans log files every minute. When a log file exceeds 50 MB, it is dumped immediately. A new **.zip** file is generated in the directory where the log file locates. For a log file, AOM stores only the latest 20 **.zip** files. When the number of **.zip** files exceeds 20, earlier **.zip** files will be deleted. After the dump is complete, the log file in AOM will be cleared. | - | | - **Disabled**: AOM does not dump log files. | - | | | - | | .. note:: | - | | | - | | - Log file rolling of AOM is implemented in the copytruncate mode. Before enabling log dumping, ensure that log files are written in the append mode. Otherwise, file holes may occur. | - | | - Currently, mainstream log components such as Log4j and Logback support log file rolling. If your log files already support rolling, skip the configuration. Otherwise, conflicts may occur. | - | | - You are advised to configure log file rolling for your own services to flexibly control the size and number of rolled files. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Multi-line Log | Some program logs (for example, Java program logs) contain a log that occupies multiple lines. By default, the log collection system collects logs by line. If you want to display logs as a single log message in the log collection system, you can enable the multi-line log function and use the log time or regular pattern mode. When a line of log message matches the preset time format or regular expression, it is considered as the start of a log message and the next line starts with this line of log message is considered as the end identifier of the log message. | - | | | - | | **Split Mode** | - | | | - | | - **Log Time**: Enter a time wildcard. For example, if the time in the log is 2017-01-01 23:59:59, the wildcard is YYYY-MM-DD hh:mm:ss. | - | | - **Regular Pattern**: Enter a regular expression. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+===========================================================================================================================================================================================================================================================================================================================================================================================================================+ + | Storage Type | - **Host Path** (hostPath): A host path is mounted to the specified container path (mount path). In the node host path, you can view the container logs output into the mount path. | + | | - **Container Path** (emptyDir): A temporary path of the node is mounted to the specified path (mount path). Log data that exists in the temporary path but is not reported by the collector to AOM will disappear after the pod is deleted. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Host Path | Enter a host path, for example, **/var/paas/sys/log/nginx**. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Container Path | Container path (for example, **/tmp**) to which the storage resources will be mounted. | + | | | + | | .. important:: | + | | | + | | NOTICE: | + | | | + | | - Do not mount storage to a system directory such as **/** or **/var/run**; this action may cause a container error to occur. You are advised to mount the container to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | + | | - When the container is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | + | | - AOM collects only the first 20 log files that have been modified recently. It collects files from 2 levels of subdirectories by default. | + | | - AOM only collects **.log**, **.trace**, and **.out** text log files in the mount paths. | + | | - For details about how to set permissions for mount points in a container, see `Configure a Security Context for a Pod or Container `__. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Extended Host Path | This parameter is mandatory only if **Storage Type** is set to **Host Path**. | + | | | + | | Extended host paths contain pod IDs or container names to distinguish different containers into which the host path is mounted. | + | | | + | | A level-3 directory is added to the original volume directory/subdirectory. You can easily obtain the files output by a single Pod. | + | | | + | | - **None**: No extended path is configured. | + | | - **PodUID**: ID of a pod. | + | | - **PodName**: name of a pod. | + | | - **PodUID/ContainerName**: ID of a pod or name of a container. | + | | - **PodName/ContainerName**: name of a pod or container. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Log Dump | Log dump refers to rotating log files on a local host. | + | | | + | | - **Enabled**: AOM scans log files every minute. When a log file exceeds 50 MB, it is dumped immediately. A new **.zip** file is generated in the directory where the log file locates. For a log file, AOM stores only the latest 20 **.zip** files. When the number of **.zip** files exceeds 20, earlier **.zip** files will be deleted. After the dump is complete, the log file in AOM will be cleared. | + | | - **Disabled**: AOM does not dump log files. | + | | | + | | .. note:: | + | | | + | | - AOM rotates log files using copytruncate. Before enabling log dumping, ensure that log files are written in the append mode. Otherwise, file holes may occur. | + | | - Currently, mainstream log components such as Log4j and Logback support log file rotation. If you have already set rotation for log files, skip the configuration. Otherwise, conflicts may occur. | + | | - You are advised to configure log file rotation for your own services to flexibly control the size and number of rolled files. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Click **OK**. -Using kubectl to Set the Container Log Storage Path ---------------------------------------------------- +YAML Example +------------ You can set the container log storage path by defining a YAML file. -As shown in the following figure, EmptyDir is mounted a temporary path to **/var/log/nginx**. In this way, the ICAgent collects logs in **/var/log/nginx**. The **policy** field is customized by CCE and allows the ICAgent to identify and collect logs. +As shown in the following figure, an emptyDir volume is mounted a temporary path to **/var/log/nginx**. In this way, the ICAgent collects logs in **/var/log/nginx**. The **policy** field is customized by CCE and allows the ICAgent to identify and collect logs. .. code-block:: @@ -144,7 +123,7 @@ As shown in the following figure, EmptyDir is mounted a temporary path to **/var imagePullSecrets: - name: default-secret -The following shows how to use the HostPath mode. Compared with the EmptyDir mode, the type of volume is changed to hostPath, and the path on the host needs to be configured for this hostPath volume. In the following example, **/tmp/log** on the host is mounted to **/var/log/nginx**. In this way, the ICAgent can collects logs in **/var/log/nginx**, without deleting the logs from **/tmp/log**. +The following shows how to use a hostPath volume. Compared with emptyDir, the type of **volumes** is changed to **hostPath**, and the path on the host needs to be configured for this hostPath volume. In the following example, **/tmp/log** on the host is mounted to **/var/log/nginx**. In this way, the ICAgent can collects logs in **/var/log/nginx**, without deleting the logs from **/tmp/log**. .. code-block:: @@ -206,16 +185,16 @@ The following shows how to use the HostPath mode. Compared with the EmptyDir mod | | | - **PodUID/ContainerName**: ID of a pod or name of a container. | | | | - **PodName/ContainerName**: name of a pod or container. | +--------------------------------+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | policy.logs.rotate | Log dumping | Log dump refers to rolling log files on a local host. | + | policy.logs.rotate | Log dump | Log dump refers to rotating log files on a local host. | | | | | | | | - **Enabled**: AOM scans log files every minute. When a log file exceeds 50 MB, it is dumped immediately. A new **.zip** file is generated in the directory where the log file locates. For a log file, AOM stores only the latest 20 **.zip** files. When the number of **.zip** files exceeds 20, earlier **.zip** files will be deleted. After the dump is complete, the log file in AOM will be cleared. | | | | - **Disabled**: AOM does not dump log files. | | | | | | | | .. note:: | | | | | - | | | - Log file rolling of AOM is implemented in the copytruncate mode. Before enabling log dumping, ensure that log files are written in the append mode. Otherwise, file holes may occur. | - | | | - Currently, mainstream log components such as Log4j and Logback support log file rolling. If your log files already support rolling, skip the configuration. Otherwise, conflicts may occur. | - | | | - You are advised to configure log file rolling for your own services to flexibly control the size and number of rolled files. | + | | | - AOM rotates log files using copytruncate. Before enabling log dumping, ensure that log files are written in the append mode. Otherwise, file holes may occur. | + | | | - Currently, mainstream log components such as Log4j and Logback support log file rotation. If you have set rotation for log files, skip the configuration. Otherwise, conflicts may occur. | + | | | - You are advised to configure log file rotation for your own services to flexibly control the size and number of rolled files. | +--------------------------------+-------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | policy.logs.annotations.format | Multi-line log matching | Some program logs (for example, Java program logs) contain a log that occupies multiple lines. By default, the log collection system collects logs by line. If you want to display logs as a single log message in the log collection system, you can enable the multi-line log function and use the log time or regular pattern mode. When a line of log message matches the preset time format or regular expression, it is considered as the start of a log message and the next line starts with this line of log message is considered as the end identifier of the log message. | | | | | @@ -258,3 +237,5 @@ You can also run the **kubectl logs** command to view the standard output of a c kubectl logs pod_name -c container_name -n namespace (one-off query) kubectl logs -f -n namespace (real-time query in tail -f mode) + +.. |image1| image:: /_static/images/en-us_image_0000001206876656.png diff --git a/umn/source/monitoring_and_alarm/custom_monitoring.rst b/umn/source/monitoring_and_alarm/custom_monitoring.rst new file mode 100644 index 0000000..73f6c3a --- /dev/null +++ b/umn/source/monitoring_and_alarm/custom_monitoring.rst @@ -0,0 +1,202 @@ +:original_name: cce_10_0201.html + +.. _cce_10_0201: + +Custom Monitoring +================= + +CCE allows you to upload custom metrics to AOM. The ICAgent on a node periodically calls the metric monitoring API configured on a workload to read monitoring data and then uploads the data to AOM. + +|image1| + +The custom metric API of a workload can be configured when the workload is created. This section uses an Nginx application as an example to describe how to report custom metrics to AOM. + +Notes and Constraints +--------------------- + +- The ICAgent is compatible with the monitoring data specifications of `Prometheus `__. The custom metrics provided by pods can be collected by the ICAgent only when they meet the monitoring data specifications of Prometheus. +- The ICAgent supports only `Gauge `__ metrics. +- The interval for the ICAgent to call the custom metric API is 1 minute, which cannot be changed. + +Prometheus Monitoring Data Collection +------------------------------------- + +Prometheus periodically calls the metric monitoring API (**/metrics** by default) of an application to obtain monitoring data. The application needs to provide the metric monitoring API for Prometheus to call, and the monitoring data must meet the following specifications of Prometheus: + +.. code-block:: + + # TYPE nginx_connections_active gauge + nginx_connections_active 2 + # TYPE nginx_connections_reading gauge + nginx_connections_reading 0 + +Prometheus provides clients in various languages. For details about the clients, see `Prometheus CLIENT LIBRARIES `__. For details about how to develop an exporter, see `WRITING EXPORTERS `__. The Prometheus community provides various third-party exporters that can be directly used. For details, see `EXPORTERS AND INTEGRATIONS `__. + +Preparing an Application +------------------------ + +Nginx has a module named **ngx_http_stub_status_module**, which provides basic monitoring functions. You can configure the **nginx.conf** file to provide an API for external systems to access Nginx monitoring data. As shown in the following figure, after the server configuration is added to **http**, Nginx can provide an API for external systems to access Nginx monitoring data. + +.. code-block:: + + user nginx; + worker_processes auto; + + error_log /var/log/nginx/error.log warn; + pid /var/run/nginx.pid; + + events { + worker_connections 1024; + } + + http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + sendfile on; + #tcp_nopush on; + keepalive_timeout 65; + #gzip on; + include /etc/nginx/conf.d/*.conf; + + server { + listen 8080; + server_name localhost; + location /stub_status { + stub_status on; + access_log off; + } + } + } + +Save the preceding configuration to the **nginx.conf** file and use the configuration to create a new image. The Dockerfile file is as follows: + +.. code-block:: + + FROM nginx:1.21.5-alpine + ADD nginx.conf /etc/nginx/nginx.conf + EXPOSE 80 + CMD ["nginx", "-g", "daemon off;"] + +Use the preceding Dockerfile file to build an image and upload it to SWR. The image name is **nginx:exporter**. + +**docker build -t nginx:exporter .** + +**docker tag nginx:exporter {swr-address}/{group}/nginx:exporter** + +**docker push {swr-address}/{group}/nginx:exporter** + +After running a container with image **nginx:exporter**, you can obtain Nginx monitoring data by calling http://**:8080/stub_status. *< ip_address >* indicates the IP address of the container. The monitoring data is as follows: + +.. code-block:: + + # curl http://127.0.0.1:8080/stub_status + Active connections: 3 + server accepts handled requests + 146269 146269 212 + Reading: 0 Writing: 1 Waiting: 2 + +Deploying an Application +------------------------ + +The data format of the monitoring data provided by **nginx:exporter** does not meet the requirements of Prometheus. You need to convert the data format to the format required by Prometheus. To convert the format of Nginx metrics, use `nginx-prometheus-exporter `__, as shown in the following figure. + +|image2| + +Deploy **nginx:exporter** and **nginx-prometheus-exporter** in the same pod. + +.. code-block:: + + kind: Deployment + apiVersion: apps/v1 + metadata: + name: nginx-exporter + namespace: default + spec: + replicas: 1 + selector: + matchLabels: + app: nginx-exporter + template: + metadata: + labels: + app: nginx-exporter + annotations: + metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"prometheus","path":"/metrics","port":"9113","names":""}]' + spec: + containers: + - name: container-0 + image: 'nginx:exporter' # Replace it with the address of the image you uploaded to SWR. + resources: + limits: + cpu: 250m + memory: 512Mi + requests: + cpu: 250m + memory: 512Mi + - name: container-1 + image: 'nginx/nginx-prometheus-exporter:0.9.0' + command: + - nginx-prometheus-exporter + args: + - '-nginx.scrape-uri=http://127.0.0.1:8080/stub_status' + imagePullSecrets: + - name: default-secret + +.. note:: + + The nginx/nginx-prometheus-exporter:0.9.0 image needs to be pulled from the public network. Therefore, each node in the cluster must have a public IP address. + +nginx-prometheus-exporter requires a startup command. **nginx-prometheus-exporter -nginx.scrape-uri=http://127.0.0.1:8080/stub_status** is used to obtain Nginx monitoring data. + +In addition, you need to add an annotation **metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"prometheus","path":"/metrics","port":"9113","names":""}]'** to the pod. + +Verification +------------ + +After an application is deployed, you can access Nginx to construct some access data and check whether the corresponding monitoring data can be obtained in AOM. + +.. code-block:: + + $ kubectl get pod + NAME READY STATUS RESTARTS AGE + nginx-exporter-78859765db-6j8sw 2/2 Running 0 4m + $ kubectl exec -it nginx-exporter-78859765db-6j8sw -- /bin/sh + Defaulting container name to container-0. + Use 'kubectl describe pod/nginx-exporter-78859765db-6j8sw -n default' to see all of the containers in this pod. + / # curl http://localhost + + + + Welcome to nginx! + + + +

Welcome to nginx!

+

If you see this page, the nginx web server is successfully installed and + working. Further configuration is required.

+ +

For online documentation and support please refer to + nginx.org.
+ Commercial support is available at + nginx.com.

+ +

Thank you for using nginx.

+ + + / # + +You can see that Nginx has been accessed once. + +Log in to AOM. In the navigation pane, choose **Monitoring** > **Metric Monitoring**. You can view Nginx-related metrics, for example, **nginx_connections_active**. + +.. |image1| image:: /_static/images/en-us_image_0000001199501262.png +.. |image2| image:: /_static/images/en-us_image_0000001243981177.png diff --git a/umn/source/monitoring_and_alarm/index.rst b/umn/source/monitoring_and_alarm/index.rst new file mode 100644 index 0000000..8632ca8 --- /dev/null +++ b/umn/source/monitoring_and_alarm/index.rst @@ -0,0 +1,16 @@ +:original_name: cce_10_0110.html + +.. _cce_10_0110: + +Monitoring and Alarm +==================== + +- :ref:`Monitoring Overview ` +- :ref:`Custom Monitoring ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + monitoring_overview + custom_monitoring diff --git a/umn/source/monitoring_and_logs/monitoring_overview.rst b/umn/source/monitoring_and_alarm/monitoring_overview.rst similarity index 65% rename from umn/source/monitoring_and_logs/monitoring_overview.rst rename to umn/source/monitoring_and_alarm/monitoring_overview.rst index 3cd3698..2578d8d 100644 --- a/umn/source/monitoring_and_logs/monitoring_overview.rst +++ b/umn/source/monitoring_and_alarm/monitoring_overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0182.html +:original_name: cce_10_0182.html -.. _cce_01_0182: +.. _cce_10_0182: Monitoring Overview =================== @@ -9,17 +9,19 @@ CCE works with AOM to comprehensively monitor clusters. When a node is created, - Resource metrics - Basic resource monitoring includes CPU, memory, and disk monitoring. For details, see :ref:`Resource Metrics `. You can view these metrics of clusters, nodes, and workloads on the CCE or AOM console. + Basic resource monitoring includes CPU, memory, and disk monitoring. For details, see :ref:`Resource Metrics `. You can view these metrics of clusters, nodes, and workloads on the CCE or AOM console. -.. important:: +- Custom metrics - AOM is available only in certain regions. + The ICAgent collects custom metrics of applications and uploads them to AOM. For details, see :ref:`Custom Monitoring `. -.. _cce_01_0182__section205486212251: +.. _cce_10_0182__section205486212251: Resource Metrics ---------------- +On the CCE console, you can view the following metrics. + .. table:: **Table 1** Resource metrics +------------------------+------------------------------------------------------------------------------+ @@ -44,14 +46,12 @@ Resource Metrics | Disk Write Rate | Indicates the data volume written to a disk per second. The unit is KB/s. | +------------------------+------------------------------------------------------------------------------+ +On the AOM console, you can view host metrics and container metrics. + Viewing Cluster Monitoring Data ------------------------------- -In the navigation pane of the CCE console, choose **Resource Management** > **Clusters**. Click |image1| on the cluster card to access the cluster monitoring page. - -|image2| - -The cluster monitoring page displays the monitoring status of cluster resources, CPU, memory, and disk usage of all nodes in a cluster, and CPU and memory allocation rates. +Click the cluster name and access the cluster console. In the navigation pane, choose **Cluster Information**. In the right pane, you can view the CPU and memory usage of all nodes (excluding master nodes) in the cluster in the last hour. **Explanation of monitoring metrics:** @@ -62,42 +62,37 @@ The cluster monitoring page displays the monitoring status of cluster resources, .. note:: - Allocatable node resources (CPU or memory) = Total amount - Reserved amount - Eviction thresholds. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. + Allocatable node resources (CPU or memory) = Total amount - Reserved amount - Eviction thresholds. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. -On the cluster monitoring page, you can also view monitoring data of nodes, workloads, and pods. You can click |image3| to view the detailed data. - -|image4| - -Viewing Monitoring Data of Master Nodes ---------------------------------------- - -CCE allows you to view monitoring data of master nodes. You can view the monitoring data of a master node in the upper right corner of the cluster details page. Clicking **More** will direct you to the AOM console. +CCE provides the status, availability zone (AZ), CPU usage, and memory usage of master nodes. Viewing Monitoring Data of Worker Nodes --------------------------------------- -In addition to the cluster monitoring page, you can also view node monitoring data on the node console by clicking **Monitoring** in the row where the node resides. +In addition to viewing monitoring data of all nodes, you can also view monitoring data of a single node. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click **Monitor** in the **Operation** column of the target node. -The node list page also displays the data about the allocable resources of the node. **Allocatable resources** indicate the upper limit of resources that can be requested by pods on a node, and are calculated based on the requests. Allocatable resources do not indicate the actual available resources of the node. - -The calculation formulas are as follows: - -- Allocatable CPU = Total CPU - Requested CPU of all pods - Reserved CPU for other resources -- Allocatable memory = Total memory - Requested memory of all pods - Reserved memory for other resources +Monitoring data comes from AOM. You can view the monitoring data of a node, including the CPU, memory, disk, network, and GPU. Viewing Workload Monitoring Data -------------------------------- -You can view monitoring data of a workload on the **Monitoring** tab page of the workload details page. +You can view monitoring data of a workload on the **Monitoring** tab page of the workload details page. Click the cluster name and access the cluster console. Choose **Workloads** in the navigation pane and click **Monitor** in the **Operation** column of the target workload. -You can also click **AOM** to go to the AOM console to view monitoring data of the workload. +Monitoring data comes from AOM. You can view the monitoring data of a workload, including the CPU, memory, network, and GPU, on the AOM console. + +**Explanation of monitoring metrics:** + +- Workload CPU usage = Maximum CPU usage in each pod of the workload +- Workload memory usage = Maximum memory usage in each pod of the workload + +You can also click **View More** to go to the AOM console and view monitoring data of the workload. Viewing Pod Monitoring Data --------------------------- You can view monitoring data of a pod on the **Pods** tab page of the workload details page. -.. |image1| image:: /_static/images/en-us_image_0000001222591781.png -.. |image2| image:: /_static/images/en-us_image_0000001221007635.png -.. |image3| image:: /_static/images/en-us_image_0000001221376671.png -.. |image4| image:: /_static/images/en-us_image_0000001176255102.png +**Explanation of monitoring metrics:** + +- Pod CPU usage = The used CPU cores/The sum of all CPU limits of the pods (If not specified, all node CPU cores are used.) +- Pod memory usage = The used physical memory/The sum of all memory limits of pods (If not specified, all node memory is used.) diff --git a/umn/source/monitoring_and_logs/index.rst b/umn/source/monitoring_and_logs/index.rst deleted file mode 100644 index 42d65d5..0000000 --- a/umn/source/monitoring_and_logs/index.rst +++ /dev/null @@ -1,16 +0,0 @@ -:original_name: cce_01_0110.html - -.. _cce_01_0110: - -Monitoring and Logs -=================== - -- :ref:`Monitoring Overview ` -- :ref:`Container Logs ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - monitoring_overview - container_logs diff --git a/umn/source/namespaces/configuring_a_namespace-level_network_policy.rst b/umn/source/namespaces/configuring_a_namespace-level_network_policy.rst deleted file mode 100644 index 56455f1..0000000 --- a/umn/source/namespaces/configuring_a_namespace-level_network_policy.rst +++ /dev/null @@ -1,59 +0,0 @@ -:original_name: cce_01_0286.html - -.. _cce_01_0286: - -Configuring a Namespace-level Network Policy -============================================ - -You can configure a namespace-level network policy after enabling network isolation. - -By default, **Network Isolation** is disabled for namespaces. For example, if network isolation is off for namespace **default**, **all workloads in the current cluster** can access the workloads in namespace **default**. - -To prevent other workloads from accessing the workloads in namespace **default**, perform the following steps: - -.. important:: - - Only clusters that use the tunnel network model support network isolation. - -Prerequisites -------------- - -- You have created a Kubernetes cluster. For details, see :ref:`Creating a CCE Cluster `. -- You have created a namespace. For details, see :ref:`Creating a Namespace `. - -Procedure ---------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Namespaces**. - -#. Select the cluster to which the namespace belongs from the **Clusters** drop-down list. - -#. At the row of a namespace (for example, **default**), switch on **Network Isolation**. - - After network isolation is enabled, workloads in namespace **default** can access each other but they cannot be accessed by workloads in other namespaces. - - - .. figure:: /_static/images/en-us_image_0000001144779784.png - :alt: **Figure 1** Namespace-level network policy - - **Figure 1** Namespace-level network policy - -Network Isolation Description ------------------------------ - -Enabling network isolation is to create a network policy in a namespace. The network policy selects all pods in the namespace and prevents pods in other namespaces from accessing. - -.. code-block:: - - kind: NetworkPolicy - apiVersion: networking.k8s.io/v1 - metadata: - name: deny-default - namespace: default - spec: - ingress: - - from: - - podSelector: {} - podSelector: {} # {} indicates that all pods are selected. - -You can also customize a network policy. For details, see :ref:`Network Policies `. diff --git a/umn/source/namespaces/creating_a_namespace.rst b/umn/source/namespaces/creating_a_namespace.rst index 5ea4491..74a688d 100644 --- a/umn/source/namespaces/creating_a_namespace.rst +++ b/umn/source/namespaces/creating_a_namespace.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0278.html +:original_name: cce_10_0278.html -.. _cce_01_0278: +.. _cce_10_0278: Creating a Namespace ==================== @@ -15,7 +15,7 @@ For example, you can deploy workloads in a development environment into one name Prerequisites ------------- -At least one cluster has been created. For details, see :ref:`Creating a CCE Cluster `. +At least one cluster has been created. Notes and Constraints --------------------- @@ -40,28 +40,24 @@ Namespaces can be created in either of the following ways: Creating a Namespace -------------------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Namespaces**. Click **Create Namespace**. +#. Log in to the CCE console and access the cluster console. -#. Set the parameters listed in :ref:`Table 1 `. The parameters marked with an asterisk (*) are mandatory. +#. Choose **Namespaces** in the navigation pane and click **Create Namespace** in the upper right corner. - .. _cce_01_0278__table5523151617575: +#. Set namespace parameters based on :ref:`Table 1 `. + + .. _cce_10_0278__table5523151617575: .. table:: **Table 1** Parameters for creating a namespace +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+==========================================================================================================================================================================================================================================================================================================+ - | \* Namespace | Unique name of the created namespace. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Cluster | Cluster to which the namespace belongs. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Node Affinity | If this parameter is set to on, workloads in this namespace will be scheduled only to nodes with specified labels. To add labels to a node, choose **Resource Management** > **Nodes** > **Manage Labels**. | - | | | - | | This parameter is displayed only for clusters of v1.13.10-r0 and later. | + | Name | Unique name of the created namespace. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Description | Description about the namespace. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Set Resource Quotas | Resource quotas can limit the amount of resources available in namespaces, achieving resource allocation by namespace. | + | Quota Management | Resource quotas can limit the amount of resources available in namespaces, achieving resource allocation by namespace. | | | | | | .. important:: | | | | @@ -70,20 +66,35 @@ Creating a Namespace | | | | | For example, the default number of pods that can be created on each node in a cluster is 110. If you create a cluster with 50 nodes, you can create a maximum of 5,500 pods. Therefore, you can set a resource quota to ensure that the total number of pods in all namespaces does not exceed 5,500. | | | | - | | Quotas can be configured for the following resources: | - | | | - | | - CPU (cores) | - | | - Memory (MiB) | - | | - StatefulSet | - | | - Deployment | - | | - Job | - | | - Cron job | - | | - Pod | - | | - Service | - | | | - | | Enter an integer. If the quota of a resource is set to **0**, no limit is posed on the resource. | + | | Enter an integer. If the quota of a resource is not specified, no limit is posed on the resource. | | | | | | If you want to limit the CPU or memory quota, you must specify the CPU or memory request value when creating a workload. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. When the configuration is complete, click **OK**. + +Using kubectl to Create a Namespace +----------------------------------- + +Define a namespace. + +.. code-block:: + + apiVersion: v1 + kind: Namespace + metadata: + name: custom-namespace + +Run the **kubectl** command to create it. + +.. code-block:: + + $ kubectl create -f custom-namespace.yaml + namespace/custom-namespace created + +You can also run the **kubectl create namespace** command to create a namespace. + +.. code-block:: + + $ kubectl create namespace custom-namespace + namespace/custom-namespace created diff --git a/umn/source/namespaces/index.rst b/umn/source/namespaces/index.rst index 726910e..ed86546 100644 --- a/umn/source/namespaces/index.rst +++ b/umn/source/namespaces/index.rst @@ -1,14 +1,13 @@ -:original_name: cce_01_0030.html +:original_name: cce_10_0030.html -.. _cce_01_0030: +.. _cce_10_0030: Namespaces ========== -- :ref:`Creating a Namespace ` -- :ref:`Managing Namespaces ` -- :ref:`Configuring a Namespace-level Network Policy ` -- :ref:`Setting a Resource Quota ` +- :ref:`Creating a Namespace ` +- :ref:`Managing Namespaces ` +- :ref:`Setting a Resource Quota ` .. toctree:: :maxdepth: 1 @@ -16,5 +15,4 @@ Namespaces creating_a_namespace managing_namespaces - configuring_a_namespace-level_network_policy setting_a_resource_quota diff --git a/umn/source/namespaces/managing_namespaces.rst b/umn/source/namespaces/managing_namespaces.rst index e3c624d..7346156 100644 --- a/umn/source/namespaces/managing_namespaces.rst +++ b/umn/source/namespaces/managing_namespaces.rst @@ -1,12 +1,12 @@ -:original_name: cce_01_0285.html +:original_name: cce_10_0285.html -.. _cce_01_0285: +.. _cce_10_0285: Managing Namespaces =================== -Selecting a Namespace ---------------------- +Using Namespaces +---------------- - When creating a workload, you can select a namespace to isolate resources or users. - When querying workloads, you can select a namespace to view all workloads in the namespace. @@ -29,7 +29,7 @@ Isolating Namespaces The following figure shows namespaces created for the development, joint debugging, and testing environments, respectively. - .. figure:: /_static/images/en-us_image_0000001098645539.png + .. figure:: /_static/images/en-us_image_0000001199021298.png :alt: **Figure 1** One namespace for one environment **Figure 1** One namespace for one environment @@ -39,7 +39,7 @@ Isolating Namespaces You are advised to use this method if a large number of workloads are deployed in the same environment. For example, in the following figure, different namespaces (APP1 and APP2) are created to logically manage workloads as different groups. Workloads in the same namespace access each other using the Service name, and workloads in different namespaces access each other using the Service name or namespace name. - .. figure:: /_static/images/en-us_image_0000001098403383.png + .. figure:: /_static/images/en-us_image_0000001243981147.png :alt: **Figure 2** Grouping workloads into different namespaces **Figure 2** Grouping workloads into different namespaces @@ -49,10 +49,8 @@ Deleting a Namespace If a namespace is deleted, all resources (such as workloads, jobs, and ConfigMaps) in this namespace will also be deleted. Exercise caution when deleting a namespace. -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Namespaces**. +#. Log in to the CCE console and access the cluster console. -#. Select the cluster to which the namespace belongs from the **Clusters** drop-down list. - -#. Select the namespace to be deleted and click **Delete**. +#. In the navigation pane, choose **Namespaces**, select the target namespace, and choose **More** > **Delete**. Follow the prompts to delete the namespace. The default namespaces cannot be deleted. diff --git a/umn/source/namespaces/setting_a_resource_quota.rst b/umn/source/namespaces/setting_a_resource_quota.rst index 8972842..7a030b5 100644 --- a/umn/source/namespaces/setting_a_resource_quota.rst +++ b/umn/source/namespaces/setting_a_resource_quota.rst @@ -1,22 +1,12 @@ -:original_name: cce_01_0287.html +:original_name: cce_10_0287.html -.. _cce_01_0287: +.. _cce_10_0287: Setting a Resource Quota ======================== Namespace-level resource quotas limit the amount of resources available to teams or users when these teams or users use the same cluster. The quotas include the total number of a type of objects and the total amount of compute resources (CPU and memory) consumed by the objects. -.. note:: - - Quotas can be set only in clusters of v1.9 or later. - -Prerequisites -------------- - -- You have created a Kubernetes cluster. For details, see :ref:`Creating a CCE Cluster `. -- You have created a namespace. For details, see :ref:`Creating a Namespace `. - Usage ----- @@ -37,9 +27,9 @@ Cluster Scale Recommended Number of Pods 2,000 nodes 50,000 pods ============= ========================== -Starting from clusters of v1.21 and later, the default `Resource Quotas `__ are created when a namespace is created. :ref:`Table 1 ` lists the resource quotas based on cluster specifications. You can modify them according to your service requirements. +Starting from clusters of v1.21 and later, the default `Resource Quotas `__ are created when a namespace is created if you have enabled **enable-resource-quota** in :ref:`Managing Cluster Components `. :ref:`Table 1 ` lists the resource quotas based on cluster specifications. You can modify them according to your service requirements. -.. _cce_01_0287__table371165714613: +.. _cce_10_0287__table371165714613: .. table:: **Table 1** Default resource quotas @@ -52,28 +42,24 @@ Starting from clusters of v1.21 and later, the default `Resource Quotas **Namespaces**. +#. Log in to the CCE console and access the cluster console. -#. Select the cluster to which the namespace belongs from the **Clusters** drop-down list. +#. In the navigation pane, click **Namespaces**. -#. In the **Operation** column of a namespace, click **Manage Quota**. +#. Click **Quota Management** next to the target namespace. This operation cannot be performed on system namespaces **kube-system** and **kube-public**. #. Set the resource quotas and click **OK**. - - **CPU (cores)**: maximum number of CPU cores that can be allocated to workload pods in the namespace. - - **Memory (MiB)**: maximum amount of memory that can be allocated to workload pods in the namespace. - - **StatefulSet**: maximum number of StatefulSets that can be created in the namespace. - - **Deployment**: maximum number of Deployments that can be created in the namespace. - - **Job**: maximum number of one-off jobs that can be created in the namespace. - - **Cron Job**: maximum number of cron jobs that can be created in the namespace. - - **Pod**: maximum number of pods that can be created in the namespace. - - **Service**: maximum number of Services that can be created in the namespace. - .. important:: - After setting CPU and memory quotas for a namespace, you must specify the request and limit values of CPU and memory resources when creating a workload. Otherwise, the workload cannot be created. If the quota of a resource is set to **0**, the resource usage is not limited. diff --git a/umn/source/networking/accessing_public_networks_from_a_container.rst b/umn/source/networking/accessing_public_networks_from_a_container.rst new file mode 100644 index 0000000..67560d5 --- /dev/null +++ b/umn/source/networking/accessing_public_networks_from_a_container.rst @@ -0,0 +1,73 @@ +:original_name: cce_10_0400.html + +.. _cce_10_0400: + +Accessing Public Networks from a Container +========================================== + +Containers can access public networks in either of the following ways: + +- Bind a public IP address to the node where the container is located if the network model is VPC network or tunnel network. +- Bind a public IP address to the pod. (When the Cloud Native Network 2.0 model is used, manually bind an EIP to the ENI or sub-ENI of the pod on the VPC console. This method is not recommended because the IP address of a pod changes after the pod is rescheduled. As a result, the new pod cannot access the public network.) +- Configure SNAT rules through NAT Gateway. + +You can use NAT Gateway to enable container pods in a VPC to access public networks. NAT Gateway provides source network address translation (SNAT), which translates private IP addresses to a public IP address by binding an elastic IP address (EIP) to the gateway, providing secure and efficient access to the Internet. :ref:`Figure 1 ` shows the SNAT architecture. The SNAT function allows the container pods in a VPC to access the Internet without being bound to an EIP. SNAT supports a large number of concurrent connections, which makes it suitable for applications involving a large number of requests and connections. + +.. _cce_10_0400__cce_bestpractice_00274_0_en-us_topic_0241700138_en-us_topic_0144420145_fig34611314153619: + +.. figure:: /_static/images/en-us_image_0000001192028618.png + :alt: **Figure 1** SNAT + + **Figure 1** SNAT + +To enable a container pod to access the Internet, perform the following steps: + +#. Assign an EIP. + + a. Log in to the management console. + b. Click |image1| in the upper left corner of the management console and select a region and a project. + c. Click |image2| at the upper left corner and choose **Networking** > **Elastic IP** in the expanded list. + d. On the **EIPs** page, click **Create** **EIP**. + e. Set parameters as required. + + .. note:: + + Set **Region** to the region where container pods are located. + +#. Create a NAT gateway. + + a. Log in to the management console. + b. Click |image3| in the upper left corner of the management console and select a region and a project. + c. Click |image4| at the upper left corner and choose **Networking** > **NAT Gateway** in the expanded list. + d. On the displayed page, click **Create** **Public NAT Gateway** in the upper right corner. + e. Set parameters as required. + + .. note:: + + Select the same VPC. + +#. Configure an SNAT rule and bind the EIP to the subnet. + + a. Log in to the management console. + b. Click |image5| in the upper left corner of the management console and select a region and a project. + c. Click |image6| at the upper left corner and choose **Networking** > **NAT Gateway** in the expanded list. + d. On the page displayed, click the name of the NAT gateway for which you want to add the SNAT rule. + e. On the **SNAT Rules** tab page, click **Add SNAT Rule**. + f. Set parameters as required. + + .. note:: + + SNAT rules take effect by CIDR block. As different container network models use different communication modes, the subnet needs to be selected according to the following rules: + + - Tunnel network and VPC network: Select the subnet where the node is located, that is, the subnet selected during node creation. + + If there are multiple CIDR blocks, you can create multiple SNAT rules or customize a CIDR block as long as the CIDR block contains the node subnet. + + After the SNAT rule is configured, workloads can access public networks from the container. Public networks can be pinged from the container. + +.. |image1| image:: /_static/images/en-us_image_0275445566.png +.. |image2| image:: /_static/images/en-us_image_0261818893.png +.. |image3| image:: /_static/images/en-us_image_0275445543.png +.. |image4| image:: /_static/images/en-us_image_0261818896.png +.. |image5| image:: /_static/images/en-us_image_0275452681.png +.. |image6| image:: /_static/images/en-us_image_0261818899.png diff --git a/umn/source/networking/configuring_intra-vpc_access.rst b/umn/source/networking/configuring_intra-vpc_access.rst new file mode 100644 index 0000000..495945a --- /dev/null +++ b/umn/source/networking/configuring_intra-vpc_access.rst @@ -0,0 +1,102 @@ +:original_name: cce_10_0399.html + +.. _cce_10_0399: + +Configuring Intra-VPC Access +============================ + +This section describes how to access an intranet from a container (outside the cluster in a VPC), including intra-VPC access and cross-VPC access. + +.. _cce_10_0399__section1940319933: + +Intra-VPC Access +---------------- + +The performance of accessing an intranet from a container varies depending on the container network models of a cluster. + +- **Container tunnel network** + + The container tunnel network encapsulates network data packets through tunnels based on the node network. A container can access other resources in the same VPC as long as the node can access the resources. If the access fails, check whether the security group of the peer resource allows access from the node where the container is located. + +- **Cloud Native Network 2.0** + + In the Cloud Native Network 2.0 model, a container is assigned an IP address from the CIDR block of a VPC. The container CIDR block is the subnet of the VPC where the node is located. The container can naturally communicate with other addresses in the VPC. If the access fails, check whether the security group of peer resources allows the access from the container CIDR block. + +- **VPC network** + + The VPC network model uses VPC routes to forward container traffic. The container CIDR block and the node VPC are not in the same CIDR block. When a container accesses other resources in the same VPC, **the security group of the peer resource must allow access of the container CIDR block**. + + For example, the CIDR block where the cluster node resides is 192.168.10.0/24, and the container CIDR block is 172.16.0.0/16. + + There is an ECS whose IP address is 192.168.10.52 in the VPC (outside the cluster). The security group of the ECS allows access of only the CIDR block of the cluster node. + + In this case, if you ping 192.168.10.52 from the container, the ping operation fails. + + .. code-block:: + + kubectl exec test01-6cbbf97b78-krj6h -it -- /bin/sh + / # ping 192.168.10.25 + PING 192.168.10.25 (192.168.10.25): 56 data bytes + ^C + --- 192.168.10.25 ping statistics --- + 104 packets transmitted, 0 packets received, 100% packet loss + + Configure the security group to allow access from the container CIDR block 172.16.0.0/16. + + In this case, 192.168.10.52 can be pinged from the container. + + .. code-block:: + + $ kubectl exec test01-6cbbf97b78-krj6h -it -- /bin/sh + / # ping 192.168.10.25 + PING 192.168.10.25 (192.168.10.25): 56 data bytes + 64 bytes from 192.168.10.25: seq=0 ttl=64 time=1.412 ms + 64 bytes from 192.168.10.25: seq=1 ttl=64 time=1.400 ms + 64 bytes from 192.168.10.25: seq=2 ttl=64 time=1.299 ms + 64 bytes from 192.168.10.25: seq=3 ttl=64 time=1.283 ms + ^C + --- 192.168.10.25 ping statistics --- + 4 packets transmitted, 4 packets received, 0% packet loss + +.. _cce_10_0399__section44190754210: + +Cross-VPC Access +---------------- + +Cross-VPC access is implemented by establishing a peering connection between VPCs. + +- In the container tunnel network model, a container can access the peer VPC only when the communication is enabled between the node network and the peer VPC. + +- Cloud Native Network 2.0 is similar to the container tunnel network. You only need to enable the communication between the subnet where the container is located and the peer VPC. + +- Each VPC network has an independent container CIDR block. In addition to the VPC CIDR block, the container CIDR block also needs to be connected. + + Assume that there are two VPCs. + + - vpc-demo: Its CIDR block is 192.168.0.0/16, the cluster is in vpc-demo, and the container CIDR block is 10.0.0.0/16. + - vpc-demo2: Its CIDR block is 10.1.0.0/16. + + Create a peering connection named **peering-demo** (the local VPC is vpc-demo and the peer VPC is vpc-demo2). Add the container CIDR block to the route of the peer VPC. + + After this configuration, you can access the container CIDR block 10.0.0.0/16 in vpc-demo2. During the access, pay attention to the security group configuration and enable the port configuration. + +Accessing Other Cloud Services +------------------------------ + +Common services that communicate with CCE through an intranet include RDS, DCS, Kafka, RabbitMQ, and ModelArts. + +In addition to the network configurations described in :ref:`Intra-VPC Access ` and :ref:`Cross-VPC Access `, you also need to check **whether these cloud services allow external access**. For example, the DCS Redis instance can be accessed only by the IP addresses in its whitelist. Generally, these cloud services can be accessed by IP addresses in the same VPC. However, the container CIDR block in the VPC network model is different from the CIDR block of the VPC. Therefore, you must add the container CIDR block to the whitelist. + +What If a Container Fails to Access an Intranet? +------------------------------------------------ + +If an intranet cannot be accessed from a container, perform the following operations: + +#. View the security group rule of the peer server to check whether the container is allowed to access the peer server. + + - The container tunnel network model needs to allow the IP address of the node where the container is located. + - The VPC network model needs to allow the container CIDR block. + - The Cloud Native Network 2.0 model needs to allow the subnet where the container is located. + +#. Check whether a whitelist is configured for the peer server. For example, the DCS Redis instance can be accessed only by the IP addresses in its whitelist. Add the container and node CIDR blocks to the whitelist. +#. Check whether the container engine is installed on the peer server and whether it conflicts with the container CIDR block in CCE. If a network conflict occurs, the access fails. diff --git a/umn/source/networking/container_network_models/cloud_native_network_2.0.rst b/umn/source/networking/container_network_models/cloud_native_network_2.0.rst index 7ae00d8..c24088f 100644 --- a/umn/source/networking/container_network_models/cloud_native_network_2.0.rst +++ b/umn/source/networking/container_network_models/cloud_native_network_2.0.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0284.html +:original_name: cce_10_0284.html -.. _cce_01_0284: +.. _cce_10_0284: Cloud Native Network 2.0 ======================== @@ -11,21 +11,17 @@ Model Definition Developed by CCE, Cloud Native Network 2.0 deeply integrates Elastic Network Interfaces (ENIs) and sub-ENIs of Virtual Private Cloud (VPC). Container IP addresses are allocated from the VPC CIDR block. ELB passthrough networking is supported to direct access requests to containers. Security groups and elastic IPs (EIPs) are bound to deliver high performance. -.. figure:: /_static/images/en-us_image_0000001231949185.png +.. figure:: /_static/images/en-us_image_0000001199181336.png :alt: **Figure 1** Cloud Native Network 2.0 **Figure 1** Cloud Native Network 2.0 **Pod-to-pod communication** +- Pods on BMS nodes use ENIs, whereas pods on ECS nodes use Sub-ENIs. Sub-ENIs are attached to ENIs through VLAN sub-interfaces. - On the same node: Packets are forwarded through the VPC ENI or sub-ENI. - Across nodes: Packets are forwarded through the VPC ENI or sub-ENI. -Notes and Constraints ---------------------- - -This network model is available only to CCE Turbo clusters. - Advantages and Disadvantages ---------------------------- @@ -45,30 +41,10 @@ Application Scenarios - High performance requirements and use of other VPC network capabilities: Cloud Native Network 2.0 directly uses VPC, which delivers almost the same performance as the VPC network. Therefore, it is applicable to scenarios that have high requirements on bandwidth and latency, such as online live broadcast and e-commerce seckill. - Large-scale networking: Cloud Native Network 2.0 supports a maximum of 2000 ECS nodes and 100,000 containers. -Container IP Address Management -------------------------------- - -In the Cloud Native Network 2.0 model, BMS nodes use ENIs and ECS nodes use sub-ENIs. The following figure shows how IP addresses are managed on these nodes. - - -.. figure:: /_static/images/en-us_image_0000001172076961.png - :alt: **Figure 2** IP address management in Cloud Native Network 2.0 - - **Figure 2** IP address management in Cloud Native Network 2.0 - -- Pod IP addresses are allocated from **Pod Subnet** you configure from the VPC. -- ENIs and sub-ENIs bound to an ECS node = Number of ENIs used to bear sub-ENIs + Number of sub-ENIs currently used by pods + Number of sub-ENIs to be bound -- ENIs bound to a BMS node = Number of ENIs currently used by pods + Number of pre-bound ENIs -- Pre-binding policy: The system periodically (every 2 minutes by default) checks whether the total number of ENIs on the node. If the low threshold is not reached, the system pre-binds ENIs. If the high threshold is exceeded, the system releases ENIs. -- On an ECS node, when the number of pre-bound sub-ENIs plus the number of sub-ENIs currently used by the pods is **smaller than** the number of sub-ENIs at the low threshold (sub-ENI quota on the node x low threshold), the system **pre-binds** sub-ENIs to make the numbers equal. -- On an ECS node, when the number of pre-bound sub-ENIs plus the number of sub-ENIs currently used by the pods is **larger than** the number of sub-ENIs at the high threshold (sub-ENI quota on the node x high threshold), the system **releases** sub-ENIs to make the numbers equal. -- On a BMS node, when the number of pre-bound ENIs plus the number of ENIs currently used by the pods is **smaller than** the number of ENIs at the low threshold (ENI quota on the node x low threshold), the system **pre-binds** ENIs to make the numbers equal. -- On a BMS node, when the number of pre-bound ENIs plus the number of ENIs currently used by the pods is **larger than** the number of ENIs at the high threshold (ENI quota on the node x high threshold), the system **releases** ENIs to make the numbers equal. - Recommendation for CIDR Block Planning -------------------------------------- -As described in :ref:`Cluster Network Structure `, network addresses in a cluster can be divided into three parts: node network, container network, and service network. When planning network addresses, consider the following aspects: +As described in :ref:`Cluster Network Structure `, network addresses in a cluster can be divided into three parts: node network, container network, and service network. When planning network addresses, consider the following aspects: - The three CIDR blocks cannot overlap. Otherwise, a conflict occurs. All subnets (including those created from the secondary CIDR block) in the VPC where the cluster resides cannot conflict with the container and Service CIDR blocks. - Ensure that each CIDR block has sufficient IP addresses. @@ -81,25 +57,19 @@ In the Cloud Native Network 2.0 model, the container CIDR block and node CIDR bl In addition, a subnet can be added to the container CIDR block after a cluster is created to increase the number of available IP addresses. In this case, ensure that the added subnet does not conflict with other subnets in the container CIDR block. -.. figure:: /_static/images/en-us_image_0000001159831938.png - :alt: **Figure 3** Configuring CIDR blocks +.. figure:: /_static/images/en-us_image_0000001244261171.png + :alt: **Figure 2** Configuring CIDR blocks - **Figure 3** Configuring CIDR blocks + **Figure 2** Configuring CIDR blocks Example of Cloud Native Network 2.0 Access ------------------------------------------ Create a CCE Turbo cluster, which contains three ECS nodes. - -.. figure:: /_static/images/en-us_image_0000001198867835.png - :alt: **Figure 4** Cluster network - - **Figure 4** Cluster network - Access the details page of one node. You can see that the node has one primary NIC and one extended NIC, and both of them are ENIs. The extended NIC belongs to the container CIDR block and is used to mount a sub-ENI to the pod. -Create a Deployment on the cluster. +Create a Deployment in the cluster. .. code-block:: diff --git a/umn/source/networking/container_network_models/container_tunnel_network.rst b/umn/source/networking/container_network_models/container_tunnel_network.rst index 2707e5f..7767566 100644 --- a/umn/source/networking/container_network_models/container_tunnel_network.rst +++ b/umn/source/networking/container_network_models/container_tunnel_network.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0282.html +:original_name: cce_10_0282.html -.. _cce_01_0282: +.. _cce_10_0282: Container Tunnel Network ======================== @@ -11,7 +11,7 @@ Container Tunnel Network Model The container tunnel network is constructed on but independent of the node network through tunnel encapsulation. This network model uses VXLAN to encapsulate Ethernet packets into UDP packets and transmits them in tunnels. Open vSwitch serves as the backend virtual switch. Though at some costs of performance, packet encapsulation and tunnel transmission enable higher interoperability and compatibility with advanced features (such as network policy-based isolation) for most common scenarios. -.. figure:: /_static/images/en-us_image_0000001145535931.png +.. figure:: /_static/images/en-us_image_0000001199341330.png :alt: **Figure 1** Container tunnel network **Figure 1** Container tunnel network @@ -27,7 +27,7 @@ Advantages and Disadvantages **Advantages** - The container network is decoupled from the node network and is not limited by the VPC quotas and response speed (such as the number of VPC routes, number of elastic ENIs, and creation speed). -- Network isolation is supported. For details, see :ref:`Network Policies `. +- Network isolation is supported. For details, see :ref:`Network Policies `. - Bandwidth limits are supported. - Large-scale networking is supported. @@ -43,8 +43,6 @@ Applicable Scenarios - Low requirements on performance: As the container tunnel network requires additional VXLAN tunnel encapsulation, it has about 5% to 15% of performance loss when compared with the other two container network models. Therefore, the container tunnel network is applicable to the scenarios that do not have high performance requirements, such as web applications, and middle-end and back-end services with a small number of access requests. - Large-scale networking: Different from the VPC network that is limited by the VPC route quota, the container tunnel network does not have any restriction on the infrastructure. In addition, the container tunnel network controls the broadcast domain to the node level. The container tunnel network supports a maximum of 2000 nodes. -.. _cce_01_0282__section182851515105616: - Container IP Address Management ------------------------------- @@ -57,7 +55,7 @@ The container tunnel network allocates container IP addresses according to the f - Pods scheduled to a node are cyclically allocated IP addresses from one or more CIDR blocks allocated to the node. -.. figure:: /_static/images/en-us_image_0000001198861255.png +.. figure:: /_static/images/en-us_image_0000001244141217.png :alt: **Figure 2** IP address allocation of the container tunnel network **Figure 2** IP address allocation of the container tunnel network @@ -69,7 +67,7 @@ For example, if the container CIDR block is 172.16.0.0/16, the number of IP addr Recommendation for CIDR Block Planning -------------------------------------- -As described in :ref:`Cluster Network Structure `, network addresses in a cluster can be divided into three parts: node network, container network, and service network. When planning network addresses, consider the following aspects: +As described in :ref:`Cluster Network Structure `, network addresses in a cluster can be divided into three parts: node network, container network, and service network. When planning network addresses, consider the following aspects: - The three CIDR blocks cannot overlap. Otherwise, a conflict occurs. - Ensure that each CIDR block has sufficient IP addresses. @@ -77,18 +75,10 @@ As described in :ref:`Cluster Network Structure `.) - Example of Container Tunnel Network Access ------------------------------------------ -Create a cluster that uses the container tunnel network model. - -Create a Deployment on the cluster. +Create a cluster that uses the container tunnel network model. Create a Deployment in the cluster. .. code-block:: diff --git a/umn/source/networking/container_network_models/index.rst b/umn/source/networking/container_network_models/index.rst index 49c9d39..644c199 100644 --- a/umn/source/networking/container_network_models/index.rst +++ b/umn/source/networking/container_network_models/index.rst @@ -1,14 +1,14 @@ -:original_name: cce_01_0280.html +:original_name: cce_10_0280.html -.. _cce_01_0280: +.. _cce_10_0280: Container Network Models ======================== -- :ref:`Overview ` -- :ref:`Container Tunnel Network ` -- :ref:`VPC Network ` -- :ref:`Cloud Native Network 2.0 ` +- :ref:`Overview ` +- :ref:`Container Tunnel Network ` +- :ref:`VPC Network ` +- :ref:`Cloud Native Network 2.0 ` .. toctree:: :maxdepth: 1 diff --git a/umn/source/networking/container_network_models/overview.rst b/umn/source/networking/container_network_models/overview.rst index 75d82a4..491b6a9 100644 --- a/umn/source/networking/container_network_models/overview.rst +++ b/umn/source/networking/container_network_models/overview.rst @@ -1,37 +1,40 @@ -:original_name: cce_01_0281.html +:original_name: cce_10_0281.html -.. _cce_01_0281: +.. _cce_10_0281: Overview ======== The container network assigns IP addresses to pods in a cluster and provides networking services. In CCE, you can select the following network models for your cluster: -- Container tunnel network -- VPC network -- Cloud Native Network 2.0 +- :ref:`Tunnel network ` +- :ref:`VPC network ` +- :ref:`Cloud Native Network 2.0 ` Network Model Comparison ------------------------ -:ref:`Table 1 ` describes the differences of network models supported by CCE. +:ref:`Table 1 ` describes the differences of network models supported by CCE. .. caution:: After a cluster is created, the network model cannot be changed. -.. _cce_01_0281__en-us_topic_0146398798_table715802210336: +.. _cce_10_0281__en-us_topic_0146398798_table715802210336: .. table:: **Table 1** Network model comparison +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ | Dimension | Tunnel Network | VPC Network | Cloud Native Network 2.0 | +========================+===================================================================================================================================+======================================================================================================================================================+============================================================================================================+ + | Application scenarios | - Common container service scenarios | - Scenarios that have high requirements on network latency and bandwidth | - Scenarios that have high requirements on network latency, bandwidth, and performance | + | | - Scenarios that do not have high requirements on network latency and bandwidth | - Containers can communicate with VMs using a microservice registration framework, such as Dubbo and CSE. | - Containers can communicate with VMs using a microservice registration framework, such as Dubbo and CSE. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ | Core technology | OVS | IPvlan and VPC route | VPC ENI/sub-ENI | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ | Applicable clusters | CCE cluster | CCE cluster | CCE Turbo cluster | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ - | Network isolation | Yes. For details, see :ref:`Network Policies `. | No | Yes. For details, see :ref:`SecurityGroups `. | + | Network isolation | Kubernetes native NetworkPolicy for pods | No | Pods support security group isolation. | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ | Passthrough networking | No | No | Yes | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ @@ -42,10 +45,7 @@ Network Model Comparison +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ | Networking scale | A maximum of 2,000 nodes are supported. | By default, 200 nodes are supported. | A maximum of 2,000 nodes are supported. | | | | | | - | | | Each time a node is added to the cluster, a route is added to the VPC routing table. Therefore, the cluster scale is limited by the VPC route table. | | - +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ - | Application scenarios | - Common container service scenarios | - Scenarios that have high requirements on network latency and bandwidth | - Scenarios that have high requirements on network latency, bandwidth, and performance | - | | - Scenarios that do not have high requirements on network latency and bandwidth | - Containers can communicate with VMs using a microservice registration framework, such as Dubbo and CSE. | - Containers can communicate with VMs using a microservice registration framework, such as Dubbo and CSE. | + | | | Each time a node is added to the cluster, a route is added to the VPC route tables. Therefore, the cluster scale is limited by the VPC route tables. | | +------------------------+-----------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------+ .. important:: @@ -53,3 +53,4 @@ Network Model Comparison #. The scale of a cluster that uses the VPC network model is limited by the custom routes of the VPC. Therefore, you need to estimate the number of required nodes before creating a cluster. #. The scale of a cluster that uses the Cloud Native Network 2.0 model depends on the size of the VPC subnet CIDR block selected for the network attachment definition. Before creating a cluster, evaluate the scale of your cluster. #. By default, VPC routing network supports direct communication between containers and hosts in the same VPC. If a peering connection policy is configured between the VPC and another VPC, the containers can directly communicate with hosts on the peer VPC. In addition, in hybrid networking scenarios such as Direct Connect and VPN, communication between containers and hosts on the peer end can also be achieved with proper planning. + #. Do not change the mask of the primary CIDR block on the VPC after a cluster is created. Otherwise, the network will be abnormal. diff --git a/umn/source/networking/container_network_models/vpc_network.rst b/umn/source/networking/container_network_models/vpc_network.rst index 847a78d..6ee0769 100644 --- a/umn/source/networking/container_network_models/vpc_network.rst +++ b/umn/source/networking/container_network_models/vpc_network.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0283.html +:original_name: cce_10_0283.html -.. _cce_01_0283: +.. _cce_10_0283: VPC Network =========== @@ -11,7 +11,7 @@ Model Definition The VPC network uses VPC routing to integrate with the underlying network. This network model is suitable for performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the VPC route quota. Each node is assigned a CIDR block of a fixed size. This networking model is free from tunnel encapsulation overhead and outperforms the container tunnel network model. In addition, as VPC routing includes routes to node IP addresses and the container CIDR block, container pods in a cluster can be directly accessed from outside the cluster. -.. figure:: /_static/images/en-us_image_0000001116237931.png +.. figure:: /_static/images/en-us_image_0000001199181338.png :alt: **Figure 1** VPC network model **Figure 1** VPC network model @@ -41,7 +41,7 @@ Applicable Scenarios - High performance requirements: As no tunnel encapsulation is required, the VPC network model delivers the performance close to that of a VPC network when compared with the container tunnel network model. Therefore, the VPC network model is applicable to scenarios that have high requirements on performance, such as AI computing and big data computing. - Small- and medium-scale networking: The VPC network is limited by the VPC route quota. Currently, a maximum of 200 nodes are supported by default. If there are large-scale networking requirements, you can increase the VPC route quota. -.. _cce_01_0283__section1574982552114: +.. _cce_10_0283__section1574982552114: Container IP Address Management ------------------------------- @@ -54,19 +54,19 @@ The VPC network allocates container IP addresses according to the following rule - Pods scheduled to a node are cyclically allocated IP addresses from CIDR blocks allocated to the node. -.. figure:: /_static/images/en-us_image_0000001153101092.png +.. figure:: /_static/images/en-us_image_0000001244261173.png :alt: **Figure 2** IP address management of the VPC network **Figure 2** IP address management of the VPC network Maximum number of nodes that can be created in the cluster using the VPC network = Number of IP addresses in the container CIDR block /Number of IP addresses in the CIDR block allocated to the node by the container CIDR block -For example, if the container CIDR block is 172.16.0.0/16, the number of IP addresses is 65536. The mask of the container CIDR block allocated to the node is 25. That is, the number of container IP addresses on each node is 128. Therefore, a maximum of 512 (65536/128) nodes can be created. The number of nodes that can be created in a cluster also depends on the node network and cluster scale. +For example, if the container CIDR block is 172.16.0.0/16, the number of IP addresses is 65536. The mask of the container CIDR block allocated to the node is 25. That is, the number of container IP addresses on each node is 128. Therefore, a maximum of 512 (65536/128) nodes can be created. In addition, the number of nodes that can be created in a cluster also depends on the node network and cluster scale. Recommendation for CIDR Block Planning -------------------------------------- -As described in :ref:`Cluster Network Structure `, network addresses in a cluster can be divided into three parts: node network, container network, and service network. When planning network addresses, consider the following aspects: +As described in :ref:`Cluster Network Structure `, network addresses in a cluster can be divided into three parts: node network, container network, and service network. When planning network addresses, consider the following aspects: - The three CIDR blocks cannot overlap. Otherwise, a conflict occurs. - Ensure that each CIDR block has sufficient IP addresses. @@ -78,20 +78,12 @@ Assume that a cluster contains 200 nodes and the network model is VPC network. In this case, the number of available IP addresses in the selected node subnet must be greater than 200. Otherwise, nodes cannot be created due to insufficient IP addresses. -The container CIDR block is 10.0.0.0/16, and the number of available IP addresses is 65536. As described in :ref:`Container IP Address Management `, the VPC network is allocated a CIDR block with the fixed size (using the mask to determine the maximum number of container IP addresses allocated to each node). For example, if the upper limit is 128, the cluster supports a maximum of 512 (65536/128) nodes, including the three master nodes. +The container CIDR block is 10.0.0.0/16, and the number of available IP addresses is 65536. As described in :ref:`Container IP Address Management `, the VPC network is allocated a CIDR block with the fixed size (using the mask to determine the maximum number of container IP addresses allocated to each node). For example, if the upper limit is 128, the cluster supports a maximum of 512 (65536/128) nodes, including the three master nodes. Example of VPC Network Access ----------------------------- -Create a cluster using the VPC network model. - - -.. figure:: /_static/images/en-us_image_0000001198980979.png - :alt: **Figure 3** Cluster network - - **Figure 3** Cluster network - -The cluster contains one node. +Create a cluster using the VPC network model. The cluster contains one node. .. code-block:: @@ -101,7 +93,7 @@ The cluster contains one node. Check the VPC routing table. The destination address 172.16.0.0/25 is the container CIDR block allocated to the node, and the next hop is the corresponding node. When the container IP address is accessed, the VPC route forwards the access request to the next-hop node. This indicates that the VPC network model uses VPC routes. -Create a Deployment on the cluster. +Create a Deployment in the cluster. .. code-block:: diff --git a/umn/source/networking/dns/dns_configuration.rst b/umn/source/networking/dns/dns_configuration.rst new file mode 100644 index 0000000..b6cb174 --- /dev/null +++ b/umn/source/networking/dns/dns_configuration.rst @@ -0,0 +1,319 @@ +:original_name: cce_10_0365.html + +.. _cce_10_0365: + +DNS Configuration +================= + +Every Kubernetes cluster has a built-in DNS add-on (Kube-DNS or CoreDNS) to provide domain name resolution for workloads in the cluster. When handling a high concurrency of DNS queries, Kube-DNS/CoreDNS may encounter a performance bottleneck, that is, it may fail occasionally to fulfill DNS queries. There are cases when Kubernetes workloads initiate unnecessary DNS queries. This makes DNS overloaded if there are many concurrent DNS queries. Tuning DNS configuration for workloads will reduce the risks of DNS query failures to some extent. + +For more information about DNS, see :ref:`coredns (System Resource Add-On, Mandatory) `. + +DNS Configuration Items +----------------------- + +Run the **cat /etc/resolv.conf** command on a Linux node or container to view the DNS resolver configuration file. The following is an example DNS resolver configuration of a container in a Kubernetes cluster: + +.. code-block:: + + nameserver 10.247.x.x + search default.svc.cluster.local svc.cluster.local cluster.local + options ndots:5 + +**Configuration Options** + +- **nameserver**: an IP address list of a name server that the resolver will query. If this parameter is set to 10.247.x.x, the resolver will query the kube-dns/CoreDNS. If this parameter is set to another IP address, the resolver will query a cloud or on-premises DNS server. + +- **search**: a search list for host-name lookup. When a domain name cannot be resolved, DNS queries will be attempted combining the domain name with each domain in the search list in turn until a match is found or all domains in the search list are tried. For CCE clusters, the search list is currently limited to three domains per container. When a nonexistent domain name is being resolved, eight DNS queries will be initiated because each domain name (including those in the search list) will be queried twice, one for IPv4 and the other for IPv6. + +- **options**: options that allow certain internal resolver variables to be modified. Common options include timeout and ndots. + + The value **ndots:5** means that if a domain name has fewer than 5 dots (.), DNS queries will be attempted by combining the domain name with each domain in the search list in turn. If no match is found after all the domains in the search list are tried, the domain name is then used for DNS query. If the domain name has 5 or more than 5 dots, it will be tried first for DNS query. In case that the domain name cannot be resolved, DNS queries will be attempted by combining the domain name with each domain in the search list in turn. + + For example, the domain name **www.***.com** has only two dots (smaller than the value of **ndots**), and therefore the sequence of DNS queries is as follows: **www.***.default.svc.cluster.local**, **www.***.com.svc.cluster.local**, **www.***.com.cluster.local**, and **www.***.com**. This means that at least seven DNS queries will be initiated before the domain name is resolved into an IP address. It is clear that when many unnecessary DNS queries will be initiated to access an external domain name. There is room for improvement in workload's DNS configuration. + +.. note:: + + For more information about configuration options in the resolver configuration file used by Linux operating systems, visit http://man7.org/linux/man-pages/man5/resolv.conf.5.html. + +Configuring DNS Using the Workload YAML +--------------------------------------- + +When creating a workload using a YAML file, you can configure the DNS settings in the YAML. The following is an example for an Nginx application: + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + namespace: default + spec: + replicas: 1 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - name: container-1 + image: nginx:latest + imagePullPolicy: IfNotPresent + imagePullSecrets: + - name: default-secret + dnsPolicy: None + dnsConfig: + options: + - name: ndots + value: '5' + - name: timeout + value: '3' + nameservers: + - 10.2.3.4 + searches: + - my.dns.search.suffix + +**dnsPolicy** + +The **dnsPolicy** field is used to configure a DNS policy for an application. The default value is **ClusterFirst**. The DNS parameters in **dnsConfig** will be merged to the DNS file generated according to **dnsPolicy**. The merge rules are later explained in :ref:`Table 2 `. Currently, **dnsPolicy** supports the following four values: + +.. _cce_10_0365__table144443315261: + +.. table:: **Table 1** dnsPolicy + + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=======================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ + | ClusterFirst (default value) | CCE cluster's CoreDNS, which is cascaded with the cloud DNS by default, is used for workloads. Containers can resolve both the cluster-internal domain names registered by a Service and the external domain names exposed to public networks. The search list (**search** option) and **ndots: 5** are present in the DNS configuration file. Therefore, when accessing an external domain name and a long cluster-internal domain name (for example, kubernetes.default.svc.cluster.local), the search list will usually be traversed first, resulting in at least six invalid DNS queries. The issue of invalid DNS queries disappears only when a short cluster-internal domain name (for example, kubernetes) is being accessed. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ClusterFirstWithHostNet | By default, the DNS configuration file that the **--resolv-conf** flag points to is configured for workloads running with **hostNetwork=true**, that is, a cloud DNS is used for CCE clusters. If workloads need to use Kube-DNS/CoreDNS of the cluster, set **dnsPolicy** to **ClusterFirstWithHostNet** and container's DNS configuration file is the same as ClusterFirst, in which invalid DNS queries still exist. | + | | | + | | .. code-block:: | + | | | + | | ... | + | | spec: | + | | containers: | + | | - image: nginx:latest | + | | imagePullPolicy: IfNotPresent | + | | name: container-1 | + | | restartPolicy: Always | + | | hostNetwork: true | + | | dnsPolicy: ClusterFirstWithHostNet | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Default | Container's DNS configuration file is the DNS configuration file that the kubelet's **--resolv-conf** flag points to. In this case, a cloud DNS is used for CCE clusters. Both **search** and **options** fields are left unspecified. This configuration can only resolve the external domain names registered with the Internet, and not cluster-internal domain names. This configuration is free from the issue of invalid DNS queries. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | None | If **dnsPolicy** is set to **None**, the **dnsConfig** field must be specified because all DNS settings are supposed to be provided using the **dnsConfig** field. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +.. note:: + + If the **dnsPolicy** field is not specified, the default value is **ClusterFirst** instead of **Default**. + +**dnsConfig** + +The **dnsConfig** field is used to configure DNS parameters for workloads. The configured parameters are merged to the DNS configuration file generated according to **dnsPolicy**. If **dnsPolicy** is set to **None**, the workload's DNS configuration file is specified by the **dnsConfig** field. If **dnsPolicy** is not set to **None**, the DNS parameters configured in **dnsConfig** are added to the DNS configuration file generated according to **dnsPolicy**. + +.. _cce_10_0365__table16581121652515: + +.. table:: **Table 2** dnsConfig + + +-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +=============+================================================================================================================================================================================================================================================================================================================================================+ + | options | An optional list of objects where each object may have a name property (required) and a value property (optional). The contents in this property will be merged to the options generated from the specified DNS policy in **dnsPolicy**. Duplicate entries are removed. | + +-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | nameservers | A list of IP addresses that will be used as DNS servers. If workload's **dnsPolicy** is set to **None**, the list must contain at least one IP address, otherwise this property is optional. The servers listed will be combined to the nameservers generated from the specified DNS policy in **dnsPolicy** with duplicate addresses removed. | + +-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | searches | A list of DNS search domains for hostname lookup in the Pod. This property is optional. When specified, the provided list will be merged into the search domain names generated from the chosen DNS policy in **dnsPolicy**. Duplicate domain names are removed. Kubernetes allows for at most 6 search domains. | + +-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +Configuring DNS for a Workload Using the Console +------------------------------------------------ + +Kubernetes provides DNS-related configuration options for applications. The use of application's DNS configuration can effectively reduce unnecessary DNS queries in certain scenarios and improve service concurrency. The following procedure uses an Nginx application as an example to describe how to add DNS configurations for a workload on the console. + +#. Log in to the CCE console, access the cluster console, select **Workloads** in the navigation pane, and click **Create Workload** in the upper right corner. +#. Configure basic information about the workload. For details, see :ref:`Creating a Deployment `. +#. In the **Advanced Settings** area, click the **DNS** tab and set the following parameters as required: + + - **DNS Policy**: The DNS policies provided on the console correspond to the **dnsPolicy** field in the YAML file. For details, see :ref:`Table 1 `. + + - **Supplement defaults**: corresponds to **dnsPolicy=ClusterFirst**. Containers can resolve both the cluster-internal domain names registered by a Service and the external domain names exposed to public networks. + - **Replace defaults**: corresponds to **dnsPolicy=None**. You must configure **IP Address** and **Search Domain**. Containers only use the user-defined IP address and search domain configurations for domain name resolution. + - **Inherit defaults**: corresponds to **dnsPolicy=Default**. Containers use the domain name resolution configuration from the node that pods run on and cannot resolve the cluster-internal domain names. + + - **Optional Objects**: The options parameters in the :ref:`dnsConfig field `. Each object may have a name property (required) and a value property (optional). After setting the properties, click **confirm to add**. + + - **timeout**: Timeout interval, in seconds. + - **ndots**: Number of dots (.) that must be present in a domain name. If a domain name has dots fewer than this value, the operating system will look up the name in the search domain. If not, the name is a fully qualified domain name (FQDN) and will be tried first as an absolute name. + + - **IP Address**: **nameservers** in the :ref:`dnsConfig field `. You can configure the domain name server for the custom domain name. The value is one or a group of DNS IP addresses. + - **Search Domain**: **searches** in the :ref:`dnsConfig field `. A list of DNS search domains for hostname lookup in the pod. This property is optional. When specified, the provided list will be merged into the search domain names generated from the chosen DNS policy in **dnsPolicy**. Duplicate domain names are removed. + +#. Click **Create Workload**. + +Configuration Examples +---------------------- + +The following example describes how to configure DNS for workloads. + +- **Use Case 1: Using Kube-DNS/CoreDNS Built in Kubernetes Clusters** + + **Scenario** + + Kubernetes in-cluster Kube-DNS/CoreDNS is applicable to resolving only cluster-internal domain names or cluster-internal domain names + external domain names. This is the default DNS for workloads. + + **Example:** + + .. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + namespace: default + name: dns-example + spec: + containers: + - name: test + image: nginx:alpine + dnsPolicy: ClusterFirst + + Container's DNS configuration file: + + .. code-block:: + + nameserver 10.247.3.10 + search default.svc.cluster.local svc.cluster.local cluster.local + options ndots:5 + +- **Use Case 2: Using a Cloud DNS** + + **Scenario** + + A DNS cannot resolve cluster-internal domain names and therefore is applicable to the scenario where workloads access only external domain names registered with the Internet. + + **Example:** + + .. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + namespace: default + name: dns-example + spec: + containers: + - name: test + image: nginx:alpine + dnsPolicy: Default//The DNS configuration file that the kubelet's --resolv-conf flag points to is used. In this case, a DNS is used for CCE clusters. + + Container's DNS configuration file: + + .. code-block:: + + nameserver 100.125.x.x + +- **Use Case 3: Using Kube-DNS/CoreDNS for Workloads Running with hostNetwork** + + **Scenario** + + By default, a DNS is used for workloads running with hostNetwork. If workloads need to use Kube-DNS/CoreDNS, set **dnsPolicy** to **ClusterFirstWithHostNet**. + + **Example:** + + .. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + name: nginx + spec: + hostNetwork: true + dnsPolicy: ClusterFirstWithHostNet + containers: + - name: nginx + image: nginx:alpine + ports: + - containerPort: 80 + + Container's DNS configuration file: + + .. code-block:: + + nameserver 10.247.3.10 + search default.svc.cluster.local svc.cluster.local cluster.local + options ndots:5 + +- **Use Case 4: Customizing Application's DNS Configuration** + + **Scenario** + + You can flexibly customize the DNS configuration file for applications. Using **dnsPolicy** and **dnsConfig** together can address almost all scenarios, including the scenarios in which an on-premises DNS will be used, multiple DNSs will be cascaded, and DNS configuration options will be modified. + + **Example 1: Using Your On-Premises DNS** + + *Set* **dnsPolicy** *to* **None** *so application's DNS configuration file is generated based on* **dnsConfig**\ *.* + + .. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + namespace: default + name: dns-example + spec: + containers: + - name: test + image: nginx:alpine + dnsPolicy: "None" + dnsConfig: + nameservers: + - 10.2.3.4 //IP address of your on-premises DNS + searches: + - ns1.svc.cluster.local + - my.dns.search.suffix + options: + - name: ndots + value: "2" + - name: timeout + value: "3" + + Container's DNS configuration file: + + .. code-block:: + + nameserver 10.2.3.4 + search ns1.svc.cluster.local my.dns.search.suffix + options timeout:3 ndots:2 + + **Example 2: Modifying the ndots Option in the DNS Configuration File to Reduce Invalid DNS Queries** + + Set **dnsPolicy** to a value other than **None** so the DNS parameters configured in **dnsConfig** are added to the DNS configuration file generated based on **dnsPolicy**. + + .. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + namespace: default + name: dns-example + spec: + containers: + - name: test + image: nginx:alpine + dnsPolicy: "ClusterFirst" + dnsConfig: + options: + - name: ndots + value: "2" //Changes the ndots:5 option in the DNS configuration file generated based on the ClusterFirst policy to ndots:2. + + Container's DNS configuration file: + + .. code-block:: + + nameserver 10.247.3.10 + search default.svc.cluster.local svc.cluster.local cluster.local + options ndots:2 diff --git a/umn/source/networking/dns/index.rst b/umn/source/networking/dns/index.rst new file mode 100644 index 0000000..0092714 --- /dev/null +++ b/umn/source/networking/dns/index.rst @@ -0,0 +1,18 @@ +:original_name: cce_10_0359.html + +.. _cce_10_0359: + +DNS +=== + +- :ref:`Overview ` +- :ref:`DNS Configuration ` +- :ref:`Using CoreDNS for Custom Domain Name Resolution ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + overview + dns_configuration + using_coredns_for_custom_domain_name_resolution diff --git a/umn/source/networking/dns/overview.rst b/umn/source/networking/dns/overview.rst new file mode 100644 index 0000000..b3b0ea8 --- /dev/null +++ b/umn/source/networking/dns/overview.rst @@ -0,0 +1,59 @@ +:original_name: cce_10_0360.html + +.. _cce_10_0360: + +Overview +======== + +Introduction to CoreDNS +----------------------- + +When you create a cluster, the :ref:`coredns add-on ` is installed to resolve domain names in the cluster. + +You can view the pod of the coredns add-on in the kube-system namespace. + +.. code-block:: + + $ kubectl get po --namespace=kube-system + NAME READY STATUS RESTARTS AGE + coredns-7689f8bdf-295rk 1/1 Running 0 9m11s + coredns-7689f8bdf-h7n68 1/1 Running 0 11m + +After coredns is installed, it becomes a DNS. After the Service is created, coredns records the Service name and IP address. In this way, the pod can obtain the Service IP address by querying the Service name from coredns. + +**nginx..svc.cluster.local** is used to access the Service. **nginx** is the Service name, **** is the namespace, and **svc.cluster.local** is the domain name suffix. In actual use, you can omit **.svc.cluster.local** in the same namespace and use the ServiceName. + +An advantage of using ServiceName is that you can write ServiceName into the program when developing the application. In this way, you do not need to know the IP address of a specific Service. + +After the coredns add-on is installed, there is also a Service in the kube-system namespace, as shown below. + +.. code-block:: + + $ kubectl get svc -n kube-system + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + coredns ClusterIP 10.247.3.10 53/UDP,53/TCP,8080/TCP 13d + +By default, after other pods are created, the address of the coredns Service is written as the address of the domain name resolution server in the **/etc/resolv.conf** file of the pod. Create a pod and view the **/etc/resolv.conf** file as follows: + +.. code-block:: + + $ kubectl exec test01-6cbbf97b78-krj6h -it -- /bin/sh + / # cat /etc/resolv.conf + nameserver 10.247.3.10 + search default.svc.cluster.local svc.cluster.local cluster.local + options ndots:5 timeout single-request-reopen + +When a user accesses the *Service name:Port* of the Nginx pod, the IP address of the Nginx Service is resolved from CoreDNS, and then the IP address of the Nginx Service is accessed. In this way, the user can access the backend Nginx pod. + + +.. figure:: /_static/images/en-us_image_0000001244261167.png + :alt: **Figure 1** Example of domain name resolution in a cluster + + **Figure 1** Example of domain name resolution in a cluster + +Related Operations +------------------ + +You can also configure DNS in a workload. For details, see :ref:`DNS Configuration `. + +You can also use coredns to implement user-defined domain name resolution. For details, see :ref:`Using CoreDNS for Custom Domain Name Resolution `. diff --git a/umn/source/networking/dns/using_coredns_for_custom_domain_name_resolution.rst b/umn/source/networking/dns/using_coredns_for_custom_domain_name_resolution.rst new file mode 100644 index 0000000..4453ad2 --- /dev/null +++ b/umn/source/networking/dns/using_coredns_for_custom_domain_name_resolution.rst @@ -0,0 +1,253 @@ +:original_name: cce_10_0361.html + +.. _cce_10_0361: + +Using CoreDNS for Custom Domain Name Resolution +=============================================== + +Challenges +---------- + +When using CCE, you may need to resolve custom internal domain names in the following scenarios: + +- In the legacy code, a fixed domain name is configured for calling other internal services. If the system decides to use Kubernetes Services, the code refactoring workload could be heavy. +- A service is created outside the cluster. Data in the cluster needs to be sent to the service through a fixed domain name. + +Solution +-------- + +There are several CoreDNS-based solutions for custom domain name resolution: + +- :ref:`Configuring the Stub Domain for CoreDNS `: You can add it on the console, which is easy to operate. +- :ref:`Using the CoreDNS Hosts plug-in to configure resolution for any domain name `: You can add any record set, which is similar to adding a record set in the local **/etc/hosts** file. +- :ref:`Using the CoreDNS Rewrite plug-in to point a domain name to a service in the cluster `: A nickname is assigned to the Kubernetes Service. You do not need to know the IP address of the resolution record in advance. +- :ref:`Using the CoreDNS Forward plug-in to set the self-built DNS as the upstream DNS `: The self-built DNS can manage a large number of resolution records. You do not need to modify the CoreDNS configuration when adding or deleting records. + +Precautions +----------- + +Improper modification on CoreDNS configuration may cause domain name resolution failures in the cluster. Perform tests before and after the modification. + +.. _cce_10_0361__section5202157467: + +Configuring the Stub Domain for CoreDNS +--------------------------------------- + +Cluster administrators can modify the ConfigMap for the CoreDNS Corefile to change how service discovery works. + +Assume that a cluster administrator has a Consul DNS server located at 10.150.0.1 and all Consul domain names have the suffix **.consul.local**. + +#. Log in to the CCE console and access the cluster console. + +#. In the navigation pane, choose **Add-ons**. On the displayed page, click **Edit** under CoreDNS. + +#. Add a stub domain in the **Parameters** area. + + Modify the **stub_domains** parameter in the format of a key-value pair. The key is a DNS suffix domain name, and the value is a DNS IP address or a group of DNS IP addresses. + + .. code-block:: + + { + "stub_domains": { + "consul.local": [ + "10.150.0.1" + ] + }, + "upstream_nameservers": [] + } + +#. Click **OK**. + +You can also modify the ConfigMap as follows: + +.. code-block:: + + $ kubectl edit configmap coredns -n kube-system + apiVersion: v1 + data: + Corefile: |- + .:5353 { + bind {$POD_IP} + cache 30 + errors + health {$POD_IP}:8080 + kubernetes cluster.local in-addr.arpa ip6.arpa { + pods insecure + fallthrough in-addr.arpa ip6.arpa + } + loadbalance round_robin + prometheus {$POD_IP}:9153 + forward . /etc/resolv.conf { + policy random + } + reload + } + + consul.local:5353 { + bind {$POD_IP} + errors + cache 30 + forward . 10.150.0.1 + } + kind: ConfigMap + metadata: + creationTimestamp: "2022-05-04T04:42:24Z" + labels: + app: coredns + k8s-app: coredns + kubernetes.io/cluster-service: "true" + kubernetes.io/name: CoreDNS + release: cceaddon-coredns + name: coredns + namespace: kube-system + resourceVersion: "8663493" + uid: bba87142-9f8d-4056-b8a6-94c3887e9e1d + +.. _cce_10_0361__section106211954135311: + +Modifying the CoreDNS Hosts Configuration File +---------------------------------------------- + +#. Use kubectl to connect to the cluster. + +#. Modify the CoreDNS configuration file and add the custom domain name to the hosts file. + + Point **www.example.com** to **192.168.1.1**. When CoreDNS resolves **www.example.com**, **192.168.1.1** is returned. + + .. important:: + + The fallthrough field must be configured. **fallthrough** indicates that when the domain name to be resolved cannot be found in the hosts file, the resolution task is transferred to the next CoreDNS plug-in. If **fallthrough** is not specified, the task ends and the domain name resolution stops. As a result, the domain name resolution in the cluster fails. + + For details about how to configure the hosts file, visit https://coredns.io/plugins/hosts/. + + .. code-block:: + + $ kubectl edit configmap coredns -n kube-system + apiVersion: v1 + data: + Corefile: |- + .:5353 { + bind {$POD_IP} + cache 30 + errors + health {$POD_IP}:8080 + kubernetes cluster.local in-addr.arpa ip6.arpa { + pods insecure + fallthrough in-addr.arpa ip6.arpa + } + hosts { + 192.168.1.1 www.example.com + fallthrough + } + loadbalance round_robin + prometheus {$POD_IP}:9153 + forward . /etc/resolv.conf + reload + } + kind: ConfigMap + metadata: + creationTimestamp: "2021-08-23T13:27:28Z" + labels: + app: coredns + k8s-app: coredns + kubernetes.io/cluster-service: "true" + kubernetes.io/name: CoreDNS + release: cceaddon-coredns + name: coredns + namespace: kube-system + resourceVersion: "460" + selfLink: /api/v1/namespaces/kube-system/configmaps/coredns + uid: be64aaad-1629-441f-8a40-a3efc0db9fa9 + + After modifying the hosts file in CoreDNS, you do not need to configure the hosts file in each pod. + +.. _cce_10_0361__section2213823544: + +Adding the CoreDNS Rewrite Configuration to Point the Domain Name to Services in the Cluster +-------------------------------------------------------------------------------------------- + +Use the Rewrite plug-in of CoreDNS to resolve a specified domain name to the domain name of a Service. + +#. Use kubectl to connect to the cluster. + +#. Modify the CoreDNS configuration file to point **example.com** to the **example** service in the **default** namespace. + + .. code-block:: + + $ kubectl edit configmap coredns -n kube-system + apiVersion: v1 + data: + Corefile: |- + .:5353 { + bind {$POD_IP} + cache 30 + errors + health {$POD_IP}:8080 + kubernetes cluster.local in-addr.arpa ip6.arpa { + pods insecure + fallthrough in-addr.arpa ip6.arpa + } + rewrite name example.com example.default.svc.cluster.local + loadbalance round_robin + prometheus {$POD_IP}:9153 + forward . /etc/resolv.conf + reload + } + kind: ConfigMap + metadata: + creationTimestamp: "2021-08-23T13:27:28Z" + labels: + app: coredns + k8s-app: coredns + kubernetes.io/cluster-service: "true" + kubernetes.io/name: CoreDNS + release: cceaddon-coredns + name: coredns + namespace: kube-system + resourceVersion: "460" + selfLink: /api/v1/namespaces/kube-system/configmaps/coredns + uid: be64aaad-1629-441f-8a40-a3efc0db9fa9 + +.. _cce_10_0361__section677819913541: + +Using CoreDNS to Cascade Self-Built DNS +--------------------------------------- + +#. Use kubectl to connect to the cluster. + +#. Modify the CoreDNS configuration file and change **/etc/resolv.conf** following **forward** to the IP address of the external DNS server. + + .. code-block:: + + $ kubectl edit configmap coredns -n kube-system + apiVersion: v1 + data: + Corefile: |- + .:5353 { + bind {$POD_IP} + cache 30 + errors + health {$POD_IP}:8080 + kubernetes cluster.local in-addr.arpa ip6.arpa { + pods insecure + fallthrough in-addr.arpa ip6.arpa + } + loadbalance round_robin + prometheus {$POD_IP}:9153 + forward . 192.168.1.1 + reload + } + kind: ConfigMap + metadata: + creationTimestamp: "2021-08-23T13:27:28Z" + labels: + app: coredns + k8s-app: coredns + kubernetes.io/cluster-service: "true" + kubernetes.io/name: CoreDNS + release: cceaddon-coredns + name: coredns + namespace: kube-system + resourceVersion: "460" + selfLink: /api/v1/namespaces/kube-system/configmaps/coredns + uid: be64aaad-1629-441f-8a40-a3efc0db9fa9 diff --git a/umn/source/networking/host_network.rst b/umn/source/networking/host_network.rst new file mode 100644 index 0000000..5f083d6 --- /dev/null +++ b/umn/source/networking/host_network.rst @@ -0,0 +1,101 @@ +:original_name: cce_10_0402.html + +.. _cce_10_0402: + +Host Network +============ + +Scenario +-------- + +Kubernetes allows pods to directly use the host/node network. + +Configuration +------------- + +Add **hostNetwork: true** to the pod definition. + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + spec: + replicas: 1 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + hostNetwork: true + containers: + - image: nginx:alpine + name: nginx + imagePullSecrets: + - name: default-secret + +The configuration succeeds if the pod IP is the same as the node IP. + +.. code-block:: + + $ kubectl get pod -owide + NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES + nginx-6fdf99c8b-6wwft 1/1 Running 0 3m41s 10.1.0.55 10.1.0.55 + +Precautions +----------- + +If a pod uses the host network, it occupies a host port. The pod IP is the host IP. To use the host network, you must confirm pods do not conflict with each other in terms of the host ports they occupy. Do not use the host network unless you know exactly which host port is used by which pod. + +When using the host network, you access the node to access a pod on it. Therefore, you need to **allow access from the security group port of the node**. Otherwise, the access fails. + +In addition, using the host network requires you to reserve host ports for the pods. When using a Deployment to deploy pods of the hostNetwork type, ensure that **the number of pods does not exceed the number of nodes**. Otherwise, multiple pods will be scheduled onto the node, and they will fail to start due to port conflicts. For example, in the preceding example nginx YAML, if two pods (setting **replicas** to **2**) are deployed in a cluster with only one node, one pod cannot be created. The pod logs will show that the Nginx cannot be started because the port is occupied. + +.. caution:: + + Do not schedule multiple pods that use the host network on the same node. Otherwise, when a ClusterIP Service is created to access a pod, the cluster IP address cannot be accessed. + +.. code-block:: + + $ kubectl get deploy + NAME READY UP-TO-DATE AVAILABLE AGE + nginx 1/2 2 1 67m + $ kubectl get pod + NAME READY STATUS RESTARTS AGE + nginx-6fdf99c8b-6wwft 1/1 Running 0 67m + nginx-6fdf99c8b-rglm7 0/1 CrashLoopBackOff 13 44m + $ kubectl logs nginx-6fdf99c8b-rglm7 + /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration + /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/ + /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh + 10-listen-on-ipv6-by-default.sh: info: Getting the checksum of /etc/nginx/conf.d/default.conf + 10-listen-on-ipv6-by-default.sh: info: Enabled listen on IPv6 in /etc/nginx/conf.d/default.conf + /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh + /docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh + /docker-entrypoint.sh: Configuration complete; ready for start up + 2022/05/11 07:18:11 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address in use) + nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to [::]:80 failed (98: Address in use) + nginx: [emerg] bind() to [::]:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address in use) + nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to [::]:80 failed (98: Address in use) + nginx: [emerg] bind() to [::]:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address in use) + nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to [::]:80 failed (98: Address in use) + nginx: [emerg] bind() to [::]:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address in use) + nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to [::]:80 failed (98: Address in use) + nginx: [emerg] bind() to [::]:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to 0.0.0.0:80 failed (98: Address in use) + nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: bind() to [::]:80 failed (98: Address in use) + nginx: [emerg] bind() to [::]:80 failed (98: Address in use) + 2022/05/11 07:18:11 [emerg] 1#1: still could not bind() + nginx: [emerg] still could not bind() diff --git a/umn/source/networking/index.rst b/umn/source/networking/index.rst index a2f5a2b..b8697dc 100644 --- a/umn/source/networking/index.rst +++ b/umn/source/networking/index.rst @@ -1,16 +1,19 @@ -:original_name: cce_01_0020.html +:original_name: cce_10_0020.html -.. _cce_01_0020: +.. _cce_10_0020: Networking ========== -- :ref:`Overview ` -- :ref:`Container Network Models ` -- :ref:`Services ` -- :ref:`Ingress ` -- :ref:`Network Policies ` -- :ref:`SecurityGroups ` +- :ref:`Overview ` +- :ref:`Container Network Models ` +- :ref:`Services ` +- :ref:`Ingresses ` +- :ref:`DNS ` +- :ref:`Configuring Intra-VPC Access ` +- :ref:`Accessing Public Networks from a Container ` +- :ref:`Network Policies ` +- :ref:`Host Network ` .. toctree:: :maxdepth: 1 @@ -19,6 +22,9 @@ Networking overview container_network_models/index services/index - ingress/index + ingresses/index + dns/index + configuring_intra-vpc_access + accessing_public_networks_from_a_container network_policies - securitygroups + host_network diff --git a/umn/source/networking/ingress/index.rst b/umn/source/networking/ingress/index.rst deleted file mode 100644 index 1927adb..0000000 --- a/umn/source/networking/ingress/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -:original_name: cce_01_0248.html - -.. _cce_01_0248: - -Ingress -======= - -- :ref:`Overview ` -- :ref:`Using ELB Ingresses on the Console ` -- :ref:`Using kubectl to Create an ELB Ingress ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - overview - using_elb_ingresses_on_the_console - using_kubectl_to_create_an_elb_ingress diff --git a/umn/source/networking/ingress/using_elb_ingresses_on_the_console.rst b/umn/source/networking/ingress/using_elb_ingresses_on_the_console.rst deleted file mode 100644 index 8ca88a5..0000000 --- a/umn/source/networking/ingress/using_elb_ingresses_on_the_console.rst +++ /dev/null @@ -1,157 +0,0 @@ -:original_name: cce_01_0251.html - -.. _cce_01_0251: - -Using ELB Ingresses on the Console -================================== - -Prerequisites -------------- - -- An ingress provides network access for backend workloads. Ensure that a workload is available in a cluster. If no workload is available, deploy a workload by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `. -- A NodePort Service has been configured for the workload. For details about how to configure the Service, see :ref:`NodePort `. - -Precautions ------------ - -- It is recommended that other resources not use the load balancer automatically created by an ingress. Otherwise, the load balancer will be occupied when the ingress is deleted, resulting in residual resources. -- After an ingress is created, upgrade and maintain the configuration of the selected load balancers on the CCE console. Do not modify the configuration on the ELB console. Otherwise, the ingress service may be abnormal. -- The URL registered in an ingress forwarding policy must be the same as the URL exposed by the backend Service. Otherwise, a 404 error will be returned. - -Adding an ELB Ingress ---------------------- - -This section uses an Nginx workload as an example to describe how to add an ELB ingress. - -#. Log in to the CCE console. - -#. In the navigation pane, choose **Resource Management** > **Network**. On the **Ingresses** tab page, select the corresponding cluster and namespace. - -#. Click **Create Ingress** to access the ingress configuration page. - - Set the ingress parameters as required. The key parameters are as follows: - - - **Access Type**: Use a load balancer to access Services. Requests can be forwarded only to NodePort Services. - - - **Ingress Name**: Specify a name of an ingress, for example, **ingress-demo**. - - - **Cluster Name**: Select the cluster to which the ingress is to be added. - - - **Namespace**: Select the namespace to which the ingress is to be added. - - - **ELB Configuration**: Ingress uses the load balancer of the ELB service to provide layer-7 network access. You can select an existing load balancer or have the system automatically create a new one. To manually create a load balancer, click **Create Load Balancer** and then click the refresh button. - - .. important:: - - - It is recommended that other resources not use the load balancer automatically created by an ingress. Otherwise, the load balancer will be occupied when the ingress is deleted, resulting in residual resources. - - Dedicated load balancers are supported only when the cluster version is 1.17 or later. - - To interconnect with an existing dedicated load balancer, ensure that HTTP is supported and the network type supports private networks. - - **Elastic Load Balancer**: The selected or created load balancer must be in the same VPC as the current cluster, and it must match the load balancer type (private or public network). - - You can create **public network** or **private network** load balancers. The default value is **Public network**. - - - **Public network**: After you attach an EIP to a load balancer, the load balancer can distribute requests from the Internet to backend servers. - - - **Enterprise Project**: Select an enterprise project in which the load balancer is created. - - **Change Configuration**: When selecting **Public network** > **Automatically created**, you can click **Change Configuration** to modify the name, specifications, billing mode, and bandwidth of the ELB instance to be created. - - - **Private network**: After you attach a private IP address to a load balancer, the load balancer can distribute requests from the clients in the same VPC to backends. - - - **Enterprise Project**: Select an enterprise project in which the load balancer is created. - - - **Listener Configuration**: Ingress configures a listener for the load balancer, which listens to requests from the load balancer and distributes traffic. After the configuration is complete, a listener is created on the load balancer. The default listener name is *k8s___*, for example, *k8s_HTTP_80*. - - - **Front-End Protocol**: **HTTP** and **HTTPS** are available. - - - **External Port**: Port number that is open to the ELB service address. The port number can be specified randomly. - - - **Server Certificate**: When an HTTPS listener is created for a load balancer, you need to bind a certificate to the load balancer to support encrypted authentication for HTTPS data transmission. For details on how to create a secret, see :ref:`Creating a Secret `. - - .. note:: - - If there is already an HTTPS ingress for the chosen port on the load balancer, the certificate of the new HTTPS ingress must be the same as the certificate of the existing ingress. This means that a listener has only one certificate. If two certificates, each with a different ingress, are added to the same listener of the same load balancer, only the certificate added earliest takes effect on the load balancer. - - - **SNI**: Click |image1| to enable the Server Name Indication (SNI) function. SNI is an extended protocol of TLS. It allows multiple TLS-based access domain names to be provided for external systems using the same IP address and port number. Different domain names can use different security certificates. After SNI is enabled, the client is allowed to submit the requested domain name when initiating a TLS handshake request. After receiving the TLS request, the load balancer searches for the certificate based on the domain name in the request. If the certificate corresponding to the domain name is found, the load balancer returns the certificate for authorization. Otherwise, the default certificate (server certificate) is returned for authorization. - - .. note:: - - - The **SNI** option is available only when **HTTPS** is selected. - - - This function is supported only for clusters of v1.15.11 and later. - - Specify the domain name for the SNI certificate. Only one domain name can be specified for each certificate. Wildcard-domain certificates are supported. - - - **Security Policy**: combinations of different TLS versions and supported cipher suites available to HTTPS listeners. - - For details about security policies, see ELB User Guide. - - .. note:: - - - **Security Policy** is available only when **HTTPS** is selected. - - This function is supported only for clusters of v1.17.9 and later. - - - **Forwarding Policies**: When the access address of a request matches the forwarding policy (a forwarding policy consists of a domain name and URL, for example, 10.117.117.117:80/helloworld), the request is forwarded to the corresponding target Service for processing. Click **Add Forwarding Policies** to add multiple forwarding policies. - - - **Domain Name**: actual domain name. Ensure that the domain name has been registered and archived. Once a domain name rule is configured, you must use the domain name for access. - - Rule Matching - - - **Prefix match**: If the URL is set to **/healthz**, the URL that meets the prefix can be accessed. For example, **/healthz/v1** and **/healthz/v2**. - - **Exact match**: The URL can be accessed only when it is fully matched. For example, if the URL is set to **/healthz**, only /healthz can be accessed. - - **Regular expression**: The URL is matched based on the regular expression. For example, if the regular expression is **/[A-Za-z0-9_.-]+/test**, all URLs that comply with this rule can be accessed, for example, **/abcA9/test** and **/v1-Ab/test**. Two regular expression standards are supported: POSIX and Perl. - - - **URL**: access path to be registered, for example, **/healthz**. - - **Target Service**: Select an existing Service or create a Service. Services that do not meet search criteria are automatically filtered out. - - **Service Access Port**: Select the access port of the target Service. - - **ELB Settings**: If multiple routes use the same Service, they are using the same Service load balancing configuration. - - - **Algorithm Type**: Three algorithms are available: weighted round robin, weighted least connections algorithm, or source IP hash. For details about the allocation policies, see :ref:`LoadBalancer `. - - - **Sticky Session**: This function is disabled by default. After this function is enabled, you need to select a sticky session type and set the sticky session duration. - - **ELB cookie**: The load balancer generates a cookie after receiving a request from the client. All subsequent requests with the cookie are routed to the same backend server for processing. - - **Application cookie**: The application deployed on the backend server generates a cookie after receiving the first request from the client. All subsequent requests that contain the cookie are routed to this backend server. This sticky session type is supported by shared load balancers. - - - **Health Check**: This function is disabled by default. To enable this function, set parameters as prompted. For details about the parameters, see `Configuring a Health Check `__. - - - **Operation**: Click **Delete** to delete the configuration. - -#. After the configuration is complete, click **Create**. After the ingress is created, it is displayed in the ingress list. - - On the ELB console, you can view the ELB automatically created through CCE. The default name is **cce-lb-ingress.UID**. Click the ELB name to access its details page. On the **Listeners** tab page, view the route settings of the ingress, including the URL, listener port, and backend server group port. - - .. important:: - - After the ingress is created, upgrade and maintain the selected load balancer on the CCE console. Do not maintain the load balancer on the ELB console. Otherwise, the ingress service may be abnormal. - -#. Access the /healthz interface of the workload, for example, workload **defaultbackend**. - - a. Obtain the access address of the /healthz interface of the workload. The access address consists of the load balancer IP address, external port, and mapping URL, for example, 10.**.**.**:80/healthz. - - b. Enter the URL of the /healthz interface, for example, http://10.**.**.**:80/healthz, in the address box of the browser to access the workload, as shown in :ref:`Figure 1 `. - - .. _cce_01_0251__fig17115192714367: - - .. figure:: /_static/images/en-us_image_0000001192723194.png - :alt: **Figure 1** Accessing the /healthz interface of defaultbackend - - **Figure 1** Accessing the /healthz interface of defaultbackend - -Updating an Ingress -------------------- - -After adding an ingress, you can update its port, domain name, and route configuration. The procedure is as follows: - -.. note:: - - You can modify the load balancer settings, including algorithm, sticky session, and health check configurations, after you select a Service in **Forwarding Policies** on the CCE console. Do not modify these configurations on the ELB console. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Network**. On the **Ingresses** tab page, filter ingresses by cluster and namespace, and click **Update** for the ingress to be updated. - -#. On the **Update Ingress** page, modify the required parameters. - - The parameters are the same as those set during creation. - -#. Click **Submit**. The ingress will be updated for the workload. - -.. |image1| image:: /_static/images/en-us_image_0000001238163131.png diff --git a/umn/source/networking/ingresses/index.rst b/umn/source/networking/ingresses/index.rst new file mode 100644 index 0000000..36f8d50 --- /dev/null +++ b/umn/source/networking/ingresses/index.rst @@ -0,0 +1,18 @@ +:original_name: cce_10_0248.html + +.. _cce_10_0248: + +Ingresses +========= + +- :ref:`Ingress Overview ` +- :ref:`Using ELB Ingresses on the Console ` +- :ref:`Using kubectl to Create an ELB Ingress ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + ingress_overview + using_elb_ingresses_on_the_console + using_kubectl_to_create_an_elb_ingress diff --git a/umn/source/networking/ingress/overview.rst b/umn/source/networking/ingresses/ingress_overview.rst similarity index 86% rename from umn/source/networking/ingress/overview.rst rename to umn/source/networking/ingresses/ingress_overview.rst index 8bdcdd1..11cc203 100644 --- a/umn/source/networking/ingress/overview.rst +++ b/umn/source/networking/ingresses/ingress_overview.rst @@ -1,20 +1,20 @@ -:original_name: cce_01_0094.html +:original_name: cce_10_0094.html -.. _cce_01_0094: +.. _cce_10_0094: -Overview -======== +Ingress Overview +================ Why We Need Ingresses --------------------- A Service is generally used to forward access requests based on TCP and UDP and provide layer-4 load balancing for clusters. However, in actual scenarios, if there is a large number of HTTP/HTTPS access requests on the application layer, the Service cannot meet the forwarding requirements. Therefore, the Kubernetes cluster provides an HTTP-based access mode, that is, ingress. -An ingress is an independent resource in the Kubernetes cluster and defines rules for forwarding external access traffic. As shown in :ref:`Figure 1 `, you can customize forwarding rules based on domain names and URLs to implement fine-grained distribution of access traffic. +An ingress is an independent resource in the Kubernetes cluster and defines rules for forwarding external access traffic. As shown in :ref:`Figure 1 `, you can customize forwarding rules based on domain names and URLs to implement fine-grained distribution of access traffic. -.. _cce_01_0094__fig18155819416: +.. _cce_10_0094__fig18155819416: -.. figure:: /_static/images/en-us_image_0000001238003081.png +.. figure:: /_static/images/en-us_image_0000001243981115.png :alt: **Figure 1** Ingress diagram **Figure 1** Ingress diagram @@ -29,15 +29,15 @@ Working Principle of ELB Ingress Controller ELB Ingress Controller developed by CCE implements layer-7 network access for the internet and intranet (in the same VPC) based on ELB and distributes access traffic to the corresponding Services using different URLs. -ELB Ingress Controller is deployed on the master node and bound to the load balancer in the VPC where the cluster resides. Different domain names, ports, and forwarding policies can be configured for the same load balancer (with the same IP address). :ref:`Figure 2 ` shows the working principle of ELB Ingress Controller. +ELB Ingress Controller is deployed on the master node and bound to the load balancer in the VPC where the cluster resides. Different domain names, ports, and forwarding policies can be configured for the same load balancer (with the same IP address). :ref:`Figure 2 ` shows the working principle of ELB Ingress Controller. #. A user creates an ingress object and configures a traffic access rule in the ingress, including the load balancer, URL, SSL, and backend service port. #. When Ingress Controller detects that the ingress object changes, it reconfigures the listener and backend server route on the ELB side according to the traffic access rule. #. When a user accesses a workload, the traffic is forwarded to the corresponding backend service port based on the forwarding policy configured on ELB, and then forwarded to each associated workload through the Service. -.. _cce_01_0094__fig122542486129: +.. _cce_10_0094__fig122542486129: -.. figure:: /_static/images/en-us_image_0000001192723190.png +.. figure:: /_static/images/en-us_image_0000001199501200.png :alt: **Figure 2** Working principle of ELB Ingress Controller **Figure 2** Working principle of ELB Ingress Controller diff --git a/umn/source/networking/ingresses/using_elb_ingresses_on_the_console.rst b/umn/source/networking/ingresses/using_elb_ingresses_on_the_console.rst new file mode 100644 index 0000000..d05c1b6 --- /dev/null +++ b/umn/source/networking/ingresses/using_elb_ingresses_on_the_console.rst @@ -0,0 +1,133 @@ +:original_name: cce_10_0251.html + +.. _cce_10_0251: + +Using ELB Ingresses on the Console +================================== + +Prerequisites +------------- + +- An ingress provides network access for backend workloads. Ensure that a workload is available in a cluster. If no workload is available, deploy a workload by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `. +- A NodePort Service has been configured for the workload. For details about how to configure the Service, see :ref:`NodePort `. +- Dedicated load balancers must be the application type (HTTP/HTTPS) supporting private networks (with a private IP). +- In ELB passthrough networking (CCE Turbo cluster + dedicated load balancer), ELB Ingress supports ClusterIP Services. In other scenarios, ELB Ingress supports NodePort Services. + +Precautions +----------- + +- It is recommended that other resources not use the load balancer automatically created by an ingress. Otherwise, the load balancer will be occupied when the ingress is deleted, resulting in residual resources. +- After an ingress is created, upgrade and maintain the configuration of the selected load balancers on the CCE console. Do not modify the configuration on the ELB console. Otherwise, the ingress service may be abnormal. +- The URL registered in an ingress forwarding policy must be the same as the URL exposed by the backend Service. Otherwise, a 404 error will be returned. +- In a cluster using the IPVS proxy mode, if the ingress and Service use the same ELB load balancer, the ingress cannot be accessed from the nodes and containers in the cluster because kube-proxy mounts the LoadBalancer Service address to the ipvs-0 bridge. This bridge intercepts the traffic of the load balancer connected to the ingress. You are advised to use different ELB load balancers for the ingress and Service. + +Adding an ELB Ingress +--------------------- + +This section uses an Nginx workload as an example to describe how to add an ELB ingress. + +#. Log in to the CCE console and access the cluster console. + +#. Choose **Networking** in the navigation pane, click the **Ingresses** tab, and click **Create Service** in the upper right corner. + +#. Set ingress parameters. + + - **Name**: Specify a name of an ingress, for example, **ingress-demo**. + + - **Load Balancer** + + Select the load balancer to interconnect. Only load balancers in the same VPC as the cluster are supported. If no load balancer is available, click **Create Load Balancer** to create one on the ELB console. + + Dedicated load balancers must support HTTP and the network type must support private networks. + + - **Listener Configuration**: Ingress configures a listener for the load balancer, which listens to requests from the load balancer and distributes traffic. After the configuration is complete, a listener is created on the load balancer. The default listener name is *k8s___*, for example, *k8s_HTTP_80*. + + - **Front-End Protocol**: **HTTP** and **HTTPS** are available. + + - **External Port**: Port number that is open to the ELB service address. The port number can be specified randomly. + + - **Server Certificate**: When an HTTPS listener is created for a load balancer, you need to bind a certificate to the load balancer to support encrypted authentication for HTTPS data transmission. + + .. note:: + + If there is already an HTTPS ingress for the chosen port on the load balancer, the certificate of the new HTTPS ingress must be the same as the certificate of the existing ingress. This means that a listener has only one certificate. If two certificates, each with a different ingress, are added to the same listener of the same load balancer, only the certificate added earliest takes effect on the load balancer. + + - **SNI**: Server Name Indication (SNI) is an extended protocol of TLS. It allows multiple TLS-based access domain names to be provided for external systems using the same IP address and port. Different domain names can use different security certificates. After SNI is enabled, the client is allowed to submit the requested domain name when initiating a TLS handshake request. After receiving the TLS request, the load balancer searches for the certificate based on the domain name in the request. If the certificate corresponding to the domain name is found, the load balancer returns the certificate for authorization. Otherwise, the default certificate (server certificate) is returned for authorization. + + .. note:: + + - The **SNI** option is available only when **HTTPS** is selected. + + - This function is supported only for clusters of v1.15.11 and later. + - Specify the domain name for the SNI certificate. Only one domain name can be specified for each certificate. Wildcard-domain certificates are supported. + + - **Security Policy**: combinations of different TLS versions and supported cipher suites available to HTTPS listeners. + + For details about security policies, see ELB User Guide. + + .. note:: + + - **Security Policy** is available only when **HTTPS** is selected. + - This function is supported only for clusters of v1.17.9 and later. + + - **Forwarding Policies**: When the access address of a request matches the forwarding policy (a forwarding policy consists of a domain name and URL, for example, 10.117.117.117:80/helloworld), the request is forwarded to the corresponding target Service for processing. Click **Add Forwarding Policies** to add multiple forwarding policies. + + - **Domain Name**: actual domain name. Ensure that the domain name has been registered and archived. Once a domain name rule is configured, you must use the domain name for access. + + - **URL Matching Rule**: + + - **Prefix match**: If the URL is set to **/healthz**, the URL that meets the prefix can be accessed. For example, **/healthz/v1** and **/healthz/v2**. + - **Exact match**: The URL can be accessed only when it is fully matched. For example, if the URL is set to **/healthz**, only /healthz can be accessed. + - **Regular expression**: The URL is matched based on the regular expression. For example, if the regular expression is **/[A-Za-z0-9_.-]+/test**, all URLs that comply with this rule can be accessed, for example, **/abcA9/test** and **/v1-Ab/test**. Two regular expression standards are supported: POSIX and Perl. + + - **URL**: access path to be registered, for example, **/healthz**. + + .. note:: + + The URL added here must exist in the backend application. Otherwise, the forwarding fails. + + For example, the default access URL of the Nginx application is **/usr/share/nginx/html**. When adding **/test** to the ingress forwarding policy, ensure that your Nginx application contains the same URL, that is, **/usr/share/nginx/html/test**, otherwise, 404 is returned. + + - **Destination Service**: Select an existing Service or create a Service. Services that do not meet search criteria are automatically filtered out. + + - .. _cce_10_0251__li118614181492: + + **Destination Service Port**: Select the access port of the destination Service. + + - **Set ELB**: + + - **Distribution Policy**: Three algorithms are available: weighted round robin, weighted least connections algorithm, or source IP hash. + + .. note:: + + - **Weighted round robin**: Requests are forwarded to different servers based on their weights, which indicate server processing performance. Backend servers with higher weights receive proportionately more requests, whereas equal-weighted servers receive the same number of requests. This algorithm is often used for short connections, such as HTTP services. + - **Weighted least connections**: In addition to the weight assigned to each server, the number of connections processed by each backend server is also considered. Requests are forwarded to the server with the lowest connections-to-weight ratio. Building on **least connections**, the **weighted least connections** algorithm assigns a weight to each server based on their processing capability. This algorithm is often used for persistent connections, such as database connections. + - **Source IP hash**: The source IP address of each request is calculated using the hash algorithm to obtain a unique hash key, and all backend servers are numbered. The generated key allocates the client to a particular server. This enables requests from different clients to be distributed in load balancing mode and ensures that requests from the same client are forwarded to the same server. This algorithm applies to TCP connections without cookies. + + - **Type**: This function is disabled by default. You can select **Load balancer cookie**. + - **Health Check**: This function is disabled by default. The health check is for the load balancer. When TCP is selected during the :ref:`port settings `, you can choose either TCP or HTTP. Currently, UDP is not supported. By default, the service port (Node Port and container port of the Service) is used for health check. You can also specify another port for health check. After the port is specified, a service port named **cce-healthz** will be added for the Service. + + - **Operation**: Click **Delete** to delete the configuration. + + - **Annotation**: Ingresses provide some advanced CCE functions, which are implemented by annotations. When you use kubectl to create a container, annotations will be used. For details, see :ref:`Creating an Ingress - Automatically Creating a Load Balancer ` and :ref:`Creating an Ingress - Interconnecting with an Existing Load Balancer `. + +#. After the configuration is complete, click **OK**. After the ingress is created, it is displayed in the ingress list. + + On the ELB console, you can view the ELB automatically created through CCE. The default name is **cce-lb-ingress.UID**. Click the ELB name to access its details page. On the **Listeners** tab page, view the route settings of the ingress, including the URL, listener port, and backend server group port. + + .. important:: + + After the ingress is created, upgrade and maintain the selected load balancer on the CCE console. Do not maintain the load balancer on the ELB console. Otherwise, the ingress service may be abnormal. + +#. Access the /healthz interface of the workload, for example, workload **defaultbackend**. + + a. Obtain the access address of the **/healthz** interface of the workload. The access address consists of the load balancer IP address, external port, and mapping URL, for example, 10.**.**.**:80/healthz. + + b. Enter the URL of the /healthz interface, for example, http://10.**.**.**:80/healthz, in the address box of the browser to access the workload, as shown in :ref:`Figure 1 `. + + .. _cce_10_0251__fig17115192714367: + + .. figure:: /_static/images/en-us_image_0000001199181230.png + :alt: **Figure 1** Accessing the /healthz interface of defaultbackend + + **Figure 1** Accessing the /healthz interface of defaultbackend diff --git a/umn/source/networking/ingress/using_kubectl_to_create_an_elb_ingress.rst b/umn/source/networking/ingresses/using_kubectl_to_create_an_elb_ingress.rst similarity index 64% rename from umn/source/networking/ingress/using_kubectl_to_create_an_elb_ingress.rst rename to umn/source/networking/ingresses/using_kubectl_to_create_an_elb_ingress.rst index 9de5872..fe30dc0 100644 --- a/umn/source/networking/ingress/using_kubectl_to_create_an_elb_ingress.rst +++ b/umn/source/networking/ingresses/using_kubectl_to_create_an_elb_ingress.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0252.html +:original_name: cce_10_0252.html -.. _cce_01_0252: +.. _cce_10_0252: Using kubectl to Create an ELB Ingress ====================================== @@ -8,25 +8,45 @@ Using kubectl to Create an ELB Ingress Scenario -------- -This section uses an :ref:`Nginx workload ` as an example to describe how to create an ELB ingress using kubectl. +This section uses an :ref:`Nginx workload ` as an example to describe how to create an ELB ingress using kubectl. -- If no load balancer is available in the same VPC, CCE can automatically create a load balancer when creating an ingress. For details, see :ref:`Creating an Ingress - Automatically Creating a Load Balancer `. -- If a load balancer is available in the same VPC, perform the operation by referring to :ref:`Creating an Ingress - Interconnecting with an Existing Load Balancer `. +- If no load balancer is available in the same VPC, CCE can automatically create a load balancer when creating an ingress. For details, see :ref:`Creating an Ingress - Automatically Creating a Load Balancer `. +- If a load balancer is available in the same VPC, perform the operation by referring to :ref:`Creating an Ingress - Interconnecting with an Existing Load Balancer `. Prerequisites ------------- -- An ingress provides network access for backend workloads. Ensure that a workload is available in a cluster. If no workload is available, deploy a sample Nginx workload by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `. -- A NodePort Service has been configured for the workload. For details about how to configure the Service, see :ref:`NodePort `. +- An ingress provides network access for backend workloads. Ensure that a workload is available in a cluster. If no workload is available, deploy a sample Nginx workload by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `. +- A NodePort Service has been configured for the workload. For details about how to configure the Service, see :ref:`NodePort `. +- Dedicated load balancers must be the application type (HTTP/HTTPS) supporting private networks (with a private IP). -.. _cce_01_0252__section3675115714214: +.. _cce_10_0252__section084115985013: + +Ingress Description of networking.k8s.io/v1 +------------------------------------------- + +In CCE clusters of v1.23 or later, the ingress version is switched to networking.k8s.io/v1. + +Compared with v1beta1, v1 has the following differences in parameters: + +- The ingress type is changed from **kubernetes.io/ingress.class** in **annotations** to **spec.ingressClassName**. +- The format of **backend** is changed. +- The **pathType** parameter must be specified for each path. The options are as follows: + + - **ImplementationSpecific**: The matching method depends on Ingress Controller. The matching method defined by **ingress.beta.kubernetes.io/url-match-mode** is used in CCE, which is the same as v1beta1. + - **Exact**: exact matching of the URL, which is case-sensitive. + - **Prefix**: matching based on the URL prefix separated by a slash (/). The match is case-sensitive, and elements in the path are matched one by one. A path element refers to a list of labels in the path separated by a slash (/). + +|image1| + +.. _cce_10_0252__section3675115714214: Creating an Ingress - Automatically Creating a Load Balancer ------------------------------------------------------------ The following describes how to run the kubectl command to automatically create a load balancer when creating an ingress. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create a YAML file named **ingress-test.yaml**. The file name can be customized. @@ -34,12 +54,45 @@ The following describes how to run the kubectl command to automatically create a .. note:: - - For clusters of v1.15 or later, the value of **apiVersion** is **networking.k8s.io/v1beta1**. - - For clusters of v1.13 or earlier, the value of **apiVersion** is **extensions/v1beta1**. + Starting from cluster v1.23, the ingress version is switched from **networking.k8s.io/v1beta1** to **networking.k8s.io/v1**. For details about the differences between v1 and v1beta1, see :ref:`Ingress Description of networking.k8s.io/v1 `. - You can create a load balancer as required. The YAML files are as follows: + **Example of a shared load balancer (public network access) for clusters of v1.23 or later:** - **Example of using a dedicated public network load balancer**: + .. code-block:: + + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: ingress-test + annotations: + kubernetes.io/elb.class: union + kubernetes.io/elb.port: '80' + kubernetes.io/elb.autocreate: + '{ + "type":"public", + "bandwidth_name":"cce-bandwidth-******", + "bandwidth_chargemode":"bandwidth", + "bandwidth_size":5, + "bandwidth_sharetype":"PER", + "eip_type":"5_bgp" + }' + spec: + rules: + - host: '' + http: + paths: + - path: '/' + backend: + service: + name: # Replace it with the name of your target Service. + port: + number: 8080 # Replace 8080 with the port number of your target Service. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce # ELB ingress is used. + + **Example of a shared load balancer (public network access) for clusters of v1.21 or earlier:** .. code-block:: @@ -49,13 +102,13 @@ The following describes how to run the kubectl command to automatically create a name: ingress-test annotations: kubernetes.io/elb.class: union - kubernetes.io/ingress.class: cce + kubernetes.io/ingress.class: cce # ELB ingress is used. kubernetes.io/elb.port: '80' kubernetes.io/elb.autocreate: '{ "type":"public", "bandwidth_name":"cce-bandwidth-******", - "bandwidth_chargemode":"traffic", + "bandwidth_chargemode":"bandwidth", "bandwidth_size":5, "bandwidth_sharetype":"PER", "eip_type":"5_bgp" @@ -72,7 +125,48 @@ The following describes how to run the kubectl command to automatically create a property: ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH - **Example of using a dedicated public network load balancer:** + **Example of a dedicated load balancer (public network access) for clusters of v1.23 or later:** + + .. code-block:: + + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: ingress-test + namespace: default + annotations: + kubernetes.io/elb.class: performance + kubernetes.io/elb.port: '80' + kubernetes.io/elb.autocreate: + '{ + "type": "public", + "bandwidth_name": "cce-bandwidth-******", + "bandwidth_chargemode": "bandwidth", + "bandwidth_size": 5, + "bandwidth_sharetype": "PER", + "eip_type": "5_bgp", + "available_zone": [ + "eu-de-01" + ], + "l7_flavor_name": "L7_flavor.elb.s1.small" + }' + spec: + rules: + - host: '' + http: + paths: + - path: '/' + backend: + service: + name: # Replace it with the name of your target Service. + port: + number: 8080 # Replace 8080 with the port number of your target Service. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce + + **Example of a dedicated load balancer (public network access) for clusters of version 1.21 or earlier:** .. code-block:: @@ -89,7 +183,7 @@ The following describes how to run the kubectl command to automatically create a '{ "type": "public", "bandwidth_name": "cce-bandwidth-******", - "bandwidth_chargemode": "traffic", + "bandwidth_chargemode": "bandwidth", "bandwidth_size": 5, "bandwidth_sharetype": "PER", "eip_type": "5_bgp", @@ -112,117 +206,115 @@ The following describes how to run the kubectl command to automatically create a .. table:: **Table 1** Key parameters - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Mandatory | Type | Description | - +===========================================+=================+=======================+==========================================================================================================================================================================================================================================+ - | kubernetes.io/elb.class | No | String | Select a proper load balancer type. | - | | | | | - | | | | The value can be: | - | | | | | - | | | | - **union**: shared load balancer | - | | | | - **performance**: dedicated load balancer.. | - | | | | | - | | | | The default value is **union**. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/ingress.class | Yes | String | **cce**: The self-developed ELBIngress is used. | - | | | | | - | | | | This parameter is mandatory when an ingress is created by calling the API. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.port | Yes | Integer | This parameter indicates the external port registered with the address of the LoadBalancer Service. | - | | | | | - | | | | Supported range: 1 to 65535 | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.subnet-id | ``-`` | String | ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | - | | | | | - | | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | - | | | | - Optional for clusters later than v1.11.7-r0. It is left blank by default. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.enterpriseID | No | String | **Kubernetes clusters of v1.15 and later versions support this field. In Kubernetes clusters earlier than v1.15, load balancers are created in the default project by default.** | - | | | | | - | | | | ID of the enterprise project in which the load balancer will be created. | - | | | | | - | | | | The value contains 1 to 100 characters. | - | | | | | - | | | | **How to obtain**: | - | | | | | - | | | | Log in to the management console and choose **Enterprise** > **Project Management** on the top menu bar. In the list displayed, click the name of the target enterprise project, and copy the ID on the enterprise project details page. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.autocreate | Yes | elb.autocreate object | Whether to automatically create a load balancer associated with an ingress. For details about the field description, see :ref:`Table 2 `. | - | | | | | - | | | | **Example** | - | | | | | - | | | | - If a public network load balancer will be automatically created, set this parameter to the following value: | - | | | | | - | | | | '{"type":"public","bandwidth_name":"cce-bandwidth-``******``","bandwidth_chargemode":"traffic","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}' | - | | | | | - | | | | - If a private network load balancer will be automatically created, set this parameter to the following value: | - | | | | | - | | | | {"type":"inner","name":"A-location-d-test"} | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | host | No | String | Domain name for accessing the Service. By default, this parameter is left blank, and the domain name needs to be fully matched. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | path | Yes | String | User-defined route path. All external access requests must match **host** and **path**. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | serviceName | Yes | String | Name of the target Service bound to the ingress. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | servicePort | Yes | Integer | Access port of the target Service. | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | ingress.beta.kubernetes.io/url-match-mode | No | String | Route matching policy. | - | | | | | - | | | | Default: **STARTS_WITH** (prefix match) | - | | | | | - | | | | Options: | - | | | | | - | | | | - **EQUAL_TO**: exact match | - | | | | - **STARTS_WITH**: prefix match | - | | | | - **REGEX**: regular expression match | - +-------------------------------------------+-----------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +===========================================+=========================================+=======================+=========================================================================================================================================================================================================================================+ + | kubernetes.io/elb.class | Yes | String | Select a proper load balancer type. | + | | | | | + | | | | The value can be: | + | | | | | + | | | | - **union**: shared load balancer | + | | | | - **performance**: dedicated load balancer.. | + | | | | | + | | | | Default: **union** | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/ingress.class | Yes | String | **cce**: The self-developed ELB ingress is used. | + | | | | | + | | (only for clusters of v1.21 or earlier) | | This parameter is mandatory when an ingress is created by calling the API. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ingressClassName | Yes | String | **cce**: The self-developed ELB ingress is used. | + | | | | | + | | (only for clusters of v1.23 or later) | | This parameter is mandatory when an ingress is created by calling the API. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.port | Yes | Integer | This parameter indicates the external port registered with the address of the LoadBalancer Service. | + | | | | | + | | | | Supported range: 1 to 65535 | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.subnet-id | ``-`` | String | ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | + | | | | | + | | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | + | | | | - Optional for clusters later than v1.11.7-r0. It is left blank by default. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.autocreate | Yes | elb.autocreate object | Whether to automatically create a load balancer associated with an ingress. For details about the field description, see :ref:`Table 2 `. | + | | | | | + | | | | **Example** | + | | | | | + | | | | - If a public network load balancer will be automatically created, set this parameter to the following value: | + | | | | | + | | | | {"type":"public","bandwidth_name":"cce-bandwidth-``******``","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"} | + | | | | | + | | | | - If a private network load balancer will be automatically created, set this parameter to the following value: | + | | | | | + | | | | {"type":"inner","name":"A-location-d-test"} | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | host | No | String | Domain name for accessing the Service. By default, this parameter is left blank, and the domain name needs to be fully matched. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | path | Yes | String | User-defined route path. All external access requests must match **host** and **path**. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | serviceName | Yes | String | Name of the target Service bound to the ingress. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | servicePort | Yes | Integer | Access port of the target Service. | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ingress.beta.kubernetes.io/url-match-mode | No | String | Route matching policy. | + | | | | | + | | | | Default: **STARTS_WITH** (prefix match) | + | | | | | + | | | | Options: | + | | | | | + | | | | - **EQUAL_TO**: exact match | + | | | | - **STARTS_WITH**: prefix match | + | | | | - **REGEX**: regular expression match | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | pathType | Yes | String | Path type. This field is supported only by clusters of v1.23 or later. | + | | | | | + | | | | - **ImplementationSpecific**: The matching method depends on Ingress Controller. The matching method defined by **ingress.beta.kubernetes.io/url-match-mode** is used in CCE. | + | | | | - **Exact**: exact matching of the URL, which is case-sensitive. | + | | | | - **Prefix**: matching based on the URL prefix separated by a slash (/). The match is case-sensitive, and elements in the path are matched one by one. A path element refers to a list of labels in the path separated by a slash (/). | + +-------------------------------------------+-----------------------------------------+-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - .. _cce_01_0252__table268711532210: + .. _cce_10_0252__table268711532210: .. table:: **Table 2** Data structure of the elb.autocreate field - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Mandatory | Type | Description | - +======================+=======================================+=================+====================================================================================================================================================================================================================================+ - | type | No | String | Network type of the load balancer. | - | | | | | - | | | | - **public**: public network load balancer | - | | | | - **inner**: private network load balancer | - | | | | | - | | | | The default value is **inner**. | - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_name | Yes for public network load balancers | String | Bandwidth name. The default value is **cce-bandwidth-*****\***. | - | | | | | - | | | | Value range: a string of 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_chargemode | Yes | String | Bandwidth billing mode. | - | | | | | - | | | | - **traffic**: billed by traffic | - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_size | Yes for public network load balancers | Integer | Bandwidth size. The value ranges from 1 Mbit/s to 2000 Mbit/s by default. The actual range varies depending on the configuration in each region. | - | | | | | - | | | | - The minimum increment for bandwidth adjustment varies depending on the bandwidth range. The details are as follows: | - | | | | | - | | | | - The minimum increment is 1 Mbit/s if the allowed bandwidth ranges from 0 Mbit/s to 300 Mbit/s (with 300 Mbit/s included). | - | | | | - The minimum increment is 50 Mbit/s if the allowed bandwidth ranges from 300 Mbit/s to 1000 Mbit/s. | - | | | | - The minimum increment is 500 Mbit/s if the allowed bandwidth is greater than 1000 Mbit/s. | - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_sharetype | Yes for public network load balancers | String | Bandwidth type. | - | | | | | - | | | | **PER**: dedicated bandwidth | - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | eip_type | Yes for public network load balancers | String | EIP type, which may vary depending on sites. For details, see the type parameter specified when `creating an EIP `__. | - | | | | | - | | | | - **5_bgp**: dynamic BGP | - | | | | - **5_gray**: dedicated load balancer | - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | name | No | String | Name of the automatically created load balancer. | - | | | | | - | | | | Value range: a string of 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | - | | | | | - | | | | Default value: **cce-lb+ingress.UID** | - +----------------------+---------------------------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +======================+=======================================+=================+===============================================================================================================================================================================================+ + | type | No | String | Network type of the load balancer. | + | | | | | + | | | | - **public**: public network load balancer | + | | | | - **inner**: private network load balancer | + | | | | | + | | | | Default: **inner** | + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_name | Yes for public network load balancers | String | Bandwidth name. The default value is **cce-bandwidth-*****\***. | + | | | | | + | | | | Value range: a string of 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_chargemode | No | String | Bandwidth mode. | + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_size | Yes for public network load balancers | Integer | Bandwidth size. The value ranges from 1 Mbit/s to 2000 Mbit/s by default. The actual range varies depending on the configuration in each region. | + | | | | | + | | | | - The minimum increment for bandwidth adjustment varies depending on the bandwidth range. The details are as follows: | + | | | | | + | | | | - The minimum increment is 1 Mbit/s if the allowed bandwidth ranges from 0 Mbit/s to 300 Mbit/s (with 300 Mbit/s included). | + | | | | - The minimum increment is 50 Mbit/s if the allowed bandwidth ranges from 300 Mbit/s to 1000 Mbit/s. | + | | | | - The minimum increment is 500 Mbit/s if the allowed bandwidth is greater than 1000 Mbit/s. | + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_sharetype | Yes for public network load balancers | String | Bandwidth type. | + | | | | | + | | | | **PER**: dedicated bandwidth. | + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | eip_type | Yes for public network load balancers | String | EIP type. | + | | | | | + | | | | - **5_bgp**: dynamic BGP | + | | | | - **5_sbgp**: static BGP | + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | name | No | String | Name of the automatically created load balancer. | + | | | | | + | | | | Value range: a string of 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | + | | | | | + | | | | Default: **cce-lb+ingress.UID** | + +----------------------+---------------------------------------+-----------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Create an ingress. @@ -243,11 +335,11 @@ The following describes how to run the kubectl command to automatically create a NAME HOSTS ADDRESS PORTS AGE ingress-test * 121.**.**.** 80 10s -#. Enter **http://121.**.**.*\*:80** in the address box of the browser to access the workload (for example, :ref:`Nginx workload `). +#. Enter **http://121.**.**.*\*:80** in the address box of the browser to access the workload (for example, :ref:`Nginx workload `). **121.**.**.*\*** indicates the IP address of the unified load balancer. -.. _cce_01_0252__section32300431736: +.. _cce_10_0252__section32300431736: Creating an Ingress - Interconnecting with an Existing Load Balancer -------------------------------------------------------------------- @@ -256,11 +348,37 @@ CCE allows you to connect to an existing load balancer when creating an ingress. .. note:: - - For clusters of v1.15 or later, the value of **apiVersion** is **networking.k8s.io/v1beta1**. - - For clusters of v1.13 or earlier, the value of **apiVersion** is **extensions/v1beta1**. - - To interconnect with an existing dedicated load balancer, ensure that HTTP is supported and the network type supports private networks. + - Existing dedicated load balancers must be the application type (HTTP/HTTPS) supporting private networks (with a private IP). -**If the cluster version is 1.15 or later, the YAML file configuration is as follows:** +**If the cluster version is 1.23 or earlier, the YAML file configuration is as follows:** + +.. code-block:: + + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: ingress-test + annotations: + kubernetes.io/elb.id: # Replace it with the ID of your existing load balancer. + kubernetes.io/elb.ip: # Replace it with your existing load balancer IP. + kubernetes.io/elb.port: '80' + spec: + rules: + - host: '' + http: + paths: + - path: '/' + backend: + service: + name: # Replace it with the name of your target Service. + port: + number: 8080 # Replace 8080 with your target service port number. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce + +**If the cluster version is 1.21 or later, the YAML file configuration is as follows:** .. code-block:: @@ -269,7 +387,6 @@ CCE allows you to connect to an existing load balancer when creating an ingress. metadata: name: ingress-test annotations: - kubernetes.io/elb.class: performance # Load balancer type kubernetes.io/elb.id: # Replace it with the ID of your existing load balancer. kubernetes.io/elb.ip: # Replace it with your existing load balancer IP. kubernetes.io/elb.port: '80' @@ -288,38 +405,28 @@ CCE allows you to connect to an existing load balancer when creating an ingress. .. table:: **Table 3** Key parameters - +-------------------------+-----------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Mandatory | Type | Description | - +=========================+=================+=================+=========================================================================================================================================================================================================+ - | kubernetes.io/elb.class | No | String | Select a proper load balancer type. | - | | | | | - | | | | The value can be: | - | | | | | - | | | | - **union**: shared load balancer | - | | | | - **performance**: dedicated load balancer.. | - | | | | | - | | | | Defaults to **union**. | - +-------------------------+-----------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.id | Yes | String | This parameter indicates the ID of a load balancer. The value can contain 1 to 100 characters. | - | | | | | - | | | | **How to obtain**: | - | | | | | - | | | | On the management console, click **Service List**, and choose **Networking** > **Elastic Load Balance**. Click the name of the target load balancer. On the **Summary** tab page, find and copy the ID. | - +-------------------------+-----------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.ip | Yes | String | This parameter indicates the service address of a load balancer. The value can be the public IP address of a public network load balancer or the private IP address of a private network load balancer. | - +-------------------------+-----------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +----------------------+-----------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +======================+=================+=================+=========================================================================================================================================================================================================+ + | kubernetes.io/elb.id | Yes | String | This parameter indicates the ID of a load balancer. The value can contain 1 to 100 characters. | + | | | | | + | | | | **How to obtain**: | + | | | | | + | | | | On the management console, click **Service List**, and choose **Networking** > **Elastic Load Balance**. Click the name of the target load balancer. On the **Summary** tab page, find and copy the ID. | + +----------------------+-----------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.ip | Yes | String | This parameter indicates the service address of a load balancer. The value can be the public IP address of a public network load balancer or the private IP address of a private network load balancer. | + +----------------------+-----------------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Configuring HTTPS Certificates ------------------------------ -Ingress supports TLS certificate configuration and provides security services in HTTPS mode. +Ingress supports TLS certificate configuration and secures your Services with HTTPS. .. note:: - - If a Service needs to be exposed using HTTPS, you must configure the TLS certificate in the ingress. For details on how to create a secret, see :ref:`Creating a Secret `. - - If HTTPS is used for the same port of the same load balancer of multiple ingresses, you must select the same certificate. + If HTTPS is enabled for the same port of the same load balancer of multiple ingresses, you must select the same certificate. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following command to create a YAML file named **ingress-test-secret.yaml** (the file name can be customized): @@ -343,7 +450,7 @@ Ingress supports TLS certificate configuration and provides security services in .. note:: - In the preceding information, **tls.crt** and **tls.key** are only examples. Replace them with the actual files. The values of **tls.crt** and **tls.key** are the content encrypted using Base64. + In the preceding information, **tls.crt** and **tls.key** are only examples. Replace them with the actual files. The values of **tls.crt** and **tls.key** are Base64-encoded. #. Create a secret. @@ -372,9 +479,11 @@ Ingress supports TLS certificate configuration and provides security services in .. note:: - Security policy (kubernetes.io/elb.tls-ciphers-policy) is supported only in clusters of v1.17.11 or later. + Default security policy (kubernetes.io/elb.tls-ciphers-policy) is supported only in clusters of v1.17.17 or later. - **Example YAML file to associate an existing load balancer:** + **The following uses the automatically created load balancer as an example. The YAML file is configured as follows:** + + **For clusters of v1.21 or earlier:** .. code-block:: @@ -383,11 +492,18 @@ Ingress supports TLS certificate configuration and provides security services in metadata: name: ingress-test annotations: - kubernetes.io/elb.class: performance # Load balancer type - kubernetes.io/elb.id: # Replace it with the ID of your existing load balancer. - kubernetes.io/elb.ip: # Replace it with the IP of your existing load balancer. + kubernetes.io/elb.class: union kubernetes.io/ingress.class: cce kubernetes.io/elb.port: '443' + kubernetes.io/elb.autocreate: + '{ + "type":"public", + "bandwidth_name":"cce-bandwidth-15511633796**", + "bandwidth_chargemode":"bandwidth", + "bandwidth_size":5, + "bandwidth_sharetype":"PER", + "eip_type":"5_bgp" + }' kubernetes.io/elb.tls-ciphers-policy: tls-1-2 spec: tls: @@ -403,12 +519,51 @@ Ingress supports TLS certificate configuration and provides security services in property: ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + **For clusters of v1.23 or later:** + + .. code-block:: + + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: ingress-test + annotations: + kubernetes.io/elb.class: union + kubernetes.io/elb.port: '443' + kubernetes.io/elb.autocreate: + '{ + "type":"public", + "bandwidth_name":"cce-bandwidth-15511633796**", + "bandwidth_chargemode":"bandwidth", + "bandwidth_size":5, + "bandwidth_sharetype":"PER", + "eip_type":"5_bgp" + }' + kubernetes.io/elb.tls-ciphers-policy: tls-1-2 + spec: + tls: + - secretName: ingress-test-secret + rules: + - host: '' + http: + paths: + - path: '/' + backend: + service: + name: # Replace it with the name of your target Service. + port: + number: 8080 # Replace 8080 with the port number of your target Service. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce + .. table:: **Table 4** Key parameters +--------------------------------------+-----------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Mandatory | Type | Description | +======================================+=================+==================+============================================================================================================================================================================================================================================+ - | kubernetes.io/elb.tls-ciphers-policy | No | String | The default value is **tls-1-2**, which is the security policy used by the listener and takes effect only when the HTTPS protocol is used. | + | kubernetes.io/elb.tls-ciphers-policy | No | String | The default value is **tls-1-2**, which is the default security policy used by the listener and takes effect only when the HTTPS protocol is used. | | | | | | | | | | Options: | | | | | | @@ -417,14 +572,14 @@ Ingress supports TLS certificate configuration and provides security services in | | | | - tls-1-2 | | | | | - tls-1-2-strict | | | | | | - | | | | For details of cipher suites for each security policy, see :ref:`Table 5 `. | + | | | | For details of cipher suites for each security policy, see :ref:`Table 5 `. | +--------------------------------------+-----------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | tls | No | Array of strings | This parameter is mandatory if HTTPS is used. Multiple independent domain names and certificates can be added to this parameter. For details, see :ref:`Configuring the Server Name Indication (SNI) `. | + | tls | No | Array of strings | This parameter is mandatory if HTTPS is used. Multiple independent domain names and certificates can be added to this parameter. For details, see :ref:`Configuring the Server Name Indication (SNI) `. | +--------------------------------------+-----------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | secretName | No | String | This parameter is mandatory if HTTPS is used. Set this parameter to the name of the created secret. | +--------------------------------------+-----------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - .. _cce_01_0252__table9419191416246: + .. _cce_10_0252__table9419191416246: .. table:: **Table 5** tls_ciphers_policy parameter description @@ -467,11 +622,94 @@ Ingress supports TLS certificate configuration and provides security services in NAME HOSTS ADDRESS PORTS AGE ingress-test * 121.**.**.** 80 10s -#. Enter **https://121.**.**.*\*:443** in the address box of the browser to access the workload (for example, :ref:`Nginx workload `). +#. Enter **https://121.**.**.*\*:443** in the address box of the browser to access the workload (for example, :ref:`Nginx workload `). **121.**.**.*\*** indicates the IP address of the unified load balancer. -.. _cce_01_0252__section0555194782414: +Using HTTP/2 +------------ + +Ingresses can use HTTP/2 to expose services. Connections from the load balancer proxy to your applications use HTTP/1.X by default. If your application is capable of receiving HTTP/2 requests, you can add the following field to the ingress annotation to enable the use of HTTP/2: + +\`kubernetes.io/elb.http2-enable: 'true'\` + +The following shows the YAML file for associating with an existing load balancer: + +**For clusters of v1.21 or earlier:** + +.. code-block:: + + apiVersion: networking.k8s.io/v1beta1 + kind: Ingress + metadata: + name: ingress-test + annotations: + kubernetes.io/elb.id: # Replace it with the ID of your existing load balancer. + kubernetes.io/elb.ip: # Replace it with the IP of your existing load balancer. + kubernetes.io/elb.port: '443' + kubernetes.io/ingress.class: cce + kubernetes.io/elb.http2-enable: 'true' # Enable HTTP/2. + spec: + tls: + - secretName: ingress-test-secret + rules: + - host: '' + http: + paths: + - path: '/' + backend: + serviceName: # Replace it with the name of your target Service. + servicePort: 80 # Replace it with the port number of your target Service. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + +**For clusters of v1.23 or later:** + +.. code-block:: + + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: ingress-test + annotations: + kubernetes.io/elb.id: # Replace it with the ID of your existing load balancer. + kubernetes.io/elb.ip: # Replace it with the IP of your existing load balancer. + kubernetes.io/elb.port: '443' + kubernetes.io/elb.http2-enable: 'true' # Enable HTTP/2. + spec: + tls: + - secretName: ingress-test-secret + rules: + - host: '' + http: + paths: + - path: '/' + backend: + service: + name: # Replace it with the name of your target Service. + port: + number: 8080 # Replace 8080 with the port number of your target Service. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce + +Table 6 HTTP/2 parameters + ++--------------------------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Parameter | Mandatory | Type | Description | ++================================+=================+=================+==================================================================================================================================================================================================================================================================================================================================+ +| kubernetes.io/elb.http2-enable | No | Bool | Whether HTTP/2 is enabled. Request forwarding using HTTP/2 improves the access performance between your application and the load balancer. However, the load balancer still uses HTTP 1.X to forward requests to the backend server. **This parameter is supported in clusters of v1.19.16-r0, v1.21.3-r0, and later versions.** | +| | | | | +| | | | Options: | +| | | | | +| | | | - **true**: enabled | +| | | | - **false**: disabled (default value) | +| | | | | +| | | | Note: **HTTP/2 can be enabled or disabled only when the listener uses HTTPS.** This parameter is invalid and defaults to **false** when the listener protocol is HTTP. | ++--------------------------------+-----------------+-----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +.. _cce_10_0252__section0555194782414: Configuring the Server Name Indication (SNI) -------------------------------------------- @@ -485,6 +723,8 @@ SNI allows multiple TLS-based access domain names to be provided for external sy You can enable SNI when the preceding conditions are met. The following uses the automatic creation of a load balancer as an example. In this example, **sni-test-secret-1** and **sni-test-secret-2** are SNI certificates. The domain names specified by the certificates must be the same as those in the certificates. +**For clusters of v1.21 or earlier:** + .. code-block:: apiVersion: networking.k8s.io/v1beta1 @@ -492,11 +732,18 @@ You can enable SNI when the preceding conditions are met. The following uses the metadata: name: ingress-test annotations: - kubernetes.io/elb.class: performance # Load balancer type - kubernetes.io/elb.id: # Replace it with the ID of your existing load balancer. - kubernetes.io/elb.ip: # Replace it with the IP of your existing load balancer. + kubernetes.io/elb.class: union kubernetes.io/ingress.class: cce kubernetes.io/elb.port: '443' + kubernetes.io/elb.autocreate: + '{ + "type":"public", + "bandwidth_name":"cce-bandwidth-******", + "bandwidth_chargemode":"bandwidth", + "bandwidth_size":5, + "bandwidth_sharetype":"PER", + "eip_type":"5_bgp" + }' kubernetes.io/elb.tls-ciphers-policy: tls-1-2 spec: tls: @@ -518,6 +765,51 @@ You can enable SNI when the preceding conditions are met. The following uses the property: ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH +**For clusters of v1.23 or later:** + +.. code-block:: + + apiVersion: networking.k8s.io/v1 + kind: Ingress + metadata: + name: ingress-test + annotations: + kubernetes.io/elb.class: union + kubernetes.io/elb.port: '443' + kubernetes.io/elb.autocreate: + '{ + "type":"public", + "bandwidth_name":"cce-bandwidth-******", + "bandwidth_chargemode":"bandwidth", + "bandwidth_size":5, + "bandwidth_sharetype":"PER", + "eip_type":"5_bgp" + }' + kubernetes.io/elb.tls-ciphers-policy: tls-1-2 + spec: + tls: + - secretName: ingress-test-secret + - hosts: + - example.top # Domain name specified a certificate is issued + secretName: sni-test-secret-1 + - hosts: + - example.com # Domain name specified a certificate is issued + secretName: sni-test-secret-2 + rules: + - host: '' + http: + paths: + - path: '/' + backend: + service: + name: # Replace it with the name of your target Service. + port: + number: 8080 # Replace 8080 with the port number of your target Service. + property: + ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + pathType: ImplementationSpecific + ingressClassName: cce + Accessing Multiple Services --------------------------- @@ -555,3 +847,5 @@ Ingresses can route requests to multiple backend Services based on different mat servicePort: 80 property: ingress.beta.kubernetes.io/url-match-mode: STARTS_WITH + +.. |image1| image:: /_static/images/en-us_image_0000001276433425.png diff --git a/umn/source/networking/network_policies.rst b/umn/source/networking/network_policies.rst index 73885f7..231d442 100644 --- a/umn/source/networking/network_policies.rst +++ b/umn/source/networking/network_policies.rst @@ -1,20 +1,26 @@ -:original_name: cce_01_0059.html +:original_name: cce_10_0059.html -.. _cce_01_0059: +.. _cce_10_0059: Network Policies ================ -As the service logic becomes increasingly complex, many applications require network calls between modules. Traditional external firewalls or application-based firewalls cannot meet the requirements. Network policies are urgently needed between modules, service logic layers, or functional teams in a large cluster. +NetworkPolicy is a Kubernetes object used to restrict pod access. In CCE, by setting network policies, you can define ingress rules specifying the addresses to access pods or egress rules specifying the addresses pods can access. This is equivalent to setting up a firewall at the application layer to further ensure network security. -CCE has enhanced the Kubernetes-based network policy feature, allowing network isolation in a cluster by configuring network policies. This means that a firewall can be set between pods. +Network policies depend on the networking add-on of the cluster to which the policies apply. -For example, to make a payment system accessible only to specified components for security purposes, you can configure network policies. +By default, if a namespace does not have any policy, pods in the namespace accept traffic from any source and send traffic to any destination. + +Network policy rules are classified into the following types: + +- **namespaceSelector**: selects particular namespaces for which all pods should be allowed as ingress sources or egress destinations. +- **podSelector**: selects particular pods in the same namespace as the network policy which should be allowed as ingress sources or egress destinations. +- **ipBlock**: selects particular IP blocks to allow as ingress sources or egress destinations. (Only egress rules support IP blocks.) Notes and Constraints --------------------- -- Only clusters that use the **tunnel network model** support network policies. +- Only clusters that use the tunnel network model support network policies. - Network isolation is not supported for IPv6 addresses. @@ -28,11 +34,6 @@ Notes and Constraints - If a cluster is upgraded to v1.23 in in-place mode, you cannot use egress rules because the node OS is not upgraded. In this case, reset the node. -Precautions ------------ - -If no network policies have been configured for a workload, such as **workload-1**, other workloads in the same cluster can access **workload-1**. - Using Ingress Rules ------------------- @@ -87,9 +88,9 @@ Using Ingress Rules - protocol: TCP port: 6379 - :ref:`Figure 2 ` shows how namespaceSelector selects ingress sources. + :ref:`Figure 2 ` shows how namespaceSelector selects ingress sources. - .. _cce_01_0059__en-us_topic_0249851123_fig127351855617: + .. _cce_10_0059__en-us_topic_0249851123_fig127351855617: .. figure:: /_static/images/en-us_image_0259558489.png :alt: **Figure 2** namespaceSelector @@ -171,46 +172,57 @@ Diagram: **Figure 4** Using both ingress and egress -Adding a Network Policy on the Console --------------------------------------- +Creating a Network Policy on the Console +---------------------------------------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Network**. On the **Network Policies** tab page, click **Create Network Policy**. +#. Log in to the CCE console and access the cluster console. +#. Choose **Networking** in the navigation pane, click the **Network Policies** tab, and click **Create Network Policy** in the upper right corner. - - **Network Policy Name**: Specify a network policy name. - - - **Cluster Name**: Select a cluster to which the network policy belongs. + - **Policy Name**: Specify a network policy name. - **Namespace**: Select a namespace in which the network policy is applied. - - **Workload** + - **Selector**: Enter a label, select the pod to be associated, and click **Add**. You can also click **Reference Workload Label** to reference the label of an existing workload. - Click **Select Workload**. In the dialog box displayed, select a workload for which the network policy is to be created, for example, **workload-1**. Then, click **OK**. + - **Inbound Rule**: Click |image1| to add an inbound rule. For details about parameter settings, see :ref:`Table 1 `. - - **Rules**: Click **Add Rule**, set the parameters listed in :ref:`Table 1 `, and click **OK**. + |image2| - .. _cce_01_0059__table26919378234: + .. _cce_10_0059__table166419994515: - .. table:: **Table 1** Parameters for adding a rule + .. table:: **Table 1** Adding an inbound rule - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+================================================================================================================================================================+ - | Direction | Only **Inbound** is supported, indicating that the whitelisted workloads access the current workload (**workload-1** in this example). | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Protocol | Select a protocol. Currently, the TCP and UDP protocols are supported. The ICMP protocol is not supported. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Destination Container Port | Specify a port on which the workload in the container image listens. The Nginx application listens on port 80. | - | | | - | | If no container port is specified, all ports can be accessed by default. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Whitelisted Workloads | Select other workloads that can access the current workload. These workloads will access the current workload at the destination container port. | - | | | - | | - **Namespace**: All workloads in the selected namespace(s) are added to the whitelist. That is, all workloads in the namespace(s) can access **workload-1**. | - | | - **Workload**: The selected workloads can access **workload-1**. Only other workloads in the same namespace as **workload-1** can be selected. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +==================+==============================================================================================================================================================+ + | Protocol & Port | Select the protocol type and port. Currently, TCP and UDP are supported. | + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Source Namespace | Select a namespace whose objects can be accessed. If this parameter is not specified, the source object belongs to the same namespace as the current policy. | + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Source Pod Label | Allow access to the pods with this label, if not specified, all pods in the namespace can be accessed. | + +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------+ -#. Click **Create**. + - **Outbound Rule**: Click |image3| to add an outbound rule. For details about parameter settings, see :ref:`Table 1 `. -#. Repeat the preceding steps to add more network policies for the current workload when other ports need to be accessed by some workloads. + |image4| - After the network policies are created, only the specified workloads or workloads in the specified namespaces can access the current workload. + .. table:: **Table 2** Adding an outbound rule + + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +========================+===================================================================================================================================================================================================================================================================================================================================================================================+ + | Protocol & Port | Select the protocol type and port. Currently, TCP and UDP are supported. If this parameter is not specified, the protocol type is not limited. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Destination CIDR Block | Allows requests to be routed to a specified CIDR block (and not to the exception CIDR blocks). Separate the destination and exception CIDR blocks by vertical bars (|), and separate multiple exception CIDR blocks by commas (,). For example, 172.17.0.0/16|172.17.1.0/24,172.17.2.0/24 indicates that 172.17.0.0/16 is accessible, but not for 172.17.1.0/24 or 172.17.2.0/24. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Destination Namespace | Select a namespace whose objects can be accessed. If this parameter is not specified, the source object belongs to the same namespace as the current policy. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Destination Pod Label | Allow access to the pods with this label, if not specified, all pods in the namespace can be accessed. | + +------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. Click **OK**. + +.. |image1| image:: /_static/images/en-us_image_0000001251716033.png +.. |image2| image:: /_static/images/en-us_image_0000001533585325.png +.. |image3| image:: /_static/images/en-us_image_0000001533586881.png +.. |image4| image:: /_static/images/en-us_image_0000001482546084.png diff --git a/umn/source/networking/overview.rst b/umn/source/networking/overview.rst index f25bd35..2211858 100644 --- a/umn/source/networking/overview.rst +++ b/umn/source/networking/overview.rst @@ -1,16 +1,16 @@ -:original_name: cce_01_0010.html +:original_name: cce_10_0010.html -.. _cce_01_0010: +.. _cce_10_0010: Overview ======== You can learn about a cluster network from the following two aspects: -- What is a cluster network like? A cluster consists of multiple nodes, and pods (or containers) are running on the nodes. Nodes and containers need to communicate with each other. For details about the cluster network types and their functions, see :ref:`Cluster Network Structure `. -- How is pod access is implemented in a cluster? Accessing a pod or container is a process of accessing services of a user. Kubernetes provides :ref:`Service ` and :ref:`Ingress ` to address pod access issues. This section summarizes common network access scenarios. You can select the proper scenario based on site requirements. For details about the network access scenarios, see :ref:`Access Scenarios `. +- What is a cluster network like? A cluster consists of multiple nodes, and pods (or containers) are running on the nodes. Nodes and containers need to communicate with each other. For details about the cluster network types and their functions, see :ref:`Cluster Network Structure `. +- How is pod access implemented in a cluster? Accessing a pod or container is a process of accessing services of a user. Kubernetes provides :ref:`Service ` and :ref:`Ingress ` to address pod access issues. This section summarizes common network access scenarios. You can select the proper scenario based on site requirements. For details about the network access scenarios, see :ref:`Access Scenarios `. -.. _cce_01_0010__section1131733719195: +.. _cce_10_0010__section1131733719195: Cluster Network Structure ------------------------- @@ -30,16 +30,16 @@ All nodes in the cluster are located in a VPC and use the VPC network. The conta Currently, CCE supports the following container network models: - Container tunnel network: The container tunnel network is constructed on but independent of the node network through tunnel encapsulation. This network model uses VXLAN to encapsulate Ethernet packets into UDP packets and transmits them in tunnels. Open vSwitch serves as the backend virtual switch. - - VPC network: The VPC network uses VPC routing to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the route quota in a VPC network. Each node is assigned a CIDR block of a fixed size. This networking model is free from tunnel encapsulation overhead and outperforms the container tunnel network model. In addition, as VPC routing includes routes to node IP addresses and the container CIDR block, container pods in the cluster can be directly accessed from outside the cluster. + - VPC network: The VPC network uses VPC routing to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the route quota in a VPC network. Each node is assigned a CIDR block of a fixed size. This networking model is free from tunnel encapsulation overhead and outperforms the container tunnel network model. In addition, as VPC routing includes routes to node IP addresses and the container CIDR block, container pods in a cluster can be directly accessed from outside the cluster. - Developed by CCE, Cloud Native Network 2.0 deeply integrates Elastic Network Interfaces (ENIs) and Sub Network Interfaces (sub-ENIs) of VPC. Container IP addresses are allocated from the VPC CIDR block. ELB passthrough networking is supported to direct access requests to containers. Security groups and elastic IPs (EIPs) are bound to deliver high performance. - The performance, networking scale, and application scenarios of a container network vary according to the container network model. For details about the functions and features of different container network models, see :ref:`Overview `. + The performance, networking scale, and application scenarios of a container network vary according to the container network model. For details about the functions and features of different container network models, see :ref:`Overview `. - **Service Network** Service is also a Kubernetes object. Each Service has a fixed IP address. When creating a cluster on CCE, you can specify the Service CIDR block. The Service CIDR block cannot overlap with the node or container CIDR block. The Service CIDR block can be used only within a cluster. -.. _cce_01_0010__section1860619221134: +.. _cce_10_0010__section1860619221134: Service ------- @@ -57,11 +57,10 @@ You can configure the following types of Services: - ClusterIP: used to make the Service only reachable from within a cluster. - NodePort: used for access from outside a cluster. A NodePort Service is accessed through the port on the node. - LoadBalancer: used for access from outside a cluster. It is an extension of NodePort, to which a load balancer routes, and external systems only need to access the load balancer. -- ENI LoadBalancer: used for access from outside the cluster. An ENI LoadBalancer Service directs traffic from a load balancer at backend pods, reducing the latency and avoiding performance loss for containerized applications. -For details about the Service, see :ref:`Overview `. +For details about the Service, see :ref:`Service Overview `. -.. _cce_01_0010__section1248852094313: +.. _cce_10_0010__section1248852094313: Ingress ------- @@ -74,9 +73,9 @@ Services forward requests using layer-4 TCP and UDP protocols. Ingresses forward **Figure 2** Ingress and Service -For details about the ingress, see :ref:`Overview `. +For details about the ingress, see :ref:`Ingress Overview `. -.. _cce_01_0010__section1286493159: +.. _cce_10_0010__section1286493159: Access Scenarios ---------------- @@ -92,12 +91,12 @@ Workload access scenarios can be categorized as follows: - External access initiated by a workload: - Accessing an intranet: The workload accesses the intranet address, but the implementation method varies depending on container network models. Ensure that the peer security group allows the access requests from the container CIDR block. - - Accessing a public network: You need to assign an EIP to the node where the workload runs, or configure SNAT rules through the NAT gateway. + - Accessing a public network: You need to assign an EIP to the node where the workload runs (when the VPC network or tunnel network model is used), bind an EIP to the pod IP address (when the Cloud Native Network 2.0 model is used), or configure SNAT rules through the NAT gateway. For details, see :ref:`Accessing Public Networks from a Container `. -.. figure:: /_static/images/en-us_image_0000001160748146.png +.. figure:: /_static/images/en-us_image_0000001244261169.png :alt: **Figure 3** Network access diagram **Figure 3** Network access diagram -.. |image1| image:: /_static/images/en-us_image_0000001159292060.png +.. |image1| image:: /_static/images/en-us_image_0000001199181334.png diff --git a/umn/source/networking/securitygroups.rst b/umn/source/networking/securitygroups.rst deleted file mode 100644 index 4187003..0000000 --- a/umn/source/networking/securitygroups.rst +++ /dev/null @@ -1,162 +0,0 @@ -:original_name: cce_01_0288.html - -.. _cce_01_0288: - -SecurityGroups -============== - -When the Cloud Native Network 2.0 model is used, pods use VPC ENIs or sub-ENIs for networking. You can directly bind security groups and EIPs to pods. CCE provides a custom resource object named **SecurityGroup** for you to associate security groups with pods in CCE. You can customize workloads with specific security isolation requirements using SecurityGroups. - -Notes and Constraints ---------------------- - -- This function is supported for CCE Turbo clusters of v1.19 and later. Upgrade your CCE Turbo clusters if their versions are earlier than v1.19. -- A workload can be bound to a maximum of five security groups. - -Using the Console ------------------ - -#. In the navigation pane of the CCE console, choose **Resource Management** > **Network**. - -#. On the **SecurityGroup** tab page, select the target cluster in the upper right corner and click **Create**. - -#. Set the parameters as described in :ref:`Table 1 `. - - .. _cce_01_0288__table572616321913: - - .. table:: **Table 1** Configuration parameters - - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ - | Parameter | Description | Example Value | - +=======================+===========================================================================================================================================================================================================================================================+======================================+ - | SecurityGroup Name | Enter a SecurityGroup name. | security-group | - | | | | - | | Enter 4 to 63 characters. The value must start with a lowercase letter and cannot end with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed. | | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ - | Cluster Name | Select a cluster. | cce-turbo | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ - | Namespace | Select a namespace. If the namespace is not created, click **Create Namespace**. | default | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ - | Workload | Select a workload. | nginx | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ - | Security Group | The selected security group will be bound to the ENI or supplementary ENI of the selected workload. A maximum of five security groups can be selected from the drop-down list. You must select one or multiple security groups to create a SecurityGroup. | 64566556-bd6f-48fb-b2c6-df8f44617953 | - | | | | - | | If no security group has not been created, click **Create Security Group**. After the security group is created, click the refresh button. | 5451f1b0-bd6f-48fb-b2c6-df8f44617953 | - | | | | - | | .. important:: | | - | | | | - | | NOTICE: | | - | | | | - | | - A maximum of 5 security groups can be selected. | | - | | - Hover the cursor on the security group name, and you can view details about the security group. | | - +-----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ - -#. After setting the parameters, click **Create**. - - After the SecurityGroup is created, the system automatically returns to the SecurityGroup list page. You can see that the newly added SecurityGroup is in the list. - -Using kubectl -------------- - -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -#. Create a description file named **securitygroup-demo.yaml**. - - **vi securitygroup-demo.yaml** - - For example, create the following SecurityGroup to bind all nginx workloads with two security groups 64566556-bd6f-48fb-b2c6-df8f44617953 and 5451f1b0-bd6f-48fb-b2c6-df8f44617953 that have been created in advance. An example is as follows: - - .. code-block:: - - apiVersion: crd.yangtse.cni/v1 - kind: SecurityGroup - metadata: - name: demo - namespace: default - spec: - podSelector: - matchLabels: - app: nginx - securityGroups: - - id: 64566556-bd6f-48fb-b2c6-df8f44617953 - - id: 5451f1b0-bd6f-48fb-b2c6-df8f44617953 - - :ref:`Table 2 ` describes the parameters in the YAML file. - - .. _cce_01_0288__table132326831016: - - .. table:: **Table 2** Description - - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | Field | Description | Mandatory | - +================+=========================================================================================+===========+ - | apiVersion | API version. The value is **crd.yangtse.cni/v1**. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | kind | Type of the object to be created. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | metadata | Metadata definition of the resource object. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | name | Name of the SecurityGroup. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | namespace | Name of the namespace. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | Spec | Detailed description of the SecurityGroup. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | podselector | Used to define the workload to be associated with security groups in the SecurityGroup. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - | SecurityGroups | Security group ID. | Yes | - +----------------+-----------------------------------------------------------------------------------------+-----------+ - -#. Run the following command to create the SecurityGroup: - - **kubectl create -f securitygroup-demo.yaml** - - If the following information is displayed, the SecurityGroup is being created. - - .. code-block:: - - securitygroup.crd.yangtse.cni/demo created - -#. Run the following command to view the SecurityGroup: - - **kubectl get sg** - - If the name of the created SecurityGroup is **demo** in the command output, the SecurityGroup is created successfully. - - .. code-block:: - - NAME POD-SELECTOR AGE - all-no map[matchLabels:map[app:nginx]] 4h1m - s001test map[matchLabels:map[app:nginx]] 19m - demo map[matchLabels:map[app:nginx]] 2m9s - -Other Operations ----------------- - -.. table:: **Table 3** Other operations - - +-----------------------------------+-------------------------------------------------------------------------------------------------+ - | Operation | Procedure | - +===================================+=================================================================================================+ - | Deletion | #. In the navigation pane of the CCE console, choose **Resource Management** > **Network**. | - | | #. On the **SecurityGroup** tab page, select the target SecurityGroup. | - | | #. Click SecurityGroup to delete the SecurityGroup. | - +-----------------------------------+-------------------------------------------------------------------------------------------------+ - | Update | #. In the navigation pane of the CCE console, choose **Resource Management** > **Network**. | - | | | - | | #. On the **SecurityGroup** tab page, click **Update** at the same row as the SecurityGroup. | - | | | - | | You can update the SecurityGroup ID and associated workload. | - +-----------------------------------+-------------------------------------------------------------------------------------------------+ - | Viewing the YAML file | #. In the navigation pane of the CCE console, choose **Resource Management** > **Network**. | - | | | - | | #. On the **SecurityGroup** tab page, click **View YAML** at the same row as the SecurityGroup. | - | | | - | | You can view, copy, and download the YAML file. | - +-----------------------------------+-------------------------------------------------------------------------------------------------+ - | Viewing events | #. In the navigation pane of the CCE console, choose **Resource Management** > **Network**. | - | | | - | | #. On the **SecurityGroup** tab page, click **View Event**. | - | | | - | | You can query the event information. | - +-----------------------------------+-------------------------------------------------------------------------------------------------+ diff --git a/umn/source/networking/services/eni_loadbalancer.rst b/umn/source/networking/services/eni_loadbalancer.rst deleted file mode 100644 index a3f469b..0000000 --- a/umn/source/networking/services/eni_loadbalancer.rst +++ /dev/null @@ -1,181 +0,0 @@ -:original_name: cce_01_0114.html - -.. _cce_01_0114: - -ENI LoadBalancer -================ - -.. _cce_01_0114__section025118182286: - -Scenario --------- - -An ENI LoadBalancer Service directs traffic from a load balancer at backend pods, reducing the latency and avoiding performance loss for containerized applications. - -External access requests are directly forwarded from a load balancer to pods. Internal access requests can be forwarded to a pod through a Service. - -|image1| - -Notes and Constraints ---------------------- - -- ENI LoadBalancer is available only in certain regions. -- Only dedicated load balancers are supported, and they must support layer-4 networking (TCP/UDP). -- After a load balancer is created, its flavor cannot be changed. Therefore, in CCE, after you create a Service, you cannot connect the automatically created load balancer to other objects. If no load balancer is automatically created, you can connect any existing one to the Service. -- The cluster version must be 1.17 or later. -- ENI LoadBalancer Services can be created only for workloads (containers) bound with elastic network interfaces (ENIs). - -.. _cce_01_0114__section17753911588: - -Adding a Service When Creating a Workload ------------------------------------------ - -You can set the Service when creating a workload on the CCE console. An Nginx workload is used as an example. - -#. In the **Set Application Access** step of :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `, click **Add Service** and set the following parameters: - - - **Access Type**: Select **ENI LoadBalancer (ELB)**. This option is available only if you have selected **Attach ENI to Pod** when specifying basic workload information during workload creation. - - **Service Name**: Specify a Service name, which can be the same as the workload name. - - **ELB Configuration** - - - **Elastic Load Balancer**: Only dedicated load balancers are supported. - - **Dedicated**: Resources are shared among load balancers, and the performance of a load balancer is not affected by other load balancers. IPv6 is supported. - - You can create **public network** or **private network** load balancers. - - - **Public network**: You can select an existing public network load balancer or have the system automatically create a new one. - - **Private network**: You can select an existing private network load balancer or have the system automatically create a new one. - - The selected or created load balancer must be in the same VPC as the current cluster, and it must match the load balancer type (private or public network). - - - **Enterprise Project**: Select an enterprise project in which the load balancer is created. - - **Specifications**: This field is displayed only when you select **Public network** and **Automatically created** for **Elastic Load Balancer**. You can click |image2| to modify the name, specifications, billing mode, and bandwidth of the load balancer. - - Configure Dedicated Load Balancer - - - **AZ**: Dedicated load balancers can be deployed across AZs to provide higher reliability. - - - **Subnet**: subnet where the backend server of the load balancer is located. - - Load balancers occupy different number of subnet IP addresses based on their specifications. Therefore, you are not advised to use the subnet CIDR blocks of other resources (such as clusters and nodes) as the load balancer CIDR block. - - - **Specifications**: Specifications determine the types of listeners that can be added to a load balancer. Select specifications that best fit your needs. For details, see `Specifications of Dedicated Load Balancers `__. - - - **Algorithm Type**: You can select **Weighted round robin**, **Weighted least connections**, or **Source IP hash**. The weight is dynamically adjusted based on the number of pods of the workload associated with the Service on each node. - - .. note:: - - - **Weighted round robin**: Requests are forwarded to different servers based on their weights, which indicate server processing performance. Backend servers with higher weights receive proportionately more requests, whereas equal-weighted servers receive the same number of requests. This algorithm is often used for short connections, such as HTTP services. - - **Weighted least connections**: In addition to the weight assigned to each server, the number of connections processed by each backend server is also considered. Requests are forwarded to the server with the lowest connections-to-weight ratio. Building on **least connections**, the **weighted least connections** algorithm assigns a weight to each server based on their processing performance. This algorithm is often used for persistent connections, such as database connections. - - **Source IP hash**: The source IP address of each request is calculated using the hash algorithm to obtain a unique hash key, and all backend servers are numbered. The generated key allocates the client to a particular server. This allows requests from different clients to be routed based on source IP addresses and ensures that a client is directed to the same server as always. This algorithm applies to TCP connections without cookies. - - - **Sticky Session**: This function is disabled by default. You can select **Based on source IP address**. Listeners ensure session stickiness based on IP addresses. Requests from the same IP address will be forwarded to the same backend server. - - **Health Check**: This function is enabled by default. Enabling it will perform health checks on your load balancer. For details about how to configure the ELB health check parameters, see `Configuring a Health Check `__. - - - **Port Settings** - - - **Protocol**: protocol used by the Service. - - **Container Port**: port defined in the container image and on which the workload listens. The Nginx application listens on port 80. - - **Access Port**: port mapped to the container port at the load balancer's IP address. The workload can be accessed at <*Load balancer's IP address*>:<*Access port*>. The port number range is 1-65535. - -#. After the configuration is complete, click **OK**. - -#. On the workload creation page, click **Next: Configure Advanced Settings**. On the page displayed, click **Create**. - -#. After the workload is successfully created, choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** on the CCE console. Click the name of the workload to view its details. On the workload details page, click the **Services** tab and obtain the access address. - -#. Click the access address. - -Adding a Service After Creating a Workload ------------------------------------------- - -You can set the Service after creating a workload. This has no impact on the workload status and takes effect immediately. The procedure is as follows: - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Network**. - -#. On the **Services** tab page, click **Create Service**. - - The parameters are the same as those in :ref:`Adding a Service When Creating a Workload `. - -#. Click **Create**. An ENI LoadBalancer Service will be added for the workload. - -Using kubectl to Create a Service (Automatically Creating a Load Balancer) --------------------------------------------------------------------------- - -An ENI LoadBalancer Service supports only dedicated ELBs. You do not need to specify NodePort when creating a Service. - -.. code-block:: - - apiVersion: v1 - kind: Service - metadata: - name: example - annotations: - kubernetes.io/elb.class: performance - kubernetes.io/elb.autocreate: - ' - { - "type": "public", - "bandwidth_name": "cce-bandwidth-1630813564682", - "bandwidth_chargemode": "traffic", - "bandwidth_size": 5, - "bandwidth_sharetype": "PER", - "eip_type": "5_bgp", - "available_zone": [ - "eu-de-01" - ], - "l7_flavor_name": "L7_flavor.elb.s2.medium", - "l4_flavor_name": "L4_flavor.elb.s1.small" - } - ' - spec: - selector: - app: example - ports: - - name: cce-service-0 - targetPort: 80 - port: 8082 - protocol: TCP - type: LoadBalancer - -For details about the parameters, see :ref:`Table 4 `. - -Using kubectl to Create a Service (Using an Existing Load Balancer) -------------------------------------------------------------------- - -When creating a Service using an existing load balancer, you only need to specify the ID of the load balancer. - -.. code-block:: - - apiVersion: v1 - kind: Service - metadata: - name: example - annotations: - kubernetes.io/elb.id: bcc44e84-d0b5-4192-8bec-b2ca55ce5025 # ID of the load balancer. Replace it with the actual value. - spec: - selector: - app: example - ports: - - name: cce-service-0 - targetPort: 80 - port: 8082 - protocol: TCP - type: LoadBalancer - -ELB Forwarding --------------- - -After an ENI LoadBalancer Service is created, you can view the listener forwarding rules of the load balancer on the ELB console. - - -.. figure:: /_static/images/en-us_image_0000001204449561.png - :alt: **Figure 1** ELB forwarding - - **Figure 1** ELB forwarding - -You can find that a listener is created for the load balancer. The backend server address is the IP address of the pod, and the service port is the container port. This is because the pod uses an ENI or sub-ENI. When traffic passes through the load balancer, it directly forwards the traffic to the pod. This is the same as that described in :ref:`Scenario `. - -.. |image1| image:: /_static/images/en-us_image_0000001152953258.png -.. |image2| image:: /_static/images/en-us_image_0000001126243447.png diff --git a/umn/source/networking/services/headless_service.rst b/umn/source/networking/services/headless_service.rst new file mode 100644 index 0000000..658fd36 --- /dev/null +++ b/umn/source/networking/services/headless_service.rst @@ -0,0 +1,68 @@ +:original_name: cce_10_0398.html + +.. _cce_10_0398: + +Headless Service +================ + +The preceding types of Services allow internal and external pod access, but not the following scenarios: + +- Accessing all pods at the same time +- Pods in a Service accessing each other + +This is where headless Service come into service. A headless Service does not create a cluster IP address, and the DNS records of all pods are returned during query. In this way, the IP addresses of all pods can be queried. :ref:`StatefulSets ` use headless Services to support mutual access between pods. + +.. code-block:: + + apiVersion: v1 + kind: Service # Object type (Service) + metadata: + name: nginx-headless + labels: + app: nginx + spec: + ports: + - name: nginx # - name: nginx # Name of the port for communication between pods + port: 80 # Port number for communication between pods + selector: + app: nginx # Select the pod whose label is app:nginx. + clusterIP: None # Set this parameter to None, indicating that a headless Service is to be created. + +Run the following command to create a headless Service: + +.. code-block:: + + # kubectl create -f headless.yaml + service/nginx-headless created + +After the Service is created, you can query the Service. + +.. code-block:: + + # kubectl get svc + NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE + nginx-headless ClusterIP None 80/TCP 5s + +Create a pod to query the DNS. You can view the records of all pods. In this way, all pods can be accessed. + +.. code-block:: + + $ kubectl run -i --tty --image tutum/dnsutils dnsutils --restart=Never --rm /bin/sh + If you do not see a command prompt, try pressing Enter. + / # nslookup nginx-0.nginx + Server: 10.247.3.10 + Address: 10.247.3.10#53 + Name: nginx-0.nginx.default.svc.cluster.local + Address: 172.16.0.31 + + / # nslookup nginx-1.nginx + Server: 10.247.3.10 + Address: 10.247.3.10#53 + Name: nginx-1.nginx.default.svc.cluster.local + Address: 172.16.0.18 + + / # nslookup nginx-2.nginx + Server: 10.247.3.10 + Address: 10.247.3.10#53 + Name: nginx-2.nginx.default.svc.cluster.local + Address: 172.16.0.19 diff --git a/umn/source/networking/services/index.rst b/umn/source/networking/services/index.rst index c534011..70d3e66 100644 --- a/umn/source/networking/services/index.rst +++ b/umn/source/networking/services/index.rst @@ -1,22 +1,24 @@ -:original_name: cce_01_0247.html +:original_name: cce_10_0247.html -.. _cce_01_0247: +.. _cce_10_0247: Services ======== -- :ref:`Overview ` -- :ref:`Intra-Cluster Access (ClusterIP) ` -- :ref:`NodePort ` -- :ref:`LoadBalancer ` -- :ref:`ENI LoadBalancer ` +- :ref:`Service Overview ` +- :ref:`Intra-Cluster Access (ClusterIP) ` +- :ref:`NodePort ` +- :ref:`LoadBalancer ` +- :ref:`Headless Service ` +- :ref:`Service Annotations ` .. toctree:: :maxdepth: 1 :hidden: - overview + service_overview intra-cluster_access_clusterip nodeport loadbalancer - eni_loadbalancer + headless_service + service_annotations diff --git a/umn/source/networking/services/intra-cluster_access_clusterip.rst b/umn/source/networking/services/intra-cluster_access_clusterip.rst index db56a0a..38ff599 100644 --- a/umn/source/networking/services/intra-cluster_access_clusterip.rst +++ b/umn/source/networking/services/intra-cluster_access_clusterip.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0011.html +:original_name: cce_10_0011.html -.. _cce_01_0011: +.. _cce_10_0011: Intra-Cluster Access (ClusterIP) ================================ @@ -12,63 +12,40 @@ ClusterIP Services allow workloads in the same cluster to use their cluster-inte The cluster-internal domain name format is **.\ **\ **.svc.cluster.local:**\ **, for example, **nginx.default.svc.cluster.local:80**. -:ref:`Figure 1 ` shows the mapping relationships between access channels, container ports, and access ports. +:ref:`Figure 1 ` shows the mapping relationships between access channels, container ports, and access ports. -.. _cce_01_0011__fig192245420557: +.. _cce_10_0011__fig192245420557: -.. figure:: /_static/images/en-us_image_0000001117575950.png +.. figure:: /_static/images/en-us_image_0000001243981117.png :alt: **Figure 1** Intra-cluster access (ClusterIP) **Figure 1** Intra-cluster access (ClusterIP) -Adding a Service When Creating a Workload ------------------------------------------ +Creating a ClusterIP Service +---------------------------- -You can set the access type (Service) when creating a workload on the CCE console. - -#. In the **Set Application Access** step of :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `, click **Add Service** and set the following parameters: - - - **Access Type**: Select **ClusterIP**. - - **Service Name**: Specify a Service name, which can be the same as the workload name. - - **Port Settings** - - - **Protocol**: protocol used by the Service. - - **Container Port**: port on which the workload listens. The Nginx application listens on port 80. - - **Access Port**: a port mapped to the container port at the cluster-internal IP address. The workload can be accessed at :. The port number range is 1-65535. - -#. After the configuration, click **OK** and then **Next: Configure Advanced Settings**. On the page displayed, click **Create**. -#. Click **View Deployment Details** or **View StatefulSet Details**. On the **Services** tab page, obtain the access address, for example, 10.247.74.100:8080. - -Adding a Service After Creating a Workload ------------------------------------------- - -You can set the Service after creating a workload. This has no impact on the workload status and takes effect immediately. The procedure is as follows: - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. On the workload list, click the name of the workload for which you will create a Service. -#. On the **Services** tab page, click **Add Service**. -#. On the **Create Service** page, select **ClusterIP** from the **Access Type** drop-down list. +#. Log in to the CCE console and access the cluster console. +#. Choose **Networking** in the navigation pane and click **Create Service** in the upper right corner. #. Set intra-cluster access parameters. - **Service Name**: Service name, which can be the same as the workload name. - - **Cluster Name**: name of the cluster where the workload runs. The value is inherited from the workload creation page and cannot be changed. - - **Namespace**: namespace where the workload is located. The value is inherited from the workload creation page and cannot be changed. - - **Workload**: workload for which you want to add a Service. + - **Service Type**: Select **ClusterIP**. + - **Namespace**: Namespace to which the workload belongs. + - **Selector**: Add a label and click **Add**. A Service selects a pod based on the added label. You can also click **Reference Workload Label** to reference the label of an existing workload. In the dialog box that is displayed, select a workload and click **OK**. - **Port Settings** - **Protocol**: protocol used by the Service. - - **Container Port**: port on which the workload listens. The Nginx application listens on port 80. - - **Access Port**: port mapped to the container port at the cluster-internal IP address. The workload can be accessed at :. The port number range is 1-65535. + - **Service Port**: port used by the Service. The port number ranges from 1 to 65535. + - **Container Port**: port on which the workload listens. For example, Nginx uses port 80 by default. -#. Click **Create**. The ClusterIP Service will be added for the workload. - -.. _cce_01_0011__section9813121512319: +#. Click **OK**. Setting the Access Type Using kubectl ------------------------------------- You can run kubectl commands to set the access type (Service). This section uses a Nginx workload as an example to describe how to implement intra-cluster access using kubectl. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create and edit the **nginx-deployment.yaml** and **nginx-clusterip-svc.yaml** files. @@ -169,7 +146,7 @@ You can run kubectl commands to set the access type (Service). This section uses .. code-block:: # kubectl run -i --tty --image nginx:alpine test --rm /bin/sh - If you don't see a command prompt, try pressing enter. + If you do not see a command prompt, try pressing Enter. / # curl 10.247.74.52:8080 diff --git a/umn/source/networking/services/loadbalancer.rst b/umn/source/networking/services/loadbalancer.rst index 2226897..20cf11c 100644 --- a/umn/source/networking/services/loadbalancer.rst +++ b/umn/source/networking/services/loadbalancer.rst @@ -1,10 +1,12 @@ -:original_name: cce_01_0014.html +:original_name: cce_10_0014.html -.. _cce_01_0014: +.. _cce_10_0014: LoadBalancer ============ +.. _cce_10_0014__section19854101411508: + Scenario -------- @@ -15,11 +17,21 @@ The LoadBalancer access address is in the format of ` is switched from the cluster level to the node level, the connection tracing table will not be cleared. You are advised not to modify the Service affinity setting after the Service is created. If you need to modify it, create a Service again. -- If the service affinity is set to the node level (that is, **externalTrafficPolicy** is set to **Local**), the cluster may fail to access the Service by using the ELB address. For details, see :ref:`Why a Cluster Fails to Access Services by Using the ELB Address `. +- After a Service is created, if the affinity setting is switched from the cluster level to the node level, the connection tracing table will not be cleared. You are advised not to modify the Service affinity setting after the Service is created. If you need to modify it, create a Service again. +- If the service affinity is set to the node level (that is, **externalTrafficPolicy** is set to **Local**), the cluster may fail to access the Service by using the ELB address. For details, see :ref:`Why a Cluster Fails to Access Services by Using the ELB Address `. - CCE Turbo clusters support only cluster-level service affinity. - Dedicated ELB load balancers can be used only in clusters of v1.17 and later. -- The specifications of dedicated load balancers must use TCP/UDP (network load balancing) and support private networks. If the Service needs to support HTTP, the specifications of dedicated load balancers must use HTTP (application load balancing) in addition to TCP/UDP (network load balancing). +- Dedicated load balancers must be the network type (TCP/UDP) supporting private networks (with a private IP). If the Service needs to support HTTP, the specifications of dedicated load balancers must use HTTP/HTTPS (application load balancing) in addition to TCP/UDP (network load balancing). - If you create a LoadBalancer Service on the CCE console, a random node port is automatically generated. If you use kubectl to create a LoadBalancer Service, a random node port is generated unless you specify one. - In a CCE cluster, if the cluster-level affinity is configured for a LoadBalancer Service, requests are distributed to the node ports of each node using SNAT when entering the cluster. The number of node ports cannot exceed the number of available node ports on the node. If the Service affinity is at the node level (local), there is no such constraint. In a CCE Turbo cluster, this constraint applies to shared ELB load balancers, but not dedicated ones. You are advised to use dedicated ELB load balancers in CCE Turbo clusters. - When the cluster service forwarding (proxy) mode is IPVS, the node IP cannot be configured as the external IP of the Service. Otherwise, the node is unavailable. -- Dedicated load balancers are available only in certain regions. +- In a cluster using the IPVS proxy mode, if the ingress and Service use the same ELB load balancer, the ingress cannot be accessed from the nodes and containers in the cluster because kube-proxy mounts the LoadBalancer Service address to the ipvs-0 bridge. This bridge intercepts the traffic of the load balancer connected to the ingress. You are advised to use different ELB load balancers for the ingress and Service. -.. _cce_01_0014__section744117150366: +Creating a LoadBalancer Service +------------------------------- -Adding a Service When Creating a Workload ------------------------------------------ - -You can set the Service when creating a workload on the CCE console. An Nginx workload is used as an example. - -#. In the **Set Application Access** step of :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `, click **Add Service** and set the following parameters: - - - **Access Type**: Select **LoadBalancer (ELB)**. +#. Log in to the CCE console and click the cluster name to access the cluster. +#. Choose **Networking** in the navigation pane and click **Create Service** in the upper right corner. +#. Set parameters. - **Service Name**: Specify a Service name, which can be the same as the workload name. - - .. _cce_01_0014__li36098269511: + - **Access Type**: Select **LoadBalancer**. - **Service Affinity:** + - **Namespace**: Namespace to which the workload belongs. - - Cluster level: The IP addresses and access ports of all nodes in a cluster can access the workload associated with the Service. Service access will cause performance loss due to route redirection, and the source IP address of the client cannot be obtained. - - Node level: Only the IP address and access port of the node where the workload is located can access the workload associated with the Service. Service access will not cause performance loss due to route redirection, and the source IP address of the client can be obtained. + - **Service Affinity**: For details, see :ref:`externalTrafficPolicy (Service Affinity) `. - **ELB Configuration** + - **Cluster level**: The IP addresses and access ports of all nodes in a cluster can be used to access the workload associated with the Service. Service access will cause performance loss due to route redirection, and the source IP address of the client cannot be obtained. + - **Node level**: Only the IP address and access port of the node where the workload is located can access the workload associated with the Service. Service access will not cause performance loss due to route redirection, and the source IP address of the client can be obtained. - - **Elastic Load Balancer**: A load balancer automatically distributes Internet access traffic to multiple nodes where the workload is located. + - **Selector**: Add a label and click **Add**. A Service selects a pod based on the added label. You can also click **Reference Workload Label** to reference the label of an existing workload. In the dialog box that is displayed, select a workload and click **OK**. - - **Shared**: Shared load balancers provide domain name- and URL-based route balancing. Resources are shared among load balancers, and the performance of a load balancer is affected by other load balancers. - - **Dedicated**: Resources are exclusively used by a load balancer, and the performance of a load balancer is not affected by other load balancers. IPv6 is supported. + - **Load Balancer** - - **AZ**: Dedicated load balancers can be deployed across AZs to provide higher reliability. + Select the load balancer to interconnect. Only load balancers in the same VPC as the cluster are supported. If no load balancer is available, click **Create Load Balancer** to create one on the ELB console. - - **Subnet**: subnet where the backend server of the load balancer is located. + You can click **Edit** and configure load balancer parameters in the **Load Balancer** dialog box. - Load balancers occupy different number of subnet IP addresses based on their specifications. Therefore, you are not advised to use the subnet CIDR blocks of other resources (such as clusters and nodes) as the load balancer subnet CIDR block. - - - **Specifications**: Specifications determine the types of listeners that can be added to a load balancer. Select specifications that best fit your needs. - - You can create **public network** or **private network** load balancers. - - - **Public network**: You can select an existing public network load balancer or have the system automatically create a new one. - - **Private network**: You can select an existing private network load balancer or have the system automatically create a new private network load balancer. - - The selected or created load balancer must be in the same VPC as the current cluster, and it must match the load balancer type (private or public network). - - - **Enterprise Project**: Select an enterprise project in which the load balancer is created. - - **Specifications**: This field is displayed only when you select **Public network** and **Automatically created** for **Elastic Load Balancer**. You can click |image1| to modify the name, specifications, billing mode, and bandwidth of the load balancer. - - **Algorithm Type**: Three algorithms are available: weighted round robin, weighted least connections algorithm, or source IP hash. + - **Distribution Policy**: Three algorithms are available: weighted round robin, weighted least connections algorithm, or source IP hash. .. note:: @@ -90,44 +84,29 @@ You can set the Service when creating a workload on the CCE console. An Nginx wo - **Weighted least connections**: In addition to the weight assigned to each server, the number of connections processed by each backend server is also considered. Requests are forwarded to the server with the lowest connections-to-weight ratio. Building on **least connections**, the **weighted least connections** algorithm assigns a weight to each server based on their processing capability. This algorithm is often used for persistent connections, such as database connections. - **Source IP hash**: The source IP address of each request is calculated using the hash algorithm to obtain a unique hash key, and all backend servers are numbered. The generated key allocates the client to a particular server. This enables requests from different clients to be distributed in load balancing mode and ensures that requests from the same client are forwarded to the same server. This algorithm applies to TCP connections without cookies. - - **Sticky Session**: This function is disabled by default. You can select **Based on source IP address**. Listeners ensure session stickiness based on IP addresses. Requests from the same IP address will be forwarded to the same backend server. - - **Health Check**: This function is enabled by default. Enabling it will perform health checks on your load balancer. By default, the Service ports (Node Port and container port of the Service) is used for health check. You can also specify another port for health check. After the port is specified, a Service port (name: **cce-healthz**; protocol: **TCP**) will be added for the Service. + - **Type**: This function is disabled by default. You can select **Source IP address**. Listeners ensure session stickiness based on IP addresses. Requests from the same IP address will be forwarded to the same backend server. + - **Health Check**: This function is disabled by default. The health check is for the load balancer. When TCP is selected during the :ref:`port settings `, you can choose either TCP or HTTP. When UDP is selected during the :ref:`port settings `, only UDP is supported.. By default, the service port (Node Port and container port of the Service) is used for health check. You can also specify another port for health check. After the port is specified, a service port named **cce-healthz** will be added for the Service. - - **Port Settings** + - .. _cce_10_0014__li388800117144: + + **Port Settings** - **Protocol**: protocol used by the Service. - - **Container Port**: port defined in the container image and on which the workload listens. The Nginx application listens on port 80. - - **Access Port**: port mapped to the container port at the load balancer's IP address. The workload can be accessed at <*Load balancer's IP address*>:<*Access port*>. The port number range is 1-65535. + - **Service Port**: port used by the Service. The port number ranges from 1 to 65535. + - **Container Port**: port on which the workload listens. For example, Nginx uses port 80 by default. -#. After the configuration is complete, click **OK**. + - **Annotation**: The LoadBalancer Service has some advanced CCE functions, which are implemented by annotations. For details, see :ref:`Service Annotations `. When you use kubectl to create a container, annotations will be used. For details, see :ref:`Using kubectl to Create a Service (Using an Existing Load Balancer) ` and :ref:`Using kubectl to Create a Service (Automatically Creating a Load Balancer) `. -#. On the workload creation page, click **Next: Configure Advanced Settings**. On the page displayed, click **Create**. +#. Click **OK**. -#. After the workload is successfully created, choose **Workloads** > **Deployments** or **Workloads** > **StatefulSets** on the CCE console. Click the name of the workload to view its details. On the workload details page, click the **Services** tab and obtain the access address. - -#. Click the access address. - -Adding a Service After Creating a Workload ------------------------------------------- - -You can set the Service after creating a workload. This has no impact on the workload status and takes effect immediately. The procedure is as follows: - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Network**. - -#. On the **Services** tab page, click **Create Service**. - - The parameters are the same as those in :ref:`Adding a Service When Creating a Workload `. - -#. Click **Create**. - -.. _cce_01_0014__section1984211714368: +.. _cce_10_0014__section1984211714368: Using kubectl to Create a Service (Using an Existing Load Balancer) ------------------------------------------------------------------- You can set the access type when creating a workload using kubectl. This section uses an Nginx workload as an example to describe how to add a LoadBalancer Service using kubectl. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create and edit the **nginx-deployment.yaml** and **nginx-elb-svc.yaml** files. @@ -164,7 +143,7 @@ You can set the access type when creating a workload using kubectl. This section Before enabling sticky session, ensure that the following conditions are met: - The workload protocol is TCP. - - Anti-affinity has been configured between pods of the workload. That is, all pods of the workload are deployed on different nodes. For details, see :ref:`Workload-Node Anti-Affinity `. + - Anti-affinity has been configured between pods of the workload. That is, all pods of the workload are deployed on different nodes. For details, see :ref:`Scheduling Policy (Affinity/Anti-affinity) `. .. code-block:: @@ -172,78 +151,78 @@ You can set the access type when creating a workload using kubectl. This section kind: Service metadata: annotations: - kubernetes.io/elb.id: 3c7caa5a-a641-4bff-801a-feace27424b6 # Load balancer ID. Replace it with the actual value. - kubernetes.io/elb.class: performance # Load balancer type + kubernetes.io/elb.id: 5083f225-9bf8-48fa-9c8b-67bd9693c4c0 # ELB ID. Replace it with the actual value. + kubernetes.io/elb.class: union # Load balancer type name: nginx spec: ports: - name: service0 - port: 80 + port: 80 # Port for accessing the Service, which is also the listener port on the load balancer. protocol: TCP - targetPort: 80 + targetPort: 80 # Port used by a Service to access the target container. This port is closely related to the applications running in a container. selector: app: nginx type: LoadBalancer .. table:: **Table 1** Key parameters - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Mandatory | Type | Description | - +===========================================+=================+==========================================================+=========================================================================================================================================================================================================+ - | kubernetes.io/elb.class | No | String | Select a proper load balancer type as required. | - | | | | | - | | | | The value can be: | - | | | | | - | | | | - **union**: shared load balancer | - | | | | - **performance**: dedicated load balancer, which can be used only in clusters of v1.17 and later. | - | | | | | - | | | | Default value: **union** | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.session-affinity-mode | No | String | Listeners ensure session stickiness based on IP addresses. Requests from the same IP address will be forwarded to the same backend server. | - | | | | | - | | | | - Disabling sticky session: Do not set this parameter. | - | | | | - Enabling sticky session: Set this parameter to **SOURCE_IP**, indicating that the sticky session is based on the source IP address. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.session-affinity-option | No | :ref:`Table 2 ` Object | This parameter specifies the sticky session timeout. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.id | Yes | String | This parameter indicates the ID of a load balancer. The value can contain 1 to 100 characters. | - | | | | | - | | | | Mandatory when an existing load balancer is to be associated. | - | | | | | - | | | | **Obtaining the load balancer ID:** | - | | | | | - | | | | On the management console, click **Service List**, and choose **Networking** > **Elastic Load Balance**. Click the name of the target load balancer. On the **Summary** tab page, find and copy the ID. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.subnet-id | ``-`` | String | This parameter indicates the ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | - | | | | | - | | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | - | | | | - Optional for clusters later than v1.11.7-r0. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.lb-algorithm | No | String | This parameter indicates the load balancing algorithm of the backend server group. The default value is **ROUND_ROBIN**. | - | | | | | - | | | | Options: | - | | | | | - | | | | - **ROUND_ROBIN**: weighted round robin algorithm | - | | | | - **LEAST_CONNECTIONS**: weighted least connections algorithm | - | | | | - **SOURCE_IP**: source IP hash algorithm | - | | | | | - | | | | When the value is **SOURCE_IP**, the weights of backend servers in the server group are invalid. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.health-check-flag | No | String | Whether to enable the ELB health check. | - | | | | | - | | | | Enabled by default. | - | | | | | - | | | | - Enabling health check: Leave blank this parameter or set it to **on**. | - | | | | - Disabling health check: Set this parameter to **off**. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.health-check-option | No | :ref:`Table 3 ` Object | ELB health check configuration items. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | port | Yes | Integer | Access port that is registered on the load balancer and mapped to the cluster-internal IP address. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | targetPort | Yes | String | Container port set on the CCE console. | - +-------------------------------------------+-----------------+----------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +===========================================+=================+==========================================================+========================================================================================================================================================================================================================================================================================================+ + | kubernetes.io/elb.class | Yes | String | Select a proper load balancer type as required. | + | | | | | + | | | | The value can be: | + | | | | | + | | | | - **union**: shared load balancer | + | | | | - **performance**: dedicated load balancer, which can be used only in clusters of v1.17 and later. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.session-affinity-mode | No | String | Listeners ensure session stickiness based on IP addresses. Requests from the same IP address will be forwarded to the same backend server. | + | | | | | + | | | | - Disabling sticky session: Do not set this parameter. | + | | | | - Enabling sticky session: Set this parameter to **SOURCE_IP**, indicating that the sticky session is based on the source IP address. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.session-affinity-option | No | :ref:`Table 2 ` Object | This parameter specifies the sticky session timeout. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.id | Yes | String | This parameter indicates the ID of a load balancer. The value can contain 1 to 100 characters. | + | | | | | + | | | | Mandatory when an existing load balancer is to be associated. | + | | | | | + | | | | **Obtaining the load balancer ID:** | + | | | | | + | | | | On the management console, click **Service List**, and choose **Networking** > **Elastic Load Balance**. Click the name of the target load balancer. On the **Summary** tab page, find and copy the ID. | + | | | | | + | | | | .. note:: | + | | | | | + | | | | The system preferentially interconnects with the load balancer based on the **kubernetes.io/elb.id** field. If this field is not specified, the **spec.loadBalancerIP** field is used (optional and available only in 1.23 and earlier versions). | + | | | | | + | | | | Do not use the **spec.loadBalancerIP** field to connect to the load balancer. This field will be discarded by Kubernetes. For details, see `Deprecation `__. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.subnet-id | ``-`` | String | This parameter indicates the ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | + | | | | | + | | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | + | | | | - Optional for clusters later than v1.11.7-r0. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.lb-algorithm | No | String | This parameter indicates the load balancing algorithm of the backend server group. The default value is **ROUND_ROBIN**. | + | | | | | + | | | | Options: | + | | | | | + | | | | - **ROUND_ROBIN**: weighted round robin algorithm | + | | | | - **LEAST_CONNECTIONS**: weighted least connections algorithm | + | | | | - **SOURCE_IP**: source IP hash algorithm | + | | | | | + | | | | When the value is **SOURCE_IP**, the weights of backend servers in the server group are invalid. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.health-check-flag | No | String | Whether to enable the ELB health check. | + | | | | | + | | | | - Enabling health check: Leave blank this parameter or set it to **on**. | + | | | | - Disabling health check: Set this parameter to **off**. | + | | | | | + | | | | If this parameter is enabled, the :ref:`kubernetes.io/elb.health-check-option ` field must also be specified at the same time. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kubernetes.io/elb.health-check-option | No | :ref:`Table 3 ` Object | ELB health check configuration items. | + +-------------------------------------------+-----------------+----------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - .. _cce_01_0014__table43592047133910: + .. _cce_10_0014__table43592047133910: .. table:: **Table 2** Data structure of the elb.session-affinity-option field @@ -255,7 +234,7 @@ You can set the access type when creating a workload using kubectl. This section | | | | Value range: 1 to 60. Default value: **60** | +---------------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------------+ - .. _cce_01_0014__table236017471397: + .. _cce_10_0014__table236017471397: .. table:: **Table 3** Data structure description of the **elb.health-check-option** field @@ -278,13 +257,13 @@ You can set the access type when creating a workload using kubectl. This section | | | | | | | | | Default value: protocol of the associated Service | | | | | | - | | | | Value options: TCP, UDP_CONNECT, or HTTP | + | | | | Value options: TCP, UDP, or HTTP | +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ | path | No | String | Health check URL. This parameter needs to be configured when the protocol is HTTP. | | | | | | | | | | Default value: **/** | | | | | | - | | | | The value contains 1 to 10,000 characters. | + | | | | The value can contain 1 to 10,000 characters. | +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ #. Create a workload. @@ -295,7 +274,7 @@ You can set the access type when creating a workload using kubectl. This section .. code-block:: - deployment "nginx" created + deployment/nginx created **kubectl get pod** @@ -304,7 +283,7 @@ You can set the access type when creating a workload using kubectl. This section .. code-block:: NAME READY STATUS RESTARTS AGE - nginx-2601814895-c1xwh 1/1 Running 0 6s + nginx-2601814895-c1xhw 1/1 Running 0 6s #. Create a Service. @@ -314,7 +293,7 @@ You can set the access type when creating a workload using kubectl. This section .. code-block:: - service "nginx" created + service/nginx created **kubectl get svc** @@ -331,17 +310,19 @@ You can set the access type when creating a workload using kubectl. This section The Nginx is accessible. - .. figure:: /_static/images/en-us_image_0276664171.png - :alt: **Figure 2** Accessing Nginx through the LoadBalancer Service + .. figure:: /_static/images/en-us_image_0000001243981181.png + :alt: **Figure 3** Accessing Nginx through the LoadBalancer Service - **Figure 2** Accessing Nginx through the LoadBalancer Service + **Figure 3** Accessing Nginx through the LoadBalancer Service + +.. _cce_10_0014__section12168131904611: Using kubectl to Create a Service (Automatically Creating a Load Balancer) -------------------------------------------------------------------------- You can add a Service when creating a workload using kubectl. This section uses an Nginx workload as an example to describe how to add a LoadBalancer Service using kubectl. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create and edit the **nginx-deployment.yaml** and **nginx-elb-svc.yaml** files. @@ -378,7 +359,7 @@ You can add a Service when creating a workload using kubectl. This section uses Before enabling sticky session, ensure that the following conditions are met: - The workload protocol is TCP. - - Anti-affinity has been configured between pods of the workload. That is, all pods of the workload are deployed on different nodes. For details, see :ref:`Workload-Node Anti-Affinity `. + - Anti-affinity has been configured between pods of the workload. That is, all pods of the workload are deployed on different nodes. For details, see :ref:`Scheduling Policy (Affinity/Anti-affinity) `. Example of a Service using a shared, public network load balancer: @@ -393,11 +374,10 @@ You can add a Service when creating a workload using kubectl. This section uses '{ "type": "public", "bandwidth_name": "cce-bandwidth-1551163379627", - "bandwidth_chargemode":"traffic", + "bandwidth_chargemode": "bandwidth", "bandwidth_size": 5, "bandwidth_sharetype": "PER", - "eip_type": "5_bgp", - "name": "james" + "eip_type": "5_bgp" }' labels: app: nginx @@ -412,7 +392,7 @@ You can add a Service when creating a workload using kubectl. This section uses app: nginx type: LoadBalancer - Example of a Service using a dedicated, public network load balancer: + Example Service using a public network dedicated load balancer (for clusters of v1.17 and later only): .. code-block:: @@ -429,12 +409,12 @@ You can add a Service when creating a workload using kubectl. This section uses '{ "type": "public", "bandwidth_name": "cce-bandwidth-1626694478577", - "bandwidth_chargemode": "traffic", + "bandwidth_chargemode": "bandwidth", "bandwidth_size": 5, "bandwidth_sharetype": "PER", "eip_type": "5_bgp", "available_zone": [ - "eu-de-01" + "" ], "l4_flavor_name": "L4_flavor.elb.s1.small" }' @@ -449,46 +429,32 @@ You can add a Service when creating a workload using kubectl. This section uses protocol: TCP type: LoadBalancer - .. _cce_01_0014__table133089105019: - .. table:: **Table 4** Key parameters +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Mandatory | Type | Description | +===========================================+=================+===============================================================+=======================================================================================================================================================================================================================================================================================+ - | kubernetes.io/elb.class | No | String | Select a proper load balancer type as required. | + | kubernetes.io/elb.class | Yes | String | Select a proper load balancer type as required. | | | | | | | | | | The value can be: | | | | | | | | | | - **union**: shared load balancer | | | | | - **performance**: dedicated load balancer, which can be used only in clusters of v1.17 and later. | - | | | | | - | | | | Default value: **union** | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | kubernetes.io/elb.subnet-id | ``-`` | String | This parameter indicates the ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | | | | | | | | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | | | | | - Optional for clusters later than v1.11.7-r0. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.enterpriseID | No | String | **Clusters of v1.15 and later versions support this field. In clusters earlier than v1.15, load balancers are created in the default project by default.** | - | | | | | - | | | | This parameter indicates the ID of the enterprise project in which the ELB load balancer will be created. | - | | | | | - | | | | If this parameter is not specified or is set to **0**, resources will be bound to the default enterprise project. | - | | | | | - | | | | **How to obtain**: | - | | | | | - | | | | Log in to the management console and choose **Enterprise** > **Project Management** on the top menu bar. In the list displayed, click the name of the target enterprise project, and copy the ID on the enterprise project details page. | + | kubernetes.io/elb.session-affinity-option | No | :ref:`Table 2 ` Object | Sticky session timeout. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.session-affinity-option | No | :ref:`Table 2 ` Object | Sticky session timeout. | - +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.autocreate | Yes | :ref:`elb.autocreate ` object | Whether to automatically create a load balancer associated with the Service. | + | kubernetes.io/elb.autocreate | Yes | :ref:`elb.autocreate ` object | Whether to automatically create a load balancer associated with the Service. | | | | | | | | | | **Example:** | | | | | | | | | | - Automatically created public network load balancer: | | | | | | - | | | | {"type":"public","bandwidth_name":"cce-bandwidth-1551163379627","bandwidth_chargemode":"traffic","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"} | + | | | | {"type":"public","bandwidth_name":"cce-bandwidth-1551163379627","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"} | | | | | | | | | | - Automatically created private network load balancer: | | | | | | @@ -506,92 +472,88 @@ You can add a Service when creating a workload using kubectl. This section uses +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | kubernetes.io/elb.health-check-flag | No | String | Whether to enable the ELB health check. | | | | | | - | | | | Disabled by default. | - | | | | | | | | | - Enabling health check: Leave blank this parameter or set it to **on**. | | | | | - Disabling health check: Set this parameter to **off**. | + | | | | | + | | | | If this parameter is enabled, the :ref:`kubernetes.io/elb.health-check-option ` field must also be specified at the same time. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.health-check-option | No | :ref:`Table 3 ` Object | ELB health check configuration items. | + | kubernetes.io/elb.health-check-option | No | :ref:`Table 3 ` Object | ELB health check configuration items. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | kubernetes.io/elb.session-affinity-mode | No | String | Listeners ensure session stickiness based on IP addresses. Requests from the same IP address will be forwarded to the same backend server. | | | | | | | | | | - Disabling sticky session: Do not set this parameter. | | | | | - Enabling sticky session: Set this parameter to **SOURCE_IP**, indicating that the sticky session is based on the source IP address. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/elb.session-affinity-option | No | :ref:`Table 2 ` Object | Sticky session timeout. | + | kubernetes.io/elb.session-affinity-option | No | :ref:`Table 2 ` Object | Sticky session timeout. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | kubernetes.io/hws-hostNetwork | No | String | This parameter indicates whether the workload Services use the host network. Setting this parameter to **true** will enable the load balancer to forward requests to the host network. | + | kubernetes.io/hws-hostNetwork | No | String | This parameter indicates whether the workload Services use the host network. Setting this parameter to **true** will enable the ELB load balancer to forward requests to the host network. | | | | | | | | | | The host network is not used by default. The value can be **true** or **false**. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | externalTrafficPolicy | No | String | If sticky session is enabled, add this parameter so that requests are transferred to a fixed node. If a LoadBalancer Service with this parameter set to **Local** is created, a client can access the target backend only if the client is installed on the same node as the backend. | +-------------------------------------------+-----------------+---------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - .. _cce_01_0014__table939522754617: + .. _cce_10_0014__table939522754617: .. table:: **Table 5** Data structure of the elb.autocreate field - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Mandatory | Type | Description | - +======================+=======================================+==================+============================================================================================================================================================================================================================================+ - | name | No | String | Name of the automatically created load balancer. | - | | | | | - | | | | Value range: a string of 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | - | | | | | - | | | | Default name: **cce-lb+service.UID** | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | type | No | String | Network type of the load balancer. | - | | | | | - | | | | - **public**: public network load balancer | - | | | | - **inner**: private network load balancer | - | | | | | - | | | | Default value: **inner** | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_name | Yes for public network load balancers | String | Bandwidth name. The default value is **cce-bandwidth-*****\***. | - | | | | | - | | | | Value range: a string of 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_chargemode | No | String | Bandwidth billing mode. | - | | | | | - | | | | - **traffic**: billed by traffic | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_size | Yes for public network load balancers | Integer | Bandwidth size. The default value is 1 to 2000 Mbit/s. Set this parameter based on the bandwidth range allowed in your region. | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | bandwidth_sharetype | Yes for public network load balancers | String | Bandwidth sharing mode. | - | | | | | - | | | | - **PER**: dedicated bandwidth | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | eip_type | Yes for public network load balancers | String | EIP type, which may vary depending on sites. For details, see the type parameter specified when `creating an EIP `__. | - | | | | | - | | | | - **5_bgp**: dynamic BGP | - | | | | - **5_gray**: dedicated load balancer | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | available_zone | Yes | Array of strings | AZ where the load balancer is located. | - | | | | | - | | | | This parameter is available only for dedicated load balancers. | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | l4_flavor_name | Yes | String | Flavor name of the layer-4 load balancer. | - | | | | | - | | | | This parameter is available only for dedicated load balancers. | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | l7_flavor_name | No | String | Flavor name of the layer-7 load balancer. | - | | | | | - | | | | This parameter is available only for dedicated load balancers. | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | elb_virsubnet_ids | No | Array of strings | Subnet where the backend server of the load balancer is located. If this parameter is left blank, the default cluster subnet is used. | - | | | | | - | | | | Load balancers occupy different number of subnet IP addresses based on their specifications. Therefore, you are not advised to use the subnet CIDR blocks of other resources (such as clusters and nodes) as the load balancer CIDR block. | - | | | | | - | | | | This parameter is available only for dedicated load balancers. | - | | | | | - | | | | Example: | - | | | | | - | | | | .. code-block:: | - | | | | | - | | | | "elb_virsubnet_ids": [ | - | | | | "14567f27-8ae4-42b8-ae47-9f847a4690dd" | - | | | | ] | - +----------------------+---------------------------------------+------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +======================+=======================================+==================+==================================================================================================================================================================================================================================================================================================================================================================================+ + | name | No | String | Name of the load balancer that is automatically created. | + | | | | | + | | | | Value range: 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | + | | | | | + | | | | Default: **cce-lb+service.UID** | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | type | No | String | Network type of the load balancer. | + | | | | | + | | | | - **public**: public network load balancer | + | | | | - **inner**: private network load balancer | + | | | | | + | | | | Default: **inner** | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_name | Yes for public network load balancers | String | Bandwidth name. The default value is **cce-bandwidth-*****\***. | + | | | | | + | | | | Value range: 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_chargemode | No | String | Bandwidth mode. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_size | Yes for public network load balancers | Integer | Bandwidth size. The default value is 1 to 2000 Mbit/s. Set this parameter based on the bandwidth range allowed in your region. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_sharetype | Yes for public network load balancers | String | Bandwidth sharing mode. | + | | | | | + | | | | - **PER**: dedicated bandwidth | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | eip_type | Yes for public network load balancers | String | EIP type. | + | | | | | + | | | | - **5_bgp**: dynamic BGP | + | | | | - **5_sbgp**: static BGP | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | available_zone | Yes | Array of strings | AZ where the load balancer is located. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | l4_flavor_name | Yes | String | Flavor name of the layer-4 load balancer. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | l7_flavor_name | No | String | Flavor name of the layer-7 load balancer. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | elb_virsubnet_ids | No | Array of strings | Subnet where the backend server of the load balancer is located. If this parameter is left blank, the default cluster subnet is used. Load balancers occupy different number of subnet IP addresses based on their specifications. Therefore, you are not advised to use the subnet CIDR blocks of other resources (such as clusters and nodes) as the load balancer CIDR block. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + | | | | | + | | | | Example: | + | | | | | + | | | | .. code-block:: | + | | | | | + | | | | "elb_virsubnet_ids": [ | + | | | | "14567f27-8ae4-42b8-ae47-9f847a4690dd" | + | | | | ] | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Create a workload. @@ -601,7 +563,7 @@ You can add a Service when creating a workload using kubectl. This section uses .. code-block:: - deployment "nginx" created + deployment/nginx created **kubectl get po** @@ -610,7 +572,7 @@ You can add a Service when creating a workload using kubectl. This section uses .. code-block:: NAME READY STATUS RESTARTS AGE - nginx-2601814895-c1xwh 1/1 Running 0 6s + nginx-2601814895-c1xhw 1/1 Running 0 6s #. Create a Service. @@ -620,7 +582,7 @@ You can add a Service when creating a workload using kubectl. This section uses .. code-block:: - service "nginx" created + service/nginx created **kubectl get svc** @@ -637,19 +599,56 @@ You can add a Service when creating a workload using kubectl. This section uses The Nginx is accessible. - .. figure:: /_static/images/en-us_image_0000001093275701.png - :alt: **Figure 3** Accessing Nginx through the LoadBalancer Service + .. figure:: /_static/images/en-us_image_0000001199021334.png + :alt: **Figure 4** Accessing Nginx through the LoadBalancer Service - **Figure 3** Accessing Nginx through the LoadBalancer Service + **Figure 4** Accessing Nginx through the LoadBalancer Service -.. _cce_01_0014__section52631714117: +ELB Forwarding +-------------- + +After a Service of the LoadBalancer type is created, you can view the listener forwarding rules of the load balancer on the ELB console. + +You can find that a listener is created for the load balancer. Its backend server is the node where the pod is located, and the backend server port is the NodePort (node port) of the Service. When traffic passes through ELB, it is forwarded to *IP address of the node where the pod is located:Node port*. That is, the Service is accessed and then the pod is accessed, which is the same as that described in :ref:`Scenario `. + +In the passthrough networking scenario (CCE Turbo + dedicated load balancer), after a LoadBalancer Service is created, you can view the listener forwarding rules of the load balancer on the ELB console. + +You can see that a listener is created for the load balancer. The backend server address is the IP address of the pod, and the service port is the container port. This is because the pod uses an ENI or sub-ENI. When traffic passes through the load balancer, it directly forwards the traffic to the pod. This is the same as that described in :ref:`Scenario `. + +.. _cce_10_0014__section52631714117: Why a Cluster Fails to Access Services by Using the ELB Address --------------------------------------------------------------- -If the service affinity of a LoadBalancer Service is set to the node level, that is, the value of **externalTrafficPolicy** is **Local**, the ELB address may fail to be accessed from the cluster (specifically, nodes or containers). +If the service affinity of a LoadBalancer Service is set to the node level, that is, the value of **externalTrafficPolicy** is **Local**, the ELB address may fail to be accessed from the cluster (specifically, nodes or containers). Information similar to the following is displayed: -This is because when the LoadBalancer Service is created, kube-proxy adds the ELB access address (external-ip) to iptables or IPVS. When the ELB address is accessed from the cluster, the ELB load balancer is not used. Instead, kube-proxy directly forwards the access request. The case depends on which container network model and service forwarding mode you use. +.. code-block:: + + upstream connect error or disconnect/reset before headers. reset reason: connection failure + +This is because when the LoadBalancer Service is created, kube-proxy adds the ELB access address as the external IP to iptables or IPVS. If a client initiates a request to access the ELB address from inside the cluster, the address is considered as the external IP address of the service and is directly forwarded by kube-proxy without passing through the ELB outside the cluster. + +When the value of **externalTrafficPolicy** is **Local**, the situation varies according to the container network model and service forwarding mode. The details are as follows: + ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| Server | Client | Tunnel Network Cluster (IPVS) | VPC Network Cluster (IPVS) | Tunnel Network Cluster (iptables) | VPC Network Cluster (iptables) | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| NodePort Service | Same node | OK. The node where the pod runs is accessible, not any other nodes. | OK. The node where the pod runs is accessible. | OK. The node where the pod runs is accessible. | OK. The node where the pod runs is accessible. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| | Cross-node | OK. The node where the pod runs is accessible, not any other nodes. | OK. The node where the pod runs is accessible. | OK. The node where the pod runs is accessible by visiting the node IP + port, not by any other ways. | OK. The node where the pod runs is accessible by visiting the node IP + port, not by any other ways. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| | Containers on the same node | OK. The node where the pod runs is accessible, not any other nodes. | OK. The node where the pod runs is not accessible. | OK. The node where the pod runs is accessible. | OK. The node where the pod runs is not accessible. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| | Containers across nodes | OK. The node where the pod runs is accessible, not any other nodes. | OK. The node where the pod runs is accessible. | OK. The node where the pod runs is accessible. | OK. The node where the pod runs is accessible. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| LoadBalancer Service using a dedicated load balancer | Same node | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| | Containers on the same node | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| Local Service of the nginx-ingress add-on using a dedicated load balancer | Same node | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ +| | Containers on the same node | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | Accessible for public networks, not private networks. | ++---------------------------------------------------------------------------+-----------------------------+---------------------------------------------------------------------+-------------------------------------------------------+------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------+ The following methods can be used to solve this problem: @@ -664,7 +663,7 @@ The following methods can be used to solve this problem: metadata: annotations: kubernetes.io/elb.class: union - kubernetes.io/elb.autocreate: '{"type":"public","bandwidth_name":"cce-bandwidth","bandwidth_chargemode":"traffic","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}' + kubernetes.io/elb.autocreate: '{"type":"public","bandwidth_name":"cce-bandwidth","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"}' labels: app: nginx name: nginx @@ -678,5 +677,3 @@ The following methods can be used to solve this problem: selector: app: nginx type: LoadBalancer - -.. |image1| image:: /_static/images/en-us_image_0298565473.png diff --git a/umn/source/networking/services/nodeport.rst b/umn/source/networking/services/nodeport.rst index 195c045..f952ae7 100644 --- a/umn/source/networking/services/nodeport.rst +++ b/umn/source/networking/services/nodeport.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0142.html +:original_name: cce_10_0142.html -.. _cce_01_0142: +.. _cce_10_0142: NodePort ======== @@ -11,7 +11,7 @@ Scenario A Service is exposed on each node's IP address at a static port (NodePort). A ClusterIP Service, to which the NodePort Service will route, is automatically created. By requesting :, you can access a NodePort Service from outside the cluster. -.. figure:: /_static/images/en-us_image_0000001163847995.png +.. figure:: /_static/images/en-us_image_0000001199501230.png :alt: **Figure 1** NodePort access **Figure 1** NodePort access @@ -20,93 +20,42 @@ Notes and Constraints --------------------- - By default, a NodePort Service is accessed within a VPC. If you need to use an EIP to access a NodePort Service through public networks, bind an EIP to the node in the cluster in advance. -- After a Service is created, if the :ref:`affinity setting ` is switched from the cluster level to the node level, the connection tracing table will not be cleared. You are advised not to modify the Service affinity setting after the Service is created. If you need to modify it, create a Service again. -- The service port of a NodePort Service created on the CCE console is the same as the configured container port. +- After a Service is created, if the affinity setting is switched from the cluster level to the node level, the connection tracing table will not be cleared. You are advised not to modify the Service affinity setting after the Service is created. If you need to modify it, create a Service again. - CCE Turbo clusters support only cluster-level service affinity. +- In VPC network mode, when container A is published through a NodePort service and the service affinity is set to the node level (that is, **externalTrafficPolicy** is set to **local**), container B deployed on the same node cannot access container A through the node IP address and NodePort service. +- When a NodePort service is created in a cluster of v1.21.7 or later, the port on the node is not displayed using **netstat** by default. If the cluster forwarding mode is **iptables**, run the **iptables -t nat -L** command to view the port. If the cluster forwarding mode is **ipvs**, run the **ipvsadm -nL** command to view the port. -Adding a Service When Creating a Workload ------------------------------------------ +Creating a NodePort Service +--------------------------- -You can set the access type when creating a workload on the CCE console. An Nginx workload is used as an example. - -#. In the **Set Application Access** step of :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a DaemonSet `, click **Add Service** and set the following parameters: - - - **Access Type**: Select **NodePort**. - - .. note:: - - If you want to use an EIP to access a NodePort Service through public networks, bind an EIP to the node in the cluster in advance. +#. Log in to the CCE console and click the cluster name to access the cluster. +#. Choose **Networking** in the navigation pane and click **Create Service** in the upper right corner. +#. Set intra-cluster access parameters. - **Service Name**: Specify a Service name, which can be the same as the workload name. + - **Service Type**: Select **NodePort**. + - **Namespace**: Namespace to which the workload belongs. + - **Service Affinity**: For details, see :ref:`externalTrafficPolicy (Service Affinity) `. - - .. _cce_01_0142__li195904442002: - - **Service Affinity**: For details, see :ref:`externalTrafficPolicy (Service Affinity) `. - - - Cluster level: The IP addresses and access ports of all nodes in a cluster can access the workload associated with the Service. Service access will cause performance loss due to route redirection, and the source IP address of the client cannot be obtained. - - Node level: Only the IP address and access port of the node where the workload is located can access the workload associated with the Service. Service access will not cause performance loss due to route redirection, and the source IP address of the client can be obtained. + - **Cluster level**: The IP addresses and access ports of all nodes in a cluster can access the workload associated with the Service. Service access will cause performance loss due to route redirection, and the source IP address of the client cannot be obtained. + - **Node level**: Only the IP address and access port of the node where the workload is located can access the workload associated with the Service. Service access will not cause performance loss due to route redirection, and the source IP address of the client can be obtained. + - **Selector**: Add a label and click **Add**. A Service selects a pod based on the added label. You can also click **Reference Workload Label** to reference the label of an existing workload. In the dialog box that is displayed, select a workload and click **OK**. - **Port Settings** - **Protocol**: protocol used by the Service. - - **Container Port**: port on which the workload in the container image listens. The value ranges from 1 to 65535. - - **Access Port**: node port (with a private IP address) to which the container port will be mapped. You are advised to select **Automatically generated**. + - **Service Port**: port used by the Service. The port number ranges from 1 to 65535. + - **Container Port**: port on which the workload listens. For example, Nginx uses port 80 by default. + - **Node Port**: You are advised to select **Auto**. You can also specify a port. The default port ranges from 30000 to 32767. - - **Automatically generated**: The system automatically assigns a port number. - - **Specified port**: You have to manually specify a fixed node port number in the range of 30000-32767. Ensure that the port is unique in a cluster. - -#. After the configuration is complete, click **OK**. -#. Click **Next: Configure Advanced Settings**. On the page displayed, click **Create**. -#. Click **View Deployment Details** or **View StatefulSet Details**. On the **Services** tab page, obtain the access address, for example, 192.168.0.160:30358. - -Adding a Service After Creating a Workload ------------------------------------------- - -You can set the Service after creating a workload. This has no impact on the workload status and takes effect immediately. The procedure is as follows: - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. On the workload list, click the name of the workload for which you will create a Service. - - .. note:: - - If the Service is associated with an ingress, the ingress is unavailable after the port information of the Service is updated. In this case, you need to delete and recreate the Service. - -#. On the **Services** tab page, click **Add Service**. -#. On the **Create Service** page, select **NodePort** from the **Access Type** drop-down list. - - .. note:: - - If you want to use an EIP to access a NodePort Service through public networks, bind an EIP to the node in the cluster in advance. - -#. Set node access parameters. - - - **Service Name**: Service name, which can be the same as the workload name. - - **Cluster Name**: name of the cluster where the workload runs. The value is inherited from the workload creation page and cannot be changed. - - **Namespace**: namespace where the workload is located. The value is inherited from the workload creation page and cannot be changed. - - **Workload**: workload for which you want to add a Service. The value is inherited from the workload creation page and cannot be changed. - - **Service Affinity** - - - Cluster level: The IP addresses and access ports of all nodes in a cluster can access the workload associated with the Service. Service access will cause performance loss due to route redirection, and the source IP address of the client cannot be obtained. - - Node level: Only the IP address and access port of the node where the workload is located can access the workload associated with the Service. Service access will not cause performance loss due to route redirection, and the source IP address of the client can be obtained. - - - **Port Settings** - - - **Protocol**: protocol used by the Service. - - **Container Port**: port on which the workload in the container image listens. The Nginx workload listens on port 80. - - **Access Port**: node port (with a private IP address) to which the container port will be mapped. You are advised to select **Automatically generated**. - - - **Automatically generated**: The system automatically assigns a port number. - - **Specified port**: You have to manually specify a fixed node port number in the range of 30000-32767. Ensure that the port is unique in a cluster. - -#. Click **Create**. A NodePort Service will be added for the workload. - -.. _cce_01_0142__section7114174773118: +#. Click **OK**. Using kubectl ------------- You can run kubectl commands to set the access type. This section uses a Nginx workload as an example to describe how to set a NodePort Service using kubectl. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create and edit the **nginx-deployment.yaml** and **nginx-nodeport-svc.yaml** files. @@ -210,7 +159,7 @@ You can run kubectl commands to set the access type. This section uses a Nginx w 10.100.0.136 Ready 152m 10.100.0.136 CentOS Linux 7 (Core) 3.10.0-1160.25.1.el7.x86_64 docker://18.9.0 10.100.0.5 Ready 152m 10.100.0.5 CentOS Linux 7 (Core) 3.10.0-1160.25.1.el7.x86_64 docker://18.9.0 # kubectl run -i --tty --image nginx:alpine test --rm /bin/sh - If you don't see a command prompt, try pressing enter. + If you do not see a command prompt, try pressing Enter. / # curl 10.100.0.136:30000 @@ -239,7 +188,7 @@ You can run kubectl commands to set the access type. This section uses a Nginx w / # -.. _cce_01_0142__section18134208069: +.. _cce_10_0142__section18134208069: externalTrafficPolicy (Service Affinity) ---------------------------------------- diff --git a/umn/source/networking/services/service_annotations.rst b/umn/source/networking/services/service_annotations.rst new file mode 100644 index 0000000..b194ce2 --- /dev/null +++ b/umn/source/networking/services/service_annotations.rst @@ -0,0 +1,186 @@ +:original_name: cce_10_0385.html + +.. _cce_10_0385: + +Service Annotations +=================== + +CCE allows you to add annotations to a YAML file to realize some advanced Service functions. The following table describes the annotations you can add. + +The annotations of a Service are the parameters that need to be specified for connecting to a load balancer. For details about how to use the annotations, see :ref:`Using kubectl to Create a Service (Automatically Creating a Load Balancer) `. + +.. table:: **Table 1** Service annotations + + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | Parameter | Type | Description | Default Value on the Console | Supported Cluster Version | + +===========================================+====================================================+=========================================================================================================================================================================================================+==============================+================================================+ + | kubernetes.io/elb.class | String | Select a proper load balancer type. | performance | v1.9 or later | + | | | | | | + | | | The value can be: | | | + | | | | | | + | | | - **union**: shared load balancer | | | + | | | - **performance**: dedicated load balancer, which can be used only in clusters of v1.17 and later. | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.id | String | ID of a load balancer. The value can contain 1 to 100 characters. | None | v1.9 or later | + | | | | | | + | | | Mandatory when an existing load balancer is to be associated. | | | + | | | | | | + | | | **How to obtain**: | | | + | | | | | | + | | | On the management console, click **Service List**, and choose **Networking** > **Elastic Load Balance**. Click the name of the target load balancer. On the **Summary** tab page, find and copy the ID. | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.subnet-id | String | ID of the subnet where the cluster is located. The value can contain 1 to 100 characters. | None | Mandatory for versions earlier than v1.11.7-r0 | + | | | | | | + | | | - Mandatory when a cluster of v1.11.7-r0 or earlier is to be automatically created. | | Discarded in versions later than v1.11.7-r0 | + | | | - Optional for clusters later than v1.11.7-r0. | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.autocreate | :ref:`Table 2 ` | Whether to automatically create a load balancer associated with the Service. | None | v1.9 or later | + | | | | | | + | | | **Example:** | | | + | | | | | | + | | | - If a public network load balancer will be automatically created, set this parameter to the following value: | | | + | | | | | | + | | | {"type":"public","bandwidth_name":"cce-bandwidth-1551163379627","bandwidth_chargemode":"bandwidth","bandwidth_size":5,"bandwidth_sharetype":"PER","eip_type":"5_bgp","name":"james"} | | | + | | | | | | + | | | - If a private network load balancer will be automatically created, set this parameter to the following value: | | | + | | | | | | + | | | {"type":"inner","name":"A-location-d-test"} | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.lb-algorithm | String | Specifies the load balancing algorithm of the backend server group. | ROUND_ROBIN | v1.9 or later | + | | | | | | + | | | Value: | | | + | | | | | | + | | | - **ROUND_ROBIN**: weighted round robin algorithm | | | + | | | - **LEAST_CONNECTIONS**: weighted least connections algorithm | | | + | | | - **SOURCE_IP**: source IP hash algorithm | | | + | | | | | | + | | | When the value is **SOURCE_IP**, the weights of backend servers in the server group are invalid. | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.health-check-flag | String | Whether to enable the ELB health check. | off | v1.9 or later | + | | | | | | + | | | - Enabling health check: Leave blank this parameter or set it to **on**. | | | + | | | - Disabling health check: Set this parameter to **off**. | | | + | | | | | | + | | | If this parameter is enabled, the :ref:`kubernetes.io/elb.health-check-option ` field must also be specified at the same time. | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.health-check-option | :ref:`Table 3 ` | ELB health check configuration items. | None | v1.9 or later | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.session-affinity-mode | String | Listeners ensure session stickiness based on IP addresses. Requests from the same IP address will be forwarded to the same backend server. | None | v1.9 or later | + | | | | | | + | | | - Disabling sticky session: Do not set this parameter. | | | + | | | - Enabling sticky session: Set this parameter to **SOURCE_IP**, indicating that the sticky session is based on the source IP address. | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/elb.session-affinity-option | :ref:`Table 4 ` | Sticky session timeout. | None | v1.9 or later | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + | kubernetes.io/hws-hostNetwork | Boolean | Whether the workload Services use the host network. Setting this parameter to **true** will enable the load balancer to forward requests to the host network. | None | v1.9 or later | + | | | | | | + | | | The value is **true** or **false**. | | | + | | | | | | + | | | The default value is **false**, indicating that the host network is not used. | | | + +-------------------------------------------+----------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------+------------------------------------------------+ + +.. _cce_10_0385__table148341447193017: + +.. table:: **Table 2** Data structure of the elb.autocreate field + + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +======================+=======================================+==================+==================================================================================================================================================================================================================================================================================================================================================================================+ + | name | No | String | Name of the load balancer that is automatically created. | + | | | | | + | | | | Value range: 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | + | | | | | + | | | | Default: **cce-lb+service.UID** | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | type | No | String | Network type of the load balancer. | + | | | | | + | | | | - **public**: public network load balancer | + | | | | - **inner**: private network load balancer | + | | | | | + | | | | Default: **inner** | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_name | Yes for public network load balancers | String | Bandwidth name. The default value is **cce-bandwidth-*****\***. | + | | | | | + | | | | Value range: 1 to 64 characters, including lowercase letters, digits, and underscores (_). The value must start with a lowercase letter and end with a lowercase letter or digit. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_chargemode | No | String | Bandwidth mode. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_size | Yes for public network load balancers | Integer | Bandwidth size. The default value is 1 to 2000 Mbit/s. Set this parameter based on the bandwidth range allowed in your region. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | bandwidth_sharetype | Yes for public network load balancers | String | Bandwidth sharing mode. | + | | | | | + | | | | - **PER**: dedicated bandwidth | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | eip_type | Yes for public network load balancers | String | EIP type. | + | | | | | + | | | | - **5_bgp**: dynamic BGP | + | | | | - **5_sbgp**: static BGP | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | available_zone | Yes | Array of strings | AZ where the load balancer is located. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | l4_flavor_name | Yes | String | Flavor name of the layer-4 load balancer. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | l7_flavor_name | No | String | Flavor name of the layer-7 load balancer. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | elb_virsubnet_ids | No | Array of strings | Subnet where the backend server of the load balancer is located. If this parameter is left blank, the default cluster subnet is used. Load balancers occupy different number of subnet IP addresses based on their specifications. Therefore, you are not advised to use the subnet CIDR blocks of other resources (such as clusters and nodes) as the load balancer CIDR block. | + | | | | | + | | | | This parameter is available only for dedicated load balancers. | + | | | | | + | | | | Example: | + | | | | | + | | | | .. code-block:: | + | | | | | + | | | | "elb_virsubnet_ids": [ | + | | | | "14567f27-8ae4-42b8-ae47-9f847a4690dd" | + | | | | ] | + +----------------------+---------------------------------------+------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +.. _cce_10_0385__table19192143412319: + +.. table:: **Table 3** Data structure description of the **elb.health-check-option** field + + +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +=================+=================+=================+====================================================================================+ + | delay | No | String | Initial waiting time (in seconds) for starting the health check. | + | | | | | + | | | | Value range: 1 to 50. Default value: **5** | + +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ + | timeout | No | String | Health check timeout, in seconds. | + | | | | | + | | | | Value range: 1 to 50. Default value: **10** | + +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ + | max_retries | No | String | Maximum number of health check retries. | + | | | | | + | | | | Value range: 1 to 10. Default value: **3** | + +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ + | protocol | No | String | Health check protocol. | + | | | | | + | | | | Default value: protocol of the associated Service | + | | | | | + | | | | Value options: TCP, UDP, or HTTP | + +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ + | path | No | String | Health check URL. This parameter needs to be configured when the protocol is HTTP. | + | | | | | + | | | | Default value: **/** | + | | | | | + | | | | The value can contain 1 to 10,000 characters. | + +-----------------+-----------------+-----------------+------------------------------------------------------------------------------------+ + +.. _cce_10_0385__table3340195463412: + +.. table:: **Table 4** Data structure of the elb.session-affinity-option field + + +---------------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Mandatory | Type | Description | + +=====================+=================+=================+==============================================================================================================================+ + | persistence_timeout | Yes | String | Sticky session timeout, in minutes. This parameter is valid only when **elb.session-affinity-mode** is set to **SOURCE_IP**. | + | | | | | + | | | | Value range: 1 to 60. Default value: **60** | + +---------------------+-----------------+-----------------+------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/networking/services/overview.rst b/umn/source/networking/services/service_overview.rst similarity index 79% rename from umn/source/networking/services/overview.rst rename to umn/source/networking/services/service_overview.rst index 27850a6..8d87e99 100644 --- a/umn/source/networking/services/overview.rst +++ b/umn/source/networking/services/service_overview.rst @@ -1,9 +1,9 @@ -:original_name: cce_01_0249.html +:original_name: cce_10_0249.html -.. _cce_01_0249: +.. _cce_10_0249: -Overview -======== +Service Overview +================ Direct Access to a Pod ---------------------- @@ -14,9 +14,9 @@ After a pod is created, the following problems may occur if you directly access - The IP address of the pod is allocated only after the pod is started. Before the pod is started, the IP address of the pod is unknown. - An application is usually composed of multiple pods that run the same image. Accessing pods one by one is not efficient. -For example, an application uses Deployments to create the frontend and backend. The frontend calls the backend for computing, as shown in :ref:`Figure 1 `. Three pods are running in the backend, which are independent and replaceable. When a backend pod is re-created, the new pod is assigned with a new IP address, of which the frontend pod is unaware. +For example, an application uses Deployments to create the frontend and backend. The frontend calls the backend for computing, as shown in :ref:`Figure 1 `. Three pods are running in the backend, which are independent and replaceable. When a backend pod is re-created, the new pod is assigned with a new IP address, of which the frontend pod is unaware. -.. _cce_01_0249__en-us_topic_0249851121_fig2173165051811: +.. _cce_10_0249__en-us_topic_0249851121_fig2173165051811: .. figure:: /_static/images/en-us_image_0258894622.png :alt: **Figure 1** Inter-pod access @@ -28,9 +28,9 @@ Using Services for Pod Access Kubernetes Services are used to solve the preceding pod access problems. A Service has a fixed IP address. (When a CCE cluster is created, a Service CIDR block is set, which is used to allocate IP addresses to Services.) A Service forwards requests accessing the Service to pods based on labels, and at the same time, perform load balancing for these pods. -In the preceding example, a Service is added for the frontend pod to access the backend pods. In this way, the frontend pod does not need to be aware of the changes on backend pods, as shown in :ref:`Figure 2 `. +In the preceding example, a Service is added for the frontend pod to access the backend pods. In this way, the frontend pod does not need to be aware of the changes on backend pods, as shown in :ref:`Figure 2 `. -.. _cce_01_0249__en-us_topic_0249851121_fig163156154816: +.. _cce_10_0249__en-us_topic_0249851121_fig163156154816: .. figure:: /_static/images/en-us_image_0258889981.png :alt: **Figure 2** Accessing pods through a Service @@ -42,18 +42,14 @@ Service Types Kubernetes allows you to specify a Service of a required type. The values and actions of different types of Services are as follows: -- :ref:`ClusterIP ` +- :ref:`ClusterIP ` A ClusterIP Service allows workloads in the same cluster to use their cluster-internal domain names to access each other. -- :ref:`NodePort ` +- :ref:`NodePort ` A NodePort Service is exposed on each node's IP at a static port. A ClusterIP Service, to which the NodePort Service routes, is automatically created. By requesting <*NodeIP*>:<*NodePort*>, you can access a NodePort Service from outside the cluster. -- :ref:`LoadBalancer ` +- :ref:`LoadBalancer ` A workload can be accessed from public networks through a load balancer, which is more secure and reliable than EIP. - -- :ref:`ENI LoadBalancer ` - - An ENI LoadBalancer Service directs traffic from a load balancer at backend pods, reducing the latency and avoiding performance loss for containerized applications. diff --git a/umn/source/node_pools/creating_a_node_pool.rst b/umn/source/node_pools/creating_a_node_pool.rst index eab7a37..73ede10 100644 --- a/umn/source/node_pools/creating_a_node_pool.rst +++ b/umn/source/node_pools/creating_a_node_pool.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0012.html +:original_name: cce_10_0012.html -.. _cce_01_0012: +.. _cce_10_0012: Creating a Node Pool ==================== @@ -8,389 +8,231 @@ Creating a Node Pool Scenario -------- -This section describes how to create a node pool and perform operations on the node pool. For details about how a node pool works, see :ref:`Node Pool Overview `. +This section describes how to create a node pool and perform operations on the node pool. For details about how a node pool works, see :ref:`Node Pool Overview `. Notes and Constraints --------------------- -- For details about how to add a node pool to a CCE Turbo cluster, see :ref:`Procedure - for CCE Turbo Clusters `. -- The autoscaler add-on needs to be installed for node auto scaling. For details about the add-on installation and parameter configuration, see :ref:`autoscaler `. +- The autoscaler add-on needs to be installed for node auto scaling. For details about the add-on installation and parameter configuration, see :ref:`autoscaler `. Procedure --------- -To create a node pool in a cluster, perform the following steps: - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. - -#. In the upper right corner of the page, click **Create Node Pool**. - -#. Set node pool parameters. - - - **Current Region**: geographic location of the node pool to be created. - - To minimize network latency and resource access time, select the region nearest to your node pool. Cloud resources are region-specific and cannot be used across regions over internal networks. - - - **Name**: name of the new node pool. By default, the name is in the format of *Cluster name*-nodepool-*Random number*. You can also use a custom name. - - - **Node Type**: Currently, only VM nodes are supported. - - - **Nodes**: number of nodes to be created for this node pool. The value cannot exceed the maximum number of nodes that can be managed by the cluster. - - - .. _cce_01_0012__li153215432027: - - **Auto Scaling**: - - - By default, this parameter is disabled. - - - After you enable autoscaler by clicking |image1|, nodes in the node pool will be automatically created or deleted based on cluster loads. - - - **Maximum Nodes** and **Minimum Nodes**: You can set the maximum and minimum number of nodes to ensure that the number of nodes to be scaled is within a proper range. - - - **Priority**: Set this parameter based on service requirements. A larger value indicates a higher priority. For example, if this parameter is set to **1** and **4** respectively for node pools A and B, B has a higher priority than A. If the priorities of multiple node pools are set to the same value, for example, **2**, the node pools are not prioritized and the system performs scaling based on the minimum resource waste principle. - - .. note:: - - CCE selects a node pool for auto scaling based on the following policies: - - a. CCE uses algorithms to determine whether a node pool meets the conditions to allow scheduling of a pod in pending state, including whether the node resources are greater than requested by the pod, and whether the nodeSelect, nodeAffinity, and taints meet the conditions. In addition, the node pools that fail to be scaled (due to insufficient resources or other reasons) and are still in the 15-minute cool-down interval are filtered. - b. If multiple node pools meet the scaling requirements, the system checks the priority of each node pool and selects the node pool with the highest priority for scaling. The value ranges from 0 to 100 and the default priority is 0. The value 100 indicates the highest priority, and the value 0 indicates the lowest priority. - c. If multiple node pools have the same priority or no priority is configured for them, the system selects the node pool that will consume the least resources based on the configured VM specification. - d. If the VM specifications of multiple node pools are the same but the node pools are deployed in different AZs, the system randomly selects a node pool to trigger scaling. - - - .. _cce_01_0012__li16733114525110: - - **Scale-In Cooling Interval**: Set this parameter in the unit of minute or hour. This parameter indicates the interval between the previous scale-out action and the next scale-in action. - - Scale-in cooling intervals can be configured in the node pool settings and the :ref:`autoscaler add-on ` settings. - - **Scale-in cooling interval configured in a node pool** - - This interval indicates the period during which nodes added to the current node pool after a scale-out operation cannot be deleted. This interval takes effect at the node pool level. - - **Scale-in cooling interval configured in the autoscaler add-on** - - The interval after a scale-out indicates the period during which the entire cluster cannot be scaled in after the autoscaler add-on triggers scale-out (due to the unschedulable pods, metrics, and scaling policies). This interval takes effect at the cluster level. - - The interval after a node is deleted indicates the period during which the cluster cannot be scaled in after the autoscaler add-on triggers scale-in. This interval takes effect at the cluster level. - - The interval after a failed scale-in indicates the period during which the cluster cannot be scaled in after the autoscaler add-on triggers scale-in. This interval takes effect at the cluster level. - - .. note:: - - You are advised not to store important data on nodes in a node pool because after auto scaling, data cannot be restored as nodes may be deleted. - - If **Autoscaler** is enabled, install the :ref:`autoscaler add-on ` to use the auto scaling feature. - - - **AZ**: An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. - - Set an AZ based on your requirements. After a node pool is created, **AZ** cannot be modified. Exercise caution when selecting an AZ for the node pool. - - To enhance workload reliability, you are advised to select **Random AZ**, allowing nodes to be randomly and evenly distributed among different AZs. - - .. note:: - - In a CCE Turbo cluster, an AZ is randomly selected from available AZs, and all nodes are created in the selected AZ. - - - **Specifications**: Select node specifications that best fit your business needs. - - - **General-purpose**: provides a balance of computing, memory, and network resources. It is a good choice for many applications, such as web servers, workload development, workload testing, and small-scale databases. - - **Memory-optimized**: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive. - - **GPU-accelerated**: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. **GPU-accelerated nodes can be created only in clusters of v1.11 or later**. GPU-accelerated nodes are available only in certain regions. - - **General computing-plus**: provides stable performance and exclusive resources to enterprise-class workloads with high and stable computing performance. - - **Disk-intensive**: supports :ref:`local disk storage ` and provides high network performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads. - - To ensure node stability, CCE automatically reserves some resources to run necessary system components. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. - - - **OS**: Select an OS for the node to be created. - - .. important:: - - Reinstalling the OS or modifying OS configurations could make the node unavailable. Exercise caution when performing these operations. - - - **VPC**: The value is the same as that of the cluster and cannot be changed. - - **This parameter is displayed only for clusters of v1.13.10-r0 and later.** - - - **Subnet**: A subnet improves network security by providing exclusive network resources that are isolated from other networks. - - You can select any subnet in the cluster VPC. Cluster nodes can belong to different subnets. - - Ensure that the DNS server in the subnet can resolve the OBS domain name. Otherwise, nodes cannot be created. - - **This parameter is displayed only for clusters of v1.13.10-r0 and later.** - - - **System Disk**: Set the system disk space of the worker node. The value ranges from 40GB to 1024 GB. The default value is 40GB. - - By default, system disks support Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) EVS disks. - - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. **This function is available only in certain regions.** - - - **Encryption** is not selected by default. - - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. - - - .. _cce_01_0012__li1931815591054: - - **Data Disk**: Set the data disk space of the worker node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. The EVS disk types provided for the data disk are the same as those for the system disk. - - .. caution:: - - If the data disk is uninstalled or damaged, the Docker service becomes abnormal and the node becomes unavailable. You are advised not to delete the data disk. - - - **LVM**: If this option is selected, CCE data disks are managed by the Logical Volume Manager (LVM). On this condition, you can adjust the disk space allocation for different resources. This option is selected for the first disk by default and cannot be unselected. You can choose to enable or disable LVM for new data disks. - - - This option is selected by default, indicating that LVM management is enabled. - - You can deselect the check box to disable LVM management. - - .. caution:: - - - Disk space of the data disks managed by LVM will be allocated according to the ratio you set. - - When creating a node in a cluster of v1.13.10 or later, if LVM is not selected for a data disk, follow instructions in :ref:`Adding a Second Data Disk to a Node in a CCE Cluster ` to fill in the pre-installation script and format the data disk. Otherwise, the data disk will still be managed by LVM. - - When creating a node in a cluster earlier than v1.13.10, you must format the data disks that are not managed by LVM. Otherwise, either these data disks or the first data disk will be managed by LVM. - - - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. - - **This function is supported only for clusters of v1.13.10 or later in certain regions,** and is not displayed for clusters of v1.13.10 or earlier. - - - **Encryption** is not selected by default. - - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. - - - **Add Data Disk**: Currently, a maximum of two data disks can be attached to a node. After the node is created, you can go to the ECS console to attach more data disks. This function is available only to clusters of certain versions. - - - **Data disk space allocation**: Click |image2| to specify the resource ratio for **Kubernetes Space** and **User Space**. Disk space of the data disks managed by LVM will be allocated according to the ratio you set. This function is available only to clusters of certain versions. - - - **Kubernetes Space**: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs. - - The Docker space cannot be less than 10%, and the space size cannot be less than 60 GB. The kubelet space cannot be less than 10%. - - The Docker space size is determined by your service requirements. For details, see :ref:`Data Disk Space Allocation `. - - - **User Space**: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted. - - .. note:: - - Note that the mount path cannot be **/**, **/home/paas**, **/var/paas**, **/var/lib**, **/var/script**, **/var/log**, **/mnt/paas**, or **/opt/cloud**, and cannot conflict with the system directories (such as **bin**, **lib**, **home**, **root**, **boot**, **dev**, **etc**, **lost+found**, **mnt**, **proc**, **sbin**, **srv**, **tmp**, **var**, **media**, **opt**, **selinux**, **sys**, and **usr**). Otherwise, the system or node installation will fail. - - **If the cluster version is v1.13.10-r0 or later and the node specification is Disk-intensive, the following options are displayed for data disks:** - - - **EVS**: Parameters are the same as those when the node type is not Disk-intensive. For details, see :ref:`Data Disk ` above. - - - **Local disk**: Local disks may break down and do not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks. - - Local disk parameters are as follows: - - - **Disk Mode**: If the node type is **disk-intensive**, the supported disk mode is HDD. - - **Read/Write Mode**: When multiple local disks exist, you can set the read/write mode. The serial and sequential modes are supported. **Sequential** indicates that data is read and written in linear mode. When a disk is used up, the next disk is used. **Serial** indicates that data is read and written in striping mode, allowing multiple local disks to be read and written at the same time. - - **Kubernetes Space**: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs. - - **User Space**: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted. - - .. important:: - - - The ratio of disk space allocated to the Kubernetes space and user space must be equal to 100% in total. You can click |image3| to refresh the data after you have modified the ratio. - - By default, disks run in the direct-lvm mode. If data disks are removed, the loop-lvm mode will be used and this will impair system stability. - - - **Login Mode**: - - - **Key pair**: Select the key pair used to log in to the node. You can select a shared key. - - A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. - - .. important:: - - When creating a node using a key pair, IAM users can select only the key pairs created by their own, regardless of whether these users are in the same group. For example, user B cannot use the key pair created by user A to create a node, and the key pair is not displayed in the drop-down list on the CCE console. - -#. **Advanced ECS Settings** (optional): Click |image4| to show advanced ECS settings. - - - **ECS Group**: An ECS group logically groups ECSs. The ECSs in the same ECS group comply with the same policy associated with the ECS group. - - - **Anti-affinity**: ECSs in an ECS group are deployed on different physical hosts to improve service reliability. - - Select an existing ECS group, or click **Create ECS Group** to create one. After the ECS group is created, click the refresh button. - - - **Resource Tags**: By adding tags to resources, you can classify resources. - - You can create predefined tags in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency. - - CCE will automatically create the "CCE-Dynamic-Provisioning-Node=node id" tag. A maximum of 5 tags can be added. - - - **Agency**: An agency is created by a tenant administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. To authorize an ECS or BMS to call cloud services, select **Cloud service** as the agency type, click **Select**, and then select **ECS BMS**. - - - **Pre-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. The script is usually used to format data disks. - - - **Post-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed after Kubernetes software is installed and will not affect the installation. The script is usually used to modify Docker parameters. - - - **Subnet IP Address**: Select **Automatically assign IP address** (recommended) or **Manually assigning IP addresses**. - -#. **Advanced Kubernetes Settings** (optional): Click |image5| to show advanced Kubernetes settings. - - - **Max Pods**: maximum number of pods that can be created on a node, including the system's default pods. If the cluster uses the **VPC network model**, the maximum value is determined by the number of IP addresses that can be allocated to containers on each node. - - This limit prevents the node from being overloaded by managing too many pods. For details, see :ref:`Maximum Number of Pods That Can Be Created on a Node `. - - - **Taints**: This field is left blank by default. Taints allow nodes to repel a set of pods. You can add a maximum of 10 taints for each node. Each taint contains the following parameters: - - - **Key**: A key must contain 1 to 63 characters starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. - - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). - - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. - - .. important:: - - - If taints are used, you must configure tolerations in the YAML files of pods. Otherwise, scale-up may fail or pods cannot be scheduled onto the added nodes. - - After a node pool is created, you can click **Edit** to modify its configuration. The modification will be synchronized to all nodes in the node pool. - - - **K8S Labels**: Labels are key/value pairs that are attached to objects, such as pods. Labels are used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system. For more information, see `Labels and Selectors `__. - - - **Maximum Data Space per Container**: maximum data space that can be used by a container. The value ranges from 10 GB to 500 GB. If the value of this field is larger than the data disk space allocated to Docker resources, the latter will override the value specified here. Typically, 90% of the data disk space is allocated to Docker resources. This parameter is displayed only for clusters of v1.13.10-r0 and later. - -#. Click **Next: Confirm** to confirm the configured service parameters and specifications. - -#. Click **Submit**. - - It takes about 6 to 10 minutes to create a node pool. You can click **Back to Node Pool List** to perform other operations on the node pool or click **Go to Node Pool Events** to view the node pool details. If the status of the node pool is Normal, the node pool is successfully created. - -.. _cce_01_0012__section953835110714: - -Procedure - for CCE Turbo Clusters ----------------------------------- - #. Log in to the CCE console. -#. Click the cluster name to open its details page, choose **Nodes** on the left, and click the **Node Pool** tab on the right. + +#. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. + #. In the upper right corner of the page, click **Create Node Pool**. -#. Configure computing parameters. - - **AZ**: An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. + **Basic Settings** - Set an AZ based on your requirements. After a node pool is created, **AZ** cannot be modified. Exercise caution when selecting an AZ for the node pool. + .. table:: **Table 1** Basic settings - To enhance workload reliability, you are advised to select **Random AZ**, allowing nodes to be randomly and evenly distributed among different AZs. + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=============================================================================================================================================================================================================================================================================================================================================================================================================================================================+ + | Node Pool Name | Name of a node pool. By default, the name is in the format of *Cluster name*-nodepool-*Random number*. If you do not want to use the default name format, you can customize the name. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Nodes | Number of nodes to be created in this node pool. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Auto Scaling | By default, auto scaling is disabled. | + | | | + | | Install the :ref:`autoscaler add-on ` to enable auto scaling. | + | | | + | | After you enable auto scaling by switching on |image1|, nodes in the node pool will be automatically created or deleted based on cluster loads. | + | | | + | | - **Maximum Nodes** and **Minimum Nodes**: You can set the maximum and minimum number of nodes to ensure that the number of nodes to be scaled is within a proper range. | + | | | + | | - **Priority**: Set this parameter based on service requirements. A larger value indicates a higher priority. For example, if this parameter is set to **1** and **4** respectively for node pools A and B, B has a higher priority than A. If the priorities of multiple node pools are set to the same value, for example, **2**, the node pools are not prioritized and the system performs scaling based on the minimum resource waste principle. | + | | | + | | .. note:: | + | | | + | | CCE selects a node pool for auto scaling based on the following policies: | + | | | + | | a. CCE uses algorithms to determine whether a node pool meets the conditions to allow scheduling of a pod in pending state, including whether the node resources are greater than requested by the pod, and whether the nodeSelect, nodeAffinity, and taints meet the conditions. In addition, the node pools that fail to be scaled (due to insufficient resources or other reasons) and are still in the 15-minute cool-down interval are filtered. | + | | b. If multiple node pools meet the scaling requirements, the system checks the priority of each node pool and selects the node pool with the highest priority for scaling. The value ranges from 0 to 100 and the default priority is 0. The value 100 indicates the highest priority, and the value 0 indicates the lowest priority. | + | | c. If multiple node pools have the same priority or no priority is configured for them, the system selects the node pool that will consume the least resources based on the configured VM specification. | + | | d. If the VM specifications of multiple node pools are the same but the node pools are deployed in different AZs, the system randomly selects a node pool to trigger scaling. | + | | | + | | - **Cooldown Period**: Requied. The unit is minute. This field indicates the period during which the nodes added in the current node pool cannot be scaled in. | + | | | + | | Scale-in cooling intervals can be configured in the node pool settings and the :ref:`autoscaler add-on ` settings. | + | | | + | | **Scale-in cooling interval configured in a node pool** | + | | | + | | This interval indicates the period during which nodes added to the current node pool after a scale-out operation cannot be deleted. This interval takes effect at the node pool level. | + | | | + | | **Scale-in cooling interval configured in the autoscaler add-on** | + | | | + | | The interval after a scale-out indicates the period during which the entire cluster cannot be scaled in after the autoscaler add-on triggers scale-out (due to the unschedulable pods, metrics, and scaling policies). This interval takes effect at the cluster level. | + | | | + | | The interval after a node is deleted indicates the period during which the cluster cannot be scaled in after the autoscaler add-on triggers scale-in. This interval takes effect at the cluster level. | + | | | + | | The interval after a failed scale-in indicates the period during which the cluster cannot be scaled in after the autoscaler add-on triggers scale-in. This interval takes effect at the cluster level. | + | | | + | | .. note:: | + | | | + | | You are advised not to store important data on nodes in a node pool because after auto scaling, data cannot be restored as nodes may be deleted. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - **Container Runtime**: runc or kata. + **Compute Settings** - For details about common containers and secure containers, see :ref:`Secure Containers and Common Containers `. + You can configure the specifications and OS of a cloud server, on which your containerized applications run. - - **Specifications**: Select node specifications that best fit your business needs. + .. table:: **Table 2** Configuration parameters - - **General-purpose**: provides a balance of computing, memory, and network resources. It is a good choice for many applications, such as web servers, workload development, workload testing, and small-scale databases. - - **Memory-optimized**: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive. - - **GPU-accelerated**: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. **GPU-accelerated nodes can be created only in clusters of v1.11 or later**. GPU-accelerated nodes are available only in certain regions. - - **General computing-plus**: provides stable performance and exclusive resources to enterprise-class workloads with high and stable computing performance. - - **Disk-intensive**: supports :ref:`local disk storage ` and provides high network performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads. + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+====================================================================================================================================================================================================================================+ + | AZ | AZ where the node is located. Nodes in a cluster can be created in different AZs for higher reliability. The value cannot be changed after the node is created. | + | | | + | | You are advised to select **Random** to deploy your node in a random AZ based on the selected node flavor. | + | | | + | | An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. To enhance workload availability, create nodes in different AZs. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Node Type | For a CCE cluster, **ECS** and **BMS** are supported. | + | | | + | | CCE Turbo clusters support ECSs of the VM and physical types. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Container Engine | CCE clusters support Docker. Starting from CCE 1.23, containerd is supported. | + | | | + | | For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Specifications | Select node specifications that best fit your business needs. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | OS | Select an OS type. Different types of nodes support different OSs. For details, see :ref:`Supported Node Specifications `. | + | | | + | | **Public image**: Select an OS for the node. | + | | | + | | **Private image**: You can use private images. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Login Mode | - **Key Pair** | + | | | + | | Select the key pair used to log in to the node. You can select a shared key. | + | | | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - To ensure node stability, CCE automatically reserves some resources to run necessary system components. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. + **Storage Settings** - - **OS**: Select an OS for the node to be created. In certain regions, only OSs are displayed and options **Public image** and **Private image** are unavailable. + Configure storage resources on a node for the containers running on it. Set the disk size according to site requirements. - - **Public image**: Select an OS for the node. - - **Private image (OBT)**: If no private image is available, click **Creating a Private Image** to create one. **This function is available only for clusters of v1.15 or later.** + .. table:: **Table 3** Configuration parameters - .. important:: + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+===============================================================================================================================================================================================================================================================================================+ + | System Disk | System disk used by the node OS. The value ranges from 40 GB to 1,024 GB. The default value is 50 GB. | + | | | + | | **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. **This function is available only in certain regions.** | + | | | + | | - **Encryption** is not selected by default. | + | | - After you select **Encryption**, you can select an existing key in the displayed dialog box. If no key is available, click **View Key List** to create a key. After the key is created, click the refresh icon. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Data Disk | Data disk used by the container runtime and kubelet on the node. | + | | | + | | **At least one data disk is required** for the container runtime and kubelet. **The data disk cannot be deleted or uninstalled. Otherwise, the node will be unavailable.** | + | | | + | | Click **Expand** to set the following parameters: | + | | | + | | - **Allocate Disk Space**: Select this option to define the disk space occupied by the container runtime to store the working directories, container image data, and image metadata. For details about how to allocate data disk space, see :ref:`Data Disk Space Allocation `. | + | | - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. **This function is available only in certain regions.** | + | | | + | | - **Encryption** is not selected by default. | + | | - After you select **Encryption**, you can select an existing key in the displayed dialog box. If no key is available, click **View Key List** to create a key. After the key is created, click the refresh icon. | + | | | + | | **Adding Multiple Data Disks** | + | | | + | | A maximum of four data disks can be added. By default, raw disks are created without any processing. You can also click **Expand** and select any of the following options: | + | | | + | | - **Default**: By default, a raw disk is created without any processing. | + | | - **Mount Disk**: The data disk is attached to a specified directory. | + | | | + | | **Local Disk Description** | + | | | + | | If the node flavor is disk-intensive or ultra-high I/O, one data disk can be a local disk. | + | | | + | | Local disks may break down and do not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - Reinstalling the OS or modifying OS configurations could make the node unavailable. Exercise caution when performing these operations. + **Network Settings** - - **Login Mode**: + Configure networking resources to allow node and containerized application access. - - **Key pair**: Select the key pair used to log in to the node. You can select a shared key. + .. table:: **Table 4** Configuration parameters - A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+======================================================================================================================================================================================+ + | Node Subnet | The node subnet selected during cluster creation is used by default. You can choose another subnet instead. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Node IP Address | Random allocation is supported. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Associate Security Group | Security group used by the nodes created in the node pool. A maximum of 5 security groups can be selected. | + | | | + | | When a cluster is created, a node security group named **{Cluster name}-cce-node-{Random ID}** is created and used by default. | + | | | + | | Traffic needs to pass through certain ports in the node security group to ensure node communications. Ensure that you have enabled these ports if you select another security group. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - .. important:: + **Advanced Settings** - When creating a node using a key pair, IAM users can select only the key pairs created by their own, regardless of whether these users are in the same group. For example, user B cannot use the key pair created by user A to create a node, and the key pair is not displayed in the drop-down list on the CCE console. + Configure advanced node capabilities such as labels, taints, and startup command. -#. Configure storage parameters. + .. table:: **Table 5** Advanced configuration parameters - - **System Disk**: Set the system disk space of the worker node. The value ranges from 40GB to 1024 GB. The default value is 50 GB. - - By default, system disks support Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) EVS disks. - - - **Data Disk**: Set the data disk space of the worker node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. The data disk space size is determined by your service requirements. For details, see :ref:`Data Disk Space Allocation `. - - If the cluster version is v1.13.10-r0 or later and the node type is Disk-intensive, data disks can be EVS disks or local disks. - - .. caution:: - - If the data disk is uninstalled or damaged, the Docker service becomes abnormal and the node becomes unavailable. You are advised not to delete the data disk. - - - Data disk space allocation: Click **Expand** and select **Allocate Disk Space** to customize the data disk space usage. - - You can customize the resource proportion for the container runtime and kubelet in the data disk. By default, 90% of the space is allocated to containers, and the remaining space is allocated to the kubelet component. - - You can also define the maximum space that can be occupied by a single container. The default value is 10 GB. - - - Adding data disks: The node must have at least one data disk, and data disks can be added. Click **Add Data Disk**. Click **Expand** to attach the new data disk to the specified directory. - - .. note:: - - Note that the mount path cannot be **/**, **/home/paas**, **/var/paas**, **/var/lib**, **/var/script**, **/var/log**, **/mnt/paas**, or **/opt/cloud**, and cannot conflict with the system directories (such as **bin**, **lib**, **home**, **root**, **boot**, **dev**, **etc**, **lost+found**, **mnt**, **proc**, **sbin**, **srv**, **tmp**, **var**, **media**, **opt**, **selinux**, **sys**, and **usr**). Otherwise, the system or node installation will fail. - - - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. - - - **Encryption** is not selected by default. - - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. - -#. Configure networking parameters. - - - **VPC**: The value is the same as that of the cluster and cannot be changed. - - **This parameter is displayed only for clusters of v1.13.10-r0 and later.** - - - **Subnet**: A subnet improves network security by providing exclusive network resources that are isolated from other networks. - - You can select any subnet in the cluster VPC. Cluster nodes can belong to different subnets. - - Ensure that the DNS server in the subnet can resolve the OBS domain name. Otherwise, nodes cannot be created. - - **This parameter is displayed only for clusters of v1.13.10-r0 and later.** - -#. Configure advanced settings. - - - **Kubernetes Label**: Kubernetes provides labels for you to run kubectl commands to filter node resources by label. - - - **Resource Tags**: Resource tags can be added to classify resources. - - You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. - - CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *Node ID*" tag. A maximum of 5 tags can be added. - - - **Taints**: Taints allow a node to repel a set of pods and work with tolerations to ensure that pods are not scheduled onto inappropriate nodes. For details, see :ref:`Configuring Node Scheduling (Tainting) `. - - - **Max Pods**: maximum number of pods that can be created on a node, including the system's default pods. If the cluster uses the **VPC network model**, the maximum value is determined by the number of IP addresses that can be allocated to containers on each node. - - This limit prevents the node from being overloaded by managing too many pods. For details, see :ref:`Maximum Number of Pods That Can Be Created on a Node `. - - - **Pre-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. It is commonly used to format data disks. - - - **Post-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed after Kubernetes software is installed and will not affect the installation. It is commonly used to modify Docker parameters. - - - **Maximum Data Space per Container**: maximum data space that can be used by a container. The value ranges from 10 GB to 500 GB. If the value of this field is larger than the data disk space allocated to Docker resources, the latter will override the value specified here. Typically, 90% of the data disk space is allocated to Docker resources. This parameter is displayed only for clusters of v1.13.10-r0 and later. + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+================================================================================================================================================================================================================================================================+ + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | + | | | + | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Resource Tag | You can add resource tags to classify resources. | + | | | + | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | + | | | + | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Taint | This parameter is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | + | | | + | | - **Key**: A key must contain 1 to 63 characters, starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | + | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | + | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | + | | | + | | For details, see :ref:`Managing Node Taints `. | + | | | + | | .. note:: | + | | | + | | For a cluster of v1.19 or earlier, the workload may have been scheduled to a node before the taint is added. To avoid such a situation, select a cluster of v1.19 or later. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Max. Pods | Maximum number of pods that can run on the node, including the default system pods. | + | | | + | | This limit prevents the node from being overloaded with pods. | + | | | + | | This number is also decided by other factors. For details, see :ref:`Maximum Number of Pods That Can Be Created on a Node `. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ECS Group | An ECS group logically groups ECSs. The ECSs in the same ECS group comply with the same policy associated with the ECS group. | + | | | + | | **Anti-affinity**: ECSs in an ECS group are deployed on different physical hosts to improve service reliability. | + | | | + | | Select an existing ECS group, or click **Add ECS Group** to create one. After the ECS group is created, click the refresh button. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pre-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Post-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed after Kubernetes software is installed and will not affect the installation. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Agency | An agency is created by the account administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. | + | | | + | | If no agency is available, click **Create Agency** on the right to create one. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ #. Click **Next: Confirm**. + #. Click **Submit**. -Viewing Node Pools in a Cluster -------------------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. -#. In the upper right corner of the node pool list, select a cluster. All node pools in the cluster will be displayed. You can view the node type, node specifications, autoscaler status, and OS of each node pool. - - .. note:: - - - A default node pool **DefaultPool** is automatically created in each cluster. The default node pool cannot be edited, deleted, or migrated. All nodes created during and after cluster creation are displayed in the default node pool. - - To display a list of nodes in **DefaultPool**, click the **Nodes** subcard in the **DefaultPool** card. - -#. To filter node pools by autoscaler status, select the autoscaler status in the upper right corner of the node pool list. -#. In the node pool list, click a node pool name. On the node pool details page, view the basic information, advanced ECS settings, advanced Kubernetes settings, and node list of the node pool. - -.. |image1| image:: /_static/images/en-us_image_0258503428.png -.. |image2| image:: /_static/images/en-us_image_0273156799.png -.. |image3| image:: /_static/images/en-us_image_0220702939.png -.. |image4| image:: /_static/images/en-us_image_0183134608.png -.. |image5| image:: /_static/images/en-us_image_0183134473.png +.. |image1| image:: /_static/images/en-us_image_0000001201381906.png diff --git a/umn/source/node_pools/index.rst b/umn/source/node_pools/index.rst index a0884de..63112c2 100644 --- a/umn/source/node_pools/index.rst +++ b/umn/source/node_pools/index.rst @@ -1,13 +1,13 @@ -:original_name: cce_01_0035.html +:original_name: cce_10_0035.html -.. _cce_01_0035: +.. _cce_10_0035: Node Pools ========== -- :ref:`Node Pool Overview ` -- :ref:`Creating a Node Pool ` -- :ref:`Managing a Node Pool ` +- :ref:`Node Pool Overview ` +- :ref:`Creating a Node Pool ` +- :ref:`Managing a Node Pool ` .. toctree:: :maxdepth: 1 diff --git a/umn/source/node_pools/managing_a_node_pool.rst b/umn/source/node_pools/managing_a_node_pool.rst index eac965f..ccac6d4 100644 --- a/umn/source/node_pools/managing_a_node_pool.rst +++ b/umn/source/node_pools/managing_a_node_pool.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0222.html +:original_name: cce_10_0222.html -.. _cce_01_0222: +.. _cce_10_0222: Managing a Node Pool ==================== @@ -17,97 +17,205 @@ CCE allows you to highly customize Kubernetes parameter settings on core compone This function is supported only in clusters of **v1.15 and later**. It is not displayed for clusters earlier than v1.15. -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. -#. In the upper right corner of the displayed page, select a cluster to filter node pools by cluster. -#. Click **Configuration** next to the node pool name. -#. On the **Configuration** page on the right, change the values of the following Kubernetes parameters: +#. Log in to the CCE console. +#. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. +#. Choose **More** > **Manage** next to the node pool name. +#. On the **Manage Component** page on the right, change the values of the following Kubernetes parameters: - .. table:: **Table 1** Kubernetes parameters + .. table:: **Table 1** kubelet - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | Component | Parameter | Description | Default Value | Remarks | - +=============+==================================+====================================================================================================================================================================================================================================================================================================================================================================================================================+===============+============================================================+ - | docker | native-umask | \`--exec-opt native.umask | normal | Cannot be changed. | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | docker-base-size | \`--storage-opts dm.basesize | 10G | Cannot be changed. | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | insecure-registry | Address of an insecure image registry | false | Cannot be changed. | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | limitcore | Limit on the number of cores | 5368709120 | ``-`` | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | default-ulimit-nofile | Limit on the number of handles in a container | {soft}:{hard} | ``-`` | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | kube-proxy | conntrack-min | sysctl -w net.nf_conntrack_max | 131072 | The values can be modified during the node pool lifecycle. | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | conntrack-tcp-timeout-close-wait | sysctl -w net.netfilter.nf_conntrack_tcp_timeout_close_wait | 1h0m0s | | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | kubelet | cpu-manager-policy | \`--cpu-manager-policy | none | The values can be modified during the node pool lifecycle. | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | 100 | | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | kube-api-burst | Burst to use while talking with kube-apiserver. | 100 | | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | max-pods | Maximum number of pods managed by kubelet. | 110 | | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | pod-pids-limit | PID limit in Kubernetes | -1 | | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | with-local-dns | Whether to use the local IP address as the ClusterDNS of the node. | false | | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ - | | allowed-unsafe-sysctls | Insecure system configuration allowed. | [] | | - | | | | | | - | | | Starting from **v1.17.17**, CCE enables pod security policies for kube-apiserver. You need to add corresponding configurations to **allowedUnsafeSysctls** of a pod security policy to make the policy take effect. (This configuration is not required for clusters earlier than v1.17.17.) For details, see :ref:`Example of Enabling Unsafe Sysctls in Pod Security Policy `. | | | - +-------------+----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------+------------------------------------------------------------+ + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | Default Value | Remarks | + +============================+====================================================================================================================================================================================================================================================================================================================================================================================================================+=================================================================================================================================+=======================================================================================================================================================================================================================================================================+ + | cpu-manager-policy | Specifies the CPU core binding configuration. For details, see :ref:`CPU Core Binding `. | none | The values can be modified during the node pool lifecycle. | + | | | | | + | | - **none**: disables pods from exclusively occupying CPUs. Select this value if you want a large pool of shareable CPU cores. | | | + | | - **static**: enables pods to exclusively occupy CPUs. Select this value if your workload is sensitive to latency in CPU cache and scheduling. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kube-api-qps | Query per second (QPS) to use while talking with kube-apiserver. | 100 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kube-api-burst | Burst to use while talking with kube-apiserver. | 100 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | max-pods | Maximum number of pods managed by kubelet. | 40 | | + | | | | | + | | | 20 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | pod-pids-limit | PID limit in Kubernetes | -1 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | with-local-dns | Whether to use the local IP address as the ClusterDNS of the node. | false | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | event-qps | QPS limit for event creation | 5 | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | allowed-unsafe-sysctls | Insecure system configuration allowed. | [] | | + | | | | | + | | Starting from **v1.17.17**, CCE enables pod security policies for kube-apiserver. You need to add corresponding configurations to **allowedUnsafeSysctls** of a pod security policy to make the policy take effect. (This configuration is not required for clusters earlier than v1.17.17.) For details, see :ref:`Example of Enabling Unsafe Sysctls in Pod Security Policy `. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | over-subscription-resource | Whether to enable node oversubscription. | true | ``-`` | + | | | | | + | | If this parameter is set to **true**, the node oversubscription feature is enabled. For details, see :ref:`Hybrid Deployment of Online and Offline Jobs `. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | colocation | Whether to enable node hybrid deployment. | true | ``-`` | + | | | | | + | | If this parameter is set to **true**, the node hybrid deployment feature is enabled. For details, see :ref:`Hybrid Deployment of Online and Offline Jobs `. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | kube-reserved-mem | Reserved node memory. | Depends on node specifications. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. | The sum of kube-reserved-mem and system-reserved-mem is less than half of the memory. | + | | | | | + | system-reserved-mem | | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | topology-manager-policy | Set the topology management policy. | none | The values can be modified during the node pool lifecycle. | + | | | | | + | | Valid values are as follows: | | .. important:: | + | | | | | + | | - **restricted**: kubelet accepts only pods that achieve optimal NUMA alignment on the requested resources. | | NOTICE: | + | | - **best-effort**: kubelet preferentially selects pods that implement NUMA alignment on CPU and device resources. | | Exercise caution when modifying topology-manager-policy and topology-manager-scope will restart kubelet and recalculate the resource allocation of pods based on the modified policy. As a result, running pods may restart or even fail to receive any resources. | + | | - **none** (default): The topology management policy is disabled. | | | + | | - **single-numa-node**: kubelet allows only pods that are aligned to the same NUMA node in terms of CPU and device resources. | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | topology-manager-scope | Set the resource alignment granularity of the topology management policy. Valid values are as follows: | container | | + | | | | | + | | - **container** (default) | | | + | | - **pod** | | | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | resolv-conf | DNS resolution configuration file specified by the container | The default value is null. | ``-`` | + +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + .. table:: **Table 2** kube-proxy + + +----------------------------------+-------------------------------------------------------------+---------------+------------------------------------------------------------+ + | Parameter | Description | Default Value | Remarks | + +==================================+=============================================================+===============+============================================================+ + | conntrack-min | sysctl -w net.nf_conntrack_max | 131072 | The values can be modified during the node pool lifecycle. | + +----------------------------------+-------------------------------------------------------------+---------------+------------------------------------------------------------+ + | conntrack-tcp-timeout-close-wait | sysctl -w net.netfilter.nf_conntrack_tcp_timeout_close_wait | 1h0m0s | | + +----------------------------------+-------------------------------------------------------------+---------------+------------------------------------------------------------+ + + .. table:: **Table 3** Network components (available only for CCE Turbo clusters) + + +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + | Parameter | Description | Default Value | Remarks | + +===========================+======================================================================================================+=================+=================+ + | nic-threshold | Low threshold of the number of bound ENIs:High threshold of the number of bound ENIs | Default: 0:0 | ``-`` | + | | | | | + | | .. note:: | | | + | | | | | + | | This parameter is being discarded. Use the dynamic pre-binding parameters of the other four ENIs. | | | + +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + | nic-minimum-target | Minimum number of ENIs bound to the nodes in the node pool | Default: 10 | ``-`` | + +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + | nic-maximum-target | Maximum number of ENIs pre-bound to a node at the node pool level | Default: 0 | ``-`` | + +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + | nic-warm-target | Number of ENIs pre-bound to a node at the node pool level | Default: 2 | ``-`` | + +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + | nic-max-above-warm-target | Reclaim number of ENIs pre-bound to a node at the node pool level | Default: 2 | ``-`` | + +---------------------------+------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + + .. table:: **Table 4** Pod security group in a node pool (available only for CCE Turbo clusters) + + +------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + | Parameter | Description | Default Value | Remarks | + +==============================+=====================================================================================================================================================================================================================================================================================================+=================+=================+ + | security_groups_for_nodepool | - Default security group used by pods in a node pool. You can enter the security group ID. If this parameter is not set, the default security group of the cluster container network is used. A maximum of five security group IDs can be specified at the same time, separated by semicolons (;). | ``-`` | ``-`` | + | | - The priority of the security group is lower than that of the security group configured for the :ref:`SecurityGroup ` resource object. | | | + +------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------+-----------------+ + + .. table:: **Table 5** Docker (available only for node pools that use Docker) + + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | Parameter | Description | Default Value | Remarks | + +=======================+===============================================================+=================+========================================================================================================+ + | native-umask | \`--exec-opt native.umask | normal | Cannot be changed. | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | docker-base-size | \`--storage-opts dm.basesize | 0 | Cannot be changed. | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | insecure-registry | Address of an insecure image registry | false | Cannot be changed. | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | limitcore | Maximum size of a core file in a container. The unit is byte. | 5368709120 | ``-`` | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ + | default-ulimit-nofile | Limit on the number of handles in a container | {soft}:{hard} | The value cannot exceed the value of the kernel parameter **nr_open** and cannot be a negative number. | + | | | | | + | | | | You can run the following command to obtain the kernel parameter **nr_open**: | + | | | | | + | | | | .. code-block:: | + | | | | | + | | | | sysctl -a | grep nr_open | + +-----------------------+---------------------------------------------------------------+-----------------+--------------------------------------------------------------------------------------------------------+ #. Click **OK**. Editing a Node Pool ------------------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. +#. Log in to the CCE console. -#. In the upper right corner of the displayed page, select a cluster to filter node pools by cluster. +#. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. -#. Click **Edit** next to the name of the node pool you will edit. In the **Edit Node Pool** dialog box, edit the following parameters: +#. Click **Edit** next to the name of the node pool you will edit. In the **Edit Node Pool** page, edit the following parameters: - .. table:: **Table 2** Node pool parameters + **Basic Settings** + + .. table:: **Table 6** Basic settings +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+=================================================================================================================================================================================================================================================================================================================================================================================================================================================+ - | Name | Name of the node pool. | + | Node Pool Name | Name of the node pool. | +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Nodes | Modify the number of nodes based on service requirements. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Autoscaler | By default, autoscaler is disabled. | + | Auto Scaling | By default, this parameter is disabled. | | | | | | After you enable autoscaler by clicking |image1|, nodes in the node pool are automatically created or deleted based on service requirements. | | | | | | - **Maximum Nodes** and **Minimum Nodes**: You can set the maximum and minimum number of nodes to ensure that the number of nodes to be scaled is within a proper range. | | | - **Priority**: A larger value indicates a higher priority. For example, if this parameter is set to **1** and **4** respectively for node pools A and B, B has a higher priority than A, and auto scaling is first triggered for B. If the priorities of multiple node pools are set to the same value, for example, **2**, the node pools are not prioritized and the system performs scaling based on the minimum resource waste principle. | + | | - **Cooldown Period**: Required. The unit is minute. This parameter indicates the interval between the previous scale-out action and the next scale-in action. | | | | - | | If the **Autoscaler** field is set to on, install the :ref:`autoscaler add-on ` to use the autoscaler feature. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Taints | - This field is left blank by default. Taints allow nodes to repel a set of pods. You can add a maximum of 10 taints for each node pool. Each taint contains the following parameters: | - | | | - | | - **Key**: A key must contain 1 to 63 characters starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | - | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | - | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | If taints are used, you must configure tolerations in the YAML files of pods. Otherwise, scale-up may fail or pods cannot be scheduled onto the added nodes. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | K8S Labels | K8S labels are key/value pairs that are attached to objects, such as pods. Labels are used to specify identifying attributes of objects that are meaningful and relevant to users, but do not directly imply semantics to the core system. For more information, see `Labels and Selectors `__. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Resource Tags | It is recommended that you use TMS's **predefined tag** function to add the same tag to different cloud resources. | - | | | - | | Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency. | - | | | - | | Tag changes do not affect the node. | + | | If the **Autoscaler** field is set to on, install the :ref:`autoscaler add-on ` to use the autoscaler feature. | +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -#. After the configuration is complete, click **Save**. + **Advanced Settings** + + .. table:: **Table 7** Advanced settings + + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+================================================================================================================================================================================================================================================================+ + | K8s label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | + | | | + | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | + | | | + | | .. note:: | + | | | + | | After a **K8s label** is modified, the inventory nodes in the node pool are updated synchronously. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Resource Tag | You can add resource tags to classify resources. | + | | | + | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | + | | | + | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | + | | | + | | .. note:: | + | | | + | | After a **resource tag** is modified, the modification automatically takes effect when a node is added. For existing nodes, you need to manually reset the nodes for the modification to take effect. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Taint | This field is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | + | | | + | | - **Key**: A key must contain 1 to 63 characters starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | + | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | + | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | + | | | + | | For details, see :ref:`Managing Node Taints `. | + | | | + | | .. note:: | + | | | + | | After a **taint** is modified, the inventory nodes in the node pool are updated synchronously. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Edit Key pair | Only node pools that use key pairs for login support key pair editing. You can select another key pair. | + | | | + | | .. note:: | + | | | + | | The edited key pair automatically takes effect when a node is added. For existing nodes, you need to manually reset the nodes for the key pair to take effect. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. Click **OK**. In the node pool list, the node pool status becomes **Scaling**. After the status changes to **Completed**, the node pool parameters are modified successfully. The modified configuration will be synchronized to all nodes in the node pool. @@ -116,21 +224,23 @@ Deleting a Node Pool Deleting a node pool will delete nodes in the pool. Pods on these nodes will be automatically migrated to available nodes in other node pools. If pods in the node pool have a specific node selector and none of the other nodes in the cluster satisfies the node selector, the pods will become unschedulable. -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. -#. In the upper right corner of the displayed page, select a cluster to filter node pools by cluster. +#. Log in to the CCE console. +#. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. #. Choose **More > Delete** next to a node pool name to delete the node pool. #. Read the precautions in the **Delete Node Pool** dialog box. -#. Enter **DELETE** in the text box and click **Yes** to confirm that you want to continue the deletion. +#. In the text box, click **Yes** to confirm that you want to continue the deletion. + +.. _cce_10_0222__section550619571556: Copying a Node Pool ------------------- You can copy the configuration of an existing node pool to create a new node pool on the CCE console. -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. -#. In the upper right corner of the displayed page, select a cluster to filter node pools by cluster. +#. Log in to the CCE console. +#. Click the cluster name and access the cluster console. Choose **Nodes** in the navigation pane and click the **Node Pools** tab on the right. #. Choose **More > Copy** next to a node pool name to copy the node pool. -#. The configuration of the selected node pool is replicated to the **Create Node Pool** page. You can edit the configuration as required and click **Next: Confirm**. +#. The configurations of the selected node pool are replicated to the **Clone Node Pool** page. You can edit the configurations as required and click **Next: Confirm**. #. On the **Confirm** page, confirm the node pool configuration and click **Create Now**. Then, a new node pool is created based on the edited configuration. Migrating a Node @@ -138,15 +248,20 @@ Migrating a Node Nodes in a node pool can be migrated. Currently, nodes in a node pool can be migrated only to the default node pool (defaultpool) in the same cluster. -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. -#. In the upper right corner of the displayed page, select a cluster to filter node pools by cluster. -#. Click **More** > **Migrate** next to the name of the node pool. -#. In the dialog box displayed, select the destination node pool and the node to be migrated. +#. Log in to the CCE console and click the cluster name to access the cluster. + +#. In the navigation pane, choose **Nodes** and switch to the **Node Pools** tab page. + +#. Click **View Node** in the **Operation** column of the node pool to be migrated. + +#. Select the nodes to be migrated and choose **More** > **Migrate** to migrate the nodes to the default node pool in batches. + + You can also choose **More** > **Migrate** in the **Operation** column of a single node to migrate the node. + +#. In the displayed **Migrate Node** window, confirm the information. .. note:: - After node migration, original resource tags, Kubernetes labels, and taints will be retained, and new Kubernetes labels and taints from the destination node pool will be added. + The migration has no impacts on the original resource tags, Kubernetes labels, and taints of the node. -#. Click **OK**. - -.. |image1| image:: /_static/images/en-us_image_0214003838.png +.. |image1| image:: /_static/images/en-us_image_0000001528627005.png diff --git a/umn/source/node_pools/node_pool_overview.rst b/umn/source/node_pools/node_pool_overview.rst index 3b57594..d75ae98 100644 --- a/umn/source/node_pools/node_pool_overview.rst +++ b/umn/source/node_pools/node_pool_overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0081.html +:original_name: cce_10_0081.html -.. _cce_01_0081: +.. _cce_10_0081: Node Pool Overview ================== @@ -22,28 +22,27 @@ This section describes how node pools work in CCE and how to create and manage n Node Pool Architecture ---------------------- - -.. figure:: /_static/images/en-us_image_0269288708.png - :alt: **Figure 1** Overall architecture of a node pool - - **Figure 1** Overall architecture of a node pool - Generally, all nodes in a node pool have the following same attributes: - Node OS +- Node specifications +- Node login mode. +- Node runtime. - Startup parameters of Kubernetes components on a node - User-defined startup script of a node -- **K8S Labels** and **Taints** +- **K8s Labels** and **Taints** CCE provides the following extended attributes for node pools: - Node pool OS - Maximum number of pods on each node in a node pool +.. _cce_10_0081__section16928123042115: + Description of DefaultPool -------------------------- -DefaultPool is not a real node pool. It only **classifies** nodes that are not in any node pool. These nodes are directly created on the console or by calling APIs. DefaultPool does not support any node pool functions, including scaling and parameter configuration. DefaultPool cannot be edited, deleted, expanded, or auto scaled, and nodes in it cannot be migrated. +DefaultPool is not a real node pool. It only **classifies** nodes that are not in the user-created node pools. These nodes are directly created on the console or by calling APIs. DefaultPool does not support any user-created node pool functions, including scaling and parameter configuration. DefaultPool cannot be edited, deleted, expanded, or auto scaled, and nodes in it cannot be migrated. Applicable Scenarios -------------------- @@ -67,40 +66,35 @@ The following table describes multiple scenarios of large-scale cluster manageme Functions and Precautions ------------------------- -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Function | Description | Notes | -+========================================+========================================================================================================================================================+====================================================================================================================================================================================+ -| Creating a node pool | Add a node pool. | It is recommended that a cluster contain no more than 100 node pools. | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Deleting a node pool | Deleting a node pool will delete nodes in the pool. Pods on these nodes will be automatically migrated to available nodes in other node pools. | If pods in the node pool have a specific node selector and none of the other nodes in the cluster satisfies the node selector, the pods will become unschedulable. | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Enabling auto scaling for a node pool | After auto scaling is enabled, nodes will be automatically created or deleted in the node pool based on the cluster loads. | You are advised not to store important data on nodes in a node pool because after auto scaling, data cannot be restored as nodes may be deleted. | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Enabling auto scaling for a node pool | After auto scaling is disabled, the number of nodes in a node pool will not automatically change with the cluster loads. | / | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Adjusting the size of a node pool | The number of nodes in a node pool can be directly adjusted. If the number of nodes is reduced, nodes are randomly removed from the current node pool. | After auto scaling is enabled, you are not advised to manually adjust the node pool size. | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Changing node pool configurations | You can modify the node pool name, node quantity, Kubernetes labels, taints, and resource tags. | The modified Kubernetes labels and taints will apply to all nodes in the node pool, which may cause pod re-scheduling. Therefore, exercise caution when performing this operation. | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Adding an existing node to a node pool | Nodes that do not belong to the cluster can be added to a node pool. The following requirements must be met: | Unless required, you are not advised to add existing nodes. You are advised to create a node pool. | -| | | | -| | - The node to be added and the CCE cluster are in the same VPC and subnet. | | -| | - The node is not used by other clusters and has the same configurations (such as specifications and billing mode) as the node pool. | | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Removing a node from a node pool | Nodes in a node pool can be migrated to the default node pool of the same cluster. | Nodes in the default node pool cannot be migrated to other node pools, and nodes in a user-created node pool cannot be migrated to other user-created node pools. | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Cloning a node pool | You can copy the configuration of an existing node pool to create a new node pool. | / | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Setting Kubernetes parameters | You can configure core components with fine granularity. | - This function is supported only for clusters of v1.15 and later. It is not displayed for versions earlier than v1.15 | -| | | - The default node pool DefaultPool does not support this type of configuration. | -+----------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Function | Description | Notes | ++=======================================+========================================================================================================================================================+========================================================================================================================================================================================================================+ +| Creating a node pool | Add a node pool. | It is recommended that a cluster contains no more than 100 node pools. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Deleting a node pool | Deleting a node pool will delete nodes in the pool. Pods on these nodes will be automatically migrated to available nodes in other node pools. | If pods in the node pool have a specific node selector and none of the other nodes in the cluster satisfies the node selector, the pods will become unschedulable. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Enabling auto scaling for a node pool | After auto scaling is enabled, nodes will be automatically created or deleted in the node pool based on the cluster loads. | You are advised not to store important data on nodes in a node pool because after auto scaling, data cannot be restored as nodes may be deleted. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Enabling auto scaling for a node pool | After auto scaling is disabled, the number of nodes in a node pool will not automatically change with the cluster loads. | / | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Adjusting the size of a node pool | The number of nodes in a node pool can be directly adjusted. If the number of nodes is reduced, nodes are randomly removed from the current node pool. | After auto scaling is enabled, you are not advised to manually adjust the node pool size. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Changing node pool configurations | You can modify the node pool name, node quantity, Kubernetes labels (and their quantity), and taints. | The deleted or added Kubernetes labels and taints (as well as their quantity) will apply to all nodes in the node pool, which may cause pod re-scheduling. Therefore, exercise caution when performing this operation. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Removing a node from a node pool | Nodes in a node pool can be migrated to the default node pool of the same cluster. | Nodes in the default node pool cannot be migrated to other node pools, and nodes in a user-created node pool cannot be migrated to other user-created node pools. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Cloning a node pool | You can copy the configuration of an existing node pool to create a new node pool. | / | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ +| Setting Kubernetes parameters | You can configure core components with fine granularity. | - This function is supported only in clusters of v1.15 and later. It is not displayed for versions earlier than v1.15. | +| | | - The default node pool DefaultPool does not support this type of configuration. | ++---------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Deploying a Workload in a Specified Node Pool --------------------------------------------- When creating a workload, you can constrain pods to run in a specified node pool. -For example, on the CCE console, you can set the affinity between the workload and the node on the **Scheduling Policies** tab page on the workload details page to forcibly deploy the workload to a specific node pool. In this way, the workload runs only on nodes in the node pool. If you need to better control where the workload is to be scheduled, you can use affinity or anti-affinity policies between workloads and nodes described in :ref:`Scheduling Policy Overview `. +For example, on the CCE console, you can set the affinity between the workload and the node on the **Scheduling Policies** tab page on the workload details page to forcibly deploy the workload to a specific node pool. In this way, the workload runs only on nodes in the node pool. If you need to better control where the workload is to be scheduled, you can use affinity or anti-affinity policies between workloads and nodes described in :ref:`Scheduling Policy (Affinity/Anti-affinity) `. For example, you can use container's resource request as a nodeSelector so that workloads will run only on the nodes that meet the resource request. @@ -111,7 +105,7 @@ Related Operations You can log in to the CCE console and refer to the following sections to perform operations on node pools: -- :ref:`Creating a Node Pool ` -- :ref:`Managing a Node Pool ` -- :ref:`Creating a Deployment ` -- :ref:`Workload-Node Affinity ` +- :ref:`Creating a Node Pool ` +- :ref:`Managing a Node Pool ` +- :ref:`Creating a Deployment ` +- :ref:`Scheduling Policy (Affinity/Anti-affinity) ` diff --git a/umn/source/nodes/creating_a_node_in_a_cce_turbo_cluster.rst b/umn/source/nodes/adding_nodes_for_management.rst similarity index 54% rename from umn/source/nodes/creating_a_node_in_a_cce_turbo_cluster.rst rename to umn/source/nodes/adding_nodes_for_management.rst index 8c28601..23e0311 100644 --- a/umn/source/nodes/creating_a_node_in_a_cce_turbo_cluster.rst +++ b/umn/source/nodes/adding_nodes_for_management.rst @@ -1,128 +1,112 @@ -:original_name: cce_01_0363.html +:original_name: cce_10_0198.html -.. _cce_01_0363: +.. _cce_10_0198: -Creating a Node in a CCE Turbo Cluster -====================================== +Adding Nodes for Management +=========================== -Prerequisites -------------- +Scenario +-------- -- At least one CCE Turbo cluster is available. For details on how to create a cluster, see :ref:`Creating a CCE Turbo Cluster `. -- A key pair has been created for identity authentication upon remote node login. +In CCE, you can :ref:`Creating a Node ` or add existing nodes (ECSs) into your cluster. + +.. important:: + + - While an ECS is being accepted into a cluster, the operating system of the ECS will be reset to the standard OS image provided by CCE to ensure node stability. The CCE console prompts you to select the operating system and the login mode during the reset. + - The system disk and data disk of an ECS will be formatted while the ECS is being accepted into a cluster. Ensure that information in the disks has been backed up. + - While an ECS is being accepted into a cluster, do not perform any operation on the ECS through the ECS console. Notes and Constraints --------------------- -- During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name. -- Nodes in a CCE Turbo cluster must be the models developed on the QingTian architecture that features software-hardware synergy. -- CCE Turbo clusters are available only in certain regions. +- The cluster version must be 1.15 or later. +- If the password or key has been set when a VM node is created, the VM node can be accepted into a cluster 10 minutes after it is available. During the management, the original password or key will become invalid. You need to reset the password or key. +- Nodes in a CCE Turbo cluster must support sub-ENIs or be bound to at least 16 ENIs. For details about the node specifications, see the nodes that can be selected on the console when you create a node. -Procedure for Creating a Node ------------------------------ +Prerequisites +------------- -After a CCE Turbo cluster is created, you can create nodes for the cluster. +A cloud server that meets the following conditions can be accepted: -#. Click **Create** **Node** in the card view of the created CCE Turbo cluster. In the **Node Configuration** step, set node parameters by referring to the following tables. +- The node to be accepted must be in the **Running** state and not used by other clusters. In addition, the node to be accepted does not carry the CCE-Dynamic-Provisioning-Node tag. +- The node to be accepted and the cluster must be in the same VPC. (If the cluster version is earlier than v1.13.10, the node to be accepted and the CCE cluster must be in the same subnet.) +- At least one data disk is attached to the node to be accepted. The data disk capacity is greater than or equal to 100 GB. +- The node to be accepted has 2-core or higher CPU, 4 GB or larger memory, and only one NIC. +- Only cloud servers with the same specifications, AZ, and data disk configuration can be added in batches. - **Computing configurations:** +Procedure +--------- - You can configure the specifications and OS of a cloud server, on which your containerized applications run. +#. Log in to the CCE console and go to the cluster where the node to be managed resides. + +#. In the navigation pane, choose **Nodes**. On the displayed page, click **Accept Node** in the upper right corner. + +#. Specify node parameters. + + **Compute Settings** .. table:: **Table 1** Configuration parameters - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+====================================================================================================================================================================================================================================+ - | AZ | AZ where the node is located. Nodes in a cluster can be created in different AZs for higher reliability. The value cannot be changed after creation. | - | | | - | | You are advised to select **Random** to deploy your node in a random AZ based on the selected node flavor. | - | | | - | | An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. To enhance workload availability, create nodes in different AZs. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container runtime | Container runtime used on the node. Different container runtimes support different node specifications and cannot be changed after the node is created. | - | | | - | | - **runc**: The runC runtime is used. By default, Docker is selected as the container engine when you create a container on the console. | - | | - kata: The Kata runtime is used. If you select this type for both nodes and workloads, the workloads run only on the nodes that use the Kata runtime. containerd is used by default. | - | | | - | | For details about common containers and secure containers, see :ref:`Secure Containers and Common Containers `. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Specifications | Select node specifications that best fit your business needs. | - | | | - | | Nodes in a CCE Turbo cluster must be the models developed on the QingTian architecture that features software-hardware synergy. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | OS | **Public image**: Select an OS for the node. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Node Name | Name of the node, which must be unique. When nodes (ECSs) are created in batches, the value of this parameter is used as the name prefix for each ECS. | - | | | - | | The system generates a default name for you, which can be modified. | - | | | - | | A node name must start with a lowercase letter and cannot end with a hyphen (-). Only digits, lowercase letters, and hyphens (-) are allowed. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Login Mode | - **Key pair**: Select the key pair used to log in to the node. You can select a shared key. | - | | | - | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=======================================================================================================================================================================================+ + | Specifications | Click **Select Cloud Server** and select the servers to be accepted. | + | | | + | | You can select multiple cloud servers for batch management. However, only the cloud servers with the same specifications, AZ, and data disk configuration can be added in batches. | + | | | + | | If a cloud server contains multiple data disks, select one of them for the container runtime and kubelet. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Container Engine | CCE clusters support Docker. Starting from CCE 1.23, containerd is supported. | + | | | + | | For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | OS | **Public image**: Select an OS for the node. | + | | | + | | **Private image**: You can use private images. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Login Mode | - **Key Pair** | + | | | + | | Select the key pair used to log in to the node. You can select a shared key. | + | | | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - **Storage configuration** + **Storage Settings** - Configure storage resources on a node for the containers running on it. Set the disk size according to site requirements. + Configure storage resources on a node for the containers running on it. .. table:: **Table 2** Configuration parameters - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===============================================================================================================================================================================================================================================================+ - | System Disk | System disk used by the node OS. The value ranges from 40 GB to 1,024 GB. The default value is 50 GB. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Data Disk | Data disk used by the container runtime and kubelet on the node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. The EVS disk types provided for the data disk are the same as those for the system disk. | - | | | - | | .. caution:: | - | | | - | | CAUTION: | - | | If the data disk is uninstalled or damaged, the Docker service becomes abnormal and the node becomes unavailable. You are advised not to delete the data disk. | - | | | - | | Click **Expand** to set the following parameters: | - | | | - | | - **Custom space allocation**: Select this option to define the disk space occupied by the container runtime to store the working directories, container image data, and image metadata. | - | | - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. | - | | | - | | - **Encryption** is not selected by default. | - | | - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - **Networking configuration** - - Configure networking resources to allow node and containerized application access. - - .. table:: **Table 3** Configuration parameters - - +-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +=============+=========================================================================================================================================================+ - | Node Subnet | The node subnet selected during cluster creation is used by default. You can choose another subnet instead. The value cannot be changed after creation. | - +-------------+---------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+====================================================================================================================================================================================================================================================================================================+ + | System Disk | Directly use the system disk of the cloud server. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Data Disk | **At least one data disk is required** for the container runtime and kubelet. **The data disk cannot be deleted or uninstalled. Otherwise, the node will be unavailable.** | + | | | + | | Click **Expand** and select **Allocate Disk Space** to define the disk space occupied by the container runtime to store the working directories, container image data, and image metadata. For details about how to allocate data disk space, see :ref:`Data Disk Space Allocation `. | + | | | + | | For other data disks, a raw disk is created without any processing by default. You can also click **Expand** and select **Mount Disk** to mount the data disk to a specified directory. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ **Advanced Settings** - Configure advanced node capabilities such as labels, taints, and startup command. - - .. table:: **Table 4** Advanced configuration parameters + .. table:: **Table 3** Advanced configuration parameters +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+================================================================================================================================================================================================================================================================+ - | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 10 labels can be added. | + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | | | | | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Resource Tags | You can add resource tags to classify resources. | + | Resource Tag | You can add resource tags to classify resources. | | | | | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | | | | | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Taints | This parameter is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | + | Taint | This field is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | | | | | | - **Key**: A key must contain 1 to 63 characters, starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | @@ -132,30 +116,20 @@ After a CCE Turbo cluster is created, you can create nodes for the cluster. | | | | | NOTICE: | | | | - | | - If taints are used, you must configure tolerations in the YAML files of pods. Otherwise, scale-out may fail or pods cannot be scheduled onto the added nodes. | + | | - If taints are used, you must configure tolerations in the YAML files of pods. Otherwise, scale-up may fail or pods cannot be scheduled onto the added nodes. | | | - After a node pool is created, you can click **Edit** to modify its configuration. The modification will be synchronized to all nodes in the node pool. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Max Pods | Maximum number of pods that can run on the node, including the default system pods. | + | Max. Pods | Maximum number of pods that can run on the node, including the default system pods. | | | | - | | This limit prevents the node from being overloaded of pods. For details, see :ref:`Maximum Number of Pods That Can Be Created on a Node `. | + | | This limit prevents the node from being overloaded with pods. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Pre-installation Script | Enter commands. A maximum of 1,000 characters are allowed. | + | Pre-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | | | | - | | The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. The commands are run to format data disks. | + | | The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Post-installation Script | Enter commands. A maximum of 1,000 characters are allowed. | + | Post-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | | | | - | | The script will be executed after Kubernetes software is installed and will not affect the installation. The commands are run to modify Docker parameters. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Agency | An agency is created by the account administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. | - | | | - | | If no agency is available, click **Create Agency** on the right to create one. | + | | The script will be executed after Kubernetes software is installed and will not affect the installation. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -#. Click **Next: Confirm** to review the configurations. - -#. Click **Submit**. - - The node list page is displayed. If the node status is **Available**, the node is created successfully. It takes about 6 to 10 minutes to create a node. - -#. Click **Back to Node List**. The node is created successfully if it changes to the **Available** state. +#. Click **Next: Confirm**. Click **Submit**. diff --git a/umn/source/nodes/creating_a_linux_lvm_disk_partition_for_docker.rst b/umn/source/nodes/creating_a_linux_lvm_disk_partition_for_docker.rst deleted file mode 100644 index e664e87..0000000 --- a/umn/source/nodes/creating_a_linux_lvm_disk_partition_for_docker.rst +++ /dev/null @@ -1,137 +0,0 @@ -:original_name: cce_01_0200.html - -.. _cce_01_0200: - -Creating a Linux LVM Disk Partition for Docker -============================================== - -Scenario --------- - -This section describes how to check whether there are **available raw disks** and **Linux LVM disk partitions** and how to create Linux LVM disk partitions. - -Prerequisites -------------- - -To improve the system stability, attach a data disk to Docker and use the direct-lvm mode. - -Procedure ---------- - -#. .. _cce_01_0200__li139011015111020: - - Check whether available raw disks exist on the current node. - - a. Log in to the target node as the **root** user. - - b. Check the raw disk device. - - **lsblk -l \| grep disk** - - If the following information is displayed, the raw disks named **xvda** and **xvdb** exist on the node. - - .. code-block:: - - xvda 202:0 0 40G 0 disk - xvdb 202:16 0 100G 0 disk - - c. Check whether the raw disk is in use. - - **lsblk /dev/**\ ** - - *devicename* indicates the raw disk name, for example, **xvda** and **xvdb** in the previous step. - - Run the **lsblk /dev/xvda** and **lsblk /dev/xvdb** commands. If the following information is displayed, **xvda** has been partitioned and used while **xvdb** is available. If no raw disk is available, bind an EVS disk to the node. It is advised that the disk space be no less than 80 GB. - - .. code-block:: - - NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT - xvda 202:0 0 40G 0 disk - ├─xvda1 202:1 0 100M 0 part /boot - └─xvda2 202:2 0 39.9G 0 part / - - .. code-block:: - - NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT - xvdb 202:16 0 100G 0 disk - -#. Check whether there are partitions available. Currently, only Linux LVM partitions are supported. - - a. Log in to the target node as the **root** user. - - b. Check the partition whose system type is Linux LVM. - - **sfdisk -l 2>>/dev/null\| grep "Linux LVM"** - - If the following information is displayed, two Linux LVM partitions, **/dev/nvme0n1p1** and **/dev/nvme0n1p2**, exist in the system. - - .. code-block:: - - /dev/nvme0n1p1 1 204800 204800 209715200 8e Linux LVM - /dev/nvme0n1p2 204801 409600 204800 209715200 8e Linux LVM - - c. Check whether the partition is in use. - - **lsblk** ** - - ** is the Linux LVM partition found in the previous step. - - In this example, run the **lsblk/dev/nvme0n1p1** and **lsblk/dev/nvme0n1p2** commands. If the following information is displayed, partition **nvme0n1p** is in use while **nvme0n1p2** is available. - - .. code-block:: - - NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT - nvme0n1p1 259:3 0 200G 0 part - └─vgpaas-thinpool_tdata 251:8 0 360G 0 lvm - └─vgpaas-thinpool 251:10 0 360G 0 lvm - - .. code-block:: - - NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT - nvme0n1p2 259:1 0 100G 0 part - - If no AZ is available, perform :ref:`3 ` to create a partition for Docker. - -#. .. _cce_01_0200__li111391316141612: - - Create a Linux LVM disk partition for Docker. - - a. Run the following command to create a disk partition. **devicename** indicates the available raw disk name, for example, **xvdb** in :ref:`1 `. - - **fdisk /dev/**\ *devicename* - - b. Enter **n** to create a new partition. Enter **p** to display the primary partition number. Enter **4** to indicate the fourth primary partition. - - - .. figure:: /_static/images/en-us_image_0144042759.png - :alt: **Figure 1** Creating a partition - - **Figure 1** Creating a partition - - c. Configure the start and last sectors as follows for example: - - .. code-block:: - - Start sector (1048578048-4294967295, 1048578048 by default): - 1048578048 - Last sector, +sector or size {K, M, or G} (1048578048-4294967294, 4294967294 by default): +100G - - This configuration indicates that partition 4 has been set to the Linux type and the size is 100 GiB. - - d. Enter **t** to change the partition system type. Enter the hex code **8e** when prompted to change the system type to Linux LVM. - - .. code-block:: - - Command (enter m to obtain help): t - Partition ID (ranging from 1 to 4, 4 by default): 4 - Hex code (enter L to list all codes): 8e - This configuration changes the type of the partition Linux to Linux LVM. - - e. Enter **w** to save the modification. - - .. code-block:: - - Command (enter m to obtain help): w - The partition table has been altered! - - f. Run the **partprobe** command to refresh the disk partition. diff --git a/umn/source/nodes/creating_a_node.rst b/umn/source/nodes/creating_a_node.rst index 6f4d620..677daed 100644 --- a/umn/source/nodes/creating_a_node.rst +++ b/umn/source/nodes/creating_a_node.rst @@ -1,225 +1,195 @@ -:original_name: cce_01_0033.html +:original_name: cce_10_0363.html -.. _cce_01_0033: +.. _cce_10_0363: Creating a Node =============== -Scenario --------- - -A node is a virtual or physical machine that provides computing resources. Sufficient nodes must be available in your project to ensure that operations, such as creating workloads, can be performed. - Prerequisites ------------- -- At least one cluster is available. For details on how to create a cluster, see :ref:`Creating a CCE Cluster `. -- A key pair has been created. The key pair will be used for identity authentication upon remote node login. +- At least one cluster has been created. +- A key pair has been created for identity authentication upon remote node login. Notes and Constraints --------------------- +- The node has 2-core or higher CPU, 4 GB or larger memory. +- To ensure node stability, a certain amount of CCE node resources will be reserved for Kubernetes components (such as kubelet, kube-proxy, and docker) based on the node specifications. Therefore, the total number of node resources and assignable node resources in Kubernetes are different. The larger the node specifications, the more the containers deployed on the node. Therefore, more node resources need to be reserved to run Kubernetes components. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. +- The node networking (such as the VM networking and container networking) is taken over by CCE. You are not allowed to add and delete NICs or change routes. If you modify the networking configuration, the availability of CCE may be affected. For example, the NIC named **gw_11cbf51a@eth0** on the node is the container network gateway and cannot be modified. - During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name. -- Only KVM nodes can be created. Non-KVM nodes cannot be used after being created. - Once a node is created, its AZ cannot be changed. -- CCE supports GPUs through an add-on named :ref:`gpu-beta `. You need to install this add-on to use GPU-enabled nodes in your cluster. Procedure --------- -#. Log in to the CCE console. Use either of the following methods to add a node: - - - In the navigation pane, choose **Resource Management** > **Nodes**. Select the cluster to which the node will belong and click **Create** **Node** on the upper part of the node list page. - - In the navigation pane, choose **Resource Management** > **Clusters**. In the card view of the cluster to which you will add nodes, click **Create** **Node**. - -#. Select a region and an AZ. - - - **Current Region**: geographic location of the nodes to be created. - - - **AZ**: Set this parameter based on the site requirements. An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. - - You are advised to deploy worker nodes in different AZs after the cluster is created to make your workloads more reliable. When creating a cluster, you can deploy nodes only in one AZ. - -#. Configure node parameters. - - - **Node Type** - - - **VM node**: A VM node will be created in the cluster. - - - **Node Name**: Enter a node name. A node name contains 1 to 56 characters starting with a lowercase letter and not ending with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed. - - - **Specifications**: Select node specifications that best fit your business needs. - - - **General-purpose**: provides a balance of computing, memory, and network resources. It is a good choice for many applications, such as web servers, workload development, workload testing, and small-scale databases. - - **Memory-optimized**: provides higher memory capacity than general-purpose nodes and is suitable for relational databases, NoSQL, and other workloads that are both memory-intensive and data-intensive. - - **GPU-accelerated**: provides powerful floating-point computing and is suitable for real-time, highly concurrent massive computing. Graphical processing units (GPUs) of P series are suitable for deep learning, scientific computing, and CAE. GPUs of G series are suitable for 3D animation rendering and CAD. **GPU-accelerated nodes can be created only in clusters of v1.11 or later**. GPU-accelerated nodes are available only in certain regions. - - **General computing-plus**: provides stable performance and exclusive resources to enterprise-class workloads with high and stable computing performance. - - **Disk-intensive**: supports :ref:`local disk storage ` and provides high network performance. It is designed for workloads requiring high throughput and data switching, such as big data workloads. - - To ensure node stability, CCE automatically reserves some resources to run necessary system components. For details, see :ref:`Formula for Calculating the Reserved Resources of a Node `. - - - **OS**: Select an OS for the node to be created. - - Reinstalling the OS or modifying OS configurations could make the node unavailable. Exercise caution when performing these operations. - - - **System Disk**: Set the system disk space of the worker node. The value ranges from 40GB to 1024 GB. The default value is 40GB. - - By default, system disks support Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) EVS disks. - - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. **This function is available only in certain regions.** - - - **Encryption** is not selected by default. - - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. - - - .. _cce_01_0033__li12223421320: - - **Data Disk**: Set the data disk space of the worker node. The value ranges from 100 GB to 32,768 GB. The default value is 100 GB. The EVS disk types provided for the data disk are the same as those for the system disk. - - .. caution:: - - If the data disk is uninstalled or damaged, the Docker service becomes abnormal and the node becomes unavailable. You are advised not to delete the data disk. - - - **LVM**: If this option is selected, CCE data disks are managed by the Logical Volume Manager (LVM). On this condition, you can adjust the disk space allocation for different resources. This option is selected for the first disk by default and cannot be unselected. You can choose to enable or disable LVM for new data disks. - - - This option is selected by default, indicating that LVM management is enabled. - - You can deselect the check box to disable LVM management. - - .. caution:: - - - Disk space of the data disks managed by LVM will be allocated according to the ratio you set. - - When creating a node in a cluster of v1.13.10 or later, if LVM is not selected for a data disk, follow instructions in :ref:`Adding a Second Data Disk to a Node in a CCE Cluster ` to fill in the pre-installation script and format the data disk. Otherwise, the data disk will still be managed by LVM. - - When creating a node in a cluster earlier than v1.13.10, you must format the data disks that are not managed by LVM. Otherwise, either these data disks or the first data disk will be managed by LVM. - - - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. - - **This function is supported only for clusters of v1.13.10 or later in certain regions,** and is not displayed for clusters of v1.13.10 or earlier. - - - **Encryption** is not selected by default. - - After you select **Encryption**, you can select an existing key in the displayed **Encryption Setting** dialog box. If no key is available, click the link next to the drop-down box to create a key. After the key is created, click the refresh icon. - - - **Add Data Disk**: Currently, a maximum of two data disks can be attached to a node. After the node is created, you can go to the ECS console to attach more data disks. This function is available only to clusters of certain versions. - - - **Data disk space allocation**: Click |image1| to specify the resource ratio for **Kubernetes Space** and **User Space**. Disk space of the data disks managed by LVM will be allocated according to the ratio you set. This function is available only to clusters of certain versions. - - - **Kubernetes Space**: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs. - - The Docker space cannot be less than 10%, and the space size cannot be less than 60 GB. The kubelet space cannot be less than 10%. - - The Docker space size is determined by your service requirements. For details, see :ref:`Data Disk Space Allocation `. - - - **User Space**: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted. - - .. note:: - - Note that the mount path cannot be **/**, **/home/paas**, **/var/paas**, **/var/lib**, **/var/script**, **/var/log**, **/mnt/paas**, or **/opt/cloud**, and cannot conflict with the system directories (such as **bin**, **lib**, **home**, **root**, **boot**, **dev**, **etc**, **lost+found**, **mnt**, **proc**, **sbin**, **srv**, **tmp**, **var**, **media**, **opt**, **selinux**, **sys**, and **usr**). Otherwise, the system or node installation will fail. - - **If the cluster version is v1.13.10-r0 or later and the node specification is Disk-intensive, the following options are displayed for data disks:** - - - **EVS**: Parameters are the same as those when the node type is not Disk-intensive. For details, see :ref:`Data Disk ` above. - - - **Local disk**: Local disks may break down and do not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks. - - Local disk parameters are as follows: - - - **Disk Mode**: If the node type is **disk-intensive**, the supported disk mode is HDD. - - **Read/Write Mode**: When multiple local disks exist, you can set the read/write mode. The serial and sequential modes are supported. **Sequential** indicates that data is read and written in linear mode. When a disk is used up, the next disk is used. **Serial** indicates that data is read and written in striping mode, allowing multiple local disks to be read and written at the same time. - - **Kubernetes Space**: You can specify the ratio of the data disk space for storing Docker and kubelet resources. Docker resources include the Docker working directory, Docker images, and image metadata. kubelet resources include pod configuration files, secrets, and emptyDirs. - - **User Space**: You can set the ratio of the disk space that is not allocated to Kubernetes resources and the path to which the user space is mounted. - - .. important:: - - - The ratio of disk space allocated to the Kubernetes space and user space must be equal to 100% in total. You can click |image2| to refresh the data after you have modified the ratio. - - By default, disks run in the direct-lvm mode. If data disks are removed, the loop-lvm mode will be used and this will impair system stability. - - - **VPC**: A VPC where the current cluster is located. This parameter cannot be changed and is displayed only for clusters of v1.13.10-r0 or later. - - - **Subnet**: A subnet improves network security by providing exclusive network resources that are isolated from other networks. You can select any subnet in the cluster VPC. Cluster nodes can belong to different subnets. - - During the node creation, software packages are downloaded from OBS using the domain name. You need to use a private DNS server to resolve the OBS domain name, and configure the subnet where the node resides with a private DNS server address. When you create a subnet, the private DNS server is used by default. If you change the subnet DNS, ensure that the DNS server in use can resolve the OBS domain name. - -#. **EIP**: an independent public IP address. If the nodes to be created require public network access, select **Automatically assign** or **Use existing**. - - An EIP bound to the node allows public network access. EIP bandwidth can be modified at any time. An ECS without a bound EIP cannot access the Internet or be accessed by public networks. - - - **Do not use**: A node without an EIP cannot be accessed from public networks. It can be used only as a cloud server for deploying services or clusters on a private network. - - - **Automatically assign**: An EIP with specified configurations is automatically assigned to each node. If the number of EIPs is smaller than the number of nodes, the EIPs are randomly bound to the nodes. - - Configure the EIP specifications, billing factor, bandwidth type, and bandwidth size as required. When creating an ECS, ensure that the elastic IP address quota is sufficient. - - - **Use existing**: Existing EIPs are assigned to the nodes to be created. - - .. note:: - - By default, VPC's SNAT feature is disabled for CCE. If SNAT is enabled, you do not need to use EIPs to access public networks. For details about SNAT, see :ref:`Custom Policies `. - -#. **Login Mode**: - - - **Key pair**: Select the key pair used to log in to the node. You can select a shared key. - - A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. - - .. important:: - - When creating a node using a key pair, IAM users can select only the key pairs created by their own, regardless of whether these users are in the same group. For example, user B cannot use the key pair created by user A to create a node, and the key pair is not displayed in the drop-down list on the CCE console. - -#. **Advanced ECS Settings** (optional): Click |image3| to show advanced ECS settings. - - - **ECS Group**: An ECS group logically groups ECSs. The ECSs in the same ECS group comply with the same policy associated with the ECS group. - - - **Anti-affinity**: ECSs in an ECS group are deployed on different physical hosts to improve service reliability. - - Select an existing ECS group, or click **Create ECS Group** to create one. After the ECS group is created, click the refresh button. - - - **Resource Tags**: By adding tags to resources, you can classify resources. - - You can create predefined tags in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use predefined tags to improve tag creation and migration efficiency. - - CCE will automatically create the "CCE-Dynamic-Provisioning-Node=node id" tag. A maximum of 5 tags can be added. - - - **Agency**: An agency is created by a tenant administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. To authorize an ECS or BMS to call cloud services, select **Cloud service** as the agency type, click **Select**, and then select **ECS BMS**. - - - **Pre-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. The script is usually used to format data disks. - - - **Post-installation Script**: Enter a maximum of 1,000 characters. - - The script will be executed after Kubernetes software is installed and will not affect the installation. The script is usually used to modify Docker parameters. - - - **Subnet IP Address**: Select **Automatically assign IP address** (recommended) or **Manually assigning IP addresses**. - -#. **Advanced Kubernetes Settings**: (Optional) Click |image4| to show advanced cluster settings. - - - **Max Pods**: maximum number of pods that can be created on a node, including the system's default pods. If the cluster uses the **VPC network model**, the maximum value is determined by the number of IP addresses that can be allocated to containers on each node. - - This limit prevents the node from being overloaded by managing too many pods. For details, see :ref:`Maximum Number of Pods That Can Be Created on a Node `. - - - **Maximum Data Space per Container**: maximum data space that can be used by a container. The value ranges from 10 GB to 500 GB. If the value of this field is larger than the data disk space allocated to Docker resources, the latter will override the value specified here. Typically, 90% of the data disk space is allocated to Docker resources. This parameter is displayed only for clusters of v1.13.10-r0 and later. - -#. **Nodes**: The value cannot exceed the management scale you select when configuring cluster parameters. Set this parameter based on service requirements and the remaining quota displayed on the page. Click |image5| to view the factors that affect the number of nodes to be added (depending on the factor with the minimum value). - -#. Click **Next: Confirm**. After confirming that the configuration is correct, click **Submit**. - - The node list page is displayed. If the node status is **Available**, the node is added successfully. It takes about 6 to 10 minutes to create a node. - - .. note:: - - - Do not delete the security groups and related rules automatically configured during cluster creation. Otherwise, the cluster will exhibit unexpected behavior. - -#. Click **Back to Node List**. The node has been created successfully if it changes to the **Available** state. - - .. note:: - - The allocatable resources are calculated based on the resource request value (**Request**), which indicates the upper limit of resources that can be requested by pods on this node, but does not indicate the actual available resources of the node. - - The calculation formula is as follows: - - - Allocatable CPUs = Total CPUs - Requested CPUs of all pods - Reserved CPUs for other resources - - Allocatable memory = Total memory - Requested memory of all pods - Reserved memory for other resources - -.. |image1| image:: /_static/images/en-us_image_0273156799.png -.. |image2| image:: /_static/images/en-us_image_0220702939.png -.. |image3| image:: /_static/images/en-us_image_0183134608.png -.. |image4| image:: /_static/images/en-us_image_0183134479.png -.. |image5| image:: /_static/images/en-us_image_0250508826.png +After a cluster is created, you can create nodes for the cluster. + +#. Log in to the CCE console. In the navigation pane, choose **Clusters**. Click the target cluster name to access its details page. + +#. In the navigation pane on the left, choose **Nodes**. On the page displayed, click **Create Node**. In the **Node Settings** step, set node parameters by referring to the following table. + + **Compute Settings** + + You can configure the specifications and OS of a cloud server, on which your containerized applications run. + + .. table:: **Table 1** Configuration parameters + + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+====================================================================================================================================================================================================================================+ + | AZ | AZ where the node is located. Nodes in a cluster can be created in different AZs for higher reliability. The value cannot be changed after the node is created. | + | | | + | | You are advised to select **Random** to deploy your node in a random AZ based on the selected node flavor. | + | | | + | | An AZ is a physical region where resources use independent power supply and networks. AZs are physically isolated but interconnected through an internal network. To enhance workload availability, create nodes in different AZs. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Node Type | CCE clusters support Elastic Cloud Servers (ECSs). | + | | | + | | CCE Turbo clusters support Elastic Cloud Servers (ECSs) and bare metal servers (BMSs). | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Container Engine | CCE clusters support Docker and containerd in some scenarios. | + | | | + | | - VPC network clusters of v1.23 and later versions support containerd. Container tunnel network clusters of v1.23.2-r0 and later versions support containerd. | + | | - For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Specifications | Select the node specifications based on service requirements. The available node specifications vary depending on AZs. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | OS | Select an OS type. Different types of nodes support different OSs. | + | | | + | | **Public image**: Select an OS for the node. | + | | | + | | **Private image**: You can use private images. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Node Name | Name of the node. When nodes (ECSs) are created in batches, the value of this parameter is used as the name prefix for each ECS. | + | | | + | | The system generates a default name for you, which can be modified. | + | | | + | | A node name must start with a lowercase letter and cannot end with a hyphen (-). Only digits, lowercase letters, and hyphens (-) are allowed. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Login Mode | - **Key Pair** | + | | | + | | Select the key pair used to log in to the node. You can select a shared key. | + | | | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + **Storage Settings** + + Configure storage resources on a node for the containers running on it. Set the disk size according to site requirements. + + .. table:: **Table 2** Configuration parameters + + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+===============================================================================================================================================================================================================================================================================================+ + | System Disk | System disk used by the node OS. The value ranges from 40 GB to 1,024 GB. The default value is 50 GB. | + | | | + | | **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. **This function is available only in certain regions.** | + | | | + | | - **Encryption** is not selected by default. | + | | - After you select **Encryption**, you can select an existing key in the displayed dialog box. If no key is available, click **View Key List** to create a key. After the key is created, click the refresh icon. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Data Disk | Data disk used by the container runtime and kubelet on the node. | + | | | + | | **At least one data disk is required** for the container runtime and kubelet. **The data disk cannot be deleted or uninstalled. Otherwise, the node will be unavailable.** | + | | | + | | Click **Expand** to set the following parameters: | + | | | + | | - **Allocate Disk Space**: Select this option to define the disk space occupied by the container runtime to store the working directories, container image data, and image metadata. For details about how to allocate data disk space, see :ref:`Data Disk Space Allocation `. | + | | - **Encryption**: Data disk encryption safeguards your data. Snapshots generated from encrypted disks and disks created using these snapshots automatically inherit the encryption function. **This function is available only in certain regions.** | + | | | + | | - **Encryption** is not selected by default. | + | | - After you select **Encryption**, you can select an existing key in the displayed dialog box. If no key is available, click **View Key List** to create a key. After the key is created, click the refresh icon. | + | | | + | | **Adding Multiple Data Disks** | + | | | + | | A maximum of four data disks can be added. By default, raw disks are created without any processing. You can also click **Expand** and select any of the following options: | + | | | + | | - **Default**: By default, a raw disk is created without any processing. | + | | - **Mount Disk**: The data disk is attached to a specified directory. | + | | | + | | **Local Disk Description** | + | | | + | | If the node flavor is disk-intensive or ultra-high I/O, one data disk can be a local disk. | + | | | + | | Local disks may break down and do not ensure data reliability. It is recommended that you store service data in EVS disks, which are more reliable than local disks. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + **Network Settings** + + Configure networking resources to allow node and containerized application access. + + .. table:: **Table 3** Configuration parameters + + +-----------------+-------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +=================+=============================================================================================================+ + | Node Subnet | The node subnet selected during cluster creation is used by default. You can choose another subnet instead. | + +-----------------+-------------------------------------------------------------------------------------------------------------+ + | Node IP Address | IP address of the specified node. By default, the value is randomly allocated. | + +-----------------+-------------------------------------------------------------------------------------------------------------+ + + **Advanced Settings** + + Configure advanced node capabilities such as labels, taints, and startup command. + + .. table:: **Table 4** Advanced configuration parameters + + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+================================================================================================================================================================================================================================================================+ + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | + | | | + | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Resource Tag | You can add resource tags to classify resources. | + | | | + | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | + | | | + | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Taint | This parameter is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | + | | | + | | - **Key**: A key must contain 1 to 63 characters, starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | + | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | + | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | + | | | + | | For details, see :ref:`Managing Node Taints `. | + | | | + | | .. note:: | + | | | + | | For a cluster of v1.19 or earlier, the workload may have been scheduled to a node before the taint is added. To avoid such a situation, select a cluster of v1.19 or later. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Max. Pods | Maximum number of pods that can run on the node, including the default system pods. | + | | | + | | This limit prevents the node from being overloaded with pods. | + | | | + | | This number is also decided by other factors. For details, see :ref:`Maximum Number of Pods That Can Be Created on a Node `. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | ECS Group | An ECS group logically groups ECSs. The ECSs in the same ECS group comply with the same policy associated with the ECS group. | + | | | + | | **Anti-affinity**: ECSs in an ECS group are deployed on different physical hosts to improve service reliability. | + | | | + | | Select an existing ECS group, or click **Add ECS Group** to create one. After the ECS group is created, click the refresh button. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pre-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Post-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed after Kubernetes software is installed and will not affect the installation. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Agency | An agency is created by the account administrator on the IAM console. By creating an agency, you can share your cloud server resources with another account, or entrust a more professional person or team to manage your resources. | + | | | + | | If no agency is available, click **Create Agency** on the right to create one. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. Click **Next: Confirm**. Confirm the configured parameters, specifications. + +#. Click **Submit**. + + The node list page is displayed. If the node status is **Running**, the node is created successfully. It takes about 6 to 10 minutes to create a node. + +#. Click **Back to Node List**. The node is created successfully if it changes to the **Running** state. diff --git a/umn/source/nodes/data_disk_space_allocation.rst b/umn/source/nodes/data_disk_space_allocation.rst deleted file mode 100644 index 4c7d7af..0000000 --- a/umn/source/nodes/data_disk_space_allocation.rst +++ /dev/null @@ -1,79 +0,0 @@ -:original_name: cce_01_0341.html - -.. _cce_01_0341: - -Data Disk Space Allocation -========================== - -When creating a node, you need to configure data disks for the node. - -The data disk is divided into Kubernetes space and user space. The user space defines the space that is not allocated to Kubernetes in the local disk. The Kubernetes space consists of the following two parts: - -- Docker space (90% by default): stores Docker working directories, Docker image data, and image metadata. -- kubelet space (10% by default): stores pod configuration files, secrets, and mounted storage such as emptyDir volumes. - -The Docker space size affects image download and container startup and running. This section describes how the Docker space is used so that you can configure the Docker space accordingly. - -Docker Space Description ------------------------- - -By default, a data disk, 100 GB for example, is divided as follows (depending on the container storage Rootfs): - -- Rootfs (Device Mapper) - - - The **/var/lib/docker** directory is used as the Docker working directory and occupies 20% of the Docker space by default. (Space size of the **/var/lib/docker** directory = Data disk space x 90% x 20%) - - - The thin pool is used to store Docker image data, image metadata, and container data, and occupies 80% of the Docker space by default. (Thin pool space = Data disk space x 90% x 80%) - - The thin pool is dynamically mounted. You can view it by running the **lsblk** command on a node, but not the **df -h** command. - - |image1| - -- Rootfs (OverlayFS): No separate thinpool. The entire Docker space is in the **/var/lib/docker** directory. - - |image2| - -Using rootfs for container storage in CCE - -- CCE cluster: EulerOS 2.9 nodes use OverlayFS, and EulerOS 2.5 nodes use Device Mapper. CentOS 7.6 nodes in clusters earlier than v1.21 use Device Mapper, and use OverlayFS in clusters of v.1.21 and later. - -You can log in to the node and run the **docker info** command to view the storage engine type. - -.. code-block:: - - # docker info - Containers: 20 - Running: 17 - Paused: 0 - Stopped: 3 - Images: 16 - Server Version: 18.09.0 - Storage Driver: devicemapper - -Docker Space and Containers ---------------------------- - -The number of pods and the space configured for each container determine whether the Docker space of a node is sufficient. - -The Docker space should be greater than the total disk space used by containers. Formula: **Docker space** > **Number of containers** x **Available data space for a single container (basesize)** - -**When device mapper is used**, although you can limit the size of the **/home** directory of a single container (to 10 GB by default), all containers on the node still share the thin pool of the node for storage. They are not completely isolated. When the sum of the thin pool space used by certain containers reaches the upper limit, other containers cannot run properly. - -In addition, after a file is deleted in the **/home** directory of the container, the thin pool space occupied by the file is not released immediately. Therefore, even if **basesize** is set to 10 GB, the thin pool space occupied by files keeps increasing until 10 GB when files are created in the container. The space released after file deletion will be reused but after a while. If **the number of containers on the node multiplied by basesize** is greater than the thin pool space size of the node, there is a possibility that the thin pool space has been used up. - -Garbage Collection Policies for Container Images ------------------------------------------------- - -When the Docker space is insufficient, image garbage collection is triggered. - -The policy for garbage collecting images takes two factors into consideration: **HighThresholdPercent** and **LowThresholdPercent**. Disk usage above the high threshold (default: 85%) will trigger garbage collection. The garbage collection will delete least recently used images until the low threshold (default: 80%) has been met. - -Docker Space Configuration Suggestions --------------------------------------- - -- The Docker space should be greater than the total disk space used by containers. Formula: **Docker space** > **Number of containers** x **Available data space for a single container (basesize)** -- You are advised to create and delete files of containerized services in local storage volumes (such as emptyDir and hostPath volumes) or cloud storage directories mounted to the containers. In this way, the thin pool space is not occupied. emptyDir volumes occupy the kubelet space. Therefore, properly plan the size of the kubelet space. -- Docker uses the OverlayFS storage mode. This mode is used in Ubuntu 18.04 nodes in CCE clusters by default. You can deploy services on these nodes to prevent that the disk space occupied by files created or deleted in containers is not released immediately. - -.. |image1| image:: /_static/images/en-us_image_0000001180446397.png -.. |image2| image:: /_static/images/en-us_image_0000001134406294.png diff --git a/umn/source/nodes/deleting_a_node.rst b/umn/source/nodes/deleting_a_node.rst index 5842a6a..2f1033e 100644 --- a/umn/source/nodes/deleting_a_node.rst +++ b/umn/source/nodes/deleting_a_node.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0186.html +:original_name: cce_10_0186.html -.. _cce_01_0186: +.. _cce_10_0186: Deleting a Node =============== @@ -15,8 +15,14 @@ Notes and Constraints - After a CCE cluster is deleted, the ECS nodes in the cluster are also deleted. -Notes ------ +- + + .. important:: + + For clusters of v1.17.11 or later, after a VM is deleted on the ECS console, the corresponding node in the CCE cluster is automatically deleted. + +Precautions +----------- - Deleting a node will lead to pod migration, which may affect services. Perform this operation during off-peak hours. - Unexpected risks may occur during the operation. Back up related data in advance. @@ -26,8 +32,9 @@ Notes Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. In the same row as the node you will delete, choose **More** > **Delete**. -#. In the **Delete Node** dialog box, enter **DELETE** and click **Yes**. +#. Log in to the CCE console and click the cluster name to access the cluster. +#. In the navigation pane, choose **Nodes**. In the same row as the node you will delete, choose **More** > **Delete**. +#. In the **Delete Node** dialog box, click **Yes**. .. note:: diff --git a/umn/source/nodes/formula_for_calculating_the_reserved_resources_of_a_node.rst b/umn/source/nodes/formula_for_calculating_the_reserved_resources_of_a_node.rst deleted file mode 100644 index f388ed2..0000000 --- a/umn/source/nodes/formula_for_calculating_the_reserved_resources_of_a_node.rst +++ /dev/null @@ -1,78 +0,0 @@ -:original_name: cce_01_0178.html - -.. _cce_01_0178: - -Formula for Calculating the Reserved Resources of a Node -======================================================== - -Some of the resources on the node need to run some necessary Kubernetes system components and resources to make the node as part of your cluster. Therefore, the total number of node resources and the number of assignable node resources in Kubernetes are different. The larger the node specifications, the more the containers deployed on the node. Therefore, Kubernetes needs to reserve more resources. - -To ensure node stability, a certain amount of CCE node resources will be reserved for Kubernetes components (such as kubelet, kube-proxy, and docker) based on the node specifications. - -CCE calculates the resources that can be allocated to user nodes as follows: - -**Allocatable resources = Total amount - Reserved amount - Eviction threshold** - -Rules for Reserving Node Memory -------------------------------- - -You can use the following formula calculate how much memory you should reserve for running containers on a node: - -Total reserved amount = Reserved memory for system components + Reserved memory for kubelet to manage pods - -.. table:: **Table 1** Reservation rules for system components - - +----------------------+-------------------------------------------------------------------------+ - | Total Memory (TM) | Reserved Memory for System Components | - +======================+=========================================================================+ - | TM <= 8 GB | 0 MB | - +----------------------+-------------------------------------------------------------------------+ - | 8 GB < TM <= 16 GB | [(TM - 8 GB) x 1024 x 10%] MB | - +----------------------+-------------------------------------------------------------------------+ - | 16 GB < TM <= 128 GB | [8 GB x 1024 x 10% + (TM - 16 GB) x 1024 x 6%] MB | - +----------------------+-------------------------------------------------------------------------+ - | TM > 128 GB | (8 GB x 1024 x 10% + 112 GB x 1024 x 6% + (TM - 128 GB) x 1024 x 2%) MB | - +----------------------+-------------------------------------------------------------------------+ - -.. table:: **Table 2** Reservation rules for kubelet - - +-------------------+---------------------------------+-------------------------------------------------+ - | Total Memory (TM) | Number of Pods | Reserved Memory for kubelet | - +===================+=================================+=================================================+ - | TM <= 2 GB | ``-`` | TM x 25% | - +-------------------+---------------------------------+-------------------------------------------------+ - | TM > 2 GB | 0 < Max. pods on a node <= 16 | 700 MB | - +-------------------+---------------------------------+-------------------------------------------------+ - | | 16 < Max. pods on a node <= 32 | [700 + (Max. pods on a node - 16) x 18.75] MB | - +-------------------+---------------------------------+-------------------------------------------------+ - | | 32 < Max. pods on a node <= 64 | [1024 + (Max. pods on a node - 32) x 6.25] MB | - +-------------------+---------------------------------+-------------------------------------------------+ - | | 64 < Max. pods on a node <= 128 | [1230 + (Max. pods on a node - 64) x 7.80] MB | - +-------------------+---------------------------------+-------------------------------------------------+ - | | Max. pods on a node > 128 | [1740 + (Max. pods on a node - 128) x 11.20] MB | - +-------------------+---------------------------------+-------------------------------------------------+ - -.. important:: - - For a small-capacity node, adjust the maximum number of instances based on the site requirements. Alternatively, when creating a node on the CCE console, you can adjust the maximum number of instances for the node based on the node specifications. - -Rules for Reserving Node CPU ----------------------------- - -.. table:: **Table 3** Node CPU reservation rules - - +----------------------------+------------------------------------------------------------------------+ - | Total CPU Cores (Total) | Reserved CPU Cores | - +============================+========================================================================+ - | Total <= 1 core | Total x 6% | - +----------------------------+------------------------------------------------------------------------+ - | 1 core < Total <= 2 cores | 1 core x 6% + (Total - 1 core) x 1% | - +----------------------------+------------------------------------------------------------------------+ - | 2 cores < Total <= 4 cores | 1 core x 6% + 1 core x 1% + (Total - 2 cores) x 0.5% | - +----------------------------+------------------------------------------------------------------------+ - | Total > 4 cores | 1 core x 6% + 1 core x 1% + 2 cores x 0.5% + (Total - 4 cores) x 0.25% | - +----------------------------+------------------------------------------------------------------------+ - -.. important:: - - CCE reserves an extra 100 MiB for kubelet eviction. diff --git a/umn/source/nodes/index.rst b/umn/source/nodes/index.rst index 2d0d77f..28f26de 100644 --- a/umn/source/nodes/index.rst +++ b/umn/source/nodes/index.rst @@ -1,44 +1,36 @@ -:original_name: cce_01_0183.html +:original_name: cce_10_0183.html -.. _cce_01_0183: +.. _cce_10_0183: Nodes ===== -- :ref:`Overview ` -- :ref:`Creating a Node ` -- :ref:`Creating a Node in a CCE Turbo Cluster ` -- :ref:`Removing a Node ` -- :ref:`Logging In to a Node ` -- :ref:`Managing Node Labels ` -- :ref:`Synchronizing Node Data ` -- :ref:`Configuring Node Scheduling (Tainting) ` -- :ref:`Resetting a Node ` -- :ref:`Deleting a Node ` -- :ref:`Stopping a Node ` -- :ref:`Performing Rolling Upgrade for Nodes ` -- :ref:`Formula for Calculating the Reserved Resources of a Node ` -- :ref:`Creating a Linux LVM Disk Partition for Docker ` -- :ref:`Data Disk Space Allocation ` -- :ref:`Adding a Second Data Disk to a Node in a CCE Cluster ` +- :ref:`Node Overview ` +- :ref:`Creating a Node ` +- :ref:`Adding Nodes for Management ` +- :ref:`Removing a Node ` +- :ref:`Resetting a Node ` +- :ref:`Logging In to a Node ` +- :ref:`Managing Node Labels ` +- :ref:`Managing Node Taints ` +- :ref:`Synchronizing Data with Cloud Servers ` +- :ref:`Deleting a Node ` +- :ref:`Stopping a Node ` +- :ref:`Performing Rolling Upgrade for Nodes ` .. toctree:: :maxdepth: 1 :hidden: - overview + node_overview/index creating_a_node - creating_a_node_in_a_cce_turbo_cluster + adding_nodes_for_management removing_a_node + resetting_a_node logging_in_to_a_node managing_node_labels - synchronizing_node_data - configuring_node_scheduling_tainting - resetting_a_node + managing_node_taints + synchronizing_data_with_cloud_servers deleting_a_node stopping_a_node performing_rolling_upgrade_for_nodes - formula_for_calculating_the_reserved_resources_of_a_node - creating_a_linux_lvm_disk_partition_for_docker - data_disk_space_allocation - adding_a_second_data_disk_to_a_node_in_a_cce_cluster diff --git a/umn/source/nodes/logging_in_to_a_node.rst b/umn/source/nodes/logging_in_to_a_node.rst index 6c43b60..5eb84a9 100644 --- a/umn/source/nodes/logging_in_to_a_node.rst +++ b/umn/source/nodes/logging_in_to_a_node.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0185.html +:original_name: cce_10_0185.html -.. _cce_01_0185: +.. _cce_10_0185: Logging In to a Node ==================== @@ -21,6 +21,8 @@ You can log in to an ECS in either of the following modes: If an ECS has no EIP, log in to the ECS console and click **Remote Login** in the same row as the ECS. + For details, see `Login Using VNC `__. + - SSH This mode applies only to ECSs running Linux. Usually, you can use a remote login tool, such as PuTTY, Xshell, and SecureCRT, to log in to your ECS. If none of the remote login tools can be used, log in to the ECS console and click **Remote Login** in the same row as the ECS to view the connection status and running status of the ECS. @@ -32,12 +34,16 @@ You can log in to an ECS in either of the following modes: .. table:: **Table 1** Linux ECS login modes - +-------------+----------------+---------------------------------------------------------+ - | EIP Binding | On-Premises OS | Connection Method | - +=============+================+=========================================================+ - | Yes | Windows | Use a remote login tool, such as PuTTY or Xshell. | - +-------------+----------------+---------------------------------------------------------+ - | Yes | Linux | Run commands. | - +-------------+----------------+---------------------------------------------------------+ - | Yes/No | Windows/Linux | Use the remote login function available on the console. | - +-------------+----------------+---------------------------------------------------------+ + +-----------------------+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | EIP Binding | On-Premises OS | Connection Method | + +=======================+=======================+===================================================================================================================================================+ + | Yes | Windows | Use a remote login tool, such as PuTTY or XShell. | + | | | | + | | | - SSH key authentication: `Login Using an SSH Key `__ | + +-----------------------+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | Yes | Linux | Run commands. | + | | | | + | | | - SSH password authentication: `Login Using an SSH Password `__ | + +-----------------------+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ + | Yes/No | Windows/Linux | Remote login using the management console: `Login Using VNC `__ | + +-----------------------+-----------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/nodes/managing_node_labels.rst b/umn/source/nodes/managing_node_labels.rst index c90ea7d..22fbd55 100644 --- a/umn/source/nodes/managing_node_labels.rst +++ b/umn/source/nodes/managing_node_labels.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0004.html +:original_name: cce_10_0004.html -.. _cce_01_0004: +.. _cce_10_0004: Managing Node Labels ==================== @@ -13,15 +13,15 @@ Node labels are mainly used in the following scenarios: - Node management: Node labels are used to classify nodes. - Affinity and anti-affinity between a workload and node: - - Some workloads require a large CPU, some require a large memory, some require a large I/O, and other workloads may be affected. In this case, you are advised to add different labels to nodes. When deploying a workload, you can select nodes with specified labels for affinity deployment to ensure the normal operation of the system. Otherwise, node anti-affinity deployment can be used. - - A system can be divided into multiple modules. Each module consists of multiple microservices. To ensure the efficiency of subsequent O&M, you can add a module label to each node so that each module can be deployed on the corresponding node, does not interfere with other modules, and can be easily developed and maintained on its node. + - Different workloads have different resource requirements such as CPU, memory, and I/O. If a workload consumes too many resources in a cluster, other workloads in the same cluster may fail to run properly. In this case, you are advised to add different labels to nodes. When deploying a workload, you can select nodes with specified labels for affinity deployment to ensure the normal operation of the system. Otherwise, node anti-affinity deployment can be used. + - A system can be divided into multiple modules. Each module consists of multiple microservices. To ensure efficient O&M, you can add a module label to each node so that each module can be deployed on the corresponding node. In this way, modules do not interfere with each other and microservices can be easily maintained on their nodes. Inherent Label of a Node ------------------------ -After a node is created, some fixed labels exist and cannot be deleted. For details about these labels, see :ref:`Table 1 `. +After a node is created, some fixed labels exist and cannot be deleted. For details about these labels, see :ref:`Table 1 `. -.. _cce_01_0004__en-us_topic_0000001199181148_table83962234533: +.. _cce_10_0004__table83962234533: .. table:: **Table 1** Inherent label of a node @@ -58,36 +58,24 @@ After a node is created, some fixed labels exist and cannot be deleted. For deta +-----------------------------------------------------+-------------------------------------------------------------+ | os.version | Node OS kernel version | +-----------------------------------------------------+-------------------------------------------------------------+ + | node.kubernetes.io/container-engine | Container engine used by the node. | + +-----------------------------------------------------+-------------------------------------------------------------+ + | accelerator | GPU node labels. | + +-----------------------------------------------------+-------------------------------------------------------------+ + | cce.cloud.com/cce-nodepool | The dedicated label of a node in a node pool. | + +-----------------------------------------------------+-------------------------------------------------------------+ -Adding a Node Label -------------------- +Adding or Deleting a Node Label +------------------------------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. +#. Log in to the CCE console. -#. In the same row as the node for which you will add labels, choose **Operation** > **More** > **Manage Labels**. +#. Click the cluster name, access the cluster details page, and choose **Nodes** in the navigation pane. On the page displayed, select a node and click **Manage Labels and Taints**. -#. In the dialog box displayed, click **Add Label** below the label list, enter the key and value of the label to be added, and click **OK**. +#. In the displayed dialog box, click **Add batch operations** under **Batch Operation**, and then choose **Add/Update** or **Delete**. - As shown in the figure, the key is **deploy_qa** and the value is **true**, indicating that the node is used to deploy the QA (test) environment. + Enter the key and value of the label to be added or deleted, and click **OK**. -#. After the label is added, click **Manage Labels**. Then, you will see the label that you have added. + For example, the key is **deploy_qa** and the value is **true**, indicating that the node is used to deploy the QA (test) environment. -Deleting a Node Label ---------------------- - -Only labels added by users can be deleted. Labels that are fixed on the node cannot be deleted. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. - -#. In the same row as the node for which you will delete labels, choose **Operation** > **More** > **Manage Labels**. - -#. Click **Delete**, and then click **OK** to delete the label. - - **Label updated successfully** is displayed. - -Searching for a Node by Label ------------------------------ - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. -#. In the upper right corner of the node list, click **Search by Label**. -#. Enter a Kubernetes label to find the target node. +#. After the label is added, check the added label in node data. diff --git a/umn/source/nodes/configuring_node_scheduling_tainting.rst b/umn/source/nodes/managing_node_taints.rst similarity index 77% rename from umn/source/nodes/configuring_node_scheduling_tainting.rst rename to umn/source/nodes/managing_node_taints.rst index 681e77a..4d41705 100644 --- a/umn/source/nodes/configuring_node_scheduling_tainting.rst +++ b/umn/source/nodes/managing_node_taints.rst @@ -1,9 +1,9 @@ -:original_name: cce_01_0352.html +:original_name: cce_10_0352.html -.. _cce_01_0352: +.. _cce_10_0352: -Configuring Node Scheduling (Tainting) -====================================== +Managing Node Taints +==================== Taints enable a node to repel specific pods to prevent these pods from being scheduled to the node. @@ -59,7 +59,22 @@ To remove a taint, run the following command with a hyphen (-) added after **NoS Taints: ... -To configure scheduling settings, log in to the CCE console, choose **Resource Management** > **Nodes** in the navigation pane, and choose **More** > **Scheduling settings** in the **Operation** column of a node in the node list. +On the CCE console, you can also manage taints of a node in batches. + +#. Log in to the CCE console. + +#. Click the cluster name, access the cluster details page, and choose **Nodes** in the navigation pane. On the page displayed, select a node and click **Manage Labels and Taints**. + +#. In the displayed dialog box, click **Add batch operations** under **Batch Operation**, choose **Add/Update**, and select **Taint**. + + Enter the key and value of the taint to be added, select the taint effect, and click **OK**. + +#. After the taint is added, check the added taint in node data. + +Node Scheduling Settings +------------------------ + +To configure scheduling settings, log in to the CCE console, click the cluster, choose **Nodes** in the navigation pane, and click **More** > **Disable Scheduling** in the **Operation** column of a node in the node list. In the dialog box that is displayed, click **OK** to set the node to be unschedulable. @@ -74,6 +89,8 @@ This operation will add a taint to the node. You can use kubectl to view the con On the CCE console, perform the same operations again to remove the taint and set the node to be schedulable. +.. _cce_10_0352__section2047442210417: + Tolerations ----------- diff --git a/umn/source/nodes/node_overview/container_engine.rst b/umn/source/nodes/node_overview/container_engine.rst new file mode 100644 index 0000000..340d1c0 --- /dev/null +++ b/umn/source/nodes/node_overview/container_engine.rst @@ -0,0 +1,70 @@ +:original_name: cce_10_0462.html + +.. _cce_10_0462: + +Container Engine +================ + +Introduction to Container Engines +--------------------------------- + +Container engines, one of the most important components of Kubernetes, manage the lifecycle of images and containers. The kubelet interacts with a container runtime through the Container Runtime Interface (CRI). + +.. _cce_10_0462__section159298451879: + +Mapping between Node OSs and Container Engines +---------------------------------------------- + +.. table:: **Table 1** Node OSs and container engines in CCE clusters + + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ + | OS | Kernel Version | Container Engine | Container Storage Rootfs | Container Runtime | + +=============+================+=================================================+=====================================================+===================+ + | CentOS 7.x | 3.x | Docker | Clusters of v1.19.16 and earlier use Device Mapper. | runC | + | | | | | | + | | | Clusters of v1.23 and later support containerd. | Clusters of v1.19.16 and later use OverlayFS. | | + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ + | EulerOS 2.5 | 3.x | Docker | Device Mapper | runC | + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ + | EulerOS 2.9 | 4.x | Docker | OverlayFS | runC | + | | | | | | + | | | Clusters of v1.23 and later support containerd. | | | + +-------------+----------------+-------------------------------------------------+-----------------------------------------------------+-------------------+ + +.. table:: **Table 2** Node OSs and container engines in CCE Turbo clusters + + +-----------------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ + | Node Type | OS | Kernel Version | Container Engine | Container Storage Rootfs | Container Runtime | + +=========================================+=============+================+==================+==========================+===================+ + | Elastic Cloud Server (VM) | CentOS 7.x | 3.x | Docker | OverlayFS | runC | + | | | | | | | + | | EulerOS 2.9 | | | | | + +-----------------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ + | Elastic Cloud Server (physical machine) | EulerOS 2.9 | 4.x | containerd | Device Mapper | Kata | + +-----------------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ + +Differences in Tracing +---------------------- + +- Docker (Kubernetes 1.23 and earlier versions): + + kubelet --> docker shim (in the kubelet process) --> docker --> containerd + +- Docker (community solution for Kubernetes v1.24 or later): + + kubelet --> cri-dockerd (kubelet uses CRI to connect to cri-dockerd) --> docker--> containerd + +- containerd: + + kubelet --> cri plugin (in the containerd process) --> containerd + +Although Docker has added functions such as swarm cluster, docker build, and Docker APIs, it also introduces bugs. Compared with containerd, Docker has one more layer of calling. **Therefore, containerd is more resource-saving and secure.** + +Container Engine Version Description +------------------------------------ + +- Docker + + - EulerOS/CentOS: docker-engine 18.9.0, a Docker version customized for CCE. Security vulnerabilities will be fixed in a timely manner. + +- containerd: 1.4.1 diff --git a/umn/source/nodes/node_overview/data_disk_space_allocation.rst b/umn/source/nodes/node_overview/data_disk_space_allocation.rst new file mode 100644 index 0000000..b094458 --- /dev/null +++ b/umn/source/nodes/node_overview/data_disk_space_allocation.rst @@ -0,0 +1,97 @@ +:original_name: cce_10_0341.html + +.. _cce_10_0341: + +Data Disk Space Allocation +========================== + +This section describes how to allocate data disk space. + +When creating a node, you need to configure a data disk whose capacity is greater than or equal to 100GB for the node. You can click **Expand** to customize the data disk space allocation. + +- :ref:`Allocate Disk Space `: CCE divides the data disk space for container engines and pods. The container engine space stores the Docker/containerd working directories, container images, and image metadata. The pod space stores kubelet components and emptyDir volumes. The available container engine space affects image download and container startup and running. + + - Container engine and container image space (90% by default): functions as the container runtime working directory and stores container image data and image metadata. + - kubelet component and emptyDir volume space (10% by default): stores pod configuration files, secrets, and mounted storage such as emptyDir volumes. + +- :ref:`Allocate Pod Basesize `: indicates the base size of a container, that is, the upper limit of the disk space occupied by each workload pod (including the space occupied by container images). This setting prevents the pods from taking all the disk space available, which may cause service exceptions. It is recommended that the value be smaller than or equal to 80% of the container engine space. This parameter is related to the node OS and container storage rootfs and is not supported in some scenarios. + +.. _cce_10_0341__section10653143445411: + +Setting Container Engine Space +------------------------------ + +A data disk, 100 GB for example, is divided as follows (depending on the container storage rootfs): + +You can log in to the node and run the **docker info** command to view the storage engine type. + +.. code-block:: + + # docker info + Containers: 20 + Running: 17 + Paused: 0 + Stopped: 3 + Images: 16 + Server Version: 18.09.0 + Storage Driver: devicemapper + +- **Rootfs (Device Mapper)** + + By default, 90% of the data disk is the container engine and container image space, which can be divided into the following two parts: + + - The **/var/lib/docker** directory is the Docker working directory and occupies 20% of the container runtime space by default. (Space size of the **/var/lib/docker** directory = **Data disk space x 90% x 20%**) + + - The thin pool stores container image data, image metadata, and container data, and occupies 80% of the container runtime space by default. (Thin pool space = **Data disk space x 90% x 80%**) + + The thin pool is dynamically mounted. You can view it by running the **lsblk** command on a node, but not the **df -h** command. + + |image1| + +- **Rootfs (OverlayFS)** + + No separate thin pool. The entire container engine and container image space (90% of the data disk by default) are in the **/var/lib/docker** directory. + + |image2| + +Using rootfs for container storage in CCE + +- CCE cluster: EulerOS 2.5 nodes use Device Mapper and EulerOS 2.9 nodes use OverlayFS. CentOS 7.x nodes in clusters earlier than v1.19.16 use Device Mapper, and use OverlayFS in clusters of v1.19.16 and later. +- CCE Turbo cluster: BMSs use Device Mapper. ECSs use OverlayFS. + +.. _cce_10_0341__section12119191161518: + +Allocating Basesize for Pods +---------------------------- + +The capability of customizing pod basesize is related to the node OS and container storage rootfs. You can log in to the node and run the **docker info** command to view the container storage rootfs. + +- Device Mapper supports custom pod basesize. The default value is 10 GB. +- When OverlayFS is used, **basesize** is not limited by default. In clusters of latest versions (1.19.16, 1.21.3, 1.23.3, and later), EulerOS 2.9 supports **basesize** if the Docker engine is used. Other OSs do not support **basesize**. + + .. note:: + + In the case of using Docker on EulerOS 2.9 nodes, **basesize** will not take effect if **CAP_SYS_RESOURCE** or **privileged** is configured for a container. + +When configuring **basesize**, consider the maximum number of pods on a node. The container engine space should be greater than the total disk space used by containers. Formula: **the container engine space and container image space (90% by default)** > **Number of containers** x **basesize**. Otherwise, the container engine space allocated to the node may be insufficient and the container cannot be started. + +For nodes that support **basesize**, when Device Mapper is used, although you can limit the size of the **/home** directory of a single container (to 10 GB by default), all containers on the node still share the thin pool of the node for storage. They are not completely isolated. When the sum of the thin pool space used by certain containers reaches the upper limit, other containers cannot run properly. + +In addition, after a file is deleted in the **/home** directory of the container, the thin pool space occupied by the file is not released immediately. Therefore, even if **basesize** is set to 10 GB, the thin pool space occupied by files keeps increasing until 10 GB when files are created in the container. The space released after file deletion will be reused but after a while. If **the number of containers on the node multiplied by basesize** is greater than the thin pool space size of the node, there is a possibility that the thin pool space has been used up. + +Garbage Collection Policies for Container Images +------------------------------------------------ + +When the container engine space is insufficient, image garbage collection is triggered. + +The policy for garbage collecting images takes two factors into consideration: **HighThresholdPercent** and **LowThresholdPercent**. Disk usage above the high threshold (default: 85%) will trigger garbage collection. The garbage collection will delete least recently used images until the low threshold (default: 80%) has been met. + +Recommended Configuration for the Container Engine Space +-------------------------------------------------------- + +- The container engine space should be greater than the total disk space used by containers. Formula: **Container engine space** > **Number of containers** x **basesize** +- You are advised to create and delete files of containerized services in local storage volumes (such as emptyDir and hostPath volumes) or cloud storage directories mounted to the containers. In this way, the thin pool space is not occupied. emptyDir volumes occupy the kubelet space. Therefore, properly plan the size of the kubelet space. +- If OverlayFS is used by in CCE clusters, you can deploy services on these nodes so that the disk space occupied by files created or deleted in containers can be released immediately. + +.. |image1| image:: /_static/images/en-us_image_0000001199021278.png +.. |image2| image:: /_static/images/en-us_image_0000001244101121.png diff --git a/umn/source/nodes/node_overview/formula_for_calculating_the_reserved_resources_of_a_node.rst b/umn/source/nodes/node_overview/formula_for_calculating_the_reserved_resources_of_a_node.rst new file mode 100644 index 0000000..5378bb2 --- /dev/null +++ b/umn/source/nodes/node_overview/formula_for_calculating_the_reserved_resources_of_a_node.rst @@ -0,0 +1,125 @@ +:original_name: cce_10_0178.html + +.. _cce_10_0178: + +Formula for Calculating the Reserved Resources of a Node +======================================================== + +Some of the resources on the node need to run some necessary Kubernetes system components and resources to make the node as part of your cluster. Therefore, the total number of node resources and the number of assignable node resources in Kubernetes are different. The larger the node specifications, the more the containers deployed on the node. Therefore, Kubernetes needs to reserve more resources. + +To ensure node stability, a certain amount of CCE node resources will be reserved for Kubernetes components (such as kubelet, kube-proxy, and docker) based on the node specifications. + +CCE calculates the resources that can be allocated to user nodes as follows: + +**Allocatable resources = Total amount - Reserved amount - Eviction threshold** + +The memory eviction threshold is fixed at 100 MB. + +When the memory consumed by all pods on a node increases, the following behaviors may occur: + +#. If the memory is greater than or equal to the allocatable amount on the node, kubelet is triggered to evict pods. +#. When the memory approaches the allocatable amount and eviction threshold (total minus reserved), OS OOM is triggered. + +Rules for Reserving Node Memory +------------------------------- + +You can use the following formula calculate how much memory you should reserve for running containers on a node: + +Total reserved amount = Reserved memory for system components + Reserved memory for kubelet to manage pods + +.. table:: **Table 1** Reservation rules for system components + + +----------------------+-------------------------------------------------------------------------+ + | Total Memory (TM) | Reserved Memory for System Components | + +======================+=========================================================================+ + | TM <= 8 GB | 0 MB | + +----------------------+-------------------------------------------------------------------------+ + | 8 GB < TM <= 16 GB | [(TM - 8 GB) x 1024 x 10%] MB | + +----------------------+-------------------------------------------------------------------------+ + | 16 GB < TM <= 128 GB | [8 GB x 1024 x 10% + (TM - 16 GB) x 1024 x 6%] MB | + +----------------------+-------------------------------------------------------------------------+ + | TM > 128 GB | (8 GB x 1024 x 10% + 112 GB x 1024 x 6% + (TM - 128 GB) x 1024 x 2%) MB | + +----------------------+-------------------------------------------------------------------------+ + +.. table:: **Table 2** Reservation rules for kubelet + + +-------------------+---------------------------------+-------------------------------------------------+ + | Total Memory (TM) | Number of Pods | Reserved Memory for kubelet | + +===================+=================================+=================================================+ + | TM <= 2 GB | ``-`` | TM x 25% | + +-------------------+---------------------------------+-------------------------------------------------+ + | TM > 2 GB | 0 < Max. pods on a node <= 16 | 700 MB | + +-------------------+---------------------------------+-------------------------------------------------+ + | | 16 < Max. pods on a node <= 32 | [700 + (Max. pods on a node - 16) x 18.75] MB | + +-------------------+---------------------------------+-------------------------------------------------+ + | | 32 < Max. pods on a node <= 64 | [1024 + (Max. pods on a node - 32) x 6.25] MB | + +-------------------+---------------------------------+-------------------------------------------------+ + | | 64 < Max. pods on a node <= 128 | [1230 + (Max. pods on a node - 64) x 7.80] MB | + +-------------------+---------------------------------+-------------------------------------------------+ + | | Max. pods on a node > 128 | [1740 + (Max. pods on a node - 128) x 11.20] MB | + +-------------------+---------------------------------+-------------------------------------------------+ + +.. important:: + + For a small-capacity node, adjust the maximum number of instances based on the site requirements. Alternatively, when creating a node on the CCE console, you can adjust the maximum number of instances for the node based on the node specifications. + +Rules for Reserving Node Memory (v2) +------------------------------------ + +For clusters of **v1.21.4-r0**, **v1.23.3-r0**, or later, the node memory reservation model is optimized to V2 and can be dynamically adjusted using the node pool parameters **kube-reserved-mem** and **system-reserved-mem**. For details, see :ref:`Managing a Node Pool `. + +The total reserved node memory of the V2 model is equal to the sum of that reserved for the OS and that reserved for CCE to manage pods. + +Reserved memory includes basic and floating parts. For the OS, the floating memory depends on the node specifications. For CCE, the floating memory depends on the number of pods on a node. + +.. table:: **Table 3** Rules for reserving node memory (v2) + + +-----------------+--------------------------------------------------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Reserved for | Basic/Floating | Reservation | Used by | + +=================+========================================================+======================+========================================================================================================================================================================================================================================+ + | OS | Basic | 400 MB (fixed) | OS service components such as sshd and systemd-journald. | + +-----------------+--------------------------------------------------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Floating (depending on the node memory) | 25MB/GB | Kernel | + +-----------------+--------------------------------------------------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | CCE | Basic | 500 MB (fixed) | Container engine components, such as kubelet and kube-proxy, when the node is unloaded | + +-----------------+--------------------------------------------------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | | Floating (depending on the number of pods on the node) | Docker: 20 MB/pod | Container engine components when the number of pods increases | + | | | | | + | | | containerd: 5 MB/pod | .. note:: | + | | | | | + | | | | When the v2 model reserves memory for a node by default, the default maximum number of pods is estimated based on the memory. For details, see :ref:`Default Maximum Number of Pods on a Node `. | + +-----------------+--------------------------------------------------------+----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +Rules for Reserving Node CPU +---------------------------- + +.. table:: **Table 4** Node CPU reservation rules + + +----------------------------+------------------------------------------------------------------------+ + | Total CPU Cores (Total) | Reserved CPU Cores | + +============================+========================================================================+ + | Total <= 1 core | Total x 6% | + +----------------------------+------------------------------------------------------------------------+ + | 1 core < Total <= 2 cores | 1 core x 6% + (Total - 1 core) x 1% | + +----------------------------+------------------------------------------------------------------------+ + | 2 cores < Total <= 4 cores | 1 core x 6% + 1 core x 1% + (Total - 2 cores) x 0.5% | + +----------------------------+------------------------------------------------------------------------+ + | Total > 4 cores | 1 core x 6% + 1 core x 1% + 2 cores x 0.5% + (Total - 4 cores) x 0.25% | + +----------------------------+------------------------------------------------------------------------+ + +.. _cce_10_0178__section1057416013173: + +Default Maximum Number of Pods on a Node +---------------------------------------- + +.. table:: **Table 5** Default maximum number of pods on a node + + ============== ============================== + Memory Default Maximum Number of Pods + ============== ============================== + 4 GB 20 + 8 GB 40 + 16 GB 60 + 32 GB 80 + 64 GB or above 110 + ============== ============================== diff --git a/umn/source/nodes/node_overview/index.rst b/umn/source/nodes/node_overview/index.rst new file mode 100644 index 0000000..506163b --- /dev/null +++ b/umn/source/nodes/node_overview/index.rst @@ -0,0 +1,24 @@ +:original_name: cce_10_0180.html + +.. _cce_10_0180: + +Node Overview +============= + +- :ref:`Precautions for Using a Node ` +- :ref:`Container Engine ` +- :ref:`Kata Containers and Common Containers ` +- :ref:`Maximum Number of Pods That Can Be Created on a Node ` +- :ref:`Formula for Calculating the Reserved Resources of a Node ` +- :ref:`Data Disk Space Allocation ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + precautions_for_using_a_node + container_engine + kata_containers_and_common_containers + maximum_number_of_pods_that_can_be_created_on_a_node + formula_for_calculating_the_reserved_resources_of_a_node + data_disk_space_allocation diff --git a/umn/source/nodes/overview.rst b/umn/source/nodes/node_overview/kata_containers_and_common_containers.rst similarity index 61% rename from umn/source/nodes/overview.rst rename to umn/source/nodes/node_overview/kata_containers_and_common_containers.rst index 4654b28..2101440 100644 --- a/umn/source/nodes/overview.rst +++ b/umn/source/nodes/node_overview/kata_containers_and_common_containers.rst @@ -1,108 +1,22 @@ -:original_name: cce_01_0180.html +:original_name: cce_10_0463.html -.. _cce_01_0180: +.. _cce_10_0463: -Overview -======== +Kata Containers and Common Containers +===================================== -Introduction ------------- +The most significant difference is that each Kata container (pod) runs on an independent micro-VM, has an independent OS kernel, and is securely isolated at the virtualization layer. CCE provides container isolation that is more secure than independent private Kubernetes clusters. With isolated OS kernels, computing resources, and networks, pod resources and data will not be preempted and stolen by other pods. -A container cluster consists of a set of worker machines, called nodes, that run containerized applications. A node can be a virtual machine (VM) or a physical machine (PM), depending on your service requirements. The components on a node include kubelet, container runtime, and kube-proxy. - -.. note:: - - A Kubernetes cluster consists of master nodes and node nodes. The nodes described in this section refer to **worker nodes**, the computing nodes of a cluster that run containerized applications. - -CCE uses high-performance Elastic Cloud Servers (ECSs) as nodes to build highly available Kubernetes clusters. - -Notes ------ - -- To ensure node stability, a certain amount of CCE node resources will be reserved for Kubernetes components (such as kubelet, kube-proxy, and docker) based on the node specifications. Therefore, the total number of node resources and the amount of allocatable node resources for your cluster are different. The larger the node specifications, the more the containers deployed on the node. Therefore, more node resources need to be reserved to run Kubernetes components. -- The node networking (such as the VM networking and container networking) is taken over by CCE. You are not allowed to add NICs or change routes. If you modify the networking configuration, the availability of CCE may be affected. - -Node Lifecycle --------------- - -A lifecycle indicates the node statuses recorded from the time when the node is created through the time when the node is deleted or released. - -.. table:: **Table 1** Node statuses - - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Status | Status Attribute | Description | - +=======================+=======================+====================================================================================================================================+ - | Available | Stable state | The node is running properly and is connected to the cluster. | - | | | | - | | | Nodes in this state can provide services. | - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Unavailable | Stable state | The node is not running properly. | - | | | | - | | | Instances in this state no longer provide services. In this case, perform the operations in :ref:`Resetting a Node `. | - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Creating | Intermediate state | The node has been created but is not running. | - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Installing | Intermediate state | The Kubernetes software is being installed on the node. | - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Deleting | Intermediate state | The node is being deleted. | - | | | | - | | | If this state stays for a long time, an exception occurs. | - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Stopped | Stable state | The node is stopped properly. | - | | | | - | | | A node in this state cannot provide services. You can start the node on the ECS console. | - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Error | Stable state | The node is abnormal. | - | | | | - | | | Instances in this state no longer provide services. In this case, perform the operations in :ref:`Resetting a Node `. | - +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ - -Mapping between Node OSs and Container Engines ----------------------------------------------- - -.. table:: **Table 2** Node OSs and container engines in CCE clusters - - +-------------+----------------+------------------+--------------------------------------------------+-------------------+ - | OS | Kernel Version | Container Engine | Container Storage Rootfs | Container Runtime | - +=============+================+==================+==================================================+===================+ - | CentOS 7.x | 3.x | Docker | Clusters of v1.19 and earlier use Device Mapper. | runC | - | | | | | | - | | | | Clusters of v1.21 and later use OverlayFS. | | - +-------------+----------------+------------------+--------------------------------------------------+-------------------+ - | EulerOS 2.5 | | | Device Mapper | | - +-------------+----------------+------------------+--------------------------------------------------+-------------------+ - -.. table:: **Table 3** Node OSs and container engines in CCE Turbo clusters - - +---------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ - | Node Type | OS | Kernel Version | Container Engine | Container Storage Rootfs | Container Runtime | - +=================================+=============+================+==================+==========================+===================+ - | VM | centos 7.x | 3.x | Docker | OverlayFS | Runc | - +---------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ - | BMS in the shared resource pool | EulerOS 2.9 | 4.x | containerd | Device Mapper | Kata | - +---------------------------------+-------------+----------------+------------------+--------------------------+-------------------+ - -.. _cce_01_0180__section7201124294111: - -Secure Containers and Common Containers ---------------------------------------- - -Secure (Kata) containers are distinguished from common containers in a few aspects. - -The most significant difference is that each secure container (pod) runs on an independent micro-VM, has an independent OS kernel, and is securely isolated at the virtualization layer. CCE provides container isolation that is more secure than independent private Kubernetes clusters. With isolated OS kernels, computing resources, and networks, pod resources and data will not be preempted and stolen by other pods. - -You can run common or secure containers on a single node in a CCE Turbo cluster. The differences between them are as follows: +You can run common or Kata containers on a single node in a CCE Turbo cluster. The differences between them are as follows: +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ -| Category | Secure Container (Kata) | Common Container (Docker) | Common Container (containerd) | +| Category | Kata Container | Common Container (Docker) | Common Container (containerd) | +==========================================================================================+=====================================================================================================================================================================================================================================================================================================+========================================================================+========================================================================+ | Node type used to run containers | Bare-metal server (BMS) | VM | VM | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ -| Container engine | containerd | Docker | containerd | -| | | | | -| | | Default value for common containers created on the console. | | +| Container Engine | containerd | Docker | containerd | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ -| Container runtime | Kata | runC | runC | +| Container Runtime | Kata | runC | runC | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ | Container kernel | Exclusive kernel | Sharing the kernel with the host | Sharing the kernel with the host | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ @@ -110,17 +24,19 @@ You can run common or secure containers on a single node in a CCE Turbo cluster. +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ | Container engine storage driver | Device Mapper | OverlayFS2 | OverlayFS | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ -| `Pod overhead `__ | Memory: 50 MiB | None | None | +| `Pod overhead `__ | Memory: 100 MiB | None | None | | | | | | | | CPU: 0.1 cores | | | | | | | | -| | Pod overhead is a feature for accounting for the resources consumed by the pod infrastructure on top of the container requests and limits. For example, if **limits.cpu** is set to 0.5 cores and **limits.memory** to 256 MiB for a pod, the pod will request 0.6-core CPUs and 306 MiB of memory. | | | +| | Pod overhead is a feature for accounting for the resources consumed by the pod infrastructure on top of the container requests and limits. For example, if **limits.cpu** is set to 0.5 cores and **limits.memory** to 256 MiB for a pod, the pod will request 0.6-core CPUs and 356 MiB of memory. | | | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ | Minimal specifications | Memory: 256 MiB | None | None | | | | | | | | CPU: 0.25 cores | | | +| | | | | +| | It is recommended that the ratio of CPU (unit: core) to memory (unit: GiB) be in the range of 1:1 to 1:8. For example, if CPU is 0.5 cores, the memory should range form 512 MiB to 4 GiB. | | | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ -| Container engine CLI | crictl | docker | crictl | +| Container engine CLI | crictl | Docker | crictl | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ | Pod computing resources | The request and limit values must be the same for both CPU and memory. | The request and limit values can be different for both CPU and memory. | The request and limit values can be different for both CPU and memory. | +------------------------------------------------------------------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------+------------------------------------------------------------------------+ diff --git a/umn/source/clusters/cluster_parameters/maximum_number_of_pods_that_can_be_created_on_a_node.rst b/umn/source/nodes/node_overview/maximum_number_of_pods_that_can_be_created_on_a_node.rst similarity index 76% rename from umn/source/clusters/cluster_parameters/maximum_number_of_pods_that_can_be_created_on_a_node.rst rename to umn/source/nodes/node_overview/maximum_number_of_pods_that_can_be_created_on_a_node.rst index 1d56b72..7cf6523 100644 --- a/umn/source/clusters/cluster_parameters/maximum_number_of_pods_that_can_be_created_on_a_node.rst +++ b/umn/source/nodes/node_overview/maximum_number_of_pods_that_can_be_created_on_a_node.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0348.html +:original_name: cce_10_0348.html -.. _cce_01_0348: +.. _cce_10_0348: Maximum Number of Pods That Can Be Created on a Node ==================================================== @@ -8,14 +8,18 @@ Maximum Number of Pods That Can Be Created on a Node The maximum number of pods that can be created on a node is determined by the following parameters: - Number of container IP addresses that can be allocated on a node (alpha.cce/fixPoolMask): Set this parameter when creating a CCE cluster. This parameter is available only when **Network Model** is **VPC network**. + - Maximum number of pods of a node (maxPods): Set this parameter when creating a node. It is a configuration item of kubelet. -- Number of ENIs of a CCE Turbo cluster node: In a CCE Turbo cluster, ECS nodes use sub-ENIs and BMS nodes use ENIs. The maximum number of pods that can be created on a node depends on the number of ENIs that can be used by the node. + +- .. _cce_10_0348__li5286959123611: + + Number of ENIs of a CCE Turbo cluster node: In a CCE Turbo cluster, ECS nodes use sub-ENIs and BMS nodes use ENIs. The maximum number of pods that can be created on a node depends on the number of ENIs that can be used by the node. The maximum number of pods that can be created on a node depends on the minimum value of these parameters. -- For a cluster using the container tunnel network model, the value depends only on :ref:`the maximum number of pods on a node `. -- For a cluster using the VPC network model, the value depends on the minimum value between :ref:`the maximum number of pods on a node ` and :ref:`the number of container IP addresses that can be allocated to a node `, that is, min(maximum number of pods on a node, number of container IP addresses that can be allocated to a node). -- For a cluster (CCE Turbo cluster) using the Cloud Native Network 2.0 model, the value depends on :ref:`the maximum number of pods on a node ` and :ref:`the number of NICs on a CCE Turbo cluster node `. +- For a cluster using the container tunnel network model, the value depends only on :ref:`the maximum number of pods on a node `. +- For a cluster using the VPC network model, the value depends on the minimum value between :ref:`the maximum number of pods on a node ` and :ref:`the number of container IP addresses that can be allocated to a node `, that is, min(maximum number of pods on a node, number of container IP addresses that can be allocated to a node). +- For a cluster (CCE Turbo cluster) using the Cloud Native Network 2.0 model, the value depends on :ref:`the maximum number of pods on a node ` and :ref:`the number of NICs on a CCE Turbo cluster node `. Container Network vs. Host Network ---------------------------------- @@ -25,34 +29,20 @@ When creating a pod, you can select the container network or host network for th - Container network (default): **Each pod is assigned an IP address by the cluster networking add-ons, which occupies the IP addresses of the container network**. - Host network: The pod uses the host network (**hostNetwork: true** needs to be configured for the pod) and occupies the host port. The pod IP address is the host IP address. The pod does not occupy the IP addresses of the container network. To use the host network, you must confirm whether the container ports conflict with the host ports. Do not use the host network unless you know exactly which host port is used by which container. -.. _cce_01_0348__section10770192193714: +.. _cce_10_0348__section10770192193714: Number of Container IP Addresses That Can Be Allocated on a Node ---------------------------------------------------------------- -If you select **VPC network** for **Network Model** when creating a CCE cluster, you also need to set the number of container IP addresses that can be allocated to each node, as shown in the following figure. +If you select **VPC network** for **Network Model** when creating a CCE cluster, you also need to set the number of container IP addresses that can be allocated to each node. This parameter affects the maximum number of pods that can be created on a node. Each pod occupies an IP address (when the container network is used). If the number of available IP addresses is insufficient, pods cannot be created. -|image1| - By default, a node occupies three container IP addresses (network address, gateway address, and broadcast address). Therefore, the number of container IP addresses that can be allocated to a node equals the number of selected container IP addresses minus 3. For example, in the preceding figure, **the number of container IP addresses that can be allocated to a node is 125 (128 - 3)**. -.. _cce_01_0348__section16296174054019: +.. _cce_10_0348__section16296174054019: Maximum Number of Pods on a Node -------------------------------- When creating a node, you can configure the maximum number of pods that can be created on the node. This parameter is a configuration item of kubelet and determines the maximum number of pods that can be created by kubelet. - -|image2| - -.. _cce_01_0348__section491514414286: - -Number of NICs on a CCE Turbo Cluster Node ------------------------------------------- - -For details about the number of NICs on a CCE Turbo cluster node, see :ref:`Cloud Native Network 2.0 `. - -.. |image1| image:: /_static/images/en-us_image_0000001195057213.png -.. |image2| image:: /_static/images/en-us_image_0000001148989534.png diff --git a/umn/source/nodes/node_overview/precautions_for_using_a_node.rst b/umn/source/nodes/node_overview/precautions_for_using_a_node.rst new file mode 100644 index 0000000..63fa3f8 --- /dev/null +++ b/umn/source/nodes/node_overview/precautions_for_using_a_node.rst @@ -0,0 +1,82 @@ +:original_name: cce_10_0461.html + +.. _cce_10_0461: + +Precautions for Using a Node +============================ + +Introduction +------------ + +A container cluster consists of a set of worker machines, called nodes, that run containerized applications. A node can be a virtual machine (VM) or a physical machine (PM), depending on your service requirements. The components on a node include kubelet, container runtime, and kube-proxy. + +.. note:: + + A Kubernetes cluster consists of master nodes and worker nodes. The nodes described in this section refer to **worker nodes**, the computing nodes of a cluster that run containerized applications. + +CCE uses high-performance Elastic Cloud Servers (ECSs) as nodes to build highly available Kubernetes clusters. + +.. _cce_10_0461__section1667513391595: + +Supported Node Specifications +----------------------------- + +Different regions support different node flavors, and node flavors may be changed or sold out. You are advised to log in to the CCE console and check whether the required node flavors are supported on the page for creating nodes. + +Underlying File Storage System of Docker +---------------------------------------- + +- In clusters of v1.15.6 or earlier, the underlying file storage system uses the XFS format. +- In clusters of v1.15.11 or later, after a node is created or reset, the underlying file storage system uses the ext4 format. + +For containerized applications that use the XFS format, pay attention to the impact of the underlying file storage format change. (The sequence of files in different file systems is different. For example, some Java applications reference a JAR package, but the directory contains multiple versions of the JAR package. If the version is not specified, the actual referenced package is determined by the system file.) + +Run the **docker info \| grep "Backing Filesystem"** command to check the format of the Docker underlying storage file used by the current node. + +Paas User and User Group +------------------------ + +When you create a node in a CCE cluster, a paas user or user group is created on the node by default. CCE components and CCE add-ons on a node run as a non-root user (paas user/user group) to minimize the running permission. If the paas user or user group is modified, CCE components and pods may fail to run properly. + +.. important:: + + The normal running of CCE components depends on the paas user or user group. Pay attention to the following requirements: + + - Do not modify the directory permission and container directory permission on a node. + - Do not change the GID and UID of the paas user or user group. + - Do not directly use the paas user or user group to set the user and group to which the service file belongs. + +Node Lifecycle +-------------- + +A lifecycle indicates the node statuses recorded from the time when the node is created through the time when the node is deleted or released. + +.. table:: **Table 1** Node statuses + + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Status | Status Attribute | Description | + +=======================+=======================+====================================================================================================================================+ + | Running | Stable state | The node is running properly and is connected to the cluster. | + | | | | + | | | Nodes in this state can provide services. | + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Unavailable | Stable state | The node is not running properly. | + | | | | + | | | Instances in this state no longer provide services. In this case, perform the operations in :ref:`Resetting a Node `. | + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Creating | Intermediate state | The node has been created but is not running. | + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Installing | Intermediate state | The Kubernetes software is being installed on the node. | + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Deleting | Intermediate state | The node is being deleted. | + | | | | + | | | If this state stays for a long time, an exception occurs. | + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Stopped | Stable state | The node is stopped properly. | + | | | | + | | | A node in this state cannot provide services. You can start the node on the ECS console. | + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Error | Stable state | The node is abnormal. | + | | | | + | | | Instances in this state no longer provide services. In this case, perform the operations in :ref:`Resetting a Node `. | + +-----------------------+-----------------------+------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/nodes/performing_rolling_upgrade_for_nodes.rst b/umn/source/nodes/performing_rolling_upgrade_for_nodes.rst index 51a063d..f6d0645 100644 --- a/umn/source/nodes/performing_rolling_upgrade_for_nodes.rst +++ b/umn/source/nodes/performing_rolling_upgrade_for_nodes.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0276.html +:original_name: cce_10_0276.html -.. _cce_01_0276: +.. _cce_10_0276: Performing Rolling Upgrade for Nodes ==================================== @@ -8,11 +8,11 @@ Performing Rolling Upgrade for Nodes Scenario -------- -In a rolling upgrade, a new node is created, existing workloads are migrated to the new node, and then the old node is deleted. :ref:`Figure 1 ` shows the migration process. +In a rolling upgrade, a new node is created, existing workloads are migrated to the new node, and then the old node is deleted. :ref:`Figure 1 ` shows the migration process. -.. _cce_01_0276__fig1689610598118: +.. _cce_10_0276__fig1689610598118: -.. figure:: /_static/images/en-us_image_0295359661.png +.. figure:: /_static/images/en-us_image_0000001199181340.png :alt: **Figure 1** Workload migration **Figure 1** Workload migration @@ -27,36 +27,13 @@ Notes and Constraints Scenario 1: The Original Node Is in DefaultPool ----------------------------------------------- -#. .. _cce_01_0276__li375022715214: +#. .. _cce_10_0276__li375022715214: - Create a node. - - a. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. - - b. Select the cluster to which the original node belongs. - - c. Click **Create Node Pool**, set the following parameters, and modify other parameters as required. For details about the parameters, see :ref:`Creating a Node Pool `. - - #. **Name**: Enter the name of the new node pool, for example, **nodepool-demo**. - #. **Nodes**: In this example, add one node. - #. **Specifications**: Select node specifications that best suit your needs. - #. **OS**: Select the operating system (OS) of the nodes to be created. - #. **Login Mode**: - - - If the login mode is **Key pair**, select a key pair for logging in to the node and select the check box to acknowledge that you have obtained the key file and that without this file you will not be able to log in to the node. - - A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. - - d. Click **Next: Confirm**. Confirm the node pool configuration and click **Submit**. - - Go back to the node pool list. In the node list, you can view that the new node pool has been created and is in the Normal state. + Create a node pool. For details, see :ref:`Creating a Node Pool `. #. Click the name of the node pool. The IP address of the new node is displayed in the node list. -3. Install and configure kubectl. - - a. In the navigation pane of the CCE console, choose **Resource Management** > **Clusters**, and click **Command Line Tool** > **Kubectl** under the cluster where the original node is located. - b. On the **Kubectl** tab page of the cluster details page, connect to the cluster as prompted. +3. Install and configure kubectl. For details, see :ref:`Connecting to a Cluster Using kubectl `. 4. Migrate the workload. @@ -84,49 +61,27 @@ Scenario 1: The Original Node Is in DefaultPool .. note:: - During workload migration, if node affinity is configured for the workload, the workload keeps displaying a message indicating that the workload is not ready. In this case, click the workload name to go to the workload details page. On the **Scheduling Policies** tab page, delete the affinity configuration of the original node and click **Add Simple Scheduling Policy** to configure the affinity and anti-affinity policies of the new node. For details, see :ref:`Simple Scheduling Policies `. + During workload migration, if node affinity is configured for the workload, the workload keeps displaying a message indicating that the workload is not ready. In this case, click the workload name to go to the workload details page. On the **Scheduling Policies** tab page, delete the affinity configuration of the original node and configure the affinity and anti-affinity policies of the new node. For details, see :ref:`Scheduling Policy (Affinity/Anti-affinity) `. - After the workload is successfully migrated, you can view that the workload is migrated to the node created in :ref:`1 ` on the **Pods** tab page of the workload details page. + After the workload is successfully migrated, you can view that the workload is migrated to the node created in :ref:`1 ` on the **Pods** tab page of the workload details page. 5. Delete the original node. - After the workload is successfully migrated and is running properly, choose **Resource Management** > **Nodes** to delete the original node. + After the workload is successfully migrated and runs properly, delete the original node. Scenario 2: The Original Node Is Not in DefaultPool --------------------------------------------------- -#. .. _cce_01_0276__li1992616214312: +#. .. _cce_10_0276__li1992616214312: - Copy the node pool and add nodes to it. + Copy the node pool and add nodes to it. For details, see :ref:`Copying a Node Pool `. - a. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Node Pools**. - - b. Select the cluster to which the original node belongs. - - In the node pool list, locate the node pool to which the original node belongs. - - c. Click **More** > **Copy** next to the node pool name. On the **Create Node Pool** page, set the following parameters and modify other parameters as required. For details about the parameters, see :ref:`Creating a Node Pool `. - - - **Name**: Enter the name of the new node pool, for example, **nodepool-demo**. - - **Nodes**: In this example, add one node. - - **Specifications**: Select node specifications that best suit your needs. - - **OS**: Select the operating system (OS) of the nodes to be created. - - **Login Mode**: - - - If the login mode is **Key pair**, select a key pair for logging in to the node and select the check box to acknowledge that you have obtained the key file and that without this file you will not be able to log in to the node. - - A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create a key pair**. - - d. Click **Next: Confirm**. Confirm the node pool configuration and click **Submit**. - - Go back to the node pool list. In the node list, you can view that the new node pool has been created and is in the Normal state. - -#. Click the name of the node pool. The IP address of the new node is displayed in the node list. +#. Click **View Node** in the **Operation** column of the node pool. The IP address of the new node is displayed in the node list. 3. Migrate the workload. - a. Click **Edit** on the right of nodepool-demo and set **Taints**. - b. Click **Add Taint**, set **Key** and **Value**, and set **Effect** to **NoExecute**. The value options of **Effect** include **NoSchedule**, **PreferNoSchedule**, or **NoExecute**. + a. Click **Edit** on the right of original node pool and set **Taints**. + b. Enter the key and value of the taint. The options of **Effect** are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. Select **NoExecute** and click **confirm to add**. - **NoSchedule**: Pods that do not tolerate this taint are not scheduled on the node; existing pods are not evicted from the node. - **PreferNoSchedule**: Kubernetes tries to avoid scheduling pods that do not tolerate this taint onto the node. @@ -134,17 +89,17 @@ Scenario 2: The Original Node Is Not in DefaultPool .. note:: - If you need to reset the taint, enter the new values or click **Delete**. + If you need to reset the taint, delete the configured taint. - c. Click **Save**. + c. Click **OK**. d. In the navigation pane of the CCE console, choose **Workloads** > **Deployments**. In the workload list, the status of the workload to be migrated changes from **Running** to **Unready**. If the workload status changes to **Running** again, the migration is successful. .. note:: - During workload migration, if node affinity is configured for the workload, the workload keeps displaying a message indicating that the workload is not ready. In this case, click the workload name to go to the workload details page. On the **Scheduling Policies** tab page, delete the affinity configuration of the original node and click **Add Simple Scheduling Policy** to configure the affinity and anti-affinity policies of the new node. For details, see :ref:`Simple Scheduling Policies `. + During workload migration, if node affinity is configured for the workload, the workload keeps displaying a message indicating that the workload is not ready. In this case, click the workload name to go to the workload details page. On the **Scheduling Policies** tab page, delete the affinity configuration of the original node and configure the affinity and anti-affinity policies of the new node. For details, see :ref:`Scheduling Policy (Affinity/Anti-affinity) `. - After the workload is successfully migrated, you can view that the workload is migrated to the node created in :ref:`1 ` on the **Pods** tab page of the workload details page. + After the workload is successfully migrated, you can view that the workload is migrated to the node created in :ref:`1 ` on the **Pods** tab page of the workload details page. 4. Delete the original node. - After the workload is successfully migrated and is running properly, choose **Resource Management** > **Node Pools** to delete the original node. + After the workload is successfully migrated and runs properly, delete the original node. diff --git a/umn/source/nodes/removing_a_node.rst b/umn/source/nodes/removing_a_node.rst index ddd7b09..045fdb0 100644 --- a/umn/source/nodes/removing_a_node.rst +++ b/umn/source/nodes/removing_a_node.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0338.html +:original_name: cce_10_0338.html -.. _cce_01_0338: +.. _cce_10_0338: Removing a Node =============== @@ -8,19 +8,19 @@ Removing a Node Scenario -------- -Removing a node from a cluster in CCE will re-install the node OS and clear CCE components on the node. +Removing a node from a cluster will re-install the node OS and clear CCE components on the node. -Removing a node will not delete the server (ECS) corresponding to the node. You are advised to remove nodes at off-peak hours to avoid impacts on your services. +Removing a node will not delete the server corresponding to the node. You are advised to remove nodes at off-peak hours to avoid impacts on your services. -After a node is removed from the cluster, the node is still running and incurs fees. +After a node is removed from the cluster, the node is still running. Notes and Constraints --------------------- -- Nodes can be removed only when the cluster is in the Available or Unavailable state. -- A CCE node can be removed only when it is in the Active, Abnormal, or Error state. +- Nodes can be removed only when the cluster is in the **Available** or **Unavailable** state. +- A CCE node can be removed only when it is in the **Active**, **Abnormal**, or **Error** state. - A CCE node in the Active state can have its OS re-installed and CCE components cleared after it is removed. -- If the OS fails to be re-installed after the node is removed, manually re-install the OS. After the re-installation, log in to the node and run the clearance script to clear CCE components. For details, see :ref:`Handling Failed OS Reinstallation `. +- If the OS fails to be re-installed after the node is removed, manually re-install the OS. After the re-installation, log in to the node and run the clearance script to clear CCE components. For details, see :ref:`Handling Failed OS Reinstallation `. Precautions ----------- @@ -33,13 +33,15 @@ Precautions Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. In the same row as the target node, choose **More** > **Remove**. +#. Log in to the CCE console and click the cluster name to access the cluster. -#. In the dialog box displayed, enter **REMOVE**, configure the login information required for re-installing the OS, and click **Yes**. Wait until the node is removed. +#. Choose **Nodes** from the navigation pane and choose **More** > **Remove** in the **Operation** column of the target node. + +#. In the dialog box displayed, configure the login information required for re-installing the OS and click **Yes**. Wait until the node is removed. After the node is removed, workload pods on the node are automatically migrated to other available nodes. -.. _cce_01_0338__section149069481111: +.. _cce_10_0338__section149069481111: Handling Failed OS Reinstallation --------------------------------- diff --git a/umn/source/nodes/resetting_a_node.rst b/umn/source/nodes/resetting_a_node.rst index ddda560..27fac2c 100644 --- a/umn/source/nodes/resetting_a_node.rst +++ b/umn/source/nodes/resetting_a_node.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0003.html +:original_name: cce_10_0003.html -.. _cce_01_0003: +.. _cce_10_0003: Resetting a Node ================ @@ -15,7 +15,7 @@ Resetting a node will reinstall the node OS and the Kubernetes software on the n Notes and Constraints --------------------- -- The cluster version must be v1.13 or later. +- For CCE clusters and CCE Turbo clusters, the version must be v1.13 or later to support node resetting. Notes ----- @@ -31,16 +31,106 @@ Notes Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. In the same row as the node you will reset, choose **More** > **Reset**. +The new console allows you to reset nodes in batches. You can also use private images to reset nodes in batches. -#. In the dialog box displayed, enter **RESET** and reconfigure the key pair for login. +#. Log in to the CCE console. +#. Click the cluster name and access the cluster details page, choose **Nodes** in the navigation pane, and select one or multiple nodes to be reset in the list on the right. Choose **More** > **Reset**. - .. figure:: /_static/images/en-us_image_0000001190302085.png - :alt: **Figure 1** Resetting the selected node +#. In the displayed dialog box, click **Yes**. - **Figure 1** Resetting the selected node + - For nodes in the DefaultPool node pool, the parameter setting page is displayed. Set the parameters by referring to :ref:`4 `. + - For a node you create in a node pool, resetting the node does not support parameter configuration. You can directly use the configuration image of the node pool to reset the node. -#. Click **Yes** and wait until the node is reset. +#. .. _cce_10_0003__li1646785611239: - After the node is reset, pods on it are automatically migrated to other available nodes. + Specify node parameters. + + **Compute Settings** + + .. table:: **Table 1** Configuration parameters + + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=======================================================================================================================================================================================+ + | Specification | Node specifications cannot be modified when you reset a node. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Container Engine | CCE clusters support Docker. | + | | | + | | For a CCE Turbo cluster, both **Docker** and **containerd** are supported. For details, see :ref:`Mapping between Node OSs and Container Engines `. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | OS | **Public image**: Select an OS for the node. | + | | | + | | **Private image**: You can use private images. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Login Mode | - **Key Pair** | + | | | + | | Select the key pair used to log in to the node. You can select a shared key. | + | | | + | | A key pair is used for identity authentication when you remotely log in to a node. If no key pair is available, click **Create Key Pair**. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + **Storage Settings** + + Configure storage resources on a node for the containers running on it. + + .. table:: **Table 2** Configuration parameters + + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+====================================================================================================================================================================================================================================================================================================+ + | System Disk | Directly use the system disk of the cloud server. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Data Disk | **At least one data disk is required** for the container runtime and kubelet. **The data disk cannot be deleted or uninstalled. Otherwise, the node will be unavailable.** | + | | | + | | Click **Expand** and select **Allocate Disk Space** to define the disk space occupied by the container runtime to store the working directories, container image data, and image metadata. For details about how to allocate data disk space, see :ref:`Data Disk Space Allocation `. | + | | | + | | For other data disks, a raw disk is created without any processing by default. You can also click **Expand** and select **Mount Disk** to mount the data disk to a specified directory. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + + **Advanced Settings** + + .. table:: **Table 3** Advanced configuration parameters + + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+================================================================================================================================================================================================================================================================+ + | Kubernetes Label | Click **Add Label** to set the key-value pair attached to the Kubernetes objects (such as pods). A maximum of 20 labels can be added. | + | | | + | | Labels can be used to distinguish nodes. With workload affinity settings, container pods can be scheduled to a specified node. For more information, see `Labels and Selectors `__. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Resource Tag | You can add resource tags to classify resources. | + | | | + | | You can create **predefined tags** in Tag Management Service (TMS). Predefined tags are visible to all service resources that support the tagging function. You can use these tags to improve tagging and resource migration efficiency. | + | | | + | | CCE will automatically create the "CCE-Dynamic-Provisioning-Node=\ *node id*" tag. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Taint | This field is left blank by default. You can add taints to set anti-affinity for the node. A maximum of 10 taints are allowed for each node. Each taint contains the following parameters: | + | | | + | | - **Key**: A key must contain 1 to 63 characters, starting with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. A DNS subdomain name can be used as the prefix of a key. | + | | - **Value**: A value must start with a letter or digit and can contain a maximum of 63 characters, including letters, digits, hyphens (-), underscores (_), and periods (.). | + | | - **Effect**: Available options are **NoSchedule**, **PreferNoSchedule**, and **NoExecute**. | + | | | + | | .. important:: | + | | | + | | NOTICE: | + | | | + | | - If taints are used, you must configure tolerations in the YAML files of pods. Otherwise, scale-up may fail or pods cannot be scheduled onto the added nodes. | + | | - After a node pool is created, you can click **Edit** to modify its configuration. The modification will be synchronized to all nodes in the node pool. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Max. Pods | Maximum number of pods that can run on the node, including the default system pods. | + | | | + | | This limit prevents the node from being overloaded with pods. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pre-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed before Kubernetes software is installed. Note that if the script is incorrect, Kubernetes software may fail to be installed. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Post-installation Command | Enter commands. A maximum of 1,000 characters are allowed. | + | | | + | | The script will be executed after Kubernetes software is installed and will not affect the installation. | + +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. Click **Next: Confirm**. + +#. Click **Submit**. diff --git a/umn/source/nodes/stopping_a_node.rst b/umn/source/nodes/stopping_a_node.rst index b3d27c5..f9d9c1a 100644 --- a/umn/source/nodes/stopping_a_node.rst +++ b/umn/source/nodes/stopping_a_node.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0036.html +:original_name: cce_10_0036.html -.. _cce_01_0036: +.. _cce_10_0036: Stopping a Node =============== @@ -21,22 +21,14 @@ Notes and Constraints Procedure --------- -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. +#. Log in to the CCE console and click the cluster name to access the cluster. -#. In the node list, click the name of the node to be stopped. +#. In the navigation pane, choose **Nodes**. In the right pane, click the name of the node to be stopped. -#. On the node details page displayed, click the node name. +#. In the upper right corner of the ECS details page, click **Stop** in the instance status area. In the displayed dialog box, click **Yes**. - .. figure:: /_static/images/en-us_image_0000001190302087.png - :alt: **Figure 1** Nodes details page + .. figure:: /_static/images/en-us_image_0000001244261119.png + :alt: **Figure 1** ECS details page - **Figure 1** Nodes details page - -#. In the upper right corner of the ECS details page, click **Stop**. In the **Stop ECS** dialog box, click **Yes**. - - - .. figure:: /_static/images/en-us_image_0000001144342232.png - :alt: **Figure 2** ECS details page - - **Figure 2** ECS details page + **Figure 1** ECS details page diff --git a/umn/source/nodes/synchronizing_data_with_cloud_servers.rst b/umn/source/nodes/synchronizing_data_with_cloud_servers.rst new file mode 100644 index 0000000..60381f0 --- /dev/null +++ b/umn/source/nodes/synchronizing_data_with_cloud_servers.rst @@ -0,0 +1,39 @@ +:original_name: cce_10_0184.html + +.. _cce_10_0184: + +Synchronizing Data with Cloud Servers +===================================== + +Scenario +-------- + +Each node in a cluster is a cloud server or physical machine. After a cluster node is created, you can change the cloud server name or specifications as required. + +Some information about CCE nodes is maintained independently from the ECS console. After you change the name, EIP, or specifications of an ECS on the ECS console, you need to **synchronize the ECS information** to the corresponding node on the CCE console. After the synchronization, information on both consoles is consistent. + +Notes and Constraints +--------------------- + +- Data, including the VM status, ECS names, number of CPUs, size of memory, ECS specifications, and public IP addresses, can be synchronized. + + If an ECS name is specified as the Kubernetes node name, the change of the ECS name cannot be synchronized to the CCE console. + +- Data, such as the OS and image ID, cannot be synchronized. (Such parameters cannot be modified on the ECS console.) + +Procedure +--------- + +#. Log in to the CCE console. + +#. Click the cluster name to access the cluster console. Choose **Nodes** in the navigation pane. + +#. Choose **More** > **Sync Server Data** next to the node. + + + .. figure:: /_static/images/en-us_image_0000001243981203.png + :alt: **Figure 1** Synchronizing server data + + **Figure 1** Synchronizing server data + + After the synchronization is complete, the **ECS data synchronization requested** message is displayed in the upper right corner. diff --git a/umn/source/nodes/synchronizing_node_data.rst b/umn/source/nodes/synchronizing_node_data.rst deleted file mode 100644 index e53f7ef..0000000 --- a/umn/source/nodes/synchronizing_node_data.rst +++ /dev/null @@ -1,32 +0,0 @@ -:original_name: cce_01_0184.html - -.. _cce_01_0184: - -Synchronizing Node Data -======================= - -Scenario --------- - -Each node in a cluster is a cloud server or physical machine. After a cluster node is created, you can change the cloud server name or specifications as required. - -Some information about CCE nodes is maintained independently from the ECS console. After you change the name, EIP, billing mode, or specifications of an ECS on the ECS console, you need to **synchronize the ECS information** to the corresponding node on the CCE console. After the synchronization, information on both consoles is consistent. - -Procedure ---------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Nodes**. - -#. In the same row as the node whose data will be synchronized, choose **More** > **Sync Node data**. - - .. note:: - - Alternatively, click the node name, and click **Sync Node Data** in the upper right corner of the node details page. - - - .. figure:: /_static/images/en-us_image_0000001144502022.png - :alt: **Figure 1** Synchronizing node data - - **Figure 1** Synchronizing node data - - After the synchronization is complete, the "Sync success" message is displayed in the upper right corner. diff --git a/umn/source/permissions_management/cluster_permissions_iam-based.rst b/umn/source/permissions_management/cluster_permissions_iam-based.rst index a10763c..93b372a 100644 --- a/umn/source/permissions_management/cluster_permissions_iam-based.rst +++ b/umn/source/permissions_management/cluster_permissions_iam-based.rst @@ -1,40 +1,40 @@ -:original_name: cce_01_0188.html +:original_name: cce_10_0188.html -.. _cce_01_0188: +.. _cce_10_0188: Cluster Permissions (IAM-based) =============================== -CCE cluster permissions are assigned based on IAM **system policies** and **custom policies**. You can use user groups to assign permissions to IAM users. +CCE cluster-level permissions are assigned based on **IAM system policies** and **custom policies**. You can use user groups to assign permissions to IAM users. .. caution:: - **Cluster permissions** are configured only for cluster-related resources (such as clusters and nodes). You must also configure :ref:`namespace permissions ` to operate Kubernetes resources (such as workloads and Services). + **Cluster permissions** are configured only for cluster-related resources (such as clusters and nodes). You must also configure :ref:`namespace permissions ` to operate Kubernetes resources (such as workloads and Services). Prerequisites ------------- -- A user with the Security Administrator role has all IAM permissions except role switching. Only these users can view user groups and their permissions on the **Permissions Management** page on the CCE console. +- A user with the Security Administrator role (for example, your account) has all IAM permissions except role switching. Only these users can view user groups and their permissions on the **Permissions** page on the CCE console. Configuration ------------- -On the CCE console, when you choose **Permissions Management** > **Cluster-Level Permissions** to create a user group, you will be directed to the IAM console to complete the process. After the user group is created and its permissions are configured, you can view the information on the **Cluster-Level Permissions** tab page. This section describes the operations in IAM. +On the CCE console, when you choose **Permissions** > **Cluster-Level Permissions** to create a user group, you will be directed to the IAM console to complete the process. After the user group is created and its permissions are configured, you can view the information on the **Cluster-Level Permissions** tab page. This section describes the operations in IAM. Process Flow ------------ -.. figure:: /_static/images/en-us_image_0000001120226646.png +.. figure:: /_static/images/en-us_image_0000001244261073.png :alt: **Figure 1** Process of assigning CCE permissions **Figure 1** Process of assigning CCE permissions -#. .. _cce_01_0188__li10176121316284: +#. .. _cce_10_0188__li10176121316284: Create a user group and assign permissions to it. - Create a user group on the IAM console, and assign CCE permissions, for example, the CCE Viewer policy to the group. + Create a user group on the IAM console, and assign CCE permissions, for example, the **CCEReadOnlyAccess** policy to the group. .. note:: @@ -42,16 +42,33 @@ Process Flow #. Create a user and add it to a user group. - Create a user on the IAM console and add the user to the group created in :ref:`1 `. + Create a user on the IAM console and add the user to the group created in :ref:`1 `. #. Log in and verify permissions. Log in to the management console as the user you created, and verify that the user has the assigned permissions. - - Log in to the management console and switch to the CCE console. Click **Create** **Cluster** in the upper right corner. If you fail to do so (assuming that only the CCE Viewer role is assigned), the permission control policy takes effect. - - Switch to the console of any other service. If a message appears indicating that you do not have the required permissions to access the service, the CCE Viewer policy takes effect. + - Log in to the management console, switch to the CCE console, and buy a cluster. If you fail to do so (assuming that only the CCEReadOnlyAccess permission is assigned), the permission control policy takes effect. + - Switch to the console of any other service. If a message appears indicating that you do not have the required permissions to access the service, the **CCEReadOnlyAccess** policy takes effect. -.. _cce_01_0188__section1437818291149: +System-defined Roles +-------------------- + +Roles are a type of coarse-grained authorization mechanism that defines service-level permissions based on user responsibilities. Only a limited number of service-level roles are available for authorization. Roles are not ideal for fine-grained authorization and least privilege access. + +The preset system role for CCE in IAM is **CCEAdministrator**. When assigning this role to a user group, you must also select other roles and policies on which this role depends, such as **Tenant Guest**, **Server Administrator**, **ELB Administrator**, **OBS Administrator**, **SFS Administrator**, **SWR Admin**, and **APM FullAccess**. + +System-defined Policies +----------------------- + +The system policies preset for CCE in IAM are **CCEFullAccess** and **CCEReadOnlyAccess**. + +- **CCE FullAccess**: common operation permissions on CCE cluster resources, excluding the namespace-level permissions for the clusters (with Kubernetes RBAC enabled) and the privileged administrator operations, such as agency configuration and cluster certificate generation +- **CCE ReadOnlyAccess**: permissions to view CCE cluster resources, excluding the namespace-level permissions of the clusters (with Kubernetes RBAC enabled) + +.. note:: + + The **CCE Admin** and **CCE Viewer** roles will be discarded soon. You are advised to use **CCE FullAccess** and **CCE ReadOnlyAccess**. Custom Policies --------------- @@ -125,17 +142,6 @@ This section provides examples of common custom CCE policies. ] } -CCE Cluster Permissions and Enterprise Projects ------------------------------------------------ - -CCE supports resource management and permission allocation by cluster and enterprise project. - -Note that: - -- IAM projects are based on physical isolation of resources, whereas enterprise projects provide global logical groups of resources, which better meet the actual requirements of enterprises. In addition, IAM policies can be managed based on enterprise projects. Therefore, you are advised to use enterprise projects for permissions management. -- When there are both IAM projects and enterprise projects, IAM preferentially matches the IAM project policies. -- When creating a cluster or node using purchased cloud resources, ensure that IAM users have been granted the required permissions in the enterprise project to use these resources. Otherwise, the cluster or node may fail to be created. - CCE Cluster Permissions and IAM RBAC ------------------------------------ @@ -171,4 +177,4 @@ When RBAC and IAM policies co-exist, the backend authentication logic for open A Using clusterCert to obtain the cluster kubeconfig: cceadm/teadmin -.. |image1| image:: /_static/images/en-us_image_0000001086743939.png +.. |image1| image:: /_static/images/en-us_image_0000001244101107.png diff --git a/umn/source/permissions_management/example_designing_and_configuring_permissions_for_users_in_a_department.rst b/umn/source/permissions_management/example_designing_and_configuring_permissions_for_users_in_a_department.rst new file mode 100644 index 0000000..684fa2c --- /dev/null +++ b/umn/source/permissions_management/example_designing_and_configuring_permissions_for_users_in_a_department.rst @@ -0,0 +1,86 @@ +:original_name: cce_10_0245.html + +.. _cce_10_0245: + +Example: Designing and Configuring Permissions for Users in a Department +======================================================================== + +Overview +-------- + +The conventional distributed task scheduling mode is being replaced by Kubernetes. CCE allows you to easily deploy, manage, and scale containerized applications in the cloud by providing support for you to use Kubernetes. + +To help enterprise administrators manage resource permissions in clusters, CCE provides multi-dimensional, fine-grained permission policies and management measures. CCE permissions are described as follows: + +- **Cluster-level permissions**: allowing a user group to perform operations on clusters, nodes, node pools, charts, and add-ons. These permissions are assigned based on IAM system policies. +- **Namespace-level permissions**: allowing a user or user group to perform operations on Kubernetes resources, such as workloads, networking, storage, and namespaces. These permissions are assigned based on Kubernetes RBAC. + +Cluster permissions and namespace permissions are independent of each other but must be used together. The permissions set for a user group apply to all users in the user group. When multiple permissions are added to a user or user group, they take effect at the same time (the union set is used). + +Permission Design +----------------- + +The following uses company X as an example. + +Generally, a company has multiple departments or projects, and each department has multiple members. Therefore, you need to design how permissions are to be assigned to different groups and projects, and set a user name for each member to facilitate subsequent user group and permissions configuration. + +The following figure shows the organizational structure of a department in a company and the permissions to be assigned to each member: + +|image1| + +Director: David +--------------- + +David is a department director of company X. To assign him all CCE permissions (both cluster and namespace permissions), you need to create the **cce-admin** user group for David on the IAM console and assign the CCE Administrator role. + +.. note:: + + **CCE Administrator**: This role has all CCE permissions. You do not need to assign other permissions. + + **CCE FullAccess and CCE ReadOnlyAccess**: These policies are related to cluster management permissions and configured only for cluster-related resources (such as clusters and nodes). You must also configure namespace permissions to perform operations on Kubernetes resources (such as workloads and Services). + +O&M Leader: James +----------------- + +James is the O&M team leader of the department. He needs the cluster permissions for all projects and the read-only permissions for all namespaces. + +To assign the permissions, create a user group named **cce-sre** on the IAM console and add James to this user group. Then, assign CCE FullAccess to the user group **cce-sre** to allow it to perform operations on clusters in all projects. + +**Assigning Read-only Permissions on All Clusters and Namespaces to All Team Leaders and Engineers** + +You can create a read-only user group named **read_only** on the IAM console and add users to the user group. + +- Although the development engineers Linda and Peter do not require cluster management permissions, they still need to view data on the CCE console. Therefore, the read-only cluster permission is required. +- For the O&M engineer William, assign the read-only permission on clusters to him in this step. +- The O&M team leader James already has the management permissions on all clusters. You can add him to the **read_only** user group to assign the read-only permission on clusters to him. + +Users James, Robert, William, Linda, and Peter are added to the **read_only** user group. + +Assign the read-only permission on clusters to the user group **read_only**. + +Return to the CCE console, and add the read-only permission on namespaces to the user group **read_only** to which the five users belong. Choose **Permissions** on the CCE console, and assign the read-only policy to the user group **read_only** for each cluster. + +After the setting is complete, James has the cluster management permissions for all projects and the read-only permissions on all namespaces, and the Robert, William, Linda, and Peter have the read-only permission on all clusters and namespaces. + +Development Team Leader: Robert +------------------------------- + +In the previous steps, Robert has been assigned the read-only permission on all clusters and namespaces. Now, assign the administrator permissions on all namespaces to Robert. + +Therefore, you need to assign the administrator permissions on all namespaces in all clusters to Robert. + +O&M Engineer: William +--------------------- + +In the previous steps, William has been assigned the read-only permission on all clusters and namespaces. He also requires the cluster management permissions. Therefore, you can log in to the IAM console, create a user group named **cce-sre-b4** and assign CCE FullAccess to William. + +Now, William has the cluster management permissions and the read-only permission on all namespaces. + +Development Engineers: Linda and Peter +-------------------------------------- + +In the previous steps, Linda and Peter have been assigned the read-only permission on clusters and namespaces. Therefore, you only need to assign the edit policy to them. + +By now, all the required permissions are assigned to the department members. + +.. |image1| image:: /_static/images/en-us_image_0000001256348238.jpg diff --git a/umn/source/permissions_management/index.rst b/umn/source/permissions_management/index.rst index cfc2814..bf98b3a 100644 --- a/umn/source/permissions_management/index.rst +++ b/umn/source/permissions_management/index.rst @@ -1,14 +1,17 @@ -:original_name: cce_01_0164.html +:original_name: cce_10_0164.html -.. _cce_01_0164: +.. _cce_10_0164: Permissions Management ====================== -- :ref:`Permissions Overview ` -- :ref:`Cluster Permissions (IAM-based) ` -- :ref:`Namespace Permissions (Kubernetes RBAC-based) ` -- :ref:`Pod Security Policies ` +- :ref:`Permissions Overview ` +- :ref:`Cluster Permissions (IAM-based) ` +- :ref:`Namespace Permissions (Kubernetes RBAC-based) ` +- :ref:`Example: Designing and Configuring Permissions for Users in a Department ` +- :ref:`Permission Dependency of the CCE Console ` +- :ref:`Pod Security ` +- :ref:`Service Account Token Security Improvement ` .. toctree:: :maxdepth: 1 @@ -17,4 +20,7 @@ Permissions Management permissions_overview cluster_permissions_iam-based namespace_permissions_kubernetes_rbac-based - pod_security_policies + example_designing_and_configuring_permissions_for_users_in_a_department + permission_dependency_of_the_cce_console + pod_security/index + service_account_token_security_improvement diff --git a/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst b/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst index 3e33bb8..8c42416 100644 --- a/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst +++ b/umn/source/permissions_management/namespace_permissions_kubernetes_rbac-based.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0189.html +:original_name: cce_10_0189.html -.. _cce_01_0189: +.. _cce_10_0189: Namespace Permissions (Kubernetes RBAC-based) ============================================= @@ -19,76 +19,77 @@ You can regulate users' or user groups' access to Kubernetes resources in a sing Role and ClusterRole specify actions that can be performed on specific resources. RoleBinding and ClusterRoleBinding bind roles to specific users, user groups, or ServiceAccounts. Illustration: -.. figure:: /_static/images/en-us_image_0000001142984374.png +.. figure:: /_static/images/en-us_image_0000001244261071.png :alt: **Figure 1** Role binding **Figure 1** Role binding On the CCE console, you can assign permissions to a user or user group to access resources in one or multiple namespaces. By default, the CCE console provides the following ClusterRoles: -- view: read-only permission on most resources in all or selected namespaces. -- edit: read and write permissions on most resources in all or selected namespaces. If this ClusterRole is configured for all namespaces, its capability is the same as the O&M permission. -- admin: read and write permissions on most resources in all namespaces, and read-only permission on nodes, storage volumes, namespaces, and quota management. -- cluster-admin: read and write permissions on all resources in all namespaces. +- view (read-only): read-only permission on most resources in all or selected namespaces. +- edit (development): read and write permissions on most resources in all or selected namespaces. If this ClusterRole is configured for all namespaces, its capability is the same as the O&M permission. +- admin (O&M): read and write permissions on most resources in all namespaces, and read-only permission on nodes, storage volumes, namespaces, and quota management. +- cluster-admin (administrator): read and write permissions on all resources in all namespaces. -.. _cce_01_0189__section207514572488: +.. _cce_10_0189__section207514572488: Cluster Permissions (IAM-based) and Namespace Permissions (Kubernetes RBAC-based) --------------------------------------------------------------------------------- -Users with different cluster permissions (assigned using IAM) have different namespace permissions (assigned using Kubernetes RBAC). :ref:`Table 1 ` lists the namespace permissions of different users. +Users with different cluster permissions (assigned using IAM) have different namespace permissions (assigned using Kubernetes RBAC). :ref:`Table 1 ` lists the namespace permissions of different users. -.. _cce_01_0189__cce_01_0187_table886210176509: +.. _cce_10_0189__cce_10_0187_table886210176509: .. table:: **Table 1** Differences in namespace permissions - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | User | Clusters Earlier Than v1.11.7-r2 | Clusters of v1.11.7-r2 | - +================================================+==================================+===================================================================================================================================+ - | User with the Tenant Administrator permissions | All namespace permissions | - Has all namespace permissions when using CCE on the console. | - | | | - Requires Kubernetes RBAC authorization when using CCE via :ref:`kubectl `. | - | | | | - | | | .. note:: | - | | | | - | | | When such a user accesses the CCE console, an administrator group is added. Therefore, the user has all namespace permissions. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | IAM user with the CCE Administrator role | All namespace permissions | - Has all namespace permissions when using CCE on the console. | - | | | - Requires Kubernetes RBAC authorization when using CCE via :ref:`kubectl `. | - | | | | - | | | .. note:: | - | | | | - | | | When such a user accesses the CCE console, an administrator group is added. Therefore, the user has all namespace permissions. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | IAM user with the CCE Viewer role | All namespace permissions | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | IAM user with the Tenant Guest role | All namespace permissions | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------------+-----------------------------------------+ + | User | Clusters of v1.13 and Later | + +=============================================================+=========================================+ + | User with the Tenant Administrator permissions | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE Administrator role | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE FullAccess or CCE ReadOnlyAccess role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the Tenant Guest role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ -Prerequisites -------------- +Precautions +----------- -- Kubernetes RBAC authorization can be used for clusters of v1.11.7-r2 and later. Ensure that you have deployed a supported cluster version. For details about upgrading a cluster, see :ref:`Performing Replace/Rolling Upgrade (v1.13 and Earlier) `. -- After you create a cluster of v1.11.7-r2 or later, CCE automatically assigns the cluster-admin permission to you, which means you have full control on all resources in all namespaces in the cluster. -- A user with the Security Administrator role has all IAM permissions except role switching. Only these users can assign permissions on the **Permissions Management** page on the CCE console. +- Kubernetes RBAC authorization can be used for clusters of v1.11.7-r2 and later. Ensure that you have deployed a supported cluster version. For details about upgrading a cluster, see :ref:`Performing Replace/Rolling Upgrade `. +- After you create a cluster of v1.11.7-r2 or later, CCE automatically assigns the cluster-admin permission to you, which means you have full control on all resources in all namespaces in the cluster. The ID of a federated user changes upon each login and logout. Therefore, the user with the permissions is displayed as deleted. In this case, do not delete the permissions. Otherwise, the authentication fails. You are advised to grant the cluster-admin permission to a user group on CCE and add federated users to the user group. +- A user with the Security Administrator role has all IAM permissions except role switching. For example, an account in the admin user group has this role by default. Only these users can assign permissions on the **Permissions** page on the CCE console. Configuring Namespace Permissions (on the Console) -------------------------------------------------- You can regulate users' or user groups' access to Kubernetes resources in a single namespace based on their Kubernetes RBAC roles. -#. Log in to the CCE console. In the navigation pane, choose **Permissions Management**. -#. On the displayed page, click the **Namespace-Level Permissions** tab. In the upper right corner of the namespace permissions list, select the cluster that contains the namespace whose access will be managed, and click **Add Permissions**. -#. Confirm the cluster name and select the namespace to assign permissions for. For example, select **All namespaces**, the target user or user group, and select the permissions. -#. Click **Create**. +#. Log in to the CCE console. In the navigation pane, choose **Permissions**. -.. _cce_01_0189__section1273861718819: +#. Select a cluster for which you want to add permissions from the drop-down list on the right. + +#. Click **Add Permissions** in the upper right corner. + +#. Confirm the cluster name and select the namespace to assign permissions for. For example, select **All namespaces**, the target user or user group, and select the permissions. + + .. note:: + + If you do not have IAM permissions, you cannot select users or user groups when configuring permissions for other users or user groups. In this case, you can enter a user ID or user group ID. + + Permissions can be customized as required. After selecting **Custom** for **Permission Type**, click **Add Custom Role** on the right of the **Custom** parameter. In the dialog box displayed, enter a name and select a rule. After the custom rule is created, you can select a value from the **Custom** drop-down list box. + +#. Click **OK**. + +.. _cce_10_0189__section1273861718819: Using kubectl to Configure Namespace Permissions ------------------------------------------------ .. note:: - When you access a cluster using kubectl, CCE uses the kubeconfig.json file generated on the cluster for authentication. This file contains user information, based on which CCE determines which Kubernetes resources can be accessed by kubectl. The permissions recorded in a kubeconfig.json file vary from user to user. The permissions that a user has are listed in :ref:`Cluster Permissions (IAM-based) and Namespace Permissions (Kubernetes RBAC-based) `. + When you access a cluster using kubectl, CCE uses the kubeconfig.json file generated on the cluster for authentication. This file contains user information, based on which CCE determines which Kubernetes resources can be accessed by kubectl. The permissions recorded in a kubeconfig.json file vary from user to user. The permissions that a user has are listed in :ref:`Cluster Permissions (IAM-based) and Namespace Permissions (Kubernetes RBAC-based) `. In addition to cluster-admin, admin, edit, and view, you can define Roles and RoleBindings to configure the permissions to add, delete, modify, and query resources, such as pods, Deployments, and Services, in the namespace. @@ -221,12 +222,11 @@ Connect to the cluster as an authorized user. If the PVs and StorageClasses can csi-disk-topology everest-csi-provisioner Delete WaitForFirstConsumer true 75d csi-nas everest-csi-provisioner Delete Immediate true 75d csi-obs everest-csi-provisioner Delete Immediate false 75d - csi-sfsturbo everest-csi-provisioner Delete Immediate true 75d Example: Assigning All Namespace Permissions (admin) ---------------------------------------------------- -The admin role contains all permissions on a namespace. You can assign permissions to users to access one or multiple namespaces. +**admin** has all permissions on namespaces. You can grant this role to a user or user group to manage one or all namespaces. In the following example kubectl output, a RoleBinding has been created, the admin role is bound to the user group **cce-role-group**, and the target namespace is the default namespace. @@ -317,4 +317,4 @@ Connect to the cluster as an authorized user. In this example, you can query res Example: Assigning Permissions for a Specific Kubernetes Resource Object ------------------------------------------------------------------------ -You can assign permissions on a specific Kubernetes resource object, such as pod, Deployment, and Service. For details, see :ref:`Using kubectl to Configure Namespace Permissions `. +You can assign permissions on a specific Kubernetes resource object, such as pod, Deployment, and Service. For details, see :ref:`Using kubectl to Configure Namespace Permissions `. diff --git a/umn/source/permissions_management/permission_dependency_of_the_cce_console.rst b/umn/source/permissions_management/permission_dependency_of_the_cce_console.rst new file mode 100644 index 0000000..57f987c --- /dev/null +++ b/umn/source/permissions_management/permission_dependency_of_the_cce_console.rst @@ -0,0 +1,88 @@ +:original_name: cce_10_0190.html + +.. _cce_10_0190: + +Permission Dependency of the CCE Console +======================================== + +Some CCE permissions policies depend on the policies of other cloud services. To view or use other cloud resources on the CCE console, you need to enable the system policy access control feature of IAM and assign dependency policies for the other cloud services. + +- Dependency policies are assigned based on the CCE FullAccess or CCE ReadOnlyAccess policy you configure. +- Only users and user groups with namespace permissions can gain the view access to resources in clusters of v1.11.7-r2 and later. + + - If a user is granted the view access to all namespaces of a cluster, the user can view all namespaced resources (except secrets) in the cluster. To view secrets in the cluster, the user must gain the **admin** or **edit** role in all namespaces of the cluster. + - HPA policies take effect only after the cluster-admin permissions are configured for the namespace. + - The **view** role within a single namespace allows users to view resources only in the specified namespace. + +Dependency Policy Configuration +------------------------------- + +To grant an IAM user the permissions to view or use resources of other cloud services on the CCE console, you must first grant the CCE Administrator, CCE FullAccess, or CCE ReadOnlyAccess policy to the user group to which the user belongs and then grant the dependency policies listed in :ref:`Table 1 ` to the user. These dependency policies will allow the IAM user to access resources of other cloud services. + +.. note:: + + CCE supports fine-grained permissions configuration, but has the following restrictions: + + - AOM does not support resource-level monitoring. After operation permissions on specific resources are configured using IAM's fine-grained cluster resource management function, IAM users can view cluster monitoring information on the **Dashboard** page of the CCE console, but cannot view the data on non-fine-grained metrics. + +.. _cce_10_0190__table99001215575: + +.. table:: **Table 1** Dependency policies + + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Console Function | Dependent Services | Roles or Policies Required | + +=====================================+==========================================+=====================================================================================================================================================================================================================================================================+ + | Dashboard | Application Operations Management (AOM) | - An IAM user with CCE Administrator assigned can use this function only after AOM FullAccess policy is assigned. | + | | | - IAM users with IAM ReadOnlyAccess, CCE FullAccess, or CCE ReadOnlyAccess assigned can directly use this function. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Workload management | Elastic Load Balance (ELB) | Except in the following cases, the user does not require any additional role to create workloads. | + | | | | + | | Application Performance Management (APM) | - To create a Service using ELB, you must have ELB FullAccess or ELB Administrator plus VPC Administrator assigned. | + | | | - To use a Java probe, you must have AOM FullAccess and APM FullAccess assigned. | + | | Application Operations Management (AOM) | - To create a Service using NAT Gateway, you must have NAT Gateway Administrator assigned. | + | | | - To use OBS, you must have OBS Administrator globally assigned. | + | | NAT Gateway | | + | | | .. note:: | + | | Object Storage Service (OBS) | | + | | | Because of the cache, it takes about 13 minutes for the RBAC policy to take effect after being granted to users, enterprise projects, and user groups. After an OBS-related system policy is granted, it takes about 5 minutes for the policy to take effect. | + | | Scalable File Service (SFS) | | + | | | - To use SFS, you must have SFS FullAccess assigned. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Cluster management | Application Operations Management (AOM) | - Auto scale-out or scale-up requires the AOM FullAccess policy. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Node management | Elastic Cloud Server (ECS) | If the permission assigned to an IAM user is CCE Administrator, creating or deleting a node requires the ECS FullAccess or ECS Administrator policy and the VPC Administrator policy. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Network management | Elastic Load Balance (ELB) | Except in the following cases, the user does not require any additional role to create a Service. | + | | | | + | | NAT Gateway | - To create a Service using ELB, you must have ELB FullAccess or ELB Administrator plus VPC Administrator assigned. | + | | | - To create a Service using NAT Gateway, you must have NAT Administrator assigned. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Storage management | Object Storage Service (OBS) | - To use OBS, you must have OBS Administrator globally assigned. | + | | | | + | | Scalable File Service (SFS) | .. note:: | + | | | | + | | | Because of the cache, it takes about 13 minutes for the RBAC policy to take effect after being granted to users, enterprise projects, and user groups. After an OBS-related system policy is granted, it takes about 5 minutes for the policy to take effect. | + | | | | + | | | - To use SFS, you must have SFS FullAccess assigned. | + | | | | + | | | The CCE Administrator role is required for importing storage devices. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Namespace management | / | / | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Chart management | / | Cloud accounts and the IAM users with CCE Administrator assigned can use this function. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Add-on management | / | Cloud accounts and the IAM users with CCE Administrator, CCE FullAccess, or CCE ReadOnlyAccess assigned can use this function. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Permissions management | / | - For cloud accounts, no additional policy/role is required. | + | | | - IAM users with CCE Administrator or global Security Administrator assigned can use this function. | + | | | - IAM users with CCE FullAccess or CCE ReadOnlyAccess assigned can use this function. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Configuration center | / | - Creating ConfigMaps does not require any additional policy. | + | | | - Viewing secrets requires that the cluster-admin, admin, or edit permission be configured for the namespace. The DEW KeypairFullAccess or DEW KeypairReadOnlyAccess policy must be assigned for dependent services. | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Help center | / | / | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Switching to other related services | Software Repository for Container (SWR) | The CCE console provides links to other related services. To view or use these services, an IAM user must be assigned required permissions for the services. | + | | | | + | | Application Operations Management (AOM) | | + +-------------------------------------+------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/permissions_management/permissions_overview.rst b/umn/source/permissions_management/permissions_overview.rst index 8053ca5..190e97f 100644 --- a/umn/source/permissions_management/permissions_overview.rst +++ b/umn/source/permissions_management/permissions_overview.rst @@ -1,13 +1,13 @@ -:original_name: cce_01_0187.html +:original_name: cce_10_0187.html -.. _cce_01_0187: +.. _cce_10_0187: Permissions Overview ==================== CCE permissions management allows you to assign permissions to IAM users and user groups under your tenant accounts. CCE combines the advantages of Identity and Access Management (IAM) and Kubernetes Role-based Access Control (RBAC) authorization to provide a variety of authorization methods, including IAM fine-grained authorization, IAM token authorization, cluster-scoped authorization, and namespace-wide authorization. -If you need to perform refined permissions management on CCE clusters and related resources, for example, to control the access of employees in different departments to cloud resources, you can perform multi-dimensional permissions management on CCE. +CCE allows you to manage permissions on clusters and related resources at a finer granularity, for example, to control the access of employees in different departments to cloud resources. This section describes the CCE permissions management mechanism and related concepts. If your account has met your service requirements, you can skip the configurations in this chapter. @@ -16,11 +16,11 @@ CCE Permissions Management CCE permissions are described as follows: -- :ref:`Cluster-level permissions `: Cluster-level permissions management evolves out of the system policy authorization feature of IAM. IAM users in the same user group have the same permissions. On IAM, you can configure system policies to describe which IAM user groups can perform which operations on cluster resources. For example, you can grant user group A to create and delete cluster X, add a node, or install an add-on, while granting user group B to view information about cluster X. +- **Cluster-level permissions**: Cluster-level permissions management evolves out of the system policy authorization feature of IAM. IAM users in the same user group have the same permissions. On IAM, you can configure system policies to describe which IAM user groups can perform which operations on cluster resources. For example, you can grant user group A to create and delete cluster X, add a node, or install an add-on, while granting user group B to view information about cluster X. - Cluster-level permissions involve CCE non-Kubernetes APIs and support fine-grained IAM policies and enterprise project management capabilities. + Cluster-level permissions involve CCE non-Kubernetes APIs and support fine-grained IAM policies. -- :ref:`Namespace-level permissions `: You can regulate users' or user groups' access to Kubernetes resources in a single namespace based on their Kubernetes RBAC roles. CCE has also been enhanced based on open-source capabilities. It supports RBAC authorization based on IAM user or user group, and RBAC authentication on access to APIs using IAM tokens. +- **Namespace-level permissions**: You can regulate users' or user groups' access to Kubernetes resources in a single namespace based on their Kubernetes RBAC roles. CCE has also been enhanced based on open-source capabilities. It supports RBAC authorization based on IAM user or user group, and RBAC authentication on access to APIs using IAM tokens. Namespace-level permissions involve CCE Kubernetes APIs and are enhanced based on the Kubernetes RBAC capabilities. Namespace-level permissions can be granted to IAM users or user groups for authentication and authorization, but are independent of fine-grained IAM policies. @@ -29,49 +29,39 @@ CCE permissions are described as follows: In general, you configure CCE permissions in two scenarios. The first is creating and managing clusters and related resources, such as nodes. The second is creating and using Kubernetes resources in the cluster, such as workloads and Services. -.. figure:: /_static/images/en-us_image_0000001168537057.png +.. figure:: /_static/images/en-us_image_0000001199181266.png :alt: **Figure 1** Illustration on CCE permissions **Figure 1** Illustration on CCE permissions These permissions allow you to manage resource users at a finer granularity. -.. _cce_01_0187__section1464135853519: +.. _cce_10_0187__section1464135853519: Cluster Permissions (IAM-based) and Namespace Permissions (Kubernetes RBAC-based) --------------------------------------------------------------------------------- -Users with different cluster permissions (assigned using IAM) have different namespace permissions (assigned using Kubernetes RBAC). :ref:`Table 1 ` lists the namespace permissions of different users. +Users with different cluster permissions (assigned using IAM) have different namespace permissions (assigned using Kubernetes RBAC). :ref:`Table 1 ` lists the namespace permissions of different users. -.. _cce_01_0187__table886210176509: +.. _cce_10_0187__table886210176509: .. table:: **Table 1** Differences in namespace permissions - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | User | Clusters Earlier Than v1.11.7-r2 | Clusters of v1.11.7-r2 | - +================================================+==================================+===================================================================================================================================+ - | User with the Tenant Administrator permissions | All namespace permissions | - Has all namespace permissions when using CCE on the console. | - | | | - Requires Kubernetes RBAC authorization when using CCE via :ref:`kubectl `. | - | | | | - | | | .. note:: | - | | | | - | | | When such a user accesses the CCE console, an administrator group is added. Therefore, the user has all namespace permissions. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | IAM user with the CCE Administrator role | All namespace permissions | - Has all namespace permissions when using CCE on the console. | - | | | - Requires Kubernetes RBAC authorization when using CCE via :ref:`kubectl `. | - | | | | - | | | .. note:: | - | | | | - | | | When such a user accesses the CCE console, an administrator group is added. Therefore, the user has all namespace permissions. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | IAM user with the CCE Viewer role | All namespace permissions | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ - | IAM user with the Tenant Guest role | All namespace permissions | Requires Kubernetes RBAC authorization. | - +------------------------------------------------+----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------------+-----------------------------------------+ + | User | Clusters of v1.13 and Later | + +=============================================================+=========================================+ + | User with the Tenant Administrator permissions | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE Administrator role | All namespace permissions | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the CCE FullAccess or CCE ReadOnlyAccess role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ + | IAM user with the Tenant Guest role | Requires Kubernetes RBAC authorization. | + +-------------------------------------------------------------+-----------------------------------------+ kubectl Permissions ------------------- -You can use :ref:`kubectl ` to access Kubernetes resources in a cluster. +You can use :ref:`kubectl ` to access Kubernetes resources in a cluster. -When you access a cluster using kubectl, CCE uses the kubeconfig.json file generated on the cluster for authentication. This file contains user information, based on which CCE determines which Kubernetes resources can be accessed by kubectl. The permissions recorded in a kubeconfig.json file vary from user to user. The permissions that a user has are listed in :ref:`Table 1 `. +When you access a cluster using kubectl, CCE uses the kubeconfig.json file generated on the cluster for authentication. This file contains user information, based on which CCE determines which Kubernetes resources can be accessed by kubectl. The permissions recorded in a kubeconfig.json file vary from user to user. The permissions that a user has are listed in :ref:`Table 1 `. diff --git a/umn/source/permissions_management/pod_security_policies.rst b/umn/source/permissions_management/pod_security/configuring_a_pod_security_policy.rst similarity index 89% rename from umn/source/permissions_management/pod_security_policies.rst rename to umn/source/permissions_management/pod_security/configuring_a_pod_security_policy.rst index 954fdeb..1ba83b4 100644 --- a/umn/source/permissions_management/pod_security_policies.rst +++ b/umn/source/permissions_management/pod_security/configuring_a_pod_security_policy.rst @@ -1,9 +1,9 @@ -:original_name: cce_01_0275.html +:original_name: cce_10_0275.html -.. _cce_01_0275: +.. _cce_10_0275: -Pod Security Policies -===================== +Configuring a Pod Security Policy +================================= A pod security policy (PSP) is a cluster-level resource that controls sensitive security aspects of the pod specification. The `PodSecurityPolicy `__ object in Kubernetes defines a group of conditions that a pod must comply with to be accepted by the system, as well as the default values of related fields. @@ -11,7 +11,8 @@ By default, the PSP access control component is enabled for clusters of v1.17.17 .. note:: - In addition to the global default PSP, the system configures independent PSPs for system components in namespace kube-system. Modifying the psp-global configuration does not affect pod creation in namespace kube-system. + - In addition to the global default PSP, the system configures independent PSPs for system components in namespace kube-system. Modifying the psp-global configuration does not affect pod creation in namespace kube-system. + - In Kubernetes 1.25, PSP has been removed and replaced by Pod Security Admission. For details, see :ref:`Configuring Pod Security Admission `. Modifying the Global Default PSP -------------------------------- @@ -24,7 +25,7 @@ Before modifying the global default PSP, ensure that a CCE cluster has been crea #. Modify the parameters as required. For details, see `PodSecurityPolicy `__. -.. _cce_01_0275__section155111941177: +.. _cce_10_0275__section155111941177: Example of Enabling Unsafe Sysctls in Pod Security Policy --------------------------------------------------------- @@ -165,6 +166,6 @@ If you have modified the default pod security policy and want to restore the ori name: system:authenticated apiGroup: rbac.authorization.k8s.io -#. Run the following commands: +#. Run the following command: **kubectl apply -f policy.yaml** diff --git a/umn/source/permissions_management/pod_security/configuring_pod_security_admission.rst b/umn/source/permissions_management/pod_security/configuring_pod_security_admission.rst new file mode 100644 index 0000000..c775122 --- /dev/null +++ b/umn/source/permissions_management/pod_security/configuring_pod_security_admission.rst @@ -0,0 +1,117 @@ +:original_name: cce_10_0466.html + +.. _cce_10_0466: + +Configuring Pod Security Admission +================================== + +Before using `Pod Security Admission `__, you need to understand Kubernetes `Pod Security Standards `__. These standards define different isolation levels for pods. They let you define how you want to restrict the behavior of pods in a clear, consistent fashion. Kubernetes offers a built-in pod security admission controller to enforce the pod security standards. Pod security restrictions are applied at the namespace level when pods are created. + +The pod security standard defines three security policy levels: + +.. _cce_10_0466__table0547553318: + +.. table:: **Table 1** Pod security policy levels + + +------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Level | Description | + +============+================================================================================================================================================================================================================+ + | privileged | Unrestricted policy, providing the widest possible level of permissions, typically aimed at system- and infrastructure-level workloads managed by privileged, trusted users, such as CNIs and storage drivers. | + +------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | baseline | Minimally restrictive policy which prevents known privilege escalations, typically targeted at non-critical workloads. This policy disables capabilities such as hostNetwork and hostPID. | + +------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | restricted | Heavily restricted policy, following current Pod hardening best practices. | + +------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +:ref:`Pod security admission ` is applied at the namespace level. The controller restricts the security context and other parameters in the pod or container in the namespace. The privileged policy does not verify the **securityContext** field of the pod and container. The baseline and restricted policies have different requirements on **securityContext**. For details, see `Pod Security Standards `__. + +Setting security context: `Configure a Security Context for a Pod or Container `__ + +Pod Security Admission Labels +----------------------------- + +Kubernetes defines three types of labels for Pod Security Admission (see :ref:`Table 2 `). You can set these labels in a namespace to define the pod security standard level to be used. However, do not change the pod security standard level in system namespaces such as kube-system. Otherwise, pods in the system namespace may be faulty. + +.. _cce_10_0466__table198561415448: + +.. table:: **Table 2** Pod security admission labels + + +---------+----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ + | Mode | Target Object | Description | + +=========+========================================+=======================================================================================================================================+ + | enforce | Pods | Policy violations will cause the pod to be rejected. | + +---------+----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ + | audit | Workloads (such as Deployment and job) | Policy violations will trigger the addition of an audit annotation to the event recorded in the audit log, but are otherwise allowed. | + +---------+----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ + | warn | Workloads (such as Deployment and job) | Policy violations will trigger a user-facing warning, but are otherwise allowed. | + +---------+----------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------+ + +.. note:: + + Pods are often created indirectly, by creating a workload object such as a Deployment or job. To help catch violations early, both the audit and warning modes are applied to the workload resources. However, the enforce mode is applied only to the resulting pod objects. + +.. _cce_10_0466__section4761636371: + +Enforcing Pod Security Admission with Namespace Labels +------------------------------------------------------ + +You can label namespaces to enforce pod security standards. Assume that a namespace is configured as follows: + +.. code-block:: + + apiVersion: v1 + kind: Namespace + metadata: + name: my-baseline-namespace + labels: + pod-security.kubernetes.io/enforce: privileged + pod-security.kubernetes.io/enforce-version: v1.25 + pod-security.kubernetes.io/audit: baseline + pod-security.kubernetes.io/audit-version: v1.25 + pod-security.kubernetes.io/warn: restricted + pod-security.kubernetes.io/warn-version: v1.25 + + # The label can be in either of the following formats: + # pod-security.kubernetes.io/: + # pod-security.kubernetes.io/-version: + # The audit and warn modes inform you of which security behaviors are violated by the load. + +Namespace labels indicate which policy level to apply for the mode. For each mode, there are two labels that determine the policy used: + +- pod-security.kubernetes.io/: + + - : must be **enforce**, **audit**, or **warn**. For details about the modes, see :ref:`Table 2 `. + - : must be **privileged**, **baseline**, or **restricted**. For details about the levels, see :ref:`Table 1 `. + +- pod-security.kubernetes.io/-version: + + Optional, which pins the policy to a given Kubernetes version. + + - : must be **enforce**, **audit**, or **warn**. For details about the modes, see :ref:`Table 2 `. + - : Kubernetes version number. For example, v1.25. You can also use **latest**. + +If pods are deployed in the preceding namespace, the following security restrictions apply: + +#. The verification in the enforce mode is skipped (enforce mode + privileged level). +#. Restrictions related to the baseline policy are verified (audit mode + baseline level). That is, if the pod or container violates the policy, the corresponding event is recorded into the audit log. +#. Restrictions related to the restricted policy are verified (warn mode + restricted level). That is, if the pod or container violates the policy, the user will receive an alarm when creating the pod. + +Migrating from Pod Security Policy to Pod Security Admission +------------------------------------------------------------ + +If you use pod security policies in a cluster earlier than v1.25 and need to replace them with pod security admission in a cluster of v1.25 or later, follow the guide in `Migrate from PodSecurityPolicy to the Built-In PodSecurity Admission Controller `__. + +.. important:: + + #. Pod security admission supports only three isolation modes, less flexible than pod security policies. If you require more control over specific constraints, you will need to use a Validating Admission Webhook to enforce those policies. + #. Pod security admission is a non-mutating admission controller, meaning it will not modify pods before validating them. If you were relying on this aspect of PSP, you will need to either modify the security context in your workloads, or use a Mutating Admission Webhook to make those changes. + #. PSP lets you bind different policies to different service accounts. This approach has many pitfalls and is not recommended, but if you require this feature anyway you will need to use a third-party webhook instead. + #. Do not apply pod security admission to namespaces where CCE components, such as kube-system, kube-public, and kube-node-lease, are deployed. Otherwise, CCE components and add-on functions will be abnormal. + +Reference +--------- + +- `Pod Security Admission `__ +- `Mapping PodSecurityPolicies to Pod Security Standards `__ +- `Enforce Pod Security Standards with Namespace Labels `__ +- `Enforce Pod Security Standards by Configuring the Built-in Admission Controller `__ diff --git a/umn/source/permissions_management/pod_security/index.rst b/umn/source/permissions_management/pod_security/index.rst new file mode 100644 index 0000000..5945ee3 --- /dev/null +++ b/umn/source/permissions_management/pod_security/index.rst @@ -0,0 +1,16 @@ +:original_name: cce_10_0465.html + +.. _cce_10_0465: + +Pod Security +============ + +- :ref:`Configuring a Pod Security Policy ` +- :ref:`Configuring Pod Security Admission ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + configuring_a_pod_security_policy + configuring_pod_security_admission diff --git a/umn/source/permissions_management/service_account_token_security_improvement.rst b/umn/source/permissions_management/service_account_token_security_improvement.rst new file mode 100644 index 0000000..3e0dbbf --- /dev/null +++ b/umn/source/permissions_management/service_account_token_security_improvement.rst @@ -0,0 +1,41 @@ +:original_name: cce_10_0477_0.html + +.. _cce_10_0477_0: + +Service Account Token Security Improvement +========================================== + +In clusters earlier than v1.21, a token is obtained by mounting the secret of the service account to a pod. Tokens obtained this way are permanent. This approach is no longer recommended starting from version 1.21. Service accounts will stop auto creating secrets in clusters from version 1.25. + +In clusters of version 1.21 or later, you can use the `TokenRequest `__ API to obtain the token and use the projected volume to mount the token to the pod. Such tokens are valid for a fixed period (one hour by default). Before expiration, Kubelet refreshes the token to ensure that the pod always uses a valid token. When the mounting pod is deleted, the token automatically becomes invalid. This approach is implemented by the `BoundServiceAccountTokenVolume `__ feature to improve the token security of the service account. Kubernetes clusters of v1.21 and later enables this approach by default. + +For smooth transition, the community extends the token validity period to one year by default. After one year, the token becomes invalid, and clients that do not support certificate reloading cannot access the API server. It is recommended that clients of earlier versions be upgraded as soon as possible. Otherwise, service faults may occur. + +If you use a Kubernetes client of a to-be-outdated version, the certificate reloading may fail. Versions of officially supported Kubernetes client libraries able to reload tokens are as follows: + +- Go: >= v0.15.7 +- Python: >= v12.0.0 +- Java: >= v9.0.0 +- Javascript: >= v0.10.3 +- Ruby: master branch +- Haskell: v0.3.0.0 +- C#: >= 7.0.5 + +For details, visit https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/1205-bound-service-account-tokens. + +.. note:: + + If you need a token that never expires, you can also `manually manage secrets for service accounts `__. Although a permanent service account token can be manually created, you are advised to use a short-lived token by calling the `TokenRequest `__ API for higher security. + +Diagnosis +--------- + +Run the following steps to check your CCE clusters of v1.21 and later: + +#. Use kubectl to connect to the cluster and run the **kubectl get --raw "/metrics" \| grep stale** command to query the metrics. Check the metric named **serviceaccount_stale_tokens_total**. + + If the value is greater than 0, some workloads in the cluster may be using an earlier client-go version. In this case, check whether this problem occurs in your deployed applications. If yes, upgrade client-go to the version specified by the community as soon as possible. The version must be at least two major versions of the CCE cluster. For example, if your cluster version is 1.23, the Kubernetes dependency library version must be at least 1.19. + + |image1| + +.. |image1| image:: /_static/images/en-us_image_0000001402494682.png diff --git a/umn/source/product_bulletin/cce_cluster_version_release_notes.rst b/umn/source/product_bulletin/cce_cluster_version_release_notes.rst index 26d79bd..50c6b0c 100644 --- a/umn/source/product_bulletin/cce_cluster_version_release_notes.rst +++ b/umn/source/product_bulletin/cce_cluster_version_release_notes.rst @@ -7,10 +7,25 @@ CCE Cluster Version Release Notes To ensure that stable and reliable Kubernetes versions are available during your use of CCE, CCE provides the Kubernetes version support mechanism. A new supported version will be released every half a year with a support period of one year. You must upgrade your Kubernetes clusters before the support period ends. +V1.25 +----- + +.. table:: **Table 1** Feature description of clusters of v1.25 + + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Kubernetes Version | Description | + +===================================+=========================================================================================================================================================================================================================================================================================================================================================================================================================================+ + | v1.25 | Main features: | + | | | + | | - Incorporates features of Kubernetes v1.25. | + | | - PodSecurityPolicy is replaced by Pod Security Admission. | + | | - The LegacyServiceAccountTokenNoAutoGeneration feature is in beta state. By default, this feature is enabled and no more secret token will be automatically generated for the service account. If you want to use a token that never expires, you need to create a secret and mount it. For details, see `Service account token Secrets `__. | + +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + V1.23 ----- -.. table:: **Table 1** Feature description of clusters of v1.23 +.. table:: **Table 2** Feature description of clusters of v1.23 +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Kubernetes Version | Description | @@ -24,7 +39,7 @@ V1.23 V1.21 ----- -.. table:: **Table 2** Feature description of clusters of v1.21 +.. table:: **Table 3** Feature description of clusters of v1.21 +-----------------------------------+------------------------------------------------------------------------------------------------------------------+ | Kubernetes Version | Description | @@ -39,7 +54,7 @@ V1.21 V1.19 ----- -.. table:: **Table 3** Feature description of clusters of v1.19 +.. table:: **Table 4** Feature description of clusters of v1.19 +-----------------------------------+------------------------------------------------------------------+ | Kubernetes Version | Description | @@ -60,7 +75,7 @@ V1.19 V1.17 ----- -.. table:: **Table 4** Feature description of clusters of v1.17 +.. table:: **Table 5** Feature description of clusters of v1.17 +-----------------------------------+-------------------------------------------------+ | Kubernetes | Description | @@ -74,28 +89,28 @@ V1.17 V1.15 ----- -.. table:: **Table 5** Feature description of clusters of v1.15 +.. table:: **Table 6** Feature description of clusters of v1.15 - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Kubernetes | Description | - +===================================+==============================================================================================================================================================================================================================================================================================+ - | v1.15.11-r1 | Main features: | - | | | - | | - EulerOS 2.5 is supported. | - | | - Incorporates features of Kubernetes v1.15.11. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | v1.15.6-r1 | Main features: | - | | | - | | - EulerOS 2.5 is supported. | - | | - Support for GPU-accelerated. | - | | - Kubernetes parameters can be dynamically configured. For details, see `Configuring Kubernetes Parameters `__ and `Managing a Node Pool `__. | - | | - CCE storage supports CSI-based cloud-native container storage systems. For details, see `everest `__. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Kubernetes | Description | + +===================================+================================================================================================================================================================================+ + | v1.15.11-r1 | Main features: | + | | | + | | - EulerOS 2.5 is supported. | + | | - Incorporates features of Kubernetes v1.15.11. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | v1.15.6-r1 | Main features: | + | | | + | | - EulerOS 2.5 is supported. | + | | - Support for GPU-accelerated. | + | | - Kubernetes parameters can be dynamically configured. For details, see :ref:`Configuring Kubernetes Parameters ` and :ref:`Managing a Node Pool `. | + | | - CCE storage supports CSI-based cloud-native container storage systems. For details, see :ref:`everest `. | + +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ V1.13 ----- -.. table:: **Table 6** Feature description of clusters of v1.13 +.. table:: **Table 7** Feature description of clusters of v1.13 +-----------------------------------+--------------------------------------------------+ | Kubernetes (CCE Enhanced Version) | Description | @@ -110,7 +125,7 @@ V1.13 V1.11 and Earlier Versions -------------------------- -.. table:: **Table 7** Feature description of clusters of v1.11 or earlier +.. table:: **Table 8** Feature description of clusters of v1.11 or earlier +-----------------------------------+-------------------------------------------------+ | Kubernetes (CCE Enhanced Version) | Description | diff --git a/umn/source/product_bulletin/index.rst b/umn/source/product_bulletin/index.rst index e7ac65b..beba27b 100644 --- a/umn/source/product_bulletin/index.rst +++ b/umn/source/product_bulletin/index.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0236.html +:original_name: cce_bulletin_0000.html -.. _cce_01_0236: +.. _cce_bulletin_0000: Product Bulletin ================ @@ -10,6 +10,7 @@ Product Bulletin - :ref:`CCE Cluster Version Release Notes ` - :ref:`OS Patch Notes for Cluster Nodes ` - :ref:`Security Vulnerability Responses ` +- :ref:`Service Account Token Security Improvement ` .. toctree:: :maxdepth: 1 @@ -20,3 +21,4 @@ Product Bulletin cce_cluster_version_release_notes os_patch_notes_for_cluster_nodes security_vulnerability_responses/index + service_account_token_security_improvement diff --git a/umn/source/product_bulletin/kubernetes_version_support_mechanism.rst b/umn/source/product_bulletin/kubernetes_version_support_mechanism.rst index 61e8603..e2d1bfc 100644 --- a/umn/source/product_bulletin/kubernetes_version_support_mechanism.rst +++ b/umn/source/product_bulletin/kubernetes_version_support_mechanism.rst @@ -5,51 +5,52 @@ Kubernetes Version Support Mechanism ==================================== -This section describes the Kubernetes version support mechanism of CCE. +This section explains versioning in CCE, and the policies for Kubernetes version support. -Cluster Version Description ---------------------------- +Version Description +------------------- -**Version number**: The format is **x.y.z-r{n}**, where **x.y** is the major version and **z** is the minor version. If the version number is followed by **-r{n}**, the version is a patch version, for example, v1.15.11-r1. +**Version number**: The format is **x.y.z**, where **x.y** is the major version and **z** is the minor version. If the version number is followed by **-r**, the version is a patch version, for example, v1.15.6-r1. |image1| -.. note:: +Version Requirements +-------------------- - Starting from Kubernetes 1.21, CCE only displays the major version number, for example, v1.21. +.. important:: -**Offline**: After a version is brought offline, a cluster of this version cannot be created on the CCE console and no new features will be released for the clusters of this version. + **Offline**: After a version is brought offline, a cluster of this version cannot be created on the CCE console and no new features will be released for the clusters of this version. -**Disuse**: After a version is disused, CCE will no longer provide technical support for the version, including supporting new features, backporting Kubernetes bug fixes, fixing vulnerabilities, and upgrading to new versions. + **Obsolete**: CCE will no longer provide support for this version, including release of new functions, community bug fixes, vulnerability management, and upgrade. -Lifecycle ---------- - -CCE releases only odd major Kubernetes versions, such as v1.23, v1.21, v1.19, and v1.17. The specific version support policies in different scenarios are as follows: +CCE releases only odd major Kubernetes versions, such as v1.25, v1.23, and v1.21. The specific version support policies in different scenarios are as follows: - Cluster creation - CCE provides you with two major Kubernetes versions of clusters, for example, v1.23 and v1.21. For example, after v1.23 is brought online for commercial use, v1.19 is brought offline synchronously. In this case, the cluster of this version cannot be created on the console. + CCE allows you to create clusters of two latest major Kubernetes versions, for example, v1.25 and v1.23. When v1.25 is commercially available, support for earlier versions (such as v1.21) will be removed. In this case, you will not be able to create clusters of v1.21 on the CCE console. - Cluster maintenance - CCE maintains four major Kubernetes versions, such as v1.17, v1.19, v1.21 and v1.23. For example, when v1.23 is put into commercial use, earlier versions such as v1.15 will be disused. + CCE maintains clusters of four major Kubernetes versions at most, such as v1.25, v1.23, v1.21, and v1.19. For example, after v1.25 is commercially available, support for v1.17 will be removed. |image2| - Cluster upgrade - CCE supports the upgrade of three major Kubernetes versions, such as v1.21, v1.19, and v1.17. For example, after v1.23 is put into commercial use, clusters of versions earlier than v1.17, such as v1.15, cannot be upgraded any more. + CCE allows you to upgrade clusters of **three major versions** at the same time. Clusters of 1.19 and later versions can be upgraded skipping one major version at most (for example, from 1.19 directly to 1.23). Each version is maintained for one year. For example, after v1.25 is available, support for earlier versions (such as v1.17) will be removed. You are advised to upgrade your Kubernetes cluster before the maintenance period ends. + + - Cluster version upgrade: After the latest major version (for example, v1.25) is available, CCE allows you to upgrade clusters to the last stable version of the second-latest major version, for example, v1.23. For details, see :ref:`Upgrade Overview `. + - Cluster patch upgrade: For existing clusters running on the live network, if there are major Kubernetes issues or vulnerabilities, CCE will perform the patch upgrade on these clusters in the background. Users are unaware of the patch upgrade. If the patch upgrade has adverse impact on user services, CCE will release a notice one week in advance. Version Release Cycle --------------------- -CCE follows the Kubernetes community to release major versions. To be specific, a new CCE version is released about six months after a new Kubernetes version is released in the community. +Kubernetes releases a major version in about four months. CCE will provide support to mirror the new Kubernetes version in about seven months after the version release. Version Constraints ------------------- After a cluster is upgraded, it cannot be rolled back to the source version. -.. |image1| image:: /_static/images/en-us_image_0000001178192666.png -.. |image2| image:: /_static/images/en-us_image_0000001223152415.png +.. |image1| image:: /_static/images/en-us_image_0000001460905374.png +.. |image2| image:: /_static/images/en-us_image_0000001461224886.png diff --git a/umn/source/product_bulletin/os_patch_notes_for_cluster_nodes.rst b/umn/source/product_bulletin/os_patch_notes_for_cluster_nodes.rst index 14152ca..3992eca 100644 --- a/umn/source/product_bulletin/os_patch_notes_for_cluster_nodes.rst +++ b/umn/source/product_bulletin/os_patch_notes_for_cluster_nodes.rst @@ -8,17 +8,16 @@ OS Patch Notes for Cluster Nodes Nodes in Hybrid Clusters ------------------------ -CCE nodes in Hybrid clusters can run on EulerOS 2.2, EulerOS 2.5, EulerOS 2.9 and CentOS 7.7. The following table lists the supported patches for these OSs. +CCE nodes in Hybrid clusters can run on EulerOS 2.5, EulerOS 2.9 and CentOS 7.7. The following table lists the supported patches for these OSs. .. table:: **Table 1** Node OS patches ========================= ========================================= OS Patch ========================= ========================================= - EulerOS release 2.0 (SP2) 3.10.0-327.62.59.83.h128.x86_64 - EulerOS release 2.0 (SP5) 3.10.0-862.14.1.5.h591.eulerosv2r7.x86_64 - EulerOS release 2.0 (SP9) 4.18.0-147.5.1.6.h541.eulerosv2r9.x86_64 - CentOS Linux release 7.7 3.10.0-1062.18.1.el7.x86_64 + EulerOS release 2.0 (SP5) 3.10.0-862.14.1.5.h687.eulerosv2r7.x86_64 + EulerOS release 2.0 (SP9) 4.18.0-147.5.1.6.h766.eulerosv2r9.x86_64 + CentOS Linux release 7.7 3.10.0-1160.76.1.el7.x86_64 ========================= ========================================= The OS patches and verification results will be updated from time to time. You can update the operating system based on your needs. diff --git a/umn/source/product_bulletin/security_vulnerability_responses/index.rst b/umn/source/product_bulletin/security_vulnerability_responses/index.rst index 520e807..08a793a 100644 --- a/umn/source/product_bulletin/security_vulnerability_responses/index.rst +++ b/umn/source/product_bulletin/security_vulnerability_responses/index.rst @@ -5,6 +5,7 @@ Security Vulnerability Responses ================================ +- :ref:`Vulnerability Fixing Policies ` - :ref:`Linux Polkit Privilege Escalation Vulnerability (CVE-2021-4034) ` - :ref:`Notice on Fixing Linux Kernel SACK Vulnerabilities ` @@ -12,5 +13,6 @@ Security Vulnerability Responses :maxdepth: 1 :hidden: + vulnerability_fixing_policies linux_polkit_privilege_escalation_vulnerability_cve-2021-4034 notice_on_fixing_linux_kernel_sack_vulnerabilities diff --git a/umn/source/product_bulletin/security_vulnerability_responses/vulnerability_fixing_policies.rst b/umn/source/product_bulletin/security_vulnerability_responses/vulnerability_fixing_policies.rst new file mode 100644 index 0000000..66fe036 --- /dev/null +++ b/umn/source/product_bulletin/security_vulnerability_responses/vulnerability_fixing_policies.rst @@ -0,0 +1,27 @@ +:original_name: cce_bulletin_0011.html + +.. _cce_bulletin_0011: + +Vulnerability Fixing Policies +============================= + +Cluster Vulnerability Fixing SLA +-------------------------------- + +- High-risk vulnerabilities: + + - CCE fixes vulnerabilities as soon as possible after the Kubernetes community detects them and releases fixing solutions. The fixing policies are the same as those of the community. + - Emergency vulnerabilities of the operating system are released according to the operating system fixing policies and procedure. Generally, after a fixing solution is provided, you need to fix the vulnerabilities by yourself. + +- Other vulnerabilities: + + Other vulnerabilities can be fixed through a normal upgrade. + +Fixing Statement +---------------- + +To prevent customers from being exposed to unexpected risks, CCE does not provide other information about the vulnerability except the vulnerability background, details, technical analysis, affected functions/versions/scenarios, solutions, and reference information. + +In addition, CCE provides the same information for all customers to protect all customers equally. CCE will not notify individual customers in advance. + +CCE does not develop or release exploitable intrusive code (or code for verification) using the vulnerabilities in the product. diff --git a/umn/source/product_bulletin/service_account_token_security_improvement.rst b/umn/source/product_bulletin/service_account_token_security_improvement.rst new file mode 100644 index 0000000..9f6ba3c --- /dev/null +++ b/umn/source/product_bulletin/service_account_token_security_improvement.rst @@ -0,0 +1,41 @@ +:original_name: cce_10_0477.html + +.. _cce_10_0477: + +Service Account Token Security Improvement +========================================== + +In clusters earlier than v1.21, a token is obtained by mounting the secret of the service account to a pod. Tokens obtained this way are permanent. This approach is no longer recommended starting from version 1.21. Service accounts will stop auto creating secrets in clusters from version 1.25. + +In clusters of version 1.21 or later, you can use the `TokenRequest `__ API to obtain the token and use the projected volume to mount the token to the pod. Such tokens are valid for a fixed period (one hour by default). Before expiration, Kubelet refreshes the token to ensure that the pod always uses a valid token. When the mounting pod is deleted, the token automatically becomes invalid. This approach is implemented by the `BoundServiceAccountTokenVolume `__ feature to improve the token security of the service account. Kubernetes clusters of v1.21 and later enables this approach by default. + +For smooth transition, the community extends the token validity period to one year by default. After one year, the token becomes invalid, and clients that do not support certificate reloading cannot access the API server. It is recommended that clients of earlier versions be upgraded as soon as possible. Otherwise, service faults may occur. + +If you use a Kubernetes client of a to-be-outdated version, the certificate reloading may fail. Versions of officially supported Kubernetes client libraries able to reload tokens are as follows: + +- Go: >= v0.15.7 +- Python: >= v12.0.0 +- Java: >= v9.0.0 +- Javascript: >= v0.10.3 +- Ruby: master branch +- Haskell: v0.3.0.0 +- C#: >= 7.0.5 + +For details, visit https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/1205-bound-service-account-tokens. + +.. note:: + + If you need a token that never expires, you can also `manually manage secrets for service accounts `__. Although a permanent service account token can be manually created, you are advised to use a short-lived token by calling the `TokenRequest `__ API for higher security. + +Diagnosis +--------- + +Run the following steps to check your CCE clusters of v1.21 and later: + +#. Use kubectl to connect to the cluster and run the **kubectl get --raw "/metrics" \| grep stale** command to query the metrics. Check the metric named **serviceaccount_stale_tokens_total**. + + If the value is greater than 0, some workloads in the cluster may be using an earlier client-go version. In this case, check whether this problem occurs in your deployed applications. If yes, upgrade client-go to the version specified by the community as soon as possible. The version must be at least two major versions of the CCE cluster. For example, if your cluster version is 1.23, the Kubernetes dependency library version must be at least 1.19. + + |image1| + +.. |image1| image:: /_static/images/en-us_image_0000001402494682.png diff --git a/umn/source/reference/index.rst b/umn/source/reference/index.rst index 54ce71a..8687eb5 100644 --- a/umn/source/reference/index.rst +++ b/umn/source/reference/index.rst @@ -5,7 +5,6 @@ Reference ========= -- :ref:`Checklist for Migrating Containerized Applications to the Cloud ` - :ref:`How Do I Troubleshoot Insufficient EIPs When a Node Is Added? ` - :ref:`How Do I Format a Data Disk Using Command Line Injection? ` - :ref:`How Do I Use heapster in Clusters of v1.13.10? ` @@ -16,17 +15,13 @@ Reference - :ref:`How Do I Add a Second Data Disk to a Node in a CCE Cluster? ` - :ref:`Workload Abnormalities ` - :ref:`What Should I Do If a Service Released in a Workload Cannot Be Accessed from Public Networks? ` -- :ref:`Selecting a Network Model When Creating a Cluster on CCE ` -- :ref:`Planning CIDR Blocks for a CCE Cluster ` - :ref:`What Is the Relationship Between Clusters, VPCs, and Subnets? ` -- :ref:`How Do I Change the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest? ` - :ref:`How Do I Harden the VPC Security Group Rules for CCE Cluster Nodes? ` .. toctree:: :maxdepth: 1 :hidden: - checklist_for_migrating_containerized_applications_to_the_cloud how_do_i_troubleshoot_insufficient_eips_when_a_node_is_added how_do_i_format_a_data_disk_using_command_line_injection how_do_i_use_heapster_in_clusters_of_v1.13.10 @@ -37,8 +32,5 @@ Reference how_do_i_add_a_second_data_disk_to_a_node_in_a_cce_cluster workload_abnormalities/index what_should_i_do_if_a_service_released_in_a_workload_cannot_be_accessed_from_public_networks - selecting_a_network_model_when_creating_a_cluster_on_cce - planning_cidr_blocks_for_a_cce_cluster what_is_the_relationship_between_clusters_vpcs_and_subnets - how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest how_do_i_harden_the_vpc_security_group_rules_for_cce_cluster_nodes diff --git a/umn/source/reference/planning_cidr_blocks_for_a_cce_cluster.rst b/umn/source/reference/planning_cidr_blocks_for_a_cce_cluster.rst deleted file mode 100644 index 157fcbd..0000000 --- a/umn/source/reference/planning_cidr_blocks_for_a_cce_cluster.rst +++ /dev/null @@ -1,143 +0,0 @@ -:original_name: cce_bestpractice_00004.html - -.. _cce_bestpractice_00004: - -Planning CIDR Blocks for a CCE Cluster -====================================== - -Before creating a cluster on CCE, determine the number of VPCs, number of subnets, container CIDR blocks, and Services for access based on service requirements. - -This section describes the functions of various addresses in a CCE cluster in a VPC and how to plan CIDR blocks. - -Basic Concepts --------------- - -**VPC CIDR Block** - -Virtual Private Cloud (VPC) enables you to provision logically isolated, configurable, and manageable virtual networks for cloud servers, cloud containers, and cloud databases. You have complete control over your virtual network, including selecting your own CIDR block, creating subnets, and configuring security groups. You can also assign EIPs and allocate bandwidth in your VPC for secure and easy access to your business system. - -**Subnet CIDR Block** - -A subnet is a network that manages ECS network planes. It supports IP address management and DNS. The IP addresses of all ECSs in a subnet belong to the subnet. - - -.. figure:: /_static/images/en-us_image_0000001223152421.png - :alt: **Figure 1** VPC CIDR block architecture - - **Figure 1** VPC CIDR block architecture - -By default, ECSs in all subnets of the same VPC can communicate with one another, while ECSs in different VPCs cannot communicate with each other. - -You can create VPC peering connections to enable ECSs in different VPCs to communicate with one another. - -**Container (Pod) CIDR Block** - -Pod is a Kubernetes object. Each pod has an IP address. - -When creating a cluster on CCE, you can specify the pod (container) CIDR block, which cannot overlap with the subnet CIDR block. For example, if the subnet CIDR block is 192.168.0.0/16, the container CIDR block cannot be 192.168.0.0/18 or 192.168.1.0/18, because these addresses are included in 192.168.0.0/16. - -**Service CIDR Block** - -Service is also a Kubernetes object. Each Service has an address. When creating a cluster on CCE, you can specify the Service CIDR block. Similarly, the Service CIDR block cannot overlap with the subnet CIDR block or the container CIDR block. The Service CIDR block can be used only within a cluster. - -For details about the relationship between these CIDR blocks, see :ref:`Figure 2 `. - -How Do I Select a CIDR Block? ------------------------------ - -**Single-VPC Single-Cluster Scenarios** - -These are the simplest scenarios. The VPC CIDR block is determined when the VPC is created. When creating a CCE cluster, select a CIDR block different from that of the current VPC. - -.. _cce_bestpractice_00004__en-us_topic_0261817695_en-us_topic_0099587154_fig15791152874920: - -.. figure:: /_static/images/en-us_image_0000001223152417.png - :alt: **Figure 2** CIDR block in the single-VPC single-cluster scenario - - **Figure 2** CIDR block in the single-VPC single-cluster scenario - -**Single-VPC Multi-Cluster Scenarios** - -Multiple CCE clusters are created in a VPC. - -In the **VPC network** mode, pod packets are forwarded through VPC routes. CCE automatically configures a routing table on the VPC routes to each container CIDR block. - -Pay attention to the following: - -- The VPC address is determined during VPC creation. When creating a cluster, select a CIDR block for each cluster that does not overlap with the VPC CIDR block or other container CIDR blocks. -- The container CIDR blocks of all clusters cannot overlap, but the Service CIDR blocks can. In this case, CCE clusters are partially interconnected. A pod of a cluster can directly access the pods of another cluster, but cannot access the Services of the cluster. -- The network scale is limited by the VPC route table. - - -.. figure:: /_static/images/en-us_image_0000001178034110.png - :alt: **Figure 3** VPC network - multi-cluster scenario - - **Figure 3** VPC network - multi-cluster scenario - -In the tunnel network model, the container network is an overlay network plane deployed over the VPC network. Though at some cost of performance, the tunnel encapsulation enables higher interoperability and compatibility with advanced features (such as network policy-based isolation), meeting the requirements of most applications. - - -.. figure:: /_static/images/en-us_image_0000001178192670.png - :alt: **Figure 4** Tunnel network - multi-cluster scenario - - **Figure 4** Tunnel network - multi-cluster scenario - -Pay attention to the following: - -- The VPC address is determined during VPC creation. When creating a cluster, select a CIDR block for each cluster that does not overlap with the VPC CIDR block or other container CIDR blocks. -- The container CIDR blocks of all clusters can overlap, so do the Service CIDR blocks. -- It is recommended that ELB be used for the cross-cluster access between containers. - -**VPC Interconnection Scenarios** - -When two VPC networks are interconnected, you can configure the packets to be sent to the peer VPC in the route table. - -In the VPC network model, after creating a peering connection, you need to add routes for the peering connection to enable communication between the two VPCs. - - -.. figure:: /_static/images/en-us_image_0000001223393899.png - :alt: **Figure 5** VPC Network - VPC interconnection scenario - - **Figure 5** VPC Network - VPC interconnection scenario - -To interconnect cluster containers across VPCs, you need to create VPC peering connections. - -Pay attention to the following: - -- The VPC address is determined during VPC creation. When creating a cluster, select a CIDR block for each cluster that does not overlap with the VPC CIDR block or other container CIDR blocks. - -- The container CIDR blocks of all clusters cannot overlap, but the Service CIDR blocks can. - -- Add the peer container CIDR block to the route table of the VPC peering connection. The following is an example: - - - .. figure:: /_static/images/en-us_image_0000001178034114.png - :alt: **Figure 6** Adding the peer container CIDR block to the local route on the VPC console - - **Figure 6** Adding the peer container CIDR block to the local route on the VPC console - -In the tunnel network model, after creating a peering connection, you need to add routes for the peering connection to enable communication between the two VPCs. - - -.. figure:: /_static/images/en-us_image_0000001223473845.png - :alt: **Figure 7** Tunnel network - VPC interconnection scenario - - **Figure 7** Tunnel network - VPC interconnection scenario - -Pay attention to the following: - -- The VPC address is determined during VPC creation. When creating a cluster, select a CIDR block for each cluster that does not overlap with the VPC CIDR block or other container CIDR blocks. - -- The container CIDR blocks of all clusters cannot overlap, but the Service CIDR blocks can. - -- Add the peer subnet CIDR block to the route table of the VPC peering connection. The following is an example: - - - .. figure:: /_static/images/en-us_image_0000001178034116.png - :alt: **Figure 8** Adding the subnet CIDR block of the peer cluster node to the local route on the VPC console - - **Figure 8** Adding the subnet CIDR block of the peer cluster node to the local route on the VPC console - -**VPC-IDC Scenarios** - -Similar to the VPC interconnection scenario, some CIDR blocks in the VPC are routed to the IDC. The pod IP addresses of CCE clusters cannot overlap with the addresses within these CIDR blocks. To access the pod IP addresses in the cluster in the IDC, you need to configure the route table to the private line VBR on the IDC. diff --git a/umn/source/reference/selecting_a_network_model_when_creating_a_cluster_on_cce.rst b/umn/source/reference/selecting_a_network_model_when_creating_a_cluster_on_cce.rst deleted file mode 100644 index a7cf537..0000000 --- a/umn/source/reference/selecting_a_network_model_when_creating_a_cluster_on_cce.rst +++ /dev/null @@ -1,66 +0,0 @@ -:original_name: cce_bestpractice_00162.html - -.. _cce_bestpractice_00162: - -Selecting a Network Model When Creating a Cluster on CCE -======================================================== - -CCE uses high-performance container networking add-ons, which support the tunnel network and VPC network models. - -.. caution:: - - After a cluster is created, the network model cannot be changed. Exercise caution when selecting a network model. - -- **Tunnel network**: The container network is an overlay tunnel network on top of a VPC network and uses the VXLAN technology. This network model is applicable when there is no high requirements on performance. VXLAN encapsulates Ethernet packets as UDP packets for tunnel transmission. Though at some cost of performance, the tunnel encapsulation enables higher interoperability and compatibility with advanced features (such as network policy-based isolation), meeting the requirements of most applications. - - - .. figure:: /_static/images/en-us_image_0000001223393893.png - :alt: **Figure 1** Container tunnel network - - **Figure 1** Container tunnel network - -- **VPC network**: The container network uses VPC routing to integrate with the underlying network. This network model is applicable to performance-intensive scenarios. The maximum number of nodes allowed in a cluster depends on the route quota in a VPC network. Each node is assigned a CIDR block of a fixed size. VPC networks are free from tunnel encapsulation overhead and outperform container tunnel networks. In addition, as VPC routing includes routes to node IP addresses and the container CIDR block, container pods in the cluster can be directly accessed from outside the cluster. - - - .. figure:: /_static/images/en-us_image_0000001178034108.png - :alt: **Figure 2** VPC network - - **Figure 2** VPC network - -The following table lists the differences between the network models. - -.. table:: **Table 1** Network comparison - - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | Dimension | Tunnel Network | VPC Network | - +==============================+==================================================================================+================================================================================================================+ - | Core component | OVS | IPVlan | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | Applicable clusters | Hybrid cluster | Hybrid cluster | - | | | | - | | VM cluster | VM cluster | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | Support for network policies | Yes | No | - | | | | - | (networkpolicy) | | | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | Support for ENI | No | Yes. The container network is deeply integrated with the VPC network, and ENI is used for pods to communicate. | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | IP address management | IP addresses can be migrated. | - Each node is allocated with a small subnet. | - | | | - A static route is added on the VPC router with the next hop set to the node IP address. | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | Network performance | Performance loss due to VXLAN tunnel encapsulation | - No performance loss as no tunnel encapsulation is required; performance comparable to bare metal networks | - | | | - Data forwarded across nodes through the VPC router | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | Networking scale | A maximum of 2,000 nodes are supported. | Limited by the VPC route table. | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | External dependency | None | Static route table of the VPC router | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - | Application scenarios | - Common container service scenarios | - Scenarios that have high requirements on network latency and bandwidth | - | | - Scenarios that do not have high requirements on network latency and bandwidth | - Containers can communicate with VMs using a microservice registration framework, such as Dubbo and CSE. | - +------------------------------+----------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------+ - -.. important:: - - #. The actual cluster scale is limited by the quota of custom routes of the VPC. Therefore, estimate the number of required nodes before creating a VPC. - #. By default, the VPC network model supports direct communication between containers and hosts in the same VPC. If a peering connection policy is configured between the VPC and another VPC, the containers can directly communicate with hosts on the peer VPC. In addition, in hybrid networking scenarios such as Direct Connect and VPN, communication between containers and hosts on the peer end can also be achieved with proper planning. diff --git a/umn/source/storage_csi/deployment_examples/evs_volumes/creating_a_pod_mounted_with_an_evs_volume.rst b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_evs_volume.rst similarity index 96% rename from umn/source/storage_csi/deployment_examples/evs_volumes/creating_a_pod_mounted_with_an_evs_volume.rst rename to umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_evs_volume.rst index 7d555e7..1e31282 100644 --- a/umn/source/storage_csi/deployment_examples/evs_volumes/creating_a_pod_mounted_with_an_evs_volume.rst +++ b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_evs_volume.rst @@ -1,9 +1,9 @@ -:original_name: cce_01_0257.html +:original_name: cce_10_0257.html -.. _cce_01_0257: +.. _cce_10_0257: -Creating a Pod Mounted with an EVS Volume -========================================= +Creating a Deployment Mounted with an EVS Volume +================================================ Scenario -------- @@ -17,7 +17,7 @@ After an EVS volume is created or imported to CCE, you can mount it to a workloa Prerequisites ------------- -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. +You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. Notes and Constraints --------------------- @@ -27,7 +27,7 @@ The following configuration example applies to clusters of Kubernetes 1.15 or la Using EVS Volumes for Deployments --------------------------------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **evs-deployment-example.yaml** file, which is used to create a Deployment. @@ -93,7 +93,7 @@ Using EVS Volumes for Deployments Using EVS Volumes for StatefulSets ---------------------------------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **evs-statefulset-example.yaml** file, which is used to create a Deployment. @@ -137,7 +137,7 @@ Using EVS Volumes for StatefulSets namespace: default labels: failure-domain.beta.kubernetes.io/region: eu-de - failure-domain.beta.kubernetes.io/zone: eu-de-01 + failure-domain.beta.kubernetes.io/zone: annotations: everest.io/disk-volume-type: SAS spec: @@ -162,7 +162,7 @@ Using EVS Volumes for StatefulSets +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ | spec.template.spec.containers.volumeMount | mountPath | Mount path of the container. In this example, the volume is mounted to the **/tmp** directory. | +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ .. note:: diff --git a/umn/source/storage_csi/deployment_examples/obs_volumes/creating_a_deployment_mounted_with_an_obs_volume.rst b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_obs_volume.rst similarity index 92% rename from umn/source/storage_csi/deployment_examples/obs_volumes/creating_a_deployment_mounted_with_an_obs_volume.rst rename to umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_obs_volume.rst index 30133c1..f204865 100644 --- a/umn/source/storage_csi/deployment_examples/obs_volumes/creating_a_deployment_mounted_with_an_obs_volume.rst +++ b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_obs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0269.html +:original_name: cce_10_0269.html -.. _cce_01_0269: +.. _cce_10_0269: Creating a Deployment Mounted with an OBS Volume ================================================ @@ -13,7 +13,7 @@ After an OBS volume is created or imported to CCE, you can mount the volume to a Prerequisites ------------- -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. +You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. Notes and Constraints --------------------- @@ -23,7 +23,7 @@ The following configuration example applies to clusters of Kubernetes 1.15 or la Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **obs-deployment-example.yaml** file, which is used to create a pod. diff --git a/umn/source/storage_csi/deployment_examples/sfs_volumes/creating_a_deployment_mounted_with_an_sfs_volume.rst b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_sfs_volume.rst similarity index 93% rename from umn/source/storage_csi/deployment_examples/sfs_volumes/creating_a_deployment_mounted_with_an_sfs_volume.rst rename to umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_sfs_volume.rst index 56b00a1..920d9d0 100644 --- a/umn/source/storage_csi/deployment_examples/sfs_volumes/creating_a_deployment_mounted_with_an_sfs_volume.rst +++ b/umn/source/storage/deployment_examples/creating_a_deployment_mounted_with_an_sfs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0263.html +:original_name: cce_10_0263.html -.. _cce_01_0263: +.. _cce_10_0263: Creating a Deployment Mounted with an SFS Volume ================================================ @@ -13,7 +13,7 @@ After an SFS volume is created or imported to CCE, you can mount the volume to a Prerequisites ------------- -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. +You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. Notes and Constraints --------------------- @@ -23,7 +23,7 @@ The following configuration example applies to clusters of Kubernetes 1.15 or la Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **sfs-deployment-example.yaml** file, which is used to create a pod. diff --git a/umn/source/storage_csi/deployment_examples/obs_volumes/creating_a_statefulset_mounted_with_an_obs_volume.rst b/umn/source/storage/deployment_examples/creating_a_statefulset_mounted_with_an_obs_volume.rst similarity index 97% rename from umn/source/storage_csi/deployment_examples/obs_volumes/creating_a_statefulset_mounted_with_an_obs_volume.rst rename to umn/source/storage/deployment_examples/creating_a_statefulset_mounted_with_an_obs_volume.rst index c4e69d0..ec406f4 100644 --- a/umn/source/storage_csi/deployment_examples/obs_volumes/creating_a_statefulset_mounted_with_an_obs_volume.rst +++ b/umn/source/storage/deployment_examples/creating_a_statefulset_mounted_with_an_obs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0268.html +:original_name: cce_10_0268.html -.. _cce_01_0268: +.. _cce_10_0268: Creating a StatefulSet Mounted with an OBS Volume ================================================= @@ -13,7 +13,7 @@ CCE allows you to use an existing OBS volume to create a StatefulSet through a P Prerequisites ------------- -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. +You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. Notes and Constraints --------------------- @@ -23,9 +23,9 @@ The following configuration example applies to clusters of Kubernetes 1.15 or la Procedure --------- -#. Create an OBS volume by referring to :ref:`PersistentVolumeClaims (PVCs) ` and obtain the PVC name. +#. Create an OBS volume by referring to :ref:`PersistentVolumeClaims (PVCs) ` and obtain the PVC name. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create a YAML file for creating the workload. Assume that the file name is **obs-statefulset-example.yaml**. @@ -80,7 +80,7 @@ Procedure +-------------+------------------------------------------------------------------------------------------------------------------------------------+ | mountPath | Mount path of a container. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ | claimName | Name of an existing PVC. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_csi/deployment_examples/sfs_volumes/creating_a_statefulset_mounted_with_an_sfs_volume.rst b/umn/source/storage/deployment_examples/creating_a_statefulset_mounted_with_an_sfs_volume.rst similarity index 96% rename from umn/source/storage_csi/deployment_examples/sfs_volumes/creating_a_statefulset_mounted_with_an_sfs_volume.rst rename to umn/source/storage/deployment_examples/creating_a_statefulset_mounted_with_an_sfs_volume.rst index a083247..ed355c0 100644 --- a/umn/source/storage_csi/deployment_examples/sfs_volumes/creating_a_statefulset_mounted_with_an_sfs_volume.rst +++ b/umn/source/storage/deployment_examples/creating_a_statefulset_mounted_with_an_sfs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0262.html +:original_name: cce_10_0262.html -.. _cce_01_0262: +.. _cce_10_0262: Creating a StatefulSet Mounted with an SFS Volume ================================================= @@ -13,7 +13,7 @@ CCE allows you to use an existing SGS volume to create a StatefulSet (by using a Prerequisites ------------- -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. +You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. Notes and Constraints --------------------- @@ -23,9 +23,9 @@ The following configuration example applies to clusters of Kubernetes 1.15 or la Procedure --------- -#. Create an SFS volume by referring to :ref:`PersistentVolumeClaims (PVCs) ` and record the volume name. +#. Create an SFS volume by referring to :ref:`PersistentVolumeClaims (PVCs) ` and record the volume name. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create a YAML file for creating the workload. Assume that the file name is **sfs-statefulset-example**.\ **yaml**. @@ -82,7 +82,7 @@ Procedure +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ | spec.template.spec.containers.volumeMounts | mountPath | Mount path of a container. | +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ | spec.template.spec.volumes.persistentVolumeClaim | claimName | Name of an existing PVC. | +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage/deployment_examples/index.rst b/umn/source/storage/deployment_examples/index.rst new file mode 100644 index 0000000..d63fdf4 --- /dev/null +++ b/umn/source/storage/deployment_examples/index.rst @@ -0,0 +1,22 @@ +:original_name: cce_10_0393.html + +.. _cce_10_0393: + +Deployment Examples +=================== + +- :ref:`Creating a Deployment Mounted with an EVS Volume ` +- :ref:`Creating a Deployment Mounted with an OBS Volume ` +- :ref:`Creating a StatefulSet Mounted with an OBS Volume ` +- :ref:`Creating a Deployment Mounted with an SFS Volume ` +- :ref:`Creating a StatefulSet Mounted with an SFS Volume ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + creating_a_deployment_mounted_with_an_evs_volume + creating_a_deployment_mounted_with_an_obs_volume + creating_a_statefulset_mounted_with_an_obs_volume + creating_a_deployment_mounted_with_an_sfs_volume + creating_a_statefulset_mounted_with_an_sfs_volume diff --git a/umn/source/storage/index.rst b/umn/source/storage/index.rst new file mode 100644 index 0000000..61cca2c --- /dev/null +++ b/umn/source/storage/index.rst @@ -0,0 +1,30 @@ +:original_name: cce_10_0374.html + +.. _cce_10_0374: + +Storage +======= + +- :ref:`Overview ` +- :ref:`Using Local Disks as Storage Volumes ` +- :ref:`PersistentVolumes (PVs) ` +- :ref:`PersistentVolumeClaims (PVCs) ` +- :ref:`StorageClass ` +- :ref:`Snapshots and Backups ` +- :ref:`Using a Custom AK/SK to Mount an OBS Volume ` +- :ref:`Setting Mount Options ` +- :ref:`Deployment Examples ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + overview + using_local_disks_as_storage_volumes + persistentvolumes_pvs + persistentvolumeclaims_pvcs + storageclass + snapshots_and_backups + using_a_custom_ak_sk_to_mount_an_obs_volume + setting_mount_options + deployment_examples/index diff --git a/umn/source/storage_csi/overview.rst b/umn/source/storage/overview.rst similarity index 94% rename from umn/source/storage_csi/overview.rst rename to umn/source/storage/overview.rst index ee45851..b2dd45e 100644 --- a/umn/source/storage_csi/overview.rst +++ b/umn/source/storage/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0307.html +:original_name: cce_10_0307.html -.. _cce_01_0307: +.. _cce_10_0307: Overview ======== @@ -8,9 +8,7 @@ Overview Volume ------ -On-disk files in a container are ephemeral, which will be lost when the container crashes and are difficult to be shared between containers running together in a pod. The Kubernetes volume abstraction solves both of these problems. Volumes cannot be independently created, but defined in the pod spec. - -All containers in a pod can access its volumes, but the volumes must have been mounted. Volumes can be mounted to any directory in a container. +On-disk files in a container are ephemeral, which will be lost when the container crashes and are difficult to be shared between containers running together in a pod. The Kubernetes volume abstraction solves both of these problems. Volumes cannot be independently created, but defined in the pod spec. All containers in a pod can access its volumes, but the volumes must have been mounted to any directory in a container. The following figure shows how a storage volume is used between containers in a pod. @@ -18,7 +16,7 @@ The following figure shows how a storage volume is used between containers in a A volume will no longer exist if the pod to which it is mounted does not exist. However, files in the volume may outlive the volume, depending on the volume type. -.. _cce_01_0307__en-us_topic_0000001199181198_section16559121287: +.. _cce_10_0307__section16559121287: Volume Types ------------ @@ -27,7 +25,7 @@ Volumes can be classified into local volumes and cloud volumes. - Local volumes - CCE supports the following five types of local volumes. For details about how to use them, see :ref:`Using Local Disks as Storage Volumes `. + CCE supports the following five types of local volumes. For details about how to use them, see :ref:`Using Local Disks as Storage Volumes `. - emptyDir: an empty volume used for temporary storage - hostPath: mounts a directory on a host (node) to your container for reading data from the host. @@ -48,7 +46,7 @@ CSI You can use Kubernetes Container Storage Interface (CSI) to develop plug-ins to support specific storage volumes. -CCE developed the storage add-on :ref:`everest ` for you to use cloud storage services, such as EVS and OBS. You can install this add-on when creating a cluster. +CCE developed the storage add-on :ref:`everest ` for you to use cloud storage services, such as EVS and OBS. You can install this add-on when creating a cluster. PV and PVC ---------- @@ -61,16 +59,16 @@ Kubernetes provides PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs) to You can bind PVCs to PVs in a pod so that the pod can use storage resources. The following figure shows the relationship between PVs and PVCs. -.. figure:: /_static/images/en-us_image_0000001409580465.png +.. figure:: /_static/images/en-us_image_0000001244141191.png :alt: **Figure 1** PVC-to-PV binding **Figure 1** PVC-to-PV binding PVs describes storage resources in the cluster. PVCs are requests for those resources. The following sections will describe how to use kubectl to connect to storage resources. -If you do not want to create storage resources or PVs manually, you can use :ref:`StorageClasses `. +If you do not want to create storage resources or PVs manually, you can use :ref:`StorageClasses `. -.. _cce_01_0307__en-us_topic_0000001199181198_section19926174743310: +.. _cce_10_0307__section19926174743310: StorageClass ------------ @@ -93,10 +91,10 @@ After a StorageClass is set, PVs can be automatically created and maintained. Yo Cloud Services for Container Storage ------------------------------------ -CCE allows you to mount local and cloud storage volumes listed in :ref:`Volume Types ` to your pods. Their features are described below. +CCE allows you to mount local and cloud storage volumes listed in :ref:`Volume Types ` to your pods. Their features are described below. -.. figure:: /_static/images/en-us_image_0000001359820608.png +.. figure:: /_static/images/en-us_image_0000001203385342.png :alt: **Figure 2** Volume types supported by CCE **Figure 2** Volume types supported by CCE @@ -124,13 +122,13 @@ CCE allows you to mount local and cloud storage volumes listed in :ref:`Volume T | | | | | | | | HPC apps here require high-speed and high-IOPS storage, such as industrial design and energy exploration. | HPC apps here require high bandwidth and shared file storage, such as gene sequencing and image rendering. | | | +----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Capacity | TB | PB | EB | TB | + | Capacity | TB | SFS 1.0: PB | EB | TB | +----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Latency | 1-2 ms | 3-10 ms | 10ms | 1-2 ms | + | Latency | 1-2 ms | SFS 1.0: 3-20 ms | 10 ms | 1-2 ms | +----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | IOPS/TPS | 33,000 for a single disk | 10,000 for a single file system | Tens of millions | 100K | + | IOPS/TPS | 33,000 for a single disk | SFS 1.0: 2K | Tens of millions | 100K | +----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Bandwidth | MB/s | GB/s | TB/s | GB/s | + | Bandwidth | MB/s | SFS 1.0: GB/s | TB/s | GB/s | +----------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Notes and Constraints @@ -151,8 +149,8 @@ Secure containers do not support OBS volumes. Notice on Using Add-ons ----------------------- -- To use the CSI plug-in (the :ref:`everest ` add-on in CCE), your cluster must be using **Kubernetes 1.15 or later**. This add-on is installed by default when you create a cluster of v1.15 or later. The FlexVolume plug-in (the :ref:`storage-driver ` add-on in CCE) is installed by default when you create a cluster of v1.13 or earlier. -- If your cluster is upgraded from v1.13 to v1.15, :ref:`storage-driver ` is replaced by everest (v1.1.6 or later) for container storage. The takeover does not affect the original storage functions. +- To use the CSI plug-in (the :ref:`everest ` add-on in CCE), your cluster must be using **Kubernetes 1.15 or later**. This add-on is installed by default when you create a cluster of v1.15 or later. The FlexVolume plug-in (the :ref:`storage-driver ` add-on in CCE) is installed by default when you create a cluster of v1.13 or earlier. +- If your cluster is upgraded from v1.13 to v1.15, :ref:`storage-driver ` is replaced by everest (v1.1.6 or later) for container storage. The takeover does not affect the original storage functions. - In version 1.2.0 of the everest add-on, **key authentication** is optimized when OBS is used. After the everest add-on is upgraded from a version earlier than 1.2.0, you need to restart all workloads that use OBS in the cluster. Otherwise, workloads may not be able to use OBS. Differences Between CSI and FlexVolume Plug-ins @@ -160,27 +158,27 @@ Differences Between CSI and FlexVolume Plug-ins .. table:: **Table 2** CSI and FlexVolume - +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Kubernetes Solution | CCE Add-on | Feature | Recommendation | - +=====================+=================+================================================================================================================================================================================================================================================================================================================================================================================================================================================+=========================================================================================================================================================================================================================================================================+ - | CSI | Everest | CSI was developed as a standard for exposing arbitrary block and file storage storage systems to containerized workloads. Using CSI, third-party storage providers can deploy plugins exposing new storage systems in Kubernetes without having to touch the core Kubernetes code. In CCE, the everest add-on is installed by default in clusters of Kubernetes v1.15 and later to connect to storage services (EVS, OBS, SFS, and SFS Turbo). | The :ref:`everest ` add-on is installed by default in clusters of **v1.15 and later**. CCE will mirror the Kubernetes community by providing continuous support for updated CSI capabilities. | - | | | | | - | | | The everest add-on consists of two parts: | | - | | | | | - | | | - **everest-csi-controller** for storage volume creation, deletion, capacity expansion, and cloud disk snapshots | | - | | | - **everest-csi-driver** for mounting, unmounting, and formatting storage volumes on nodes | | - | | | | | - | | | For details, see :ref:`everest `. | | - +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Flexvolume | storage-driver | FlexVolume is an out-of-tree plugin interface that has existed in Kubernetes since version 1.2 (before CSI). CCE provided FlexVolume volumes through the storage-driver add-on installed in clusters of Kubernetes v1.13 and earlier versions. This add-on connects clusters to storage services (EVS, OBS, SFS, and SFS Turbo). | For clusters of v1.13 or earlier that have been created, the installed FlexVolume plug-in (the storage-driver add-on in CCE) can still be used. CCE stops providing update support for this add-on, and you are advised to :ref:`upgrade these clusters `. | - | | | | | - | | | For details, see :ref:`storage-driver `. | | - +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Kubernetes Solution | CCE Add-on | Feature | Recommendation | + +=====================+=================+================================================================================================================================================================================================================================================================================================================================================================================================================================================+================================================================================================================================================================================================================================================================================+ + | CSI | Everest | CSI was developed as a standard for exposing arbitrary block and file storage storage systems to containerized workloads. Using CSI, third-party storage providers can deploy plugins exposing new storage systems in Kubernetes without having to touch the core Kubernetes code. In CCE, the everest add-on is installed by default in clusters of Kubernetes v1.15 and later to connect to storage services (EVS, OBS, SFS, and SFS Turbo). | The :ref:`everest ` add-on is installed by default in clusters of **v1.15 and later**. CCE will mirror the Kubernetes community by providing continuous support for updated CSI capabilities. | + | | | | | + | | | The everest add-on consists of two parts: | | + | | | | | + | | | - **everest-csi-controller** for storage volume creation, deletion, capacity expansion, and cloud disk snapshots | | + | | | - **everest-csi-driver** for mounting, unmounting, and formatting storage volumes on nodes | | + | | | | | + | | | For details, see :ref:`everest `. | | + +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Flexvolume | storage-driver | FlexVolume is an out-of-tree plugin interface that has existed in Kubernetes since version 1.2 (before CSI). CCE provided FlexVolume volumes through the storage-driver add-on installed in clusters of Kubernetes v1.13 and earlier versions. This add-on connects clusters to storage services (EVS, OBS, SFS, and SFS Turbo). | For the created clusters of **v1.13 or earlier**, the installed FlexVolume plug-in (CCE add-on :ref:`storage-driver `) can still be used. CCE stops providing update support for this add-on, and you are advised to :ref:`upgrade these clusters `. | + | | | | | + | | | For details, see :ref:`storage-driver `. | | + +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ .. note:: - A cluster can use only one type of storage plug-ins. - - The FlexVolume plug-in cannot be replaced by the CSI plug-in in clusters of v1.13 or earlier. You can only upgrade these clusters. For details, see :ref:`Cluster Upgrade Between Major Versions `. + - The FlexVolume plug-in cannot be replaced by the CSI plug-in in clusters of v1.13 or earlier. You can only upgrade these clusters. For details, see :ref:`Cluster Upgrade `. Checking Storage Add-ons ------------------------ @@ -190,4 +188,4 @@ Checking Storage Add-ons #. Click the **Add-on Instance** tab. #. Select a cluster in the upper right corner. The default storage add-on installed during cluster creation is displayed. -.. |image1| image:: /_static/images/en-us_image_0000001409860177.png +.. |image1| image:: /_static/images/en-us_image_0000001199501276.png diff --git a/umn/source/storage_csi/persistentvolumeclaims_pvcs.rst b/umn/source/storage/persistentvolumeclaims_pvcs.rst similarity index 83% rename from umn/source/storage_csi/persistentvolumeclaims_pvcs.rst rename to umn/source/storage/persistentvolumeclaims_pvcs.rst index 81bd442..a90b650 100644 --- a/umn/source/storage_csi/persistentvolumeclaims_pvcs.rst +++ b/umn/source/storage/persistentvolumeclaims_pvcs.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0378.html +:original_name: cce_10_0378.html -.. _cce_01_0378: +.. _cce_10_0378: PersistentVolumeClaims (PVCs) ============================= @@ -28,7 +28,7 @@ When a PVC is created, the system checks whether there is an available PV with t | Storage class | storageclass | storageclass | The settings must be consistent. | +---------------+-------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------+ -.. _cce_01_0378__en-us_topic_0000001199501146_section43881411172418: +.. _cce_10_0378__section43881411172418: Volume Access Modes ------------------- @@ -57,27 +57,29 @@ StorageClass describes the storage class used in the cluster. You need to specif **Using the CCE Console** #. Log in to the CCE console. -#. Go to the cluster details page, choose **Storage** from the navigation pane, and click the **PersistentVolumeClaims (PVCs)** tab. +#. Click the cluster name and go to the cluster console. Choose **Storage** from the navigation pane, and click the **PersistentVolumeClaims (PVCs)** tab. #. Click **Create PVC** in the upper right corner. In the dialog box displayed, set the PVC parameters. - - **Creation Method**: Select **Storage class**. + - **Storage Volume Claim Type**: Select a storage type as required. - **PVC Name**: Enter a PVC name. - - **Storage Class**: Select the required storage class. The following storage resources can be dynamically provisioned: + - **Creation Method**: Select **Dynamic creation**. + - **Storage Classes**: Select the required storage class. The following storage resources can be dynamically provisioned: - **csi-disk**: EVS disk. + - **csi-nas**: SFS Capacity-Oriented file storage. - **csi-obs**: OBS bucket. - **AZ** (supported only by EVS): Select the AZ where the EVS disk is located. - - **Disk Type**: Select an EVS disk type. EVS disk types vary in different regions. + - **Disk Type** (supported only by EVS disks): Select an EVS disk type as required. EVS disk types vary in different regions. - Common I/O - High I/O - Ultra-high I/O - - **Access Mode**: **ReadWriteOnce** and **ReadWriteMany** are supported. For details, see :ref:`Volume Access Modes `. - - **Capacity (GiB)** (supported only by EVS and SFS): storage capacity. This parameter is not available for OBS. + - **Access Mode**: **ReadWriteOnce** and **ReadWriteMany** are supported. For details, see :ref:`Volume Access Modes `. + - **Capacity (GiB)** (only EVS and SFS are supported): storage capacity. This parameter is not available for OBS. - **Encryption** (supported only for EVS and SFS): Select **Encryption**. After selecting this option, you need to select a key. - - **Secret** (supported only for OBS): Select an access key for OBS. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. + - **Secret** (supported only for OBS): Select an access key for OBS. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. #. Click **Create**. @@ -106,7 +108,7 @@ Example YAML for EVS labels: failure-domain.beta.kubernetes.io/region: eu-de - failure-domain.beta.kubernetes.io/zone: eu-de-01 + failure-domain.beta.kubernetes.io/zone: spec: accessModes: - ReadWriteOnce # The value must be ReadWriteOnce for EVS. @@ -115,6 +117,27 @@ Example YAML for EVS storage: 10Gi # EVS disk capacity, ranging from 1 to 32768. storageClassName: csi-disk # The storage class type is EVS. +Example YAML for file storage: + +.. code-block:: + + apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: pvc-sfs-auto-example + namespace: default + annotations: + everest.io/crypt-key-id: 0992dbda-6340-470e-a74e-4f0db288ed82 # (Optional) Key ID. The key is used to encrypt file systems. + everest.io/crypt-alias: sfs/default # (Optional) Key name. Mandatory for encrypted volumes. + everest.io/crypt-domain-id: 2cd7ebd02e4743eba4e6342c09e49344 # (Optional) ID of the tenant to which the encrypted volume belongs. Mandatory for encrypted volumes. + spec: + accessModes: + - ReadWriteMany # The value must be ReadWriteMany for SFS. + resources: + requests: + storage: 10Gi # SFS file system size. + storageClassName: csi-nas # The storage class type is SFS. + Example YAML for OBS: .. code-block:: @@ -144,19 +167,13 @@ If a PV has been created, you can create a PVC to apply for PV resources. **Using the CCE Console** #. Log in to the CCE console. -#. Go to the cluster details page, choose **Storage** from the navigation pane, and click the **PersistentVolumeClaims (PVCs)** tab. +#. Click the cluster name and go to the cluster console. Choose **Storage** from the navigation pane, and click the **PersistentVolumeClaims (PVCs)** tab. #. Click **Create PVC** in the upper right corner. In the dialog box displayed, set the PVC parameters. - - **Creation Method**: Select **Existing volume**. - - **PVC Name**: Enter a PVC name. - - **Volume Type**: Select your required volume type. - - - EVS - - SFS - - OBS - - SFS Turbo - - - **Associate Volume**: Select the volume to be associated, that is, the PV. + - **Storage Volume Claim Type**: Select a storage type as required. + - **PVC Name**: name of a PVC. + - **Creation Method**: Select **Existing storage volume**. + - **PV**: Select the volume to be associated, that is, the PV. #. Click **Create**. @@ -186,7 +203,7 @@ Example YAML for EVS labels: failure-domain.beta.kubernetes.io/region: eu-de - failure-domain.beta.kubernetes.io/zone: eu-de-01 + failure-domain.beta.kubernetes.io/zone: spec: accessModes: - ReadWriteOnce # The value must be ReadWriteOnce for EVS. @@ -212,9 +229,9 @@ Example YAML for SFS: - ReadWriteMany # The value must be ReadWriteMany for SFS. resources: requests: - storage: 100Gi # Requested PVC capacity. - storageClassName: csi-nas # Storage class name. The value is csi-nas for SFS. - volumeName: cce-sfs-test # PV name. + storage: 100Gi # Requested PVC capacity + storageClassName: csi-nas # Storage class name + volumeName: cce-sfs-test # PV name Example YAML for OBS: @@ -269,16 +286,11 @@ The disk type, encryption setting, and disk mode of the created EVS PVC are cons **Using the CCE Console** #. Log in to the CCE console. -#. Go to the cluster details page, choose **Storage** from the navigation pane, and click the **PersistentVolumeClaims (PVCs)** tab. -#. Click **Create PVC** in the upper right corner. In the dialog box displayed, set the PVC parameters. - - - **Creation Mode**: Select **Snapshot**. - - **PVC Name**: name of a PVC. - - **Snapshot**: Select the snapshot to be used. - +#. Click the cluster name and go to the cluster console. Choose **Storage** from the navigation pane, and click the **Snapshots and Backups** tab. +#. Locate the snapshot for which you want to create a PVC, click **Create PVC**, and specify the PVC name in the displayed dialog box. #. Click **Create**. -**Using YAML** +**Creating from YAML** .. code-block:: diff --git a/umn/source/storage_csi/persistentvolumes_pvs.rst b/umn/source/storage/persistentvolumes_pvs.rst similarity index 95% rename from umn/source/storage_csi/persistentvolumes_pvs.rst rename to umn/source/storage/persistentvolumes_pvs.rst index 4fbafd9..3a447d0 100644 --- a/umn/source/storage_csi/persistentvolumes_pvs.rst +++ b/umn/source/storage/persistentvolumes_pvs.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0379.html +:original_name: cce_10_0379.html -.. _cce_01_0379: +.. _cce_10_0379: PersistentVolumes (PVs) ======================= @@ -23,7 +23,7 @@ Volume Access Modes PVs can be mounted to the host system only in the mode supported by underlying storage resources. For example, a file storage system can be read and written by multiple nodes, but an EVS disk can be read and written by only one node. - ReadWriteOnce: A volume can be mounted as read-write by a single node. This access mode is supported by EVS. -- ReadWriteMany: A volume can be mounted as read-write by multiple nodes. This access mode is supported by SFS, SFS Turbo, and OBS. +- ReadWriteMany: A volume can be mounted as read-write by multiple nodes. This access mode is supported by SFS, OBS, and SFS Turbo. .. table:: **Table 1** Access modes supported by cloud storage @@ -36,7 +36,7 @@ PVs can be mounted to the host system only in the mode supported by underlying s SFS Turbo x Y ============ ============= ============= -.. _cce_01_0379__en-us_topic_0000001244101041_section19999142414413: +.. _cce_10_0379__section19999142414413: PV Reclaim Policy ----------------- @@ -63,14 +63,14 @@ Creating an EVS Volume **Using the CCE Console** #. Log in to the CCE console. -#. Access the cluster details page, choose **Storage** from the navigation pane, and click the **Volumes** tab. +#. Click the cluster name and access the cluster console. Choose **Storage** from the navigation pane, and click the **PersistentVolumes (PVs)** tab. #. Click **Create Volume** in the upper right corner. In the dialog box displayed, set the volume parameters. - **Volume Type**: Select **EVS**. - **EVS**: - **PV Name**: Enter a PV name. - **Access Mode**: ReadWriteOnce - - **Reclaim Policy**: Select **Delete** or **Retain** as required. For details, see :ref:`PV Reclaim Policy `. + - **Reclaim Policy**: Select **Delete** or **Retain** as required. For details, see :ref:`PV Reclaim Policy `. #. Click **Create**. @@ -87,7 +87,7 @@ Creating an EVS Volume name: cce-evs-test labels: failure-domain.beta.kubernetes.io/region: eu-de - failure-domain.beta.kubernetes.io/zone: eu-de-01 + failure-domain.beta.kubernetes.io/zone: spec: accessModes: - ReadWriteOnce # Access mode. The value is fixed to ReadWriteOnce for EVS. @@ -157,6 +157,21 @@ Creating an SFS Volume - The SFS file system and the cluster must be in the same VPC. +**Using the CCE Console** + +#. Log in to the CCE console. +#. Click the cluster name and access the cluster console. Choose **Storage** from the navigation pane, and click the **PersistentVolumes (PVs)** tab. +#. Click **Create Volume** in the upper right corner. In the dialog box displayed, set the volume parameters. + + - **Volume Type**: Select **SFS**. + - Select SFS resources. + - **PV Name**: Enter a PV name. + - **Access Mode**: ReadWriteMany + - **Reclaim Policy**: Select **Delete** or **Retain** as required. For details, see :ref:`PV Reclaim Policy `. + - **Mount Options**: mount options. For details about the options, see :ref:`Setting Mount Options `. + +#. Click **Create**. + **Using YAML** .. code-block:: @@ -181,7 +196,7 @@ Creating an SFS Volume everest.io/share-export-location: # Shared path of the file storage storage.kubernetes.io/csiProvisionerIdentity: everest-csi-provisioner persistentVolumeReclaimPolicy: Retain # Reclaim policy. - storageClassName: csi-nas # Storage class name. The value must be csi-nas for SFS. + storageClassName: csi-nas # Storage class name mountOptions: [] # Mount options .. table:: **Table 3** Key parameters @@ -195,9 +210,9 @@ Creating an SFS Volume | | | | | This field is valid only when the everest version is 1.2.9 or later and the reclaim policy is Delete. If the reclaim policy is Delete and the current value is **retain-volume-only**, the associated PV is deleted while the underlying storage volume is retained, when a PVC is deleted. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeHandle | File system ID. | + | volumeHandle | - If SFS Capacity-Oriented file storage is used, enter the file storage ID. | | | | - | | On the management console, choose **Service List** > **Storage** > **Scalable File Service**. In the SFS file system list, click the name of the target file system and copy the content following **ID** on the page displayed. | + | | On the management console, choose **Service List** > **Storage** > **Scalable File Service**. In the SFS file system list, click the name of the target file system and copy the content following **ID** on the page displayed. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | everest.io/share-export-location | Shared path of the file system. | | | | @@ -205,7 +220,7 @@ Creating an SFS Volume +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | mountOptions | Mount options. | | | | - | | If not specified, the following configurations are used by default. For details, see :ref:`SFS Volume Mount Options `. | + | | If not specified, the following configurations are used by default. For details, see :ref:`SFS Volume Mount Options `. | | | | | | .. code-block:: | | | | @@ -243,16 +258,16 @@ Creating an OBS Volume **Using the CCE Console** #. Log in to the CCE console. -#. Access the cluster details page, choose **Storage** from the navigation pane, and click the **Volumes** tab. +#. Click the cluster name and access the cluster console. Choose **Storage** from the navigation pane, and click the **PersistentVolumes (PVs)** tab. #. Click **Create Volume** in the upper right corner. In the dialog box displayed, set the volume parameters. - **Volume Type**: Select **OBS**. - Select OBS resources. - **PV Name**: Enter a PV name. - **Access Mode**: ReadWriteMany - - **Reclaim Policy**: Select **Delete** or **Retain** as required. For details, see :ref:`PV Reclaim Policy `. - - **Key**: You can customize the access key (AK/SK) for mounting an OBS volume. You can use the AK/SK to create a secret and mount the secret to the PV. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. - - **Mount Options**: mount options. For details about the options, see :ref:`Setting Mount Options `. + - **Reclaim Policy**: Select **Delete** or **Retain** as required. For details, see :ref:`PV Reclaim Policy `. + - **Secret**: You can customize the access key (AK/SK) for mounting an OBS volume. You can use the AK/SK to create a secret and mount the secret to the PV. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. + - **Mount Options**: mount options. For details about the options, see :ref:`Setting Mount Options `. #. Click **Create**. @@ -309,9 +324,9 @@ Creating an OBS Volume | | | | | For details about the value of **region**, see `Regions and Endpoints `__. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | nodePublishSecretRef | Access key (AK/SK) used for mounting the object storage volume. You can use the AK/SK to create a secret and mount it to the PV. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. | + | nodePublishSecretRef | Access key (AK/SK) used for mounting the object storage volume. You can use the AK/SK to create a secret and mount it to the PV. For details, see :ref:`Using a Custom AK/SK to Mount an OBS Volume `. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | mountOptions | Mount options. For details, see :ref:`OBS Volume Mount Options `. | + | mountOptions | Mount options. For details, see :ref:`OBS Volume Mount Options `. | +-----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | persistentVolumeReclaimPolicy | A reclaim policy is supported when the cluster version is equal to or later than 1.19.10 and the everest version is equal to or later than 1.2.9. | | | | @@ -337,15 +352,15 @@ Creating an SFS Turbo Volume **Using the CCE Console** #. Log in to the CCE console. -#. Access the cluster details page, choose **Storage** from the navigation pane, and click the **Volumes** tab. +#. Click the cluster name and access the cluster console. Choose **Storage** from the navigation pane, and click the **PersistentVolumes (PVs)** tab. #. Click **Create Volume** in the upper right corner. In the dialog box displayed, set the volume parameters. - **Volume Type**: Select **SFS Turbo**. - **SFS Turbo**: Select SFS Turbo resources. - **PV Name**: Enter a PV name. - **Access Mode**: ReadWriteMany - - **Reclaim Policy**: Select **Retain**. For details, see :ref:`PV Reclaim Policy `. - - **Mount Options**: mount options. For details about the options, see :ref:`Setting Mount Options `. + - **Reclaim Policy**: Select **Retain**. For details, see :ref:`PV Reclaim Policy `. + - **Mount Options**: mount options. For details about the options, see :ref:`Setting Mount Options `. #. Click **Create**. @@ -388,7 +403,7 @@ Creating an SFS Turbo Volume +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | mountOptions | Mount options. | | | | - | | If not specified, the following configurations are used by default. For details, see :ref:`SFS Volume Mount Options `. | + | | If not specified, the following configurations are used by default. For details, see :ref:`SFS Volume Mount Options `. | | | | | | .. code-block:: | | | | diff --git a/umn/source/storage_csi/setting_mount_options.rst b/umn/source/storage/setting_mount_options.rst similarity index 89% rename from umn/source/storage_csi/setting_mount_options.rst rename to umn/source/storage/setting_mount_options.rst index c71d8fa..8d9508a 100644 --- a/umn/source/storage_csi/setting_mount_options.rst +++ b/umn/source/storage/setting_mount_options.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0337.html +:original_name: cce_10_0337.html -.. _cce_01_0337: +.. _cce_10_0337: Setting Mount Options ===================== @@ -12,16 +12,16 @@ You can mount cloud storage volumes to your containers and use these volumes as This section describes how to set mount options when mounting SFS and OBS volumes. You can set mount options in a PV and bind the PV to a PVC. Alternatively, set mount options in a StorageClass and use the StorageClass to create a PVC. In this way, PVs can be dynamically created and inherit mount options configured in the StorageClass by default. -.. _cce_01_0337__en-us_topic_0000001199021188_section14888047833: +.. _cce_10_0337__section14888047833: SFS Volume Mount Options ------------------------ -The everest add-on in CCE presets the options described in :ref:`Table 1 ` for mounting SFS volumes. You can set other mount options if needed. For details, see `Mounting an NFS File System to ECSs (Linux) `__. +The everest add-on in CCE presets the options described in :ref:`Table 1 ` for mounting SFS volumes. You can set other mount options if needed. For details, see `Mounting an NFS File System to ECSs (Linux) `__. -.. _cce_01_0337__en-us_topic_0000001199021188_table128754351546: +.. _cce_10_0337__table128754351546: -.. table:: **Table 1** Preset mount options for SFS volumes +.. table:: **Table 1** SFS volume mount options +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Option | Description | @@ -40,14 +40,14 @@ The everest add-on in CCE presets the options described in :ref:`Table 1 ` and :ref:`Table 3 ` by default. The options in :ref:`Table 2 ` are mandatory. +When mounting file storage, the everest add-on presets the options described in :ref:`Table 2 ` and :ref:`Table 3 ` by default. The options in :ref:`Table 2 ` are mandatory. -.. _cce_01_0337__en-us_topic_0000001199021188_table1688593020213: +.. _cce_10_0337__table1688593020213: .. table:: **Table 2** Mandatory mount options configured by default @@ -71,14 +71,14 @@ When mounting file storage, the everest add-on presets the options described in | sigv2 | Specifies the signature version. Used by default in object buckets. | +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------+ -.. _cce_01_0337__en-us_topic_0000001199021188_table9886123010217: +.. _cce_10_0337__table9886123010217: .. table:: **Table 3** Optional mount options configured by default +-----------------------+--------------------------------------------------------------------------------------------------------------------+ | Option | Description | +=======================+====================================================================================================================+ - | max_write=131072 | If specified, obsfs allocates the **inode** number. Enabled by default in read/write mode. | + | max_write=131072 | This parameter is valid only when **big_writes** is configured. The recommended value is **128 KB**. | +-----------------------+--------------------------------------------------------------------------------------------------------------------+ | ssl_verify_hostname=0 | Disables verifying the SSL certificate based on the host name. | +-----------------------+--------------------------------------------------------------------------------------------------------------------+ @@ -115,7 +115,7 @@ Mount options cannot be configured for secure containers. Setting Mount Options in a PV ----------------------------- -You can use the **mountOptions** field to set mount options in a PV. The options you can configure in **mountOptions** are listed in :ref:`SFS Volume Mount Options ` and :ref:`OBS Volume Mount Options `. +You can use the **mountOptions** field to set mount options in a PV. The options you can configure in **mountOptions** are listed in :ref:`SFS Volume Mount Options ` and :ref:`OBS Volume Mount Options `. .. code-block:: @@ -155,7 +155,7 @@ After a PV is created, you can create a PVC and bind it to the PV, and then moun Setting Mount Options in a StorageClass --------------------------------------- -You can use the **mountOptions** field to set mount options in a StorageClass. The options you can configure in **mountOptions** are listed in :ref:`SFS Volume Mount Options ` and :ref:`OBS Volume Mount Options `. +You can use the **mountOptions** field to set mount options in a StorageClass. The options you can configure in **mountOptions** are listed in :ref:`SFS Volume Mount Options ` and :ref:`OBS Volume Mount Options `. .. code-block:: diff --git a/umn/source/storage_csi/snapshots_and_backups.rst b/umn/source/storage/snapshots_and_backups.rst similarity index 86% rename from umn/source/storage_csi/snapshots_and_backups.rst rename to umn/source/storage/snapshots_and_backups.rst index 341bfc4..e6b3937 100644 --- a/umn/source/storage_csi/snapshots_and_backups.rst +++ b/umn/source/storage/snapshots_and_backups.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0211.html +:original_name: cce_10_0381.html -.. _cce_01_0211: +.. _cce_10_0381: Snapshots and Backups ===================== @@ -46,15 +46,15 @@ Creating a Snapshot **Using the CCE Console** #. Log in to the CCE console. -#. Go to the cluster details page, choose **Storage** from the navigation pane, and click the **Snapshots and Backups** tab. +#. Click the cluster name and go to the cluster console. Choose **Storage** from the navigation pane, and click the **Snapshots and Backups** tab. #. Click **Create Snapshot** in the upper right corner. In the dialog box displayed, set related parameters. - **Snapshot Name**: Enter a snapshot name. - - **Storage**: Select the PVC for which you want to create a snapshot. + - **Storage**: Select a PVC. Only EVS PVCs can create a snapshot. #. Click **Create**. -**Using YAML** +**Creating from YAML** .. code-block:: @@ -79,16 +79,11 @@ The disk type, encryption setting, and disk mode of the created EVS PVC are cons **Using the CCE Console** #. Log in to the CCE console. -#. Go to the cluster details page, choose **Storage** from the navigation pane, and click the **PersistentVolumeClaims (PVCs)** tab. -#. Click **Create PVC** in the upper right corner. In the dialog box displayed, set the PVC parameters. - - - **Creation Mode**: Select **Snapshot**. - - **PVC Name**: name of a PVC. - - **Snapshot**: Select the snapshot to be used. - +#. Click the cluster name and go to the cluster console. Choose **Storage** from the navigation pane, and click the **Snapshots and Backups** tab. +#. Locate the snapshot for which you want to create a PVC, click **Create PVC**, and specify the PVC name in the displayed dialog box. #. Click **Create**. -**Using YAML** +**Creating from YAML** .. code-block:: @@ -101,7 +96,7 @@ The disk type, encryption setting, and disk mode of the created EVS PVC are cons everest.io/disk-volume-type: SSD # EVS disk type, which must be the same as that of the source EVS disk of the snapshot. labels: failure-domain.beta.kubernetes.io/region: eu-de - failure-domain.beta.kubernetes.io/zone: eu-de-01 + failure-domain.beta.kubernetes.io/zone: spec: accessModes: - ReadWriteOnce diff --git a/umn/source/storage_csi/storageclass.rst b/umn/source/storage/storageclass.rst similarity index 97% rename from umn/source/storage_csi/storageclass.rst rename to umn/source/storage/storageclass.rst index 325250b..9607a61 100644 --- a/umn/source/storage_csi/storageclass.rst +++ b/umn/source/storage/storageclass.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0380.html +:original_name: cce_10_0380.html -.. _cce_01_0380: +.. _cce_10_0380: StorageClass ============ @@ -14,7 +14,7 @@ You can run the following command to query the storage classes that CCE supports # kubectl get sc NAME PROVISIONER AGE csi-disk everest-csi-provisioner 17d # Storage class for EVS disks - csi-nas everest-csi-provisioner 17d # Storage class for SFS file systems + csi-nas everest-csi-provisioner 17d # Storage class for SFS 1.0 file systems csi-obs everest-csi-provisioner 17d # Storage class for OBS buckets After a StorageClass is set, PVs can be automatically created and maintained. You only need to specify the StorageClass when creating a PVC, which greatly reduces the workload. @@ -122,6 +122,10 @@ For an ultra-high I/O storage class, you can set the class name to **csi-disk-ss - **Delete**: When a PVC is deleted, both the PV and the EVS disk are deleted. - **Retain**: When a PVC is deleted, the PV and underlying storage resources are not deleted. Instead, you must manually delete these resources. After that, the PV resource is in the **Released** state and cannot be bound to the PVC again. +.. note:: + + The reclamation policy set here has no impact on the SFS Turbo storage. Therefore, the yearly/monthly SFS Turbo resources will not be reclaimed when the cluster or PVC is deleted. + If high data security is required, you are advised to select **Retain** to prevent data from being deleted by mistake. After the definition is complete, run the **kubectl create** commands to create storage resources. diff --git a/umn/source/storage_csi/using_a_custom_ak_sk_to_mount_an_obs_volume.rst b/umn/source/storage/using_a_custom_ak_sk_to_mount_an_obs_volume.rst similarity index 98% rename from umn/source/storage_csi/using_a_custom_ak_sk_to_mount_an_obs_volume.rst rename to umn/source/storage/using_a_custom_ak_sk_to_mount_an_obs_volume.rst index 356f1e3..8049691 100644 --- a/umn/source/storage_csi/using_a_custom_ak_sk_to_mount_an_obs_volume.rst +++ b/umn/source/storage/using_a_custom_ak_sk_to_mount_an_obs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0336.html +:original_name: cce_10_0336.html -.. _cce_01_0336: +.. _cce_10_0336: Using a Custom AK/SK to Mount an OBS Volume =========================================== @@ -264,6 +264,8 @@ You can use a secret of an IAM user to mount an OBS volume. Assume that a worklo |image2| + |image3| + #. Write data into the mouth path again. In this example, the write operation succeeded. **kubectl exec obs-secret-5cd558f76f-vxslv -- touch /temp/test** @@ -278,5 +280,6 @@ You can use a secret of an IAM user to mount an OBS volume. Assume that a worklo -rwxrwxrwx 1 root root 0 Jun 7 01:52 test -.. |image1| image:: /_static/images/en-us_image_0000001409700093.png -.. |image2| image:: /_static/images/en-us_image_0000001360140132.png +.. |image1| image:: /_static/images/en-us_image_0000001199181232.png +.. |image2| image:: /_static/images/en-us_image_0000001244141105.png +.. |image3| image:: /_static/images/en-us_image_0000001244261069.png diff --git a/umn/source/storage_csi/using_local_disks_as_storage_volumes.rst b/umn/source/storage/using_local_disks_as_storage_volumes.rst similarity index 92% rename from umn/source/storage_csi/using_local_disks_as_storage_volumes.rst rename to umn/source/storage/using_local_disks_as_storage_volumes.rst index 252c8c6..2b23dbd 100644 --- a/umn/source/storage_csi/using_local_disks_as_storage_volumes.rst +++ b/umn/source/storage/using_local_disks_as_storage_volumes.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0053.html +:original_name: cce_10_0377.html -.. _cce_01_0053: +.. _cce_10_0377: Using Local Disks as Storage Volumes ==================================== @@ -12,14 +12,14 @@ Using Local Volumes CCE supports four types of local volumes. -- :ref:`hostPath `: mounts a file directory of the host where the container is located to the specified mount point of the container. For example, if the container needs to access **/etc/hosts**, you can use a hostPath volume to map **/etc/hosts**. -- :ref:`emptyDir `: stores data temporarily. An emptyDir volume is first created when a pod is assigned to a node, and exists as long as that pod is running on that node. When a container pod is terminated, **EmptyDir** will be deleted and the data is permanently lost. -- :ref:`ConfigMap `: A ConfigMap can be mounted as a volume, and all contents stored in its key are mounted onto the specified container directory. A ConfigMap is a type of resource that stores configuration information required by a workload. Its content is user-defined. For details about how to create a ConfigMap, see :ref:`Creating a ConfigMap `. For details about how to use a ConfigMap, see :ref:`Using a ConfigMap `. -- :ref:`Secret `: You can store sensitive information such as passwords, in secrets and mount them as files for use by pods. A secret is a type of resource that holds sensitive data, such as authentication and key information. All content is user-defined. For details about how to create a secret, see :ref:`Creating a Secret `. For details about how to use a secret, see :ref:`Using a Secret `. +- :ref:`hostPath `: mounts a file directory of the host where the container is located to the specified mount point of the container. For example, if the container needs to access **/etc/hosts**, you can use a hostPath volume to map **/etc/hosts**. +- :ref:`emptyDir `: stores data temporarily. An emptyDir volume is first created when a pod is assigned to a node, and exists as long as that pod is running on that node. When a container pod is terminated, **EmptyDir** will be deleted and the data is permanently lost. +- :ref:`ConfigMap `: A ConfigMap can be mounted as a volume, and all contents stored in its key are mounted onto the specified container directory. A ConfigMap is a type of resource that stores configuration information required by a workload. Its content is user-defined. For details about how to create a ConfigMap, see :ref:`Creating a ConfigMap `. For details about how to use a ConfigMap, see :ref:`Using a ConfigMap `. +- :ref:`Secret mounting `: Data in the secret is mounted to a path of the container. A secret is a type of resource that holds sensitive data, such as authentication and key information. All content is user-defined. For details about how to create a secret, see :ref:`Creating a Secret `. For details on how to use a secret, see :ref:`Using a Secret `. The following describes how to mount these four types of volumes. -.. _cce_01_0053__en-us_topic_0000001199341206_section196700523438: +.. _cce_10_0377__section196700523438: hostPath -------- @@ -28,18 +28,18 @@ You can mount a path on the host to a specified container path. A hostPath volum #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image1|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **hostPath** from the drop-down list. -#. Set parameters for adding a local volume, as listed in :ref:`Table 1 `. +#. Set parameters for adding a local volume, as listed in :ref:`Table 1 `. - .. _cce_01_0053__en-us_topic_0000001199341206_table14312815449: + .. _cce_10_0377__table14312815449: .. table:: **Table 1** Setting parameters for mounting a hostPath volume +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===================================+=====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ - | Storage Type | Select **HostPath**. | + | Storage Type | Select **hostPath**. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Host Path | Path of the host to which the local volume is to be mounted, for example, **/etc/hosts**. | | | | @@ -75,10 +75,10 @@ You can mount a path on the host to a specified container path. A hostPath volum | | - **Read-only**: You can only read the data volumes mounted to the path. | | | - **Read/Write**: You can modify the data volumes mounted to the path. Newly written data is not migrated if the container is migrated, which may cause a data loss. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image1| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -.. _cce_01_0053__en-us_topic_0000001199341206_section550555216467: +.. _cce_10_0377__section550555216467: emptyDir -------- @@ -87,11 +87,11 @@ emptyDir applies to temporary data storage, disaster recovery, and runtime data #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image2|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **emptyDir** from the drop-down list. -#. Set the local volume type to **emptyDir** and set parameters for adding a local volume, as described in :ref:`Table 2 `. +#. Set the local volume type to **emptyDir** and set parameters for adding a local volume, as described in :ref:`Table 2 `. - .. _cce_01_0053__en-us_topic_0000001199341206_table1867417102475: + .. _cce_10_0377__table1867417102475: .. table:: **Table 2** Setting parameters for mounting an emptyDir volume @@ -100,7 +100,7 @@ emptyDir applies to temporary data storage, disaster recovery, and runtime data +===================================+=====================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================+ | Storage Type | Select **emptyDir**. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Medium | - **Default**: Data is stored in hard disks, which is applicable to a large amount of data with low requirements on reading and writing efficiency. | + | Storage Medium | - **Default**: Data is stored in hard disks, which is applicable to a large amount of data with low requirements on reading and writing efficiency. | | | - **Memory**: Selecting this option can improve the running speed, but the storage capacity is subject to the memory size. This mode applies to scenarios where the data volume is small and the read and write efficiency is high. | | | | | | .. note:: | @@ -129,23 +129,23 @@ emptyDir applies to temporary data storage, disaster recovery, and runtime data | | - **Read-only**: You can only read the data volumes mounted to the path. | | | - **Read/Write**: You can modify the data volumes mounted to the path. Newly written data is not migrated if the container is migrated, which may cause a data loss. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image2| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -.. _cce_01_0053__en-us_topic_0000001199341206_section18638191594712: +.. _cce_10_0377__section18638191594712: ConfigMap --------- -The data stored in a ConfigMap can be referenced in a volume of type ConfigMap. You can mount such a volume to a specified container path. The platform supports the separation of workload codes and configuration files. ConfigMap volumes are used to store workload configuration parameters. Before that, you need to create ConfigMaps in advance. For details, see :ref:`Creating a ConfigMap `. +The data stored in a ConfigMap can be referenced in a volume of type ConfigMap. You can mount such a volume to a specified container path. The platform supports the separation of workload codes and configuration files. ConfigMap volumes are used to store workload configuration parameters. Before that, you need to create ConfigMaps in advance. For details, see :ref:`Creating a ConfigMap `. #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image3|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **ConfigMap** from the drop-down list. -#. Set the local volume type to **ConfigMap** and set parameters for adding a local volume, as shown in :ref:`Table 3 `. +#. Set the local volume type to **ConfigMap** and set parameters for adding a local volume, as shown in :ref:`Table 3 `. - .. _cce_01_0053__en-us_topic_0000001199341206_table1776324831114: + .. _cce_10_0377__table1776324831114: .. table:: **Table 3** Setting parameters for mounting a ConfigMap volume @@ -156,13 +156,15 @@ The data stored in a ConfigMap can be referenced in a volume of type ConfigMap. +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Option | Select the desired ConfigMap name. | | | | - | | A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. | + | | A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Add Container Path | Configure the following parameters: | | | | | | a. **subPath**: Enter a subpath, for example, **tmp**. | | | | - | | A subpath is used to mount a local disk so that the same data volume is used in a single pod. If this parameter is left blank, the root path is used by default. | + | | - A subpath is used to mount a local volume so that the same data volume is used in a single pod. | + | | - The subpath can be the key and value of a ConfigMap or secret. If the subpath is a key-value pair that does not exist, the data import does not take effect. | + | | - The data imported by specifying a subpath will not be updated along with the ConfigMap/secret updates. | | | | | | b. **Container Path**: Enter the path of the container, for example, **/tmp**. | | | | @@ -175,23 +177,23 @@ The data stored in a ConfigMap can be referenced in a volume of type ConfigMap. | | | | | c. Set the permission to **Read-only**. Data volumes in the path are read-only. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image3| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -.. _cce_01_0053__en-us_topic_0000001199341206_section10197243134710: +.. _cce_10_0377__section10197243134710: Secret ------ -You can mount a secret as a volume to the specified container path. Contents in a secret are user-defined. Before that, you need to create a secret. For details, see :ref:`Creating a Secret `. +You can mount a secret as a volume to the specified container path. Contents in a secret are user-defined. Before that, you need to create a secret. For details, see :ref:`Creating a Secret `. #. Log in to the CCE console. -#. When creating a workload, click **Data Storage** in the **Container Settings**. Click the **Local Volumes** tab and click |image4|. +#. When creating a workload, click **Data Storage** in the **Container Settings**. Click **Add Volume** and choose **Secret** from the drop-down list. -#. Set the local volume type to **Secret** and set parameters for adding a local volume, as shown in :ref:`Table 4 `. +#. Set the local volume type to **Secret** and set parameters for adding a local volume, as shown in :ref:`Table 4 `. - .. _cce_01_0053__en-us_topic_0000001199341206_table861818920109: + .. _cce_10_0377__table861818920109: .. table:: **Table 4** Setting parameters for mounting a secret volume @@ -202,13 +204,15 @@ You can mount a secret as a volume to the specified container path. Contents in +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Secret | Select the desired secret name. | | | | - | | A secret must be created in advance. For details, see :ref:`Creating a Secret `. | + | | A secret must be created in advance. For details, see :ref:`Creating a Secret `. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Add Container Path | Configure the following parameters: | | | | | | a. **subPath**: Enter a subpath, for example, **tmp**. | | | | - | | A subpath is used to mount a local disk so that the same data volume is used in a single pod. If this parameter is left blank, the root path is used by default. | + | | - A subpath is used to mount a local volume so that the same data volume is used in a single pod. | + | | - The subpath can be the key and value of a ConfigMap or secret. If the subpath is a key-value pair that does not exist, the data import does not take effect. | + | | - The data imported by specifying a subpath will not be updated along with the ConfigMap/secret updates. | | | | | | b. **Container Path**: Enter the path of the container, for example, **/tmp**. | | | | @@ -221,7 +225,7 @@ You can mount a secret as a volume to the specified container path. Contents in | | | | | c. Set the permission to **Read-only**. Data volumes in the path are read-only. | | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | + | | You can click |image4| to add multiple paths and subpaths. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Mounting a hostPath Volume Using kubectl @@ -229,7 +233,7 @@ Mounting a hostPath Volume Using kubectl You can use kubectl to mount a file directory of the host where the container is located to a specified mount path of the container. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **hostPath-pod-example.yaml** file, which is used to create a pod. @@ -339,7 +343,7 @@ You can use kubectl to mount a file directory of the host where the container is -rw-r--r-- 1 root root 0 Jun 1 08:12 test1 -rw-r--r-- 1 root root 0 Jun 1 08:14 test2 -.. |image1| image:: /_static/images/en-us_image_0000001409700089.png -.. |image2| image:: /_static/images/en-us_image_0000001359980148.png -.. |image3| image:: /_static/images/en-us_image_0000001360140128.png -.. |image4| image:: /_static/images/en-us_image_0000001409740389.png +.. |image1| image:: /_static/images/en-us_image_0000001465197524.png +.. |image2| image:: /_static/images/en-us_image_0000001515917789.png +.. |image3| image:: /_static/images/en-us_image_0000001464878016.png +.. |image4| image:: /_static/images/en-us_image_0000001515838557.png diff --git a/umn/source/storage_csi/deployment_examples/evs_volumes/index.rst b/umn/source/storage_csi/deployment_examples/evs_volumes/index.rst deleted file mode 100644 index 4f51810..0000000 --- a/umn/source/storage_csi/deployment_examples/evs_volumes/index.rst +++ /dev/null @@ -1,16 +0,0 @@ -:original_name: cce_01_0044.html - -.. _cce_01_0044: - -EVS Volumes -=========== - -- :ref:`Using EVS Volumes ` -- :ref:`Creating a Pod Mounted with an EVS Volume ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - using_evs_volumes - creating_a_pod_mounted_with_an_evs_volume diff --git a/umn/source/storage_csi/deployment_examples/evs_volumes/using_evs_volumes.rst b/umn/source/storage_csi/deployment_examples/evs_volumes/using_evs_volumes.rst deleted file mode 100644 index bcac480..0000000 --- a/umn/source/storage_csi/deployment_examples/evs_volumes/using_evs_volumes.rst +++ /dev/null @@ -1,194 +0,0 @@ -:original_name: cce_01_0254.html - -.. _cce_01_0254: - -Using EVS Volumes -================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. - -Notes and Constraints ---------------------- - -- EVS disks cannot be attached across AZs and cannot be used by multiple workloads, multiple pods of the same workload, or multiple jobs. - -- Data in a shared disk cannot be shared between nodes in a CCE cluster. If the same EVS disk is attached to multiple nodes, read and write conflicts and data cache conflicts may occur. When creating a Deployment, you are advised to create only one pod if you want to use EVS disks. - -- For clusters earlier than v1.19.10, if an HPA policy is used to scale out a workload with EVS volumes mounted, the existing pods cannot be read or written when a new pod is scheduled to another node. - - For clusters of v1.19.10 and later, if an HPA policy is used to scale out a workload with EVS volume mounted, a new pod cannot be started because EVS disks cannot be attached. - -- When you create a StatefulSet and add a cloud storage volume, existing EVS volumes cannot be used. - -- EVS disks that have partitions or have non-ext4 file systems cannot be imported. - -- Container storage in CCE clusters of Kubernetes 1.13 or later version supports encryption. Currently, E2E encryption is supported only in certain regions. - -- EVS volumes cannot be created in specified enterprise projects. Only the default enterprise project is supported. - -Creating an EVS Disk --------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. Click **Create** **EVS Disk**. - -#. Configure basic disk information. :ref:`Table 1 ` describes the parameters. - - .. _cce_01_0254__table20328123218464: - - .. table:: **Table 1** Configuring basic disk information - - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+=====================================================================================================================================================================================================================================================================================================================================================================================+ - | \* PVC Name | **New PVC Name**: name of the PVC to be created. A storage volume is automatically created when a PVC is created. One PVC corresponds to one storage volume. The storage volume name is automatically generated when the PVC is created. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Name | Cluster where the EVS disk is deployed. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Select the namespace where the EVS disk is deployed. If you do not need to select a namespace, retain the default value. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Volume Capacity (GB) | Size of the storage to be created. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Access Mode | Access permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteOnce** (RWO): The volume can be mounted as read-write by a single node, and data reading and writing are supported based on a non-shared EVS volume. EVS volumes in RWO mode are supported since v1.13.10-r1. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Primary AZ | AZ to which the volume belongs. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Type | Type of the new EVS disk. | - | | | - | | - **Common I/O**: uses Serial Advanced Technology Attachment (SATA) drives to store data. | - | | - **High I/O**: uses serial attached SCSI (SAS) drives to store data. | - | | - **Ultra-high I/O**: uses solid state disk (SSD) drives to store data. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage Format | The default value is **CSI** and cannot be changed. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Encryption | **KMS Encryption** is deselected by default. | - | | | - | | After **KMS Encryption** is selected, Key Management Service (KMS), an easy-to-use and highly secure cloud service for your keys, will be used for EVS disks. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **EVSAccessKMS** indicates that EVS is granted the permission to access KMS. After EVS is authorized successfully, it can obtain KMS keys to encrypt and decrypt EVS systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - | | | - | | This function is supported only for clusters of v1.13.10 and later in certain regions. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Review your order, click **Submit**, and wait until the creation is successful. - - The file system is displayed in the list. When its status becomes **Normal**, the file system is created successfully. - -#. Click the volume name to view detailed information about the volume. - -Adding an EVS Volume --------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a Job `. During creation, expand **Data Storage** after adding a container. On the **Cloud Volume** tab page, click **Add Cloud Volume**. -#. Set the storage volume type to **EVS**. - - .. table:: **Table 2** Parameters required for mounting an EVS volume - - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+=================================================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | **EVS**: You can use EVS disks the same way you use traditional hard disks on servers. EVS disks deliver higher data reliability and I/O throughput and are easy to use. They can be used for file systems, databases, or other system software and applications that require block storage resources. | - | | | - | | .. caution:: | - | | | - | | CAUTION: | - | | | - | | - To attach an EVS disk to a workload, you must set the number of pods to **1** when creating the workload. If multiple pods are configured, you cannot attach EVS disks. | - | | - When you create a StatefulSet and add a cloud storage volume, existing EVS volumes cannot be used. | - | | - EVS disks cannot be attached across AZs and cannot be used by multiple workloads, multiple pods of the same workload, or multiple jobs. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | Select a created disk. If no disk is available, follow the prompts to create one. | - | | | - | | For the same cluster and namespace, you can use an existing storage volume when creating a Deployment (with **Allocation Mode** set to **Manual**). | - | | | - | | When creating a StatefulSet, you can only use a volume automatically allocated by the system (only **Automatic** is available for **Allocation Mode**). | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Automatic | If you select **Automatic**, you need to configure the following items: | - | | | - | | a. **Access Mode**: permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteOnce** (RWO): The volume can be mounted as read-write by a single node, and data reading and writing are supported based on a non-shared EVS volume. EVS volumes in RWO mode are supported since v1.13.10-r1. | - | | | - | | b. **Availability Zone**: AZ where the storage volume is located. Only the AZ where the node is located can be selected. | - | | | - | | c. **Sub-Type**: Select a storage subtype. | - | | | - | | - **Common I/O**: uses Serial Advanced Technology Attachment (SATA) drives to store data. | - | | - **High I/O**: uses serial attached SCSI (SAS) drives to store data. | - | | - **Ultra-high I/O**: uses solid state disk (SSD) drives to store data. | - | | | - | | d. **Storage Capacity**: Enter the storage capacity in the unit of GB. Ensure that the storage capacity quota is not exceeded; otherwise, creation will fail. | - | | | - | | e. **Storage Format**: The default value is **CSI**. | - | | | - | | The container storage interface (CSI) is used to establish a set of standard storage management interfaces between Kubernetes and external storage systems to provide storage services for containers. | - | | | - | | f. After you select **KMS Encryption**, Key Management Service (KMS), an easy-to-use and highly secure service, will be enabled for EVS disks. This function is supported only for clusters of v1.13.10 and later in certain regions. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **EVSAccessKMS** indicates that EVS is granted the permission to access KMS. After EVS is authorized successfully, it can obtain KMS keys to encrypt and decrypt EVS systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | a. Click **Add Container Path**. | - | | b. **Container Path**: Enter the container path to which the data volume is mounted. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | | - | | - Do not mount a data volume to a system directory such as **/** or **/var/run**; this action may cause a container error to occur. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | - If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | c. Set permissions. | - | | | - | | - **Read-only**: You can only read the data volumes mounted to the path. | - | | - **Read/Write**: You can modify the data volumes mounted to the path. Newly written data is not migrated if the container is migrated, which may cause a data loss. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Importing an EVS Disk ---------------------- - -CCE allows you to import existing EVS disks. - -.. note:: - - An EVS disk can be imported into only one namespace. If an EVS disk has been imported into a namespace, it is invisible in other namespaces and cannot be imported again. **If you want to import an EVS disk that has file system (ext4) formatted, ensure that no partition has been created for the disk. Otherwise, data may be lost.** - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **EVS** tab page, click **Import**. -#. Select one or more EVS disks that you want to import. Then, click **OK**. - -Unbinding an EVS Disk ---------------------- - -After an EVS volume is successfully created or imported, the EVS volume is automatically bound to the current cluster and cannot be used by other clusters. When the volume is unbound from the cluster, other clusters can still use the volume. - -If the EVS volume has been mounted to a workload, it cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the EVS disk list, click **Unbind** next to the target EVS disk. -#. Confirm the unbinding, and click **OK**. - -Related Operations ------------------- - -After an EVS volume is created, you can perform operations described in :ref:`Table 3 `. - -.. _cce_01_0254__table1619535674020: - -.. table:: **Table 3** Other operations - - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==========================================================================================+ - | Deleting an EVS volume | #. Select the EVS volume to be deleted and click **Delete** in the **Operation** column. | - | | #. Follow the prompts to delete the EVS volume. | - +-----------------------------------+------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_csi/deployment_examples/index.rst b/umn/source/storage_csi/deployment_examples/index.rst deleted file mode 100644 index 7f14503..0000000 --- a/umn/source/storage_csi/deployment_examples/index.rst +++ /dev/null @@ -1,20 +0,0 @@ -:original_name: cce_01_0393.html - -.. _cce_01_0393: - -Deployment Examples -=================== - -- :ref:`EVS Volumes ` -- :ref:`SFS Turbo Volumes ` -- :ref:`OBS Volumes ` -- :ref:`SFS Volumes ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - evs_volumes/index - sfs_turbo_volumes/index - obs_volumes/index - sfs_volumes/index diff --git a/umn/source/storage_csi/deployment_examples/obs_volumes/index.rst b/umn/source/storage_csi/deployment_examples/obs_volumes/index.rst deleted file mode 100644 index f4567c3..0000000 --- a/umn/source/storage_csi/deployment_examples/obs_volumes/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -:original_name: cce_01_0160.html - -.. _cce_01_0160: - -OBS Volumes -=========== - -- :ref:`Using OBS Volumes ` -- :ref:`Creating a Deployment Mounted with an OBS Volume ` -- :ref:`Creating a StatefulSet Mounted with an OBS Volume ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - using_obs_volumes - creating_a_deployment_mounted_with_an_obs_volume - creating_a_statefulset_mounted_with_an_obs_volume diff --git a/umn/source/storage_csi/deployment_examples/obs_volumes/using_obs_volumes.rst b/umn/source/storage_csi/deployment_examples/obs_volumes/using_obs_volumes.rst deleted file mode 100644 index 9b62f3e..0000000 --- a/umn/source/storage_csi/deployment_examples/obs_volumes/using_obs_volumes.rst +++ /dev/null @@ -1,200 +0,0 @@ -:original_name: cce_01_0265.html - -.. _cce_01_0265: - -Using OBS Volumes -================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. - -Notes and Constraints ---------------------- - -- CCE clusters of v1.7.3-r8 and earlier do not support OBS volumes. You need to upgrade these clusters or create clusters of a later version that supports OBS. -- Volumes cannot be created in specified enterprise projects. Only the default enterprise project is supported. - -Preparations ------------- - -To mount reliable and stable OBS buckets as volumes, you must create AK/SK before you create OBS buckets. - -The procedure for configuring the AK/SK is as follows: - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. - -#. On the **OBS** tab page, click **AK/SK** in the notice. - - - .. figure:: /_static/images/en-us_image_0000001190538605.png - :alt: **Figure 1** Configuring the AK/SK - - **Figure 1** Configuring the AK/SK - -#. Click |image1|, select a key file, and click **Upload** to upload the key file. - -#. Select the corresponding workload and click **Restart**. - -.. important:: - - When creating an OBS volume, you must use the AK/SK. If the key file is not uploaded, the pod will fail to be started or OBS data access will be abnormal due to the volume mounting failure. - -Creating an OBS Volume ----------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. - -#. Click the **OBS** tab and click **Create OBS Bucket**. - -#. Configure basic information, as shown in :ref:`Table 1 `. - - .. _cce_01_0265__table20328123218464: - - .. table:: **Table 1** Parameters for creating an OBS volume - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Parameter Description | - +===================================+================================================================================================================================================================================================================================================================================+ - | \* PVC Name | Name of the new PVC, which is different from the volume name. The actual volume name is automatically generated when the PV is created by the PVC. | - | | | - | | The name contains 3 to 55 characters (excluding the prefix). It must contain lowercase letters, digits, and hyphens (-), and cannot start or end with a hyphen (-). | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Name | Cluster to which the OBS volume belongs. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace to which the volume belongs. The default value is **default**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Instance Type | Type of the storage instance created on OBS. | - | | | - | | - **Parallel file system**: **If the cluster version is v1.15 or later and the everest add-on version is 1.0.2 or later**, parallel file systems that can be mounted by obsfs can be created. | - | | - **Object bucket**: A bucket is a container for storing objects in OBS. OBS provides flat storage in the form of buckets and objects. Unlike the conventional multi-layer directory structure of file systems, all objects in a bucket are stored at the same logical layer. | - | | | - | | .. note:: | - | | | - | | Parallel file systems are optimized OBS objects. You are advised to **use parallel file systems** instead of object buckets to mount OBS volumes to containers. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage Class | This parameter is displayed when you select **Object bucket** for **Instance Type**. | - | | | - | | This parameter indicates the storage classes supported by OBS. | - | | | - | | - **Standard**\ : applicable to scenarios where a large number of hotspot files or small-sized files need to be accessed frequently (multiple times per month on average) and require fast access response. | - | | - **Infrequent access**: applicable to scenarios where data is not frequently accessed (less than 12 times per year on average) but requires fast access response. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage Policy | Object storage has the following policies: | - | | | - | | **Private**: Only the bucket owner has full control over the bucket. Unauthorized users do not have permissions to access the bucket. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Access Mode | Access permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteMany** (RWX): The volume is mounted as read-write by multiple nodes. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage Format | The default type is **CSI**. | - | | | - | | The container storage interface (CSI) is used to establish a set of standard storage management interfaces between Kubernetes and external storage systems to provide storage services for containers. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **Create**. - - After the OBS volume is successfully created, it is displayed in the OBS volume list. Click the PVC name to view detailed information about the OBS volume. - -Adding an OBS Volume --------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, :ref:`Creating a DaemonSet `, or :ref:`Creating a Job `. After you have added a container, choose **Data Storage** > **Cloud Volume**, and then click **Add Cloud Volume**. -#. Set **Type** to **OBS**. - - .. table:: **Table 2** OBS volume parameters - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+========================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | Select **OBS**. | - | | | - | | **OBS**: Standard and Infrequent Access OBS buckets are supported. OBS buckets are commonly used for big data analytics, cloud native applications, static website hosting, and backup/active archiving. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | **Name**: Select a created OBS volume. | - | | | - | | **Sub-Type**: class of the selected volume. The value can be **Standard** or **Infrequent access**, and you do not need to set this parameter. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Automatic | Type of the storage instance created on OBS. | - | | | - | | - **Parallel file system**: **If the cluster version is v1.15 or later and the everest add-on version is 1.0.2 or later**, parallel file systems that can be mounted by obsfs can be created. | - | | | - | | **Storage Format**: The default value is **CSI**. | - | | | - | | - **Object bucket**: A bucket is a container for storing objects in OBS. | - | | | - | | **Sub-Type**: Select **Standard** or **Infrequent access**. | - | | | - | | **Storage Format**: The default value is **CSI**. | - | | | - | | .. note:: | - | | | - | | Parallel file systems are optimized OBS objects. You are advised to **use parallel file systems** instead of object buckets to mount OBS volumes to containers. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | Configure the following parameters: | - | | | - | | a. **Container Path**: Enter the mount path in the container, for example, **/tmp**. | - | | | - | | The mount path must not be a system directory, such as **/** and **/var/run**. Otherwise, an exception occurs. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | b. Set permissions. | - | | | - | | - **Read-only**: You can only read the data in the mounted volumes. | - | | - **Read/Write**: You can modify the data in the mounted volumes. Newly written data is not migrated if the container is migrated, which causes a data loss. | - | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Importing an OBS Volume ------------------------ - -CCE allows you to import existing OBS volumes. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **OBS** tab page, click **Import**. -#. Select one or more OBS volumes that you want to import. - - .. note:: - - Parallel file systems are optimized OBS objects. You are advised to **use parallel file systems** instead of object buckets to mount OBS volumes to containers. - -#. Select the target cluster and namespace. -#. Click **OK**. - -Unbinding an OBS Volume ------------------------ - -When an OBS volume is successfully created, the OBS volume is automatically bound to the current cluster. Other clusters can also use the OBS volume. When the volume is unbound from the cluster, other clusters can still use the volume. - -If the volume has been mounted to a workload, the volume cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the OBS volume list, click **Unbind** next to the target OBS volume. -#. In the dialog box displayed, click **Yes**. - -Related Operations ------------------- - -After an OBS volume is created, you can perform the operation described in :ref:`Table 3 `. - -.. _cce_01_0265__table1619535674020: - -.. table:: **Table 3** Other Operations - - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==========================================================================================+ - | Deleting an OBS volume | #. Select the OBS volume to be deleted and click **Delete** in the **Operation** column. | - | | #. Follow the prompts to delete the volume. | - +-----------------------------------+------------------------------------------------------------------------------------------+ - -.. |image1| image:: /_static/images/en-us_image_0000001088110417.png diff --git a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/creating_a_deployment_mounted_with_an_sfs_turbo_volume.rst b/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/creating_a_deployment_mounted_with_an_sfs_turbo_volume.rst deleted file mode 100644 index 1f753c5..0000000 --- a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/creating_a_deployment_mounted_with_an_sfs_turbo_volume.rst +++ /dev/null @@ -1,85 +0,0 @@ -:original_name: cce_01_0274.html - -.. _cce_01_0274: - -Creating a Deployment Mounted with an SFS Turbo Volume -====================================================== - -Scenario --------- - -After an SFS Turbo volume is created or imported to CCE, you can mount the volume to a workload. - -Prerequisites -------------- - -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. - -Notes and Constraints ---------------------- - -The following configuration example applies to clusters of Kubernetes 1.15 or later. - -Procedure ---------- - -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -#. Run the following commands to configure the **sfsturbo-deployment-example.yaml** file, which is used to create a Deployment: - - **touch sfsturbo-deployment-example.yaml** - - **vi sfsturbo-deployment-example.yaml** - - Example of mounting an SFS Turbo volume to a Deployment (PVC-based, shared volume): - - .. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: sfsturbo-deployment-example # Workload name - namespace: default - spec: - replicas: 1 - selector: - matchLabels: - app: sfsturbo-deployment-example - template: - metadata: - labels: - app: sfsturbo-deployment-example - spec: - containers: - - image: nginx - name: container-0 - volumeMounts: - - mountPath: /tmp # Mount path - name: pvc-sfsturbo-example - restartPolicy: Always - imagePullSecrets: - - name: default-secret - volumes: - - name: pvc-sfsturbo-example - persistentVolumeClaim: - claimName: pvc-sfsturbo-example # PVC name - - .. table:: **Table 1** Key parameters - - +-----------+---------------------------------------------------------------------------+ - | Parameter | Description | - +===========+===========================================================================+ - | name | Name of the created Deployment. | - +-----------+---------------------------------------------------------------------------+ - | app | Name of the Deployment. | - +-----------+---------------------------------------------------------------------------+ - | mountPath | Mount path of the container. In this example, the mount path is **/tmp**. | - +-----------+---------------------------------------------------------------------------+ - - .. note:: - - **spec.template.spec.containers.volumeMounts.name** and **spec.template.spec.volumes.name** must be consistent because they have a mapping relationship. - -#. Run the following command to create the workload: - - **kubectl create -f sfsturbo-deployment-example.yaml** diff --git a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/creating_a_statefulset_mounted_with_an_sfs_turbo_volume.rst b/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/creating_a_statefulset_mounted_with_an_sfs_turbo_volume.rst deleted file mode 100644 index 6de402a..0000000 --- a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/creating_a_statefulset_mounted_with_an_sfs_turbo_volume.rst +++ /dev/null @@ -1,177 +0,0 @@ -:original_name: cce_01_0273.html - -.. _cce_01_0273: - -Creating a StatefulSet Mounted with an SFS Turbo Volume -======================================================= - -Scenario --------- - -CCE allows you to use an existing SFS Turbo volume to create a StatefulSet. - -Prerequisites -------------- - -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. - -Notes and Constraints ---------------------- - -The following configuration example applies to clusters of Kubernetes 1.15 or later. - -Procedure ---------- - -#. Create an SFS Turbo volume and record the volume name. - -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - -#. Create a YAML file for creating the workload. Assume that the file name is **sfsturbo-statefulset-example.yaml**. - - **touch sfsturbo-statefulset-example.yaml** - - **vi sfsturbo-statefulset-example.yaml** - - Configuration example: - - .. code-block:: - - apiVersion: apps/v1 - kind: StatefulSet - metadata: - name: sfsturbo-statefulset-example - namespace: default - spec: - replicas: 1 - selector: - matchLabels: - app: sfsturbo-statefulset-example - template: - metadata: - labels: - app: sfsturbo-statefulset-example - spec: - volumes: - - name: pvc-sfsturbo-example - persistentVolumeClaim: - claimName: pvc-sfsturbo-example - containers: - - name: container-0 - image: 'nginx:latest' - volumeMounts: - - name: pvc-sfsturbo-example - mountPath: /tmp - restartPolicy: Always - imagePullSecrets: - - name: default-secret - serviceName: sfsturbo-statefulset-example-headless - updateStrategy: - type: RollingUpdate - - .. table:: **Table 1** Key parameters - - +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +=============+====================================================================================================================================+ - | replicas | Number of pods. | - +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | name | Name of the new workload. | - +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | image | Image used by the workload. | - +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | mountPath | Mount path of a container. | - +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | - +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | claimName | Name of an existing PVC. | - +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - - .. note:: - - **spec.template.spec.containers.volumeMounts.name** and **spec.template.spec.volumes.name** must be consistent because they have a mapping relationship. - -#. Create the StatefulSet. - - **kubectl create -f sfsturbo-statefulset-example.yaml** - -Verifying Persistent Storage of an SFS Turbo Volume ---------------------------------------------------- - -#. Query the pod and SFS Turbo volume of the deployed workload (for example, **sfsturbo-statefulset-example**). - - a. Run the following command to query the pod name of the workload: - - .. code-block:: - - kubectl get po | grep sfsturbo-statefulset-example - - Expected outputs: - - .. code-block:: - - sfsturbo-statefulset-example-0 1/1 Running 0 2m5s - - b. Run the following command to check whether an SFS Turbo volume is mounted to the **/tmp** directory: - - .. code-block:: - - kubectl exec sfsturbo-statefulset-example-0 -- mount|grep /tmp - - Expected outputs: - - .. code-block:: - - 192.168.0.108:/ on /tmp type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,nolock,noresvport,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.0.108,mountvers=3,mountport=20048,mountproto=tcp,local_lock=all,addr=192.168.0.108) - -#. Run the following command to create a file named **test** in the **/tmp** directory: - - .. code-block:: - - kubectl exec sfsturbo-statefulset-example-0 -- touch /tmp/test - -#. Run the following command to view the file in the **/tmp** directory: - - .. code-block:: - - kubectl exec sfsturbo-statefulset-example-0 -- ls -l /tmp - - Expected outputs: - - .. code-block:: - - -rw-r--r-- 1 root root 0 Jun 1 02:50 test - -#. Run the following command to delete the pod named **sfsturbo-statefulset-example-0**: - - .. code-block:: - - kubectl delete po sfsturbo-statefulset-example-0 - -#. Check whether the file still exists after the pod is rebuilt. - - a. Run the following command to query the name of the rebuilt pod: - - .. code-block:: - - kubectl get po - - Expected outputs: - - .. code-block:: - - sfsturbo-statefulset-example-0 1/1 Running 0 2m - - b. Run the following command to view the file in the **/tmp** directory: - - .. code-block:: - - kubectl exec sfsturbo-statefulset-example-0 -- ls -l /tmp - - Expected outputs: - - .. code-block:: - - -rw-r--r-- 1 root root 0 Jun 1 02:50 test - - The **test** file still exists after the pod is rebuilt, indicating that the data in the SFS Turbo volume can be persistently stored. diff --git a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/index.rst b/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/index.rst deleted file mode 100644 index 27654b6..0000000 --- a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -:original_name: cce_01_0125.html - -.. _cce_01_0125: - -SFS Turbo Volumes -================= - -- :ref:`Using SFS Turbo Volumes ` -- :ref:`Creating a Deployment Mounted with an SFS Turbo Volume ` -- :ref:`Creating a StatefulSet Mounted with an SFS Turbo Volume ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - using_sfs_turbo_volumes - creating_a_deployment_mounted_with_an_sfs_turbo_volume - creating_a_statefulset_mounted_with_an_sfs_turbo_volume diff --git a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/using_sfs_turbo_volumes.rst b/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/using_sfs_turbo_volumes.rst deleted file mode 100644 index 4f63cb9..0000000 --- a/umn/source/storage_csi/deployment_examples/sfs_turbo_volumes/using_sfs_turbo_volumes.rst +++ /dev/null @@ -1,83 +0,0 @@ -:original_name: cce_01_0271.html - -.. _cce_01_0271: - -Using SFS Turbo Volumes -======================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. - -Notes and Constraints ---------------------- - -- SFS Turbo volumes are available only in certain regions. -- Currently, SFS Turbo file systems cannot be directly created on CCE. -- Only an SFS Turbo file system in the same VPC as the cluster and in the same subnet as the node can be imported. -- Inbound ports (111, 445, 2049, 2051, and 20048) must be enabled for the security group to which the SFS Turbo file system belongs. - -.. _cce_01_0271__section57261325121712: - -Importing an SFS Turbo Volume ------------------------------ - -CCE allows you to import existing SFS Turbo volumes. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **SFS Turbo** tab page, click **Import**. -#. Select one or more SFS Turbo volumes that you want to import. -#. Select the cluster and namespace to which you want to import the volumes. -#. Click **Next**. The volumes are displayed in the list. When **PVC Status** becomes **Bound**, the volumes are imported successfully. - -Adding an SFS Turbo Volume --------------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, :ref:`Creating a DaemonSet `, or :ref:`Creating a Job `. After you have added a container, choose **Data Storage** > **Cloud Volume**, and then click **Add Cloud Volume**. -#. Set the storage volume type to **SFS Turbo**. - - .. table:: **Table 1** Parameters for configuring an SFS Turbo volume - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Parameter Description | - +===================================+========================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | **SFS Turbo**: applicable to DevOps, containerized microservices, and enterprise office applications. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | Select an existing SFS Turbo volume. You need to import SFS Turbo volumes in advance. For details, see :ref:`Importing an SFS Turbo Volume `. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | Configure the following parameters: | - | | | - | | a. **subPath**: Enter the subpath of the file storage, for example, **/tmp**. | - | | | - | | This parameter specifies a subpath inside the referenced volume instead of its root. If this parameter is not specified, the root path is used. Currently, only file storage is supported. The value must be a relative path and cannot start with a slash (/) or ../. | - | | | - | | b. **Container Path**: Enter the mount path in the container, for example, **/tmp**. | - | | | - | | The mount path must not be a system directory, such as **/** and **/var/run**. Otherwise, an exception occurs. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | c. Set permissions. | - | | | - | | - **Read-only**: You can only read the data in the mounted volumes. | - | | - **Read/Write**: You can modify the data in the mounted volumes. Newly written data is not migrated if the container is migrated, which causes a data loss. | - | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Unbinding an SFS Turbo Volume ------------------------------ - -When an SFS Turbo volume is successfully imported to a cluster, the volume is bound to the cluster. The volume can also be imported to other clusters. When the volume is unbound from the cluster, other clusters can still import and use the volume. - -If the SFS Turbo volume has been mounted to a workload, the volume cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the SFS Turbo volume list, click **Unbind** next to the target volume. -#. In the dialog box displayed, click **OK**. diff --git a/umn/source/storage_csi/deployment_examples/sfs_volumes/index.rst b/umn/source/storage_csi/deployment_examples/sfs_volumes/index.rst deleted file mode 100644 index 8ff230f..0000000 --- a/umn/source/storage_csi/deployment_examples/sfs_volumes/index.rst +++ /dev/null @@ -1,18 +0,0 @@ -:original_name: cce_01_0111.html - -.. _cce_01_0111: - -SFS Volumes -=========== - -- :ref:`Using SFS Volumes ` -- :ref:`Creating a Deployment Mounted with an SFS Volume ` -- :ref:`Creating a StatefulSet Mounted with an SFS Volume ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - using_sfs_volumes - creating_a_deployment_mounted_with_an_sfs_volume - creating_a_statefulset_mounted_with_an_sfs_volume diff --git a/umn/source/storage_csi/deployment_examples/sfs_volumes/using_sfs_volumes.rst b/umn/source/storage_csi/deployment_examples/sfs_volumes/using_sfs_volumes.rst deleted file mode 100644 index 49f0ebb..0000000 --- a/umn/source/storage_csi/deployment_examples/sfs_volumes/using_sfs_volumes.rst +++ /dev/null @@ -1,169 +0,0 @@ -:original_name: cce_01_0259.html - -.. _cce_01_0259: - -Using SFS Volumes -================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the CSI plug-in (:ref:`everest `) in the cluster. - -Notes and Constraints ---------------------- - -- SFS volumes are available only in certain regions. -- Container storage in CCE clusters of Kubernetes 1.13 or later version supports encryption. Currently, E2E encryption is supported only in certain regions. -- Volumes cannot be created in specified enterprise projects. Only the default enterprise project is supported. - -.. _cce_01_0259__section1191025105819: - -Creating an SFS Volume ----------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. - -#. On the **SFS** tab, click **Create SFS File System**. - -#. Configure basic information, as shown in :ref:`Table 1 `. - - .. _cce_01_0259__table20328123218464: - - .. table:: **Table 1** Parameters for creating an SFS volume - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Parameter Description | - +===================================+======================================================================================================================================================================================================================================================================================================================================================================================+ - | \* PVC Name | Name of the new PVC, which is different from the volume name. The actual volume name is automatically generated when the PV is created by the PVC. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Name | Cluster to which the file system volume belongs. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace in which the volume is created. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Total Capacity | The total capacity is the capacity of a single volume. Fees are charged by actual usage. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Access Mode | Access permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteMany** (RWX): The SFS volume can be mounted as read-write by multiple nodes. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage Format | The default value is **CSI** and cannot be changed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Encryption | **KMS Encryption** is deselected by default. | - | | | - | | After **KMS Encryption** is selected, Key Management Service (KMS), an easy-to-use and highly secure key service, will be used for SFS file systems. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **SFSAccessKMS** indicates that SFS is granted the permission to access KMS. After SFS is authorized successfully, it can obtain KMS keys to encrypt and decrypt file systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - | | | - | | This function is supported only for clusters of v1.13.10 and later in certain regions. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **Create**. - - The volume is displayed in the list. When **PVS Status** becomes **Bound**, the volume is created successfully. - -#. Click the volume name to view detailed information about the volume. - -Adding an SFS Volume --------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, :ref:`Creating a DaemonSet `, or :ref:`Creating a Job `. During creation, expand **Data Storage** after adding a container. On the **Cloud Volume** tab page, click **Add Cloud Volume**. -#. Set the storage class to **SFS**. - - .. table:: **Table 2** Parameters for mounting an SFS volume - - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Parameter Description | - +===================================+============================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | **File Storage (NFS)**: This type applies to a wide range of scenarios, including media processing, content management, big data, and application analysis. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | - **Name**: Select a created file system. You need to create a file system in advance. For details about how to create a file system, see :ref:`Creating an SFS Volume `. | - | | - **Sub-Type**: subtype of the created file storage. | - | | - **Storage Capacity**: This field is one of the PVC attributes. If the storage capacity has been expanded on the IaaS side, it is normal that the capacity values are inconsistent. The PVC capacity is the same as the storage entity capacity only after end-to-end container storage capacity expansion is supported for CCE clusters of v1.13. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Automatic | An SFS volume is created automatically. You need to enter the storage capacity. | - | | | - | | - **Sub-Type**: Select **NFS**. | - | | | - | | - **Storage Capacity**: Specify the total storage capacity, in GB. Ensure that the storage capacity quota is not exceeded; otherwise, creation will fail. | - | | | - | | - **Storage Format**: The default value is **CSI**. | - | | | - | | The container storage interface (CSI) is used to establish a set of standard storage management interfaces between Kubernetes and external storage systems to provide storage services for containers. | - | | | - | | - After you select **KMS Encryption**, Key Management Service (KMS), an easy-to-use and highly secure service, will be enabled for file systems. This function is supported only for clusters of v1.13.10 and later in certain regions. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **SFSAccessKMS** indicates that SFS is granted the permission to access KMS. After SFS is authorized successfully, it can obtain KMS keys to encrypt and decrypt file systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | Configure the following parameters: | - | | | - | | a. **subPath**: Enter the subpath of the file storage, for example, **/tmp**. | - | | | - | | If this parameter is not specified, the root path of the data volume is used by default. Currently, only file storage is supported. The value must be a relative path and cannot start with a slash (/) or ../. | - | | | - | | b. **Container Path**: Enter the path of the container, for example, **/tmp**. | - | | | - | | The container path must not be a system directory, such as **/** and **/var/run**. Otherwise, an exception occurs. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | c. Set permissions. | - | | | - | | - **Read-only**: You can only read the data volumes mounted to the path. | - | | - **Read/Write**: You can modify the data volumes mounted to the path. Newly written data is not migrated if the container is migrated, which may cause a data loss. | - | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Importing an SFS Volume ------------------------ - -CCE allows you to import existing SFS volumes. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **SFS** tab page, click **Import**. -#. Select one or more SFS volumes that you want to attach. -#. Select the target cluster and namespace. Then, click **OK**. - -Unbinding an SFS Volume ------------------------ - -When an SFS volume is successfully created or imported, the volume is automatically bound to the current cluster. Other clusters can also use the volume. When the SFS volume is unbound from the cluster, other clusters can still import and use the volume. - -If the SFS volume has been attached to a workload, the volume cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the SFS volume list, click **Unbind** next to the target volume. -#. Confirm the unbinding, and click **OK**. - -Related Operations ------------------- - -After an SFS volume is created, you can perform the operation described in :ref:`Table 3 `. - -.. _cce_01_0259__table1619535674020: - -.. table:: **Table 3** Other operations - - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==========================================================================================+ - | Deleting an SFS volume | #. Select the SFS volume to be deleted and click **Delete** in the **Operation** column. | - | | #. Follow the prompts to delete the EVS disk. | - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Importing an SFS volume | CCE allows you to import existing SFS volumes. | - | | | - | | #. On the **SFS** tab page, click **Import**. | - | | #. Select one or more SFS volumes that you want to attach. | - | | #. Select the target cluster and namespace. | - | | #. Click **Yes**. | - +-----------------------------------+------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_csi/index.rst b/umn/source/storage_csi/index.rst deleted file mode 100644 index 2636423..0000000 --- a/umn/source/storage_csi/index.rst +++ /dev/null @@ -1,30 +0,0 @@ -:original_name: cce_01_0042.html - -.. _cce_01_0042: - -Storage (CSI) -============= - -- :ref:`Overview ` -- :ref:`Using Local Disks as Storage Volumes ` -- :ref:`PersistentVolumes (PVs) ` -- :ref:`PersistentVolumeClaims (PVCs) ` -- :ref:`StorageClass ` -- :ref:`Snapshots and Backups ` -- :ref:`Using a Custom AK/SK to Mount an OBS Volume ` -- :ref:`Setting Mount Options ` -- :ref:`Deployment Examples ` - -.. toctree:: - :maxdepth: 1 - :hidden: - - overview - using_local_disks_as_storage_volumes - persistentvolumes_pvs - persistentvolumeclaims_pvcs - storageclass - snapshots_and_backups - using_a_custom_ak_sk_to_mount_an_obs_volume - setting_mount_options - deployment_examples/index diff --git a/umn/source/storage_flexvolume/flexvolume_overview.rst b/umn/source/storage_flexvolume/flexvolume_overview.rst index 0974f56..52d84ec 100644 --- a/umn/source/storage_flexvolume/flexvolume_overview.rst +++ b/umn/source/storage_flexvolume/flexvolume_overview.rst @@ -1,15 +1,15 @@ -:original_name: cce_01_0306.html +:original_name: cce_10_0306.html -.. _cce_01_0306: +.. _cce_10_0306: FlexVolume Overview =================== In container storage, you can use different types of volumes and mount them to containers in pods as many as you want. -In CCE, container storage is backed both by Kubernetes-native objects, such as emptyDir/hostPath volumes, secrets, and ConfigMaps, and by storage services. +In CCE, container storage is backed both by Kubernetes-native objects, such as emptyDir, hostPath, secret, and ConfigMap, and by cloud storage services. -CCE clusters of **1.13 and earlier versions** use the :ref:`storage-driver ` add-on to connect to storage services to support Kubernetes FlexVolume driver for container storage. The FlexVolume driver has been deprecated in favor of the Container Storage Interface (CSI). **The everest add-on for CSI is installed in CCE clusters of 1.15 and later versions by default.** For details, see :ref:`Overview `. +CCE clusters of **1.13 and earlier versions** use the :ref:`storage-driver ` add-on to connect to cloud storage services to support Kubernetes FlexVolume driver for container storage. The FlexVolume driver has been deprecated in favor of the Container Storage Interface (CSI). **The everest add-on for CSI is installed in CCE clusters of 1.15 and later versions by default.** For details, see :ref:`Overview `. .. note:: @@ -19,44 +19,8 @@ CCE clusters of **1.13 and earlier versions** use the :ref:`storage-driver `) is compatible with the CSI plug-in (:ref:`everest `). Clusters of v1.17 and later versions do not support FlexVolume any more. You need to use the everest add-on. For details about CSI and FlexVolume, see :ref:`Differences Between CSI and FlexVolume Plug-ins `. -- The FlexVolume plug-in will be maintained by Kubernetes developers, but new functionality will only be added to CSI. You are advised not to create storage that connects to the FlexVolume plug-in (storage-driver) in CCE any more. Otherwise, the storage resources may not function normally. - -.. _cce_01_0306__section86752053123513: - -Differences Between CSI and FlexVolume Plug-ins ------------------------------------------------ - -.. table:: **Table 1** CSI and FlexVolume - - +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Kubernetes Solution | CCE Add-on | Feature | Usage | - +=====================+=================+================================================================================================================================================================================================================================================================================================================================================================================================================================================+=========================================================================================================================================================================================================================================================================+ - | CSI | everest | CSI was developed as a standard for exposing arbitrary block and file storage storage systems to containerized workloads. Using CSI, third-party storage providers can deploy plugins exposing new storage systems in Kubernetes without having to touch the core Kubernetes code. In CCE, the everest add-on is installed by default in clusters of Kubernetes v1.15 and later to connect to storage services (EVS, OBS, SFS, and SFS Turbo). | The :ref:`everest ` add-on is installed by default in clusters of **v1.15 and later**. CCE will mirror the Kubernetes community by providing continuous support for updated CSI capabilities. | - | | | | | - | | | The everest add-on consists of two parts: | | - | | | | | - | | | - **everest-csi-controller** for storage volume creation, deletion, capacity expansion, and cloud disk snapshots | | - | | | - **everest-csi-driver** for mounting, unmounting, and formatting storage volumes on nodes | | - | | | | | - | | | For details, see :ref:`everest `. | | - +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | FlexVolume | storage-driver | FlexVolume is an out-of-tree plugin interface that has existed in Kubernetes since version 1.2 (before CSI). CCE provided FlexVolume volumes through the storage-driver add-on installed in clusters of Kubernetes v1.13 and earlier versions. This add-on connects clusters to storage services (EVS, OBS, SFS, and SFS Turbo). | For clusters of v1.13 or earlier that have been created, the installed FlexVolume plug-in (the storage-driver add-on in CCE) can still be used. CCE stops providing update support for this add-on, and you are advised to :ref:`upgrade these clusters `. | - | | | | | - | | | For details, see :ref:`storage-driver `. | | - +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -.. note:: - - - A cluster can use only one type of storage plug-ins. - - The FlexVolume plug-in cannot be replaced by the CSI plug-in in clusters of v1.13 or earlier. You can only upgrade these clusters. For details, see :ref:`Cluster Upgrade Between Major Versions `. - -Notice on Using Add-ons ------------------------ - -- To use the CSI plug-in (the :ref:`everest ` add-on in CCE), your cluster must be using **Kubernetes 1.15 or later**. This add-on is installed by default when you create a cluster of v1.15 or later. The FlexVolume plug-in (the :ref:`storage-driver ` add-on in CCE) is installed by default when you create a cluster of v1.13 or earlier. -- If your cluster is upgraded from v1.13 to v1.15, :ref:`storage-driver ` is replaced by everest (v1.1.6 or later) for container storage. The takeover does not affect the original storage functions. -- In version 1.2.0 of the everest add-on, **key authentication** is optimized when OBS is used. After the everest add-on is upgraded from a version earlier than 1.2.0, you need to restart all workloads that use OBS in the cluster. Otherwise, workloads may not be able to use OBS. +- For clusters created in CCE, Kubernetes v1.15.11 is a transitional version in which the FlexVolume plug-in (:ref:`storage-driver `) is compatible with the CSI plug-in (:ref:`everest `). Clusters of v1.17 and later versions do not support FlexVolume anymore. You need to use the everest add-on. +- The FlexVolume plug-in will be maintained by Kubernetes developers, but new functionality will only be added to CSI. You are advised not to create storage that connects to the FlexVolume plug-in (storage-driver) in CCE anymore. Otherwise, the storage resources may not function normally. Checking Storage Add-ons ------------------------ @@ -65,3 +29,30 @@ Checking Storage Add-ons #. In the navigation tree on the left, click **Add-ons**. #. Click the **Add-on Instance** tab. #. Select a cluster in the upper right corner. The default storage add-on installed during cluster creation is displayed. + +Differences Between CSI and FlexVolume Plug-ins +----------------------------------------------- + +.. table:: **Table 1** CSI and FlexVolume + + +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Kubernetes Solution | CCE Add-on | Feature | Recommendation | + +=====================+=================+================================================================================================================================================================================================================================================================================================================================================================================================================================================+================================================================================================================================================================================================================================================================================+ + | CSI | Everest | CSI was developed as a standard for exposing arbitrary block and file storage storage systems to containerized workloads. Using CSI, third-party storage providers can deploy plugins exposing new storage systems in Kubernetes without having to touch the core Kubernetes code. In CCE, the everest add-on is installed by default in clusters of Kubernetes v1.15 and later to connect to storage services (EVS, OBS, SFS, and SFS Turbo). | The :ref:`everest ` add-on is installed by default in clusters of **v1.15 and later**. CCE will mirror the Kubernetes community by providing continuous support for updated CSI capabilities. | + | | | | | + | | | The everest add-on consists of two parts: | | + | | | | | + | | | - **everest-csi-controller** for storage volume creation, deletion, capacity expansion, and cloud disk snapshots | | + | | | - **everest-csi-driver** for mounting, unmounting, and formatting storage volumes on nodes | | + | | | | | + | | | For details, see :ref:`everest `. | | + +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Flexvolume | storage-driver | FlexVolume is an out-of-tree plugin interface that has existed in Kubernetes since version 1.2 (before CSI). CCE provided FlexVolume volumes through the storage-driver add-on installed in clusters of Kubernetes v1.13 and earlier versions. This add-on connects clusters to storage services (EVS, OBS, SFS, and SFS Turbo). | For the created clusters of **v1.13 or earlier**, the installed FlexVolume plug-in (CCE add-on :ref:`storage-driver `) can still be used. CCE stops providing update support for this add-on, and you are advised to :ref:`upgrade these clusters `. | + | | | | | + | | | For details, see :ref:`storage-driver `. | | + +---------------------+-----------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +.. note:: + + - A cluster can use only one type of storage plug-ins. + - The FlexVolume plug-in cannot be replaced by the CSI plug-in in clusters of v1.13 or earlier. You can only upgrade these clusters. For details, see :ref:`Cluster Upgrade `. diff --git a/umn/source/storage_flexvolume/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst b/umn/source/storage_flexvolume/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst index bc3bcb2..7f529c5 100644 --- a/umn/source/storage_flexvolume/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst +++ b/umn/source/storage_flexvolume/how_do_i_change_the_storage_class_used_by_a_cluster_of_v1.15_from_flexvolume_to_csi_everest.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0343.html +:original_name: cce_10_0343.html -.. _cce_01_0343: +.. _cce_10_0343: How Do I Change the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest? ============================================================================================ @@ -18,7 +18,7 @@ Procedure #. (Optional) Back up data to prevent data loss in case of exceptions. -#. .. _cce_01_0343__en-us_topic_0285037038_li1219802032512: +#. .. _cce_10_0343__cce_bestpractice_0107_li1219802032512: Configure a YAML file of the PV in the CSI format according to the PV in the FlexVolume format and associate the PV with the existing storage. @@ -223,9 +223,9 @@ Procedure | storageClassName | Name of the Kubernetes storage class. Set this field to **csi-sfsturbo** for SFS Turbo volumes. | +----------------------------------+-------------------------------------------------------------------------------------------------------------------------+ -#. .. _cce_01_0343__en-us_topic_0285037038_li1710710385418: +#. .. _cce_10_0343__cce_bestpractice_0107_li1710710385418: - Configure a YAML file of the PVC in the CSI format according to the PVC in the FlexVolume format and associate the PVC with the PV created in :ref:`2 `. + Configure a YAML file of the PVC in the CSI format according to the PVC in the FlexVolume format and associate the PVC with the PV created in :ref:`2 `. To be specific, run the following commands to configure the pvc-example.yaml file, which is used to create a PVC. @@ -268,7 +268,7 @@ Procedure +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | failure-domain.beta.kubernetes.io/zone | AZ where the EVS disk is deployed. Use the same value as that of the FlexVolume PVC. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | everest.io/disk-volume-type | Storage class of the EVS disk. The value can be **SAS** or **SSD**. Set this parameter to the same value as that of the PV created in :ref:`2 `. | + | everest.io/disk-volume-type | Storage class of the EVS disk. The value can be **SAS** or **SSD**. Set this parameter to the same value as that of the PV created in :ref:`2 `. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | name | PVC name, which must be unique in the namespace. The value must be unique in the namespace. (If the PVC is dynamically created by a stateful application, the value of this parameter must be the same as the name of the FlexVolume PVC.) | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -276,7 +276,7 @@ Procedure +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage | Requested capacity of the PVC, which must be the same as the storage size of the existing PV. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`2 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`2 `. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storageClassName | Name of the Kubernetes storage class. Set this field to **csi-disk** for EVS disks. | +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -316,7 +316,7 @@ Procedure +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storageClassName | Set this field to **csi-nas**. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`2 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV in :ref:`2 `. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Configuration example of **a PVC for an OBS volume**: @@ -348,7 +348,7 @@ Procedure +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +============================+============================================================================================================================================================================================================================================+ - | everest.io/obs-volume-type | OBS volume type, which can be **STANDARD** (standard bucket) and **WARM** (infrequent access bucket). Set this parameter to the same value as that of the PV created in :ref:`2 `. | + | everest.io/obs-volume-type | OBS volume type, which can be **STANDARD** (standard bucket) and **WARM** (infrequent access bucket). Set this parameter to the same value as that of the PV created in :ref:`2 `. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | csi.storage.k8s.io/fstype | File type, which can be **obsfs** or **s3fs**. The value must be the same as that of **fsType** of the static OBS volume PV. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -360,7 +360,7 @@ Procedure +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storageClassName | Name of the Kubernetes storage class. Set this field to **csi-obs**. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`2 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`2 `. | +----------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ Configuration example of **a PVC for an SFS Turbo volume**: @@ -398,10 +398,10 @@ Procedure +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage | Storage capacity, in the unit of Gi. The value must be the same as the storage size of the existing PV. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`2 `. | + | volumeName | Name of the PV. Set this parameter to the name of the static PV created in :ref:`2 `. | +------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -#. .. _cce_01_0343__en-us_topic_0285037038_li487255772614: +#. .. _cce_10_0343__cce_bestpractice_0107_li487255772614: Upgrade the workload to use a new PVC. @@ -415,7 +415,7 @@ Procedure .. note:: - Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. + Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. b. Go to the CCE console. On the workload upgrade page, click **Upgrade** > **Advanced Settings** > **Data Storage** > **Cloud Storage**. @@ -435,7 +435,7 @@ Procedure .. note:: - Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. + Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. b. Run the **kubectl edit** command to edit the StatefulSet and use the newly created PVC. @@ -473,7 +473,7 @@ Procedure .. note:: - Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. + Replace the example file name **pvc-example.yaml** in the preceding commands with the names of the YAML files configured in :ref:`2 ` and :ref:`3 `. e. Change the number of pods back to the original value and wait until the pods are running. @@ -518,7 +518,7 @@ Procedure storage: 10Gi storageClassName: csi-disk - The parameter value must be the same as the PVC of the EVS volume created in :ref:`3 `. + The parameter value must be the same as the PVC of the EVS volume created in :ref:`3 `. Configuration example of **volumeClaimTemplates for an SFS volume**: @@ -537,7 +537,7 @@ Procedure storage: 10Gi storageClassName: csi-nas - The parameter value must be the same as the PVC of the SFS volume created in :ref:`3 `. + The parameter value must be the same as the PVC of the SFS volume created in :ref:`3 `. Configuration example of **volumeClaimTemplates for an OBS volume**: @@ -559,7 +559,7 @@ Procedure storage: 1Gi storageClassName: csi-obs - The parameter value must be the same as the PVC of the OBS volume created in :ref:`3 `. + The parameter value must be the same as the PVC of the OBS volume created in :ref:`3 `. - Delete the StatefulSet. @@ -576,7 +576,7 @@ Procedure .. note:: - If a rollback is required, perform :ref:`4 `. Select the PVC in FlexVolume format and upgrade the application. + If a rollback is required, perform :ref:`4 `. Select the PVC in FlexVolume format and upgrade the application. #. Uninstall the PVC in the FlexVolume format. diff --git a/umn/source/storage_flexvolume/index.rst b/umn/source/storage_flexvolume/index.rst index f6e2600..dfdecfc 100644 --- a/umn/source/storage_flexvolume/index.rst +++ b/umn/source/storage_flexvolume/index.rst @@ -1,16 +1,16 @@ -:original_name: cce_01_0305.html +:original_name: cce_10_0305.html -.. _cce_01_0305: +.. _cce_10_0305: Storage (FlexVolume) ==================== -- :ref:`FlexVolume Overview ` -- :ref:`How Do I Change the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest? ` -- :ref:`Using EVS Disks as Storage Volumes ` -- :ref:`Using SFS Turbo File Systems as Storage Volumes ` -- :ref:`Using OBS Buckets as Storage Volumes ` -- :ref:`Using SFS File Systems as Storage Volumes ` +- :ref:`FlexVolume Overview ` +- :ref:`How Do I Change the Storage Class Used by a Cluster of v1.15 from FlexVolume to CSI Everest? ` +- :ref:`Using EVS Disks as Storage Volumes ` +- :ref:`Using SFS Turbo File Systems as Storage Volumes ` +- :ref:`Using OBS Buckets as Storage Volumes ` +- :ref:`Using SFS File Systems as Storage Volumes ` .. toctree:: :maxdepth: 1 diff --git a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/index.rst b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/index.rst index 217601c..e2eedae 100644 --- a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/index.rst +++ b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/index.rst @@ -1,22 +1,20 @@ -:original_name: cce_01_0309.html +:original_name: cce_10_0309.html -.. _cce_01_0309: +.. _cce_10_0309: Using EVS Disks as Storage Volumes ================================== -- :ref:`Overview ` -- :ref:`Using EVS Volumes ` -- :ref:`(kubectl) Automatically Creating an EVS Disk ` -- :ref:`(kubectl) Creating a PV from an Existing EVS Disk ` -- :ref:`(kubectl) Creating a Pod Mounted with an EVS Volume ` +- :ref:`Overview ` +- :ref:`(kubectl) Automatically Creating an EVS Disk ` +- :ref:`(kubectl) Creating a PV from an Existing EVS Disk ` +- :ref:`(kubectl) Creating a Pod Mounted with an EVS Volume ` .. toctree:: :maxdepth: 1 :hidden: overview - using_evs_volumes kubectl_automatically_creating_an_evs_disk kubectl_creating_a_pv_from_an_existing_evs_disk kubectl_creating_a_pod_mounted_with_an_evs_volume diff --git a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_automatically_creating_an_evs_disk.rst b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_automatically_creating_an_evs_disk.rst index a23d4cc..950b26c 100644 --- a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_automatically_creating_an_evs_disk.rst +++ b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_automatically_creating_an_evs_disk.rst @@ -1,20 +1,10 @@ -:original_name: cce_01_0312.html +:original_name: cce_10_0312.html -.. _cce_01_0312: +.. _cce_10_0312: (kubectl) Automatically Creating an EVS Disk ============================================ -Scenario --------- - -CCE supports creating EVS volumes through PersistentVolumeClaims (PVCs). - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -23,7 +13,7 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **pvc-evs-auto-example.yaml** file, which is used to create a PVC. @@ -54,27 +44,21 @@ Procedure .. table:: **Table 1** Key parameters - +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +==========================================+======================================================================================================================================+ - | volume.beta.kubernetes.io/storage-class | EVS disk type. The value is in lowercase. | - | | | - | | Supported values: Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) | - +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | - | | | - | | For details about the value of **region**, see `Regions and Endpoints `__. | - +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | - | | | - | | For details about the value of **zone**, see `Regions and Endpoints `__. | - +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | storage | Storage capacity in the unit of Gi. | - +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | accessModes | Read/write mode of the volume. | - | | | - | | You can set this parameter to **ReadWriteMany** (shared volume) and **ReadWriteOnce** (non-shared volume). | - +------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ + +------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +==========================================+============================================================================================================+ + | volume.beta.kubernetes.io/storage-class | EVS disk type. The value is in lowercase. | + +------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | + +------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | + +------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | storage | Storage capacity in the unit of Gi. | + +------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | accessModes | Read/write mode of the volume. | + | | | + | | You can set this parameter to **ReadWriteMany** (shared volume) and **ReadWriteOnce** (non-shared volume). | + +------------------------------------------+------------------------------------------------------------------------------------------------------------+ #. Run the following command to create a PVC. diff --git a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pod_mounted_with_an_evs_volume.rst b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pod_mounted_with_an_evs_volume.rst index 371c843..a24f45b 100644 --- a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pod_mounted_with_an_evs_volume.rst +++ b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pod_mounted_with_an_evs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0314.html +:original_name: cce_10_0314.html -.. _cce_01_0314: +.. _cce_10_0314: (kubectl) Creating a Pod Mounted with an EVS Volume =================================================== @@ -14,11 +14,6 @@ After an EVS volume is created or imported to CCE, you can mount it to a workloa EVS disks cannot be attached across AZs. Before mounting a volume, you can run the **kubectl get pvc** command to query the available PVCs in the AZ where the current cluster is located. -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -27,7 +22,7 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **evs-deployment-example.yaml** file, which is used to create a Deployment. @@ -141,7 +136,7 @@ Procedure +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ | spec.template.spec.containers.volumeMount | mountPath | Mount path of the container. In this example, the volume is mounted to the **/tmp** directory. | +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ .. note:: diff --git a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst index f2dca19..cfe4975 100644 --- a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst +++ b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_evs_disk.rst @@ -1,20 +1,10 @@ -:original_name: cce_01_0313.html +:original_name: cce_10_0313.html -.. _cce_01_0313: +.. _cce_10_0313: (kubectl) Creating a PV from an Existing EVS Disk ================================================= -Scenario --------- - -CCE allows you to create a PersistentVolume (PV) using an existing EVS disk. After the PV is created, you can create a PersistentVolumeClaim (PVC) and bind it to the PV. - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -25,25 +15,25 @@ Procedure #. Log in to the EVS console, create an EVS disk, and record the volume ID, capacity, and disk type of the EVS disk. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create two YAML files for creating the PersistentVolume (PV) and PersistentVolumeClaim (PVC). Assume that the file names are **pv-evs-example.yaml** and **pvc-evs-example.yaml**. **touch pv-evs-example.yaml** **pvc-evs-example.yaml** +-------------------------------+--------------------------------+-----------------------------------------------------+ - | Kubernetes Version | Description | YAML Example | + | Kubernetes Cluster Version | Description | YAML Example | +===============================+================================+=====================================================+ - | 1.11.7 <= K8s version <= 1.13 | Clusters from v1.11.7 to v1.13 | :ref:`Example YAML ` | + | 1.11.7 <= K8s version <= 1.13 | Clusters from v1.11.7 to v1.13 | :ref:`Example YAML ` | +-------------------------------+--------------------------------+-----------------------------------------------------+ - | 1.11 <= K8s version < 1.11.7 | Clusters from v1.11 to v1.11.7 | :ref:`Example YAML ` | + | 1.11 <= K8s version < 1.11.7 | Clusters from v1.11 to v1.11.7 | :ref:`Example YAML ` | +-------------------------------+--------------------------------+-----------------------------------------------------+ - | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | + | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | +-------------------------------+--------------------------------+-----------------------------------------------------+ **Clusters from v1.11.7 to v1.13** - - .. _cce_01_0313__li0648350102513: + - .. _cce_10_0313__li0648350102513: **Example YAML file for the PV:** @@ -84,16 +74,12 @@ Procedure | Parameter | Description | +==========================================+===========================================================================================================================================================================================================================================================================================================================+ | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | - | | | - | | For details about the value of **region**, see `Regions and Endpoints `__. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | - | | | - | | For details about the value of **zone**, see `Regions and Endpoints `__. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage | EVS volume capacity in the unit of Gi. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | storageClassName | EVS disk type. Supported values: Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) | + | storageClassName | EVS disk type. Supported values: Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD) | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | driver | Storage driver. | | | | @@ -143,31 +129,27 @@ Procedure .. table:: **Table 2** Key parameters - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===============================================+======================================================================================================================================+ - | volume.beta.kubernetes.io/storage-class | Storage class, which must be the same as that of the existing PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | volume.beta.kubernetes.io/storage-provisioner | The field must be set to **flexvolume-huawei.com/fuxivol**. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | - | | | - | | For details about the value of **region**, see `Regions and Endpoints `__. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | - | | | - | | For details about the value of **zone**, see `Regions and Endpoints `__. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | storage | Requested capacity in the PVC, in Gi. | - | | | - | | The value must be the same as the storage size of the existing PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===============================================+=============================================================================================+ + | volume.beta.kubernetes.io/storage-class | Storage class, which must be the same as that of the existing PV. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | volume.beta.kubernetes.io/storage-provisioner | The field must be set to **flexvolume-huawei.com/fuxivol**. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | storage | Requested capacity in the PVC, in Gi. | + | | | + | | The value must be the same as the storage size of the existing PV. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | volumeName | Name of the PV. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ **Clusters from v1.11 to v1.11.7** - - .. _cce_01_0313__li19211184720504: + - .. _cce_10_0313__li19211184720504: **Example YAML file for the PV:** @@ -200,16 +182,12 @@ Procedure | Parameter | Description | +==========================================+===========================================================================================================================================================================================================================================================================================================================+ | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | - | | | - | | For details about the value of **region**, see `Regions and Endpoints `__. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | - | | | - | | For details about the value of **zone**, see `Regions and Endpoints `__. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage | EVS volume capacity in the unit of Gi. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | storageClassName | EVS disk type. Supported values: Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) | + | storageClassName | EVS disk type. Supported values: Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD) | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | driver | Storage driver. | | | | @@ -251,31 +229,27 @@ Procedure .. table:: **Table 4** Key parameters - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===============================================+======================================================================================================================================+ - | volume.beta.kubernetes.io/storage-class | Storage class. The value can be **sas** or **ssd**. The value must be the same as that of the existing PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | volume.beta.kubernetes.io/storage-provisioner | The field must be set to **flexvolume-huawei.com/fuxivol**. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | - | | | - | | For details about the value of **region**, see `Regions and Endpoints `__. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | - | | | - | | For details about the value of **zone**, see `Regions and Endpoints `__. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | storage | Requested capacity in the PVC, in Gi. | - | | | - | | The value must be the same as the storage size of the existing PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===============================================+============================================================================================================+ + | volume.beta.kubernetes.io/storage-class | Storage class. The value can be **sas** or **ssd**. The value must be the same as that of the existing PV. | + +-----------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | volume.beta.kubernetes.io/storage-provisioner | The field must be set to **flexvolume-huawei.com/fuxivol**. | + +-----------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | + +-----------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | + +-----------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | storage | Requested capacity in the PVC, in Gi. | + | | | + | | The value must be the same as the storage size of the existing PV. | + +-----------------------------------------------+------------------------------------------------------------------------------------------------------------+ + | volumeName | Name of the PV. | + +-----------------------------------------------+------------------------------------------------------------------------------------------------------------+ **Clusters of v1.9** - - .. _cce_01_0313__li813222310297: + - .. _cce_10_0313__li813222310297: **Example YAML file for the PV:** @@ -310,16 +284,12 @@ Procedure | Parameter | Description | +==========================================+===========================================================================================================================================================================================================================================================================================================================+ | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | - | | | - | | For details about the value of **region**, see `Regions and Endpoints `__. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | - | | | - | | For details about the value of **zone**, see `Regions and Endpoints `__. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage | EVS volume capacity in the unit of Gi. | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | storageClassName | EVS disk type. Supported values: Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD)High I/O (SAS) and Ultra-high I/O (SSD) | + | storageClassName | EVS disk type. Supported values: Common I/O (SATA), High I/O (SAS), and Ultra-high I/O (SSD) | +------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | driver | Storage driver. | | | | @@ -362,27 +332,23 @@ Procedure .. table:: **Table 6** Key parameters - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===============================================+======================================================================================================================================+ - | volume.beta.kubernetes.io/storage-class | Storage class, which must be the same as that of the existing PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | volume.beta.kubernetes.io/storage-provisioner | The field must be set to **flexvolume-huawei.com/fuxivol**. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | - | | | - | | For details about the value of **region**, see `Regions and Endpoints `__. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | - | | | - | | For details about the value of **zone**, see `Regions and Endpoints `__. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | storage | Requested capacity in the PVC, in Gi. | - | | | - | | The value must be the same as the storage size of the existing PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ - | volumeName | Name of the PV. | - +-----------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------+ + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===============================================+=============================================================================================+ + | volume.beta.kubernetes.io/storage-class | Storage class, which must be the same as that of the existing PV. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | volume.beta.kubernetes.io/storage-provisioner | The field must be set to **flexvolume-huawei.com/fuxivol**. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/region | Region where the cluster is located. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | failure-domain.beta.kubernetes.io/zone | AZ where the EVS volume is created. It must be the same as the AZ planned for the workload. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | storage | Requested capacity in the PVC, in Gi. | + | | | + | | The value must be the same as the storage size of the existing PV. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ + | volumeName | Name of the PV. | + +-----------------------------------------------+---------------------------------------------------------------------------------------------+ #. Create the PV. @@ -400,11 +366,11 @@ Procedure If you skip this step in this example or when creating a static PV or PVC, ensure that the EVS disk associated with the static PV has been unbound from the node before you delete the node. - a. .. _cce_01_0313__li6891526204113: + a. .. _cce_10_0313__li6891526204113: Obtain the tenant token. For details, see `Obtaining a User Token `__. - b. .. _cce_01_0313__li17017349418: + b. .. _cce_10_0313__li17017349418: Obtain the EVS access address **EVS_ENDPOINT**. For details, see `Regions and Endpoints `__. @@ -422,9 +388,9 @@ Procedure +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | +===============+========================================================================================================================================================================================================================================================+ - | EVS_ENDPOINT | EVS access address. Set this parameter to the value obtained in :ref:`6.b `. | + | EVS_ENDPOINT | EVS access address. Set this parameter to the value obtained in :ref:`6.b `. | +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | project_id | Project ID. You can click the login user in the upper right corner of the console page, select **My Credentials** from the drop-down list, and view the project ID on the **Projects** tab page. | + | project_id | Project ID. | +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | volume_id | ID of the associated EVS disk. Set this parameter to **volume_id** of the static PV to be created. You can also log in to the EVS console, click the name of the EVS disk to be imported, and obtain the ID from **Summary** on the disk details page. | +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ @@ -432,7 +398,7 @@ Procedure +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | pvc_namespace | Namespace where the PVC is to be bound. | +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | TOKEN | User token. Set this parameter to the value obtained in :ref:`6.a `. | + | TOKEN | User token. Set this parameter to the value obtained in :ref:`6.a `. | +---------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ For example, run the following commands: diff --git a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/overview.rst b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/overview.rst index f7bb8c9..5720bcb 100644 --- a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/overview.rst +++ b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0310.html +:original_name: cce_10_0310.html -.. _cce_01_0310: +.. _cce_10_0310: Overview ======== @@ -8,7 +8,7 @@ Overview To achieve persistent storage, CCE allows you to mount the storage volumes created from Elastic Volume Service (EVS) disks to a path of a container. When the container is migrated, the mounted EVS volumes are also migrated. By using EVS volumes, you can mount the remote file directory of storage system into a container so that data in the data volume is permanently preserved even when the container is deleted. -.. figure:: /_static/images/en-us_image_0276664178.png +.. figure:: /_static/images/en-us_image_0000001248663503.png :alt: **Figure 1** Mounting EVS volumes to CCE **Figure 1** Mounting EVS volumes to CCE diff --git a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/using_evs_volumes.rst b/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/using_evs_volumes.rst deleted file mode 100644 index 5c84e89..0000000 --- a/umn/source/storage_flexvolume/using_evs_disks_as_storage_volumes/using_evs_volumes.rst +++ /dev/null @@ -1,178 +0,0 @@ -:original_name: cce_01_0311.html - -.. _cce_01_0311: - -Using EVS Volumes -================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - -Notes and Constraints ---------------------- - -- EVS disks cannot be attached across AZs and cannot be used by multiple workloads, multiple pods of the same workload, or multiple jobs. -- Data in a shared disk cannot be shared between nodes in a CCE cluster. If the same EVS disk is attached to multiple nodes, read and write conflicts and data cache conflicts may occur. When creating a Deployment, you are advised to create only one pod if you want to use EVS disks. -- When you create a StatefulSet and add a cloud storage volume, existing EVS volumes cannot be used. -- EVS disks that have partitions or have non-ext4 file systems cannot be imported. -- Container storage in CCE clusters of Kubernetes 1.13 or later version supports encryption. Currently, E2E encryption is supported only in certain regions. -- Volumes cannot be created in specified enterprise projects. Only the default enterprise project is supported. -- The following operations apply to clusters of Kubernetes 1.13 or earlier. - -Buying an EVS Disk ------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. Click **Create EVS Disk**. - -#. Configure basic disk information. :ref:`Table 1 ` describes the parameters. - - .. _cce_01_0311__table20328123218464: - - .. table:: **Table 1** Configuring basic disk information - - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+=====================================================================================================================================================================================================================================================================================================================================================================================+ - | \* PVC Name | **New PVC Name**: name of the PVC to be created. A storage volume is automatically created when a PVC is created. One PVC corresponds to one storage volume. The storage volume name is automatically generated when the PVC is created. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Name | Cluster where the EVS disk is deployed. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace where the EVS disk is deployed. You can retain the default value or specify one. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Volume Capacity (GB) | Size of the storage to be created. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Access Mode | Access permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteOnce** (RWO): The volume can be mounted as read-write by a single node, and data reading and writing are supported based on a non-shared EVS volume. EVS volumes in RWO mode are supported since v1.13.10-r1. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | AZ | AZ to which the disk belongs. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Type | Type of the new EVS disk. | - | | | - | | - **Common I/O**: uses Serial Advanced Technology Attachment (SATA) drives to store data. | - | | - **High I/O**: uses serial attached SCSI (SAS) drives to store data. | - | | - **Ultra-high I/O**: uses solid state disk (SSD) drives to store data. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Encryption | **KMS Encryption** is deselected by default. | - | | | - | | After **KMS Encryption** is selected, Key Management Service (KMS), an easy-to-use and highly secure cloud service for your keys, will be used for EVS disks. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **EVSAccessKMS** indicates that EVS is granted the permission to access KMS. After EVS is authorized successfully, it can obtain KMS keys to encrypt and decrypt EVS systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - | | | - | | This function is supported only for clusters of v1.13.10 and later in certain regions. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Review your order, click **Submit**, and wait until the creation is successful. - - The file system is displayed in the list. When its status becomes **Normal**, the file system is created successfully. - -#. Click the volume name to view detailed information about the volume. - -Adding an EVS Volume --------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, or :ref:`Creating a Job `. During creation, expand **Data Storage** after adding a container. On the **Cloud Volume** tab page, click **Add Cloud Volume**. -#. Set the storage volume type to **EVS**. - - .. table:: **Table 2** Parameters required for mounting an EVS volume - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+====================================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | **EVS**: You can use EVS disks the same way you use traditional hard disks on servers. EVS disks deliver higher data reliability and I/O throughput and are easy to use. They can be used for file systems, databases, or other system software and applications that require block storage resources. | - | | | - | | .. caution:: | - | | | - | | CAUTION: | - | | | - | | - To attach an EVS disk to a workload, you must set the number of pods to **1** when creating the workload. If multiple pods are created, you cannot attach EVS disks. | - | | - When you create a StatefulSet and add a cloud storage volume, existing EVS volumes cannot be used. | - | | - EVS disks cannot be attached across AZs and cannot be used by multiple workloads, multiple pods of the same workload, or multiple jobs. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | Select a created disk. If no disk is available, follow the prompts to create one. | - | | | - | | For the same cluster and namespace, you can use an existing storage volume when creating a Deployment (with **Allocation Mode** set to **Manual**). | - | | | - | | When creating a StatefulSet, you can only use a volume automatically allocated by the system (only **Automatic** is available for **Allocation Mode**). | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Automatic | If you select **Automatic**, you need to configure the following items: | - | | | - | | a. **Access Mode**: permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteOnce** (RWO): A non-shared EVS volume is mounted as read-write to a pod by a single node. EVS volumes in RWO mode are supported since v1.13.10-r1. | - | | | - | | b. **Availability Zone**: AZ where the storage volume is located. Only the AZ where the worker node is located can be selected. | - | | c. **Sub-Type**: Select a storage subtype. | - | | | - | | - **Common I/O**: uses Serial Advanced Technology Attachment (SATA) drives to store data. | - | | - **High I/O**: uses serial attached SCSI (SAS) drives to store data. | - | | - **Ultra-high I/O**: uses solid state disk (SSD) drives to store data. | - | | | - | | d. **Storage Capacity**: Enter the storage capacity in the unit of GB. Ensure that the storage capacity quota is not exceeded; otherwise, creation will fail. | - | | e. After you select **KMS Encryption**, Key Management Service (KMS), an easy-to-use and highly secure service, will be enabled for EVS disks. This function is supported only for clusters of v1.13.10 and later in certain regions. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **EVSAccessKMS** indicates that EVS is granted the permission to access KMS. After EVS is authorized successfully, it can obtain KMS keys to encrypt and decrypt EVS systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | a. Click **Add Container Path**. | - | | b. **Container Path**: Enter the container path to which the volume is mounted. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | | - | | - Do not mount the volume to a system directory such as **/** or **/var/run**; this action may cause container errors. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | - If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | c. Set permissions. | - | | | - | | - **Read-only**: You can only read the data in the mounted volumes. | - | | - **Read/Write**: You can modify the data in the mounted volumes. Newly written data is not migrated if the container is migrated, which causes a data loss. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Importing an EVS Disk ---------------------- - -CCE allows you to import existing EVS disks. - -.. note:: - - An EVS disk can be imported into only one namespace. If an EVS disk has been imported into a namespace, it is invisible in other namespaces and cannot be imported again. **If you want to import an EVS disk that has file system (ext4) formatted, ensure that no partition has been created for the disk. Otherwise, data may be lost.** - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **EVS** tab page, click **Import**. -#. Select one or more EVS disks that you want to import. Then, click **OK**. - -Unbinding an EVS Disk ---------------------- - -After an EVS volume is successfully created or imported, the EVS volume is automatically bound to the current cluster and cannot be used by other clusters. When the volume is unbound from the cluster, other clusters can still use the volume. - -If the EVS volume has been mounted to a workload, it cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the EVS disk list, click **Unbind** next to the target EVS disk. -#. Confirm the unbinding, and click **OK**. - -Related Operations ------------------- - -After an EVS volume is created, you can perform operations described in :ref:`Table 3 `. - -.. _cce_01_0311__cce_01_0254_table1619535674020: - -.. table:: **Table 3** Other operations - - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==========================================================================================+ - | Deleting an EVS volume | #. Select the EVS volume to be deleted and click **Delete** in the **Operation** column. | - | | #. Follow the prompts to delete the EVS volume. | - +-----------------------------------+------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/index.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/index.rst index 0cdc976..e279a1a 100644 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/index.rst +++ b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/index.rst @@ -1,23 +1,21 @@ -:original_name: cce_01_0322.html +:original_name: cce_10_0322.html -.. _cce_01_0322: +.. _cce_10_0322: Using OBS Buckets as Storage Volumes ==================================== -- :ref:`Overview ` -- :ref:`Using OBS Volumes ` -- :ref:`(kubectl) Automatically Creating an OBS Volume ` -- :ref:`(kubectl) Creating a PV from an Existing OBS Bucket ` -- :ref:`(kubectl) Creating a Deployment Mounted with an OBS Volume ` -- :ref:`(kubectl) Creating a StatefulSet Mounted with an OBS Volume ` +- :ref:`Overview ` +- :ref:`(kubectl) Automatically Creating an OBS Volume ` +- :ref:`(kubectl) Creating a PV from an Existing OBS Bucket ` +- :ref:`(kubectl) Creating a Deployment Mounted with an OBS Volume ` +- :ref:`(kubectl) Creating a StatefulSet Mounted with an OBS Volume ` .. toctree:: :maxdepth: 1 :hidden: overview - using_obs_volumes kubectl_automatically_creating_an_obs_volume kubectl_creating_a_pv_from_an_existing_obs_bucket kubectl_creating_a_deployment_mounted_with_an_obs_volume diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_automatically_creating_an_obs_volume.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_automatically_creating_an_obs_volume.rst index 570f8a5..0fbff41 100644 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_automatically_creating_an_obs_volume.rst +++ b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_automatically_creating_an_obs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0325.html +:original_name: cce_10_0325.html -.. _cce_01_0325: +.. _cce_10_0325: (kubectl) Automatically Creating an OBS Volume ============================================== @@ -10,12 +10,6 @@ Scenario During the use of OBS, expected OBS buckets can be automatically created and mounted as volumes. Currently, standard and infrequent access OBS buckets are supported, which correspond to **obs-standard** and **obs-standard-ia**, respectively. -Prerequisites -------------- - -- You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. -- The AK/SK has been uploaded. For details, see :ref:`Preparations `. - Notes and Constraints --------------------- @@ -24,7 +18,7 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **pvc-obs-auto-example.yaml** file, which is used to create a PVC. @@ -64,7 +58,7 @@ Procedure | storage | Storage capacity in the unit of Gi. For OBS buckets, this field is used only for verification (cannot be empty or 0). Its value is fixed at **1**, and any value you set does not take effect for OBS buckets. | +-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -#. Run the following command to create the PVC. +#. Run the following command to create a PVC: **kubectl create -f pvc-obs-auto-example.yaml** diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_obs_volume.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_obs_volume.rst index 37659e0..10e71ef 100644 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_obs_volume.rst +++ b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_obs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0327.html +:original_name: cce_10_0327.html -.. _cce_01_0327: +.. _cce_10_0327: (kubectl) Creating a Deployment Mounted with an OBS Volume ========================================================== @@ -10,12 +10,6 @@ Scenario After an OBS volume is created or imported to CCE, you can mount the volume to a workload. -Prerequisites -------------- - -- You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. -- The AK/SK has been uploaded. For details, see :ref:`Preparations `. - Notes and Constraints --------------------- @@ -24,7 +18,7 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **obs-deployment-example.yaml** file, which is used to create a pod. @@ -160,7 +154,7 @@ Procedure +-------------+------------------------------------------------------------------------------------------------------------------------------------+ | mountPath | Mount path in the container. In this example, the volume is mounted to the **/tmp** directory. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ .. note:: diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst index 3609cbd..5efd49b 100644 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst +++ b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_obs_bucket.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0326.html +:original_name: cce_10_0326.html -.. _cce_01_0326: +.. _cce_10_0326: (kubectl) Creating a PV from an Existing OBS Bucket =================================================== @@ -10,12 +10,6 @@ Scenario CCE allows you to use an existing OBS bucket to create a PersistentVolume (PV). You can create a PersistentVolumeClaim (PVC) and bind it to the PV. -Prerequisites -------------- - -- You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. -- The AK/SK has been uploaded. For details, see :ref:`Preparations `. - Notes and Constraints --------------------- @@ -26,23 +20,23 @@ Procedure #. Log in to the OBS console, create an OBS bucket, and record the bucket name and storage class. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create two YAML files for creating the PV and PVC. Assume that the file names are **pv-obs-example.yaml** and **pvc-obs-example.yaml**. **touch pv-obs-example.yaml** **pvc-obs-example.yaml** +-----------------------------+------------------------------+-----------------------------------------------------+ - | Kubernetes Version | Description | YAML Example | + | Kubernetes Cluster Version | Description | YAML Example | +=============================+==============================+=====================================================+ - | 1.11 <= K8s version <= 1.13 | Clusters from v1.11 to v1.13 | :ref:`Example YAML ` | + | 1.11 <= K8s version <= 1.13 | Clusters from v1.11 to v1.13 | :ref:`Example YAML ` | +-----------------------------+------------------------------+-----------------------------------------------------+ - | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | + | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | +-----------------------------+------------------------------+-----------------------------------------------------+ **Clusters from v1.11 to v1.13** - - .. _cce_01_0326__li45671840132016: + - .. _cce_10_0326__li45671840132016: **Example YAML file for the PV:** @@ -84,7 +78,7 @@ Procedure +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage_class | Storage class, including **STANDARD** (standard bucket) and **STANDARD_IA** (infrequent access bucket). | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | region | For details about the value of **region**, see `Regions and Endpoints `__. | + | region | Region where the cluster is located. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | volumeID | OBS bucket name. | | | | @@ -139,7 +133,7 @@ Procedure **Clusters of v1.9** - - .. _cce_01_0326__li154036581589: + - .. _cce_10_0326__li154036581589: **Example YAML file for the PV:** @@ -176,7 +170,7 @@ Procedure +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | storage_class | Storage class, including **STANDARD** (standard bucket) and **STANDARD_IA** (infrequent access bucket). | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | region | For details about the value of **region**, see `Regions and Endpoints `__. | + | region | Region where the cluster is located. | +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | volumeID | OBS bucket name. | | | | diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_obs_volume.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_obs_volume.rst index 5f32d22..1fb3491 100644 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_obs_volume.rst +++ b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_obs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0328.html +:original_name: cce_10_0328.html -.. _cce_01_0328: +.. _cce_10_0328: (kubectl) Creating a StatefulSet Mounted with an OBS Volume =========================================================== @@ -10,12 +10,6 @@ Scenario CCE allows you to use an existing OBS volume to create a StatefulSet through a PersistentVolumeClaim (PVC). -Prerequisites -------------- - -- You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. -- The AK/SK has been uploaded. For details, see :ref:`Preparations `. - Notes and Constraints --------------------- @@ -24,9 +18,9 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Create an OBS volume by referring to :ref:`Creating an OBS Volume ` and obtain the PVC name. +#. Create an OBS volume by referring to :ref:`(kubectl) Automatically Creating an OBS Volume ` and obtain the PVC name. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create a YAML file for creating the workload. Assume that the file name is **obs-statefulset-example.yaml**. @@ -86,7 +80,7 @@ Procedure +-------------+------------------------------------------------------------------------------------------------------------------------------------+ | mountPath | Mount path in the container. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ | claimName | Name of an existing PVC. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/overview.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/overview.rst index 03014dc..c05a7c4 100644 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/overview.rst +++ b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0323.html +:original_name: cce_10_0323.html -.. _cce_01_0323: +.. _cce_10_0323: Overview ======== @@ -8,7 +8,7 @@ Overview CCE allows you to mount a volume created from an Object Storage Service (OBS) bucket to a container to store data persistently. Object storage is commonly used in cloud workloads, data analysis, content analysis, and hotspot objects. -.. figure:: /_static/images/en-us_image_0276664570.png +.. figure:: /_static/images/en-us_image_0000001249023453.png :alt: **Figure 1** Mounting OBS volumes to CCE **Figure 1** Mounting OBS volumes to CCE diff --git a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/using_obs_volumes.rst b/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/using_obs_volumes.rst deleted file mode 100644 index debd06d..0000000 --- a/umn/source/storage_flexvolume/using_obs_buckets_as_storage_volumes/using_obs_volumes.rst +++ /dev/null @@ -1,184 +0,0 @@ -:original_name: cce_01_0324.html - -.. _cce_01_0324: - -Using OBS Volumes -================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - -Notes and Constraints ---------------------- - -- CCE clusters of v1.7.3-r8 and earlier do not support OBS volumes. You need to upgrade these clusters or create clusters of a later version that supports OBS. -- Volumes cannot be created in specified enterprise projects. Only the default enterprise project is supported. -- The following operations apply to clusters of Kubernetes 1.13 or earlier. - -.. _cce_01_0324__section14271608324: - -Preparations ------------- - -To mount reliable and stable OBS buckets as volumes, you must create AK/SK before you create OBS buckets. - -The procedure for configuring the AK/SK is as follows: - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. - -#. On the **OBS** tab page, click **AK/SK** in the notice. - - - .. figure:: /_static/images/en-us_image_0000001190538605.png - :alt: **Figure 1** Configuring the AK/SK - - **Figure 1** Configuring the AK/SK - -#. Click |image1|, select a key file, and click **Upload** to upload the key file. - -#. Select the corresponding workload and click **Restart**. - -.. important:: - - When creating an OBS volume, you must use the AK/SK. If the key file is not uploaded, the pod will fail to be started or OBS data access will be abnormal due to the volume mounting failure. - -.. _cce_01_0324__section172788131291: - -Creating an OBS Volume ----------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. - -#. Click the **OBS** tab and click **Create OBS Bucket**. - -#. Configure basic information, as shown in :ref:`Table 1 `. - - .. _cce_01_0324__table20328123218464: - - .. table:: **Table 1** Parameters for creating an OBS volume - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+================================================================================================================================================================================================================================================================================+ - | \* PVC Name | Name of the new PVC, which is different from the volume name. The actual volume name is automatically generated when the PV is created by the PVC. | - | | | - | | The name contains 3 to 55 characters (excluding the prefix). It must contain lowercase letters, digits, and hyphens (-), and cannot start or end with a hyphen (-). | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Name | Cluster to which the OBS volume belongs. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace to which the volume belongs. The default value is **default**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Instance Type | Type of the storage instance created on OBS. | - | | | - | | - **Parallel file system**: supported when the cluster version is 1.15 or later and the everest add-on version is 1.0.2 or later. | - | | - **Object bucket**: A bucket is a container for storing objects in OBS. OBS provides flat storage in the form of buckets and objects. Unlike the conventional multi-layer directory structure of file systems, all objects in a bucket are stored at the same logical layer. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage Class | This parameter is displayed when you select **Object bucket** for **Instance Type**. | - | | | - | | This parameter indicates the storage classes supported by OBS. | - | | | - | | - **Standard**\ : applicable to scenarios where a large number of hotspot files or small-sized files need to be accessed frequently (multiple times per month on average) and require fast access response. | - | | - **Infrequent access**: applicable to scenarios where data is not frequently accessed (less than 12 times per year on average) but requires fast access response. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Storage Policy | Object storage has the following policies: | - | | | - | | **Private**: Only the bucket owner has full control over the bucket. Unauthorized users do not have permissions to access the bucket. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Access Mode | Access permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteMany** (RWX): The volume is mounted as read-write by multiple nodes. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **Create**. - - After the OBS volume is successfully created, it is displayed in the OBS volume list. Click the PVC name to view detailed information about the OBS volume. - -Adding an OBS Volume --------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, :ref:`Creating a DaemonSet `, or :ref:`Creating a Job `. After you have added a container, choose **Data Storage** > **Cloud Volume**, and then click **Add Cloud Volume**. -#. Set **Type** to **OBS**. - - .. table:: **Table 2** OBS volume parameters - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+========================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | Select **OBS**. | - | | | - | | **OBS**: Standard and Infrequent Access OBS buckets are supported. OBS buckets are commonly used for big data analytics, cloud native applications, static website hosting, and backup/active archiving. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | **Name**: Select a created OBS volume. | - | | | - | | **Sub-Type**: class of the selected volume. The value can be **Standard** or **Infrequent access**, and you do not need to set this parameter. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Automatic | Type of the storage instance created on OBS. | - | | | - | | - **Parallel file system**: supported when the cluster version is 1.15 or later and the everest add-on version is 1.0.2 or later. | - | | | - | | - **Object bucket**: A bucket is a container for storing objects in OBS. | - | | | - | | **Sub-Type**: Select **Standard** or **Infrequent access**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | Configure the following parameters: | - | | | - | | a. **Container Path**: Enter the mount path in the container, for example, **/tmp**. | - | | | - | | The mount path must not be a system directory, such as **/** and **/var/run**. Otherwise, an exception occurs. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | b. Set permissions. | - | | | - | | - **Read-only**: You can only read the data in the mounted volumes. | - | | - **Read/Write**: You can modify the data in the mounted volumes. Newly written data is not migrated if the container is migrated, which causes a data loss. | - | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Importing an OBS Volume ------------------------ - -CCE allows you to import existing OBS volumes. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **OBS** tab page, click **Import**. -#. Select one or more OBS volumes that you want to import. -#. Select the target cluster and namespace. -#. Click **OK**. - -Unbinding an OBS Volume ------------------------ - -When an OBS volume is successfully created, the OBS volume is automatically bound to the current cluster. Other clusters can also use the OBS volume. When the volume is unbound from the cluster, other clusters can still use the volume. - -If the volume has been mounted to a workload, the volume cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the OBS volume list, click **Unbind** next to the target OBS volume. -#. In the dialog box displayed, click **Yes**. - -Related Operations ------------------- - -After an OBS volume is created, you can perform the operation described in :ref:`Table 3 `. - -.. _cce_01_0324__table1619535674020: - -.. table:: **Table 3** Other operations - - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==========================================================================================+ - | Deleting an OBS volume | #. Select the OBS volume to be deleted and click **Delete** in the **Operation** column. | - | | #. Follow the prompts to delete the volume. | - +-----------------------------------+------------------------------------------------------------------------------------------+ - -.. |image1| image:: /_static/images/en-us_image_0000001088110417.png diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/index.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/index.rst index 01cab6c..d0cc819 100644 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/index.rst +++ b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/index.rst @@ -1,23 +1,21 @@ -:original_name: cce_01_0315.html +:original_name: cce_10_0315.html -.. _cce_01_0315: +.. _cce_10_0315: Using SFS File Systems as Storage Volumes ========================================= -- :ref:`Overview ` -- :ref:`Using SFS Volumes ` -- :ref:`(kubectl) Automatically Creating an SFS Volume ` -- :ref:`(kubectl) Creating a PV from an Existing SFS File System ` -- :ref:`(kubectl) Creating a Deployment Mounted with an SFS Volume ` -- :ref:`(kubectl) Creating a StatefulSet Mounted with an SFS Volume ` +- :ref:`Overview ` +- :ref:`(kubectl) Automatically Creating an SFS Volume ` +- :ref:`(kubectl) Creating a PV from an Existing SFS File System ` +- :ref:`(kubectl) Creating a Deployment Mounted with an SFS Volume ` +- :ref:`(kubectl) Creating a StatefulSet Mounted with an SFS Volume ` .. toctree:: :maxdepth: 1 :hidden: overview - using_sfs_volumes kubectl_automatically_creating_an_sfs_volume kubectl_creating_a_pv_from_an_existing_sfs_file_system kubectl_creating_a_deployment_mounted_with_an_sfs_volume diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_automatically_creating_an_sfs_volume.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_automatically_creating_an_sfs_volume.rst index b0567c9..cb320c6 100644 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_automatically_creating_an_sfs_volume.rst +++ b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_automatically_creating_an_sfs_volume.rst @@ -1,20 +1,10 @@ -:original_name: cce_01_0318.html +:original_name: cce_10_0318.html -.. _cce_01_0318: +.. _cce_10_0318: (kubectl) Automatically Creating an SFS Volume ============================================== -Scenario --------- - -CCE supports creating SFS volumes through PersistentVolumeClaims (PVCs). - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -23,7 +13,7 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **pvc-sfs-auto-example.yaml** file, which is used to create a PVC. @@ -63,7 +53,7 @@ Procedure | storage | Storage capacity in the unit of Gi. | +-----------------------------------------+---------------------------------------------------------------------------------------+ -#. Run the following command to create the PVC. +#. Run the following command to create a PVC: **kubectl create -f pvc-sfs-auto-example.yaml** diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_volume.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_volume.rst index 800c9ba..3470ba8 100644 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_volume.rst +++ b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0320.html +:original_name: cce_10_0320.html -.. _cce_01_0320: +.. _cce_10_0320: (kubectl) Creating a Deployment Mounted with an SFS Volume ========================================================== @@ -10,11 +10,6 @@ Scenario After an SFS volume is created or imported to CCE, you can mount the volume to a workload. -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -23,7 +18,7 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **sfs-deployment-example.yaml** file, which is used to create a pod. @@ -136,7 +131,7 @@ Procedure +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ | spec.template.spec.containers.volumeMount | mountPath | Mount path in the container. In this example, the mount path is **/tmp**. | +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +-------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ .. note:: diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst index d5434b1..bed800b 100644 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst +++ b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_file_system.rst @@ -1,20 +1,10 @@ -:original_name: cce_01_0319.html +:original_name: cce_10_0319.html -.. _cce_01_0319: +.. _cce_10_0319: (kubectl) Creating a PV from an Existing SFS File System ======================================================== -Scenario --------- - -CCE allows you to use an existing file system to create a PersistentVolume (PV). After the creation is successful, create the corresponding PersistentVolumeClaim (PVC) and bind it to the PV. - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -25,23 +15,23 @@ Procedure #. Log in to the SFS console, create a file system, and record the file system ID, shared path, and capacity. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create two YAML files for creating the PV and PVC. Assume that the file names are **pv-sfs-example.yaml** and **pvc-sfs-example.yaml**. **touch pv-sfs-example.yaml** **pvc-sfs-example.yaml** - +-----------------------------+------------------------------+-----------------------------------------------------+ - | Kubernetes Version | Description | YAML Example | - +=============================+==============================+=====================================================+ - | 1.11 <= K8s version <= 1.13 | Clusters from v1.11 to v1.13 | :ref:`Example YAML ` | - +-----------------------------+------------------------------+-----------------------------------------------------+ - | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | - +-----------------------------+------------------------------+-----------------------------------------------------+ + +----------------------------+------------------------------+-----------------------------------------------------+ + | Kubernetes Cluster Version | Description | YAML Example | + +============================+==============================+=====================================================+ + | 1.11 <= K8s version < 1.13 | Clusters from v1.11 to v1.13 | :ref:`Example YAML ` | + +----------------------------+------------------------------+-----------------------------------------------------+ + | K8s version = 1.9 | Clusters of v1.9 | :ref:`Example YAML ` | + +----------------------------+------------------------------+-----------------------------------------------------+ **Clusters from v1.11 to v1.13** - - .. _cce_01_0319__li1252510101515: + - .. _cce_10_0319__li1252510101515: **Example YAML file for the PV:** @@ -137,7 +127,7 @@ Procedure **Clusters of v1.9** - - .. _cce_01_0319__li10858156164514: + - .. _cce_10_0319__li10858156164514: **Example YAML file for the PV:** diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_volume.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_volume.rst index b2700a4..2e25ca0 100644 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_volume.rst +++ b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0321.html +:original_name: cce_10_0321.html -.. _cce_01_0321: +.. _cce_10_0321: (kubectl) Creating a StatefulSet Mounted with an SFS Volume =========================================================== @@ -10,11 +10,6 @@ Scenario CCE allows you to use an existing SFS volume to create a StatefulSet through a PersistentVolumeClaim (PVC). -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -23,9 +18,9 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Create an SFS volume by referring to :ref:`Creating an SFS Volume ` and record the volume name. +#. Create an SFS volume by referring to :ref:`(kubectl) Automatically Creating an SFS Volume ` and record the volume name. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create a YAML file for creating the workload. Assume that the file name is **sfs-statefulset-example**.\ **yaml**. @@ -83,7 +78,7 @@ Procedure +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ | spec.template.spec.containers.volumeMounts | mountPath | Mount path in the container. | +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | spec | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ | spec.template.spec.volumes.persistentVolumeClaim | claimName | Name of an existing PVC. | +--------------------------------------------------+-------------+------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/overview.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/overview.rst index ec847a0..6567c03 100644 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/overview.rst +++ b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0316.html +:original_name: cce_10_0316.html -.. _cce_01_0316: +.. _cce_10_0316: Overview ======== @@ -8,7 +8,7 @@ Overview CCE allows you to mount a volume created from a Scalable File Service (SFS) file system to a container to store data persistently. SFS volumes are commonly used in ReadWriteMany scenarios, such as media processing, content management, big data analysis, and workload process analysis. -.. figure:: /_static/images/en-us_image_0276664213.png +.. figure:: /_static/images/en-us_image_0000001201823500.png :alt: **Figure 1** Mounting SFS volumes to CCE **Figure 1** Mounting SFS volumes to CCE diff --git a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/using_sfs_volumes.rst b/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/using_sfs_volumes.rst deleted file mode 100644 index 1e964cf..0000000 --- a/umn/source/storage_flexvolume/using_sfs_file_systems_as_storage_volumes/using_sfs_volumes.rst +++ /dev/null @@ -1,161 +0,0 @@ -:original_name: cce_01_0317.html - -.. _cce_01_0317: - -Using SFS Volumes -================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - -Notes and Constraints ---------------------- - -- SFS volumes are available only in certain regions. -- Container storage in CCE clusters of Kubernetes 1.13 or later version supports encryption. Currently, E2E encryption is supported only in certain regions. -- Volumes cannot be created in specified enterprise projects. Only the default enterprise project is supported. - -.. _cce_01_0317__cce_01_0259_section1191025105819: - -Creating an SFS Volume ----------------------- - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. - -#. On the **SFS** tab, click **Create SFS File System**. - -#. Configure basic information, as shown in :ref:`Table 1 `. - - .. _cce_01_0317__cce_01_0259_table20328123218464: - - .. table:: **Table 1** Parameters for Creating a File System Volume - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Parameter Description | - +===================================+======================================================================================================================================================================================================================================================================================================================================================================================+ - | \* PVC Name | Name of the new PVC, which is different from the volume name. The actual volume name is automatically generated when the PV is created by the PVC. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Cluster Name | Cluster to which the file system volume belongs. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Namespace | Namespace with which the snapshot is associated. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Total Capacity | The total capacity is the capacity of a single volume. Fees are charged by actual usage. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Access Mode | Access permissions of user applications on storage resources (PVs). | - | | | - | | - **ReadWriteMany** (RWX): The SFS volume can be mounted as read-write by multiple nodes. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Encryption | **KMS Encryption** is deselected by default. | - | | | - | | After **KMS Encryption** is selected, Key Management Service (KMS), an easy-to-use and highly secure key service, will be used for SFS file systems. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **SFSAccessKMS** indicates that SFS is granted the permission to access KMS. After SFS is authorized successfully, it can obtain KMS keys to encrypt and decrypt file systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - | | | - | | This function is supported only for clusters of v1.13.10 and later in certain regions. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **Create**. - - The volume is displayed in the list. When **PVS Status** becomes **Bound**, the volume is created successfully. - -#. Click the volume name to view detailed information about the volume. - -Adding an SFS Volume --------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, :ref:`Creating a DaemonSet `, or :ref:`Creating a Job `. During creation, expand **Data Storage** after adding a container. On the **Cloud Volume** tab page, click **Add Cloud Volume**. -#. Set the storage class to **SFS**. - - .. table:: **Table 2** Parameters for mounting a file system - - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Parameter Description | - +===================================+============================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | **File Storage (NFS)**: This type applies to a wide range of scenarios, including media processing, content management, big data, and application analysis. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | - **Name**: Select a created file system. You need to create a file system in advance. For details about how to create a file system, see :ref:`Creating an SFS Volume `. | - | | - **Sub-Type**: subtype of the created file storage. | - | | - **Storage Capacity**: This field is one of the PVC attributes. If the storage capacity has been expanded on the IaaS side, it is normal that the capacity values are inconsistent. The PVC capacity is the same as the storage entity capacity only after end-to-end container storage capacity expansion is supported for CCE clusters of v1.13. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Automatic | An SFS volume is created automatically. You need to enter the storage capacity. | - | | | - | | - **Sub-Type**: Select **NFS**. | - | | - **Storage Capacity**: Specify the total storage capacity, in GB. Ensure that the storage capacity quota is not exceeded; otherwise, creation will fail. | - | | - After you select **KMS Encryption**, Key Management Service (KMS), an easy-to-use and highly secure service, will be enabled for file systems. This function is supported only for clusters of v1.13.10 and later in certain regions. If no agency has been created, click **Create Agency** and set the following parameters: | - | | | - | | - **Agency Name**: Agencies can be used to assign permissions to trusted accounts or cloud services for a specific period of time. If no agency is created, click **Create Agency**. The agency name **SFSAccessKMS** indicates that SFS is granted the permission to access KMS. After SFS is authorized successfully, it can obtain KMS keys to encrypt and decrypt file systems. | - | | - **Key Name**: After a key is created, it can be loaded and used in containerized applications. | - | | - **Key ID**: generated by default. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | Configure the following parameters: | - | | | - | | a. **subPath**: Enter the subpath of the file storage, for example, **/tmp**. | - | | | - | | If this parameter is not specified, the root path of the data volume is used by default. Currently, only file storage is supported. The value must be a relative path and cannot start with a slash (/) or ../. | - | | | - | | b. **Container Path**: Enter the path of the container, for example, **/tmp**. | - | | | - | | The container path must not be a system directory, such as **/** and **/var/run**. Otherwise, an exception occurs. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | c. Set permissions. | - | | | - | | - **Read-only**: You can only read the data volumes mounted to the path. | - | | - **Read/Write**: You can modify the data volumes mounted to the path. Newly written data is not migrated if the container is migrated, which may cause a data loss. | - | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Importing an SFS Volume ------------------------ - -CCE allows you to import existing SFS volumes. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **SFS** tab page, click **Import**. -#. Select one or more SFS volumes that you want to attach. -#. Select the target cluster and namespace. Then, click **OK**. - -Unbinding an SFS Volume ------------------------ - -When an SFS volume is successfully created or imported, the volume is automatically bound to the current cluster. Other clusters can also use the volume. When the SFS volume is unbound from the cluster, other clusters can still import and use the volume. - -If the SFS volume has been attached to a workload, the volume cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the SFS volume list, click **Unbind** next to the target volume. -#. Confirm the unbinding, and click **OK**. - -Related Operations ------------------- - -After an SFS volume is created, you can perform the operation described in :ref:`Table 3 `. - -.. _cce_01_0317__cce_01_0259_table1619535674020: - -.. table:: **Table 3** Other operations - - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==========================================================================================+ - | Deleting an SFS volume | #. Select the SFS volume to be deleted and click **Delete** in the **Operation** column. | - | | #. Follow the prompts to delete the EVS disk. | - +-----------------------------------+------------------------------------------------------------------------------------------+ - | Importing an SFS volume | CCE allows you to import existing SFS volumes. | - | | | - | | #. On the **SFS** tab page, click **Import**. | - | | #. Select one or more SFS volumes that you want to attach. | - | | #. Select the target cluster and namespace. | - | | #. Click **Yes**. | - +-----------------------------------+------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/index.rst b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/index.rst index 0c02d12..ac171ac 100644 --- a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/index.rst +++ b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/index.rst @@ -1,22 +1,20 @@ -:original_name: cce_01_0329.html +:original_name: cce_10_0329.html -.. _cce_01_0329: +.. _cce_10_0329: Using SFS Turbo File Systems as Storage Volumes =============================================== -- :ref:`Overview ` -- :ref:`Using SFS Turbo Volumes ` -- :ref:`(kubectl) Creating a PV from an Existing SFS Turbo File System ` -- :ref:`(kubectl) Creating a Deployment Mounted with an SFS Turbo Volume ` -- :ref:`(kubectl) Creating a StatefulSet Mounted with an SFS Turbo Volume ` +- :ref:`Overview ` +- :ref:`(kubectl) Creating a PV from an Existing SFS Turbo File System ` +- :ref:`(kubectl) Creating a Deployment Mounted with an SFS Turbo Volume ` +- :ref:`(kubectl) Creating a StatefulSet Mounted with an SFS Turbo Volume ` .. toctree:: :maxdepth: 1 :hidden: overview - using_sfs_turbo_volumes kubectl_creating_a_pv_from_an_existing_sfs_turbo_file_system kubectl_creating_a_deployment_mounted_with_an_sfs_turbo_volume kubectl_creating_a_statefulset_mounted_with_an_sfs_turbo_volume diff --git a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_turbo_volume.rst b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_turbo_volume.rst index 0d482ae..e9355c9 100644 --- a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_turbo_volume.rst +++ b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_deployment_mounted_with_an_sfs_turbo_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0333.html +:original_name: cce_10_0333.html -.. _cce_01_0333: +.. _cce_10_0333: (kubectl) Creating a Deployment Mounted with an SFS Turbo Volume ================================================================ @@ -10,11 +10,6 @@ Scenario After an SFS Turbo volume is created or imported to CCE, you can mount the volume to a workload. -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -23,7 +18,7 @@ The following configuration example applies to clusters of Kubernetes 1.13 or ea Procedure --------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Run the following commands to configure the **efs-deployment-example.yaml** file, which is used to create a Deployment: diff --git a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_turbo_file_system.rst b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_turbo_file_system.rst index 82d4caf..7892ff7 100644 --- a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_turbo_file_system.rst +++ b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_pv_from_an_existing_sfs_turbo_file_system.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0332.html +:original_name: cce_10_0332.html -.. _cce_01_0332: +.. _cce_10_0332: (kubectl) Creating a PV from an Existing SFS Turbo File System ============================================================== @@ -10,11 +10,6 @@ Scenario CCE allows you to use an existing SFS Turbo file system to create a PersistentVolume (PV). After the creation is successful, you can create a PersistentVolumeClaim (PVC) and bind it to the PV. -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -25,7 +20,7 @@ Procedure #. Log in to the SFS console, create a file system, and record the file system ID, shared path, and capacity. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create two YAML files for creating the PV and PVC. Assume that the file names are **pv-efs-example.yaml** and **pvc-efs-example.yaml**. diff --git a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_turbo_volume.rst b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_turbo_volume.rst index 7462458..7d3a0fc 100644 --- a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_turbo_volume.rst +++ b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/kubectl_creating_a_statefulset_mounted_with_an_sfs_turbo_volume.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0334.html +:original_name: cce_10_0334.html -.. _cce_01_0334: +.. _cce_10_0334: (kubectl) Creating a StatefulSet Mounted with an SFS Turbo Volume ================================================================= @@ -10,11 +10,6 @@ Scenario CCE allows you to use an existing SFS Turbo volume to create a StatefulSet. -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - Notes and Constraints --------------------- @@ -25,7 +20,7 @@ Procedure #. Create an SFS Turbo volume and record the volume name. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create a YAML file for creating the workload. Assume that the file name is **efs-statefulset-example.yaml**. @@ -108,7 +103,7 @@ Procedure +-------------+------------------------------------------------------------------------------------------------------------------------------------+ | mountPath | Mount path in the container. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ - | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | + | serviceName | Service corresponding to the workload. For details about how to create a Service, see :ref:`Creating a StatefulSet `. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ | claimName | Name of an existing PVC. | +-------------+------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/overview.rst b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/overview.rst index 45f6032..68c379d 100644 --- a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/overview.rst +++ b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0330.html +:original_name: cce_10_0330.html -.. _cce_01_0330: +.. _cce_10_0330: Overview ======== @@ -8,7 +8,7 @@ Overview CCE allows you to mount a volume created from an SFS Turbo file system to a container to store data persistently. Provisioned on demand and fast, SFS Turbo is suitable for DevOps, container microservices, and enterprise OA scenarios. -.. figure:: /_static/images/en-us_image_0276664792.png +.. figure:: /_static/images/en-us_image_0000001202103502.png :alt: **Figure 1** Mounting SFS Turbo volumes to CCE **Figure 1** Mounting SFS Turbo volumes to CCE diff --git a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/using_sfs_turbo_volumes.rst b/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/using_sfs_turbo_volumes.rst deleted file mode 100644 index 5c902c9..0000000 --- a/umn/source/storage_flexvolume/using_sfs_turbo_file_systems_as_storage_volumes/using_sfs_turbo_volumes.rst +++ /dev/null @@ -1,82 +0,0 @@ -:original_name: cce_01_0331.html - -.. _cce_01_0331: - -Using SFS Turbo Volumes -======================= - -Prerequisites -------------- - -You have created a CCE cluster and installed the FlexVolume plug-in (:ref:`storage-driver `) in the cluster. - -Notes and Constraints ---------------------- - -- SFS Turbo volumes are available only in certain regions. -- Currently, SFS Turbo file systems cannot be directly created on CCE. -- The following operations apply to clusters of Kubernetes 1.13 or earlier. - -.. _cce_01_0331__section57261325121712: - -Importing an SFS Turbo Volume ------------------------------ - -CCE allows you to import existing SFS Turbo volumes. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. On the **SFS Turbo** tab page, click **Import**. -#. Select one or more SFS Turbo volumes that you want to import. -#. Select the cluster and namespace to which you want to import the volumes. -#. Click **OK**. The volumes are displayed in the list. When **PVS Status** becomes **Bound**, the volumes are imported successfully. - -Adding an SFS Turbo Volume --------------------------- - -#. Create a workload or job by referring to :ref:`Creating a Deployment `, :ref:`Creating a StatefulSet `, :ref:`Creating a DaemonSet `, or :ref:`Creating a Job `. After you have added a container, choose **Data Storage** > **Cloud Volume**, and then click **Add Cloud Volume**. -#. Set the storage volume type to **SFS Turbo**. - - .. table:: **Table 1** Parameters for configuring an SFS Turbo volume - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+========================================================================================================================================================================================================================================================================================================================================================================================================+ - | **Type** | **SFS Turbo**: applicable to DevOps, containerized microservices, and enterprise OA applications. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | **Allocation Mode** | | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Manual | Select an existing SFS Turbo volume. You need to import SFS Turbo volumes in advance. For details, see :ref:`Importing an SFS Turbo Volume `. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Add Container Path | Configure the following parameters: | - | | | - | | a. **subPath**: Enter the subpath of the file storage, for example, **/tmp**. | - | | | - | | This parameter specifies a subpath inside the referenced volume instead of its root. If this parameter is not specified, the root path is used. Currently, only file storage is supported. The value must be a relative path and cannot start with a slash (/) or ../. | - | | | - | | b. **Container Path**: Enter the mount path in the container, for example, **/tmp**. | - | | | - | | The mount path must not be a system directory, such as **/** and **/var/run**. Otherwise, an exception occurs. You are advised to mount the volume to an empty directory. If the directory is not empty, ensure that there are no files affecting container startup in the directory. Otherwise, such files will be replaced, resulting in failures to start the container and create the workload. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | If the volume is mounted to a high-risk directory, you are advised to use an account with minimum permissions to start the container; otherwise, high-risk files on the host machine may be damaged. | - | | | - | | c. Set permissions. | - | | | - | | - **Read-only**: You can only read the data in the mounted volumes. | - | | - **Read/Write**: You can modify the data in the mounted volumes. Newly written data is not migrated if the container is migrated, which causes a data loss. | - | | | - | | Click **Add Container Path** to add multiple settings. Then, click **OK**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **OK**. - -Unbinding an SFS Turbo Volume ------------------------------ - -When an SFS Turbo volume is successfully imported to a cluster, the volume is bound to the cluster. The volume can also be imported to other clusters. When the volume is unbound from the cluster, other clusters can still import and use the volume. - -If the SFS Turbo volume has been mounted to a workload, the volume cannot be unbound from the cluster. - -#. Log in to the CCE console. In the navigation pane, choose **Resource Management** > **Storage**. In the SFS Turbo volume list, click **Unbind** next to the target volume. -#. In the dialog box displayed, click **OK**. diff --git a/umn/source/workloads/configuring_a_container/configuring_an_image_pull_policy.rst b/umn/source/workloads/configuring_a_container/configuring_an_image_pull_policy.rst new file mode 100644 index 0000000..0eb5389 --- /dev/null +++ b/umn/source/workloads/configuring_a_container/configuring_an_image_pull_policy.rst @@ -0,0 +1,39 @@ +:original_name: cce_10_0353.html + +.. _cce_10_0353: + +Configuring an Image Pull Policy +================================ + +When a workload is created, the container image is pulled from the image repository to the node. The image is also pulled when the workload is restarted or upgraded. + +By default, **imagePullPolicy** is set to **IfNotPresent**, indicating that if the image exists on the node, the existing image is used. If the image does not exist on the node, the image is pulled from the image repository. + +The image pull policy can also be set to **Always**, indicating that the image is pulled from the image repository and overwrites the image on the node regardless of whether the image exists on the node. + +.. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + name: nginx + spec: + containers: + - image: nginx:alpine + name: container-0 + resources: + limits: + cpu: 100m + memory: 200Mi + requests: + cpu: 100m + memory: 200Mi + imagePullPolicy: Always + imagePullSecrets: + - name: default-secret + +You can also set the image pull policy when creating a workload on the CCE console. As shown in the following figure, if you select **Always**, the image is always pulled. If you do not select it, the policy will be **IfNotPresent**, which means that the image is not pulled. + +.. important:: + + You are advised to use a new tag each time you create an image. If you do not update the tag but only update the image, when **Pull Policy** is set to **IfNotPresent**, CCE considers that an image with the tag already exists on the current node and will not pull the image again. diff --git a/umn/source/workloads/configuring_a_container/configuring_the_workload_upgrade_policy.rst b/umn/source/workloads/configuring_a_container/configuring_the_workload_upgrade_policy.rst new file mode 100644 index 0000000..9894d82 --- /dev/null +++ b/umn/source/workloads/configuring_a_container/configuring_the_workload_upgrade_policy.rst @@ -0,0 +1,92 @@ +:original_name: cce_10_0397.html + +.. _cce_10_0397: + +Configuring the Workload Upgrade Policy +======================================= + +In actual applications, upgrade is a common operation. A Deployment, StatefulSet, or DaemonSet can easily support application upgrade. + +You can set different upgrade policies: + +- **Rolling upgrade**: New pods are created gradually and then old pods are deleted. This is the default policy. +- **Replace upgrade**: The current pods are deleted and then new pods are created. + +Upgrade Parameters +------------------ + +- **Max. Surge** (maxSurge) + + Specifies the maximum number of pods that can exist over **spec.replicas**. The default value is 25%. For example, if **spec.replicas** is set to **4**, no more than 5 pods can exist during the upgrade process, that is, the upgrade step is 1. The absolute number is calculated from the percentage by rounding up. The value can also be set to an absolute number. + + This parameter is supported only by Deployments. + +- **Max. Unavailable Pods** (maxUnavailable) + + Specifies the maximum number of pods that can be unavailable during the update process. The default value is 25%. For example, if **spec.replicas** is set to **4**, at least 3 pods exist during the upgrade process, that is, the deletion step is 1. The value can also be set to an absolute number. + + This parameter is supported only by Deployments. + +- **Min. Ready Seconds** (minReadySeconds) + + A pod is considered available only when the minimum readiness time is exceeded without any of its containers crashing. The default value is **0** (the pod is considered available immediately after it is ready). + +- **Revision History Limit** (revisionHistoryLimit) + + Specifies the number of old ReplicaSets to retain to allow rollback. These old ReplicaSets consume resources in etcd and crowd the output of **kubectl get rs**. The configuration of each Deployment revision is stored in its ReplicaSets. Therefore, once the old ReplicaSet is deleted, you lose the ability to roll back to that revision of Deployment. By default, 10 old ReplicaSets will be kept, but the ideal value depends on the frequency and stability of the new Deployments. + +- **Max. Upgrade Duration** (progressDeadlineSeconds) + + Specifies the number of seconds that the system waits for a Deployment to make progress before reporting a Deployment progress failure. It is surfaced as a condition with Type=Progressing, Status=False, and Reason=ProgressDeadlineExceeded in the status of the resource. The Deployment controller will keep retrying the Deployment. In the future, once automatic rollback will be implemented, the Deployment controller will roll back a Deployment as soon as it observes such a condition. + + If this parameter is specified, the value of this parameter must be greater than that of **.spec.minReadySeconds**. + +- **Scale-In Time Window** (terminationGracePeriodSeconds) + + Graceful deletion time. The default value is 30 seconds. When a pod is deleted, a SIGTERM signal is sent and the system waits for the applications in the container to terminate. If the application is not terminated within the time specified by **terminationGracePeriodSeconds**, a SIGKILL signal is sent to forcibly terminate the pod. + +Upgrade Example +--------------- + +The Deployment can be upgraded in a declarative mode. That is, you only need to modify the YAML definition of the Deployment. For example, you can run the **kubectl edit** command to change the Deployment image to **nginx:alpine**. After the modification, query the ReplicaSet and pod. The query result shows that a new ReplicaSet is created and the pod is re-created. + +.. code-block:: + + $ kubectl edit deploy nginx + + $ kubectl get rs + NAME DESIRED CURRENT READY AGE + nginx-6f9f58dffd 2 2 2 1m + nginx-7f98958cdf 0 0 0 48m + + $ kubectl get pods + NAME READY STATUS RESTARTS AGE + nginx-6f9f58dffd-tdmqk 1/1 Running 0 1m + nginx-6f9f58dffd-tesqr 1/1 Running 0 1m + +The Deployment can use the **maxSurge** and **maxUnavailable** parameters to control the proportion of pods to be re-created during the upgrade, which is useful in many scenarios. The configuration is as follows: + +.. code-block:: + + spec: + strategy: + rollingUpdate: + maxSurge: 1 + maxUnavailable: 0 + type: RollingUpdate + +In the preceding example, the value of **spec.replicas** is **2**. If both **maxSurge** and **maxUnavailable** are the default value 25%, **maxSurge** allows a maximum of three pods to exist (2 x 1.25 = 2.5, rounded up to 3), and **maxUnavailable** does not allow a maximum of two pods to be unavailable (2 x 0.75 = 1.5, rounded up to 2). That is, during the upgrade process, there will always be two pods running. Each time a new pod is created, an old pod is deleted, until all pods are new. + +Rollback +-------- + +Rollback is to roll an application back to the earlier version when a fault occurs during the upgrade. A Deployment can be easily rolled back to the earlier version. + +For example, if the upgraded image is faulty, you can run the **kubectl rollout undo** command to roll back the Deployment. + +.. code-block:: + + $ kubectl rollout undo deployment nginx + deployment.apps/nginx rolled back + +A Deployment can be easily rolled back because it uses a ReplicaSet to control a pod. After the upgrade, the previous ReplicaSet still exists. The Deployment is rolled back by using the previous ReplicaSet to re-create the pod. The number of ReplicaSets stored in a Deployment can be restricted by the **revisionHistoryLimit** parameter. The default value is **10**. diff --git a/umn/source/workloads/configuring_a_container/configuring_time_zone_synchronization.rst b/umn/source/workloads/configuring_a_container/configuring_time_zone_synchronization.rst new file mode 100644 index 0000000..44767e3 --- /dev/null +++ b/umn/source/workloads/configuring_a_container/configuring_time_zone_synchronization.rst @@ -0,0 +1,43 @@ +:original_name: cce_10_0354.html + +.. _cce_10_0354: + +Configuring Time Zone Synchronization +===================================== + +When creating a workload, you can configure containers to use the same time zone as the node. You can enable time zone synchronization when creating a workload. + +The time zone synchronization function depends on the local disk (hostPath) mounted to the container. After time zone synchronization is enabled, **/etc/localtime** of the node is mounted to **/etc/localtime** of the container in HostPath mode, in this way, the node and container use the same time zone configuration file. + +.. code-block:: + + kind: Deployment + apiVersion: apps/v1 + metadata: + name: test + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: test + template: + metadata: + labels: + app: test + spec: + volumes: + - name: vol-162979628557461404 + hostPath: + path: /etc/localtime + type: '' + containers: + - name: container-0 + image: 'nginx:alpine' + volumeMounts: + - name: vol-162979628557461404 + readOnly: true + mountPath: /etc/localtime + imagePullPolicy: IfNotPresent + imagePullSecrets: + - name: default-secret diff --git a/umn/source/workloads/configuring_a_container/index.rst b/umn/source/workloads/configuring_a_container/index.rst index 34ebd91..90ffafd 100644 --- a/umn/source/workloads/configuring_a_container/index.rst +++ b/umn/source/workloads/configuring_a_container/index.rst @@ -1,24 +1,32 @@ -:original_name: cce_01_0130.html +:original_name: cce_10_0130.html -.. _cce_01_0130: +.. _cce_10_0130: Configuring a Container ======================= -- :ref:`Using a Third-Party Image ` -- :ref:`Setting Container Specifications ` -- :ref:`Setting Container Lifecycle Parameters ` -- :ref:`Setting Container Startup Commands ` -- :ref:`Setting Health Check for a Container ` -- :ref:`Setting an Environment Variable ` +- :ref:`Setting Basic Container Information ` +- :ref:`Using a Third-Party Image ` +- :ref:`Setting Container Specifications ` +- :ref:`Setting Container Lifecycle Parameters ` +- :ref:`Setting Health Check for a Container ` +- :ref:`Setting an Environment Variable ` +- :ref:`Configuring an Image Pull Policy ` +- :ref:`Configuring Time Zone Synchronization ` +- :ref:`Configuring the Workload Upgrade Policy ` +- :ref:`Scheduling Policy (Affinity/Anti-affinity) ` .. toctree:: :maxdepth: 1 :hidden: + setting_basic_container_information using_a_third-party_image setting_container_specifications setting_container_lifecycle_parameters - setting_container_startup_commands setting_health_check_for_a_container setting_an_environment_variable + configuring_an_image_pull_policy + configuring_time_zone_synchronization + configuring_the_workload_upgrade_policy + scheduling_policy_affinity_anti-affinity diff --git a/umn/source/workloads/configuring_a_container/scheduling_policy_affinity_anti-affinity.rst b/umn/source/workloads/configuring_a_container/scheduling_policy_affinity_anti-affinity.rst new file mode 100644 index 0000000..8952f33 --- /dev/null +++ b/umn/source/workloads/configuring_a_container/scheduling_policy_affinity_anti-affinity.rst @@ -0,0 +1,487 @@ +:original_name: cce_10_0232.html + +.. _cce_10_0232: + +Scheduling Policy (Affinity/Anti-affinity) +========================================== + +A nodeSelector provides a very simple way to constrain pods to nodes with particular labels, as mentioned in :ref:`Creating a DaemonSet `. The affinity and anti-affinity feature greatly expands the types of constraints you can express. + +Kubernetes supports node-level and pod-level affinity and anti-affinity. You can configure custom rules to achieve affinity and anti-affinity scheduling. For example, you can deploy frontend pods and backend pods together, deploy the same type of applications on a specific node, or deploy different applications on different nodes. + +Node Affinity (nodeAffinity) +---------------------------- + +Labels are the basis of affinity rules. Let's look at the labels on nodes in a cluster. + +.. code-block:: + + $ kubectl describe node 192.168.0.212 + Name: 192.168.0.212 + Roles: + Labels: beta.kubernetes.io/arch=amd64 + beta.kubernetes.io/os=linux + failure-domain.beta.kubernetes.io/is-baremetal=false + failure-domain.beta.kubernetes.io/region=****** + failure-domain.beta.kubernetes.io/zone=****** + kubernetes.io/arch=amd64 + kubernetes.io/availablezone=****** + kubernetes.io/eniquota=12 + kubernetes.io/hostname=192.168.0.212 + kubernetes.io/os=linux + node.kubernetes.io/subnetid=fd43acad-33e7-48b2-a85a-24833f362e0e + os.architecture=amd64 + os.name=EulerOS_2.0_SP5 + os.version=3.10.0-862.14.1.5.h328.eulerosv2r7.x86_64 + +These labels are automatically added by CCE during node creation. The following describes a few that are frequently used during scheduling. + +- **failure-domain.beta.kubernetes.io/region**: region where the node is located. +- **failure-domain.beta.kubernetes.io/zone**: availability zone to which the node belongs. +- **kubernetes.io/hostname**: host name of the node. + +When you deploy pods, you can use a nodeSelector, as described in :ref:`DaemonSet `, to constrain pods to nodes with specific labels. The following example shows how to use a nodeSelector to deploy pods only on the nodes with the **gpu=true** label. + +.. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + name: nginx + spec: + nodeSelector: # Node selection. A pod is deployed on a node only when the node has the gpu=true label. + gpu: true + ... + +Node affinity rules can achieve the same results, as shown in the following example. + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: gpu + labels: + app: gpu + spec: + selector: + matchLabels: + app: gpu + replicas: 3 + template: + metadata: + labels: + app: gpu + spec: + containers: + - image: nginx:alpine + name: gpu + resources: + requests: + cpu: 100m + memory: 200Mi + limits: + cpu: 100m + memory: 200Mi + imagePullSecrets: + - name: default-secret + affinity: + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: gpu + operator: In + values: + - "true" + +Even though the node affinity rule requires more lines, it is more expressive, which will be further described later. + +**requiredDuringSchedulingIgnoredDuringExecution** seems to be complex, but it can be easily understood as a combination of two parts. + +- requiredDuringScheduling indicates that pods can be scheduled to the node only when all the defined rules are met (required). +- IgnoredDuringExecution indicates that pods already running on the node do not need to meet the defined rules. That is, a label on the node is ignored, and pods that require the node to contain that label will not be re-scheduled. + +In addition, the value of **operator** is **In**, indicating that the label value must be in the values list. Other available operator values are as follows: + +- **NotIn**: The label value is not in a list. +- **Exists**: A specific label exists. +- **DoesNotExist**: A specific label does not exist. +- **Gt**: The label value is greater than a specified value (string comparison). +- **Lt**: The label value is less than a specified value (string comparison). + +Note that there is no such thing as nodeAntiAffinity because operators **NotIn** and **DoesNotExist** provide the same function. + +The following describes how to check whether the rule takes effect. Assume that a cluster has three nodes. + +.. code-block:: + + $ kubectl get node + NAME STATUS ROLES AGE VERSION + 192.168.0.212 Ready 13m v1.15.6-r1-20.3.0.2.B001-15.30.2 + 192.168.0.94 Ready 13m v1.15.6-r1-20.3.0.2.B001-15.30.2 + 192.168.0.97 Ready 13m v1.15.6-r1-20.3.0.2.B001-15.30.2 + +Add the **gpu=true** label to the **192.168.0.212** node. + +.. code-block:: + + $ kubectl label node 192.168.0.212 gpu=true + node/192.168.0.212 labeled + + $ kubectl get node -L gpu + NAME STATUS ROLES AGE VERSION GPU + 192.168.0.212 Ready 13m v1.15.6-r1-20.3.0.2.B001-15.30.2 true + 192.168.0.94 Ready 13m v1.15.6-r1-20.3.0.2.B001-15.30.2 + 192.168.0.97 Ready 13m v1.15.6-r1-20.3.0.2.B001-15.30.2 + +Create the Deployment. You can find that all pods are deployed on the **192.168.0.212** node. + +.. code-block:: + + $ kubectl create -f affinity.yaml + deployment.apps/gpu created + + $ kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + gpu-6df65c44cf-42xw4 1/1 Running 0 15s 172.16.0.37 192.168.0.212 + gpu-6df65c44cf-jzjvs 1/1 Running 0 15s 172.16.0.36 192.168.0.212 + gpu-6df65c44cf-zv5cl 1/1 Running 0 15s 172.16.0.38 192.168.0.212 + +Node Preference Rule +-------------------- + +The preceding **requiredDuringSchedulingIgnoredDuringExecution** rule is a hard selection rule. There is another type of selection rule, that is, **preferredDuringSchedulingIgnoredDuringExecution**. It is used to specify which nodes are preferred during scheduling. + +To achieve this effect, add a node attached with SAS disks to the cluster, add the **DISK=SAS** label to the node, and add the **DISK=SSD** label to the other three nodes. + +.. code-block:: + + $ kubectl get node -L DISK,gpu + NAME STATUS ROLES AGE VERSION DISK GPU + 192.168.0.100 Ready 7h23m v1.15.6-r1-20.3.0.2.B001-15.30.2 SAS + 192.168.0.212 Ready 8h v1.15.6-r1-20.3.0.2.B001-15.30.2 SSD true + 192.168.0.94 Ready 8h v1.15.6-r1-20.3.0.2.B001-15.30.2 SSD + 192.168.0.97 Ready 8h v1.15.6-r1-20.3.0.2.B001-15.30.2 SSD + +Define a Deployment. Use the **preferredDuringSchedulingIgnoredDuringExecution** rule to set the weight of nodes with the SSD disk installed as **80** and nodes with the **gpu=true** label as **20**. In this way, pods are preferentially deployed on the nodes with the SSD disk installed. + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: gpu + labels: + app: gpu + spec: + selector: + matchLabels: + app: gpu + replicas: 10 + template: + metadata: + labels: + app: gpu + spec: + containers: + - image: nginx:alpine + name: gpu + resources: + requests: + cpu: 100m + memory: 200Mi + limits: + cpu: 100m + memory: 200Mi + imagePullSecrets: + - name: default-secret + affinity: + nodeAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 80 + preference: + matchExpressions: + - key: DISK + operator: In + values: + - SSD + - weight: 20 + preference: + matchExpressions: + - key: gpu + operator: In + values: + - "true" + +After the deployment, there are five pods deployed on the node **192.168.0.212** (label: **DISK=SSD** and **GPU=true**), three pods deployed on the node **192.168.0.97** (label: **DISK=SSD**), and two pods deployed on the node **192.168.0.100** (label: **DISK=SAS**). + +From the preceding output, you can find that no pods of the Deployment are scheduled to node **192.168.0.94** (label: **DISK=SSD**). This is because the node already has many pods on it and its resource usage is high. This also indicates that the **preferredDuringSchedulingIgnoredDuringExecution** rule defines a preference rather than a hard requirement. + +.. code-block:: + + $ kubectl create -f affinity2.yaml + deployment.apps/gpu created + + $ kubectl get po -o wide + NAME READY STATUS RESTARTS AGE IP NODE + gpu-585455d466-5bmcz 1/1 Running 0 2m29s 172.16.0.44 192.168.0.212 + gpu-585455d466-cg2l6 1/1 Running 0 2m29s 172.16.0.63 192.168.0.97 + gpu-585455d466-f2bt2 1/1 Running 0 2m29s 172.16.0.79 192.168.0.100 + gpu-585455d466-hdb5n 1/1 Running 0 2m29s 172.16.0.42 192.168.0.212 + gpu-585455d466-hkgvz 1/1 Running 0 2m29s 172.16.0.43 192.168.0.212 + gpu-585455d466-mngvn 1/1 Running 0 2m29s 172.16.0.48 192.168.0.97 + gpu-585455d466-s26qs 1/1 Running 0 2m29s 172.16.0.62 192.168.0.97 + gpu-585455d466-sxtzm 1/1 Running 0 2m29s 172.16.0.45 192.168.0.212 + gpu-585455d466-t56cm 1/1 Running 0 2m29s 172.16.0.64 192.168.0.100 + gpu-585455d466-t5w5x 1/1 Running 0 2m29s 172.16.0.41 192.168.0.212 + +In the preceding example, the node scheduling priority is as follows. Nodes with both **SSD** and **gpu=true** labels have the highest priority. Nodes with the **SSD** label but no **gpu=true** label have the second priority (weight: 80). Nodes with the **gpu=true** label but no **SSD** label have the third priority. Nodes without any of these two labels have the lowest priority. + + +.. figure:: /_static/images/en-us_image_0000001202101148.png + :alt: **Figure 1** Scheduling priority + + **Figure 1** Scheduling priority + +Workload Affinity (podAffinity) +------------------------------- + +Node affinity rules affect only the affinity between pods and nodes. Kubernetes also supports configuring inter-pod affinity rules. For example, the frontend and backend of an application can be deployed together on one node to reduce access latency. There are also two types of inter-pod affinity rules: **requiredDuringSchedulingIgnoredDuringExecution** and **preferredDuringSchedulingIgnoredDuringExecution**. + +Assume that the backend of an application has been created and has the **app=backend** label. + +.. code-block:: + + $ kubectl get po -o wide + NAME READY STATUS RESTARTS AGE IP NODE + backend-658f6cb858-dlrz8 1/1 Running 0 2m36s 172.16.0.67 192.168.0.100 + +You can configure the following pod affinity rule to deploy the frontend pods of the application to the same node as the backend pods. + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: frontend + labels: + app: frontend + spec: + selector: + matchLabels: + app: frontend + replicas: 3 + template: + metadata: + labels: + app: frontend + spec: + containers: + - image: nginx:alpine + name: frontend + resources: + requests: + cpu: 100m + memory: 200Mi + limits: + cpu: 100m + memory: 200Mi + imagePullSecrets: + - name: default-secret + affinity: + podAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - topologyKey: kubernetes.io/hostname + labelSelector: + matchExpressions: + - key: app + operator: In + values: + - backend + +Deploy the frontend and you can find that the frontend is deployed on the same node as the backend. + +.. code-block:: + + $ kubectl create -f affinity3.yaml + deployment.apps/frontend created + + $ kubectl get po -o wide + NAME READY STATUS RESTARTS AGE IP NODE + backend-658f6cb858-dlrz8 1/1 Running 0 5m38s 172.16.0.67 192.168.0.100 + frontend-67ff9b7b97-dsqzn 1/1 Running 0 6s 172.16.0.70 192.168.0.100 + frontend-67ff9b7b97-hxm5t 1/1 Running 0 6s 172.16.0.71 192.168.0.100 + frontend-67ff9b7b97-z8pdb 1/1 Running 0 6s 172.16.0.72 192.168.0.100 + +The **topologyKey** field specifies the selection range. The scheduler selects nodes within the range based on the affinity rule defined. The effect of **topologyKey** is not fully demonstrated in the preceding example because all the nodes have the **kubernetes.io/hostname** label, that is, all the nodes are within the range. + +To see how **topologyKey** works, assume that the backend of the application has two pods, which are running on different nodes. + +.. code-block:: + + $ kubectl get po -o wide + NAME READY STATUS RESTARTS AGE IP NODE + backend-658f6cb858-5bpd6 1/1 Running 0 23m 172.16.0.40 192.168.0.97 + backend-658f6cb858-dlrz8 1/1 Running 0 2m36s 172.16.0.67 192.168.0.100 + +Add the **prefer=true** label to nodes **192.168.0.97** and **192.168.0.94**. + +.. code-block:: + + $ kubectl label node 192.168.0.97 prefer=true + node/192.168.0.97 labeled + $ kubectl label node 192.168.0.94 prefer=true + node/192.168.0.94 labeled + + $ kubectl get node -L prefer + NAME STATUS ROLES AGE VERSION PREFER + 192.168.0.100 Ready 44m v1.15.6-r1-20.3.0.2.B001-15.30.2 + 192.168.0.212 Ready 91m v1.15.6-r1-20.3.0.2.B001-15.30.2 + 192.168.0.94 Ready 91m v1.15.6-r1-20.3.0.2.B001-15.30.2 true + 192.168.0.97 Ready 91m v1.15.6-r1-20.3.0.2.B001-15.30.2 true + +Define **topologyKey** in the **podAffinity** section as **prefer**. + +.. code-block:: + + affinity: + podAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - topologyKey: prefer + labelSelector: + matchExpressions: + - key: app + operator: In + values: + - backend + +The scheduler recognizes the nodes with the **prefer** label, that is, **192.168.0.97** and **192.168.0.94**, and then find the pods with the **app=backend** label. In this way, all frontend pods are deployed onto **192.168.0.97**. + +.. code-block:: + + $ kubectl create -f affinity3.yaml + deployment.apps/frontend created + + $ kubectl get po -o wide + NAME READY STATUS RESTARTS AGE IP NODE + backend-658f6cb858-5bpd6 1/1 Running 0 26m 172.16.0.40 192.168.0.97 + backend-658f6cb858-dlrz8 1/1 Running 0 5m38s 172.16.0.67 192.168.0.100 + frontend-67ff9b7b97-dsqzn 1/1 Running 0 6s 172.16.0.70 192.168.0.97 + frontend-67ff9b7b97-hxm5t 1/1 Running 0 6s 172.16.0.71 192.168.0.97 + frontend-67ff9b7b97-z8pdb 1/1 Running 0 6s 172.16.0.72 192.168.0.97 + +Workload Anti-Affinity (podAntiAffinity) +---------------------------------------- + +Unlike the scenarios in which pods are preferred to be scheduled onto the same node, sometimes, it could be the exact opposite. For example, if certain pods are deployed together, they will affect the performance. + +The following example defines an inter-pod anti-affinity rule, which specifies that pods must not be scheduled to nodes that already have pods with the **app=frontend** label, that is, to deploy the pods of the frontend to different nodes with each node has only one replica. + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: frontend + labels: + app: frontend + spec: + selector: + matchLabels: + app: frontend + replicas: 5 + template: + metadata: + labels: + app: frontend + spec: + containers: + - image: nginx:alpine + name: frontend + resources: + requests: + cpu: 100m + memory: 200Mi + limits: + cpu: 100m + memory: 200Mi + imagePullSecrets: + - name: default-secret + affinity: + podAntiAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + - topologyKey: kubernetes.io/hostname + labelSelector: + matchExpressions: + - key: app + operator: In + values: + - frontend + +Deploy the frontend and query the deployment results. You can find that each node has only one frontend pod and one pod of the Deployment is **Pending**. This is because when the scheduler is deploying the fifth pod, all nodes already have one pod with the **app=frontend** label on them. There is no available node. Therefore, the fifth pod will remain in the **Pending** status. + +.. code-block:: + + $ kubectl create -f affinity4.yaml + deployment.apps/frontend created + + $ kubectl get po -o wide + NAME READY STATUS RESTARTS AGE IP NODE + frontend-6f686d8d87-8dlsc 1/1 Running 0 18s 172.16.0.76 192.168.0.100 + frontend-6f686d8d87-d6l8p 0/1 Pending 0 18s + frontend-6f686d8d87-hgcq2 1/1 Running 0 18s 172.16.0.54 192.168.0.97 + frontend-6f686d8d87-q7cfq 1/1 Running 0 18s 172.16.0.47 192.168.0.212 + frontend-6f686d8d87-xl8hx 1/1 Running 0 18s 172.16.0.23 192.168.0.94 + +Configuring Scheduling Policies +------------------------------- + +#. Log in to the CCE console. + +#. When creating a workload, click **Scheduling** in the **Advanced Settings** area. + + .. table:: **Table 1** Node affinity settings + + +-----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===========+===========================================================================================================================================================================================================================================================================+ + | Required | This is a hard rule that must be met for scheduling. It corresponds to **requiredDuringSchedulingIgnoredDuringExecution** in Kubernetes. Multiple required rules can be set, and scheduling will be performed if only one of them is met. | + +-----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Preferred | This is a soft rule specifying preferences that the scheduler will try to enforce but will not guarantee. It corresponds to **preferredDuringSchedulingIgnoredDuringExecution** in Kubernetes. Scheduling is performed when one rule is met or none of the rules are met. | + +-----------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +#. Under **Node Affinity**, **Workload Affinity**, and **Workload Anti-Affinity**, click |image1| to add scheduling policies. In the dialog box displayed, add a policy directly or by specifying a node or an AZ. + + Specifying a node or an AZ is essentially implemented through labels. The **kubernetes.io/hostname** label is used when you specify a node, and the **failure-domain.beta.kubernetes.io/zone** label is used when you specify an AZ. + + .. table:: **Table 2** Scheduling policy configuration + + +-----------------------------------+------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+============================================================================================================+ + | Label | Node label. You can use the default label or customize a label. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------+ + | Operator | The following relations are supported: **In**, **NotIn**, **Exists**, **DoesNotExist**, **Gt**, and **Lt** | + | | | + | | - **In**: A label exists in the label list. | + | | - **NotIn**: A label does not exist in the label list. | + | | - **Exists**: A specific label exists. | + | | - **DoesNotExist**: A specific label does not exist. | + | | - **Gt**: The label value is greater than a specified value (string comparison). | + | | - **Lt**: The label value is less than a specified value (string comparison). | + +-----------------------------------+------------------------------------------------------------------------------------------------------------+ + | Label Value | Label value. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------+ + | Namespace | This parameter is available only in a workload affinity or anti-affinity scheduling policy. | + | | | + | | Namespace for which the scheduling policy takes effect. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------+ + | Topology Key | This parameter can be used only in a workload affinity or anti-affinity scheduling policy. | + | | | + | | Select the scope specified by **topologyKey** and then select the content defined by the policy. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------+ + | Weight | This parameter can be set only in a **Preferred** scheduling policy. | + +-----------------------------------+------------------------------------------------------------------------------------------------------------+ + +.. |image1| image:: /_static/images/en-us_image_0000001203031716.png diff --git a/umn/source/workloads/configuring_a_container/setting_an_environment_variable.rst b/umn/source/workloads/configuring_a_container/setting_an_environment_variable.rst index 892a491..164e5cc 100644 --- a/umn/source/workloads/configuring_a_container/setting_an_environment_variable.rst +++ b/umn/source/workloads/configuring_a_container/setting_an_environment_variable.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0113.html +:original_name: cce_10_0113.html -.. _cce_01_0113: +.. _cce_10_0113: Setting an Environment Variable =============================== @@ -12,57 +12,131 @@ An environment variable is a variable whose value can affect the way a running c The function of setting environment variables on CCE is the same as that of specifying **ENV** in a Dockerfile. -CCE provides three ways to add environment variables: Manually add environment variables, import environment variables from a secret, and import environment variables from a configMap. - .. important:: After a container is started, do not modify configurations in the container. If configurations in the container are modified (for example, passwords, certificates, and environment variables of a containerized application are added to the container), the configurations will be lost after the container restarts and container services will become abnormal. An example scenario of container restart is pod rescheduling due to node anomalies. Configurations must be imported to a container as arguments. Otherwise, configurations will be lost after the container restarts. -Manually Adding Environment Variables -------------------------------------- +Environment variables can be set in the following modes: -#. When creating a workload, add a container image. Then, expand **Environment Variables** and click **Add Environment Variables**. +- **Custom** +- **Added from ConfigMap**: Import all keys in a ConfigMap as environment variables. +- **Added from ConfigMap key**: Import a key in a ConfigMap as the value of an environment variable. For example, if you import **configmap_value** of **configmap_key** in a ConfigMap as the value of environment variable **key1**, an environment variable named **key1** with its value **is configmap_value** exists in the container. +- **Added from secret**: Import all keys in a secret as environment variables. +- **Added from secret key**: Import the value of a key in a secret as the value of an environment variable. For example, if you import **secret_value** of **secret_key** in secret **secret-example** as the value of environment variable **key2**, an environment variable named **key2** with its value **secret_value** exists in the container. +- **Variable value/reference**: Use the field defined by a pod as the value of the environment variable, for example, the pod name. +- **Resource Reference**: Use the field defined by a container as the value of the environment variable, for example, the CPU limit of the container. -#. Configure the following parameters as required: +Adding Environment Variables +---------------------------- - - **Type**: Set this to **Added manually**. - - **Variable Name**: Enter a variable name, for example, demo. - - **Variable Value/Reference**: Enter a variable value, for example, value. +#. Log in to the CCE console. When creating a workload, select **Environment Variables** under **Container Settings**. +#. Set environment variables. - .. figure:: /_static/images/en-us_image_0000001190302095.png - :alt: **Figure 1** Manually adding environment variables + |image1| - **Figure 1** Manually adding environment variables +YAML Example +------------ -Importing Environment Variables from a Secret ---------------------------------------------- +.. code-block:: -#. You need to create a key first. For details, see :ref:`Creating a Secret `. + apiVersion: apps/v1 + kind: Deployment + metadata: + name: env-example + namespace: default + spec: + replicas: 1 + selector: + matchLabels: + app: env-example + template: + metadata: + labels: + app: env-example + spec: + containers: + - name: container-1 + image: nginx:alpine + imagePullPolicy: Always + resources: + requests: + cpu: 250m + memory: 512Mi + limits: + cpu: 250m + memory: 512Mi + env: + - name: key # Custom + value: value + - name: key1 # Added from ConfigMap key + valueFrom: + configMapKeyRef: + name: configmap-example + key: key1 + - name: key2 # Added from secret key + valueFrom: + secretKeyRef: + name: secret-example + key: key2 + - name: key3 # Variable reference, which uses the field defined by a pod as the value of the environment variable. + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + - name: key4 # Resource reference, which uses the field defined by a container as the value of the environment variable. + valueFrom: + resourceFieldRef: + containerName: container1 + resource: limits.cpu + divisor: 1 + envFrom: + - configMapRef: # Added from ConfigMap + name: configmap-example + - secretRef: # Added from secret + name: secret-example + imagePullSecrets: + - name: default-secret -#. When creating a workload, add a container image. Then, expand **Environment Variables** and click **Add Environment Variables**. +Viewing Environment Variables +----------------------------- -#. Configure the following parameters as required: +If the contents of **configmap-example** and **secret-example** are as follows: - - **Type**: Set this to **Added from Secret**. - - **Variable Name**: Enter a variable name. - - **Variable Value/Reference**: Select the corresponding secret name and key. +.. code-block:: + $ kubectl get configmap configmap-example -oyaml + apiVersion: v1 + data: + configmap_key: configmap_value + kind: ConfigMap + ... - .. figure:: /_static/images/en-us_image_0000001190302097.png - :alt: **Figure 2** Importing environment variables from a secret + $ kubectl get secret secret-example -oyaml + apiVersion: v1 + data: + secret_key: c2VjcmV0X3ZhbHVl # c2VjcmV0X3ZhbHVl is the value of secret_value in Base64 mode. + kind: Secret + ... - **Figure 2** Importing environment variables from a secret +The environment variables in the pod are as follows: -Importing Environment Variables from a ConfigMap ------------------------------------------------- +.. code-block:: -#. Create a ConfigMap first. For details, see :ref:`Creating a ConfigMap `. -#. When creating a workload, add a container image. Then, expand **Environment Variables** and click **Add Environment Variables**. -#. Configure the following parameters as required: + $ kubectl get pod + NAME READY STATUS RESTARTS AGE + env-example-695b759569-lx9jp 1/1 Running 0 17m - - **Type**: Set this to **Added from ConfigMap**. - - **Variable Name**: Enter a variable name. - - **Variable Value/Reference**: Select the corresponding ConfigMap name and key. + $ kubectl exec env-example-695b759569-lx9jp -- printenv + / # env + key=value # Custom environment variable + ey1=configmap_value # Added from ConfigMap key + key2=secret_value # Added from secret key + key3=env-example-695b759569-lx9jp # metadata.name defined by the pod + key4=1 # limits.cpu defined by container1. The value is rounded up, in unit of cores. + configmap_key=configmap_value # Added from ConfigMap. The key value in the original ConfigMap key is directly imported. + secret_key=secret_value # Added from key. The key value in the original secret is directly imported. + +.. |image1| image:: /_static/images/en-us_image_0000001247802971.png diff --git a/umn/source/workloads/configuring_a_container/setting_basic_container_information.rst b/umn/source/workloads/configuring_a_container/setting_basic_container_information.rst new file mode 100644 index 0000000..57a934d --- /dev/null +++ b/umn/source/workloads/configuring_a_container/setting_basic_container_information.rst @@ -0,0 +1,46 @@ +:original_name: cce_10_0396.html + +.. _cce_10_0396: + +Setting Basic Container Information +=================================== + +A workload is an abstract model of a group of pods. One pod can encapsulate one or more containers. You can click **Add Container** in the upper right corner to add multiple container images and set them separately. + +.. table:: **Table 1** Image parameters + + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +===================================+=====================================================================================================================================================================================================================================================================================+ + | Container Name | Name the container. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Image Name | Click **Select Image** and select the image used by the container. | + | | | + | | If you need to use a third-party image, see :ref:`Using a Third-Party Image `. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Image Tag | Select the image tag to be deployed. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Pull Policy | Image update or pull policy. If you select **Always**, the image is pulled from the image repository each time. If you do not select **Always**, the existing image of the node is preferentially used. If the image does not exist, the image is pulled from the image repository. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | CPU Quota | - **Request**: minimum number of CPU cores required by a container. The default value is 0.25 cores. | + | | - **Limit**: maximum number of CPU cores available for a container. Do not leave **Limit** unspecified. Otherwise, intensive use of container resources will occur and your workload may exhibit unexpected behavior. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Memory Quota | - **Request**: minimum amount of memory required by a container. The default value is 512 MiB. | + | | - **Limit**: maximum amount of memory available for a container. When memory usage exceeds the specified memory limit, the container will be terminated. | + | | | + | | For more information about **Request** and **Limit**, see :ref:`Setting Container Specifications `. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | GPU Quota | It is configurable only when the cluster contains GPU nodes. | + | | | + | | - **All**: The GPU is not used. | + | | - **Dedicated**: GPU resources are exclusively used by the container. | + | | - **Shared**: percentage of GPU resources used by the container. For example, if this parameter is set to **10%**, the container uses 10% of GPU resources. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Privileged Container | Programs in a privileged container have certain privileges. | + | | | + | | If **Privileged Container** is enabled, the container is assigned privileges. For example, privileged containers can manipulate network devices on the host machine and modify kernel parameters. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Init Container | Indicates whether to use the container as an init container. | + | | | + | | An init container is a special container that run before app containers in a pod. For details, see `Init Container `__. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/workloads/configuring_a_container/setting_container_lifecycle_parameters.rst b/umn/source/workloads/configuring_a_container/setting_container_lifecycle_parameters.rst index b4f744b..1700061 100644 --- a/umn/source/workloads/configuring_a_container/setting_container_lifecycle_parameters.rst +++ b/umn/source/workloads/configuring_a_container/setting_container_lifecycle_parameters.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0105.html +:original_name: cce_10_0105.html -.. _cce_01_0105: +.. _cce_10_0105: Setting Container Lifecycle Parameters ====================================== @@ -12,21 +12,25 @@ CCE provides callback functions for the lifecycle management of containerized ap CCE provides the following lifecycle callback functions: -- **Start Command**: executed to start a container. For details, see :ref:`Setting Container Startup Commands `. -- **Post-Start**: executed immediately after a container is started. For details, see :ref:`Post-Start Processing `. -- **Pre-Stop**: executed before a container is stopped. The pre-stop processing function helps you ensure that the services running on the pods can be completed in advance in the case of pod upgrade or deletion. For details, see :ref:`Pre-Stop Processing `. +- **Startup Command**: executed to start a container. For details, see :ref:`Startup Commands `. +- **Post-Start**: executed immediately after a container is started. For details, see :ref:`Post-Start Processing `. +- **Pre-Stop**: executed before a container is stopped. The pre-stop processing function helps you ensure that the services running on the pods can be completed in advance in the case of pod upgrade or deletion. For details, see :ref:`Pre-Stop Processing `. -Commands and Parameters Used to Run a Container ------------------------------------------------ +.. _cce_10_0105__section54912655316: + +Startup Commands +---------------- + +By default, the default command during image start. To run a specific command or rewrite the default image value, you must perform specific settings: A Docker image has metadata that stores image information. If lifecycle commands and arguments are not set, CCE runs the default commands and arguments, that is, Docker instructions **ENTRYPOINT** and **CMD**, provided during image creation. If the commands and arguments used to run a container are set during application creation, the default commands **ENTRYPOINT** and **CMD** are overwritten during image build. The rules are as follows: -.. table:: **Table 1** Commands and parameters used to run a container +.. table:: **Table 1** Commands and arguments used to run a container +------------------+--------------+----------------------------+-------------------------------+--------------------+ - | Image Entrypoint | Image CMD | Command to Run a Container | Parameters to Run a Container | Command Executed | + | Image ENTRYPOINT | Image CMD | Command to Run a Container | Parameters to Run a Container | Command Executed | +==================+==============+============================+===============================+====================+ | [touch] | [/root/test] | Not set | Not set | [touch /root/test] | +------------------+--------------+----------------------------+-------------------------------+--------------------+ @@ -37,23 +41,36 @@ If the commands and arguments used to run a container are set during application | [touch] | [/root/test] | [mkdir] | [/opt/test] | [mkdir /opt/test] | +------------------+--------------+----------------------------+-------------------------------+--------------------+ -Startup Commands ----------------- +#. Log in to the CCE console. When creating a workload, configure container information and select **Lifecycle**. +#. Enter a command and arguments on the **Startup Command** tab page. -By default, the default command during image start. To run a specific command or rewrite the default image value, you must perform specific settings: For details, see :ref:`Setting Container Startup Commands `. + .. table:: **Table 2** Container startup command -.. _cce_01_0105__section15243544163715: + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ + | Configuration Item | Procedure | + +===================================+=============================================================================================================================================+ + | Command | Enter an executable command, for example, **/run/server**. | + | | | + | | If there are multiple commands, separate them with spaces. If the command contains a space, you need to add a quotation mark (""). | + | | | + | | .. note:: | + | | | + | | In the case of multiple commands, you are advised to run **/bin/sh** or other **shell** commands. Other commands are used as parameters. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ + | Args | Enter the argument that controls the container running command, for example, **--port=8080**. | + | | | + | | If there are multiple arguments, separate them in different lines. | + +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------+ + +.. _cce_10_0105__section15243544163715: Post-Start Processing --------------------- -#. Log in to the CCE console. Expand **Lifecycle** when adding a container during workload creation. +#. Log in to the CCE console. When creating a workload, configure container information and select **Lifecycle**. +#. Set the post-start processing parameters on the **Post-Start** tab page. -#. Set the post-start processing parameters, as listed in :ref:`Table 2 `. - - .. _cce_01_0105__table823614643810: - - .. table:: **Table 2** Post-start processing parameters + .. table:: **Table 3** Post-start processing parameters +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | @@ -75,18 +92,18 @@ Post-Start Processing | | | | | - **Path**: (optional) request URL. | | | - **Port**: (mandatory) request port. | - | | - **Host Address**: (optional) IP address of the request. The default value is the IP address of the node where the container resides. | + | | - **Host**: (optional) IP address of the request. The default value is the IP address of the node where the container resides. | +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -.. _cce_01_0105__section2334114473712: +.. _cce_10_0105__section2334114473712: Pre-Stop Processing ------------------- -#. When creating a workload and adding a container, expand **Lifecycle**. -#. Set **pre-stop** parameters, as shown in :ref:`Table 2 `. +#. Log in to the CCE console. When creating a workload, configure container information and select **Lifecycle**. +#. Set the pre-start processing parameters on the **Pre-Stop** tab page. - .. table:: **Table 3** Pre-stop parameters + .. table:: **Table 4** Pre-stop processing parameters +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Parameter | Description | @@ -108,84 +125,48 @@ Pre-Stop Processing | | | | | - **Path**: (optional) request URL. | | | - **Port**: (mandatory) request port. | - | | - **Host Address**: (optional) IP address of the request. The default value is the IP address of the node where the container resides. | + | | - **Host**: (optional) IP address of the request. The default value is the IP address of the node where the container resides. | +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -Container Restart Policy ------------------------- - -The **restartPolicy** field is used to specify the pod restart policy. The restart policy type can be **Always**, **OnFailure**, or **Never**. The default value is **Always**. - -When **restartPolicy** is used, containers are restarted only through kubelet on the same node. - -+----------------+------------------------------------------------------------------------------------------------------------+ -| Restart Policy | Description | -+================+============================================================================================================+ -| Always | When a container fails, kubelet automatically restarts the container. | -+----------------+------------------------------------------------------------------------------------------------------------+ -| OnFailure | When the container stops running and the exit code is not 0, kubelet automatically restarts the container. | -+----------------+------------------------------------------------------------------------------------------------------------+ -| Never | kubelet does not restart the container regardless of the container running status. | -+----------------+------------------------------------------------------------------------------------------------------------+ - -.. note:: - - Controllers that can manage pods include ReplicaSet Controllers, jobs, DaemonSets, and kubelet (static pod). - - - ReplicaSet Controller and DaemonSet: The policy must be set to **Always** to ensure that containers run continuously. - - Job: The policy can be set to **OnFailure** or **Never** to ensure that containers are not restarted after being executed. - - kubelet will restart a pod whenever it fails, regardless of the value of **restartPolicy**. In addition, no health check is performed on the pod. - -.. _cce_01_0105__section151181981167: - -Example YAML for Setting the Container Lifecycle ------------------------------------------------- +Example YAML +------------ This section uses Nginx as an example to describe how to set the container lifecycle. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +In the following configuration file, the **postStart** command is defined to run the **install.sh** command in the **/bin/bash** directory. **preStop** is defined to run the **uninstall.sh** command. -#. Create and edit the **nginx-deployment.yaml** file. **nginx-deployment.yaml** is an example file name, and you can change it as required. +.. code-block:: - **vi nginx-deployment.yaml** - - In the following configuration file, the **postStart** command is defined to run the **install.sh** command in the **/bin/bash** directory. **preStop** is defined to run the **uninstall.sh** command. - - .. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - spec: - replicas: 1 - selector: - matchLabels: - app: nginx - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: nginx - spec: - restartPolicy: Always #Restart policy - containers: - - image: nginx - command: - - sleep 3600 #Startup command - imagePullPolicy: Always - lifecycle: - postStart: - exec: - command: - - /bin/bash - - install.sh #Post-start command - preStop: - exec: - command: - - /bin/bash - - uninstall.sh #Pre-stop command - name: nginx - imagePullSecrets: - - name: default-secret + apiVersion: apps/v1 + kind: Deployment + metadata: + name: nginx + spec: + replicas: 1 + selector: + matchLabels: + app: nginx + template: + metadata: + labels: + app: nginx + spec: + containers: + - image: nginx + command: + - sleep 3600 #Startup command + imagePullPolicy: Always + lifecycle: + postStart: + exec: + command: + - /bin/bash + - install.sh #Post-start command + preStop: + exec: + command: + - /bin/bash + - uninstall.sh #Pre-stop command + name: nginx + imagePullSecrets: + - name: default-secret diff --git a/umn/source/workloads/configuring_a_container/setting_container_specifications.rst b/umn/source/workloads/configuring_a_container/setting_container_specifications.rst index 86d6e14..c78d517 100644 --- a/umn/source/workloads/configuring_a_container/setting_container_specifications.rst +++ b/umn/source/workloads/configuring_a_container/setting_container_specifications.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0163.html +:original_name: cce_10_0163.html -.. _cce_01_0163: +.. _cce_10_0163: Setting Container Specifications ================================ @@ -8,30 +8,25 @@ Setting Container Specifications Scenario -------- -CCE allows you to set resource limits for added containers during workload creation. You can request and limit the CPU and memory quotas used by each pod in the workload. +CCE allows you to set resource limits for added containers during workload creation. You can apply for and limit the CPU and memory quotas used by each pod in a workload. Meanings -------- For **CPU** and **Memory**, the meanings of **Request** and **Limit** are as follows: -- If **Request** is selected, the system schedules the pod to the node that meets the requirements for workload deployment based on the request value. -- If **Request** is deselected, the system schedules the pod to a random node for workload deployment. -- If **Limit** is selected, the system limits the resources used by the workload based on the preset value. -- If **Limit** is deselected, the system does not limit the resources used by the pod. If the memory resources used by the pod exceed the memory allocated to the node, the workload or node may be unavailable. +- **Request**: Schedules the pod to the node that meets the requirements for workload deployment. +- **Limit**: Limits the resources used by the workload. .. note:: When creating a workload, you are advised to set the upper and lower limits of CPU and memory resources. If the upper and lower resource limits are not set for a workload, a resource leak of this workload will make resources unavailable for other workloads deployed on the same node. In addition, workloads that do not have upper and lower resource limits cannot be accurately monitored. -For **GPU** quotas, the meanings of **Use** and **Any GPU type** are as follows: - -- If **Use** is selected, the system schedules the pod to a node that meets the requirements for workload deployment based on the configured value. -- **Any GPU type** is selected by default and cannot be deselected. This option indicates that the resources used by pods are not limited. - Configuration Description ------------------------- +In actual production services, the recommended ratio of **Request** to **Limit** is about 1:1.5. For some sensitive services, the recommended ratio is 1:1. If the **Request** is too small and the **Limit** is too large, node resources are overcommitted. During service peaks, the memory or CPU of a node may be used up. As a result, the node is unavailable. + - CPU quotas: .. table:: **Table 1** Description of CPU quotas diff --git a/umn/source/workloads/configuring_a_container/setting_container_startup_commands.rst b/umn/source/workloads/configuring_a_container/setting_container_startup_commands.rst deleted file mode 100644 index 6a4cf5a..0000000 --- a/umn/source/workloads/configuring_a_container/setting_container_startup_commands.rst +++ /dev/null @@ -1,208 +0,0 @@ -:original_name: cce_01_0008.html - -.. _cce_01_0008: - -Setting Container Startup Commands -================================== - -Scenario --------- - -When creating a workload or job, you can use an image to specify the processes running in the container. - -By default, the image runs the default command. To run a specific command or rewrite the default image value, you must perform the following settings: - -- **Working directory**: working directory of the command. - - If the working directory is not specified in the image or on the console, the default value is **/**. - -- **Command**: command that controls the running of an image. - -- **Args**: parameters transferred to the running command. - -.. important:: - - After a container is started, do not modify configurations in the container. If configurations in the container are modified (for example, passwords, certificates, and environment variables of a containerized application are added to the container), the configurations will be lost after the container restarts and container services will become abnormal. An example scenario of container restart is pod rescheduling due to node anomalies. - - Configurations must be imported to a container as arguments. Otherwise, configurations will be lost after the container restarts. - -Commands and Arguments Used to Run a Container ----------------------------------------------- - -A Docker image has metadata that stores image information. If lifecycle commands and arguments are not set, CCE runs the default commands and arguments, that is, Docker instructions **ENTRYPOINT** and **CMD**, provided during image creation. - -If the commands and arguments used to run a container are set during application creation, the default commands **ENTRYPOINT** and **CMD** are overwritten during image build. The rules are as follows: - -.. table:: **Table 1** Commands and parameters used to run a container - - +------------------+--------------+----------------------------+-------------------------+--------------------+ - | Image Entrypoint | Image CMD | Command to Run a Container | Args to Run a Container | Command Executed | - +==================+==============+============================+=========================+====================+ - | [touch] | [/root/test] | Not set | Not set | [touch /root/test] | - +------------------+--------------+----------------------------+-------------------------+--------------------+ - | [touch] | [/root/test] | [mkdir] | Not set | [mkdir] | - +------------------+--------------+----------------------------+-------------------------+--------------------+ - | [touch] | [/root/test] | Not set | [/opt/test] | [touch /opt/test] | - +------------------+--------------+----------------------------+-------------------------+--------------------+ - | [touch] | [/root/test] | [mkdir] | [/opt/test] | [mkdir /opt/test] | - +------------------+--------------+----------------------------+-------------------------+--------------------+ - -Setting the Startup Command ---------------------------- - -#. Log in to the CCE console. Expand **Lifecycle** when adding a container during workload or job creation. - -#. Enter the running command and parameters, as shown in :ref:`Table 2 `. - - .. note:: - - - The current startup command is provided as a string array and corresponds to the Entrypoint startup command of Docker. The format is as follows: ["executable", "param1", "param2",..]. For details about how to start Kubernetes containers, click `here `__. - - The lifecycle of a container is the same as that of the startup command. That is, the lifecycle of the container ends after the command is executed. - - .. _cce_01_0008__table15533234825: - - .. table:: **Table 2** Container startup command - - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ - | Configuration Item | Procedure | - +===================================+===========================================================================================================================================+ - | Command | Enter an executable command, for example, **/run/server**. | - | | | - | | If there are multiple commands, separate them with spaces. If the command contains a space, you need to add a quotation mark (""). | - | | | - | | .. note:: | - | | | - | | If there are multiple commands, you are advised to run the **/bin/sh** or other shell commands. Other commands are used as parameters. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ - | Args | Enter the argument that controls the container running command, for example, **--port=8080**. | - | | | - | | If there are multiple arguments, separate them in different lines. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------+ - - The following uses Nginx as an example to describe three typical application scenarios of the container startup command: - - Example code: - - .. code-block:: - - nginx -c nginx.conf - - - Scenario 1: Both the **command** and **arguments** are set. - - - .. figure:: /_static/images/en-us_image_0000001190302089.png - :alt: **Figure 1** Setting the startup command and parameters - - **Figure 1** Setting the startup command and parameters - - Example YAML file: - - .. code-block:: - - command: - - nginx - args: - - '-c' - - nginx.conf - - - Scenario 2: Only the **command** is set. - - - .. figure:: /_static/images/en-us_image_0000001144342236.png - :alt: **Figure 2** Setting the startup command - - **Figure 2** Setting the startup command - - .. note:: - - A command must be enclosed in double quotes. If no double quotes are added, the command is split into multiple commands based on space character. - - Example YAML file: - - .. code-block:: - - command: - - nginx -c nginx.conf - args: - - - Scenario 3: Only **arguments** are set. - - - .. figure:: /_static/images/en-us_image_0000001190302091.png - :alt: **Figure 3** Setting startup arguments - - **Figure 3** Setting startup arguments - - .. note:: - - If the container startup command is not added to the system path, run the **/bin/sh** command to execute the container startup command. The container startup command must be enclosed in double quotes. - - Example YAML file: - - .. code-block:: - - command: - - /bin/sh - args: - - '-c' - - '"nginx -c nginx.conf"' - -#. Check or modify the YAML file. - - - When creating a workload, in the **Configure Advanced Settings** step, click YAML on the right. - - - .. figure:: /_static/images/en-us_image_0000001144342238.png - :alt: **Figure 4** Checking or editing a YAML file - - **Figure 4** Checking or editing a YAML file - - - After the workload is created, go to the workload list. In the same row as the workload, choose **More** > **Edit YAML**. - - - After the workload is created, go to the workload details page. On the displayed page, click **Edit YAML** in the upper right corner. - -Example YAML for Setting Container Startup Commands ---------------------------------------------------- - -This section uses Nginx as an example to describe how to set container startup commands using kubectl. - -Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. See :ref:`Using kubectl to create a Deployment ` or :ref:`Using kubectl to create a StatefulSet `. For more details on how to set container startup commands, see `official Kubernetes documentation `__. - -.. code-block:: - - apiVersion: apps/v1 - kind: Deployment - metadata: - name: nginx - spec: - replicas: 1 - selector: - matchLabels: - app: nginx - strategy: - type: RollingUpdate - template: - metadata: - labels: - app: nginx - spec: - containers: - - image: nginx - command: - - sleep - - '3600' #Startup command - imagePullPolicy: Always - lifecycle: - postStart: - exec: - command: - - /bin/bash - - install.sh #Post-start command - preStop: - exec: - command: - - /bin/bash - - uninstall.sh #Pre-stop command - name: nginx - imagePullSecrets: - - name: default-secret diff --git a/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst b/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst index fb2414d..5a41319 100644 --- a/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst +++ b/umn/source/workloads/configuring_a_container/setting_health_check_for_a_container.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0112.html +:original_name: cce_10_0112.html -.. _cce_01_0112: +.. _cce_10_0112: Setting Health Check for a Container ==================================== @@ -8,21 +8,22 @@ Setting Health Check for a Container Scenario -------- -Health check regularly checks the health status of containers during container running. If the health check function is not configured, a pod cannot detect service exceptions or automatically restart the service to restore it. This will result in a situation where the pod status is normal but the service in the pod is abnormal. +Health check regularly checks the health status of containers during container running. If the health check function is not configured, a pod cannot detect application exceptions or automatically restart the application to restore it. This will result in a situation where the pod status is normal but the application in the pod is abnormal. -CCE provides the following health check probes: +Kubernetes provides the following health check probes: -- **Liveness probe**: checks whether a container is still alive. It is similar to the **ps** command that checks whether a process exists. If the liveness check of a container fails, the cluster restarts the container. If the liveness check is successful, no operation is executed. -- **Readiness probe**: checks whether a container is ready to process user requests. Upon that the container is detected unready, service traffic will not be directed to the container. It may take a long time for some applications to start up before they can provide services. This is because that they need to load disk data or rely on startup of an external module. In this case, the application process is running, but the application cannot provide services. To address this issue, this health check probe is used. If the container readiness check fails, the cluster masks all requests sent to the container. If the container readiness check is successful, the container can be accessed. +- **Liveness probe** (livenessProbe): checks whether a container is still alive. It is similar to the **ps** command that checks whether a process exists. If the liveness check of a container fails, the cluster restarts the container. If the liveness check is successful, no operation is executed. +- **Readiness probe** (readinessProbe): checks whether a container is ready to process user requests. Upon that the container is detected unready, service traffic will not be directed to the container. It may take a long time for some applications to start up before they can provide services. This is because that they need to load disk data or rely on startup of an external module. In this case, the application process is running, but the application cannot provide services. To address this issue, this health check probe is used. If the container readiness check fails, the cluster masks all requests sent to the container. If the container readiness check is successful, the container can be accessed. +- **Startup probe** (startupProbe): checks when a container application has started. If such a probe is configured, it disables liveness and readiness checks until it succeeds, ensuring that those probes do not interfere with the application startup. This can be used to adopt liveness checks on slow starting containers, avoiding them getting killed by the kubelet before they are started. -Health Check Methods --------------------- +Check Method +------------ - **HTTP request** This health check mode is applicable to containers that provide HTTP/HTTPS services. The cluster periodically initiates an HTTP/HTTPS GET request to such containers. If the return code of the HTTP/HTTPS response is within 200-399, the probe is successful. Otherwise, the probe fails. In this health check mode, you must specify a container listening port and an HTTP/HTTPS request path. - For example, for a container that provides HTTP services, the HTTP check path is **/health-check**, the port is 80, and the host address is optional (which defaults to the container IP address). Here, 172.16.0.186 is used as an example, and we can get such a request: GET http://172.16.0.186:80/health-check. The cluster periodically initiates this request to the container. + For example, for a container that provides HTTP services, the HTTP check path is **/health-check**, the port is 80, and the host address is optional (which defaults to the container IP address). Here, 172.16.0.186 is used as an example, and we can get such a request: GET http://172.16.0.186:80/health-check. The cluster periodically initiates this request to the container. You can also add one or more headers to an HTTP request. For example, set the request header name to **Custom-Header** and the corresponding value to **example**. - **TCP port** @@ -49,19 +50,85 @@ Health Check Methods - Put the program to be executed in the container image so that the program can be executed. - If the command to be executed is a shell script, do not directly specify the script as the command, but add a script parser. For example, if the script is **/data/scripts/health_check.sh**, you must specify **sh/data/scripts/health_check.sh** for command execution. The reason is that the cluster is not in the terminal environment when executing programs in a container. -Common Parameter Description ----------------------------- +- **gRPC Check** + + gRPC checks can configure startup, liveness, and readiness probes for your gRPC application without exposing any HTTP endpoint, nor do you need an executable. Kubernetes can connect to your workload via gRPC and query its status. + + .. important:: + + - The gRPC check is supported only in CCE clusters of v1.25 or later. + - To use gRPC for check, your application must support the `gRPC health checking protocol `__. + - Similar to HTTP and TCP probes, if the port is incorrect or the application does not support the health checking protocol, the check fails. + +Common Parameters +----------------- .. table:: **Table 1** Common parameter description - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===================================================================================================================================================================================================================================================================+ - | Initial Delay (s) | Check delay time in seconds. Set this parameter according to the normal startup time of services. | - | | | - | | For example, if this parameter is set to 30, the health check will be started 30 seconds after the container is started. The time is reserved for containerized services to start. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Timeout (s) | Timeout duration. Unit: second. | - | | | - | | For example, if this parameter is set to **10**, the timeout wait time for performing a health check is 10s. If the wait time elapses, the health check is regarded as a failure. If the parameter is left blank or set to **0**, the default timeout time is 1s. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Parameter | Description | + +==========================================+============================================================================================================================================================================================================================================================================+ + | **Period** (periodSeconds) | Indicates the probe detection period, in seconds. | + | | | + | | For example, if this parameter is set to **30**, the detection is performed every 30 seconds. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Delay** (initialDelaySeconds) | Check delay time in seconds. Set this parameter according to the normal startup time of services. | + | | | + | | For example, if this parameter is set to **30**, the health check will be started 30 seconds after the container is started. The time is reserved for containerized services to start. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Timeout** (timeoutSeconds) | Number of seconds after which the probe times out. Unit: second. | + | | | + | | For example, if this parameter is set to **10**, the timeout wait time for performing a health check is 10s. If the wait time elapses, the health check is regarded as a failure. If the parameter is left blank or set to **0**, the default timeout time is 1s. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Success Threshold** (successThreshold) | Minimum consecutive successes for the probe to be considered successful after having failed. For example, if this parameter is set to **1**, the workload status is normal only when the health check is successful for one consecutive time after the health check fails. | + | | | + | | The default value is **1**, which is also the minimum value. | + | | | + | | The value of this parameter is fixed to **1** in **Liveness Probe** and **Startup Probe**. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | **Failure Threshold** (failureThreshold) | Number of retry times when the detection fails. | + | | | + | | Giving up in case of liveness probe means to restart the container. In case of readiness probe the pod will be marked Unready. | + | | | + | | The default value is **3**. The minimum value is **1**. | + +------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +YAML Example +------------ + +.. code-block:: + + apiVersion: v1 + kind: Pod + metadata: + labels: + test: liveness + name: liveness-http + spec: + containers: + - name: liveness + image: nginx:alpine + args: + - /server + livenessProbe: + httpGet: + path: /healthz + port: 80 + httpHeaders: + - name: Custom-Header + value: Awesome + initialDelaySeconds: 3 + periodSeconds: 3 + readinessProbe: + exec: + command: + - cat + - /tmp/healthy + initialDelaySeconds: 5 + periodSeconds: 5 + startupProbe: + httpGet: + path: /healthz + port: 80 + failureThreshold: 30 + periodSeconds: 10 diff --git a/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst b/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst index 6c26fc0..026dac6 100644 --- a/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst +++ b/umn/source/workloads/configuring_a_container/using_a_third-party_image.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0009.html +:original_name: cce_10_0009.html -.. _cce_01_0009: +.. _cce_10_0009: Using a Third-Party Image ========================= @@ -15,31 +15,27 @@ Generally, a third-party image repository can be accessed only after authenticat Prerequisites ------------- -The node where the workload is running is accessible from public networks. You can access public networks through :ref:`LoadBalancer `. +The node where the workload is running is accessible from public networks. Using the Console ----------------- -#. .. _cce_01_0009__li16481144064414: +#. .. _cce_10_0009__li16481144064414: Create a secret for accessing a third-party image repository. - In the navigation pane, choose **Configuration Center** > **Secret**, and click **Create Secret**. **Type** must be set to **kubernetes.io/dockerconfigjson**. For details, see :ref:`Creating a Secret `. + Click the cluster name and access the cluster console. In the navigation pane, choose **ConfigMaps and Secrets**. On the **Secrets** tab page, click **Create Secret** in the upper right corner. Set **Secret Type** to **kubernetes.io/dockerconfigjson**. For details, see :ref:`Creating a Secret `. Enter the user name and password used to access the third-party image repository. -#. Create a workload. For details, see :ref:`Creating a Deployment ` or :ref:`Creating a StatefulSet `. If the workload will be created from a third-party image, set the image parameters as follows: +#. When creating a workload, you can enter a private image path in the format of **domainname/namespace/imagename:tag** in **Image Name** and select the key created in :ref:`1 `. - a. Set **Secret Authentication** to **Yes**. - b. Select the secret created in step :ref:`1 `. - c. Enter the image address. - -#. Click **Create**. +#. Set other parameters and click **Create Workload**. Using kubectl ------------- -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create a secret of the dockercfg type using kubectl. diff --git a/umn/source/workloads/cpu_core_binding/binding_cpu_cores.rst b/umn/source/workloads/cpu_core_binding/binding_cpu_cores.rst new file mode 100644 index 0000000..754725a --- /dev/null +++ b/umn/source/workloads/cpu_core_binding/binding_cpu_cores.rst @@ -0,0 +1,80 @@ +:original_name: cce_10_0351.html + +.. _cce_10_0351: + +Binding CPU Cores +================= + +By default, kubelet uses `CFS quotas `__ to enforce pod CPU limits. When the node runs many CPU-bound pods, the workload can move to different CPU cores depending on whether the pod is throttled and which CPU cores are available at scheduling time. Many workloads are not sensitive to this migration and thus work fine without any intervention. Some applications are CPU-sensitive. They are sensitive to: + +- CPU throttling +- Context switching +- Processor cache misses +- Cross-socket memory access +- Hyperthreads that are expected to run on the same physical CPU card + +If your workloads are sensitive to any of these items and CPU cache affinity and scheduling latency significantly affect workload performance, kubelet allows alternative CPU management policies to determine some placement preferences on the node. The CPU manager preferentially allocates resources on a socket and full physical cores to avoid interference. + +Binding CPU Cores to a Pod +-------------------------- + +Prerequisites: + +- The static core binding policy is enabled on the node. For details, see :ref:`Enabling the CPU Management Policy `. +- Both **requests** and **limits** must be set in the pod definition and their values must be the same. +- The value of **requests** must be an integer for the container. +- For an init container, it is recommended that you set its **requests** to the same as that of the service container. Otherwise, the service container does not inherit the CPU allocation result of the init container, and the CPU manager reserves more CPU resources than supposed. For more information, see `App Containers can't inherit Init Containers CPUs - CPU Manager Static Policy `__. + +You can use :ref:`Scheduling Policy (Affinity/Anti-affinity) ` to schedule the configured pods to the nodes where the static CPU policy is enabled. In this way, cores can be bound. + +.. _cce_10_0351__section173918176434: + +Enabling the CPU Management Policy +---------------------------------- + +A `CPU management policy `__ is specified by the kubelet flag **--cpu-manager-policy**. The following policies are supported: + +- Disabled (**none**): the default policy. The **none** policy explicitly enables the existing default CPU affinity scheme, providing no affinity beyond what the OS scheduler does automatically. +- Enabled (**static**): The **static** policy allows containers in **Guaranteed** pods with integer CPU requests to be granted increased CPU affinity and exclusivity on the node. + +When creating a cluster, you can configure the CPU management policy in **Advanced Settings**, as shown in the following figure. + +|image1| + +You can also configure the policy in a node pool. The configuration will change the kubelet flag **--cpu-manager-policy** on the node. Log in to the CCE console, click the cluster name, access the cluster details page, and choose **Nodes** in the navigation pane. On the page displayed, click the **Node Pools** tab. Choose **More** > **Manage** in the **Operation** column of the target node pool, and change the value of **cpu-manager-policy** to **static**. + +Pod Configuration +----------------- + +For CPU, both **requests** and **limits** must be set to the same, and **requests** must be an integer. + +.. code-block:: + + kind: Deployment + apiVersion: apps/v1 + metadata: + name: test + spec: + replicas: 1 + selector: + matchLabels: + app: test + template: + metadata: + labels: + app: test + spec: + containers: + - name: container-1 + image: nginx:alpine + resources: + requests: + cpu: 2 # The value must be an integer and must be the same as that in limits. + memory: 2048Mi + limits: + cpu: 2 # The value must be an integer and must be the same as that in requests. + memory: 2048Mi + imagePullSecrets: + - name: default-secret + +.. |image1| image:: /_static/images/en-us_image_0000001244261055.png diff --git a/umn/source/workloads/cpu_core_binding/index.rst b/umn/source/workloads/cpu_core_binding/index.rst new file mode 100644 index 0000000..05441bd --- /dev/null +++ b/umn/source/workloads/cpu_core_binding/index.rst @@ -0,0 +1,14 @@ +:original_name: cce_10_0551.html + +.. _cce_10_0551: + +CPU Core Binding +================ + +- :ref:`Binding CPU Cores ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + binding_cpu_cores diff --git a/umn/source/workloads/creating_a_cron_job.rst b/umn/source/workloads/creating_a_cron_job.rst index c3bad14..1fb71f1 100644 --- a/umn/source/workloads/creating_a_cron_job.rst +++ b/umn/source/workloads/creating_a_cron_job.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0151.html +:original_name: cce_10_0151.html -.. _cce_01_0151: +.. _cce_10_0151: Creating a Cron Job =================== @@ -23,136 +23,76 @@ The typical usage of a cron job is as follows: Prerequisites ------------- -Resources have been created. For details, see :ref:`Creating a Node `. If clusters and nodes are available, you need not create them again. +Resources have been created. For details, see :ref:`Creating a Node `. -Procedure ---------- +Using the CCE Console +--------------------- -#. (Optional) If you use a private container image to create your cron job, upload the container image to the image repository. +#. Log in to the CCE console. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Cron Jobs**. Then, click **Create Cron Job**. +#. Click the cluster name to go to the cluster console, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. -#. Configure the basic cron job information listed in :ref:`Table 1 `. The parameters marked with an asterisk (*) are mandatory. +#. Set basic information about the workload. - .. _cce_01_0151__tfadd3a11520b424d9063fe347c9c8c46: + **Basic Info** - .. table:: **Table 1** Basic cron job information + - **Workload Type**: Select **Cron Job**. For details about workload types, see :ref:`Overview `. + - **Workload Name**: Enter the name of the workload. + - **Namespace**: Select the namespace of the workload. The default value is **default**. You can also click **Create Namespace** to create one. For details, see :ref:`Creating a Namespace `. + - **Container Runtime**: A CCE cluster uses runC by default, whereas a CCE Turbo cluster supports both runC and Kata. For details about the differences between runC and Kata, see :ref:`Kata Containers and Common Containers `. - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===========================================================================================================================================================+ - | \* Job Name | Name of a new cron job. The name must be unique. | - | | | - | | Enter 4 to 52 characters starting with a lowercase letter and ending with a letter or digit. Only lowercase letters, digits, and hyphens (-) are allowed. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Cluster | Cluster to which a new cron job belongs. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Namespace | Namespace to which a cron job belongs. If you do not specify this parameter, the value **default** is used by default. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Description | Description of a cron job. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------+ + **Container Settings** -#. Click **Next: Configure Timing Rule**. + - Container Information -#. Set the timing rule. + Multiple containers can be configured in a pod. You can click **Add Container** on the right to configure multiple containers for the pod. - .. table:: **Table 2** Timing rule parameters + - **Basic Info**: See :ref:`Setting Basic Container Information `. + - **Lifecycle**: See :ref:`Setting Container Lifecycle Parameters `. + - **Environment Variables**: See :ref:`Setting an Environment Variable `. - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===========================================================================================================================================================================+ - | \* Concurrency Policy | The following policies are supported: | - | | | - | | - Forbid: A new job cannot be created before the previous job is complete. | - | | - Allow: The cron job allows concurrently running jobs, which preempt cluster resources. | - | | - Replace: A new job replaces the previous job when it is time to create the job but the previous job is not complete. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Schedule | Time when a new cron job is executed. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Job Records | You can set the number of jobs that are successfully executed or fail to be executed. Setting a limit to **0** corresponds to keeping none of the jobs after they finish. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Image Access Credential**: Select the credential used for accessing the image repository. The default value is **default-secret**. You can use default-secret to access images in SWR. For details about **default-secret**, see :ref:`default-secret `. -#. Click **Next: Add Container** to add a container. + - **GPU graphics card**: **All** is selected by default. The workload instance will be scheduled to the node with the specified GPU graphics card type. - a. Click **Select Container Image** to select the image to be deployed. + **Schedule** - - **My Images**: displays all image repositories you created. - - **Third-Party Images**: Create a job using an image from any third-party image repository. When you create a job using a third-party image, ensure that the node where the job is running can access public networks. For details about how to use a third-party image, see :ref:`Using a Third-Party Image `. + - **Concurrency Policy**: The following three modes are supported: - - If your image repository does not require authentication, set **Secret Authentication** to **No**, enter an image address in **Image Address**, and then click **OK**. - - If your image repository must be authenticated (account and password), you need to create a secret and then use a third-party image. For details, see :ref:`Using a Third-Party Image `. + - **Forbid**: A new job cannot be created before the previous job is completed. + - **Allow**: The cron job allows concurrently running jobs, which preempt cluster resources. + - **Replace**: A new job replaces the previous job when it is time to create a job but the previous job is not completed. - - **Shared Images**: The images shared by other tenants using the SWR service are displayed here. You can create workloads based on the shared images. + - **Policy Settings**: specifies when a new cron job is executed. Policy settings in YAML are implemented using cron expressions. - b. Set image parameters. + - A cron job is executed at a fixed interval. The unit can be minute, hour, day, or month. For example, if a cron job is executed every 30 minutes, the cron expression is **\*/30 \* \* \* \***, the execution time starts from 0 in the unit range, for example, **00:00:00**, **00:30:00**, **01:00:00**, and **...**. + - The cron job is executed at a fixed time (by month). For example, if a cron job is executed at 00:00 on the first day of each month, the cron expression is **0 0 1 \*/1 \***, and the execution time is **\****-01-01 00:00:00**, **\****-02-01 00:00:00**, and **...**. + - The cron job is executed at a fixed time (by week). For example, if a cron job is executed at 00:00 every Monday, the cron expression is **0 0 \* \* 1**, and the execution time is **\****-**-01 00:00:00 on Monday**, **\****-**-08 00:00:00 on Monday**, and **...**. + - For details about how to use cron expressions, see `cron `__. - .. table:: **Table 3** Image parameters + .. note:: - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================================+ - | Image | Name of the image. You can click **Change Image** to update it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Image Version | Select the image tag to be deployed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Container Name | Name of the container. You can modify it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Resources | **CPU** | - | | | - | | - **Request**: minimum number of CPU cores required by a container. The default value is 0.25 cores. | - | | - **Limit**: maximum number of CPU cores available for a container. Do not leave **Limit** unspecified. Otherwise, intensive use of container resources will occur and your workload may exhibit unexpected behavior. | - | | | - | | **Memory** | - | | | - | | - **Request**: minimum amount of memory required by a container. The default value is 0.5 GiB. | - | | - **Limit**: maximum amount of memory available for a container. When memory usage exceeds the specified memory limit, the container will be terminated. | - | | | - | | For more information about **Request** and **Limit**, see :ref:`Setting Container Specifications `. | - | | | - | | **GPU**: configurable only when the cluster contains GPU nodes. | - | | | - | | It indicates the percentage of GPU resources reserved for a container. Select **Use** and set the percentage. For example, if this parameter is set to 10%, the container is allowed to use 10% of GPU resources. If you do not select **Use** or set this parameter to **0**, no GPU resources can be used. | - | | | - | | **GPU/Graphics Card**: The workload's pods will be scheduled to the node with the specified GPU. | - | | | - | | If **Any GPU type** is selected, the container uses a random GPU in the node. If you select a specific GPU, the container uses that GPU accordingly. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - If a cron job is executed at a fixed time (by month) and the number of days in a month does not exist, the cron job will not be executed in this month. For example, if the number of days is set to 30 but February does not have the 30th day, the cron job skips this month and continues on March 30. - c. (Optional) Configure advanced settings. + - Due to the definition of the cron expression, the fixed period is not a strict period. The time unit range is divided from 0 by period. For example, if the unit is minute, the value ranges from 0 to 59. If the value cannot be exactly divided, the last period is reset. Therefore, an accurate period can be represented only when the period can evenly divide its time unit range. - .. table:: **Table 4** Advanced settings + For example, the unit of the period is hour. Because **/2, /3, /4, /6, /8, and /12** can be divided by 24, the accurate period can be represented. If another period is used, the last period will be reset at the beginning of a new day. For example, if the cron expression is **\* \*/12 \* \* \***, the execution time is **00:00:00** and **12:00:00** every day. If the cron expression is **\* \*/13 \* \* \***, the execution time is **00:00:00** and **13:00:00** every day. At 00:00 on the next day, the execution time is updated even if the period does not reach 13 hours. - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===================================================================================================================================================================================================================================================================================+ - | Lifecycle | Actions defined in the lifecycle script definition are taken for the lifecycle events of container tasks. | - | | | - | | - **Start Command**: You can set the command to be executed immediately after the container is started. For details, see :ref:`Configuring a Container `. | - | | - **Post-Start**: The command is triggered after a job starts. For details, see :ref:`Setting Container Lifecycle Parameters `. | - | | - **Pre-Stop**: The command is triggered before a job is stopped. For details, see :ref:`Setting Container Lifecycle Parameters `. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Environment Variables | Environment variables can be added to a container. In general, environment variables are used to set parameters. On the **Environment Variables** tab page, click **Add Environment Variable**. Currently, environment variables can be added using any of the following methods: | - | | | - | | - **Added manually**: Set **Variable Name** and **Variable Value/Reference**. | - | | - **Added from Secret**: Set **Variable Name** and select the desired secret name and data. A secret must be created in advance. For details, see :ref:`Creating a Secret `. | - | | - **Added from ConfigMap**: Set **Variable Name** and select the desired ConfigMap name and data. A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Job Records**: You can set the number of jobs that are successfully executed or fail to be executed. Setting a limit to **0** corresponds to keeping none of the jobs after they finish. - d. (Optional) One job pod contains one or more related containers. If your cron job contains multiple containers, click **Add Container** to add containers. + **Advanced Settings** -#. Click **Create**. + - **Labels and Annotations**: See :ref:`Pod Labels and Annotations `. - If the status is **Started**, the cron job has been created successfully. - -.. _cce_01_0151__section13519162224919: +#. Click **Create Workload** in the lower right corner. Using kubectl ------------- A cron job has the following configuration parameters: -- **.spec.schedule**: takes a `Cron `__ format string, for example, **0 \* \* \* \*** or **@hourly**, as schedule time of jobs to be created and executed. -- **.spec.jobTemplate**: specifies jobs to be run, and has the same schema as when you are :ref:`Creating a Job Using kubectl `. +- **.spec.schedule**: takes a `Cron `__ format string, for example, **0 \* \* \* \*** or **@hourly**, as schedule time of jobs to be created and executed. +- **.spec.jobTemplate**: specifies jobs to be run, and has the same schema as when you are :ref:`Creating a Job Using kubectl `. - **.spec.startingDeadlineSeconds**: specifies the deadline for starting a job. - **.spec.concurrencyPolicy**: specifies how to treat concurrent executions of a job created by the Cron job. The following options are supported: @@ -239,23 +179,23 @@ The following is an example cron job, which is saved in the **cronjob.yaml** fil Related Operations ------------------ -After a cron job is created, you can perform operations listed in :ref:`Table 5 `. +After a cron job is created, you can perform operations listed in :ref:`Table 1 `. -.. _cce_01_0151__t6d520710097a4ee098eae42bcb508608: +.. _cce_10_0151__t6d520710097a4ee098eae42bcb508608: -.. table:: **Table 5** Other operations +.. table:: **Table 1** Other operations +-----------------------------------+----------------------------------------------------------------------------------------------------+ | Operation | Description | +===================================+====================================================================================================+ - | Editing a YAML file | Click **More** > **View YAML** next to the cron job name to view the YAML file of the current job. | + | Editing a YAML file | Click **More** > **Edit YAML** next to the cron job name to edit the YAML file of the current job. | +-----------------------------------+----------------------------------------------------------------------------------------------------+ | Stopping a cron job | #. Select the job to be stopped and click **Stop** in the **Operation** column. | - | | #. Click **OK**. | + | | #. Click **Yes**. | +-----------------------------------+----------------------------------------------------------------------------------------------------+ | Deleting a cron job | #. Select the cron job to be deleted and click **More** > **Delete** in the **Operation** column. | | | | - | | #. Click **OK**. | + | | #. Click **Yes**. | | | | | | Deleted jobs cannot be restored. Therefore, exercise caution when deleting a job. | +-----------------------------------+----------------------------------------------------------------------------------------------------+ diff --git a/umn/source/workloads/creating_a_daemonset.rst b/umn/source/workloads/creating_a_daemonset.rst index 015e425..c448262 100644 --- a/umn/source/workloads/creating_a_daemonset.rst +++ b/umn/source/workloads/creating_a_daemonset.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0216.html +:original_name: cce_10_0216.html -.. _cce_01_0216: +.. _cce_10_0216: Creating a DaemonSet ==================== @@ -23,173 +23,131 @@ You can deploy a DaemonSet for each type of daemons on all nodes, or deploy mult Prerequisites ------------- -You must have one cluster available before creating a DaemonSet. For details on how to create a cluster, see :ref:`Creating a CCE Cluster `. +You must have one cluster available before creating a DaemonSet. For details on how to create a cluster, see :ref:`Creating a CCE Cluster `. -Procedure ---------- +Using the CCE Console +--------------------- #. Log in to the CCE console. -#. In the navigation pane on the left, choose **Workloads** > **DaemonSets**. Click **Create DaemonSet** in the upper right corner of the page. Set basic workload parameters as described in :ref:`Table 1 `. The parameters marked with an asterisk (*) are mandatory. +#. Click the cluster name to go to the cluster console, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. - .. _cce_01_0216__table18511927357: +#. Set basic information about the workload. - .. table:: **Table 1** Basic workload parameters + **Basic Info** - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===========================================================================================================================================================================================================================================+ - | \* Workload Name | Name of the containerized workload to be created. The name must be unique. | - | | | - | | Enter 4 to 63 characters starting with a letter and ending with a letter or digit. Only lowercase letters, digits, and hyphens (-) are allowed. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Cluster Name | Cluster to which the workload belongs. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Namespace | In a single cluster, data in different namespaces is isolated from each other. This enables applications to share the services of the same cluster without interfering each other. If no namespace is set, the default namespace is used. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Time Zone Synchronization | If this parameter is enabled, the container and the node use the same time zone. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | After time zone synchronization is enabled, disks of the hostPath type will be automatically added and listed in the **Data Storage** > **Local Volume** area. Do not modify or delete the disks. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Description | Description of the workload. | - +-----------------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Workload Type**: Select **DaemonSet**. For details about workload types, see :ref:`Overview `. + - **Workload Name**: Enter the name of the workload. + - **Namespace**: Select the namespace of the workload. The default value is **default**. You can also click **Create Namespace** to create one. For details, see :ref:`Creating a Namespace `. + - **Container Runtime**: A CCE cluster uses runC by default, whereas a CCE Turbo cluster supports both runC and Kata. For details about the differences between runC and Kata, see :ref:`Kata Containers and Common Containers `. + - **Time Zone Synchronization**: Specify whether to enable time zone synchronization. After time zone synchronization is enabled, the container and node use the same time zone. The time zone synchronization function depends on the local disk mounted to the container. Do not modify or delete the time zone. For details, see :ref:`Configuring Time Zone Synchronization `. -#. Click **Next: Add Container**. + **Container Settings** - a. Click **Add Container** and select the image to be deployed. + - Container Information - - **My Images**: Create a workload using an image in the image repository you created. - - **Third-Party Images**: Create a workload using an image from any third-party image repository. When you create a workload using a third-party image, ensure that the node where the workload is running can access public networks. For details on how to create a workload using a third-party image, see :ref:`Using a Third-Party Image `. + Multiple containers can be configured in a pod. You can click **Add Container** on the right to configure multiple containers for the pod. - - If your image repository does not require authentication, set **Secret Authentication** to **No**, enter an image pull address, and then click **OK**. - - If your image repository must be authenticated (account and password), you need to create a secret and then use a third-party image. For details, see :ref:`Using a Third-Party Image `. - - - **Shared Images**: Create a workload using an image shared by another tenant through the SWR service. - - b. Configure basic image information. - - A workload is an abstract model of a group of pods. One pod can encapsulate one or more containers. You can click **Add Container** in the upper right corner to add multiple container images and set them separately. - - .. table:: **Table 2** Image parameters - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================================+ - | Image Name | Name of the image. You can click **Change Image** to update it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Image Version | Select the image tag to be deployed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Container Name | Name of the container. You can modify it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Privileged Container | Programs in a privileged container have certain privileges. | - | | | - | | If **Privileged Container** is **On**, the container is granted superuser permissions. For example, privileged containers can manipulate network devices on the host machine and modify kernel parameters. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Resources | **CPU** | - | | | - | | - **Request**: minimum number of CPU cores required by a container. The default value is 0.25 cores. | - | | - **Limit**: maximum number of CPU cores available for a container. Do not leave **Limit** unspecified. Otherwise, intensive use of container resources will occur and your workload may exhibit unexpected behavior. | - | | | - | | **Memory** | - | | | - | | - **Request**: minimum amount of memory required by a container. The default value is 512 MiB. | - | | - **Limit**: maximum amount of memory available for a container. When memory usage exceeds the specified memory limit, the container will be terminated. | - | | | - | | For more information about **Request** and **Limit**, see :ref:`Setting Container Specifications `. | - | | | - | | **GPU**: configurable only when the cluster contains GPU nodes. | - | | | - | | It indicates the percentage of GPU resources reserved for a container. Select **Use** and set the percentage. For example, if this parameter is set to 10%, the container is allowed to use 10% of GPU resources. If you do not select **Use** or set this parameter to **0**, no GPU resources can be used. | - | | | - | | **GPU/Graphics Card**: The workload's pods will be scheduled to the node with the specified GPU. | - | | | - | | If **Any GPU type** is selected, the container uses a random GPU in the node. If you select a specific GPU, the container uses this GPU accordingly. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - c. **Lifecycle**: Commands for starting and running containers can be set. - - - **Start Command**: executed when the workload is started. For details, see :ref:`Setting Container Startup Commands `. - - **Post-Start**: executed after the workload runs successfully. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - **Pre-Stop**: executed to delete logs or temporary files before the workload ends. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - d. **Health Check**: CCE provides two types of probes: liveness probe and readiness probe. They are used to determine whether containers and user services are running properly. For more information, see :ref:`Setting Health Check for a Container `. - - - **Liveness Probe**: used to restart the unhealthy container. - - **Readiness Probe**: used to change the container to the unready state when detecting that the container is unhealthy. In this way, service traffic will not be directed to the container. - - e. **Environment Variables**: Environment variables can be added to a container. In general, environment variables are used to set parameters. - - On the **Environment Variables** tab page, click **Add Environment Variable**. Currently, three types of environment variables are supported: - - - **Added manually**: Set **Variable Name** and **Variable Value/Reference**. - - **Added from Secret**: Set **Variable Name** and select the desired secret name and data. A secret must be created in advance. For details, see :ref:`Creating a Secret `. - - **Added from ConfigMap**: Set **Variable Name** and select the desired ConfigMap name and data. A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. + - **Basic Info**: See :ref:`Setting Basic Container Information `. + - **Lifecycle**: See :ref:`Setting Container Lifecycle Parameters `. + - **Health Check**: See :ref:`Setting Health Check for a Container `. + - **Environment Variables**: See :ref:`Setting an Environment Variable `. + - **Data Storage**: See :ref:`Overview `. .. note:: - To edit an environment variable that has been set, click **Edit**. To delete an environment variable that has been set, click **Delete**. + If the workload contains more than one pod, EVS volumes cannot be mounted. - f. **Data Storage**: Data storage can be mounted to containers for persistent storage and high disk I/O. Local volume and cloud storage are supported. For details, see :ref:`Storage (CSI) `. + - **Security Context**: Set container permissions to protect the system and other containers from being affected. Enter the user ID to set container permissions and prevent systems and other containers from being affected. + - **Logging**: See :ref:`Using ICAgent to Collect Container Logs `. - .. note:: + - **Image Access Credential**: Select the credential used for accessing the image repository. The default value is **default-secret**. You can use default-secret to access images in SWR. For details about **default-secret**, see :ref:`default-secret `. - Currently, cloud storage cannot be mounted to secure (Kata) containers in a CCE Turbo cluster. + - **GPU graphics card**: **All** is selected by default. The workload instance will be scheduled to the node with the specified GPU graphics card type. - g. **Security Context**: Container permissions can be configured to protect CCE and other containers from being affected. + **Service Settings** - Enter the user ID to set container permissions and prevent systems and other containers from being affected. + A Service is used for pod access. With a fixed IP address, a Service forwards access traffic to pods and performs load balancing for these pods. - h. **Log Policies**: Log collection policies and log directory can be configured to collect container logs for unified management and analysis. For details, see :ref:`Container Logs `. + You can also create a Service after creating a workload. For details about the Service, see :ref:`Service Overview `. -#. Click **Next: Set Application Access**. Then, click **Add Service** and set the workload access type. + **Advanced Settings** - If your workload will be reachable to other workloads or public networks, add a Service to define the workload access type. + - **Upgrade**: See :ref:`Configuring the Workload Upgrade Policy `. + - **Scheduling**: See :ref:`Scheduling Policy (Affinity/Anti-affinity) `. + - **Labels and Annotations**: See :ref:`Pod Labels and Annotations `. + - **Toleration**: Using both taints and tolerations allows (not forcibly) the pod to be scheduled to a node with the matching taints, and controls the pod eviction policies after the node where the pod is located is tainted. For details, see :ref:`Tolerations `. + - **DNS**: See :ref:`DNS Configuration `. - The workload access type determines the network attributes of the workload. Workloads with different access types can provide different network capabilities. For details, see :ref:`Overview `. +#. Click **Create Workload** in the lower right corner. -#. Click **Next: Configure Advanced Settings** to configure advanced policies. +Using kubectl +------------- - - **Upgrade Policy**: +The following procedure uses Nginx as an example to describe how to create a workload using kubectl. - - **Upgrade Mode**: Only **Rolling upgrade** is supported. During a rolling upgrade, old pods are gradually replaced with new ones. During the upgrade, service traffic is evenly distributed to both pods to ensure service continuity. - - **Maximum Number of Unavailable Pods**: Maximum number of unavailable pods allowed in a rolling upgrade. If the number is equal to the total number of pods, services may be interrupted. Minimum number of alive pods = Total pods - Maximum number of unavailable pods +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. - - **Graceful Deletion**: +#. Create and edit the **nginx-daemonset.yaml** file. **nginx-daemonset.yaml** is an example file name, and you can change it as required. - **Graceful Time Window**: Enter the time. The graceful scale-in policy provides a time window for workload deletion and is reserved for executing commands in the PreStop phase in the lifecycle. If workload processes are not terminated after the time window elapses, the workload will be forcibly deleted. + **vi nginx-daemonset.yaml** - - **Scheduling Policies**: You can combine static global scheduling policies or dynamic runtime scheduling policies as required. For details, see :ref:`Scheduling Policy Overview `. + The content of the description file is as follows: The following provides an example. For more information on DaemonSets, see `Kubernetes documents `__. - - **Advanced Pod Settings** + .. code-block:: - - **Pod Label**: The built-in **app** label is specified when the workload is created. It is used to set affinity and anti-affinity scheduling and cannot be modified. You can click **Add Label** to add labels. + apiVersion: apps/v1 + kind: DaemonSet + metadata: + name: nginx-daemonset + labels: + app: nginx-daemonset + spec: + selector: + matchLabels: + app: nginx-daemonset + template: + metadata: + labels: + app: nginx-daemonset + spec: + nodeSelector: # Node selection. A pod is created on a node only when the node meets daemon=need. + daemon: need + containers: + - name: nginx-daemonset + image: nginx:alpine + resources: + limits: + cpu: 250m + memory: 512Mi + requests: + cpu: 250m + memory: 512Mi + imagePullSecrets: + - name: default-secret + The **replicas** parameter used in defining a Deployment or StatefulSet does not exist in the above configuration for a DaemonSet, because each node has only one replica. It is fixed. - .. figure:: /_static/images/en-us_image_0220765374.png - :alt: **Figure 1** Advanced pod settings + The nodeSelector in the preceding pod template specifies that a pod is created only on the nodes that meet **daemon=need**, as shown in the following figure. If you want to create a pod on each node, delete the label. - **Figure 1** Advanced pod settings +#. Create a DaemonSet. - - **Client DNS Configuration**: A CCE cluster has a built-in DNS add-on (CoreDNS) to provide domain name resolution for workloads in the cluster. + **kubectl create -f nginx-daemonset.yaml** - - **DNS Policy** + If the following information is displayed, the DaemonSet is being created. - - **ClusterFirst**: The default DNS configuration overrides the **Nameserver** and **DNS Search Domain** configurations of the client. - - **None**: Only the **Nameserver** and **DNS Search Domain** configurations are used for domain name resolution. - - **Default**: The pod inherits the DNS configuration from the node on which the pod runs. + .. code-block:: - - **Nameserver**: You can configure a domain name server for a user-defined domain name. The value is one or a group of DNS IP addresses, for example, 1.2.3.4. - - **DNS Search Domain**: a search list for host-name lookup. When a domain name cannot be resolved, DNS queries will be attempted combining the domain name with each domain in the search list in turn until a match is found or all domains in the search list are tried. - - **Timeout (s)**: amount of time the resolver will wait for a response from a remote name server before retrying the query on a different name server. Set it based on the site requirements. - - **ndots**: threshold for the number of dots that must appear in a domain name before an initial absolute query will be made. If a domain name has **ndots** or more than **ndots** dots, the name is a fully qualified domain name (FQDN) and will be tried first as an absolute name. If a domain name has less than **ndots** dots, the operating system will look up the name in a list of search domain names. + daemonset.apps/nginx-daemonset created -#. After the preceding configurations are complete, click **Create**. On the page displayed, click **Return to Workload List** to view the workload status. +#. Query the DaemonSet status. - If the workload is in the **Running** state, it has been successfully created. + **kubectl get ds** - Workload status is not updated in real time. Click |image1| in the upper right corner or press **F5** to refresh the page. + .. code-block:: -.. |image1| image:: /_static/images/en-us_image_0183674977.png + $ kubectl get ds + NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE + nginx-daemonset 1 1 0 1 0 daemon=need 116s + +#. If the workload will be accessed through a ClusterIP or NodePort Service, set the corresponding workload access type. For details, see :ref:`Networking `. diff --git a/umn/source/workloads/creating_a_deployment.rst b/umn/source/workloads/creating_a_deployment.rst index f0d1127..3972467 100644 --- a/umn/source/workloads/creating_a_deployment.rst +++ b/umn/source/workloads/creating_a_deployment.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0047.html +:original_name: cce_10_0047.html -.. _cce_01_0047: +.. _cce_10_0047: Creating a Deployment ===================== @@ -13,7 +13,7 @@ Deployments are workloads (for example, Nginx) that do not store any data or sta Prerequisites ------------- -- Before creating a containerized workload, you must have an available cluster. For details on how to create a cluster, see :ref:`Creating a CCE Cluster `. +- Before creating a containerized workload, you must have an available cluster. For details on how to create a cluster, see :ref:`Creating a CCE Cluster `. - To enable public access to a workload, ensure that an EIP or load balancer has been bound to at least one node in the cluster. .. note:: @@ -23,218 +23,68 @@ Prerequisites Using the CCE Console --------------------- -CCE provides multiple methods for creating a workload. You can use any of the following methods: +#. Log in to the CCE console. -- Use an image in Third-Party Images. You do not need to upload any image before using it. -- Use an image that you have uploaded to SWR. -- Use a **shared image** to create a workload. Specifically, other tenants share an image with you by using the **SWR service**. -- Use a YAML file to create a workload. You can click **Create YAML** on the right of the **Configure Advanced Settings** page when creating a Deployment. For details about YAML, see :ref:`Table 3 `. After the YAML file is written, click **Create** to create a workload. +#. Click the cluster name to access the cluster details page, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. - .. note:: +#. Set basic information about the workload. - Settings in the YAML file are synchronized with those on the console. You can edit the YAML file on the console to create a workload. For example: + **Basic Info** - - If you enter a workload name on the console, the name will automatically appear in the YAML file. - - If you add an image on the console, the image will be automatically added to the YAML file. + - **Workload Type**: Select **Deployment**. For details about workload types, see :ref:`Overview `. + - **Workload Name**: Enter the name of the workload. + - **Namespace**: Select the namespace of the workload. The default value is **default**. You can also click **Create Namespace** to create one. For details, see :ref:`Creating a Namespace `. + - **Pods**: Enter the number of pods. + - **Container Runtime**: A CCE cluster uses runC by default, whereas a CCE Turbo cluster supports both runC and Kata. For details about the differences between runC and Kata, see :ref:`Kata Containers and Common Containers `. + - **Time Zone Synchronization**: Specify whether to enable time zone synchronization. After time zone synchronization is enabled, the container and node use the same time zone. The time zone synchronization function depends on the local disk mounted to the container. Do not modify or delete the time zone. For details, see :ref:`Configuring Time Zone Synchronization `. - When you click **Create YAML** on the right of the console, do not create multiple YAML files in the YAML definition pane displayed. You need to create them one by one. Otherwise, an error will be reported during the creation. + **Container Settings** -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. On the page displayed, click **Create Deployment**. Set basic workload parameters as described in :ref:`Table 1 `. The parameters marked with an asterisk (*) are mandatory. + - Container Information - .. _cce_01_0047__table12741447488: + Multiple containers can be configured in a pod. You can click **Add Container** on the right to configure multiple containers for the pod. - .. table:: **Table 1** Basic workload parameters - - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+======================================================================================================================================================================================================================================================+ - | \* Workload Name | Name of the workload to be created. The name must be unique. | - | | | - | | Enter 4 to 63 characters starting with a letter and ending with a letter or digit. Only lowercase letters, digits, and hyphens (-) are allowed. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Cluster Name | Cluster to which the workload belongs. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Namespace | In a single cluster, data in different namespaces is isolated from each other. This enables applications to share the services of the same cluster without interfering each other. If no namespace is set, the default namespace is used. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Instances | Number of pods in the workload. A workload can have one or more pods. You can set the number of pods. The default value is **2** and can be set to **1**. | - | | | - | | Each workload pod consists of the same containers. Configuring multiple pods for a workload ensures that the workload can still run properly even if a pod is faulty. If only one pod is used, a node or pod exception may cause service exceptions. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Container runtime | Select a container runtime, which cannot be changed after creation. **This parameter is available only for CCE Turbo clusters.** | - | | | - | | - **runc**: Common containers will run on the node. | - | | - **kata**: Secure containers will be used and the workload can run only on the node that uses the secure runtime. | - | | | - | | For details about common containers and secure containers, see :ref:`Secure Containers and Common Containers `. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Time Zone Synchronization | If this parameter is enabled, the container and the node use the same time zone. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | After time zone synchronization is enabled, disks of the hostPath type will be automatically added and listed in the **Data Storage** > **Local Volume** area. Do not modify or delete the disks. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Description | Description of the workload. | - +-----------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **Next: Add Container**. - - a. Click **Add Container** and select the image to be deployed. - - - **My Images**: Create a workload using an image in the image repository you created. - - **Third-Party Images**: Create a workload using an image from any third-party image repository. When you create a workload using a third-party image, ensure that the node where the workload is running can access public networks. For details on how to create a workload using a third-party image, see :ref:`Using a Third-Party Image `. - - - If your image repository does not require authentication, set **Secret Authentication** to **No**, enter an image pull address, and then click **OK**. - - If your image repository must be authenticated (account and password), you need to create a secret and then use a third-party image. For details, see :ref:`Using a Third-Party Image `. - - - **Shared Images**: Create a workload using an image shared by another tenant through the SWR service. - - b. Configure basic image information. - - A workload is an abstract model of a group of pods. One pod can encapsulate one or more containers. You can click **Add Container** in the upper right corner to add multiple container images and set them separately. - - .. table:: **Table 2** Image parameters - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================================+ - | Image Name | Name of the image. You can click **Change Image** to update it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Image Version | Select the image tag to be deployed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Container Name | Name of the container. You can modify it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Privileged Container | Programs in a privileged container have certain privileges. | - | | | - | | If **Privileged Container** is **On**, the container is granted superuser permissions. For example, privileged containers can manipulate network devices on the host machine and modify kernel parameters. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Resources | **CPU** | - | | | - | | - **Request**: minimum number of CPU cores required by a container. The default value is 0.25 cores. | - | | - **Limit**: maximum number of CPU cores available for a container. Do not leave **Limit** unspecified. Otherwise, intensive use of container resources will occur and your workload may exhibit unexpected behavior. | - | | | - | | **Memory** | - | | | - | | - **Request**: minimum amount of memory required by a container. The default value is 512 MiB. | - | | - **Limit**: maximum amount of memory available for a container. When memory usage exceeds the specified memory limit, the container will be terminated. | - | | | - | | For more information about **Request** and **Limit**, see :ref:`Setting Container Specifications `. | - | | | - | | **GPU**: configurable only when the cluster contains GPU nodes. | - | | | - | | It indicates the percentage of GPU resources reserved for a container. Select **Use** and set the percentage. For example, if this parameter is set to 10%, the container is allowed to use 10% of GPU resources. If you do not select **Use** or set this parameter to **0**, no GPU resources can be used. | - | | | - | | **GPU/Graphics Card**: The workload's pods will be scheduled to the node with the specified GPU. | - | | | - | | If **Any GPU type** is selected, the container uses a random GPU in the node. If you select a specific GPU, the container uses this GPU accordingly. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - c. **Lifecycle**: Commands for starting and running containers can be set. - - - **Start Command**: executed when the workload is started. For details, see :ref:`Setting Container Startup Commands `. - - **Post-Start**: executed after the workload runs successfully. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - **Pre-Stop**: executed to delete logs or temporary files before the workload ends. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - d. **Health Check**: CCE provides two types of probes: liveness probe and readiness probe. They are used to determine whether containers and user services are running properly. For more information, see :ref:`Setting Health Check for a Container `. - - - **Liveness Probe**: used to restart the unhealthy container. - - **Readiness Probe**: used to change the container to the unready state when detecting that the container is unhealthy. In this way, service traffic will not be directed to the container. - - e. **Environment Variables**: Environment variables can be added to a container. In general, environment variables are used to set parameters. - - On the **Environment Variables** tab page, click **Add Environment Variable**. Currently, three types of environment variables are supported: - - - **Added manually**: Set **Variable Name** and **Variable Value/Reference**. - - **Added from Secret**: Set **Variable Name** and select the desired secret name and data. A secret must be created in advance. For details, see :ref:`Creating a Secret `. - - **Added from ConfigMap**: Set **Variable Name** and select the desired ConfigMap name and data. A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. + - **Basic Info**: See :ref:`Setting Basic Container Information `. + - **Lifecycle**: See :ref:`Setting Container Lifecycle Parameters `. + - **Health Check**: See :ref:`Setting Health Check for a Container `. + - **Environment Variables**: See :ref:`Setting an Environment Variable `. + - **Data Storage**: See :ref:`Overview `. .. note:: - To edit an environment variable that has been set, click **Edit**. To delete an environment variable that has been set, click **Delete**. + If the workload contains more than one pod, EVS volumes cannot be mounted. - f. **Data Storage**: Data storage can be mounted to containers for persistent storage and high disk I/O. Local volume and cloud storage are supported. For details, see :ref:`Storage (CSI) `. + - **Security Context**: Set container permissions to protect the system and other containers from being affected. Enter the user ID to set container permissions and prevent systems and other containers from being affected. + - **Logging**: See :ref:`Using ICAgent to Collect Container Logs `. - .. note:: + - **Image Access Credential**: Select the credential used for accessing the image repository. The default value is **default-secret**. You can use default-secret to access images in SWR. For details about **default-secret**, see :ref:`default-secret `. - Currently, cloud storage cannot be mounted to secure (Kata) containers in a CCE Turbo cluster. + - **GPU graphics card**: **All** is selected by default. The workload instance will be scheduled to the node with the specified GPU graphics card type. - g. **Security Context**: Container permissions can be configured to protect CCE and other containers from being affected. + **Service Settings** - Enter the user ID to set container permissions and prevent systems and other containers from being affected. + A Service is used for pod access. With a fixed IP address, a Service forwards access traffic to pods and performs load balancing for these pods. - h. **Log Policies**: Log collection policies and log directory can be configured to collect container logs for unified management and analysis. For details, see :ref:`Container Logs `. + You can also create a Service after creating a workload. For details about the Service, see :ref:`Service Overview `. -#. Click **Next: Set Application Access**. Then, click **Add Service** and set the workload access type. + **Advanced Settings** - If your workload will be reachable to other workloads or public networks, add a Service to define the workload access type. + - **Upgrade**: See :ref:`Configuring the Workload Upgrade Policy `. + - **Scheduling**: See :ref:`Scheduling Policy (Affinity/Anti-affinity) `. + - **Labels and Annotations**: See :ref:`Pod Labels and Annotations `. + - **Toleration**: Using both taints and tolerations allows (not forcibly) the pod to be scheduled to a node with the matching taints, and controls the pod eviction policies after the node where the pod is located is tainted. For details, see :ref:`Tolerations `. + - **DNS**: See :ref:`DNS Configuration `. - The workload access type determines the network attributes of the workload. Workloads with different access types can provide different network capabilities. For details, see :ref:`Overview `. +#. Click **Create Workload** in the lower right corner. -#. Click **Next: Configure Advanced Settings** to configure advanced policies. - - - **Upgrade Mode**: You can specify the upgrade mode of a Deployment, including **Rolling upgrade** and **In-place upgrade**. - - - **Rolling upgrade**: Old pods are gradually replaced with new ones. During the upgrade, service traffic is evenly distributed to both pods to ensure service continuity. - - - **Maximum Number of Unavailable Pods**: maximum number of unavailable pods allowed in a rolling upgrade. If the number is equal to the total number of pods, services may be interrupted. Minimum number of alive pods = Total pods - Maximum number of unavailable pods - - - **In-place upgrade**: Old pods are deleted before new pods are created. Services will be interrupted during an in-place upgrade. - - - **Graceful Deletion**: A time window can be set for workload deletion and reserved for executing commands in the pre-stop phase in the lifecycle. If workload processes are not terminated after the time window elapses, the workload will be forcibly deleted. - - - **Graceful Time Window (s)**: Set a time window (0-9999s) for pre-stop commands to finish execution before a workload is deleted. The default value is 30s. - - **Scale Order**: Choose **Prioritize new pods** or **Prioritize old pods** based on service requirements. **Prioritize new pods** indicates that new pods will be first deleted when a scale-in is triggered. - - - **Migration Policy**: When the node where a workload's pods are located is unavailable for the specified amount of time, the pods will be rescheduled to other available nodes. - - - **Migration Time Window (s)**: Set a time window for migration. The default value is 300s. - - - **Scheduling Policies**: You can combine static global scheduling policies or dynamic runtime scheduling policies as required. For details, see :ref:`Scheduling Policy Overview `. - - - **Advanced Pod Settings** - - - **Pod Label**: The built-in **app** label is specified when the workload is created. It is used to set affinity and anti-affinity scheduling and cannot be modified. You can click **Add Label** to add labels. - - - .. figure:: /_static/images/en-us_image_0220765374.png - :alt: **Figure 1** Advanced pod settings - - **Figure 1** Advanced pod settings - - - **Client DNS Configuration**: A CCE cluster has a built-in DNS add-on (CoreDNS) to provide domain name resolution for workloads in the cluster. - - - **DNS Policy** - - - **ClusterFirst**: The default DNS configuration overrides the **Nameserver** and **DNS Search Domain** configurations of the client. - - **None**: Only the **Nameserver** and **DNS Search Domain** configurations are used for domain name resolution. - - **Default**: The pod inherits the DNS configuration from the node on which the pod runs. - - - **Nameserver**: You can configure a domain name server for a user-defined domain name. The value is one or a group of DNS IP addresses, for example, 1.2.3.4. - - **DNS Search Domain**: a search list for host-name lookup. When a domain name cannot be resolved, DNS queries will be attempted combining the domain name with each domain in the search list in turn until a match is found or all domains in the search list are tried. - - **Timeout (s)**: amount of time the resolver will wait for a response from a remote name server before retrying the query on a different name server. Set it based on the site requirements. - - **ndots**: threshold for the number of dots that must appear in a domain name before an initial absolute query will be made. If a domain name has **ndots** or more than **ndots** dots, the name is a fully qualified domain name (FQDN) and will be tried first as an absolute name. If a domain name has less than **ndots** dots, the operating system will look up the name in a list of search domain names. - -#. After the preceding configurations are complete, click **Create**. On the page displayed, click **Return to Workload List** to view the workload status. - - If the workload is in the **Running** state, it has been successfully created. - - Workload status is not updated in real time. Click |image1| in the upper right corner or press **F5** to refresh the page. - -#. To access the workload in a browser, go to the workload list on the **Deployments** page. Copy the corresponding **External Access Address** and paste it into the address box in the browser. - - .. note:: - - - External access addresses are available only if the Deployment access type is set to **NodePort** and an EIP is assigned to any node in the cluster, or if the Deployment access type is set to **LoadBalancer (ELB)**. - - If the workload list contains more than 500 records, the Kubernetes pagination mechanism will be used. Specifically, you can only go to the first page or the next page, but cannot go to the previous page. In addition, if resources are divided into discrete pages, the total number of resources displayed is the maximum number of resources that can be queried at a time, not the actual total number of resources. - -.. _cce_01_0047__section155246177178: +.. _cce_10_0047__section155246177178: Using kubectl ------------- The following procedure uses Nginx as an example to describe how to create a workload using kubectl. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. #. Create and edit the **nginx-deployment.yaml** file. **nginx-deployment.yaml** is an example file name. You can rename it as required. @@ -267,11 +117,11 @@ The following procedure uses Nginx as an example to describe how to create a wor imagePullSecrets: - name: default-secret - For details about these parameters, see :ref:`Table 3 `. + For details about these parameters, see :ref:`Table 1 `. - .. _cce_01_0047__table132326831016: + .. _cce_10_0047__table132326831016: - .. table:: **Table 3** Deployment YAML parameters + .. table:: **Table 1** Deployment YAML parameters +-----------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ | Parameter | Description | Mandatory/Optional | @@ -343,12 +193,10 @@ The following procedure uses Nginx as an example to describe how to create a wor **Parameter description** - - **NAME**: pod name - - **READY**: number of pod replicas that have been deployed - - **STATUS**: status of the Deployment - - **RESTARTS**: restart times + - **NAME**: Name of the application running in the pod. + - **READY**: indicates the number of available workloads. The value is displayed as "the number of available pods/the number of expected pods". + - **UP-TO-DATE**: indicates the number of replicas that have been updated. + - **AVAILABLE**: indicates the number of available pods. - **AGE**: period the Deployment keeps running -#. If the Deployment will be accessed through a ClusterIP or NodePort Service, add the corresponding Service. For details, see :ref:`Networking `. - -.. |image1| image:: /_static/images/en-us_image_0183674977.png +#. If the Deployment will be accessed through a ClusterIP or NodePort Service, add the corresponding Service. For details, see :ref:`Networking `. diff --git a/umn/source/workloads/creating_a_job.rst b/umn/source/workloads/creating_a_job.rst index 6b22af7..1e54800 100644 --- a/umn/source/workloads/creating_a_job.rst +++ b/umn/source/workloads/creating_a_job.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0150.html +:original_name: cce_10_0150.html -.. _cce_01_0150: +.. _cce_10_0150: Creating a Job ============== @@ -21,118 +21,57 @@ A job is started and terminated at specific times, while a long-term servo workl Prerequisites ------------- -Resources have been created. For details, see :ref:`Creating a Node `. If clusters and nodes are available, you need not create them again. +Resources have been created. For details, see :ref:`Creating a Node `. If clusters and nodes are available, you need not create them again. -Procedure ---------- +Using the CCE Console +--------------------- -#. (Optional) If you use a private container image to create your job, upload the container image to the image repository. +#. Log in to the CCE console. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Jobs**. Click **Create Job**. +#. Click the cluster name to go to the cluster console, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. -#. Configure the basic job information listed in :ref:`Table 1 `. The parameters marked with an asterisk (*) are mandatory. +#. Set basic information about the workload. - .. _cce_01_0150__t70ce3a99637a44ce8f7274857fb245b1: + **Basic Info** - .. table:: **Table 1** Basic job information + - **Workload Type**: Select **Job**. For details about workload types, see :ref:`Overview `. + - **Workload Name**: Enter the name of the workload. + - **Namespace**: Select the namespace of the workload. The default value is **default**. You can also click **Create Namespace** to create one. For details, see :ref:`Creating a Namespace `. + - **Pods**: Enter the number of pods. + - **Container Runtime**: A CCE cluster uses runC by default, whereas a CCE Turbo cluster supports both runC and Kata. For details about the differences between runC and Kata, see :ref:`Kata Containers and Common Containers `. - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==========================================================================================================================================================================+ - | \* Job Name | Name of a new job. The name must be unique. | - | | | - | | Enter 4 to 63 characters starting with a lowercase letter and ending with a letter or digit. Only lowercase letters, digits, and hyphens (-) are allowed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Cluster | Cluster to which a new job belongs. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Namespace | Namespace to which the new job belongs. By default, this parameter is set to **default**. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Instances | Number of pods in this job. A job can have one or more pods. You can specify the number of pods. The default value is **1**. | - | | | - | | Each job pod consists of the same containers. Configuring multiple job pods can ensure high availability. The job can continue to run even if one of the pods is faulty. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Description | Description of a job. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + **Container Settings** -#. Click **Next: Add Container** to add a container and an image. + - Container Information - a. Click **Select Container Image** to select the image to be deployed. + Multiple containers can be configured in a pod. You can click **Add Container** on the right to configure multiple containers for the pod. - - **My Images**: displays all image repositories you created. - - **Third-Party Images**: Create a job using an image from any third-party image repository. When you create a job using a third-party image, ensure that the node where the job is running can access public networks. For details about how to use a third-party image, see :ref:`Using a Third-Party Image `. + - **Basic Info**: See :ref:`Setting Basic Container Information `. + - **Lifecycle**: See :ref:`Setting Container Lifecycle Parameters `. + - **Environment Variables**: See :ref:`Setting an Environment Variable `. + - **Data Storage**: See :ref:`Overview `. - - If your image repository does not require authentication, set **Secret Authentication** to **No**, enter an image address in **Image Address**, and then click **OK**. - - If your image repository must be authenticated (account and password), you need to create a secret and then use a third-party image. For details, see :ref:`Using a Third-Party Image `. + .. note:: - - **Shared Images**: The images shared by other tenants using the SWR service are displayed here. You can create workloads based on the shared images. + If the workload contains more than one pod, EVS volumes cannot be mounted. - b. Set image parameters. + - **Logging**: See :ref:`Using ICAgent to Collect Container Logs `. - .. table:: **Table 2** Image parameters + - **Image Access Credential**: Select the credential used for accessing the image repository. The default value is **default-secret**. You can use default-secret to access images in SWR. For details about **default-secret**, see :ref:`default-secret `. - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================================+ - | Image | Name of the image. You can click **Change Image** to update it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Image Version | Select the image tag to be deployed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Container Name | Name of the container. You can modify it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Resources | **CPU** | - | | | - | | - **Request**: minimum number of CPU cores required by a container. The default value is 0.25 cores. | - | | - **Limit**: maximum number of CPU cores available for a container. Do not leave **Limit** unspecified. Otherwise, intensive use of container resources will occur and your workload may exhibit unexpected behavior. | - | | | - | | **Memory** | - | | | - | | - **Request**: minimum amount of memory required by a container. The default value is 0.5 GiB. | - | | - **Limit**: maximum amount of memory available for a container. When memory usage exceeds the specified memory limit, the container will be terminated. | - | | | - | | For more information about **Request** and **Limit**, see :ref:`Setting Container Specifications `. | - | | | - | | **GPU**: configurable only when the cluster contains GPU nodes. | - | | | - | | It indicates the percentage of GPU resources reserved for a container. Select **Use** and set the percentage. For example, if this parameter is set to 10%, the container is allowed to use 10% of GPU resources. If you do not select **Use** or set this parameter to **0**, no GPU resources can be used. | - | | | - | | **GPU/Graphics Card**: The workload's pods will be scheduled to the node with the specified GPU. | - | | | - | | If **Any GPU type** is selected, the container uses a random GPU in the node. If you select a specific GPU, the container uses that GPU accordingly. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **GPU graphics card**: **All** is selected by default. The workload instance will be scheduled to the node with the specified GPU graphics card type. - c. (Optional) Configure advanced settings. + **Advanced Settings** - .. table:: **Table 3** Advanced settings + - **Labels and Annotations**: See :ref:`Pod Labels and Annotations `. + - **Job Settings**: - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===================================================================================================================================================================================================================================================================================+ - | Lifecycle | Lifecycle scripts define the actions taken for container-related jobs when a lifecycle event occurs. | - | | | - | | - **Start Command**: You can set the command to be executed immediately after the container is started. For details, see :ref:`Configuring a Container `. | - | | - **Post-Start**: The command is triggered after a job starts. For details, see :ref:`Setting Container Lifecycle Parameters `. | - | | - **Pre-Stop**: The command is triggered before a job is stopped. For details, see :ref:`Setting Container Lifecycle Parameters `. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Environment Variables | Environment variables can be added to a container. In general, environment variables are used to set parameters. On the **Environment Variables** tab page, click **Add Environment Variable**. Currently, environment variables can be added using any of the following methods: | - | | | - | | - **Added manually**: Set **Variable Name** and **Variable Value/Reference**. | - | | - **Added from Secret**: Set **Variable Name** and select the desired secret name and data. A secret must be created in advance. For details, see :ref:`Creating a Secret `. | - | | - **Added from ConfigMap**: Set **Variable Name** and select the desired ConfigMap name and data. A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Data Storage | The local disk or cloud storage can be mounted to a container to implement persistent data file storage. | - | | | - | | For details, see :ref:`Storage (CSI) `. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Log Policies | Set a log policy and log path for collecting workload logs and preventing logs from being over-sized. For details, see :ref:`Container Logs `. | - +-----------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + - **Parallel Pods**: Maximum number of pods that can run in parallel during job execution. The value cannot be greater than the total number of pods in the job. + - **Timeout (s)**: Once a job reaches this time, the job status becomes failed and all pods in this job will be deleted. If you leave this parameter blank, the job will never time out. - d. (Optional) One job pod contains one or more related containers. If your job contains multiple containers, click **Add Container** to add containers. +#. Click **Create Workload** in the lower right corner. -#. Click **Create**. - - If the status of the job is **Executing**, the job has been created successfully. - -.. _cce_01_0150__section450152719412: +.. _cce_10_0150__section450152719412: Using kubectl ------------- @@ -149,7 +88,7 @@ A job has the following configuration parameters: Based on the **.spec.completions** and **.spec.Parallelism** settings, jobs are classified into the following types. -.. table:: **Table 4** Job types +.. table:: **Table 1** Job types +---------------------------------------------+-----------------------------------------------------------------------+-------------------------------------------------------+ | Job Type | Description | Example | @@ -235,20 +174,20 @@ The following is an example job, which calculates Pi till the 2000\ :sup:`th` di Related Operations ------------------ -After a one-off job is created, you can perform operations listed in :ref:`Table 5 `. +After a one-off job is created, you can perform operations listed in :ref:`Table 2 `. -.. _cce_01_0150__t84075653e7544394939d13740fad0c20: +.. _cce_10_0150__t84075653e7544394939d13740fad0c20: -.. table:: **Table 5** Other operations +.. table:: **Table 2** Other operations - +-----------------------------------+--------------------------------------------------------------------------------------------------+ - | Operation | Description | - +===================================+==================================================================================================+ - | Viewing a YAML | Click **View YAML** next to the job name to view the YAML file corresponding to the current job. | - +-----------------------------------+--------------------------------------------------------------------------------------------------+ - | Deleting a one-off job | #. Select the job to be deleted and click **Delete** in the **Operation** column. | - | | | - | | #. Click **OK**. | - | | | - | | Deleted jobs cannot be restored. Exercise caution when deleting a job. | - +-----------------------------------+--------------------------------------------------------------------------------------------------+ + +-----------------------------------+-------------------------------------------------------------------------------------------------------------+ + | Operation | Description | + +===================================+=============================================================================================================+ + | Editing a YAML file | Click **More** > **Edit YAML** next to the job name to edit the YAML file corresponding to the current job. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------+ + | Deleting a job | #. Select the job to be deleted and click **Delete** in the **Operation** column. | + | | | + | | #. Click **Yes**. | + | | | + | | Deleted jobs cannot be restored. Exercise caution when deleting a job. | + +-----------------------------------+-------------------------------------------------------------------------------------------------------------+ diff --git a/umn/source/workloads/creating_a_statefulset.rst b/umn/source/workloads/creating_a_statefulset.rst index 15b4516..609566a 100644 --- a/umn/source/workloads/creating_a_statefulset.rst +++ b/umn/source/workloads/creating_a_statefulset.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0048.html +:original_name: cce_10_0048.html -.. _cce_01_0048: +.. _cce_10_0048: Creating a StatefulSet ====================== @@ -12,10 +12,18 @@ StatefulSets are a type of workloads whose data or status is stored while they a A container can be migrated between different hosts, but data is not stored on the hosts. To store StatefulSet data persistently, attach HA storage volumes provided by CCE to the container. +Notes and Constraints +--------------------- + +- When you delete or scale a StatefulSet, the system does not delete the storage volumes associated with the StatefulSet to ensure data security. +- When you delete a StatefulSet, reduce the number of replicas to **0** before deleting the StatefulSet so that pods in the StatefulSet can be stopped in order. +- When you create a StatefulSet, a headless Service is required for pod access. For details, see :ref:`Headless Service `. +- When a node is unavailable, pods become **Unready**. In this case, you need to manually delete the pods of the StatefulSet so that the pods can be migrated to a normal node. + Prerequisites ------------- -- Before creating a workload, you must have an available cluster. For details on how to create a cluster, see :ref:`Creating a CCE Cluster `. +- Before creating a workload, you must have an available cluster. For details on how to create a cluster, see :ref:`Creating a CCE Cluster `. - To enable public access to a workload, ensure that an EIP or load balancer has been bound to at least one node in the cluster. .. note:: @@ -25,227 +33,91 @@ Prerequisites Using the CCE Console --------------------- -CCE provides multiple methods for creating a workload. You can use any of the following methods: +#. Log in to the CCE console. -#. Use an image in Third-Party Images. You do not need to upload any image before using it. -#. Use an image that you have uploaded to SWR. -#. Use a shared image to create a workload. Specifically, other tenants share an image with you by using the SWR service. -#. Use a YAML file to create a workload. You can click **Create YAML** on the right of the **Create StatefulSet** page. For details about YAML, see :ref:`Using kubectl `. After the YAML file is written, click **Create** to create a workload. +#. Click the cluster name to access the cluster details page, choose **Workloads** in the navigation pane, and click the **Create Workload** in the upper right corner. - .. note:: +#. Set basic information about the workload. - Settings in the YAML file are synchronized with those on the console. You can edit the YAML file on the console to create a workload. For example: + **Basic Info** - - If you enter a workload name on the console, the name will automatically appear in the YAML file. - - If you add an image on the console, the image will be automatically added to the YAML file. + - **Workload Type**: Select **StatefulSet**. For details about workload types, see :ref:`Overview `. + - **Workload Name**: Enter the name of the workload. + - **Namespace**: Select the namespace of the workload. The default value is **default**. You can also click **Create Namespace** to create one. For details, see :ref:`Creating a Namespace `. + - **Pods**: Enter the number of pods. + - **Container Runtime**: A CCE cluster uses runC by default, whereas a CCE Turbo cluster supports both runC and Kata. For details about the differences between runC and Kata, see :ref:`Kata Containers and Common Containers `. + - **Time Zone Synchronization**: Specify whether to enable time zone synchronization. After time zone synchronization is enabled, the container and node use the same time zone. The time zone synchronization function depends on the local disk mounted to the container. Do not modify or delete the time zone. For details, see :ref:`Configuring Time Zone Synchronization `. - When you click **Create YAML** on the right of the console, do not create multiple YAML files in the YAML definition pane displayed. You need to create them one by one. Otherwise, an error will be reported during the creation. + **Container Settings** -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **StatefulSets**. On the displayed page, click **Create StatefulSet**. Set basic workload parameters as described in :ref:`Table 1 `. The parameters marked with an asterisk (*) are mandatory. + - Container Information - .. _cce_01_0048__table12741447488: + Multiple containers can be configured in a pod. You can click **Add Container** on the right to configure multiple containers for the pod. - .. table:: **Table 1** Basic workload parameters - - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================+ - | \* Workload Name | Name of a workload, which must be unique. | - | | | - | | Enter 4 to 52 characters starting with a lowercase letter and ending with a letter or digit. Only lowercase letters, digits, and hyphens (-) are allowed. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Cluster Name | Cluster to which the workload belongs. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Namespace | In a single cluster, data in different namespaces is isolated from each other. This enables applications to share the services of the same cluster without interfering each other. If no namespace is set, the **default** namespace is used. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \* Instances | Number of pods in a workload. A workload can have one or more pods. The default value is **2**. You can customize the value, for example, setting it to **1**. | - | | | - | | Each workload pod consists of the same containers. You can configure multiple pods for a workload to ensure high reliability. For such a workload, if one pod is faulty, the workload can still run properly. If only one pod is used, a node or pod exception may cause service exceptions. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Time Zone Synchronization | If this parameter is enabled, the container and the node use the same time zone. | - | | | - | | .. important:: | - | | | - | | NOTICE: | - | | After time zone synchronization is enabled, disks of the hostPath type will be automatically added and listed in the **Data Storage** > **Local Volume** area. Do not modify or delete the disks. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Description | Description of the workload. | - +-----------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - -#. Click **Next: Add Container**. - - a. Click **Add Container** and select the image to be deployed. - - - **My Images**: Create a workload using an image in the image repository you created. - - **Third-Party Images**: Create a workload using an image from any third-party image repository. When you create a workload using a third-party image, ensure that the node where the workload is running can access public networks. For details on how to create a workload using a third-party image, see :ref:`Using a Third-Party Image `. - - - If your image repository does not require authentication, set **Secret Authentication** to **No**, enter an image pull address, and then click **OK**. - - If your image repository must be authenticated (account and password), you need to create a secret and then use a third-party image. For details, see :ref:`Using a Third-Party Image `. - - - **Shared Images**: Create a workload using an image shared by another tenant through the SWR service. - - b. Configure basic image information. - - A workload is an abstract model of a group of pods. One pod can encapsulate one or more containers. You can click **Add Container** in the upper right corner to add multiple container images and set them separately. - - .. table:: **Table 2** Image parameters - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+==============================================================================================================================================================================================================================================================================================================+ - | Image Name | Name of the image. You can click **Change Image** to update it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Image Version | Select the image tag to be deployed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | \*Container Name | Name of the container. You can modify it. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Privileged Container | Programs in a privileged container have certain privileges. | - | | | - | | If **Privileged Container** is **On**, the container is granted superuser permissions. For example, privileged containers can manipulate network devices on the host machine and modify kernel parameters. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Resources | **CPU** | - | | | - | | - **Request**: minimum number of CPU cores required by a container. The default value is 0.25 cores. | - | | - **Limit**: maximum number of CPU cores available for a container. Do not leave **Limit** unspecified. Otherwise, intensive use of container resources will occur and your workload may exhibit unexpected behavior. | - | | | - | | **Memory** | - | | | - | | - **Request**: minimum amount of memory required by a container. The default value is 512 MiB. | - | | - **Limit**: maximum amount of memory available for a container. When memory usage exceeds the specified memory limit, the container will be terminated. | - | | | - | | For more information about **Request** and **Limit**, see :ref:`Setting Container Specifications `. | - | | | - | | **GPU**: configurable only when the cluster contains GPU nodes. | - | | | - | | It indicates the percentage of GPU resources reserved for a container. Select **Use** and set the percentage. For example, if this parameter is set to 10%, the container is allowed to use 10% of GPU resources. If you do not select **Use** or set this parameter to **0**, no GPU resources can be used. | - | | | - | | **GPU/Graphics Card**: The workload's pods will be scheduled to the node with the specified GPU. | - | | | - | | If **Any GPU type** is selected, the container uses a random GPU in the node. If you select a specific GPU, the container uses this GPU accordingly. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - c. **Lifecycle**: Commands for starting and running containers can be set. - - - **Start Command**: executed when the workload is started. For details, see :ref:`Setting Container Startup Commands `. - - **Post-Start**: executed after the workload runs successfully. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - **Pre-Stop**: executed to delete logs or temporary files before the workload ends. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - d. **Health Check**: CCE provides two types of probes: liveness probe and readiness probe. They are used to determine whether containers and user services are running properly. For more information, see :ref:`Setting Health Check for a Container `. - - - **Liveness Probe**: used to restart the unhealthy container. - - **Readiness Probe**: used to change the container to the unready state when detecting that the container is unhealthy. In this way, service traffic will not be directed to the container. - - e. **Environment Variables**: Environment variables can be added to a container. In general, environment variables are used to set parameters. - - On the **Environment Variables** tab page, click **Add Environment Variable**. Currently, three types of environment variables are supported: - - - **Added manually**: Set **Variable Name** and **Variable Value/Reference**. - - **Added from Secret**: Set **Variable Name** and select the desired secret name and data. A secret must be created in advance. For details, see :ref:`Creating a Secret `. - - **Added from ConfigMap**: Set **Variable Name** and select the desired ConfigMap name and data. A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. + - **Basic Info**: See :ref:`Setting Basic Container Information `. + - **Lifecycle**: See :ref:`Setting Container Lifecycle Parameters `. + - **Health Check**: See :ref:`Setting Health Check for a Container `. + - **Environment Variables**: See :ref:`Setting an Environment Variable `. + - **Data Storage**: See :ref:`Overview `. .. note:: - To edit an environment variable that has been set, click **Edit**. To delete an environment variable that has been set, click **Delete**. + - StatefulSets support dynamically provisioned EVS volumes. - f. **Data Storage**: Data storage can be mounted to containers for persistent storage and high disk I/O. Local volume and cloud storage are supported. For details, see :ref:`Storage (CSI) `. + Dynamic mounting is achieved by using the `volumeClaimTemplates `__ field and depends on the dynamic creation capability of StorageClass. A StatefulSet associates each pod with a unique PVC using the **volumeClaimTemplates** field, and the PVCs are bound to their corresponding PVs. Therefore, after the pod is rescheduled, the original data can still be mounted thanks to the PVC. - .. note:: + - After a workload is created, the storage that is dynamically mounted cannot be updated. - You can add data storage volumes only when creating a StatefulSet. + - **Security Context**: Set container permissions to protect the system and other containers from being affected. Enter the user ID to set container permissions and prevent systems and other containers from being affected. + - **Logging**: See :ref:`Using ICAgent to Collect Container Logs `. - g. **Security Context**: Container permissions can be configured to protect CCE and other containers from being affected. + - **Image Access Credential**: Select the credential used for accessing the image repository. The default value is **default-secret**. You can use default-secret to access images in SWR. For details about **default-secret**, see :ref:`default-secret `. - Enter the user ID to set container permissions and prevent systems and other containers from being affected. + - **GPU graphics card**: **All** is selected by default. The workload instance will be scheduled to the node with the specified GPU graphics card type. - h. **Log Policies**: Log collection policies and log directory can be configured to collect container logs for unified management and analysis. For details, see :ref:`Container Logs `. + **Headless Service Parameters** -#. Click **Next: Set Application Access** and set **Headless Service** and workload access type. + A headless Service is used to solve the problem of mutual access between pods in a StatefulSet. The headless Service provides a fixed access domain name for each pod. For details, see :ref:`Headless Service `. - :ref:`Table 3 ` describes the parameters in the **Headless Service** area. + **Service Settings** - .. _cce_01_0048__table2293204814496: + A Service is used for pod access. With a fixed IP address, a Service forwards access traffic to pods and performs load balancing for these pods. - .. table:: **Table 3** Parameter description + You can also create a Service after creating a workload. For details about the Service, see :ref:`Service Overview `. - +----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +================+==========================================================================================================================================================================================================+ - | Service Name | Name of the Service corresponding to the workload for mutual access between pods. This Service is used for internal discovery of pods, and does not require an independent IP address or load balancing. | - +----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Port Name | Name of the container port. You are advised to enter a name that indicates the function of the port. | - +----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Container Port | Listening port inside the container. | - +----------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + **Advanced Settings** - Click **Add Service** and set the workload access type. + - **Upgrade**: See :ref:`Configuring the Workload Upgrade Policy `. - If your workload will be reachable to other workloads or public networks, add a Service to define the workload access type. + - **Scheduling**: See :ref:`Scheduling Policy (Affinity/Anti-affinity) `. - The workload access type determines the network attributes of the workload. Workloads with different access types can provide different network capabilities. For details, see :ref:`Overview `. + - **Instances Management Policies** -#. Click **Next: Configure Advanced Settings**. + For some distributed systems, the StatefulSet sequence is unnecessary and/or should not occur. These systems require only uniqueness and identifiers. - - **Upgrade Policy**: Only **Rolling upgrade** is supported. + - **OrderedReady**: The StatefulSet will deploy, delete, or scale pods in order and one by one. (The StatefulSet continues only after the previous pod is ready or deleted.) This is the default policy. + - **Parallel**: The StatefulSet will create pods in parallel to match the desired scale without waiting, and will delete all pods at once. - During a rolling upgrade, old pods are gradually replaced with new ones, and service traffic is evenly distributed to both pods to ensure service continuity. + - **Toleration**: Using both taints and tolerations allows (not forcibly) the pod to be scheduled to a node with the matching taints, and controls the pod eviction policies after the node where the pod is located is tainted. For details, see :ref:`Tolerations `. - - **Pod Management Policy**: There are two types of policies: ordered and parallel. + - **Labels and Annotations**: See :ref:`Pod Labels and Annotations `. - **Ordered**: The StatefulSet will deploy, delete, or scale pods in order and one by one (the StatefulSet waits until each pod is ready before continuing). This is the default policy. + - **DNS**: See :ref:`DNS Configuration `. - **Parallel**: The StatefulSet will create pods in parallel to match the desired scale without waiting, and will delete all pods at once. - - - **Graceful Deletion**: A time window can be set for workload deletion and reserved for executing commands in the pre-stop phase in the lifecycle. If workload processes are not terminated after the time window elapses, the workload will be forcibly deleted. - - - **Graceful Time Window (s)**: Set a time window (0-9999s) for pre-stop commands to finish execution before a workload is deleted. The default value is 30s. - - **Scale Order**: Choose **Prioritize new pods** or **Prioritize old pods** based on service requirements. **Prioritize new pods** indicates that new pods will be first deleted when a scale-in is triggered. - - - **Scheduling Policies**: You can combine static global scheduling policies or dynamic runtime scheduling policies as required. For details, see :ref:`Scheduling Policy Overview `. - - - **Advanced Pod Settings** - - - **Pod Label**: The built-in **app** label is specified when the workload is created. It is used to set affinity and anti-affinity scheduling and cannot be modified. You can click **Add Label** to add labels. - - - .. figure:: /_static/images/en-us_image_0220765374.png - :alt: **Figure 1** Advanced pod settings - - **Figure 1** Advanced pod settings - - - **Client DNS Configuration**: A CCE cluster has a built-in DNS add-on (CoreDNS) to provide domain name resolution for workloads in the cluster. - - - **DNS Policy** - - - **ClusterFirst**: The default DNS configuration overrides the **Nameserver** and **DNS Search Domain** configurations of the client. - - **None**: Only the **Nameserver** and **DNS Search Domain** configurations are used for domain name resolution. - - **Default**: The pod inherits the DNS configuration from the node on which the pod runs. - - - **Nameserver**: You can configure a domain name server for a user-defined domain name. The value is one or a group of DNS IP addresses, for example, 1.2.3.4. - - **DNS Search Domain**: a search list for host-name lookup. When a domain name cannot be resolved, DNS queries will be attempted combining the domain name with each domain in the search list in turn until a match is found or all domains in the search list are tried. - - **Timeout (s)**: amount of time the resolver will wait for a response from a remote name server before retrying the query on a different name server. Set it based on the site requirements. - - **ndots**: threshold for the number of dots that must appear in a domain name before an initial absolute query will be made. If a domain name has **ndots** or more than **ndots** dots, the name is a fully qualified domain name (FQDN) and will be tried first as an absolute name. If a domain name has less than **ndots** dots, the operating system will look up the name in a list of search domain names. - -#. Click **Create** and then **Back to StatefulSet List**. If the workload is in the **Running** state, it has been successfully created. If the workload status is not updated, click |image1| in the upper right corner or press **F5** to refresh the page. - - .. note:: - - - When a node is unavailable, pods become **Unready**. In this case, you need to manually delete the pods of the StatefulSet so that the pods can be migrated to a normal node. - - If the workload list contains more than 500 records, the Kubernetes pagination mechanism will be used. Specifically, you can only go to the first page or the next page, but cannot go to the previous page. In addition, if resources are divided into discrete pages, the total number of resources displayed is the maximum number of resources that can be queried at a time, not the actual total number of resources. - -.. _cce_01_0048__section113441881214: +#. Click **Create Workload** in the lower right corner. Using kubectl ------------- -The following procedure uses an etcd workload as an example to describe how to create a workload using kubectl. +In this example, an nginx workload is used and the EVS volume is dynamically mounted to it using the **volumeClaimTemplates** field. -#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. -#. Create and edit the **etcd-statefulset.yaml** file. +#. Create and edit the **nginx-statefulset.yaml** file. - **etcd-statefulset.yaml** is an example file name, and you can change it as required. + **nginx-statefulset.yaml** is an example file name, and you can change it as required. - **vi etcd-statefulset.yaml** + **vi nginx-statefulset.yaml** The following provides an example of the file contents. For more information on StatefulSet, see the `Kubernetes documentation `__. @@ -254,72 +126,99 @@ The following procedure uses an etcd workload as an example to describe how to c apiVersion: apps/v1 kind: StatefulSet metadata: - name: etcd + name: nginx spec: - replicas: 2 selector: matchLabels: - app: etcd - serviceName: etcd-svc + app: nginx template: metadata: labels: - app: etcd + app: nginx spec: containers: - - env: - - name: PAAS_APP_NAME - value: tesyhhj - - name: PAAS_NAMESPACE - value: default - - name: PAAS_PROJECT_ID - value: 9632fae707ce4416a0ab1e3e121fe555 - image: etcd # If you use an image in My Images, obtain the image path from SWR. - imagePullPolicy: IfNotPresent - name: container-0 + - name: container-1 + image: nginx:latest + imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 250m + memory: 512Mi + limits: + cpu: 250m + memory: 512Mi + volumeMounts: + - name: test + readOnly: false + mountPath: /usr/share/nginx/html + subPath: '' + imagePullSecrets: + - name: default-secret + dnsPolicy: ClusterFirst + volumes: [] + serviceName: nginx-svc + replicas: 2 + volumeClaimTemplates: # Dynamically mounts the EVS volume to the workload. + - apiVersion: v1 + kind: PersistentVolumeClaim + metadata: + name: test + namespace: default + annotations: + everest.io/disk-volume-type: SAS # SAS EVS volume type. + labels: + failure-domain.beta.kubernetes.io/region: eu-de # region where the EVS volume is created. + failure-domain.beta.kubernetes.io/zone: # AZ where the EVS volume is created. It must be the same as the AZ of the node. + spec: + accessModes: + - ReadWriteOnce # The value must be ReadWriteOnce for the EVS volume. + resources: + requests: + storage: 10Gi + storageClassName: csi-disk # Storage class name. The value is csi-disk for the EVS volume. updateStrategy: type: RollingUpdate - **vi etcd-headless.yaml** + **vi nginx-headless.yaml** .. code-block:: apiVersion: v1 kind: Service metadata: + name: nginx-svc + namespace: default labels: - app: etcd - name: etcd-svc + app: nginx spec: + selector: + app: nginx + version: v1 clusterIP: None ports: - - name: etcd-svc - port: 3120 - protocol: TCP - targetPort: 3120 - selector: - app: etcd - sessionAffinity: None + - name: nginx + targetPort: 80 + nodePort: 0 + port: 80 + protocol: TCP type: ClusterIP #. Create a workload and the corresponding headless service. - **kubectl create -f etcd-statefulset.yaml** + **kubectl create -f nginx-statefulset.yaml** If the following information is displayed, the StatefulSet has been successfully created. .. code-block:: - statefulset.apps/etcd created + statefulset.apps/nginx created - **kubectl create -f etcd-headless.yaml** + **kubectl create -f nginx-headless.yaml** If the following information is displayed, the headless service has been successfully created. .. code-block:: - service/etcd-svc created + service/nginx-svc created -#. If the workload will be accessed through a ClusterIP or NodePort Service, set the corresponding workload access type. For details, see :ref:`Networking `. - -.. |image1| image:: /_static/images/en-us_image_0300973777.png +#. If the workload will be accessed through a ClusterIP or NodePort Service, set the corresponding workload access type. For details, see :ref:`Networking `. diff --git a/umn/source/workloads/gpu_scheduling.rst b/umn/source/workloads/gpu_scheduling.rst new file mode 100644 index 0000000..e571069 --- /dev/null +++ b/umn/source/workloads/gpu_scheduling.rst @@ -0,0 +1,138 @@ +:original_name: cce_10_0345.html + +.. _cce_10_0345: + +GPU Scheduling +============== + +You can use GPUs in CCE containers. + +Prerequisites +------------- + +- A GPU node has been created. For details, see :ref:`Creating a Node `. + +- The gpu-beta add-on has been installed. During the installation, select the GPU driver on the node. For details, see :ref:`gpu-beta `. + +- gpu-beta mounts the driver directory to **/usr/local/nvidia/lib64**. To use GPU resources in a container, you need to add **/usr/local/nvidia/lib64** to the **LD_LIBRARY_PATH** environment variable. + + Generally, you can use any of the following methods to add a file: + + #. Configure the **LD_LIBRARY_PATH** environment variable in the Dockerfile used for creating an image. (Recommended) + + .. code-block:: + + ENV LD_LIBRARY_PATH /usr/local/nvidia/lib64:$LD_LIBRARY_PATH + + #. Configure the **LD_LIBRARY_PATH** environment variable in the image startup command. + + .. code-block:: + + /bin/bash -c "export LD_LIBRARY_PATH=/usr/local/nvidia/lib64:$LD_LIBRARY_PATH && ..." + + #. Define the **LD_LIBRARY_PATH** environment variable when creating a workload. (Ensure that this variable is not configured in the container. Otherwise, it will be overwritten.) + + .. code-block:: + + env: + - name: LD_LIBRARY_PATH + value: /usr/local/nvidia/lib64 + +Using GPUs +---------- + +Create a workload and request GPUs. You can specify the number of GPUs as follows: + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: gpu-test + namespace: default + spec: + replicas: 1 + selector: + matchLabels: + app: gpu-test + template: + metadata: + labels: + app: gpu-test + spec: + containers: + - image: nginx:perl + name: container-0 + resources: + requests: + cpu: 250m + memory: 512Mi + nvidia.com/gpu: 1 # Number of requested GPUs + limits: + cpu: 250m + memory: 512Mi + nvidia.com/gpu: 1 # Maximum number of GPUs that can be used + imagePullSecrets: + - name: default-secret + +**nvidia.com/gpu** specifies the number of GPUs to be requested. The value can be smaller than **1**. For example, **nvidia.com/gpu: 0.5** indicates that multiple pods share a GPU. In this case, all the requested GPU resources come from the same GPU card. + +After **nvidia.com/gpu** is specified, workloads will not be scheduled to nodes without GPUs. If the node is GPU-starved, Kubernetes events similar to the following are reported: + +- 0/2 nodes are available: 2 Insufficient nvidia.com/gpu. +- 0/4 nodes are available: 1 InsufficientResourceOnSingleGPU, 3 Insufficient nvidia.com/gpu. + +To use GPUs on the CCE console, select the GPU quota and specify the percentage of GPUs reserved for the container when creating a workload. + + +.. figure:: /_static/images/en-us_image_0000001397733101.png + :alt: **Figure 1** Using GPUs + + **Figure 1** Using GPUs + +GPU Node Labels +--------------- + +CCE will label GPU-enabled nodes after they are created. Different types of GPU-enabled nodes have different labels. + +.. code-block:: + + $ kubectl get node -L accelerator + NAME STATUS ROLES AGE VERSION ACCELERATOR + 10.100.2.179 Ready 8m43s v1.19.10-r0-CCE21.11.1.B006-21.11.1.B006 nvidia-t4 + +When using GPUs, you can enable the affinity between pods and nodes based on labels so that the pods can be scheduled to the correct nodes. + +.. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: gpu-test + namespace: default + spec: + replicas: 1 + selector: + matchLabels: + app: gpu-test + template: + metadata: + labels: + app: gpu-test + spec: + nodeSelector: + accelerator: nvidia-t4 + containers: + - image: nginx:perl + name: container-0 + resources: + requests: + cpu: 250m + memory: 512Mi + nvidia.com/gpu: 1 # Number of requested GPUs + limits: + cpu: 250m + memory: 512Mi + nvidia.com/gpu: 1 # Maximum number of GPUs that can be used + imagePullSecrets: + - name: default-secret diff --git a/umn/source/workloads/index.rst b/umn/source/workloads/index.rst index 049836d..fd71f38 100644 --- a/umn/source/workloads/index.rst +++ b/umn/source/workloads/index.rst @@ -1,20 +1,23 @@ -:original_name: cce_01_0046.html +:original_name: cce_10_0046.html -.. _cce_01_0046: +.. _cce_10_0046: Workloads ========= -- :ref:`Overview ` -- :ref:`Creating a Deployment ` -- :ref:`Creating a StatefulSet ` -- :ref:`Creating a DaemonSet ` -- :ref:`Creating a Job ` -- :ref:`Creating a Cron Job ` -- :ref:`Managing Pods ` -- :ref:`Managing Workloads and Jobs ` -- :ref:`Scaling a Workload ` -- :ref:`Configuring a Container ` +- :ref:`Overview ` +- :ref:`Creating a Deployment ` +- :ref:`Creating a StatefulSet ` +- :ref:`Creating a DaemonSet ` +- :ref:`Creating a Job ` +- :ref:`Creating a Cron Job ` +- :ref:`Managing Workloads and Jobs ` +- :ref:`Configuring a Container ` +- :ref:`GPU Scheduling ` +- :ref:`CPU Core Binding ` +- :ref:`Pod Labels and Annotations ` +- :ref:`Volcano Scheduling ` +- :ref:`Security Group Policies ` .. toctree:: :maxdepth: 1 @@ -26,7 +29,10 @@ Workloads creating_a_daemonset creating_a_job creating_a_cron_job - managing_pods managing_workloads_and_jobs - scaling_a_workload configuring_a_container/index + gpu_scheduling + cpu_core_binding/index + pod_labels_and_annotations + volcano_scheduling/index + security_group_policies diff --git a/umn/source/workloads/managing_pods.rst b/umn/source/workloads/managing_pods.rst deleted file mode 100644 index 21be366..0000000 --- a/umn/source/workloads/managing_pods.rst +++ /dev/null @@ -1,75 +0,0 @@ -:original_name: cce_01_0013.html - -.. _cce_01_0013: - -Managing Pods -============= - -Scenario --------- - -A pod is the smallest and simplest unit in the Kubernetes object model that you create or deploy. A pod encapsulates an application's container (or, in some cases, multiple containers), storage resources, a unique network identity (IP address), as well as options that govern how the container(s) should run. A pod represents a single instance of an application in Kubernetes, which might consist of either a single container or a small number of containers that are tightly coupled and that share resources. - -Pods in a Kubernetes cluster can be used in either of the following ways: - -- **Pods that run a single container.** The "one-container-per-pod" model is the most common Kubernetes use case. In this case, a pod functions as a wrapper around a single container, and Kubernetes manages the pods rather than the containers directly. -- **Pods that run multiple containers that need to work together.** A pod might encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. The possible scenarios are as follows: - - - Content management systems, file and data loaders, local cache managers, etc; - - Log and checkpoint backup, compression, rotation, snapshotting, etc; - - Data change watchers, log tailers, logging and monitoring adapters, event publishers, etc; - - Proxies, bridges, adapters, etc; - - Controllers, managers, configurators, and updaters - -You can easily manage pods on CCE, such as editing YAML files and monitoring pods. - -Editing a YAML File -------------------- - -To edit and download the YAML file of a pod online, do as follows: - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Pods**. -#. Click **Edit YAML** at the same row as the target pod. In the **Edit YAML** dialog box displayed, modify the YAML file of the pod. -#. Click **Edit** and then **OK** to save the changes. - - .. note:: - - If a pod is created by another workload, its YAML file cannot be modified individually on the **Pods** page. - -#. (Optional) In the **Edit YAML** window, click **Download** to download the YAML file. - -Monitoring Pods ---------------- - -On the CCE console, you can view the CPU and memory usage, upstream and downstream rates, and disk read/write rates of a workload pod to determine the required resource specifications. - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Pods**. -#. Click **Monitoring** at the same row as the target pod to view the CPU and memory usage, upstream and downstream rates, and disk read/write rates of the pod. - - .. note:: - - You cannot view the monitoring data of a pod that is not running. - -Deleting a Pod --------------- - -If a pod is no longer needed, you can delete it. Deleted pods cannot be recovered. Exercise caution when performing this operation. - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Pods**. - -#. Click **Delete** at the same row as the pod to be deleted. - - Read the system prompts carefully. A pod cannot be restored after it is deleted. Exercise caution when performing this operation. - -#. Click **Yes** to delete the pod. - - .. note:: - - - If the node where the pod is located is unavailable or shut down and the workload cannot be deleted, you can forcibly delete the pod from the pod list on the workload details page. - - Ensure that the storage volumes to be deleted are not used by other workloads. If these volumes are imported or have snapshots, you can only unbind them. - -Helpful Links -------------- - -- `The Distributed System Toolkit: Patterns for Composite Containers `__ -- `Container Design Patterns `__ diff --git a/umn/source/workloads/managing_workloads_and_jobs.rst b/umn/source/workloads/managing_workloads_and_jobs.rst index c14b83b..93652dc 100644 --- a/umn/source/workloads/managing_workloads_and_jobs.rst +++ b/umn/source/workloads/managing_workloads_and_jobs.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0007.html +:original_name: cce_10_0007.html -.. _cce_01_0007: +.. _cce_10_0007: Managing Workloads and Jobs =========================== @@ -8,215 +8,138 @@ Managing Workloads and Jobs Scenario -------- -After a workload is created, you can scale, upgrade, monitor, roll back, or delete the workload, as well as edit its YAML file. +After a workload is created, you can upgrade, monitor, roll back, or delete the workload, as well as edit its YAML file. .. table:: **Table 1** Workload/Job management - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Operation | Description | - +==============================================================+==============================================================================================================================================================================================================================================+ - | :ref:`Logging ` | You can view logs of Deployments, StatefulSets, DaemonSets, and jobs. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Upgrade ` | You can replace images or image tags to quickly upgrade Deployments, StatefulSets, and DaemonSets without interrupting services. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Editing a YAML file ` | You can modify and download the YAML files of Deployments, StatefulSets, DaemonSets, and pods on the CCE console. YAML files of jobs and cron jobs can only be viewed, copied, and downloaded. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Scaling ` | A workload can be automatically resized according to scaling policies, freeing you from the efforts to manually adjust resources for fluctuating service traffic. This saves you big on both resources and labors. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Monitoring ` | You can view the CPU and memory usage of Deployments, DaemonSets, and pods on the CCE console to determine the resource specifications you may need. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Rollback ` | Only Deployments can be rolled back. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Pausing ` | Only Deployments can be paused. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Resuming ` | Only Deployments can be resumed. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Labeling ` | Labels are key-value pairs and can be attached to workloads for affinity and anti-affinity scheduling. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Deletion ` | You can delete a workload or job that is no longer needed. Deleted workloads or jobs cannot be recovered. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Access settings ` | You can determine how your workloads can be accessed. For details, see :ref:`Overview `. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Scheduling policies ` | CCE supports custom and simple scheduling policies. **Custom scheduling policies** allow you to customize node affinity, workload affinity, and workload anti-affinity. **Simple scheduling policies** allow easy and convenient scheduling. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | :ref:`Event ` | CCE provides event names, event types, number of occurrences, Kubernetes events, first occurrence time, and last occurrence time by workload or pod. | - +--------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Operation | Description | + +=====================================================================================+================================================================================================================================================================================================+ + | :ref:`Monitor ` | You can view the CPU and memory usage of workloads and pods on the CCE console. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`View Log ` | You can view the logs of workloads. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`Upgrade ` | You can replace images or image tags to quickly upgrade Deployments, StatefulSets, and DaemonSets without interrupting services. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`Edit YAML ` | You can modify and download the YAML files of Deployments, StatefulSets, DaemonSets, and pods on the CCE console. YAML files of jobs and cron jobs can only be viewed, copied, and downloaded. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`Roll Back ` | Only Deployments can be rolled back. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`Redeploy ` | You can redeploy a workload. After the workload is redeployed, all pods in the workload will be restarted. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`Enabling/Disabling the Upgrade ` | Only Deployments support this operation. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`Manage Label ` | Labels are key-value pairs and can be attached to workloads for affinity and anti-affinity scheduling. Jobs and Cron Jobs do not support this operation. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`Delete ` | You can delete a workload or job that is no longer needed. Deleted workloads or jobs cannot be recovered. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | :ref:`View Events ` | You can view event names, event types, number of occurrences, Kubernetes events, first occurrence time, and last occurrence time. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ + | Stop/Start | You can only start or stop a cron job. | + +-------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -.. _cce_01_0007__section51511928173817: +.. _cce_10_0007__section7200124254011: + +Monitoring a Workload +--------------------- + +You can view the CPU and memory usage of Deployments and pods on the CCE console to determine the resource specifications you may need. This section uses a Deployment as an example to describe how to monitor a workload. + +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. Click the **Deployments** tab and click **Monitor** of the target workload. On the page that is displayed, you can view CPU usage and memory usage of the workload. +#. Click the workload name. On the **Pods** tab page, click the **Monitor** of the target pod to view its CPU and memory usage. + +.. _cce_10_0007__cce_01_0007_section51511928173817: Viewing Logs ------------ You can view logs of Deployments, StatefulSets, DaemonSets, and jobs. This section uses a Deployment as an example to describe how to view logs. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. -#. In the same row as the workload you will view, click **Logs**. +#. Click the **Deployments** tab and click the **View Log** of the target workload. - In the displayed **Logs** window, view the logs generated in the last 5 minutes, 30 minutes, or 1 hour. + On the displayed **View Log** window, you can view logs by time. -.. _cce_01_0007__section17604174417381: +.. _cce_10_0007__cce_01_0007_section17604174417381: Upgrading a Workload -------------------- -You can replace images or image tags to quickly upgrade Deployments, StatefulSets, and DaemonSets without interrupting services. +You quickly upgrade Deployments, StatefulSets, and DaemonSets on the CCE console. This section uses a Deployment as an example to describe how to upgrade a workload. Before replacing an image or image version, upload the new image to the SWR service. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**, and click **Upgrade** for the Deployment to be upgraded. +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. Click the **Deployments** tab and click **Upgrade** of the target workload. .. note:: - Workloads cannot be upgraded in batches. - Before performing an in-place StatefulSet upgrade, you must manually delete old pods. Otherwise, the upgrade status is always displayed as **Upgrading**. -#. Upgrade the Deployment. +#. Upgrade the workload based on service requirements. The method for setting parameter is the same as that for creating a workload. +#. After the update is complete, click **Upgrade Workload**, manually confirm the YAML file, and submit the upgrade. - - **Image Name**: To replace the Deployment image, click **Replace Image** and select a new image. - - **Image Version**: To replace the Deployment image version, select a new version from the **Image Version** drop-down list. - - **Container Name**: To change the container name, click |image1| next to **Container Name** and enter a new name. - - **Privileged Container**: After this function is enabled, the container can access all devices on the host. - - **Container Resources**: You can set the CPU, memory and GPU quotas. - - **Advanced Settings**: - - - **Lifecycle**: Commands for starting and running containers can be set. - - - **Start Command**: executed when the workload is started. For details, see :ref:`Setting Container Startup Commands `. - - **Post-Start**: executed after the workload runs successfully. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - **Pre-Stop**: executed to delete logs or temporary files before the workload ends. For more information, see :ref:`Setting Container Lifecycle Parameters `. - - - **Health Check**: CCE provides two types of probes: liveness probe and readiness probe. They are used to determine whether containers and user services are running properly. For more information, see :ref:`Setting Health Check for a Container `. - - - **Liveness Probe**: used to restart the unhealthy container. - - **Readiness Probe**: used to change the container to the unready state when detecting that the container is unhealthy. In this way, service traffic will not be directed to the container. - - - **Environment Variables**: Environment variables can be added to a container. In general, environment variables are used to set parameters. - - On the **Environment Variables** tab page, click **Add Environment Variable**. Currently, three types of environment variables are supported: - - - **Added manually**: Set **Variable Name** and **Variable Value/Reference**. - - **Added from Secret**: Set **Variable Name** and select the desired secret name and data. A secret must be created in advance. For details, see :ref:`Creating a Secret `. - - **Added from ConfigMap**: Set **Variable Name** and select the desired ConfigMap name and data. A ConfigMap must be created in advance. For details, see :ref:`Creating a ConfigMap `. - - .. note:: - - To edit an environment variable that has been set, click **Edit**. To delete an environment variable that has been set, click **Delete**. - - - **Data Storage**: Data storage can be mounted to containers for persistent storage and high disk I/O. Local disks and cloud storage volumes are supported. For details, see :ref:`Storage (CSI) `. - - .. note:: - - You can add data storage volumes only when creating a StatefulSet. - - - **Security Context**: Container permissions can be configured to protect CCE and other containers from being affected. - - Enter the user ID to set container permissions and prevent systems and other containers from being affected. - - - **Log Policies**: Log collection policies and log directory can be configured to collect container logs for unified management and analysis. For details, see :ref:`Container Logs `. - -#. Click **Submit**. - -.. _cce_01_0007__section21669213390: +.. _cce_10_0007__cce_01_0007_section21669213390: Editing a YAML file ------------------- You can modify and download the YAML files of Deployments, StatefulSets, DaemonSets, and pods on the CCE console. YAML files of jobs and cron jobs can only be viewed, copied, and downloaded. This section uses a Deployment as an example to describe how to edit the YAML file. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. -#. In the same row as the workload you will edit, choose **Operation** > **More** > **Edit YAML**. In the **Edit YAML** window, edit the YAML file of the current workload. +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. Click the **Deployments** tab and choose **More** > **Edit YAML** in the **Operation** column of the target workload. In the dialog box that is displayed, modify the YAML file. #. Click **Edit** and then **OK** to save the changes. #. (Optional) In the **Edit YAML** window, click **Download** to download the YAML file. -.. _cce_01_0007__section11703514131711: - -Scaling a Workload ------------------- - -A workload can be automatically resized according to custom scaling policies, freeing you from the efforts to manually adjust the amount of resources for fluctuating service traffic. This saves you big on both resources and labors. This section uses a Deployment as an example to describe how to scale a workload. - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. - -#. In the same row as the workload for which you will add a scaling policy, choose **Operation** > **More** > **Scaling**. - -#. On the **Scaling** tab page, add or edit scaling policies. Scaling policies are classified as auto and manual scaling policies. - - For details, see :ref:`Scaling a Workload `. - -.. _cce_01_0007__section15303324141816: - -Monitoring a Workload ---------------------- - -You can view the CPU and memory usage of Deployments, DaemonSets, and pods on the CCE console to determine the resource specifications you may need. This section uses a Deployment as an example to describe how to monitor a workload. - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. -#. Click the name of the Deployment to be monitored. On the displayed Deployment details page, click the **Monitoring** tab to view CPU usage and memory usage of the Deployment. -#. Click the **Pods** tab. Click |image2| next to a pod to be monitored and click **Monitoring**. -#. Check CPU usage and memory usage of the pod. - - - CPU usage - - The horizontal axis indicates time while the vertical axis indicates the CPU usage. The green line indicates the CPU usage while the red line indicates the CPU usage limit. - - .. note:: - - It takes some time to calculate CPU usage. Therefore, when CPU and memory usage are displayed for the first time, CPU usage is displayed about one minute later than memory usage. - - CPU and memory usage are displayed only for pods in the running state. - - - Memory usage - - The horizontal axis indicates time while the vertical axis indicates the memory usage. The green line indicates the memory usage while the red line indicates the memory usage limit. - - .. note:: - - Memory usage is displayed only for a running pod. - -.. _cce_01_0007__section13324541124815: +.. _cce_10_0007__cce_01_0007_section13324541124815: Rolling Back a Workload (Available Only for Deployments) -------------------------------------------------------- CCE records the release history of all Deployments. You can roll back a Deployment to a specified version. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. -#. In the same row as the Deployment you will roll back, choose **Operation** > **More** > **Roll Back**. -#. In the **Roll Back to This Version** drop-down list, select the version to which you will roll back the Deployment. Then, click **OK**. +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. Click the **Deployments** tab, choose **More > Roll Back** in the **Operation** column of the target workload. +#. Switch to the **Change History** tab page, click **Roll Back to This Version** of the target version, manually confirm the YAML file, and click **OK**. -.. _cce_01_0007__section10207209154017: +.. _cce_10_0007__section132451237607: -Pausing a Workload (Available Only for Deployments) ---------------------------------------------------- +Redeploying a Workload +---------------------- -You can pause Deployments. After a Deployment is paused, the upgrade command can be successfully issued but will not be applied to the pods. +After you redeploy a workload, all pods in the workload will be restarted. This section uses Deployments as an example to illustrate how to redeploy a workload. -If you are performing a rolling upgrade, the rolling upgrade stops after the pause command is issued. In this case, the new and old pods coexist. +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. Click the **Deployments** tab and choose **More** > **Redeploy** in the **Operation** column of the target workload. +#. In the dialog box that is displayed, click **Yes** to redeploy the workload. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. -#. In the same row as the Deployment you will pause, choose **Operation** > **More** > **Pause**. -#. In the displayed **Pause Workload** dialog box, click **OK**. -#. Click **OK**. +.. _cce_10_0007__cce_01_0007_section12087915401: - .. important:: +Disabling/Enabling Upgrade (Available Only for Deployments) +----------------------------------------------------------- - Deployments in the paused state cannot be rolled back. +Only Deployments support this operation. -.. _cce_01_0007__section12087915401: +- After the upgrade is disabled, the upgrade command can be delivered but will not be applied to the pods. -Resuming a Workload (Available Only for Deployments) ----------------------------------------------------- + If you are performing a rolling upgrade, the rolling upgrade stops after the disabling upgrade command is delivered. In this case, the new and old pods co-exist. -You can resume paused Deployments. After a Deployment is resumed, it can be upgraded or rolled back. Its pods will inherit the latest updates of the Deployment. If they are inconsistent, the pods are upgraded automatically according to the latest information of the Deployment. +- If a Deployment is being upgraded, it can be upgraded or rolled back. Its pods will inherit the latest updates of the Deployment. If they are inconsistent, the pods are upgraded automatically according to the latest information of the Deployment. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. -#. In the same row as the Deployment you will resume, choose **Operation** > **More** > **Resume**. -#. In the displayed **Resume Workload** dialog box, click **OK**. +.. important:: -.. _cce_01_0007__section5931193015488: + Deployments in the disable upgrade state cannot be rolled back. + +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. Click the **Deployments** tab and choose **More** > **Disable/Enable Upgrade** in the **Operation** column of the workload. +#. In the dialog box that is displayed, click **Yes**. + +.. _cce_10_0007__cce_01_0007_section5931193015488: Managing Labels --------------- @@ -234,27 +157,27 @@ In the following figure, three labels (release, env, and role) are defined for w If you set **key** to **role** and **value** to **frontend** when using workload scheduling or another function, APP 1 and APP 2 will be selected. -.. figure:: /_static/images/en-us_image_0165888686.png +.. figure:: /_static/images/en-us_image_0000001408895746.png :alt: **Figure 1** Label example **Figure 1** Label example -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. -#. Click the name of the workload whose labels will be managed. -#. On the workload details page, click **Manage Label**. In the displayed dialog box, click **Add Label**. Enter the label key and value, and click **OK**. +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. Click the **Deployments** tab and choose **More** > **Manage Label** in the **Operation** column of the target workload. +#. Click **Add**, enter a key and a value, and click **OK**. .. note:: A key-value pair must contain 1 to 63 characters starting and ending with a letter or digit. Only letters, digits, hyphens (-), underscores (_), and periods (.) are allowed. -.. _cce_01_0007__section14423721191418: +.. _cce_10_0007__cce_01_0007_section14423721191418: Deleting a Workload/Job ----------------------- You can delete a workload or job that is no longer needed. Deleted workloads or jobs cannot be recovered. Exercise caution when you perform this operation. This section uses a Deployment as an example to describe how to delete a workload. -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments**. +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. #. In the same row as the workload you will delete, choose **Operation** > **More** > **Delete**. @@ -267,16 +190,16 @@ You can delete a workload or job that is no longer needed. Deleted workloads or - If the node where the pod is located is unavailable or shut down and the workload cannot be deleted, you can forcibly delete the pod from the pod list on the workload details page. - Ensure that the storage volumes to be deleted are not used by other workloads. If these volumes are imported or have snapshots, you can only unbind them. -.. _cce_01_0007__section1947616516301: +.. _cce_10_0007__cce_01_0007_section1947616516301: -Events ------- +Viewing Events +-------------- -On the workload details page, click the **Events** or **Pods** tab to view the events, event types, number of occurrences, Kubernetes events, first occurrence time, and last occurrence time. +This section uses Deployments as an example to illustrate how to view events of a workload. To view the event of a job or cron jon, click **View Event** in the **Operation** column of the target workload. -.. note:: +#. Log in to the CCE console, go to an existing cluster, and choose **Workloads** in the navigation pane. +#. On the **Deployments** tab page, click the target workload. In the **Pods** tab page, click the **View Events** to view the event name, event type, number of occurrences, Kubernetes event, first occurrence time, and last occurrence time. - Event data will be retained for one hour and then automatically deleted. + .. note:: -.. |image1| image:: /_static/images/en-us_image_0195434915.png -.. |image2| image:: /_static/images/en-us_image_0121749065.png + Event data will be retained for one hour and then automatically deleted. diff --git a/umn/source/workloads/overview.rst b/umn/source/workloads/overview.rst index e264cb0..fadd735 100644 --- a/umn/source/workloads/overview.rst +++ b/umn/source/workloads/overview.rst @@ -1,6 +1,6 @@ -:original_name: cce_01_0006.html +:original_name: cce_10_0006.html -.. _cce_01_0006: +.. _cce_10_0006: Overview ======== @@ -16,9 +16,9 @@ Pods can be used in either of the following ways: - A container is running in a pod. This is the most common usage of pods in Kubernetes. You can view the pod as a single encapsulated container, but Kubernetes directly manages pods instead of containers. -- Multiple containers that need to be coupled and share resources run in a pod. In this scenario, an application contains a main container and several sidecar containers, as shown in :ref:`Figure 1 `. For example, the main container is a web server that provides file services from a fixed directory, and a sidecar container periodically downloads files to the directory. +- Multiple containers that need to be coupled and share resources run in a pod. In this scenario, an application contains a main container and several sidecar containers, as shown in :ref:`Figure 1 `. For example, the main container is a web server that provides file services from a fixed directory, and a sidecar container periodically downloads files to the directory. - .. _cce_01_0006__en-us_topic_0254767870_fig347141918551: + .. _cce_10_0006__en-us_topic_0254767870_fig347141918551: .. figure:: /_static/images/en-us_image_0258392378.png :alt: **Figure 1** Pod diff --git a/umn/source/workloads/pod_labels_and_annotations.rst b/umn/source/workloads/pod_labels_and_annotations.rst new file mode 100644 index 0000000..ba4d631 --- /dev/null +++ b/umn/source/workloads/pod_labels_and_annotations.rst @@ -0,0 +1,56 @@ +:original_name: cce_10_0386.html + +.. _cce_10_0386: + +Pod Labels and Annotations +========================== + +Pod Annotations +--------------- + +CCE allows you to add annotations to a YAML file to realize some advanced pod functions. The following table describes the annotations you can add. + +.. table:: **Table 1** Pod annotations + + +----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ + | Annotation | Description | Default Value | + +==============================================+=======================================================================================================================================================================================+=======================+ + | kubernetes.AOM.log.stdout | Standard output parameter. If not specified, the standard log output of all containers is reported to AOM. You can collect stdout logs from certain containers or ignore them at all. | ``-`` | + | | | | + | | Example: | | + | | | | + | | - Collecting none of the stdout logs: | | + | | | | + | | kubernetes.AOM.log.stdout: '[]' | | + | | | | + | | - Collecting stdout logs of container-1 and container-2: | | + | | | | + | | kubernetes.AOM.log.stdout: '["container-1","container-2"]' | | + +----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ + | metrics.alpha.kubernetes.io/custom-endpoints | Parameter for reporting AOM monitoring metrics that you specify. | ``-`` | + | | | | + | | For details, see :ref:`Custom Monitoring `. | | + +----------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------+ + +Pod Labels +---------- + +When you create a workload on the console, the following labels are added to the pod by default. The value of **app** is the workload name. You can add labels as required. + +The pod labels added here will be added to the **selector.matchLabels** parameter in the workload definition. The following is an example YAML file: + +.. code-block:: + + ... + spec: + selector: + matchLabels: + app: nginx + version: v1 + template: + metadata: + labels: + app: nginx + version: v1 + spec: + ... diff --git a/umn/source/workloads/scaling_a_workload.rst b/umn/source/workloads/scaling_a_workload.rst deleted file mode 100644 index cbfcc68..0000000 --- a/umn/source/workloads/scaling_a_workload.rst +++ /dev/null @@ -1,151 +0,0 @@ -:original_name: cce_01_0057.html - -.. _cce_01_0057: - -Scaling a Workload -================== - -- Auto scaling: You can set metric-based, scheduled, and periodic policies. After configuration, pods can be automatically added or deleted based on resource changes or the specified schedule. -- Manual scaling: Pods are immediately added or deleted after the configuration is complete. - -.. note:: - - **Scaling policy priority**: If you do not manually adjust the number of pods, auto scaling policies will take effect for resource scheduling. If :ref:`manual scaling ` is triggered, auto scaling policies will be temporarily invalid. - -Auto Scaling - HPA ------------------- - -HPA policies can be used for auto scaling. **You can view all policies or perform more operations in** :ref:`Auto Scaling `. - -Auto Scaling - AOM ------------------- - -You can define auto scaling policies as required, which can intelligently adjust resources in response to service changes and data traffic spikes. - -Auto scaling can be backed by Application Operations Management (AOM), but not for clusters of v1.17 and later. - -Currently, CCE supports the following types of auto scaling policies: - -:ref:`Metric-based policy `: After a workload is created, pods will be automatically scaled when the workload's CPU or memory usage exceeds or falls below a preset limit. - -:ref:`Scheduled policy `: scaling at a specified time. Scheduled auto scaling is applicable flash sales, premier shopping events, and other regular events that bring a high burst of traffic load. - -:ref:`Periodic policy `: scaling at a specified time on a daily, weekly, or monthly basis. Periodic scheduling is applicable to scenarios where traffic changes periodically. - -- .. _cce_01_0057__li16804196913: - - **Metric-based policy**: Supports auto scaling of a workload based on the CPU/memory usage. - - #. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments** or **StatefulSets**. In the same row as the target workload, choose **More** > **Scaling**. - - #. In the **Auto Scaling** area, click **Add Scaling Policy**. - - #. Set the policy parameters as listed in :ref:`Table 1 `. - - .. _cce_01_0057__table297454613303: - - .. table:: **Table 1** Parameters for adding a metric-based policy - - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+====================================================================================================================================================================================================================================================================================================================+ - | Policy Name | Enter the name of the scaling policy. | - | | | - | | The policy name must be 1 to 64 characters in length and start with a letter. Only letters, digits, underscores (_), and hyphens (-) are allowed. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Policy Type | Set this parameter to **Metric-based policy**. | - | | | - | | The alarm policy is triggered based on historical data. The system checks whether the indicators set by the user in the monitoring window meet the triggering conditions **every minute**. If the triggering conditions are met for N consecutive periods, the system performs the action specified by the policy. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Metric | Set the metrics that describe the resource performance data or status. | - | | | - | | - **CPU Usage**: CPU usage of the measured object. The value is the percentage of the used CPU cores to the total CPU cores. | - | | - **Physical Memory Usage**: percentage of the physical memory size used by the measured object to the physical memory size that the measured object has applied for. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Trigger Condition | The value can be higher (>) or lower (<) than a threshold. When the usage of the preceding metrics reaches the specified value, the scaling policy is triggered. | - | | | - | | For example, if **Metric** is set to **CPU Usage** and this parameter is set to **> 70%**, the scaling policy is triggered when the CPU usage exceeds 70%. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Monitoring window | Size of the data aggregation window. | - | | | - | | If the value is set to **60**, metric statistics are collected every 60 seconds. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Threshold Crossings | Number of consecutive times that the threshold is reached within the monitoring window. The calculation cycle is fixed at one minute. | - | | | - | | If the parameter is set to **3**, the action is triggered if threshold is reached for three consecutive measurement periods. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - | Action | Action executed after a policy is triggered. Two actions are available: add or reduce pods. | - +-----------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ - - #. Click **OK**. - - #. In the **Auto Scaling** area, check that the policy has been started. - - When the trigger condition is met, the auto scaling policy starts automatically. - -- .. _cce_01_0057__li1595211281895: - - **Scheduled policy:** scaling at a specified time. - - #. In the **Auto Scaling** area, click **Add Scaling Policy**. Select **Scheduled policy**. - - .. table:: **Table 2** Parameters for adding a scheduled policy - - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===================================================================================================================================================+ - | Policy Name | Enter the name of the scaling policy. | - | | | - | | The policy name must be 1 to 64 characters in length and start with a letter. Only letters, digits, underscores (_), and hyphens (-) are allowed. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Policy Type | Set this parameter to **Scheduled policy**. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Trigger Time | Time at which the policy is enforced. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Action | Action executed after a policy is triggered. Three actions are available: add pods, reduce pods, and set the number of pods. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - - #. Click **OK**. - - #. In the **Auto Scaling** area, check that the policy has been started. - - When the trigger time is reached, you can see on the **Pods** tab page that the auto scaling policy has taken effect. - -- .. _cce_01_0057__li35861531491: - - **Periodic policy:** scaling at a specified time on a daily, weekly, or monthly basis. - - #. In the **Auto Scaling** area, click **Add Scaling Policy**. Select **Periodic policy**. - - .. table:: **Table 3** Parameters for adding a periodic policy - - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Parameter | Description | - +===================================+===================================================================================================================================================+ - | Policy Name | Enter the name of the scaling policy. | - | | | - | | The policy name must be 1 to 64 characters in length and start with a letter. Only letters, digits, underscores (_), and hyphens (-) are allowed. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Policy Type | Set this parameter to **Periodic policy**. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Time Range | Specify the time for triggering the policy. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - | Action | Action executed after a policy is triggered. Three actions are available: add pods, reduce pods, and set the number of pods. | - +-----------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------+ - - #. Click **OK**. - - #. In the **Auto Scaling** area, check that the policy has been started. - - When the trigger condition is met, the auto scaling policy starts automatically. - -.. _cce_01_0057__section1050418516503: - -Manual Scaling --------------- - -#. Log in to the CCE console. In the navigation pane, choose **Workloads** > **Deployments** or **StatefulSets**. In the same row as the target workload, choose **More** > **Scaling**. -#. In the **Manual Scaling** area, click |image1| and change the number of pods to, for example, **3**. Then, click **Save**. The scaling takes effect immediately. -#. On the **Pods** tab page, check that a new pod is being created. When the pod status becomes **Running**, pod scaling is complete. - -.. |image1| image:: /_static/images/en-us_image_0144045351.png diff --git a/umn/source/workloads/security_group_policies.rst b/umn/source/workloads/security_group_policies.rst new file mode 100644 index 0000000..127a1df --- /dev/null +++ b/umn/source/workloads/security_group_policies.rst @@ -0,0 +1,127 @@ +:original_name: cce_10_0288.html + +.. _cce_10_0288: + +Security Group Policies +======================= + +When the Cloud Native Network 2.0 model is used, pods use VPC ENIs or sub-ENIs for networking. You can directly bind security groups and EIPs to pods. CCE provides a custom resource object named **SecurityGroup** for you to associate security groups with pods in CCE. You can customize workloads with specific security isolation requirements using SecurityGroups. + +Notes and Constraints +--------------------- + +- This function is supported for CCE Turbo clusters of v1.19 and later. Upgrade your CCE Turbo clusters if their versions are earlier than v1.19. +- A workload can be bound to a maximum of five security groups. + +Using the Console +----------------- + +#. Log in to the CCE console and access the cluster console. + +#. In the navigation pane, choose **Workloads**. On the displayed page, click the name of the target workload. + +#. Switch to the **Security Group Policy** tab page and click **Create**. + +#. Set the parameters as described in :ref:`Table 1 `. + + .. _cce_10_0288__table572616321913: + + .. table:: **Table 1** Configuration parameters + + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ + | Parameter | Description | Example Value | + +============================+===========================================================================================================================================================================================================================================================+======================================+ + | Security Group Policy Name | Enter a security policy name. | security-group | + | | | | + | | Enter 1 to 63 characters. The value must start with a lowercase letter and cannot end with a hyphen (-). Only lowercase letters, digits, and hyphens (-) are allowed. | | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ + | Associate Security Group | The selected security group will be bound to the ENI or supplementary ENI of the selected workload. A maximum of five security groups can be selected from the drop-down list. You must select one or multiple security groups to create a SecurityGroup. | 64566556-bd6f-48fb-b2c6-df8f44617953 | + | | | | + | | If no security group has not been created, click **Create Security Group**. After the security group is created, click the refresh button. | 5451f1b0-bd6f-48fb-b2c6-df8f44617953 | + | | | | + | | .. important:: | | + | | | | + | | NOTICE: | | + | | | | + | | - A maximum of 5 security groups can be selected. | | + | | - Hover the cursor on next to the security group name, and you can view details about the security group. | | + +----------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------+ + +#. After setting the parameters, click **OK**. + + After the security group is created, the system automatically returns to the security group list page where you can see the new security group. + +Using kubectl +------------- + +#. Use kubectl to connect to the cluster. For details, see :ref:`Connecting to a Cluster Using kubectl `. + +#. Create a description file named **securitygroup-demo.yaml**. + + **vi securitygroup-demo.yaml** + + For example, create the following SecurityGroup to bind all nginx workloads with two security groups 64566556-bd6f-48fb-b2c6-df8f44617953 and 5451f1b0-bd6f-48fb-b2c6-df8f44617953 that have been created in advance. An example is as follows: + + .. code-block:: + + apiVersion: crd.yangtse.cni/v1 + kind: SecurityGroup + metadata: + name: demo + namespace: default + spec: + podSelector: + matchLabels: + app: nginx + securityGroups: + - id: 64566556-bd6f-48fb-b2c6-df8f44617953 + - id: 5451f1b0-bd6f-48fb-b2c6-df8f44617953 + + :ref:`Table 2 ` describes the parameters in the YAML file. + + .. _cce_10_0288__table132326831016: + + .. table:: **Table 2** Description + + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | Field | Description | Mandatory | + +================+=========================================================================================+===========+ + | apiVersion | API version. The value is **crd.yangtse.cni/v1**. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | kind | Type of the object to be created. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | metadata | Metadata definition of the resource object. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | name | Name of the SecurityGroup. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | namespace | Name of the namespace. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | spec | Detailed description of the SecurityGroup. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | podSelector | Used to define the workload to be associated with security groups in the SecurityGroup. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + | securityGroups | Security group ID. | Yes | + +----------------+-----------------------------------------------------------------------------------------+-----------+ + +#. Run the following command to create the SecurityGroup: + + **kubectl create -f securitygroup-demo.yaml** + + If the following information is displayed, the SecurityGroup is being created. + + .. code-block:: + + securitygroup.crd.yangtse.cni/demo created + +#. Run the following command to view the SecurityGroup: + + **kubectl get sg** + + If the name of the created SecurityGroup is **demo** in the command output, the SecurityGroup is created successfully. + + .. code-block:: + + NAME POD-SELECTOR AGE + all-no map[matchLabels:map[app:nginx]] 4h1m + s001test map[matchLabels:map[app:nginx]] 19m + demo map[matchLabels:map[app:nginx]] 2m9s diff --git a/umn/source/workloads/volcano_scheduling/hybrid_deployment_of_online_and_offline_jobs.rst b/umn/source/workloads/volcano_scheduling/hybrid_deployment_of_online_and_offline_jobs.rst new file mode 100644 index 0000000..49a4ad0 --- /dev/null +++ b/umn/source/workloads/volcano_scheduling/hybrid_deployment_of_online_and_offline_jobs.rst @@ -0,0 +1,521 @@ +:original_name: cce_10_0384.html + +.. _cce_10_0384: + +Hybrid Deployment of Online and Offline Jobs +============================================ + +Online and Offline Jobs +----------------------- + +Jobs can be classified into online jobs and offline jobs based on whether services are always online. + +- **Online job**: Such jobs run for a long time, with regular traffic surges, tidal resource requests, and high requirements on SLA, such as advertising and e-commerce services. +- **Offline jobs**: Such jobs run for a short time, have high computing requirements, and can tolerate high latency, such as AI and big data services. + +Resource Oversubscription and Hybrid Deployment +----------------------------------------------- + +Many services see surges in traffic. To ensure performance and stability, resources are often requested at the maximum needed. However, the surges may ebb very shortly and resources, if not released, are wasted in non-peak hours. Especially for online jobs that request a large quantity of resources to ensure SLA, resource utilization can be as low as it gets. + +Resource oversubscription is the process of making use of idle requested resources. Oversubscribed resources are suitable for deploying offline jobs, which focus on throughput but have low SLA requirements and can tolerate certain failures. + +Hybrid deployment of online and offline jobs in a cluster can better utilize cluster resources. + + +.. figure:: /_static/images/en-us_image_0000001378942548.png + :alt: **Figure 1** Resource oversubscription + + **Figure 1** Resource oversubscription + +Oversubscription for Hybrid Deployment +-------------------------------------- + +Hybrid deployment is supported, and CPU and memory resources can be oversubscribed. The key features are as follows: + +- Offline jobs preferentially run on oversubscribed nodes. + + If both oversubscribed and non-oversubscribed nodes exist, the former will score higher than the latter and offline jobs are preferentially scheduled to oversubscribed nodes. + +- Online jobs can use only non-oversubscribed resources if scheduled to an oversubscribed node. + + Offline jobs can use both oversubscribed and non-oversubscribed resources of an oversubscribed node. + +- In the same scheduling period, online jobs take precedence over offline jobs. + + If both online and offline jobs exist, online jobs are scheduled first. When the node resource usage exceeds the upper limit and the node requests exceed 100%, offline jobs will be evicted. + +- CPU/memory isolation is provided by kernels. + + CPU isolation: Online jobs can quickly preempt CPU resources of offline jobs and suppress the CPU usage of the offline jobs. + + Memory isolation: When system memory resources are used up and OOM Kill is triggered, the kernel evicts offline jobs first. + +- kubelet offline jobs admission rules: + + After the the pod is scheduled to a node, kubelet starts the pod only when the node resources can meet the pod request (predicateAdmitHandler.Admit). kubelet starts the pod when both of the following conditions are met: + + - The total request of pods to be started and online running jobs < allocatable nodes + - The total request of pods to be started and online/offline running job < allocatable nodes+oversubscribed nodes + +- Resource oversubscription and hybrid deployment: + + If only hybrid deployment is used, you need to configure the label **volcano.sh/colocation=true** for the node and delete the node label **volcano.sh/oversubscription** or set its value to **false**. + + If the label **volcano.sh/colocation=true** is configured for a node, hybrid deployment is enabled. If the label **volcano.sh/oversubscription=true** is configured, resource oversubscription is enabled. The following table lists the available feature combinations after hybrid deployment or resource oversubscription is enabled. + + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | Hybrid Deployment Enabled (volcano.sh/colocation=true) | Resource oversubscription Enabled (volcano.sh/oversubscription=true) | Use Oversubscribed Resources? | Conditions for Evicting Offline Pods | + +========================================================+======================================================================+===============================+========================================================================================+ + | No | No | No | None | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | Yes | No | No | The node resource usage exceeds the high threshold. | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | No | Yes | Yes | The node resource usage exceeds the high threshold, and the node request exceeds 100%. | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + | Yes | Yes | Yes | The node resource usage exceeds the high threshold. | + +--------------------------------------------------------+----------------------------------------------------------------------+-------------------------------+----------------------------------------------------------------------------------------+ + +Notes and Constraints +--------------------- + +**Specifications** + +- Kubernetes version: + + - 1.19: 1.19.16-r4 or later + - 1.21: 1.21.7-r0 or later + - 1.23: 1.23.5-r0 or later + +- Cluster Type: CCE or CCE Turbo +- Node OS: EulerOS 2.9 (kernel-4.18.0-147.5.1.6.h729.6.eulerosv2r9.x86_64) +- Node Type: ECS +- The volcano add-on version: 1.7.0 or later + +**Constraints** + +- Before enabling the volcano oversubscription plug-in, ensure that the overcommit plug-in is not enabled. +- Modifying the label of an oversubscribed node does not affect the running pods. +- Running pods cannot be converted between online and offline services. To convert services, you need to rebuild pods. +- If the label **volcano.sh/oversubscription=true** is configured for a node in the cluster, the **oversubscription** configuration must be added to the volcano add-on. Otherwise, the scheduling of oversubscribed nodes will be abnormal. Ensure that you have correctly configure labels because the scheduler does not check the add-on and node configurations. For details about the labels, see :ref:`Configuring Oversubscription Labels for Scheduling `. +- To disable oversubscription, perform the following operations: + + - Remove the **volcano.sh/oversubscription** label from the oversubscribed node. + - Set **over-subscription-resource** to **false**. + - Modify the configmap of the volcano scheduler named **volcano-scheduler-configmap** and remove the oversubscription add-on. + +- If **cpu-manager-policy** is set to static core binding on a node, do not assign the QoS class of Guaranteed to offline pods. If core binding is required, change the pods to online pods. Otherwise, offline pods may occupy the CPUs of online pods, causing online pod startup failures, and offline pods fail to be started although they are successfully scheduled. +- If **cpu-manager-policy** is set to static core binding on a node, do not bind cores to all online pods. Otherwise, online pods occupy all CPU or memory resources, leaving a small number of oversubscribed resources. + +.. _cce_10_0384__section1940910414220: + +Configuring Oversubscription Labels for Scheduling +-------------------------------------------------- + +If the label **volcano.sh/oversubscription=true** is configured for a node in the cluster, the **oversubscription** configuration must be added to the volcano add-on. Otherwise, the scheduling of oversubscribed nodes will be abnormal. For details about the related configuration, see :ref:`Table 1 `. + +Ensure that you have correctly configure labels because the scheduler does not check the add-on and node configurations. + +.. _cce_10_0384__table152481219311: + +.. table:: **Table 1** Configuring oversubscription labels for scheduling + + +----------------------------+--------------------------------+----------------------------------------------------+ + | Oversubscription in Add-on | Oversubscription Label on Node | Scheduling | + +============================+================================+====================================================+ + | Yes | Yes | Triggered by oversubscription | + +----------------------------+--------------------------------+----------------------------------------------------+ + | Yes | No | Triggered | + +----------------------------+--------------------------------+----------------------------------------------------+ + | No | No | Triggered | + +----------------------------+--------------------------------+----------------------------------------------------+ + | No | Yes | Not triggered or failed. Avoid this configuration. | + +----------------------------+--------------------------------+----------------------------------------------------+ + +Using Hybrid Deployment +----------------------- + +#. Configure the volcano add-on. + + a. Use kubectl to connect to the cluster. + + b. Install the volcano plug-in and add the **oversubscription** plug-in to **volcano-scheduler-configmap**. Ensure that the plug-in configuration does not contain the **overcommit** plug-in. If **- name: overcommit** exists, delete this configuration. + + .. code-block:: + + # kubectl edit cm volcano-scheduler-configmap -n kube-system + apiVersion: v1 + data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: gang + - name: priority + - name: conformance + - name: oversubscription + - plugins: + - name: drf + - name: predicates + - name: nodeorder + - name: binpack + - plugins: + - name: cce-gpu-topology-predicate + - name: cce-gpu-topology-priority + - name: cce-gpu + +#. Enable the node oversubscription feature. + + A label can be configured to use oversubscribed resources only after the oversubscription feature is enabled for a node. Related nodes can be created only in a node pool. To enable the oversubscription feature, perform the following steps: + + a. Create a node pool. + b. Choose **More** > **Manage** in the **Operation** column of the created node pool. + c. In the **Manage Component** window that is displayed, set **over-subscription-resource** under **kubelet** to **true** and click **OK**. + + |image1| + +#. Set the node oversubscription label. + + The **volcano.sh/oversubscription** label needs to be configured for an oversubscribed node. If this label is set for a node and the value is **true**, the node is an oversubscribed node. Otherwise, the node is not an oversubscribed node. + + .. code-block:: + + kubectl label node 192.168.0.0 volcano.sh/oversubscription=true + + An oversubscribed node also supports the oversubscription thresholds, as listed in :ref:`Table 2 `. For example: + + .. code-block:: + + kubectl annotate node 192.168.0.0 volcano.sh/evicting-cpu-high-watermark=70 + + Querying the node information + + .. code-block:: + + # kubectl describe node 192.168.0.0 + Name: 192.168.0.0 + Roles: + Labels: ... + volcano.sh/oversubscription=true + Annotations: ... + volcano.sh/evicting-cpu-high-watermark: 70 + + .. _cce_10_0384__table1853397191112: + + .. table:: **Table 2** Node oversubscription annotations + + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | Name | Description | + +===========================================+====================================================================================================================================+ + | volcano.sh/evicting-cpu-high-watermark | When the CPU usage of a node exceeds the specified value, offline job eviction is triggered and the node becomes unschedulable. | + | | | + | | The default value is **80**, indicating that offline job eviction is triggered when the CPU usage of a node exceeds 80%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/evicting-cpu-low-watermark | After eviction is triggered, the scheduling starts again when the CPU usage of a node is lower than the specified value. | + | | | + | | The default value is **30**, indicating that scheduling starts again when the CPU usage of a node is lower than 30%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/evicting-memory-high-watermark | When the memory usage of a node exceeds the specified value, offline job eviction is triggered and the node becomes unschedulable. | + | | | + | | The default value is **60**, indicating that offline job eviction is triggered when the memory usage of a node exceeds 60%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/evicting-memory-low-watermark | After eviction is triggered, the scheduling starts again when the memory usage of a node is lower than the specified value. | + | | | + | | The default value is **30**, indicating that the scheduling starts again when the memory usage of a node is less than 30%. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + | volcano.sh/oversubscription-types | Oversubscribed resource type. The options are as follows: | + | | | + | | - CPU (oversubscribed CPU) | + | | - memory (oversubscribed memory) | + | | - cpu,memory (oversubscribed CPU and memory) | + | | | + | | The default value is **cpu,memory**. | + +-------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------+ + +#. Deploy online and offline jobs. + + The **volcano.sh/qos-level** label needs to be added to annotation to distinguish offline jobs. The value is an integer ranging from -7 to 7. If the value is less than 0, the job is an offline job. If the value is greater than or equal to 0, the job is a high-priority job, that is, online job. You do not need to set this label for online jobs. For both online and offline jobs, set **schedulerName** to **volcano** to enable the Volcano scheduler. + + .. note:: + + The priorities of online/online and offline/offline jobs are not differentiated, and the value validity is not verified. If the value of **volcano.sh/qos-level** of an offline job is not a negative integer ranging from -7 to 0, the job is processed as an online job. + + For an offline job: + + .. code-block:: + + kind: Deployment + apiVersion: apps/v1 + spec: + replicas: 4 + template: + metadata: + annotations: + metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"","path":"","port":"","names":""}]' + volcano.sh/qos-level: "-1" # Offline job label + spec: + schedulerName: volcano # The Volcano scheduler is used. + ... + + For an online job: + + .. code-block:: + + kind: Deployment + apiVersion: apps/v1 + spec: + replicas: 4 + template: + metadata: + annotations: + metrics.alpha.kubernetes.io/custom-endpoints: '[{"api":"","path":"","port":"","names":""}]' + spec: + schedulerName: volcano # The Volcano scheduler is used. + ... + +#. Run the following command to check the number of oversubscribed resources and the resource usage: + + kubectl describe node ** + + .. code-block:: + + # kubectl describe node 192.168.0.0 + Name: 192.168.0.0 + Roles: + Labels: ... + volcano.sh/oversubscription=true + Annotations: ... + volcano.sh/oversubscription-cpu: 2335 + volcano.sh/oversubscription-memory: 341753856 + Allocatable: + cpu: 3920m + memory: 6263988Ki + Allocated resources: + (Total limits may be over 100 percent, i.e., overcommitted.) + Resource Requests Limits + -------- -------- ------ + cpu 4950m (126%) 4950m (126%) + memory 1712Mi (27%) 1712Mi (27%) + +Hybrid Deployment Example +------------------------- + +The following uses an example to describe how to deploy online and offline jobs in hybrid mode. + +#. Assume that a cluster has two nodes: one oversubscribed node and one non-oversubscribed node. + + .. code-block:: + + # kubectl get node + NAME STATUS ROLES AGE VERSION + 192.168.0.173 Ready 4h58m v1.19.16-r2-CCE22.5.1 + 192.168.0.3 Ready 148m v1.19.16-r2-CCE22.5.1 + + - 192.168.0.173 is an oversubscribed node (with the **volcano.sh/oversubscirption=true** label). + - 192.168.0.3 is a non-oversubscribed node (without the **volcano.sh/oversubscirption=true** label). + + .. code-block:: + + # kubectl describe node 192.168.0.173 + Name: 192.168.0.173 + Roles: + Labels: beta.kubernetes.io/arch=amd64 + ... + volcano.sh/oversubscription=true + +#. Submit offline job creation requests. If resources are sufficient, all offline jobs will be scheduled to the oversubscribed node. + + The offline job template is as follows: + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: offline + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: offline + template: + metadata: + labels: + app: offline + annotations: + volcano.sh/qos-level: "-1" #Offline job label + spec: + schedulerName: volcano # The Volcano scheduler is used. + containers: + - name: container-1 + image: nginx:latest + imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 500m + memory: 512Mi + limits: + cpu: "1" + memory: 512Mi + imagePullSecrets: + - name: default-secret + + Offline jobs are scheduled to the oversubscribed node. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + offline-69cdd49bf4-pmjp8 1/1 Running 0 5s 192.168.10.178 192.168.0.173 + offline-69cdd49bf4-z8kxh 1/1 Running 0 5s 192.168.10.131 192.168.0.173 + +#. Submit online job creation requests. If resources are sufficient, the online jobs will be scheduled to the non-oversubscribed node. + + The online job template is as follows: + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: online + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: online + template: + metadata: + labels: + app: online + spec: + schedulerName: volcano # The Volcano scheduler is used. + containers: + - name: container-1 + image: resource_consumer:latest + imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 1400m + memory: 512Mi + limits: + cpu: "2" + memory: 512Mi + imagePullSecrets: + - name: default-secret + + Online jobs are scheduled to the non-oversubscribed node. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + online-ffb46f656-4mwr6 1/1 Running 0 5s 192.168.10.146 192.168.0.3 + online-ffb46f656-dqdv2 1/1 Running 0 5s 192.168.10.67 192.168.0.3 + +#. Improve the resource usage of the oversubscribed node and observe whether offline job eviction is triggered. + + Deploy online jobs to the oversubscribed node (192.168.0.173). + + .. code-block:: + + apiVersion: apps/v1 + kind: Deployment + metadata: + name: online + namespace: default + spec: + replicas: 2 + selector: + matchLabels: + app: online + template: + metadata: + labels: + app: online + spec: + affinity: # Submit an online job to an oversubscribed node. + nodeAffinity: + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: + - key: kubernetes.io/hostname + operator: In + values: + - 192.168.0.173 + schedulerName: volcano # The Volcano scheduler is used. + containers: + - name: container-1 + image: resource_consumer:latest + imagePullPolicy: IfNotPresent + resources: + requests: + cpu: 700m + memory: 512Mi + limits: + cpu: 700m + memory: 512Mi + imagePullSecrets: + - name: default-secret + + Submit the online or offline jobs to the oversubscribed node (192.168.0.173) at the same time. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + offline-69cdd49bf4-pmjp8 1/1 Running 0 13m 192.168.10.178 192.168.0.173 + offline-69cdd49bf4-z8kxh 1/1 Running 0 13m 192.168.10.131 192.168.0.173 + online-6f44bb68bd-b8z9p 1/1 Running 0 3m4s 192.168.10.18 192.168.0.173 + online-6f44bb68bd-g6xk8 1/1 Running 0 3m12s 192.168.10.69 192.168.0.173 + + Observe the oversubscribed node (192.168.0.173). You can find that oversubscribed resources exist and the CPU allocation rate exceeds 100%. + + .. code-block:: + + # kubectl describe node 192.168.0.173 + Name: 192.168.0.173 + Roles: + Labels: … + volcano.sh/oversubscription=true + Annotations: … + volcano.sh/oversubscription-cpu: 2343 + volcano.sh/oversubscription-memory: 3073653200 + … + Allocated resources: + (Total limits may be over 100 percent, i.e., overcommitted.) + Resource Requests Limits + -------- -------- ------ + cpu 4750m (121%) 7350m (187%) + memory 3760Mi (61%) 4660Mi (76%) + … + + Increase the CPU usage of online jobs on the node. Offline job eviction is triggered. + + .. code-block:: + + # kubectl get pod -o wide + NAME READY STATUS RESTARTS AGE IP NODE + offline-69cdd49bf4-bwdm7 1/1 Running 0 11m 192.168.10.208 192.168.0.3 + offline-69cdd49bf4-pmjp8 0/1 Evicted 0 26m 192.168.0.173 + offline-69cdd49bf4-qpdss 1/1 Running 0 11m 192.168.10.174 192.168.0.3 + offline-69cdd49bf4-z8kxh 0/1 Evicted 0 26m 192.168.0.173 + online-6f44bb68bd-b8z9p 1/1 Running 0 24m 192.168.10.18 192.168.0.173 + online-6f44bb68bd-g6xk8 1/1 Running 0 24m 192.168.10.69 192.168.0.173 + +Handling Suggestions +-------------------- + +- After kubelet of the oversubscribed node is restarted, the resource view of the Volcano scheduler is not synchronized with that of kubelet. As a result, OutOfCPU occurs in some newly scheduled jobs, which is normal. After a period of time, the Volcano scheduler can properly schedule online and offline jobs. + +- After online and offline jobs are submitted, you are not advised to dynamically change the job type (adding or deleting annotation volcano.sh/qos-level: "-1") because the current kernel does not support the change of an offline job to an online job. + +- CCE collects the resource usage (CPU/memory) of all pods running on a node based on the status information in the cgroups system. The resource usage may be different from the monitored resource usage, for example, the resource statistics displayed by running the **top** command. + +- You can add oversubscribed resources (such as CPU and memory) at any time. + + You can reduce the oversubscribed resource types only when the resource allocation rate does not exceed 100%. + +.. |image1| image:: /_static/images/en-us_image_0000001207511384.png diff --git a/umn/source/workloads/volcano_scheduling/index.rst b/umn/source/workloads/volcano_scheduling/index.rst new file mode 100644 index 0000000..71363a4 --- /dev/null +++ b/umn/source/workloads/volcano_scheduling/index.rst @@ -0,0 +1,14 @@ +:original_name: cce_10_0423.html + +.. _cce_10_0423: + +Volcano Scheduling +================== + +- :ref:`Hybrid Deployment of Online and Offline Jobs ` + +.. toctree:: + :maxdepth: 1 + :hidden: + + hybrid_deployment_of_online_and_offline_jobs