forked from docs/doc-exports
Chen, Junjie
dd8a3a658b
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Chen, Junjie <chenjunjie@huawei.com> Co-committed-by: Chen, Junjie <chenjunjie@huawei.com>
108 lines
11 KiB
HTML
108 lines
11 KiB
HTML
<a name="Kafka-specification"></a><a name="Kafka-specification"></a>
|
|
|
|
<h1 class="topictitle1">Specifications</h1>
|
|
<div id="body1552013649930"><div class="section" id="Kafka-specification__section21958537463"><h4 class="sectiontitle">Kafka Instance Specifications</h4><p id="Kafka-specification__p17171424696">Kafka instances are compatible with open-source Kafka 1.1.0, 2.3.0, and 2.7. The instance specifications are classified based on bandwidth, namely, 100 MB/s, 300 MB/s, 600 MB/s, and 1200 MB/s.</p>
|
|
|
|
<div class="tablenoborder"><a name="Kafka-specification__en-us_topic_0159429488_table78751014154818"></a><a name="en-us_topic_0159429488_table78751014154818"></a><table cellpadding="4" cellspacing="0" summary="" id="Kafka-specification__en-us_topic_0159429488_table78751014154818" frame="border" border="1" rules="all"><caption><b>Table 1 </b>TPS and the maximum number of partitions supported by different instance specifications and I/O types</caption><thead align="left"><tr id="Kafka-specification__en-us_topic_0159429488_row887661474817"><th align="left" class="cellrowborder" valign="top" width="16.91169116911691%" id="mcps1.3.1.3.2.6.1.1"><p id="Kafka-specification__en-us_topic_0159429488_p187611474813">Bandwidth</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="19.79197919791979%" id="mcps1.3.1.3.2.6.1.2"><p id="Kafka-specification__en-us_topic_0159429488_p087601415481">I/O Type</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="23.54235423542354%" id="mcps1.3.1.3.2.6.1.3"><p id="Kafka-specification__en-us_topic_0159429488_p7876121417484">TPS (High-Throughput)</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="21.53215321532153%" id="mcps1.3.1.3.2.6.1.4"><p id="Kafka-specification__en-us_topic_0159429488_p2876514144817">TPS (Synchronous Replication)</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="18.22182218221822%" id="mcps1.3.1.3.2.6.1.5"><p id="Kafka-specification__en-us_topic_0159429488_p473420062316">Maximum Partitions</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="Kafka-specification__en-us_topic_0159429488_row14876161419486"><td class="cellrowborder" rowspan="2" valign="top" width="16.91169116911691%" headers="mcps1.3.1.3.2.6.1.1 "><p id="Kafka-specification__en-us_topic_0159429488_p887611420481">100 MB/s</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.79197919791979%" headers="mcps1.3.1.3.2.6.1.2 "><p id="Kafka-specification__en-us_topic_0159429488_p587641494812">High I/O</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.54235423542354%" headers="mcps1.3.1.3.2.6.1.3 "><p id="Kafka-specification__en-us_topic_0159429488_p6876191454812">100,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.53215321532153%" headers="mcps1.3.1.3.2.6.1.4 "><p id="Kafka-specification__en-us_topic_0159429488_p13876214144813">60,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="18.22182218221822%" headers="mcps1.3.1.3.2.6.1.5 "><p id="Kafka-specification__en-us_topic_0159429488_p1973490132315">300</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="Kafka-specification__en-us_topic_0159429488_row987671494818"><td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.1 "><p id="Kafka-specification__en-us_topic_0159429488_p14877191414814">Ultra-high I/O</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.2 "><p id="Kafka-specification__en-us_topic_0159429488_p38771414124820">100,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.3 "><p id="Kafka-specification__en-us_topic_0159429488_p1787711464810">80,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.4 "><p id="Kafka-specification__en-us_topic_0159429488_p177341503239">300</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="Kafka-specification__en-us_topic_0159429488_row1987731414488"><td class="cellrowborder" rowspan="2" valign="top" width="16.91169116911691%" headers="mcps1.3.1.3.2.6.1.1 "><p id="Kafka-specification__en-us_topic_0159429488_p158771145486">300 MB/s</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.79197919791979%" headers="mcps1.3.1.3.2.6.1.2 "><p id="Kafka-specification__en-us_topic_0159429488_p81946616198">High I/O</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.54235423542354%" headers="mcps1.3.1.3.2.6.1.3 "><p id="Kafka-specification__en-us_topic_0159429488_p1219412617194">300,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.53215321532153%" headers="mcps1.3.1.3.2.6.1.4 "><p id="Kafka-specification__en-us_topic_0159429488_p19194186201918">150,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="18.22182218221822%" headers="mcps1.3.1.3.2.6.1.5 "><p id="Kafka-specification__en-us_topic_0159429488_p19734170132314">900</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="Kafka-specification__en-us_topic_0159429488_row1987711149481"><td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.1 "><p id="Kafka-specification__en-us_topic_0159429488_p919618691910">Ultra-high I/O</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.2 "><p id="Kafka-specification__en-us_topic_0159429488_p8196186121912">300,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.3 "><p id="Kafka-specification__en-us_topic_0159429488_p141960610191">200,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" headers="mcps1.3.1.3.2.6.1.4 "><p id="Kafka-specification__en-us_topic_0159429488_p373400192312">900</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="Kafka-specification__en-us_topic_0159429488_row10877414124815"><td class="cellrowborder" valign="top" width="16.91169116911691%" headers="mcps1.3.1.3.2.6.1.1 "><p id="Kafka-specification__en-us_topic_0159429488_p19877111424819">600 MB/s</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.79197919791979%" headers="mcps1.3.1.3.2.6.1.2 "><p id="Kafka-specification__en-us_topic_0159429488_p6887132015">Ultra-high I/O</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.54235423542354%" headers="mcps1.3.1.3.2.6.1.3 "><p id="Kafka-specification__en-us_topic_0159429488_p1088181172014">600,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.53215321532153%" headers="mcps1.3.1.3.2.6.1.4 "><p id="Kafka-specification__en-us_topic_0159429488_p1088417205">300,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="18.22182218221822%" headers="mcps1.3.1.3.2.6.1.5 "><p id="Kafka-specification__en-us_topic_0159429488_p1973450182313">1800</p>
|
|
</td>
|
|
</tr>
|
|
<tr id="Kafka-specification__en-us_topic_0159429488_row1020392861910"><td class="cellrowborder" valign="top" width="16.91169116911691%" headers="mcps1.3.1.3.2.6.1.1 "><p id="Kafka-specification__en-us_topic_0159429488_p1320352841912">1200 MB/s</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="19.79197919791979%" headers="mcps1.3.1.3.2.6.1.2 "><p id="Kafka-specification__en-us_topic_0159429488_p1841726206">Ultra-high I/O</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="23.54235423542354%" headers="mcps1.3.1.3.2.6.1.3 "><p id="Kafka-specification__en-us_topic_0159429488_p6842252018">1,200,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="21.53215321532153%" headers="mcps1.3.1.3.2.6.1.4 "><p id="Kafka-specification__en-us_topic_0159429488_p98420232017">400,000</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="18.22182218221822%" headers="mcps1.3.1.3.2.6.1.5 "><p id="Kafka-specification__en-us_topic_0159429488_p187342062312">1800</p>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
<div class="note" id="Kafka-specification__note2446913161813"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><p id="Kafka-specification__p1744714132187">For Kafka instances, the number of transactions per second (TPS) is the maximum number of messages that can be written per second. The preceding TPS is calculated with each message being 1 KB.</p>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="Kafka-specification__section57651849164311"><h4 class="sectiontitle">Bandwidth Selection</h4><p id="Kafka-specification__p1243993263911">The bandwidth of a Kafka instance refers to the maximum read or write bandwidth. You are advised to select a bandwidth 30% higher than what is required.</p>
|
|
<ul id="Kafka-specification__ul11187183018442"><li id="Kafka-specification__li512618337443">100 MB/s<p id="Kafka-specification__p1430012344445"><a name="Kafka-specification__li512618337443"></a><a name="li512618337443"></a>Recommended for up to 3000 client connections, 60 consumer groups, and 70 MB/s service traffic.</p>
|
|
</li><li id="Kafka-specification__li29688391442">300 MB/s<p id="Kafka-specification__p95551406444"><a name="Kafka-specification__li29688391442"></a><a name="li29688391442"></a>Recommended for up to 10,000 client connections, 300 consumer groups, and 210 MB/s service traffic.</p>
|
|
</li><li id="Kafka-specification__li18814185114414">600 MB/s<p id="Kafka-specification__p581412525445"><a name="Kafka-specification__li18814185114414"></a><a name="li18814185114414"></a>Recommended for up to 20,000 client connections, 600 consumer groups, and 420 MB/s service traffic.</p>
|
|
</li><li id="Kafka-specification__li2143165910441">1200 MB/s<p id="Kafka-specification__p997100124514"><a name="Kafka-specification__li2143165910441"></a><a name="li2143165910441"></a>Recommended for up to 20,000 client connections, 600 consumer groups, and 840 MB/s service traffic.</p>
|
|
</li></ul>
|
|
</div>
|
|
<div class="section" id="Kafka-specification__section2792821194417"><h4 class="sectiontitle">Storage Space Selection</h4><p id="Kafka-specification__p4545191495211">Kafka instances support storage with 1 to 3 replicas. The storage space is consumed by all replicas. When creating an instance, specify its storage space based on the expected service message size and the number of replicas.</p>
|
|
<p id="Kafka-specification__p22993455134">For example, if the estimated message size is 100 GB, the disk capacity must be at least: 100 GB x Number of replicas + 100 GB (reserved space).</p>
|
|
</div>
|
|
<div class="section" id="Kafka-specification__section8847182451410"><h4 class="sectiontitle">Topic Quantity</h4><p id="Kafka-specification__p347717213107">There are limits on the topic quantity and the aggregate number of partitions in the topics. When the partition quantity limit is reached, you can no longer create topics.</p>
|
|
<p id="Kafka-specification__p9441156121114">The number of topics is related to the maximum number of partitions allowed and the specified number of partitions in each topic (see <a href="#Kafka-specification__en-us_topic_0159429488_table78751014154818">Table 1</a>).</p>
|
|
<p id="Kafka-specification__p03915410289"><strong id="Kafka-specification__b161201519682">The maximum number of partitions for a 100 MB/s instance is 300.</strong></p>
|
|
<ul id="Kafka-specification__ul1677240123719"><li id="Kafka-specification__li67714014378">If the number of partitions of each topic in the instance is 3, the maximum number of topics is 300/3 = 100.</li><li id="Kafka-specification__li16761142193719">If the number of partitions of each topic in the instance is 1, the maximum number of topics is 300/1 = 300.</li></ul>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="kafka-ug-0723001.html">Service Overview</a></div>
|
|
</div>
|
|
</div>
|
|
|