forked from docs/doc-exports
Yang, Tong
6182f91ba8
Reviewed-by: Hasko, Vladimir <vladimir.hasko@t-systems.com> Co-authored-by: Yang, Tong <yangtong2@huawei.com> Co-committed-by: Yang, Tong <yangtong2@huawei.com>
57 lines
14 KiB
HTML
57 lines
14 KiB
HTML
<a name="mrs_01_1937"></a><a name="mrs_01_1937"></a>
|
|
|
|
<h1 class="topictitle1">Creating a Spark SQL Role</h1>
|
|
<div id="body1595920205992"><div class="section" id="mrs_01_1937__s92d44b7b4fc24f42ac831038a3391d3c"><h4 class="sectiontitle">Scenario</h4><p id="mrs_01_1937__aef5e79bdfe3140c790420f9a1eefaa34">This section describes how to create and configure a SparkSQL role on Manager as the system administrator. The Spark SQL role can be configured with the <span id="mrs_01_1937__ph1556925410519">Spark </span>administrator permission or the permission of performing operations on the table data.</p>
|
|
<p id="mrs_01_1937__ac6167e132f79422db1b3037ce52bcdc0">Creating a database with Hive requires users to join in the <strong id="mrs_01_1937__b1155544547112055">hive</strong> group, without granting a role. Users have all permissions on the databases or tables created by themselves in Hive or HDFS. They can create tables, select, delete, insert, or update data, and grant permissions to other users to allow them to access the tables and corresponding HDFS directories and files. The created databases or tables are saved in the <strong id="mrs_01_1937__b667697035112055">/user/hive/warehouse</strong> directory of HDFS by default.</p>
|
|
<div class="note" id="mrs_01_1937__n6757f921e8a64929a0cc23c62f40d757"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1937__ul1582111910116"><li id="mrs_01_1937__li10209261510">If the current component uses Ranger for permission control, you need to configure permission management policies based on Ranger. For details, see <a href="mrs_01_1860.html">Adding a Ranger Access Permission Policy for Spark2x</a>.</li><li id="mrs_01_1937__li1269751117331">After Ranger authentication is enabled or disabled on Spark2x, you need to restart Spark2x and download the client again or update the client configuration file <strong id="mrs_01_1937__b44561111207">spark/conf/spark-defaults.conf</strong>.<p id="mrs_01_1937__p19489174814011">Enable Ranger authentication: <strong id="mrs_01_1937__b517055016452">spark.ranger.plugin.authorization.enable=true</strong></p>
|
|
<p id="mrs_01_1937__li1269751117331p1">Disable Ranger authentication: <strong id="mrs_01_1937__b1937211313467">spark.ranger.plugin.authorization.enable=false</strong></p>
|
|
</li></ul>
|
|
</div></div>
|
|
</div>
|
|
<div class="section" id="mrs_01_1937__s9565281564ee438fbd4cd214624e0d48"><h4 class="sectiontitle">Procedure</h4><ol id="mrs_01_1937__o696d088ecb98405eb0564042ac472e82"><li id="mrs_01_1937__l517d874fa204422680421f698a4f5538">Log in to Manager, and choose <span class="parmname" id="mrs_01_1937__p6ff23cc3f0c44bc6b96662fd80c3d19e"><b>System > <span id="mrs_01_1937__text95011812125513">Permission > </span>Role</b></span>.</li><li id="mrs_01_1937__lc019f8ec1db14c958cdc0db4397a54bd">Click <span class="parmname" id="mrs_01_1937__p51f02dac5c3541a08516c5848cebb638"><b>Create Role</b></span> and set a role name and enter description.</li><li id="mrs_01_1937__l7bd2c5ddb36946b5966d7a7ea9a25e76">Set <span class="parmname" id="mrs_01_1937__parmname101671928113718"><b><span id="mrs_01_1937__text1791432815554">Configure Resource </span>Permission</b></span>. For details, see <a href="#mrs_01_1937__t5dabb9a552a94c148817ff3e1e9997df">Table 1</a>.<ul id="mrs_01_1937__ua2bf10c6bffc4ad08b372a2eb0f6ff4b"><li id="mrs_01_1937__lf4d2adf24e05458ea07cc291a668365c"><span class="parmname" id="mrs_01_1937__p0271dc3f965a46b3bb02af9d5698ffbf"><b>Hive Admin Privilege</b></span>: Hive administrator permissions.</li><li id="mrs_01_1937__l73914a93d12940598477b134e9772e85"><span class="parmname" id="mrs_01_1937__p9b8718a20f2949a09c52047c8e8dc11d"><b>Hive Read Write Privileges</b></span>: Hive data table management permission, which is the operation permission to set and manage the data of created tables.<div class="note" id="mrs_01_1937__n5d7eaa8bc5cd4afcb4ef9508ca31cc19"><img src="public_sys-resources/note_3.0-en-us.png"><span class="notetitle"> </span><div class="notebody"><ul id="mrs_01_1937__uee10f77094b5492c8e8b56e83672aa8a"><li id="mrs_01_1937__l368334d07df64c4eae0b83b6cc854c1d">Hive role management supports the <span id="mrs_01_1937__ph65121816523">Hive </span>administrator permission, and the permissions of accessing tables and views, without granting the database permission.</li><li id="mrs_01_1937__l70e120ac0159485eb3924f37292d292a">The permissions of the Hive administrator do not include the permission to manage HDFS.</li><li id="mrs_01_1937__l9c408d029a294c73bc237a7288fe132f">If there are too many tables in the database or too many files in tables, the permission granting may last a while. For example, if a table contains 10,000 files, the permission granting lasts about 2 minutes.</li></ul>
|
|
</div></div>
|
|
|
|
<div class="tablenoborder"><a name="mrs_01_1937__t5dabb9a552a94c148817ff3e1e9997df"></a><a name="t5dabb9a552a94c148817ff3e1e9997df"></a><table cellpadding="4" cellspacing="0" summary="" id="mrs_01_1937__t5dabb9a552a94c148817ff3e1e9997df" frame="border" border="1" rules="all"><caption><b>Table 1 </b>Setting a role</caption><thead align="left"><tr id="mrs_01_1937__r6234f2cd87c647b2bf064a55d1a1a876"><th align="left" class="cellrowborder" valign="top" width="25.14%" id="mcps1.3.2.2.3.3.2.3.2.3.1.1"><p id="mrs_01_1937__a2e614ea3d0344020b5a69460102c9fa5">Task</p>
|
|
</th>
|
|
<th align="left" class="cellrowborder" valign="top" width="74.86%" id="mcps1.3.2.2.3.3.2.3.2.3.1.2"><p id="mrs_01_1937__a92d14f474ddf48e5a110efe77a0fd4ba">Operation</p>
|
|
</th>
|
|
</tr>
|
|
</thead>
|
|
<tbody><tr id="mrs_01_1937__r85674f0172764ab6972898535b53bc8d"><td class="cellrowborder" valign="top" width="25.14%" headers="mcps1.3.2.2.3.3.2.3.2.3.1.1 "><p id="mrs_01_1937__a64a94517509b424b9c9351eb9ae19c0b">Hive administrator permission</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="74.86%" headers="mcps1.3.2.2.3.3.2.3.2.3.1.2 "><p id="mrs_01_1937__a1b812cd946174f8c9d383d642a65833d">In the <span class="parmname" id="mrs_01_1937__parmname16197648103811"><b><span id="mrs_01_1937__text1951111103310">Configure Resource </span>Permission</b></span> table, choose <em id="mrs_01_1937__i761021459112055">Name of the desired cluster</em> > <strong id="mrs_01_1937__b296245663112055">Hive</strong> and select <span class="parmname" id="mrs_01_1937__p11c4e482b7fd43bba2ba5138eb9fcb46"><b>Hive Admin Privilege</b></span>.</p>
|
|
<div class="p" id="mrs_01_1937__ac637a1891ca740dba1f7c8b82861b89b">After being bound to the Hive administrator role, perform the following operations during each maintenance operation:<ol type="a" id="mrs_01_1937__o476b19aa3fe44e788a73451c9a3f0edf"><li id="mrs_01_1937__l3f6b1013492f40c3afc022e4b0c11414">Log in to the node where the Spark2x client is installed as the client installation user.</li><li id="mrs_01_1937__la9e2050ed858424aae6292703a9f20c6">Run the following command to configure environment variables:<p id="mrs_01_1937__ae97e52f14530484aa85d5b7e1d8f6c76"><a name="mrs_01_1937__la9e2050ed858424aae6292703a9f20c6"></a><a name="la9e2050ed858424aae6292703a9f20c6"></a>For example, if the Spark2x client installation directory is <span class="filepath" id="mrs_01_1937__f8a303d1d5d9342c4ac3e2d98e99c4610"><b>/opt/client</b></span>, run <strong id="mrs_01_1937__b455017462422">source /opt/client/bigdata_env</strong>.</p>
|
|
<p id="mrs_01_1937__p63206412501"><strong id="mrs_01_1937__b12881052174213">source /opt/client/Spark2x/component_env</strong></p>
|
|
</li><li id="mrs_01_1937__la715ae8cdbde4444a1a92c469aebf54f">Run the following command to perform user authentication:<p id="mrs_01_1937__ae6741da5747b451389c3e93edf7341c3"><a name="mrs_01_1937__la715ae8cdbde4444a1a92c469aebf54f"></a><a name="la715ae8cdbde4444a1a92c469aebf54f"></a><i><b><span class="cmdname" style="font-family:Arial" id="mrs_01_1937__cmdname21831294019">kinit</span></b></i> <i><span class="varname" id="mrs_01_1937__v1cc42cc6495045b7b189af4612e5e059">Hive service user</span></i></p>
|
|
</li><li id="mrs_01_1937__l20c28a7c2eec42869d5ab27cc957e37d">Run the following command to log in to the client tool:<p id="mrs_01_1937__a4c9a4bbb76134d139b266340078c19f7"><a name="mrs_01_1937__l20c28a7c2eec42869d5ab27cc957e37d"></a><a name="l20c28a7c2eec42869d5ab27cc957e37d"></a><strong id="mrs_01_1937__b10887548181512">/opt/client/Spark2x/spark/bin/beeline -u "jdbc:hive2://<zkNode1_IP>:<zkNode1_Port>,<zkNode2_IP>:<zkNode2_Port>,<zkNode3_IP>:<zkNode3_Port>/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=sparkthriftserver2x;user.principal=spark2x/hadoop.</strong><em id="mrs_01_1937__i16892204871519"><system domain name></em><strong id="mrs_01_1937__b5869135291513">@</strong><em id="mrs_01_1937__i18872125271520"><system domain name></em><strong id="mrs_01_1937__b15397816168">;saslQop=auth-conf;auth=KERBEROS;principal=spark2x/hadoop.</strong><em id="mrs_01_1937__i1954248201617"><system domain name></em><strong id="mrs_01_1937__b318621291610">@</strong><em id="mrs_01_1937__i13189412121620"><system domain name></em><strong id="mrs_01_1937__b191871012111618">;"</strong></p>
|
|
<div class="note" id="mrs_01_1937__n08b8dfec09c74ff2a4570ff11fd1936b"><span class="notetitle"> NOTE: </span><div class="notebody"><ul id="mrs_01_1937__u796a53c4a0c74edc8226b507645f82ac"><li id="mrs_01_1937__ld5f86def574e4545a5256647dad0ac17"><span class="parmvalue" id="mrs_01_1937__p403b64716df4483fb8ab296426acc442"><b><zkNode1_IP>:<zkNode1_Port>, <zkNode2_IP>:<zkNode2_Port>, <zkNode3_IP>:<zkNode3_Port></b></span> indicates the ZooKeeper URL, for example, 192.168.81.37:2181,192.168.195.232:2181,192.168.169.84:2181.</li><li id="mrs_01_1937__l5ea26e65b2cb416ba815243e762a7871"><span class="parmvalue" id="mrs_01_1937__pfb233b58d8d441ee8d06a84ce066db45"><b>sparkthriftserver</b></span> indicates a ZooKeeper directory, from which a random TriftServer or ProxyThriftServer is connected by the client.</li><li id="mrs_01_1937__li118613311649">You can log in to Manager, choose <strong id="mrs_01_1937__b1473356737112055">System</strong> > <strong id="mrs_01_1937__b111866962112055">Permission</strong> > <strong id="mrs_01_1937__b507463264112055">Domain and Mutual Trust</strong>, and view the value of <strong id="mrs_01_1937__b1922109090112055">Local Domain</strong>, which is the current system domain name. <strong id="mrs_01_1937__b316959841112055">spark2x/hadoop.</strong><<em id="mrs_01_1937__i688141620112055">System domain name</em>> is the username. All letters in the system domain name contained in the username are lowercase letters. For example, <strong id="mrs_01_1937__b988129214112055">Local Domain</strong> is set to <strong id="mrs_01_1937__b3421880112055">9427068F-6EFA-4833-B43E-60CB641E5B6C.COM</strong>, and the username is <strong id="mrs_01_1937__b1972829776112055">spark2x/hadoo.9427068f-6efa-4833-b43e-60cb641e5b6c.com</strong>.</li></ul>
|
|
</div></div>
|
|
</li><li id="mrs_01_1937__lcfc9d06dc3464c3fb54abf9cdab13109">Run the following command to update the administrator permissions:<p id="mrs_01_1937__a6648ef0d02bb441cbe974142036be71a"><a name="mrs_01_1937__lcfc9d06dc3464c3fb54abf9cdab13109"></a><a name="lcfc9d06dc3464c3fb54abf9cdab13109"></a><strong id="mrs_01_1937__b1894218321223">set role admin</strong><strong id="mrs_01_1937__b7634741144113">;</strong></p>
|
|
</li></ol>
|
|
</div>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1937__ra6c634b448214dba8e42ef2b7fede0c7"><td class="cellrowborder" valign="top" width="25.14%" headers="mcps1.3.2.2.3.3.2.3.2.3.1.1 "><p id="mrs_01_1937__ac5e30918ce1f455fbe4c423843e4a3c7">Setting the permission to query a table of another user in the default database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="74.86%" headers="mcps1.3.2.2.3.3.2.3.2.3.1.2 "><ol type="a" id="mrs_01_1937__oc839faf8e0ff4b5d87fd7278305c3e27"><li id="mrs_01_1937__la556ffb541794365af310f320a39d12d">In the <span class="parmname" id="mrs_01_1937__parmname14961932193213"><b><span id="mrs_01_1937__text1222182413311">Configure Resource </span>Permission</b></span> table, choose <em id="mrs_01_1937__i1081896099112055">Name of the desired cluster</em> > <strong id="mrs_01_1937__b1679612691112055">Hive</strong> > <strong id="mrs_01_1937__b1370170041112055">Hive Read Write Privileges</strong>.</li><li id="mrs_01_1937__l32c6598a660c4c95a2ec08770b8ef766">Click the name of the specified database in the database list. Tables in the database are displayed.</li><li id="mrs_01_1937__le5ec51c981304a63a9808a8bd2ba629f">In the <span class="parmname" id="mrs_01_1937__pddc687dd053941a4931623a207f840c7"><b>Permission</b></span> column of the specified table, select <span class="parmname" id="mrs_01_1937__pcf7a98390d0449868ff41221ee9ae753"><b>SELECT</b></span>.</li></ol>
|
|
</td>
|
|
</tr>
|
|
<tr id="mrs_01_1937__r0841b72272114e279ed2a70d6bcce8ae"><td class="cellrowborder" valign="top" width="25.14%" headers="mcps1.3.2.2.3.3.2.3.2.3.1.1 "><p id="mrs_01_1937__a10b6f02d328f4799858578caf3c38469">Setting the permission to import data to a table of another user in the default database</p>
|
|
</td>
|
|
<td class="cellrowborder" valign="top" width="74.86%" headers="mcps1.3.2.2.3.3.2.3.2.3.1.2 "><ol type="a" id="mrs_01_1937__o392a14d40b1041c3bb845c079606a1bb"><li id="mrs_01_1937__l8684fcd214c24a03b2a48a6c8cae3fcc">In the <span class="parmname" id="mrs_01_1937__pc7ea9fa3dbb84dfcb1e99ea017efd41c"><b><span id="mrs_01_1937__text3792123110334">Configure Resource </span>Permission</b></span> table, choose <em id="mrs_01_1937__i1904638285112055">Name of the desired cluster</em> > <strong id="mrs_01_1937__b2039289644112055">Hive</strong> > <strong id="mrs_01_1937__b1312700853112055">Hive Read Write Privileges</strong>.</li><li id="mrs_01_1937__l2303a3c7ffb84d1791668d0d9f0014dc">Click the name of the specified database in the database list. Tables in the database are displayed.</li><li id="mrs_01_1937__l69e315d925114e77a64fd0ed17207cc9">In the <span class="parmname" id="mrs_01_1937__paa8f70e1337b47888fae18dfd3f34295"><b>Permission</b></span> column of the specified table, select <span class="parmname" id="mrs_01_1937__pba1baef2fbf2430b81ff7a626a07e56f"><b>DELETE</b></span> and <span class="parmname" id="mrs_01_1937__p57d5c6ed31d241b4ac42876988fdd48d"><b>INSERT</b></span>.</li></ol>
|
|
</td>
|
|
</tr>
|
|
</tbody>
|
|
</table>
|
|
</div>
|
|
</li></ul>
|
|
</li><li id="mrs_01_1937__l2a9c8341bc76439db2f32e0f7e7078cd">Click <span class="parmname" id="mrs_01_1937__pcd738567cf96452dbb760f83ce087833"><b>OK</b></span>.</li></ol>
|
|
</div>
|
|
</div>
|
|
<div>
|
|
<div class="familylinks">
|
|
<div class="parentlink"><strong>Parent topic:</strong> <a href="mrs_01_1935.html">SparkSQL Permission Management(Security Mode)</a></div>
|
|
</div>
|
|
</div>
|
|
|