vpruthi/doc-exports
1
0
Fork 0
forked from docs/doc-exports
Code Pull Requests Activity
doc-exports/docs/dws/dev/dws_06_0106.html
Lu, Huayi e6fa411af0 DWS DEV 830.201 version 
Reviewed-by: Pruthi, Vineet <vineet.pruthi@t-systems.com>
Co-authored-by: Lu, Huayi <luhuayi@huawei.com>
Co-committed-by: Lu, Huayi <luhuayi@huawei.com>
2024-05-16 07:24:04 +00:00

194 lines
24 KiB
HTML
Raw Blame History

<a name="EN-US_TOPIC_0000001188110522"></a><a name="EN-US_TOPIC_0000001188110522"></a>
<h1 class="topictitle1">Synonym Dictionary</h1>
<div id="body1561195448345"><p id="EN-US_TOPIC_0000001188110522__p1798110596188">A synonym dictionary is used to define, identify, and convert synonyms of tokens. Phrases are not supported (use the thesaurus dictionary in <a href="dws_06_0107.html">Thesaurus Dictionary</a>).</p>
<div class="section" id="EN-US_TOPIC_0000001188110522__section10442451826"><h4 class="sectiontitle">Examples</h4><ul id="EN-US_TOPIC_0000001188110522__ul48271054104413"><li id="EN-US_TOPIC_0000001188110522__li16827135434414">A synonym dictionary can be used to overcome linguistic problems, for example, to prevent an English stemmer dictionary from reducing the word "Paris" to "pari". It is enough to have a <strong id="EN-US_TOPIC_0000001188110522__b714616536239">Paris paris</strong> line in the synonym dictionary and put it before the <strong id="EN-US_TOPIC_0000001188110522__b8322316152420">english_stem</strong> dictionary.<div class="notice" id="EN-US_TOPIC_0000001188110522__note29448236569"><span class="noticetitle"><img src="public_sys-resources/notice_3.0-en-us.png"> </span><div class="noticebody"><p id="EN-US_TOPIC_0000001188110522__p1794462317569">// Hard-coded or plaintext AK and SK are risky. For security purposes, encrypt your AK and SK and store them in the configuration file or environment variables.</p>
</div></div>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188110522__screen61051947201711"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span>
<span class="normal">18</span>
<span class="normal">19</span>
<span class="normal">20</span>
<span class="normal">21</span>
<span class="normal">22</span>
<span class="normal">23</span>
<span class="normal">24</span>
<span class="normal">25</span>
<span class="normal">26</span>
<span class="normal">27</span>
<span class="normal">28</span>
<span class="normal">29</span>
<span class="normal">30</span>
<span class="normal">31</span>
<span class="normal">32</span>
<span class="normal">33</span>
<span class="normal">34</span>
<span class="normal">35</span>
<span class="normal">36</span>
<span class="normal">37</span>
<span class="normal">38</span>
<span class="normal">39</span>
<span class="normal">40</span>
<span class="normal">41</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">ts_debug</span><span class="p">(</span><span class="s1">'english'</span><span class="p">,</span><span class="w"> </span><span class="s1">'Paris'</span><span class="p">);</span>
<span class="w"> </span><span class="k">alias</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">description</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">token</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">dictionaries</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">dictionary</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">lexemes</span><span class="w"> </span>
<span class="c1">-----------+-----------------+-------+----------------+--------------+---------</span>
<span class="w"> </span><span class="n">asciiword</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Word</span><span class="p">,</span><span class="w"> </span><span class="k">all</span><span class="w"> </span><span class="n">ASCII</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Paris</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">english_stem</span><span class="err">}</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">english_stem</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">pari</span><span class="err">}</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">CREATE</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">SEARCH</span><span class="w"> </span><span class="k">DICTIONARY</span><span class="w"> </span><span class="n">my_synonym</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">TEMPLATE</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">synonym</span><span class="p">,</span>
<span class="w"> </span><span class="n">SYNONYMS</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">my_synonyms</span><span class="p">,</span>
<span class="w"> </span><span class="n">FILEPATH</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s1">'obs://bucket01/obs.xxx.xxx.com accesskey=xxxxx secretkey=xxxxx region=xx-xx-xx'</span>
<span class="p">);</span>
<span class="k">ALTER</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">SEARCH</span><span class="w"> </span><span class="n">CONFIGURATION</span><span class="w"> </span><span class="n">english</span>
<span class="w"> </span><span class="k">ALTER</span><span class="w"> </span><span class="n">MAPPING</span><span class="w"> </span><span class="k">FOR</span><span class="w"> </span><span class="n">asciiword</span>
<span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="n">my_synonym</span><span class="p">,</span><span class="w"> </span><span class="n">english_stem</span><span class="p">;</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">ts_debug</span><span class="p">(</span><span class="s1">'english'</span><span class="p">,</span><span class="w"> </span><span class="s1">'Paris'</span><span class="p">);</span>
<span class="w"> </span><span class="k">alias</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">description</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">token</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">dictionaries</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">dictionary</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">lexemes</span><span class="w"> </span>
<span class="c1">-----------+-----------------+-------+---------------------------+------------+---------</span>
<span class="w"> </span><span class="n">asciiword</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Word</span><span class="p">,</span><span class="w"> </span><span class="k">all</span><span class="w"> </span><span class="n">ASCII</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Paris</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">my_synonym</span><span class="p">,</span><span class="n">english_stem</span><span class="err">}</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">my_synonym</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">paris</span><span class="err">}</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">ts_debug</span><span class="p">(</span><span class="s1">'english'</span><span class="p">,</span><span class="w"> </span><span class="s1">'paris'</span><span class="p">);</span>
<span class="w"> </span><span class="k">alias</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">description</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">token</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">dictionaries</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">dictionary</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">lexemes</span><span class="w"> </span>
<span class="c1">-----------+-----------------+-------+---------------------------+------------+---------</span>
<span class="w"> </span><span class="n">asciiword</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Word</span><span class="p">,</span><span class="w"> </span><span class="k">all</span><span class="w"> </span><span class="n">ASCII</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Paris</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">my_synonym</span><span class="p">,</span><span class="n">english_stem</span><span class="err">}</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">my_synonym</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">paris</span><span class="err">}</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">ALTER</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">SEARCH</span><span class="w"> </span><span class="k">DICTIONARY</span><span class="w"> </span><span class="n">my_synonym</span><span class="w"> </span><span class="p">(</span><span class="w"> </span><span class="n">CASESENSITIVE</span><span class="o">=</span><span class="k">true</span><span class="p">);</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">ts_debug</span><span class="p">(</span><span class="s1">'english'</span><span class="p">,</span><span class="w"> </span><span class="s1">'Paris'</span><span class="p">);</span>
<span class="w"> </span><span class="k">alias</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">description</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">token</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">dictionaries</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">dictionary</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">lexemes</span><span class="w"> </span>
<span class="c1">-----------+-----------------+-------+---------------------------+------------+---------</span>
<span class="w"> </span><span class="n">asciiword</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Word</span><span class="p">,</span><span class="w"> </span><span class="k">all</span><span class="w"> </span><span class="n">ASCII</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Paris</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">my_synonym</span><span class="p">,</span><span class="n">english_stem</span><span class="err">}</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">my_synonym</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">paris</span><span class="err">}</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">SELECT</span><span class="w"> </span><span class="o">*</span><span class="w"> </span><span class="k">FROM</span><span class="w"> </span><span class="n">ts_debug</span><span class="p">(</span><span class="s1">'english'</span><span class="p">,</span><span class="w"> </span><span class="s1">'paris'</span><span class="p">);</span>
<span class="w"> </span><span class="k">alias</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">description</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">token</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">dictionaries</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="k">dictionary</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">lexemes</span><span class="w"> </span>
<span class="c1">-----------+-----------------+-------+---------------------------+------------+---------</span>
<span class="w"> </span><span class="n">asciiword</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Word</span><span class="p">,</span><span class="w"> </span><span class="k">all</span><span class="w"> </span><span class="n">ASCII</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">Paris</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">my_synonym</span><span class="p">,</span><span class="n">english_stem</span><span class="err">}</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="n">my_synonym</span><span class="w"> </span><span class="o">|</span><span class="w"> </span><span class="err">{</span><span class="n">pari</span><span class="err">}</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001188110522__p161114594517">The full name of the synonym dictionary file is <strong id="EN-US_TOPIC_0000001188110522__b930888361">my_synonyms.syn</strong>, and the dictionary is stored in the <strong id="EN-US_TOPIC_0000001188110522__b782344694610">obs://bucket01/obs.xxx.xxx.com accesskey=xxxxx secretkey=xxxxx region=<em id="EN-US_TOPIC_0000001188110522__i1389171081113"><span id="EN-US_TOPIC_0000001188110522__ph108911210151118">xx-xx-xx</span></em></strong> directory. For details about the syntax and parameters for creating a synonym dictionary, see <a href="dws_06_0183.html">CREATE TEXT SEARCH DICTIONARY</a>.</p>
</li><li id="EN-US_TOPIC_0000001188110522__li1342001654917">An asterisk (*) can be placed at the end of a synonym in the configuration file. This indicates that the synonym is a prefix. The asterisk is ignored when the entry is used in to_tsvector(), but when it is used in to_tsquery(), the result will be a query item with the prefix match marker (see <a href="dws_06_0098.html">Handling TSQuery</a>).<p id="EN-US_TOPIC_0000001188110522__p14645211182011">Assume that the content in the dictionary file <strong id="EN-US_TOPIC_0000001188110522__b97781691144">synonym_sample.syn</strong> is as follows:</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188110522__screen928520169418"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal">1</span>
<span class="normal">2</span>
<span class="normal">3</span>
<span class="normal">4</span>
<span class="normal">5</span></pre></div></td><td class="code"><div><pre><span></span><span class="n">postgres</span><span class="w"> </span><span class="n">pgsql</span>
<span class="n">postgresql</span><span class="w"> </span><span class="n">pgsql</span><span class="w"> </span>
<span class="n">postgre</span><span class="w"> </span><span class="n">pgsql</span><span class="w"> </span>
<span class="n">gogle</span><span class="w"> </span><span class="n">googl</span><span class="w"> </span>
<span class="n">indices</span><span class="w"> </span><span class="k">index</span><span class="o">*</span>
</pre></div></td></tr></table></div>
</div>
<p id="EN-US_TOPIC_0000001188110522__p17318191418578">Create and use a dictionary.</p>
<div class="codecoloring" codetype="Sql" id="EN-US_TOPIC_0000001188110522__screen133702144519"><div class="highlight"><table class="highlighttable"><tr><td class="linenos"><div class="linenodiv"><pre><span class="normal"> 1</span>
<span class="normal"> 2</span>
<span class="normal"> 3</span>
<span class="normal"> 4</span>
<span class="normal"> 5</span>
<span class="normal"> 6</span>
<span class="normal"> 7</span>
<span class="normal"> 8</span>
<span class="normal"> 9</span>
<span class="normal">10</span>
<span class="normal">11</span>
<span class="normal">12</span>
<span class="normal">13</span>
<span class="normal">14</span>
<span class="normal">15</span>
<span class="normal">16</span>
<span class="normal">17</span>
<span class="normal">18</span>
<span class="normal">19</span>
<span class="normal">20</span>
<span class="normal">21</span>
<span class="normal">22</span>
<span class="normal">23</span>
<span class="normal">24</span>
<span class="normal">25</span>
<span class="normal">26</span>
<span class="normal">27</span>
<span class="normal">28</span>
<span class="normal">29</span>
<span class="normal">30</span>
<span class="normal">31</span>
<span class="normal">32</span>
<span class="normal">33</span>
<span class="normal">34</span>
<span class="normal">35</span>
<span class="normal">36</span>
<span class="normal">37</span>
<span class="normal">38</span></pre></div></td><td class="code"><div><pre><span></span><span class="k">CREATE</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">SEARCH</span><span class="w"> </span><span class="k">DICTIONARY</span><span class="w"> </span><span class="n">syn</span><span class="w"> </span><span class="p">(</span>
<span class="w"> </span><span class="k">TEMPLATE</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">synonym</span><span class="p">,</span>
<span class="w"> </span><span class="n">SYNONYMS</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="n">synonym_sample</span>
<span class="p">);</span>
<span class="k">SELECT</span><span class="w"> </span><span class="n">ts_lexize</span><span class="p">(</span><span class="s1">'syn'</span><span class="p">,</span><span class="s1">'indices'</span><span class="p">);</span>
<span class="w"> </span><span class="n">ts_lexize</span><span class="w"> </span>
<span class="c1">-----------</span>
<span class="w"> </span><span class="err">{</span><span class="k">index</span><span class="err">}</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">CREATE</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">SEARCH</span><span class="w"> </span><span class="n">CONFIGURATION</span><span class="w"> </span><span class="n">tst</span><span class="w"> </span><span class="p">(</span><span class="k">copy</span><span class="o">=</span><span class="k">simple</span><span class="p">);</span>
<span class="k">ALTER</span><span class="w"> </span><span class="nb">TEXT</span><span class="w"> </span><span class="k">SEARCH</span><span class="w"> </span><span class="n">CONFIGURATION</span><span class="w"> </span><span class="n">tst</span><span class="w"> </span><span class="k">ALTER</span><span class="w"> </span><span class="n">MAPPING</span><span class="w"> </span><span class="k">FOR</span><span class="w"> </span><span class="n">asciiword</span><span class="w"> </span><span class="k">WITH</span><span class="w"> </span><span class="n">syn</span><span class="p">;</span>
<span class="k">SELECT</span><span class="w"> </span><span class="n">to_tsvector</span><span class="p">(</span><span class="s1">'tst'</span><span class="p">,</span><span class="s1">'indices'</span><span class="p">);</span>
<span class="w"> </span><span class="n">to_tsvector</span><span class="w"> </span>
<span class="c1">-------------</span>
<span class="w"> </span><span class="s1">'index'</span><span class="p">:</span><span class="mi">1</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">SELECT</span><span class="w"> </span><span class="n">to_tsquery</span><span class="p">(</span><span class="s1">'tst'</span><span class="p">,</span><span class="s1">'indices'</span><span class="p">);</span>
<span class="w"> </span><span class="n">to_tsquery</span><span class="w"> </span>
<span class="c1">------------</span>
<span class="w"> </span><span class="s1">'index'</span><span class="p">:</span><span class="o">*</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">SELECT</span><span class="w"> </span><span class="s1">'indexes are very useful'</span><span class="p">::</span><span class="n">tsvector</span><span class="p">;</span>
<span class="w"> </span><span class="n">tsvector</span><span class="w"> </span>
<span class="c1">---------------------------------</span>
<span class="w"> </span><span class="s1">'are'</span><span class="w"> </span><span class="s1">'indexes'</span><span class="w"> </span><span class="s1">'useful'</span><span class="w"> </span><span class="s1">'very'</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
<span class="k">SELECT</span><span class="w"> </span><span class="s1">'indexes are very useful'</span><span class="p">::</span><span class="n">tsvector</span><span class="w"> </span><span class="o">@@</span><span class="w"> </span><span class="n">to_tsquery</span><span class="p">(</span><span class="s1">'tst'</span><span class="p">,</span><span class="s1">'indices'</span><span class="p">);</span>
<span class="w"> </span><span class="o">?</span><span class="k">column</span><span class="o">?</span><span class="w"> </span>
<span class="c1">----------</span>
<span class="w"> </span><span class="n">t</span>
<span class="p">(</span><span class="mi">1</span><span class="w"> </span><span class="k">row</span><span class="p">)</span>
</pre></div></td></tr></table></div>
</div>
</li></ul>
</div>
<p id="EN-US_TOPIC_0000001188110522__p541744303613"></p>
</div>
<div>
<div class="familylinks">
<div class="parentlink"><strong>Parent topic:</strong> <a href="dws_06_0102.html">Dictionaries</a></div>
</div>
</div>
View Git Blame Copy Permalink