02fc414 · searx · git.iko.soy

git.iko.soy/searx

on master help sign in

[mod] dictzone, translated, currency_convert: use engine_type online_curency and online_dictionnary

02fc414Alexandre Flament6 years ago

docs/dev/engine_overview.rst+47−19

compare with master

unifiedsplit

@@ -37,15 +37,16 @@ settings. However, the standard way is the following:

37engine file

38-----------

40−======================= =========== ===========================================

40+======================= =========== ========================================================

41argument type information

42−======================= =========== ===========================================

42+======================= =========== ========================================================

43categories list pages, in which the engine is working

44paging boolean support multible pages

45language_support boolean support language choosing

46time_range_support boolean support search time range

47−engine_type str ``online`` by default, ``offline``

47+engine_type str ``online`` by default, other possibles values are

48+ ``offline``, ``online_dictionnary``, ``online_currency``

48−======================= =========== ===========================================

49+======================= =========== ========================================================

50.. _engine settings:

51.. _engine settings:

@@ -111,22 +112,49 @@ passed arguments

111These arguments can be used to construct the search query. Furthermore,

112These arguments can be used to construct the search query. Furthermore,

112parameters with default value can be redefined for special purposes.

113parameters with default value can be redefined for special purposes.

113

114

115+If the ``engine_type`` is ``online```:

116+

117+====================== ============== ========================================================================

118+argument type default-value, information

119+====================== ============== ========================================================================

120+url str ``''``

121+method str ``'GET'``

122+headers set ``{}``

123+data set ``{}``

124+cookies set ``{}``

125+verify bool ``True``

126+headers.User-Agent str a random User-Agent

127+category str current category, like ``'general'``

128+safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)

129+time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``

130+pageno int current pagenumber

131+language str specific language code like ``'en_US'``, or ``'all'`` if unspecified

132+====================== ============== ========================================================================

133+

134+

135+If the ``engine_type`` is ``online_dictionnary```, in addition to the ``online`` arguments:

136+

114====================== ============ ========================================================================

137====================== ============ ========================================================================

115argument type default-value, information

138argument type default-value, information

116====================== ============ ========================================================================

139====================== ============ ========================================================================

117−url string ``''``

140+from_lang str specific language code like ``'en_US'``

118−method string ``'GET'``

119−headers set ``{}``

120−data set ``{}``

121−cookies set ``{}``

122−verify boolean ``True``

123−headers.User-Agent string a random User-Agent

124−category string current category, like ``'general'``

125−started datetime current date-time

126−pageno int current pagenumber

127−language string specific language code like ``'en_US'``, or ``'all'`` if unspecified

141+to_lang str specific language code like ``'en_US'``

142+query str the text query without the languages

128====================== ============ ========================================================================

143====================== ============ ========================================================================

129

144

145+If the ``engine_type`` is ``online_currency```, in addition to the ``online`` arguments:

146+

147+====================== ============ ========================================================================

148+argument type default-value, information

149+====================== ============ ========================================================================

150+amount float the amount to convert

151+from str ISO 4217 code

152+to str ISO 4217 code

153+from_name str currency name

154+to_name str currency name

155+====================== ============ ========================================================================

156+

157+

130parsed arguments

158parsed arguments

131----------------

159----------------

132

160

@@ -137,12 +165,12 @@ request:

137=================== =========== ==========================================================================

165=================== =========== ==========================================================================

138argument type information

166argument type information

139=================== =========== ==========================================================================

167=================== =========== ==========================================================================

140−url string requested url

168+url str requested url

141−method string HTTP request method

169+method str HTTP request method

142headers set HTTP header information

170headers set HTTP header information

143−data set HTTP data information (parsed if ``method != 'GET'``)

171+data set HTTP data information

144cookies set HTTP cookies

172cookies set HTTP cookies

145−verify boolean Performing SSL-Validity check

173+verify bool Performing SSL-Validity check

146max_redirects int maximum redirects, hard limit

174max_redirects int maximum redirects, hard limit

147soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine

175soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine

148raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300

176raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300

searx/engines/currency_convert.py+2−40

@@ -1,54 +1,16 @@

1import json

2−import re

3−import unicodedata

4−from searx.data import CURRENCIES # NOQA

4+engine_type = 'online_currency'

7categories = []

5categories = []

8url = 'https://duckduckgo.com/js/spice/currency/1/{0}/{1}'

6url = 'https://duckduckgo.com/js/spice/currency/1/{0}/{1}'

9weight = 100

7weight = 100

11−parser_re = re.compile('.*?(\\d+(?:\\.\\d+)?) ([^.0-9]+) (?:in|to) ([^.0-9]+)', re.I)

12https_support = True

9https_support = True

15−def normalize_name(name):

16− name = name.lower().replace('-', ' ').rstrip('s')

17− name = re.sub(' +', ' ', name)

18− return unicodedata.normalize('NFKD', name).lower()

19−

20−

21−def name_to_iso4217(name):

22− global CURRENCIES

23−

24− name = normalize_name(name)

25− currency = CURRENCIES['names'].get(name, [name])

26− return currency[0]

27−

28−

29−def iso4217_to_name(iso4217, language):

30− global CURRENCIES

31−

32− return CURRENCIES['iso4217'].get(iso4217, {}).get(language, iso4217)

33−

34−

35def request(query, params):

12def request(query, params):

36− m = parser_re.match(query)

37− if not m:

38− # wrong query

39− return params

40− amount, from_currency, to_currency = m.groups()

41− amount = float(amount)

42− from_currency = name_to_iso4217(from_currency.strip())

43− to_currency = name_to_iso4217(to_currency.strip())

44−

45− params['url'] = url.format(from_currency, to_currency)

13+ params['url'] = url.format(params['from'], params['to'])

46− params['amount'] = amount

47− params['from'] = from_currency

48− params['to'] = to_currency

49− params['from_name'] = iso4217_to_name(from_currency, 'en')

50− params['to_name'] = iso4217_to_name(to_currency, 'en')

51−

52 return params

14 return params

searx/engines/dictzone.py+6−18

@@ -9,36 +9,24 @@

9 @parse url, title, content

10"""

12−import re

13from urllib.parse import urljoin

12from urllib.parse import urljoin

14from lxml import html

13from lxml import html

15−from searx.utils import is_valid_lang, eval_xpath

14+from searx.utils import eval_xpath

16+

17+engine_type = 'online_dictionnary'

17categories = ['general']

18categories = ['general']

18url = 'https://dictzone.com/{from_lang}-{to_lang}-dictionary/{query}'

19url = 'https://dictzone.com/{from_lang}-{to_lang}-dictionary/{query}'

19weight = 100

20weight = 100

21−parser_re = re.compile('.*?([a-z]+)-([a-z]+) ([^ ]+)$', re.I)

22results_xpath = './/table[@id="r"]/tr'

23https_support = True

26def request(query, params):

27− m = parser_re.match(query)

28− if not m:

29− return params

30−

31− from_lang, to_lang, query = m.groups()

32−

33− from_lang = is_valid_lang(from_lang)

34− to_lang = is_valid_lang(to_lang)

35−

36− if not from_lang or not to_lang:

37− return params

38−

39− params['url'] = url.format(from_lang=from_lang[2],

27+ params['url'] = url.format(from_lang=params['from_lang'][2],

40− to_lang=to_lang[2],

28+ to_lang=params['to_lang'][2],

41− query=query)

29+ query=params['query'])

43 return params

31 return params

searx/engines/translated.py+4−22

@@ -8,44 +8,26 @@

8 @stable yes

9 @parse url, title, content

10"""

11−import re

12−from searx.utils import is_valid_lang

12+engine_type = 'online_dictionnary'

14categories = ['general']

13categories = ['general']

15url = 'https://api.mymemory.translated.net/get?q={query}&langpair={from_lang}|{to_lang}{key}'

14url = 'https://api.mymemory.translated.net/get?q={query}&langpair={from_lang}|{to_lang}{key}'

16web_url = 'https://mymemory.translated.net/en/{from_lang}/{to_lang}/{query}'

15web_url = 'https://mymemory.translated.net/en/{from_lang}/{to_lang}/{query}'

17weight = 100

16weight = 100

18https_support = True

17https_support = True

20−parser_re = re.compile('.*?([a-z]+)-([a-z]+) (.{2,})$', re.I)

21api_key = ''

19api_key = ''

24def request(query, params):

22def request(query, params):

25− m = parser_re.match(query)

26− if not m:

27− return params

28−

29− from_lang, to_lang, query = m.groups()

30−

31− from_lang = is_valid_lang(from_lang)

32− to_lang = is_valid_lang(to_lang)

33−

34− if not from_lang or not to_lang:

35− return params

36−

37 if api_key:

23 if api_key:

38 key_form = '&key=' + api_key

24 key_form = '&key=' + api_key

39 else:

25 else:

40 key_form = ''

26 key_form = ''

41− params['url'] = url.format(from_lang=from_lang[1],

27+ params['url'] = url.format(from_lang=params['from_lang'][1],

42− to_lang=to_lang[1],

28+ to_lang=params['to_lang'][1],

43− query=query,

29+ query=params['query'],

44 key=key_form)

30 key=key_form)

45− params['query'] = query

46− params['from_lang'] = from_lang

47− params['to_lang'] = to_lang

48−

49 return params

31 return params