git.iko.soy/searx
on 484dc99 help sign in
searx/docs/dev/engine_overview.rst
305 lines 13 kB history raw
2.. _engines-dev:
4===============
5Engine overview
6===============
8.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
10searx is a metasearch-engine_, so it uses different search engines to provide
11better results.
13Because there is no general search API which could be used for every search
14engine, an adapter has to be built between searx and the external search
15engines. Adapters are stored under the folder :origin:`searx/engines`.
17.. contents::
18 :depth: 3
19 :backlinks: entry
22.. _general engine configuration:
24general engine configuration
25============================
27It is required to tell searx the type of results the engine provides. The
28arguments can be set in the engine file or in the settings file
29(normally ``settings.yml``). The arguments in the settings file override
30the ones in the engine file.
32It does not matter if an option is stored in the engine file or in the
33settings. However, the standard way is the following:
35.. _engine file:
37engine file
38-----------
40======================= =========== ========================================================
41argument type information
42======================= =========== ========================================================
43categories list pages, in which the engine is working
44paging boolean support multible pages
45language_support boolean support language choosing
46time_range_support boolean support search time range
47engine_type str ``online`` by default, other possibles values are
48 ``offline``, ``online_dictionnary``, ``online_currency``
49======================= =========== ========================================================
51.. _engine settings:
53settings.yml
54------------
56======================= =========== =============================================
57argument type information
58======================= =========== =============================================
59name string name of search-engine
60engine string name of searx-engine
61 (filename without ``.py``)
62shortcut string shortcut of search-engine
63timeout string specific timeout for search-engine
64display_error_messages boolean display error messages on the web UI
65proxies dict set proxies for a specific engine
66 (e.g. ``proxies : {http: socks5://proxy:port,
67 https: socks5://proxy:port}``)
68======================= =========== =============================================
71overrides
72---------
74A few of the options have default values in the engine, but are often
75overwritten by the settings. If ``None`` is assigned to an option in the engine
76file, it has to be redefined in the settings, otherwise searx will not start
77with that engine.
79The naming of overrides is arbitrary. But the recommended overrides are the
80following:
82======================= =========== ===========================================
83argument type information
84======================= =========== ===========================================
85base_url string base-url, can be overwritten to use same
86 engine on other URL
87number_of_results int maximum number of results per request
88language string ISO code of language and country like en_US
89api_key string api-key if required by engine
90======================= =========== ===========================================
92example code
93------------
95.. code:: python
97 # engine dependent config
98 categories = ['general']
99 paging = True
100 language_support = True
103making a request
104================
106To perform a search an URL have to be specified. In addition to specifying an
107URL, arguments can be passed to the query.
109passed arguments
110----------------
112These arguments can be used to construct the search query. Furthermore,
113parameters with default value can be redefined for special purposes.
115If the ``engine_type`` is ``online```:
117====================== ============== ========================================================================
118argument type default-value, information
119====================== ============== ========================================================================
120url str ``''``
121method str ``'GET'``
122headers set ``{}``
123data set ``{}``
124cookies set ``{}``
125verify bool ``True``
126headers.User-Agent str a random User-Agent
127category str current category, like ``'general'``
128safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
129time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
130pageno int current pagenumber
131language str specific language code like ``'en_US'``, or ``'all'`` if unspecified
132====================== ============== ========================================================================
135If the ``engine_type`` is ``online_dictionnary```, in addition to the ``online`` arguments:
137====================== ============ ========================================================================
138argument type default-value, information
139====================== ============ ========================================================================
140from_lang str specific language code like ``'en_US'``
141to_lang str specific language code like ``'en_US'``
142query str the text query without the languages
143====================== ============ ========================================================================
145If the ``engine_type`` is ``online_currency```, in addition to the ``online`` arguments:
147====================== ============ ========================================================================
148argument type default-value, information
149====================== ============ ========================================================================
150amount float the amount to convert
151from str ISO 4217 code
152to str ISO 4217 code
153from_name str currency name
154to_name str currency name
155====================== ============ ========================================================================
158parsed arguments
159----------------
161The function ``def request(query, params):`` always returns the ``params``
162variable. Inside searx, the following paramters can be used to specify a search
163request:
165=================== =========== ==========================================================================
166argument type information
167=================== =========== ==========================================================================
168url str requested url
169method str HTTP request method
170headers set HTTP header information
171data set HTTP data information
172cookies set HTTP cookies
173verify bool Performing SSL-Validity check
174max_redirects int maximum redirects, hard limit
175soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
176raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
177=================== =========== ==========================================================================
180example code
181------------
183.. code:: python
185 # search-url
186 base_url = 'https://example.com/'
187 search_string = 'search?{query}&page={page}'
189 # do search-request
190 def request(query, params):
191 search_path = search_string.format(
192 query=urlencode({'q': query}),
193 page=params['pageno'])
195 params['url'] = base_url + search_path
197 return params
200returned results
201================
203Searx is able to return results of different media-types. Currently the
204following media-types are supported:
206- default_
207- images_
208- videos_
209- torrent_
210- map_
212To set another media-type as default, the parameter ``template`` must be set to
213the desired type.
215default
216-------
218========================= =====================================================
219result-parameter information
220========================= =====================================================
221url string, url of the result
222title string, title of the result
223content string, general result-text
224publishedDate :py:class:`datetime.datetime`, time of publish
225========================= =====================================================
227images
228------
230To use this template, the parameter:
232========================= =====================================================
233result-parameter information
234========================= =====================================================
235template is set to ``images.html``
236url string, url to the result site
237title string, title of the result *(partly implemented)*
238content *(partly implemented)*
239publishedDate :py:class:`datetime.datetime`,
240 time of publish *(partly implemented)*
241img\_src string, url to the result image
242thumbnail\_src string, url to a small-preview image
243========================= =====================================================
245videos
246------
248========================= =====================================================
249result-parameter information
250========================= =====================================================
251template is set to ``videos.html``
252url string, url of the result
253title string, title of the result
254content *(not implemented yet)*
255publishedDate :py:class:`datetime.datetime`, time of publish
256thumbnail string, url to a small-preview image
257========================= =====================================================
259torrent
260-------
262.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
264========================= =====================================================
265result-parameter information
266========================= =====================================================
267template is set to ``torrent.html``
268url string, url of the result
269title string, title of the result
270content string, general result-text
271publishedDate :py:class:`datetime.datetime`,
272 time of publish *(not implemented yet)*
273seed int, number of seeder
274leech int, number of leecher
275filesize int, size of file in bytes
276files int, number of files
277magnetlink string, magnetlink_ of the result
278torrentfile string, torrentfile of the result
279========================= =====================================================
282map
283---
285========================= =====================================================
286result-parameter information
287========================= =====================================================
288url string, url of the result
289title string, title of the result
290content string, general result-text
291publishedDate :py:class:`datetime.datetime`, time of publish
292latitude latitude of result (in decimal format)
293longitude longitude of result (in decimal format)
294boundingbox boundingbox of result (array of 4. values
295 ``[lat-min, lat-max, lon-min, lon-max]``)
296geojson geojson of result (https://geojson.org/)
297osm.type type of osm-object (if OSM-Result)
298osm.id id of osm-object (if OSM-Result)
299address.name name of object
300address.road street name of object
301address.house_number house number of object
302address.locality city, place of object
303address.postcode postcode of object
304address.country country of object
305========================= =====================================================