git.iko.soy/searx
on master help sign in
searx/docs/dev/engine_overview.rst
306 lines 13 kB history raw
2.. _engines-dev:
4===============
5Engine overview
6===============
8.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
10searx is a metasearch-engine_, so it uses different search engines to provide
11better results.
13Because there is no general search API which could be used for every search
14engine, an adapter has to be built between searx and the external search
15engines. Adapters are stored under the folder :origin:`searx/engines`.
17.. contents::
18 :depth: 3
19 :backlinks: entry
22.. _general engine configuration:
24general engine configuration
25============================
27It is required to tell searx the type of results the engine provides. The
28arguments can be set in the engine file or in the settings file
29(normally ``settings.yml``). The arguments in the settings file override
30the ones in the engine file.
32It does not matter if an option is stored in the engine file or in the
33settings. However, the standard way is the following:
35.. _engine file:
37engine file
38-----------
40======================= =========== ========================================================
41argument type information
42======================= =========== ========================================================
43categories list pages, in which the engine is working
44paging boolean support multible pages
45time_range_support boolean support search time range
46engine_type str ``online`` by default, other possibles values are
47 ``offline``, ``online_dictionnary``, ``online_currency``
48======================= =========== ========================================================
50.. _engine settings:
52settings.yml
53------------
55======================= =========== =============================================
56argument type information
57======================= =========== =============================================
58name string name of search-engine
59engine string name of searx-engine
60 (filename without ``.py``)
61enable_http bool enable HTTP
62 (by default only HTTPS is enabled).
63shortcut string shortcut of search-engine
64timeout string specific timeout for search-engine
65display_error_messages boolean display error messages on the web UI
66proxies dict set proxies for a specific engine
67 (e.g. ``proxies : {http: socks5://proxy:port,
68 https: socks5://proxy:port}``)
69======================= =========== =============================================
72overrides
73---------
75A few of the options have default values in the engine, but are often
76overwritten by the settings. If ``None`` is assigned to an option in the engine
77file, it has to be redefined in the settings, otherwise searx will not start
78with that engine.
80The naming of overrides is arbitrary. But the recommended overrides are the
81following:
83======================= =========== ===========================================
84argument type information
85======================= =========== ===========================================
86base_url string base-url, can be overwritten to use same
87 engine on other URL
88number_of_results int maximum number of results per request
89language string ISO code of language and country like en_US
90api_key string api-key if required by engine
91======================= =========== ===========================================
93example code
94------------
96.. code:: python
98 # engine dependent config
99 categories = ['general']
100 paging = True
103making a request
104================
106To perform a search an URL have to be specified. In addition to specifying an
107URL, arguments can be passed to the query.
109passed arguments
110----------------
112These arguments can be used to construct the search query. Furthermore,
113parameters with default value can be redefined for special purposes.
115If the ``engine_type`` is ``online```:
117====================== ============== ========================================================================
118argument type default-value, information
119====================== ============== ========================================================================
120url str ``''``
121method str ``'GET'``
122headers set ``{}``
123data set ``{}``
124cookies set ``{}``
125verify bool ``True``
126headers.User-Agent str a random User-Agent
127category str current category, like ``'general'``
128safesearch int ``0``, between ``0`` and ``2`` (normal, moderate, strict)
129time_range Optional[str] ``None``, can be ``day``, ``week``, ``month``, ``year``
130pageno int current pagenumber
131language str specific language code like ``'en_US'``, or ``'all'`` if unspecified
132====================== ============== ========================================================================
135If the ``engine_type`` is ``online_dictionnary```, in addition to the ``online`` arguments:
137====================== ============ ========================================================================
138argument type default-value, information
139====================== ============ ========================================================================
140from_lang str specific language code like ``'en_US'``
141to_lang str specific language code like ``'en_US'``
142query str the text query without the languages
143====================== ============ ========================================================================
145If the ``engine_type`` is ``online_currency```, in addition to the ``online`` arguments:
147====================== ============ ========================================================================
148argument type default-value, information
149====================== ============ ========================================================================
150amount float the amount to convert
151from str ISO 4217 code
152to str ISO 4217 code
153from_name str currency name
154to_name str currency name
155====================== ============ ========================================================================
158parsed arguments
159----------------
161The function ``def request(query, params):`` always returns the ``params``
162variable. Inside searx, the following paramters can be used to specify a search
163request:
165=================== =========== ==========================================================================
166argument type information
167=================== =========== ==========================================================================
168url str requested url
169method str HTTP request method
170headers set HTTP header information
171data set HTTP data information
172cookies set HTTP cookies
173verify bool Performing SSL-Validity check
174follow_redirects bool Follow redirects
175max_redirects int maximum redirects, hard limit
176soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
177raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
178=================== =========== ==========================================================================
181example code
182------------
184.. code:: python
186 # search-url
187 base_url = 'https://example.com/'
188 search_string = 'search?{query}&page={page}'
190 # do search-request
191 def request(query, params):
192 search_path = search_string.format(
193 query=urlencode({'q': query}),
194 page=params['pageno'])
196 params['url'] = base_url + search_path
198 return params
201returned results
202================
204Searx is able to return results of different media-types. Currently the
205following media-types are supported:
207- default_
208- images_
209- videos_
210- torrent_
211- map_
213To set another media-type as default, the parameter ``template`` must be set to
214the desired type.
216default
217-------
219========================= =====================================================
220result-parameter information
221========================= =====================================================
222url string, url of the result
223title string, title of the result
224content string, general result-text
225publishedDate :py:class:`datetime.datetime`, time of publish
226========================= =====================================================
228images
229------
231To use this template, the parameter:
233========================= =====================================================
234result-parameter information
235========================= =====================================================
236template is set to ``images.html``
237url string, url to the result site
238title string, title of the result *(partly implemented)*
239content *(partly implemented)*
240publishedDate :py:class:`datetime.datetime`,
241 time of publish *(partly implemented)*
242img\_src string, url to the result image
243thumbnail\_src string, url to a small-preview image
244========================= =====================================================
246videos
247------
249========================= =====================================================
250result-parameter information
251========================= =====================================================
252template is set to ``videos.html``
253url string, url of the result
254title string, title of the result
255content *(not implemented yet)*
256publishedDate :py:class:`datetime.datetime`, time of publish
257thumbnail string, url to a small-preview image
258========================= =====================================================
260torrent
261-------
263.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
265========================= =====================================================
266result-parameter information
267========================= =====================================================
268template is set to ``torrent.html``
269url string, url of the result
270title string, title of the result
271content string, general result-text
272publishedDate :py:class:`datetime.datetime`,
273 time of publish *(not implemented yet)*
274seed int, number of seeder
275leech int, number of leecher
276filesize int, size of file in bytes
277files int, number of files
278magnetlink string, magnetlink_ of the result
279torrentfile string, torrentfile of the result
280========================= =====================================================
283map
284---
286========================= =====================================================
287result-parameter information
288========================= =====================================================
289url string, url of the result
290title string, title of the result
291content string, general result-text
292publishedDate :py:class:`datetime.datetime`, time of publish
293latitude latitude of result (in decimal format)
294longitude longitude of result (in decimal format)
295boundingbox boundingbox of result (array of 4. values
296 ``[lat-min, lat-max, lon-min, lon-max]``)
297geojson geojson of result (https://geojson.org/)
298osm.type type of osm-object (if OSM-Result)
299osm.id id of osm-object (if OSM-Result)
300address.name name of object
301address.road street name of object
302address.house_number house number of object
303address.locality city, place of object
304address.postcode postcode of object
305address.country country of object
306========================= =====================================================