on 9f8ebec help sign in
searx/docs/dev/engine_overview.rst
277 lines 11 kB history raw
2.. _engines-dev:
4===============
5Engine overview
6===============
8.. _metasearch-engine: https://en.wikipedia.org/wiki/Metasearch_engine
10searx is a metasearch-engine_, so it uses different search engines to provide
11better results.
13Because there is no general search API which could be used for every search
14engine, an adapter has to be built between searx and the external search
15engines. Adapters are stored under the folder :origin:`searx/engines`.
17.. contents::
18 :depth: 3
19 :backlinks: entry
22.. _general engine configuration:
24general engine configuration
25============================
27It is required to tell searx the type of results the engine provides. The
28arguments can be set in the engine file or in the settings file
29(normally ``settings.yml``). The arguments in the settings file override
30the ones in the engine file.
32It does not matter if an option is stored in the engine file or in the
33settings. However, the standard way is the following:
35.. _engine file:
37engine file
38-----------
40======================= =========== ===========================================
41argument type information
42======================= =========== ===========================================
43categories list pages, in which the engine is working
44paging boolean support multible pages
45language_support boolean support language choosing
46time_range_support boolean support search time range
47offline boolean engine runs offline
48======================= =========== ===========================================
50.. _engine settings:
52settings.yml
53------------
55======================= =========== =============================================
56argument type information
57======================= =========== =============================================
58name string name of search-engine
59engine string name of searx-engine
60 (filename without ``.py``)
61shortcut string shortcut of search-engine
62timeout string specific timeout for search-engine
63display_error_messages boolean display error messages on the web UI
64proxies dict set proxies for a specific engine
65 (e.g. ``proxies : {http: socks5://proxy:port,
66 https: socks5://proxy:port}``)
67======================= =========== =============================================
70overrides
71---------
73A few of the options have default values in the engine, but are often
74overwritten by the settings. If ``None`` is assigned to an option in the engine
75file, it has to be redefined in the settings, otherwise searx will not start
76with that engine.
78The naming of overrides is arbitrary. But the recommended overrides are the
79following:
81======================= =========== ===========================================
82argument type information
83======================= =========== ===========================================
84base_url string base-url, can be overwritten to use same
85 engine on other URL
86number_of_results int maximum number of results per request
87language string ISO code of language and country like en_US
88api_key string api-key if required by engine
89======================= =========== ===========================================
91example code
92------------
94.. code:: python
96 # engine dependent config
97 categories = ['general']
98 paging = True
99 language_support = True
102making a request
103================
105To perform a search an URL have to be specified. In addition to specifying an
106URL, arguments can be passed to the query.
108passed arguments
109----------------
111These arguments can be used to construct the search query. Furthermore,
112parameters with default value can be redefined for special purposes.
114====================== ============ ========================================================================
115argument type default-value, information
116====================== ============ ========================================================================
117url string ``''``
118method string ``'GET'``
119headers set ``{}``
120data set ``{}``
121cookies set ``{}``
122verify boolean ``True``
123headers.User-Agent string a random User-Agent
124category string current category, like ``'general'``
125started datetime current date-time
126pageno int current pagenumber
127language string specific language code like ``'en_US'``, or ``'all'`` if unspecified
128====================== ============ ========================================================================
130parsed arguments
131----------------
133The function ``def request(query, params):`` always returns the ``params``
134variable. Inside searx, the following paramters can be used to specify a search
135request:
137=================== =========== ==========================================================================
138argument type information
139=================== =========== ==========================================================================
140url string requested url
141method string HTTP request method
142headers set HTTP header information
143data set HTTP data information (parsed if ``method != 'GET'``)
144cookies set HTTP cookies
145verify boolean Performing SSL-Validity check
146max_redirects int maximum redirects, hard limit
147soft_max_redirects int maximum redirects, soft limit. Record an error but don't stop the engine
148raise_for_httperror bool True by default: raise an exception if the HTTP code of response is >= 300
149=================== =========== ==========================================================================
152example code
153------------
155.. code:: python
157 # search-url
158 base_url = 'https://example.com/'
159 search_string = 'search?{query}&page={page}'
161 # do search-request
162 def request(query, params):
163 search_path = search_string.format(
164 query=urlencode({'q': query}),
165 page=params['pageno'])
167 params['url'] = base_url + search_path
169 return params
172returned results
173================
175Searx is able to return results of different media-types. Currently the
176following media-types are supported:
178- default_
179- images_
180- videos_
181- torrent_
182- map_
184To set another media-type as default, the parameter ``template`` must be set to
185the desired type.
187default
188-------
190========================= =====================================================
191result-parameter information
192========================= =====================================================
193url string, url of the result
194title string, title of the result
195content string, general result-text
196publishedDate :py:class:`datetime.datetime`, time of publish
197========================= =====================================================
199images
200------
202To use this template, the parameter:
204========================= =====================================================
205result-parameter information
206========================= =====================================================
207template is set to ``images.html``
208url string, url to the result site
209title string, title of the result *(partly implemented)*
210content *(partly implemented)*
211publishedDate :py:class:`datetime.datetime`,
212 time of publish *(partly implemented)*
213img\_src string, url to the result image
214thumbnail\_src string, url to a small-preview image
215========================= =====================================================
217videos
218------
220========================= =====================================================
221result-parameter information
222========================= =====================================================
223template is set to ``videos.html``
224url string, url of the result
225title string, title of the result
226content *(not implemented yet)*
227publishedDate :py:class:`datetime.datetime`, time of publish
228thumbnail string, url to a small-preview image
229========================= =====================================================
231torrent
232-------
234.. _magnetlink: https://en.wikipedia.org/wiki/Magnet_URI_scheme
236========================= =====================================================
237result-parameter information
238========================= =====================================================
239template is set to ``torrent.html``
240url string, url of the result
241title string, title of the result
242content string, general result-text
243publishedDate :py:class:`datetime.datetime`,
244 time of publish *(not implemented yet)*
245seed int, number of seeder
246leech int, number of leecher
247filesize int, size of file in bytes
248files int, number of files
249magnetlink string, magnetlink_ of the result
250torrentfile string, torrentfile of the result
251========================= =====================================================
254map
255---
257========================= =====================================================
258result-parameter information
259========================= =====================================================
260url string, url of the result
261title string, title of the result
262content string, general result-text
263publishedDate :py:class:`datetime.datetime`, time of publish
264latitude latitude of result (in decimal format)
265longitude longitude of result (in decimal format)
266boundingbox boundingbox of result (array of 4. values
267 ``[lat-min, lat-max, lon-min, lon-max]``)
268geojson geojson of result (https://geojson.org/)
269osm.type type of osm-object (if OSM-Result)
270osm.id id of osm-object (if OSM-Result)
271address.name name of object
272address.road street name of object
273address.house_number house number of object
274address.locality city, place of object
275address.postcode postcode of object
276address.country country of object
277========================= =====================================================